KVM: X86: Fix and clean up for register caches
cover KVM: X86: Fix and clean up for register caches
From: Lai Jiangshan <[email protected]>
The patchset was started when I read the code of nested_svm_load_cr3() and found that it marks CR3 available other than dirty when changing vcpu->arch.cr3. I thought its caller has ensured that vmcs.GUEST_CR3 will be or already be set to @cr3 so that it doesn't need to be marked dirty. And later I found that it is not true and it must be a bug in a rare case before I realized that all the code just (ab)uses vcpu->arch.regs_avail for VCPU_EXREG_CR3 and there is not such bug of using regs_avail here.
(The above finding becomes a low meaning patch_15 rather than a fix)
The unhappyness of the reading code made me do some cleanup for regs_avail and regs_dirty and kvm_register_xxx() functions in the hope that the code become clearer with less misunderstanding.
Major focus was on VCPU_EXREG_CR3 and VCPU_EXREG_PDPTR. They are ensured to be marked the correct tags (available or dirty), and the value is ensured to be synced to architecture before run if it is marked dirty.
When cleaning VCPU_EXREG_PDPTR, I also checked if the corresponding cr0/cr4 pdptr bits are all intercepted when !tdp_enabled, and I think it is not clear enough, so X86_CR4_PDPTR_BITS is added as self-comments in the code.
Lai Jiangshan (15):
KVM: X86: Ensure the dirty PDPTEs to be loaded
KVM: VMX: Mark VCPU_EXREG_PDPTR available in ept_save_pdptrs()
KVM: SVM: Always clear available of VCPU_EXREG_PDPTR in svm_vcpu_run()
KVM: VMX: Add and use X86_CR4_TLB_BITS when !enable_ept
KVM: VMX: Add and use X86_CR4_PDPTR_BITS when !enable_ept
KVM: X86: Move CR0 pdptr_bits into header file as X86_CR0_PDPTR_BITS
KVM: SVM: Remove outdate comment in svm_load_mmu_pgd()
KVM: SVM: Remove useless check in svm_load_mmu_pgd()
KVM: SVM: Remove the unneeded code to mark available for CR3
KVM: X86: Mark CR3 dirty when vcpu->arch.cr3 is changed
KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
KVM: VMX: Reset the bits that are meaningful to be reset in
vmx_register_cache_reset()
KVM: SVM: Add and use svm_register_cache_reset()
KVM: X86: Remove kvm_register_clear_available()
KVM: nVMX: Always write vmcs.GUEST_CR3 during nested VM-Exit
arch/x86/kvm/kvm_cache_regs.h | 13 ++++++------ arch/x86/kvm/svm/nested.c | 1 - arch/x86/kvm/svm/svm.c | 17 ++++++++-------- arch/x86/kvm/svm/svm.h | 26 ++++++++++++++++++++++++ arch/x86/kvm/vmx/nested.c | 30 ++++++++++++++++++---------- arch/x86/kvm/vmx/vmx.c | 12 +++++++----- arch/x86/kvm/vmx/vmx.h | 37 +++++++++++++++++++++++++---------- arch/x86/kvm/x86.c | 13 ++++++------ 8 files changed, 101 insertions(+), 48 deletions(-)
Signature · 1 line
2.19.1.6.gb485710b
PATCH 1/15 KVM: X86: Ensure the dirty PDPTEs to be loaded
From: Lai Jiangshan <[email protected]>
For VMX, the dirty PDPTEs needs to be loaded before the coming VMENTER via vmx_load_mmu_pgd() if EPT is enabled.
But not all paths that call load_pdptrs() will cause vmx_load_mmu_pgd() to be invoked. Normally, kvm_mmu_reset_context() and KVM_REQ_LOAD_MMU_PGD are used to launch later vmx_load_mmu_pgd().
The commit d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed") skips kvm_mmu_reset_context() after load_pdptrs() when changing CR0.CD and CR0.NW.
The commit 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush") skips KVM_REQ_LOAD_MMU_PGD after load_pdptrs() when rewriting the CR3 with the same value.
The commit a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE") skips kvm_mmu_reset_context() after load_pdptrs() when changing CR4.PGE.
--- arch/x86/kvm/x86.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kvm/x86.c +2 -0
@@ -830,6 +830,8 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); +/* Ensure the dirty PDPTEs to be loaded. */ +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); vcpu->arch.pdptrs_from_userspace = false;return 1 ;
Signature · 1 line
2.19.1.6.gb485710b
PATCH 2/15 KVM: VMX: Mark VCPU_EXREG_PDPTR available in ept_save_pdptrs()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
mmu->pdptrs[] and vmcs.GUEST_PDPTR[0-3] are synced, so mmu->pdptrs is available and GUEST_PDPTR[0-3] is not dirty.
--- arch/x86/kvm/vmx/vmx.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -3001,7 +3001,7 @@ void ept_save_pdptrs(struct kvm_vcpu *vcpu) mmu->pdptrs[2 ] =vmcs_read64 (GUEST_PDPTR2 ); mmu->pdptrs[3 ] =vmcs_read64 (GUEST_PDPTR3 ); -kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); +kvm_register_mark_available (vcpu,VCPU_EXREG_PDPTR ); }#define CR3_EXITING_BITS (CPU_BASED_CR3_LOAD_EXITING | \
Signature · 1 line
2.19.1.6.gb485710b
PATCH 3/15 KVM: SVM: Always clear available of VCPU_EXREG_PDPTR in svm_vcpu_run()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Make it the same logic to handle the availability of VCPU_EXREG_PDPTR as VMX and also remove a branch in svm_vcpu_run().
--- arch/x86/kvm/svm/svm.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/svm.c +9 -4
@@ -1583,10 +1583,16 @@ static void svm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags)static void svm_cache_reg (struct kvm_vcpu *vcpu,enum kvm_reg reg) { +kvm_register_mark_available (vcpu, reg); +switch (reg) {case VCPU_EXREG_PDPTR : -BUG_ON (!npt_enabled); -load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu)); +/* +* When !npt_enabled, mmu->pdptrs[] is already available since +* it is always updated per SDM when moving to CRs. +*/ +if (npt_enabled) +load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu));break ;default :KVM_BUG_ON (1 , vcpu->kvm);
@@ -3964,8 +3970,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) svm->vmcb->control.tlb_ctl =TLB_CONTROL_DO_NOTHING ;vmcb_mark_all_clean (svm->vmcb); -if (npt_enabled) -kvm_register_clear_available (vcpu,VCPU_EXREG_PDPTR ); +kvm_register_clear_available (vcpu,VCPU_EXREG_PDPTR );/* * We need to handle MC intercepts here before the vcpu has a chance to
Signature · 1 line
2.19.1.6.gb485710b
PATCH 4/15 KVM: VMX: Add and use X86_CR4_TLB_BITS when !enable_ept
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
In set_cr4_guest_host_mask(), X86_CR4_PGE is set to be intercepted when !enable_ept just because X86_CR4_PGE is the only bit that is responsible for flushing TLB but listed in KVM_POSSIBLE_CR4_GUEST_BITS.
It is clearer and self-documented to use X86_CR4_TLB_BITS instead.
No functionality changed.
--- arch/x86/kvm/kvm_cache_regs.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/kvm_cache_regs.h +2 -0
@@ -9,6 +9,8 @@ (X86_CR4_PVI |X86_CR4_DE |X86_CR4_PCE |X86_CR4_OSFXSR \ | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD | X86_CR4_FSGSBASE) +#define X86_CR4_TLB_BITS (X86_CR4_PGE | X86_CR4_PCIDE | X86_CR4_PAE | X86_CR4_SMEP) +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ { \
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -4107,7 +4107,7 @@ void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) vcpu->arch.cr4_guest_owned_bits =KVM_POSSIBLE_CR4_GUEST_BITS & ~vcpu->arch.cr4_guest_rsvd_bits;if (!enable_ept) - vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_PGE ; + vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_TLB_BITS ;if (is_guest_mode (&vmx->vcpu)) vcpu->arch.cr4_guest_owned_bits &= ~get_vmcs12 (vcpu)->cr4_guest_host_mask;
Signature · 1 line
2.19.1.6.gb485710b
Paolo Bonzini
maintainerRe: [PATCH 04/15] KVM: VMX: Add and use X86_CR4_TLB_BITS when !enable_ept
Quoted 7 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
In
set_cr4_guest_host_mask(),X86_CR4_PGEis set to be intercepted when !enable_ept just becauseX86_CR4_PGEis the only bit that is responsible for flushing TLB but listed inKVM_POSSIBLE_CR4_GUEST_BITS.It is clearer and self-documented to use
X86_CR4_TLB_BITSinstead.
Very good idea, but I'd go with a slightly clearer X86_CR4_TLBFLUSH_BITS.
Paolo
Quoted 35 lines
No functionality changed.
Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/kvm_cache_regs.h | 2 ++ arch/x86/kvm/vmx/vmx.c | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-)arch/x86/kvm/kvm_cache_regs.h +2
-0@@ -9,6 +9,8 @@ (X86_CR4_PVI |X86_CR4_DE |X86_CR4_PCE |X86_CR4_OSFXSR \ | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD | X86_CR4_FSGSBASE) +#define X86_CR4_TLB_BITS (X86_CR4_PGE | X86_CR4_PCIDE | X86_CR4_PAE | X86_CR4_SMEP) +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ { \arch/x86/kvm/vmx/vmx.c +1
-1@@ -4107,7 +4107,7 @@ void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) vcpu->arch.cr4_guest_owned_bits =KVM_POSSIBLE_CR4_GUEST_BITS & ~vcpu->arch.cr4_guest_rsvd_bits;if (!enable_ept) - vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_PGE ; + vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_TLB_BITS ;if (is_guest_mode (&vmx->vcpu)) vcpu->arch.cr4_guest_owned_bits &= ~get_vmcs12 (vcpu)->cr4_guest_host_mask;
PATCH 5/15 KVM: VMX: Add and use X86_CR4_PDPTR_BITS when !enable_ept
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
In set_cr4_guest_host_mask(), all cr4 pdptr bits are already set to be intercepted in an unclear way.
Add X86_CR4_PDPTR_BITS to make it clear and self-documented.
No functionality changed.
--- arch/x86/kvm/kvm_cache_regs.h | 1 + arch/x86/kvm/vmx/vmx.c | 4 +++- arch/x86/kvm/x86.c | 4 +--- 3 files changed, 5 insertions(+), 4 deletions(-)
arch/x86/kvm/kvm_cache_regs.h +1 -0
@@ -10,6 +10,7 @@ | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD | X86_CR4_FSGSBASE)#define X86_CR4_TLB_BITS (X86_CR4_PGE | X86_CR4_PCIDE | X86_CR4_PAE | X86_CR4_SMEP) +#define X86_CR4_PDPTR_BITS (X86_CR4_PGE | X86_CR4_PSE | X86_CR4_PAE | X86_CR4_SMEP)#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\
arch/x86/kvm/vmx/vmx.c +3 -1
@@ -4106,8 +4106,10 @@ void set_cr4_guest_host_mask(struct vcpu_vmx *vmx) vcpu->arch.cr4_guest_owned_bits =KVM_POSSIBLE_CR4_GUEST_BITS & ~vcpu->arch.cr4_guest_rsvd_bits; -if (!enable_ept) +if (!enable_ept) { vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_TLB_BITS ; + vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_PDPTR_BITS ; + }if (is_guest_mode (&vmx->vcpu)) vcpu->arch.cr4_guest_owned_bits &= ~get_vmcs12 (vcpu)->cr4_guest_host_mask;
arch/x86/kvm/x86.c +1 -3
@@ -1052,8 +1052,6 @@ EXPORT_SYMBOL_GPL(kvm_post_set_cr4);int kvm_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4) {unsigned long old_cr4 =kvm_read_cr4 (vcpu); -unsigned long pdptr_bits =X86_CR4_PGE |X86_CR4_PSE |X86_CR4_PAE | -X86_CR4_SMEP ;if (!kvm_is_valid_cr4 (vcpu, cr4))return 1 ;
@@ -1064,7 +1062,7 @@ int kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)if ((cr4 ^ old_cr4) &X86_CR4_LA57 )return 1 ; }else if (is_paging (vcpu) && (cr4 &X86_CR4_PAE ) - && ((cr4 ^ old_cr4) & pdptr_bits) + && ((cr4 ^ old_cr4) &X86_CR4_PDPTR_BITS ) && !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu)))return 1 ;
Signature · 1 line
2.19.1.6.gb485710b
PATCH 6/15 KVM: X86: Move CR0 pdptr_bits into header file as X86_CR0_PDPTR_BITS
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Not functionality changed.
--- arch/x86/kvm/kvm_cache_regs.h | 3 +++ arch/x86/kvm/x86.c | 3 +-- 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/kvm_cache_regs.h +3 -0
@@ -9,9 +9,12 @@ (X86_CR4_PVI |X86_CR4_DE |X86_CR4_PCE |X86_CR4_OSFXSR \ | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD | X86_CR4_FSGSBASE) +#define X86_CR0_PDPTR_BITS (X86_CR0_CD | X86_CR0_NW | X86_CR0_PG)#define X86_CR4_TLB_BITS (X86_CR4_PGE | X86_CR4_PCIDE | X86_CR4_PAE | X86_CR4_SMEP)#define X86_CR4_PDPTR_BITS (X86_CR4_PGE | X86_CR4_PSE | X86_CR4_PAE | X86_CR4_SMEP) +static_assert (!(KVM_POSSIBLE_CR0_GUEST_BITS &X86_CR0_PDPTR_BITS )); +#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\ { \
arch/x86/kvm/x86.c +1 -2
@@ -858,7 +858,6 @@ EXPORT_SYMBOL_GPL(kvm_post_set_cr0);int kvm_set_cr0 (struct kvm_vcpu *vcpu,unsigned long cr0) {unsigned long old_cr0 =kvm_read_cr0 (vcpu); -unsigned long pdptr_bits =X86_CR0_CD |X86_CR0_NW |X86_CR0_PG ; cr0 |=X86_CR0_ET ;
@@ -888,7 +887,7 @@ int kvm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) }#endif if (!(vcpu->arch.efer &EFER_LME ) && (cr0 &X86_CR0_PG ) && -is_pae (vcpu) && ((cr0 ^ old_cr0) & pdptr_bits) && +is_pae (vcpu) && ((cr0 ^ old_cr0) &X86_CR0_PDPTR_BITS ) && !load_pdptrs (vcpu, vcpu->arch.walk_mmu,kvm_read_cr3 (vcpu)))return 1 ;
Signature · 1 line
2.19.1.6.gb485710b
PATCH 7/15 KVM: SVM: Remove outdated comment in svm_load_mmu_pgd()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
The comment had been added in the commit 689f3bf21628 ("KVM: x86: unify callbacks to load paging root") and its related code was removed later, and it has nothing to do with the next line of code.
So the comment should be removed too.
--- arch/x86/kvm/svm/svm.c | 1 - 1 file changed, 1 deletion(-)
arch/x86/kvm/svm/svm.c +0 -1
@@ -4000,7 +4000,6 @@ static void svm_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,hv_track_root_tdp (vcpu, root_hpa); -/* Loading L2's CR3 is handled by enter_svm_guest_mode. */ if (!test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail))return ; cr3 = vcpu->arch.cr3;
Signature · 1 line
2.19.1.6.gb485710b
PATCH 8/15 KVM: SVM: Remove useless check in svm_load_mmu_pgd()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
VCPU_EXREG_CR3 is never cleared from vcpu->arch.regs_avail in SVM so the if-branch is always false and useless, just remove it.
--- arch/x86/kvm/svm/svm.c | 2 -- 1 file changed, 2 deletions(-)
arch/x86/kvm/svm/svm.c +0 -2
@@ -4000,8 +4000,6 @@ static void svm_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,hv_track_root_tdp (vcpu, root_hpa); -if (!test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) -return ; cr3 = vcpu->arch.cr3; }else if (vcpu->arch.mmu->shadow_root_level >=PT64_ROOT_4LEVEL ) { cr3 =__sme_set (root_hpa) |kvm_get_active_pcid (vcpu);
Signature · 1 line
2.19.1.6.gb485710b
PATCH 9/15 KVM: SVM: Remove the unneeded code to mark available for CR3
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
VCPU_EXREG_CR3 is never cleared from vcpu->arch.regs_avail in SVM so marking available for CR3 is mere an NOP, just remove it.
And it is not required to mark it dirty since VCPU_EXREG_CR3 is neither never cleared from vcpu->arch.regs_dirty and SVM doesn't use the dirty information of VCPU_EXREG_CR3.
--- arch/x86/kvm/svm/nested.c | 1 - 1 file changed, 1 deletion(-)
arch/x86/kvm/svm/nested.c +0 -1
@@ -444,7 +444,6 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,kvm_mmu_new_pgd (vcpu, cr3); vcpu->arch.cr3 = cr3; -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 );/* Re-initialize the MMU, e.g. to pick up CR4 MMU role changes. */ kvm_init_mmu (vcpu);
Signature · 1 line
2.19.1.6.gb485710b
Paolo Bonzini
maintainerRe: [PATCH 09/15] KVM: SVM: Remove the unneeded code to mark available for CR3
Quoted 20 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
VCPU_EXREG_CR3is never cleared fromvcpu->arch.regs_availin SVM so marking available for CR3 is mere an NOP, just remove it.And it is not required to mark it dirty since
VCPU_EXREG_CR3is neither never cleared fromvcpu->arch.regs_dirtyand SVM doesn't use the dirty information ofVCPU_EXREG_CR3.Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/svm/nested.c | 1 - 1 file changed, 1 deletion(-)arch/x86/kvm/svm/nested.c +0
-0
arch/x86/kvm/svm/nested.c static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, · line 449
Quoted 7 lines
arch/x86/kvm/svm/nested.c +0
-1@@ -444,7 +444,6 @@ static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,kvm_mmu_new_pgd (vcpu, cr3); vcpu->arch.cr3 = cr3; -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 );/* Re-initialize the MMU, e.g. to pick up CR4 MMU role changes. */ kvm_init_mmu (vcpu);
These two patches can be merged, I think, with a commit message like
VCPU_EXREG_CR3 is never cleared from vcpu->arch.regs_avail or
vcpu->arch.regs_dirty in SVM; therefore, marking CR3 as available is
merely a NOP, and testing it will likewise always succeed.Paolo
PATCH 10/15 KVM: X86: Mark CR3 dirty when vcpu->arch.cr3 is changed
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
When vcpu->arch.cr3 is changed, it should be marked dirty unless it is being updated to the value of the architecture guest CR3 (i.e. VMX.GUEST_CR3 or vmcb->save.cr3 when tdp is enabled).
This patch has no functionality changed because kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3) is superset of kvm_register_mark_available(vcpu, VCPU_EXREG_CR3) with additional change to vcpu->arch.regs_dirty, but no code uses regs_dirty for VCPU_EXREG_CR3. (vmx_load_mmu_pgd() uses vcpu->arch.regs_avail instead to test if VCPU_EXREG_CR3 dirty which means current code (ab)uses regs_avail for VCPU_EXREG_CR3 dirty information.)
--- arch/x86/kvm/vmx/nested.c | 2 +- arch/x86/kvm/x86.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/nested.c +1 -1
@@ -1134,7 +1134,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,kvm_mmu_new_pgd (vcpu, cr3); vcpu->arch.cr3 = cr3; -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );/* Re-initialize the MMU, e.g. to pick up CR4 MMU role changes. */ kvm_init_mmu (vcpu);
arch/x86/kvm/x86.c +2 -2
@@ -1159,7 +1159,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)kvm_mmu_new_pgd (vcpu, cr3); vcpu->arch.cr3 = cr3; -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); handle_tlb_flush:/*
@@ -10591,7 +10591,7 @@ static int __set_sregs_common(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs, vcpu->arch.cr2 = sregs->cr2; *mmu_reset_needed |=kvm_read_cr3 (vcpu) != sregs->cr3; vcpu->arch.cr3 = sregs->cr3; -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );kvm_set_cr8 (vcpu, sregs->cr8);
Signature · 1 line
2.19.1.6.gb485710b
PATCH 11/15 KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
When vcpu->arch.cr3 is changed, it is marked dirty, so vmcs.GUEST_CR3 can be updated only when kvm_register_is_dirty(vcpu, VCPU_EXREG_CR3).
--- arch/x86/kvm/vmx/vmx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx/vmx.c +2 -2
@@ -3126,9 +3126,9 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,if (!enable_unrestricted_guest && !is_paging (vcpu)) guest_cr3 =to_kvm_vmx (kvm)->ept_identity_map_addr; -else if (test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) +else if (kvm_register_is_dirty (vcpu,VCPU_EXREG_CR3 )) guest_cr3 = vcpu->arch.cr3; -else /* vmcs01.GUEST_CR3 is already up-to-date. */ +else /* vmcs.GUEST_CR3 is already up-to-date. */ update_guest_cr3 = false;vmx_ept_load_pdptrs (vcpu); }else {
Signature · 1 line
2.19.1.6.gb485710b
Re: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
Quoted 16 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
When
vcpu->arch.cr3is changed, it is marked dirty, so vmcs.GUEST_CR3 can be updated only whenkvm_register_is_dirty(vcpu, VCPU_EXREG_CR3).Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/vmx.c +0
-0@@ -3126,9 +3126,9 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,
arch/x86/kvm/vmx/vmx.c static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, · line 3134
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +2
-2if (!enable_unrestricted_guest && !is_paging (vcpu)) guest_cr3 =to_kvm_vmx (kvm)->ept_identity_map_addr; -else if (test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) +else if (kvm_register_is_dirty (vcpu,VCPU_EXREG_CR3 )) guest_cr3 = vcpu->arch.cr3; -else /* vmcs01.GUEST_CR3 is already up-to-date. */ +else /* vmcs.GUEST_CR3 is already up-to-date. */ update_guest_cr3 = false;vmx_ept_load_pdptrs (vcpu); }else {
I just bisected this patch to break booting a VM with ept=1 but unrestricted_guest=0 (I needed to re-test unrestricted_guest=0 bug related to SMM, but didn't want to boot without EPT. With ept=0,the VM boots with this patch applied).
Reverting this patch on top of kvm/queue also works.
Best regards,
Maxim levitsky
Re: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
Quoted 17 lines · Maxim Levitsky
Quoted 16 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
When
vcpu->arch.cr3is changed, it is marked dirty, so vmcs.GUEST_CR3 can be updated only whenkvm_register_is_dirty(vcpu, VCPU_EXREG_CR3).Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/vmx.c +0
-0@@ -3126,9 +3126,9 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,
Quoted 16 lines
arch/x86/kvm/vmx/vmx.c static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, · line 3134
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +2
-2if (!enable_unrestricted_guest && !is_paging (vcpu)) guest_cr3 =to_kvm_vmx (kvm)->ept_identity_map_addr; -else if (test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) +else if (kvm_register_is_dirty (vcpu,VCPU_EXREG_CR3 )) guest_cr3 = vcpu->arch.cr3; -else /* vmcs01.GUEST_CR3 is already up-to-date. */ +else /* vmcs.GUEST_CR3 is already up-to-date. */ update_guest_cr3 = false;vmx_ept_load_pdptrs (vcpu); }else {I just bisected this patch to break booting a VM with ept=1 but unrestricted_guest=0 (I needed to re-test unrestricted_guest=0 bug related to SMM, but didn't want to boot without EPT. With ept=0,the VM boots with this patch applied).
Thanks for reporting.
Sorry, I never tested it with unrestricted_guest=0. I can't reproduce it now shortly with unrestricted_guest=0. Maybe it can be reproduced easily if I try more guests or I write a piece of guest code to deliberate hit it if the following analyses is correct.
All the paths changing %cr3 are followed with kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3) and GUEST_CR3 will be expected to be updated.
What I missed is the case of "if (!enable_unrestricted_guest && !is_paging(vcpu))" in vmx_load_mmu_pgd() which doesn't load GUEST_CR3 but clears dirty of VCPU_EXREG_CR3 (when after next run).
So when CR0 !PG -> PG, VCPU_EXREG_CR3 dirty bit should be set.
Maybe adding the following patch on top of the original patch can work.
Thanks
Lai
arch/x86/kvm/x86.c +1 -0
@@ -858,6 +858,7 @@ void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lonif ((cr0 ^ old_cr0) &X86_CR0_PG ) {kvm_clear_async_pf_completion_queue (vcpu);kvm_async_pf_hash_reset (vcpu); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); }if ((cr0 ^ old_cr0) &KVM_MMU_CR0_ROLE_BITS )
Re: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
Quoted 4 lines · Lai Jiangshan
What I missed is the case of "if (!enable_unrestricted_guest && !is_paging(vcpu))" in
vmx_load_mmu_pgd()which doesn't loadGUEST_CR3but clears dirty ofVCPU_EXREG_CR3(when after next run).
Oops.
What I missed is the case of "if (!enable_unrestricted_guest && !is_paging(vcpu))" in vmx_load_mmu_pgd() which doesn't load GUEST_CR3 with the guest cr3 and VCPU_EXREG_CR3 dirty bit is cleared after VMEXIT. When !PG -> PG, GUEST_CR3 is still the ept_identity_map_addr, and VCPU_EXREG_CR3 dirty bit is not set, so vmx_load_mmu_pgd() doesn't update GUEST_CR3.
Quoted 2 lines
So when CR0 !PG -> PG,
VCPU_EXREG_CR3dirty bit should be set.
Sean Christopherson
maintainerRe: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
In reply to Lai JiangshanQuoted 62 lines · Lai Jiangshan
Quoted 23 lines · Maxim Levitsky
Quoted 22 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
When
vcpu->arch.cr3is changed, it is marked dirty, so vmcs.GUEST_CR3 can be updated only whenkvm_register_is_dirty(vcpu, VCPU_EXREG_CR3).Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/vmx.c +2
-2@@ -3126,9 +3126,9 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,if (!enable_unrestricted_guest && !is_paging (vcpu)) guest_cr3 =to_kvm_vmx (kvm)->ept_identity_map_addr; -else if (test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) +else if (kvm_register_is_dirty (vcpu,VCPU_EXREG_CR3 )) guest_cr3 = vcpu->arch.cr3; -else /* vmcs01.GUEST_CR3 is already up-to-date. */ +else /* vmcs.GUEST_CR3 is already up-to-date. */ Quoted 9 lines
arch/x86/kvm/vmx/vmx.c static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, · line 3133
Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +0
-0update_guest_cr3 = false;vmx_ept_load_pdptrs (vcpu); }else {I just bisected this patch to break booting a VM with ept=1 but unrestricted_guest=0 (I needed to re-test unrestricted_guest=0 bug related to SMM, but didn't want to boot without EPT. With ept=0,the VM boots with this patch applied).
Thanks for reporting.
Sorry, I never tested it with unrestricted_guest=0. I can't reproduce it now shortly with unrestricted_guest=0. Maybe it can be reproduced easily if I try more guests or I write a piece of guest code to deliberate hit it if the following analyses is correct.
All the paths changing %cr3 are followed with
kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3)andGUEST_CR3will be expected to be updated.What I missed is the case of "if (!enable_unrestricted_guest && !is_paging(vcpu))" in
vmx_load_mmu_pgd()which doesn't loadGUEST_CR3but clears dirty ofVCPU_EXREG_CR3(when after next run).So when CR0 !PG -> PG,
VCPU_EXREG_CR3dirty bit should be set.Maybe adding the following patch on top of the original patch can work.
Thanks
Laiarch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lon · line 861
Quoted 4 lines
arch/x86/kvm/x86.c +1
-0@@ -858,6 +858,7 @@ void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lonif ((cr0 ^ old_cr0) &X86_CR0_PG ) {kvm_clear_async_pf_completion_queue (vcpu);kvm_async_pf_hash_reset (vcpu); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );
The better place for this is the path in vmx_set_cr0() that handles EPT + !URG. I believe nested_vmx_restore_host_state(), which is used to restore host state if hardware detects a VM-Fail that KVM misses on nested VM-Enter, is also missing a call to mark CR3 dirty.
arch/x86/kvm/vmx/nested.c +1 -1
@@ -4426,7 +4426,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)nested_ept_uninit_mmu_context (vcpu); vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );/* * Use ept_save_pdptrs(vcpu) to load the MMU's cached PDPTRs
arch/x86/kvm/vmx/vmx.c +3 -1
@@ -3066,8 +3066,10 @@ void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) }/* Note, vmx_set_cr4() consumes the new vcpu->arch.cr0. */ -if ((old_cr0_pg ^ cr0) &X86_CR0_PG ) +if ((old_cr0_pg ^ cr0) &X86_CR0_PG ) {vmx_set_cr4 (vcpu,kvm_read_cr4 (vcpu)); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); + } }/* depends on vcpu->arch.cr0 to be set to a new value */
Paolo Bonzini
maintainerRe: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
Quoted 10 lines · Sean Christopherson
The better place for this is the path in
vmx_set_cr0()that handles EPT + !URG. I believenested_vmx_restore_host_state(), which is used to restore host state if hardware detects a VM-Fail that KVM misses on nested VM-Enter, is also missing a call to mark CR3 dirty.arch/x86/kvm/vmx/nested.c +0
-0
arch/x86/kvm/vmx/nested.c static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu) · line 4432
Quoted 8 lines
arch/x86/kvm/vmx/nested.c +1
-1@@ -4426,7 +4426,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)nested_ept_uninit_mmu_context (vcpu); vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );/* * Use ept_save_pdptrs(vcpu) to load the MMU's cached PDPTRs
Yes, I agree. Jiangshan, please send all your fixes tomorrow, or tell me if you prefer that I push the reverts to kvm/next.
Paolo
Re: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
In reply to Sean ChristophersonQuoted 90 lines · Sean Christopherson
Quoted 59 lines · Lai Jiangshan
Quoted 22 lines · Maxim Levitsky
Quoted 21 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
When
vcpu->arch.cr3is changed, it is marked dirty, so vmcs.GUEST_CR3 can be updated only whenkvm_register_is_dirty(vcpu, VCPU_EXREG_CR3).Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/vmx.c +1
-2@@ -3126,9 +3126,9 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,if (!enable_unrestricted_guest && !is_paging (vcpu)) guest_cr3 =to_kvm_vmx (kvm)->ept_identity_map_addr; -else if (test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) +else if (kvm_register_is_dirty (vcpu,VCPU_EXREG_CR3 )) guest_cr3 = vcpu->arch.cr3; -else /* vmcs01.GUEST_CR3 is already up-to-date. */ Quoted 9 lines
arch/x86/kvm/vmx/vmx.c static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, · line 3133
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1
-0+else /* vmcs.GUEST_CR3 is already up-to-date. */ update_guest_cr3 = false;vmx_ept_load_pdptrs (vcpu); }else {I just bisected this patch to break booting a VM with ept=1 but unrestricted_guest=0 (I needed to re-test unrestricted_guest=0 bug related to SMM, but didn't want to boot without EPT. With ept=0,the VM boots with this patch applied).
Thanks for reporting.
Sorry, I never tested it with unrestricted_guest=0. I can't reproduce it now shortly with unrestricted_guest=0. Maybe it can be reproduced easily if I try more guests or I write a piece of guest code to deliberate hit it if the following analyses is correct.
All the paths changing %cr3 are followed with
kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3)andGUEST_CR3will be expected to be updated.What I missed is the case of "if (!enable_unrestricted_guest && !is_paging(vcpu))" in
vmx_load_mmu_pgd()which doesn't loadGUEST_CR3but clears dirty ofVCPU_EXREG_CR3(when after next run).So when CR0 !PG -> PG,
VCPU_EXREG_CR3dirty bit should be set.Maybe adding the following patch on top of the original patch can work.
Thanks
Laiarch/x86/kvm/x86.c +0
-0arch/x86/kvm/x86.c void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lon · line 861
Quoted 4 lines
arch/x86/kvm/x86.c +1
-0@@ -858,6 +858,7 @@ void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lonif ((cr0 ^ old_cr0) &X86_CR0_PG ) {kvm_clear_async_pf_completion_queue (vcpu);kvm_async_pf_hash_reset (vcpu); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );The better place for this is the path in
vmx_set_cr0()that handles EPT + !URG. I believenested_vmx_restore_host_state(), which is used to restore host state if hardware detects a VM-Fail that KVM misses on nested VM-Enter, is also missing a call to mark CR3 dirty.arch/x86/kvm/vmx/nested.c +1
-1@@ -4426,7 +4426,7 @@ static void nested_vmx_restore_host_state(struct kvm_vcpu *vcpu)nested_ept_uninit_mmu_context (vcpu); vcpu->arch.cr3 =vmcs_readl (GUEST_CR3 ); -kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 );/* * Use ept_save_pdptrs(vcpu) to load the MMU's cached PDPTRs arch/x86/kvm/vmx/vmx.c +0
-0@@ -3066,8 +3066,10 @@ void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) }
arch/x86/kvm/vmx/vmx.c void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · line 3075
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +3
-1/* Note, vmx_set_cr4() consumes the new vcpu->arch.cr0. */ -if ((old_cr0_pg ^ cr0) &X86_CR0_PG ) +if ((old_cr0_pg ^ cr0) &X86_CR0_PG ) {vmx_set_cr4 (vcpu,kvm_read_cr4 (vcpu)); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); + } }/* depends on vcpu->arch.cr0 to be set to a new value */
I tested this patch and my guests boot as well. I didn't test more stuff like migration or so, will do tomorrow.
Best regards,
Maxim Levitsky
Re: [PATCH 11/15] KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
In reply to Lai JiangshanQuoted 61 lines · Lai Jiangshan
Quoted 17 lines · Maxim Levitsky
Quoted 16 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
When
vcpu->arch.cr3is changed, it is marked dirty, so vmcs.GUEST_CR3 can be updated only whenkvm_register_is_dirty(vcpu, VCPU_EXREG_CR3).Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/vmx/vmx.c +0
-0@@ -3126,9 +3126,9 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa,Quoted 15 lines
arch/x86/kvm/vmx/vmx.c static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, · line 3134
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +2
-2if (!enable_unrestricted_guest && !is_paging (vcpu)) guest_cr3 =to_kvm_vmx (kvm)->ept_identity_map_addr; -else if (test_bit (VCPU_EXREG_CR3 , (ulong *)&vcpu->arch.regs_avail)) +else if (kvm_register_is_dirty (vcpu,VCPU_EXREG_CR3 )) guest_cr3 = vcpu->arch.cr3; -else /* vmcs01.GUEST_CR3 is already up-to-date. */ +else /* vmcs.GUEST_CR3 is already up-to-date. */ update_guest_cr3 = false;vmx_ept_load_pdptrs (vcpu); }else {I just bisected this patch to break booting a VM with ept=1 but unrestricted_guest=0 (I needed to re-test unrestricted_guest=0 bug related to SMM, but didn't want to boot without EPT. With ept=0,the VM boots with this patch applied).
Thanks for reporting.
Sorry, I never tested it with unrestricted_guest=0. I can't reproduce it now shortly with unrestricted_guest=0. Maybe it can be reproduced easily if I try more guests or I write a piece of guest code to deliberate hit it if the following analyses is correct.
All the paths changing %cr3 are followed with
kvm_register_mark_dirty(vcpu, VCPU_EXREG_CR3)andGUEST_CR3will be expected to be updated.What I missed is the case of "if (!enable_unrestricted_guest && !is_paging(vcpu))" in
vmx_load_mmu_pgd()which doesn't loadGUEST_CR3but clears dirty ofVCPU_EXREG_CR3(when after next run).So when CR0 !PG -> PG,
VCPU_EXREG_CR3dirty bit should be set.Maybe adding the following patch on top of the original patch can work.
Thanks
Laiarch/x86/kvm/x86.c +0
-0
arch/x86/kvm/x86.c void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lon · line 864
Quoted 7 lines
arch/x86/kvm/x86.c +1
-0@@ -858,6 +858,7 @@ void kvm_post_set_cr0(struct kvm_vcpu *vcpu, unsigned long old_cr0, unsigned lonif ((cr0 ^ old_cr0) &X86_CR0_PG ) {kvm_clear_async_pf_completion_queue (vcpu);kvm_async_pf_hash_reset (vcpu); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); }if ((cr0 ^ old_cr0) &KVM_MMU_CR0_ROLE_BITS )
Tested this patch and my guests boot. I didn't test more stuff like migration or so, will do tomorrow.
Best regards,
Maxim Levitsky
PATCH 12/15 KVM: VMX: Reset the bits that are meaningful to be reset in vmx_register_cache_reset()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
Add meaningful bits as VMX_REGS_AVAIL_SET and VMX_REGS_DIRTY_SET.
--- arch/x86/kvm/vmx/vmx.h | 37 +++++++++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx/vmx.h +27 -10
@@ -437,16 +437,33 @@ BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL)static inline void vmx_register_cache_reset (struct kvm_vcpu *vcpu) { - vcpu->arch.regs_avail = ~((1 <<VCPU_REGS_RIP ) | (1 <<VCPU_REGS_RSP ) - | (1 <<VCPU_EXREG_RFLAGS ) - | (1 <<VCPU_EXREG_PDPTR ) - | (1 <<VCPU_EXREG_SEGMENTS ) - | (1 <<VCPU_EXREG_CR0 ) - | (1 <<VCPU_EXREG_CR3 ) - | (1 <<VCPU_EXREG_CR4 ) - | (1 <<VCPU_EXREG_EXIT_INFO_1 ) - | (1 <<VCPU_EXREG_EXIT_INFO_2 )); - vcpu->arch.regs_dirty =0 ; +/* +* VMX_REGS_AVAIL_SET - The set of registers that will be updated in cache on +* demand. Other registers not listed here are synced to +* the cache immediately after VM-Exit. +* +* VMX_REGS_DIRTY_SET - The set of registers that might be outdated in +* architecture. Other registers not listed here are synced +* to the architecture immediately when modifying. +*/ +#define VMX_REGS_AVAIL_SET ((1 << VCPU_REGS_RIP) |\ + (1 << VCPU_REGS_RSP) |\ + (1 << VCPU_EXREG_RFLAGS) |\ + (1 << VCPU_EXREG_PDPTR) |\ + (1 << VCPU_EXREG_SEGMENTS) |\ + (1 << VCPU_EXREG_CR0) |\ + (1 << VCPU_EXREG_CR3) |\ + (1 << VCPU_EXREG_CR4) |\ + (1 << VCPU_EXREG_EXIT_INFO_1) |\ + (1 << VCPU_EXREG_EXIT_INFO_2)) + +#define VMX_REGS_DIRTY_SET ((1 << VCPU_REGS_RIP) |\ + (1 << VCPU_REGS_RSP) |\ + (1 << VCPU_EXREG_PDPTR) |\ + (1 << VCPU_EXREG_CR3)) + + vcpu->arch.regs_avail &= ~VMX_REGS_AVAIL_SET ; + vcpu->arch.regs_dirty &= ~VMX_REGS_DIRTY_SET ; }static inline struct kvm_vmx *to_kvm_vmx (struct kvm *kvm)
Signature · 1 line
2.19.1.6.gb485710b
Paolo Bonzini
maintainerRe: [PATCH 12/15] KVM: VMX: Reset the bits that are meaningful to be reset in vmx_register_cache_reset()
Quoted 8 lines · Lai Jiangshan
+/*
+ *VMX_REGS_AVAIL_SET- The set of registers that will be updated in cache on
+ * demand. Other registers not listed here are synced to
+ * the cache immediately after VM-Exit.
+ *
+ *VMX_REGS_DIRTY_SET- The set of registers that might be outdated in
+ * architecture. Other registers not listed here are synced
+ * to the architecture immediately when modifying.
Slightly more expressive:
/*
* VMX_REGS_LAZY_LOAD_SET - The set of registers that will be updated in the
* cache on demand. Other registers not listed here are synced to
* the cache immediately after VM-Exit.
*/
...
/*
* VMX_REGS_LAZY_UPDATE_SET - The set of registers that might be outdated in
* VMCS. Other registers not listed here are synced to the VMCS
* immediately when modified.
*/
...
BUILD_BUG_ON(VMX_REGS_LAZY_UPDATE_SET & ~VMX_REGS_LAZY_LOAD_SET);
vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET;
vcpu->arch.regs_dirty &= ~VMX_REGS_LAZY_UPDATE_SET;That is lazily loaded registers become unavailable, and lazily updated registers become unavailable and dirty.
Paolo
PATCH 13/15 KVM: SVM: Add and use svm_register_cache_reset()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
It resets all the appropriate bits like vmx.
--- arch/x86/kvm/svm/svm.c | 3 +-- arch/x86/kvm/svm/svm.h | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/svm.c +1 -2
@@ -3969,8 +3969,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) svm->vmcb->control.tlb_ctl =TLB_CONTROL_DO_NOTHING ;vmcb_mark_all_clean (svm->vmcb); - -kvm_register_clear_available (vcpu,VCPU_EXREG_PDPTR ); +svm_register_cache_reset (vcpu);/* * We need to handle MC intercepts here before the vcpu has a chance to
arch/x86/kvm/svm/svm.h +26 -0
@@ -274,6 +274,32 @@ static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit)return !test_bit (bit, (unsigned long *)&vmcb->control.clean); } +static inline void svm_register_cache_reset (struct kvm_vcpu *vcpu) +{ +/* +* SVM_REGS_AVAIL_SET - The set of registers that will be updated in cache on +* demand. Other registers not listed here are synced to +* the cache immediately after VM-Exit. +* +* SVM_REGS_DIRTY_SET - The set of registers that might be outdated in +* architecture. Other registers not listed here are synced +* to the architecture immediately when modifying. +* +* Special case: VCPU_EXREG_CR3 should be in this set due +* to the fact. But KVM_REQ_LOAD_MMU_PGD is always +* requested when the cache vcpu->arch.cr3 is changed and +* svm_load_mmu_pgd() always syncs the new CR3 value into +* the architecture. So the dirty information of +* VCPU_EXREG_CR3 is not used which means VCPU_EXREG_CR3 +* isn't required to be put in this set. +*/ +#define SVM_REGS_AVAIL_SET (1 << VCPU_EXREG_PDPTR) +#define SVM_REGS_DIRTY_SET (0) + + vcpu->arch.regs_avail &= ~SVM_REGS_AVAIL_SET ; + vcpu->arch.regs_dirty &= ~SVM_REGS_DIRTY_SET ; +} +static inline struct vcpu_svm *to_svm (struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_svm, vcpu);
Signature · 1 line
2.19.1.6.gb485710b
Paolo Bonzini
maintainerRe: [PATCH 13/15] KVM: SVM: Add and use svm_register_cache_reset()
Quoted 47 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
It resets all the appropriate bits like vmx.
Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/svm/svm.c | 3 +-- arch/x86/kvm/svm/svm.h | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+), 2 deletions(-)arch/x86/kvm/svm/svm.c +1
-2@@ -3969,8 +3969,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) svm->vmcb->control.tlb_ctl =TLB_CONTROL_DO_NOTHING ;vmcb_mark_all_clean (svm->vmcb); - -kvm_register_clear_available (vcpu,VCPU_EXREG_PDPTR ); +svm_register_cache_reset (vcpu);/* * We need to handle MC intercepts here before the vcpu has a chance to arch/x86/kvm/svm/svm.h +15
-0@@ -274,6 +274,32 @@ static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit)return !test_bit (bit, (unsigned long *)&vmcb->control.clean); } +static inline void svm_register_cache_reset (struct kvm_vcpu *vcpu) +{ +/* +* SVM_REGS_AVAIL_SET - The set of registers that will be updated in cache on +* demand. Other registers not listed here are synced to +* the cache immediately after VM-Exit. +* +* SVM_REGS_DIRTY_SET - The set of registers that might be outdated in +* architecture. Other registers not listed here are synced +* to the architecture immediately when modifying. +* +* Special case: VCPU_EXREG_CR3 should be in this set due +* to the fact. But KVM_REQ_LOAD_MMU_PGD is always +* requested when the cache vcpu->arch.cr3 is changed and +* svm_load_mmu_pgd() always syncs the new CR3 value into
arch/x86/kvm/svm/svm.h static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit) · line 301
Quoted 10 lines
arch/x86/kvm/svm/svm.h +10
-0+* the architecture. So the dirty information of +* VCPU_EXREG_CR3 is not used which means VCPU_EXREG_CR3 +* isn't required to be put in this set. +*/ +#define SVM_REGS_AVAIL_SET (1 << VCPU_EXREG_PDPTR) +#define SVM_REGS_DIRTY_SET (0) + + vcpu->arch.regs_avail &= ~SVM_REGS_AVAIL_SET ; + vcpu->arch.regs_dirty &= ~SVM_REGS_DIRTY_SET ; +}
I think touching regs_dirty is confusing here, so I'd go with this:
vcpu->arch.regs_avail &= ~SVM_REGS_LAZY_LOAD_SET;
/*
* SVM does not use vcpu->arch.regs_dirty. The only register that
* might be out of date in the VMCB is CR3, but KVM_REQ_LOAD_MMU_PGD
* is always requested when the cache vcpu->arch.cr3 is changed and
* svm_load_mmu_pgd() always syncs the new CR3 value into the VMCB. */
(VMX instead needs VCPU_EXREG_CR3 mostly because it does not want to update it unconditionally on exit).
Paolo
Re: [PATCH 13/15] KVM: SVM: Add and use svm_register_cache_reset()
Quoted 49 lines · Paolo Bonzini
Quoted 48 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
It resets all the appropriate bits like vmx.
Signed-off-by Lai Jiangshan <[email protected]>---
arch/x86/kvm/svm/svm.c| 3 +--arch/x86/kvm/svm/svm.h| 26 ++++++++++++++++++++++++++2 files changed, 27 insertions(+), 2 deletions(-)arch/x86/kvm/svm/svm.c +1
-2@@ -3969,8 +3969,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) svm->vmcb->control.tlb_ctl = TLB_CONTROL_DO_NOTHING; vmcb_mark_all_clean(svm->vmcb); - - kvm_register_clear_available(vcpu, VCPU_EXREG_PDPTR); + svm_register_cache_reset(vcpu); /** We need to handle MC intercepts here before the vcpu has a chance to arch/x86/kvm/svm/svm.h +19
-0@@ -274,6 +274,32 @@ static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit) return !test_bit(bit, (unsigned long *)&vmcb->control.clean); } +static inline void svm_register_cache_reset (struct kvm_vcpu *vcpu) +{ +/* +* SVM_REGS_AVAIL_SET - The set of registers that will be updated in cache on +* demand. Other registers not listed here are synced to +* the cache immediately after VM-Exit. +* +* SVM_REGS_DIRTY_SET - The set of registers that might be outdated in +* architecture. Other registers not listed here are synced +* to the architecture immediately when modifying. +* +* Special case: VCPU_EXREG_CR3 should be in this set due +* to the fact. But KVM_REQ_LOAD_MMU_PGD is always +* requested when the cache vcpu->arch.cr3 is changed and +* svm_load_mmu_pgd() always syncs the new CR3 value into +* the architecture. So the dirty information of +* VCPU_EXREG_CR3 is not used which means VCPU_EXREG_CR3 +* isn't required to be put in this set. +*/
Quoted 8 lines
arch/x86/kvm/svm/svm.h static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit) · line 301
Quoted 6 lines
arch/x86/kvm/svm/svm.h +6
-0+#define SVM_REGS_AVAIL_SET (1 << VCPU_EXREG_PDPTR) +#define SVM_REGS_DIRTY_SET (0) + + vcpu->arch.regs_avail &= ~SVM_REGS_AVAIL_SET; + vcpu->arch.regs_dirty &= ~SVM_REGS_DIRTY_SET; +}I think touching
regs_dirtyis confusing here, so I'd go with this:
It makes the code the same as vmx by clearing all the SVM_REGS_DIRTY_SET bits. And the compiler will remove this line of code since SVM_REGS_DIRTY_SET is (0), and regs_dirty is not touched.
Using VMX_REGS_DIRTY_SET and SVM_REGS_DIRTY_SET and making the code similar is my intent for patch12,13. If it causes confusing, I would like to make a second thought. SVM_REGS_DIRTY_SET does be special in svm where VCPU_EXREG_CR3 is in it by definition, but it is not added into SVM_REGS_DIRTY_SET in the patch just for optimization to allow the compiler optimizes the line of code out.
I'm Ok to not queue patch12,13,14.
Quoted 14 lines
vcpu->arch.regs_avail&= ~SVM_REGS_LAZY_LOAD_SET;/*
* SVM does not use vcpu->arch.regs_dirty. The only register that
* might be out of date in the VMCB is CR3, butKVM_REQ_LOAD_MMU_PGD
* is always requested when the cachevcpu->arch.cr3is changed and
*svm_load_mmu_pgd()always syncs the new CR3 value into the VMCB. */(VMX instead needs
VCPU_EXREG_CR3mostly because it does not want to update it unconditionally on exit).Paolo
Paolo Bonzini
maintainerRe: [PATCH 13/15] KVM: SVM: Add and use svm_register_cache_reset()
Quoted 6 lines · Lai Jiangshan
Using
VMX_REGS_DIRTY_SETandSVM_REGS_DIRTY_SETand making the code similar is my intent for patch12,13. If it causes confusing, I would like to make a second thought.SVM_REGS_DIRTY_SETdoes be special in svm whereVCPU_EXREG_CR3is in it by definition, but it is not added intoSVM_REGS_DIRTY_SETin the patch just for optimization to allow the compiler optimizes the line of code out.
I think this is where we disagree. In my opinion it is enough to document that CR3 _can_ be out of date, but it doesn't have to be marked dirty because its dirty bit is effectively KVM_REQ_LOAD_MMU_PGD.
For VMX, it is important to clear VCPU_EXREG_CR3 because the combination "avail=0, dirty=1" is nonsensical:
av d 0 0 in VMCS 0 1 *INVALID* 1 0 in vcpu->arch 1 1 in vcpu->arch, needs store
But on SVM, VCPU_EXREG_CR3 is always available.
Thinking more about it, it makes more sense for VMX to reset _all_ bits of dirty to 0, just like it was before your change, but doing so even earlier in vmx_vcpu_run.
I appreciate that VMX_REGS_LAZY_UPDATE_SET is useful for documentation, but it's also important that the values in avail/dirty make sense as a pair.
So here is what I would do:
arch/x86/kvm/kvm_cache_regs.h +7 -0
@@ -43,6 +43,13 @@ BUILD_KVM_GPR_ACCESSORS(r14, R14) BUILD_KVM_GPR_ACCESSORS(r15, R15)#endif +/* +* avail dirty +* 0 0 register in VMCS/VMCB +* 0 1 *INVALID* +* 1 0 register in vcpu->arch +* 1 1 register in vcpu->arch, needs to be stored back +*/ static inline bool kvm_register_is_available (struct kvm_vcpu *vcpu,enum kvm_reg reg) {
arch/x86/kvm/vmx/vmx.c +2 -1
@@ -6635,6 +6635,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)vmcs_writel (GUEST_RSP , vcpu->arch.regs[VCPU_REGS_RSP ]);if (kvm_register_is_dirty (vcpu,VCPU_REGS_RIP ))vmcs_writel (GUEST_RIP , vcpu->arch.regs[VCPU_REGS_RIP ]); + vcpu->arch.regs_dirty =0 ; cr3 =__get_current_cr3_fast ();if (unlikely (cr3 != vmx->loaded_vmcs->host_state.cr3)) {
@@ -6729,7 +6730,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)loadsegment (es, __USER_DS);#endif -vmx_register_cache_reset (vcpu); + vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET ;pt_guest_exit (vmx);
arch/x86/kvm/vmx/vmx.h +15 -13
@@ -473,19 +473,21 @@ BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL)BUILD_CONTROLS_SHADOW (secondary_exec,SECONDARY_VM_EXEC_CONTROL ) -static inline void vmx_register_cache_reset (struct kvm_vcpu *vcpu) -{ - vcpu->arch.regs_avail = ~((1 <<VCPU_REGS_RIP ) | (1 <<VCPU_REGS_RSP ) - | (1 <<VCPU_EXREG_RFLAGS ) - | (1 <<VCPU_EXREG_PDPTR ) - | (1 <<VCPU_EXREG_SEGMENTS ) - | (1 <<VCPU_EXREG_CR0 ) - | (1 <<VCPU_EXREG_CR3 ) - | (1 <<VCPU_EXREG_CR4 ) - | (1 <<VCPU_EXREG_EXIT_INFO_1 ) - | (1 <<VCPU_EXREG_EXIT_INFO_2 )); - vcpu->arch.regs_dirty =0 ; -} +/* +* VMX_REGS_LAZY_LOAD_SET - The set of registers that will be updated in the +* cache on demand. Other registers not listed here are synced to +* the cache immediately after VM-Exit. + */ +#define VMX_REGS_LAZY_LOAD_SET ((1 << VCPU_REGS_RIP) | \ + (1 << VCPU_REGS_RSP) | \ + (1 << VCPU_EXREG_RFLAGS) | \ + (1 << VCPU_EXREG_PDPTR) | \ + (1 << VCPU_EXREG_SEGMENTS) | \ + (1 << VCPU_EXREG_CR0) | \ + (1 << VCPU_EXREG_CR3) | \ + (1 << VCPU_EXREG_CR4) | \ + (1 << VCPU_EXREG_EXIT_INFO_1) | \ + (1 << VCPU_EXREG_EXIT_INFO_2))static inline struct kvm_vmx *to_kvm_vmx (struct kvm *kvm) {
and likewise for SVM:
arch/x86/kvm/svm/svm.c +2 -1
@@ -3944,6 +3944,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) vcpu->arch.regs[VCPU_REGS_RSP ] = svm->vmcb->save.rsp; vcpu->arch.regs[VCPU_REGS_RIP ] = svm->vmcb->save.rip; } + vcpu->arch.regs_dirty =0 ;if (unlikely (svm->vmcb->control.exit_code ==SVM_EXIT_NMI ))kvm_before_interrupt (vcpu);
@@ -3978,7 +3978,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); -kvm_register_clear_available (vcpu,VCPU_EXREG_PDPTR ); + vcpu->arch.regs_avail &= ~SVM_REGS_LAZY_LOAD_SET ;/* * We need to handle MC intercepts here before the vcpu has a chance to
arch/x86/kvm/svm/svm.h +10 -0
@@ -321,6 +321,16 @@ static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit)return !test_bit (bit, (unsigned long *)&vmcb->control.clean); } +/* +* Only the PDPTRs are loaded on demand into the shadow MMU. All other +* fields are synchronized in handle_exit, because accessing the VMCB is cheap. +* +* CR3 might be out of date in the VMCB but it is not marked dirty; instead, +* KVM_REQ_LOAD_MMU_PGD is always requested when the cached vcpu->arch.cr3 +* is changed. svm_load_mmu_pgd() then syncs the new CR3 value into the VMCB. +*/ +#define SVM_REGS_LAZY_LOAD_SET (1 << VCPU_EXREG_PDPTR) +static inline struct vcpu_svm *to_svm (struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_svm, vcpu);
Lai Jiangshan
Reviewed-byRe: [PATCH 13/15] KVM: SVM: Add and use svm_register_cache_reset()
Quoted 27 lines · Paolo Bonzini
Quoted 6 lines · Lai Jiangshan
Using
VMX_REGS_DIRTY_SETandSVM_REGS_DIRTY_SETand making the code similar is my intent for patch12,13. If it causes confusing, I would like to make a second thought.SVM_REGS_DIRTY_SETdoes be special in svm whereVCPU_EXREG_CR3is in it by definition, but it is not added intoSVM_REGS_DIRTY_SETin the patch just for optimization to allow the compiler optimizes the line of code out.I think this is where we disagree. In my opinion it is enough to document that CR3 _can_ be out of date, but it doesn't have to be marked dirty because its dirty bit is effectively
KVM_REQ_LOAD_MMU_PGD.For VMX, it is important to clear
VCPU_EXREG_CR3because the combination "avail=0, dirty=1" is nonsensical:av d
0 0 in VMCS
0 1 *INVALID*
1 0 invcpu->arch
1 1 invcpu->arch, needs storeBut on SVM,
VCPU_EXREG_CR3is always available.Thinking more about it, it makes more sense for VMX to reset _all_ bits of dirty to 0, just like it was before your change, but doing so even earlier in
vmx_vcpu_run.
Quoted 5 lines
I appreciate that
VMX_REGS_LAZY_UPDATE_SETis useful for documentation, but it's also important that the values in avail/dirty make sense as a pair.So here is what I would do:
Quoted 125 lines
arch/x86/kvm/kvm_cache_regs.h +7
-0@@ -43,6 +43,13 @@ BUILD_KVM_GPR_ACCESSORS(r14, R14) BUILD_KVM_GPR_ACCESSORS(r15, R15) #endif +/* +* avail dirty +* 0 0 register in VMCS/VMCB +* 0 1 *INVALID* +* 1 0 register in vcpu->arch +* 1 1 register in vcpu->arch, needs to be stored back +*/ static inline bool kvm_register_is_available(struct kvm_vcpu *vcpu, enum kvm_reg reg) {arch/x86/kvm/vmx/vmx.c +2
-1@@ -6635,6 +6635,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) vmcs_writel(GUEST_RSP, vcpu->arch.regs[VCPU_REGS_RSP]); if (kvm_register_is_dirty(vcpu, VCPU_REGS_RIP)) vmcs_writel(GUEST_RIP, vcpu->arch.regs[VCPU_REGS_RIP]); + vcpu->arch.regs_dirty = 0; cr3 = __get_current_cr3_fast(); if (unlikely(cr3 != vmx->loaded_vmcs->host_state.cr3)) {@@ -6729,7 +6730,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) loadsegment(es, __USER_DS); #endif - vmx_register_cache_reset(vcpu); + vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET; pt_guest_exit(vmx);arch/x86/kvm/vmx/vmx.h +15
-13@@ -473,19 +473,21 @@ BUILD_CONTROLS_SHADOW(pin, PIN_BASED_VM_EXEC_CONTROL) BUILD_CONTROLS_SHADOW(exec, CPU_BASED_VM_EXEC_CONTROL) BUILD_CONTROLS_SHADOW(secondary_exec, SECONDARY_VM_EXEC_CONTROL) -static inline void vmx_register_cache_reset (struct kvm_vcpu *vcpu) -{ - vcpu->arch.regs_avail = ~((1 << VCPU_REGS_RIP) | (1 << VCPU_REGS_RSP) - | (1 << VCPU_EXREG_RFLAGS) - | (1 << VCPU_EXREG_PDPTR) - | (1 << VCPU_EXREG_SEGMENTS) - | (1 << VCPU_EXREG_CR0) - | (1 << VCPU_EXREG_CR3) - | (1 << VCPU_EXREG_CR4) - | (1 << VCPU_EXREG_EXIT_INFO_1) - | (1 << VCPU_EXREG_EXIT_INFO_2)); - vcpu->arch.regs_dirty = 0; -} +/* +* VMX_REGS_LAZY_LOAD_SET - The set of registers that will be updated in the +* cache on demand. Other registers not listed here are synced to +* the cache immediately after VM-Exit. + */ +#define VMX_REGS_LAZY_LOAD_SET ((1 << VCPU_REGS_RIP) | \ + (1 << VCPU_REGS_RSP) | \ + (1 << VCPU_EXREG_RFLAGS) | \ + (1 << VCPU_EXREG_PDPTR) | \ + (1 << VCPU_EXREG_SEGMENTS) | \ + (1 << VCPU_EXREG_CR0) | \ + (1 << VCPU_EXREG_CR3) | \ + (1 << VCPU_EXREG_CR4) | \ + (1 << VCPU_EXREG_EXIT_INFO_1) | \ + (1 << VCPU_EXREG_EXIT_INFO_2)) static inline struct kvm_vmx *to_kvm_vmx(struct kvm *kvm) {and likewise for SVM:
arch/x86/kvm/svm/svm.c +2
-1@@ -3944,6 +3944,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) vcpu->arch.regs[VCPU_REGS_RSP] = svm->vmcb->save.rsp; vcpu->arch.regs[VCPU_REGS_RIP] = svm->vmcb->save.rip; } + vcpu->arch.regs_dirty = 0; if (unlikely(svm->vmcb->control.exit_code == SVM_EXIT_NMI)) kvm_before_interrupt(vcpu);@@ -3978,7 +3978,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); - kvm_register_clear_available(vcpu, VCPU_EXREG_PDPTR); + vcpu->arch.regs_avail &= ~SVM_REGS_LAZY_LOAD_SET; /** We need to handle MC intercepts here before the vcpu has a chance to arch/x86/kvm/svm/svm.h +10
-0@@ -321,6 +321,16 @@ static inline bool vmcb_is_dirty(struct vmcb *vmcb, int bit) return !test_bit(bit, (unsigned long *)&vmcb->control.clean); } +/* +* Only the PDPTRs are loaded on demand into the shadow MMU. All other +* fields are synchronized in handle_exit, because accessing the VMCB is cheap. +* +* CR3 might be out of date in the VMCB but it is not marked dirty; instead, +* KVM_REQ_LOAD_MMU_PGD is always requested when the cached vcpu->arch.cr3 +* is changed. svm_load_mmu_pgd() then syncs the new CR3 value into the VMCB. +*/ +#define SVM_REGS_LAZY_LOAD_SET (1 << VCPU_EXREG_PDPTR) + static inline struct vcpu_svm *to_svm(struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_svm, vcpu);
PATCH 14/15 KVM: X86: Remove kvm_register_clear_available()
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
It has no user.
--- arch/x86/kvm/kvm_cache_regs.h | 7 ------- 1 file changed, 7 deletions(-)
arch/x86/kvm/kvm_cache_regs.h +0 -7
@@ -61,13 +61,6 @@ static inline void kvm_register_mark_available(struct kvm_vcpu *vcpu,__set_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); } -static inline void kvm_register_clear_available (struct kvm_vcpu *vcpu, -enum kvm_reg reg) -{ -__clear_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); -__clear_bit (reg, (unsigned long *)&vcpu->arch.regs_dirty); -} -static inline void kvm_register_mark_dirty (struct kvm_vcpu *vcpu,enum kvm_reg reg) {
Signature · 1 line
2.19.1.6.gb485710b
PATCH 15/15 KVM: nVMX: Always write vmcs.GUEST_CR3 during nested VM-Exit
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
For VM-Enter, vmcs.GUEST_CR3 and vcpu->arch.cr3 are synced and it is better to mark VCPU_EXREG_CR3 available rather than dirty to reduce a redundant vmwrite(GUEST_CR3) in vmx_load_mmu_pgd().
But nested_vmx_load_cr3() is also served for VM-Exit which doesn't set vmcs.GUEST_CR3.
This patch moves writing to vmcs.GUEST_CR3 into nested_vmx_load_cr3() for both nested VM-Eneter/Exit and use kvm_register_mark_available().
This patch doesn't cause any extra writing to vmcs.GUEST_CR3 and if userspace is modifying CR3 with KVM_SET_SREGS later, the dirty info for VCPU_EXREG_CR3 would be set for next writing to vmcs.GUEST_CR3 and no update will be lost.
--- arch/x86/kvm/vmx/nested.c | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx/nested.c +21 -11
@@ -1133,8 +1133,28 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,if (!nested_ept)kvm_mmu_new_pgd (vcpu, cr3); +/* +* Immediately write vmcs.GUEST_CR3 when changing vcpu->arch.cr3. +* +* VCPU_EXREG_CR3 is marked available rather than dirty because +* vcpu->arch.cr3 and vmcs.GUEST_CR3 are synced when enable_ept and +* vmcs.GUEST_CR3 is irrelevant to vcpu->arch.cr3 when !enable_ept. +* +* For VM-Enter case, it will be propagated to vmcs12 on nested +* VM-Exit, which can occur without actually running L2 and thus +* without hitting vmx_load_mmu_pgd(), e.g. if L1 is entering L2 with +* vmcs12.GUEST_ACTIVITYSTATE=HLT, in which case KVM will intercept +* the transition to HLT instead of running L2. +* +* For VM-Exit case, it is likely that vmcs.GUEST_CR3 == cr3 here, but +* L1 may set HOST_CR3 to a value other than its CR3 before VM-Entry, +* so we just update it unconditionally. +*/ +if (enable_ept) +vmcs_writel (GUEST_CR3 , cr3); + vcpu->arch.cr3 = cr3; -kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 );/* Re-initialize the MMU, e.g. to pick up CR4 MMU role changes. */ kvm_init_mmu (vcpu);
@@ -2600,16 +2620,6 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, from_vmentry, entry_failure_code))return -EINVAL ; -/* -* Immediately write vmcs02.GUEST_CR3. It will be propagated to vmcs12 -* on nested VM-Exit, which can occur without actually running L2 and -* thus without hitting vmx_load_mmu_pgd(), e.g. if L1 is entering L2 with -* vmcs12.GUEST_ACTIVITYSTATE=HLT, in which case KVM will intercept the -* transition to HLT instead of running L2. -*/ -if (enable_ept) -vmcs_writel (GUEST_CR3 , vmcs12->guest_cr3); -/* Late preparation of GUEST_PDPTRs now that EFER and CRs are set. */ if (load_guest_pdptrs_vmcs12 &&nested_cpu_has_ept (vmcs12) &&is_pae_paging (vcpu)) {
Signature · 1 line
2.19.1.6.gb485710b
Paolo Bonzini
maintainerRe: [PATCH 15/15] KVM: nVMX: Always write vmcs.GUEST_CR3 during nested VM-Exit
Quoted 64 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
For VM-Enter, vmcs.GUEST_CR3 and
vcpu->arch.cr3are synced and it is better to markVCPU_EXREG_CR3available rather than dirty to reduce a redundantvmwrite(GUEST_CR3)invmx_load_mmu_pgd().But
nested_vmx_load_cr3()is also served for VM-Exit which doesn't set vmcs.GUEST_CR3.This patch moves writing to vmcs.GUEST_CR3 into
nested_vmx_load_cr3()for both nested VM-Eneter/Exit and usekvm_register_mark_available().This patch doesn't cause any extra writing to vmcs.GUEST_CR3 and if userspace is modifying CR3 with
KVM_SET_SREGSlater, the dirty info forVCPU_EXREG_CR3would be set for next writing to vmcs.GUEST_CR3 and no update will be lost.Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/vmx/nested.c | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-)arch/x86/kvm/vmx/nested.c +21
-4@@ -1133,8 +1133,28 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,if (!nested_ept)kvm_mmu_new_pgd (vcpu, cr3); +/* +* Immediately write vmcs.GUEST_CR3 when changing vcpu->arch.cr3. +* +* VCPU_EXREG_CR3 is marked available rather than dirty because +* vcpu->arch.cr3 and vmcs.GUEST_CR3 are synced when enable_ept and +* vmcs.GUEST_CR3 is irrelevant to vcpu->arch.cr3 when !enable_ept. +* +* For VM-Enter case, it will be propagated to vmcs12 on nested +* VM-Exit, which can occur without actually running L2 and thus +* without hitting vmx_load_mmu_pgd(), e.g. if L1 is entering L2 with +* vmcs12.GUEST_ACTIVITYSTATE=HLT, in which case KVM will intercept +* the transition to HLT instead of running L2. +* +* For VM-Exit case, it is likely that vmcs.GUEST_CR3 == cr3 here, but +* L1 may set HOST_CR3 to a value other than its CR3 before VM-Entry, +* so we just update it unconditionally. +*/ +if (enable_ept) +vmcs_writel (GUEST_CR3 , cr3); + vcpu->arch.cr3 = cr3; -kvm_register_mark_dirty (vcpu,VCPU_EXREG_CR3 ); +kvm_register_mark_available (vcpu,VCPU_EXREG_CR3 );/* Re-initialize the MMU, e.g. to pick up CR4 MMU role changes. */ kvm_init_mmu (vcpu);@@ -2600,16 +2620,6 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, from_vmentry, entry_failure_code))return -EINVAL ; -/* -* Immediately write vmcs02.GUEST_CR3. It will be propagated to vmcs12 -* on nested VM-Exit, which can occur without actually running L2 and
arch/x86/kvm/vmx/nested.c static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, · line 2625
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +0
-7-* thus without hitting vmx_load_mmu_pgd(), e.g. if L1 is entering L2 with -* vmcs12.GUEST_ACTIVITYSTATE=HLT, in which case KVM will intercept the -* transition to HLT instead of running L2. -*/ -if (enable_ept) -vmcs_writel (GUEST_CR3 , vmcs12->guest_cr3); -/* Late preparation of GUEST_PDPTRs now that EFER and CRs are set. */ if (load_guest_pdptrs_vmcs12 &&nested_cpu_has_ept (vmcs12) &&is_pae_paging (vcpu)) {
I have to think more about this one and patch 17. I queued the rest.
Paolo
[PATCH 16/15] KVM: X86: Update mmu->pdptrs only when it is changed
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
It is unchanged in most cases.
--- arch/x86/kvm/x86.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +7 -4
@@ -828,10 +828,13 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) } } -memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs)); -kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +kvm_register_mark_available (vcpu,VCPU_EXREG_PDPTR ); +if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) { +memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs)); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); +/* Ensure the dirty PDPTEs to be loaded. */ +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); + } vcpu->arch.pdptrs_from_userspace = false;return 1 ;
Signature · 1 line
2.19.1.6.gb485710b
Sean Christopherson
maintainerRe: [PATCH 16/15] KVM: X86: Update mmu->pdptrs only when it is changed
Quoted 18 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
It is unchanged in most cases.
Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/x86.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)arch/x86/kvm/x86.c +0
-1@@ -828,10 +828,13 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) } } -memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 837
Quoted 10 lines
arch/x86/kvm/x86.c +7
-3-kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +kvm_register_mark_available (vcpu,VCPU_EXREG_PDPTR ); +if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) { +memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs)); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); +/* Ensure the dirty PDPTEs to be loaded. */ +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); + }
Can this be unqueued until there's sufficient justification that (a) this is correct and (b) actually provides a meaningful performance optimization? There have been far too many PDPTR caching bugs to make this change without an analysis of why it's safe, e.g. what guarantees the that PDPTRs in the VMCS are sync'd with mmu->pdptrs? I'm not saying they aren't, I just want the changelog to prove that they are.
The next patch does add a fairly heavy unload of the current root for !TDP, but that's a bug fix and should be ordered before any optimizations anyways.
Quoted 6 lines
arch/x86/kvm/x86.c +0
-0vcpu->arch.pdptrs_from_userspace = false;return 1 ;Signature · 1 line
2.19.1.6.gb485710b
Re: [PATCH 16/15] KVM: X86: Update mmu->pdptrs only when it is changed
Quoted 28 lines · Sean Christopherson
Quoted 27 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
It is unchanged in most cases.
Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/x86.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)arch/x86/kvm/x86.c +6
-4@@ -828,10 +828,13 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) } } -memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs)); -kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +kvm_register_mark_available (vcpu,VCPU_EXREG_PDPTR ); +if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) { +memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs)); +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); +/* Ensure the dirty PDPTEs to be loaded. */ +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu);
Quoted 8 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 837
Quoted 1 line
arch/x86/kvm/x86.c +1
-0+ }Can this be unqueued until there's sufficient justification that (a) this is correct and (b) actually provides a meaningful performance optimization? There have been far too many PDPTR caching bugs to make this change without an analysis of why it's safe, e.g. what guarantees the that PDPTRs in the VMCS are sync'd with
mmu->pdptrs? I'm not saying they aren't, I just want the changelog to prove that they are.
I have zero intent to improve performance for 32bit guests.
For correctness, the patch needs to be revaluated, I agree it to be unqueued.
Quoted 3 lines
The next patch does add a fairly heavy unload of the current root for !TDP, but that's a bug fix and should be ordered before any optimizations anyways.
I don't have strong option on the patch ordering and I have a preference which is already applied to other patchset.
This patch is a preparation patch for next patch. It has very limited or even negative value without next patch and it was not in the original 15-patch patchset until I found the defect fixed by the next patch that I appended both as "16/15" and "17/15".
But kvm has many small defects for so many years, I has fixed several in this circle and there are some pending.
And the kvm works for so many years even with these small defects and they only hit when the guest deliberately do something, and even the guest does these things, it only reveals that the kvm doesn't fully conform the SDM, it can't hurt host any degree.
For important fix or for bug that can hurt host, I would definitely make the bug fix first and other patches are ordered later.
For defect like this, I prefer "clean" patches policy: several cleanup or preparation patches first to give the code a better basic and then fix the defects.
It is OK for me if fixes like this (normal guest doesn't hit it and it can't hurt host) go into or doesn't go into the stable tree.
For example, I used my patch ordering policy in "permission_fault() for SMAP" patchset where the fix went in patch3 which fixes implicit supervisor access when the CPL < 3. For next spin, I would be a new preparation patch first to add "implicit access" in pfec. The fix itself can't go as the first patch.
This is a reply not specified to this patch. And for this patch and next patch I will take your suggestion or another possible approach.
Thanks
Lai
Paolo Bonzini
maintainerRe: [PATCH 16/15] KVM: X86: Update mmu->pdptrs only when it is changed
In reply to Sean ChristophersonQuoted 3 lines · Sean Christopherson
what guarantees the that PDPTRs in the VMCS are sync'd with
mmu->pdptrs? I'm not saying they aren't, I just want the changelog to prove that they are.
If they aren't synced you should *already* have dirty VCPU_EXREG_PDPTR and pending KVM_REQ_LOAD_MMU_PGD, shouldn't you? As long as the caching invariants are respected, this patch is fairly safe, and if they aren't there are plenty of preexisting bugs anyway.
Paolo
Quoted 3 lines
The next patch does add a fairly heavy unload of the current root for !TDP, but that's a bug fix and should be ordered before any optimizations anyways.
Re: [PATCH 16/15] KVM: X86: Update mmu->pdptrs only when it is changed
Quoted 7 lines · Paolo Bonzini
Quoted 3 lines · Sean Christopherson
what guarantees the that PDPTRs in the VMCS are sync'd with mmu->pdptrs? I'm not saying they aren't, I just want the changelog to prove that they are.
If they aren't synced you should *already* have dirty
VCPU_EXREG_PDPTRand pendingKVM_REQ_LOAD_MMU_PGD, shouldn't you? As long as the caching invariants are respected, this patch is fairly safe, and if they aren't there are plenty of preexisting bugs anyway.
They can be not synced in other side: not available.
If (!kvm_register_is_available(vcpu, VCPU_EXREG_PDPTR)) it will make no sense to compare mmu->pdptrs when EPT is enabled.
Because vmcs might have different copy, it is better to just mark it dirty in load_pdptrs().
(SVM is OK even with NPT enabled, since vmcb doesn't have a copy)
I haven't investigated enough then and today. It is quit complicated.
Thanks
Lai
Quoted 4 lines
Quoted 4 lines
The next patch does add a fairly heavy unload of the current root for !TDP, but that's a bug fix and should be ordered before any optimizations anyways.
[PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
In reply to Lai JiangshanFrom: Lai Jiangshan <[email protected]>
For shadow paging, the pae_root needs to be reconstructed before the coming VMENTER if the guest PDPTEs is changed.
But not all paths that call load_pdptrs() will cause the pae_root to be reconstructed. Normally, kvm_mmu_reset_context() and kvm_mmu_free_roots() are used to launch later reconstruction.
The commit d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed") skips kvm_mmu_reset_context() after load_pdptrs() when changing CR0.CD and CR0.NW.
The commit 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush") skips kvm_mmu_free_roots() after load_pdptrs() when rewriting the CR3 with the same value.
The commit a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE") skips kvm_mmu_reset_context() after load_pdptrs() when changing CR4.PGE.
Normally, the guest doesn't change the PDPTEs before doing only the above operation without touching other bits that can force pae_root to be reconstructed. Guests like linux would keep the PDPTEs unchaged for every instance of pagetable.
--- arch/x86/kvm/x86.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +8 -2
@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT +* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT ); } vcpu->arch.pdptrs_from_userspace = false;
Signature · 1 line
2.19.1.6.gb485710b
Re: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Hello, Paolo
any thought/concern about this one?
Thanks
Lai
Quoted 56 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
For shadow paging, the
pae_rootneeds to be reconstructed before the coming VMENTER if the guest PDPTEs is changed.But not all paths that call
load_pdptrs()will cause thepae_rootto be reconstructed. Normally,kvm_mmu_reset_context()andkvm_mmu_free_roots()are used to launch later reconstruction.The commit
d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed")skipskvm_mmu_reset_context()afterload_pdptrs()when changing CR0.CD and CR0.NW.The commit 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush") skips
kvm_mmu_free_roots()afterload_pdptrs()when rewriting the CR3 with the same value.The commit
a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE")skipskvm_mmu_reset_context()afterload_pdptrs()when changing CR4.PGE.Normally, the guest doesn't change the PDPTEs before doing only the above operation without touching other bits that can force
pae_rootto be reconstructed. Guests like linux would keep the PDPTEs unchaged for every instance of pagetable.Fixes d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed")Fixes 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush")Fixes a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE")Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/x86.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +8
-2@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT +* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT ); } vcpu->arch.pdptrs_from_userspace = false;
Sean Christopherson
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
In reply to Lai JiangshanQuoted 9 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
For shadow paging, the
pae_rootneeds to be reconstructed before the coming VMENTER if the guest PDPTEs is changed.But not all paths that call
load_pdptrs()will cause thepae_rootto be reconstructed. Normally,kvm_mmu_reset_context()andkvm_mmu_free_roots()are used to launch later reconstruction.
Quoted 7 lines
The commit
d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed")skipskvm_mmu_reset_context()afterload_pdptrs()when changing CR0.CD and CR0.NW.The commit 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush") skips
kvm_mmu_free_roots()afterload_pdptrs()when rewriting the CR3 with the same value.
This isn't accurate, prior to that commit KVM wasn't guaranteed to do kvm_mmu_free_roots() if it got a hit on the current CR3 or if a previous CR3 in the cache matched the new CR3 (the "cache" has done some odd things in the past).
So I think this particular flavor would be:
Quoted 25 lines
The commit
a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE")skipskvm_mmu_reset_context()afterload_pdptrs()when changing CR4.PGE.Normally, the guest doesn't change the PDPTEs before doing only the above operation without touching other bits that can force
pae_rootto be reconstructed. Guests like linux would keep the PDPTEs unchaged for every instance of pagetable.Fixes d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed")Fixes 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush")Fixes a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE")Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/x86.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +0
-0@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR );
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 10 lines
arch/x86/kvm/x86.c +8
-2-/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT +* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );
Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of vcpu->arch.mmuvcpu->arch.mmu.
To avoid a dependency on the previous patch, I think it makes sense to have this be:
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) kvm_mmu_free_roots(vcpu, mmu, KVM_MMU_ROOT_CURRENT);
before the memcpy().
Then we can decide independently if skipping the KVM_REQ_LOAD_MMU_PGD if the PDPTRs are unchanged with respect to the MMU is safe.
Re: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Quoted 5 lines · Sean Christopherson
Quoted 4 lines
The commit 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush") skips
kvm_mmu_free_roots()afterload_pdptrs()when rewriting the CR3 with the same value.
Quoted 7 lines
This isn't accurate, prior to that commit KVM wasn't guaranteed to do
kvm_mmu_free_roots()if it got a hit on the current CR3 or if a previous CR3 in the cache matched the new CR3 (the "cache" has done some odd things in the past).So I think this particular flavor would be:
Fixes 7c390d350f8b ("kvm: x86: Add fast CR3 switch code path")
If guest is 32bit, fast_cr3_switch() always return false, and kvm_mmu_free_roots() is always called, and no cr3 goes in prev_root.
And from 21823fbda552, fast_cr3_switch() and kvm_mmu_free_roots() are both skipped when cr3 is unchanged.
Quoted 31 lines
Quoted 31 lines
The commit
a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE")skipskvm_mmu_reset_context()afterload_pdptrs()when changing CR4.PGE.Normally, the guest doesn't change the PDPTEs before doing only the above operation without touching other bits that can force
pae_rootto be reconstructed. Guests like linux would keep the PDPTEs unchaged for every instance of pagetable.Fixes d81135a57aa6("KVM: x86: do not reset mmu if CR0.CD and CR0.NW are changed")Fixes 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush")Fixes a91a7c709600("KVM: X86: Don't reset mmu context when toggling X86_CR4_PGE")Signed-off-by Lai Jiangshan <[email protected]>--- arch/x86/kvm/x86.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)arch/x86/kvm/x86.c +3
-2@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT +* enabled or pae_root to be reconstructed for shadow paging.
Quoted 8 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 5 lines
arch/x86/kvm/x86.c +5
-0+*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of
vcpu->arch.mmuvcpu->arch.mmu.
@mmu is the "guest mmu" (vcpu->arch.walk_mmu), which is used to walk including loading pdptr.
vcpu->arch.mmu is for host constructing mmu for shadowed or tdp mmu which is used in host side management including kvm_mmu_free_roots().
Even they are the same pointer now for !tdp, the meaning is different. I prefer to distinguish them even before kvm_mmu is split different for guest mmu (vcpu->arch.walk_mmu) and host constructing mmu (vcpu->arch.mmu).
(I once searched all the usage of undistinguished usage of kvm_mmu *mmu, and found a bug, see "Use vcpu->arch.walk_mmu for kvm_mmu_invlpg()")
I think Paolo is doing the splitting, unless I would take the job because I have some patches pending depended them.
Quoted 6 lines
To avoid a dependency on the previous patch, I think it makes sense to have this be:
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) kvm_mmu_free_roots(vcpu, mmu, KVM_MMU_ROOT_CURRENT);
Yes, it is a good idea to add this first.
Thanks for review and suggestion.
Lai
Sean Christopherson
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Quoted 12 lines · Lai Jiangshan
Quoted 9 lines · Sean Christopherson
Quoted 4 lines
The commit 21823fbda552("KVM: x86: Invalidate all PGDs for the current PCID on MOV CR3 w/ flush") skips
kvm_mmu_free_roots()afterload_pdptrs()when rewriting the CR3 with the same value.This isn't accurate, prior to that commit KVM wasn't guaranteed to do
kvm_mmu_free_roots()if it got a hit on the current CR3 or if a previous CR3 in the cache matched the new CR3 (the "cache" has done some odd things in the past).
Quoted 6 lines
Quoted 3 lines
So I think this particular flavor would be:
Fixes 7c390d350f8b ("kvm: x86: Add fast CR3 switch code path")If guest is 32bit,
fast_cr3_switch()always return false, andkvm_mmu_free_roots()is always called, and no cr3 goes inprev_root.
Oh, duh, PAE paging.
Quoted 15 lines
Quoted 11 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 8 lines
arch/x86/kvm/x86.c +8
-0+/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT +* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of
vcpu->arch.mmuvcpu->arch.mmu.@mmu is the "guest mmu" (
vcpu->arch.walk_mmu), which is used to walk including loading pdptr.
Quoted 6 lines
vcpu->arch.mmuis for host constructing mmu for shadowed or tdp mmu which is used in host side management includingkvm_mmu_free_roots().Even they are the same pointer now for !tdp, the meaning is different. I prefer to distinguish them even before
kvm_mmuis split different for guest mmu (vcpu->arch.walk_mmu) and host constructing mmu (vcpu->arch.mmu).
I see where you're coming from. I was viewing it from the perspective of, "they're the same thing for shadow paging, why reread mmu?".
I'm ok with explicitly referencing arch.mmu, but maybe add a comment?
Paolo Bonzini
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
In reply to Sean ChristophersonQuoted 8 lines · Sean Christopherson
Quoted 8 lines
arch/x86/kvm/x86.c +2
-2@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT
Quoted 8 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of
vcpu->arch.mmuvcpu->arch.mmu.
In kvm/next actually there's no mmu parameter to load_pdptrs, so it's okay to keep vcpu->arch.mmu.
Quoted 9 lines
To avoid a dependency on the previous patch, I think it makes sense to have this be:
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) kvm_mmu_free_roots(vcpu, mmu, KVM_MMU_ROOT_CURRENT);before the
memcpy().Then we can decide independently if skipping the
KVM_REQ_LOAD_MMU_PGDif the PDPTRs are unchanged with respect to the MMU is safe.
Do you disagree that there's already an invariant that the PDPTRs can only be dirty if KVM_REQ_LOAD_MMU_PGD---and therefore a previous change to the PDPTRs would have triggered KVM_REQ_LOAD_MMU_PGD? This is opposed to the guest TLB flush due to MOV CR3; that one has to be done unconditionally for PAE paging, and it is handled separately within kvm_set_cr3.
Paolo
Sean Christopherson
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Quoted 26 lines · Paolo Bonzini
Quoted 8 lines · Sean Christopherson
Quoted 8 lines
arch/x86/kvm/x86.c +2
-2@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT Quoted 8 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of
vcpu->arch.mmuvcpu->arch.mmu.In kvm/next actually there's no mmu parameter to
load_pdptrs, so it's okay to keepvcpu->arch.mmu.Quoted 5 lines
To avoid a dependency on the previous patch, I think it makes sense to have this be:
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) kvm_mmu_free_roots(vcpu, mmu, KVM_MMU_ROOT_CURRENT);
Quoted 8 lines
Quoted 4 lines
before the
memcpy().Then we can decide independently if skipping the
KVM_REQ_LOAD_MMU_PGDif the PDPTRs are unchanged with respect to the MMU is safe.Do you disagree that there's already an invariant that the PDPTRs can only be dirty if KVM_REQ_LOAD_MMU_PGD---and therefore a previous change to the PDPTRs would have triggered
KVM_REQ_LOAD_MMU_PGD?
What I think is moot, because commit 24cd19a28cb7 ("KVM: X86: Update mmu->pdptrs only when it is changed") breaks nested VMs with EPT in L0 and PAE shadow paging in L2. Reproducing is trivial, just disable EPT in L1 and run a VM. I haven't investigating how it breaks things, because why it's broken is secondary for me.
My primary concern is that we would even consider optimizing the PDPTR logic without a mountain of evidence that any patch is correct for all scenarios. We had to add an entire ioctl() just to get PDPTRs functional. This apparently wasn't validated against a simple use case, let alone against things like migration with nested VMs, multliple L2s, etc...
Sean Christopherson
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Quoted 32 lines · Sean Christopherson
Quoted 31 lines · Paolo Bonzini
Quoted 8 lines · Sean Christopherson
Quoted 8 lines
arch/x86/kvm/x86.c +2
-2@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT Quoted 8 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of
vcpu->arch.mmuvcpu->arch.mmu.In kvm/next actually there's no mmu parameter to
load_pdptrs, so it's okay to keepvcpu->arch.mmu.Quoted 9 lines
To avoid a dependency on the previous patch, I think it makes sense to have this be:
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) kvm_mmu_free_roots(vcpu, mmu, KVM_MMU_ROOT_CURRENT);before the
memcpy().Then we can decide independently if skipping the
KVM_REQ_LOAD_MMU_PGDif the PDPTRs are unchanged with respect to the MMU is safe.
Quoted 7 lines
Quoted 3 lines
Do you disagree that there's already an invariant that the PDPTRs can only be dirty if KVM_REQ_LOAD_MMU_PGD---and therefore a previous change to the PDPTRs would have triggered
KVM_REQ_LOAD_MMU_PGD?What I think is moot, because commit
24cd19a28cb7("KVM: X86: Updatemmu->pdptrsonly when it is changed") breaks nested VMs with EPT in L0 and PAE shadow paging in L2. Reproducing is trivial, just disable EPT in L1 and run a VM. I haven't
Doh, s/L2/L1
Quoted 7 lines
investigating how it breaks things, because why it's broken is secondary for me.
My primary concern is that we would even consider optimizing the PDPTR logic without a mountain of evidence that any patch is correct for all scenarios. We had to add an entire
ioctl()just to get PDPTRs functional. This apparently wasn't validated against a simple use case, let alone against things like migration with nested VMs, multliple L2s, etc...
Re: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
In reply to Sean ChristophersonQuoted 41 lines · Sean Christopherson
Quoted 26 lines · Paolo Bonzini
Quoted 8 lines · Sean Christopherson
Quoted 8 lines
arch/x86/kvm/x86.c +2
-2@@ -832,8 +832,14 @@ int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3)if (memcmp (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs))) {memcpy (mmu->pdptrs, pdpte,sizeof (mmu->pdptrs));kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); -/* Ensure the dirty PDPTEs to be loaded. */ -kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +/* +* Ensure the dirty PDPTEs to be loaded for VMX with EPT Quoted 8 lines
arch/x86/kvm/x86.c int load_pdptrs(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, unsigned long cr3) · line 842
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+* enabled or pae_root to be reconstructed for shadow paging. +*/ +if (tdp_enabled) +kvm_make_request (KVM_REQ_LOAD_MMU_PGD , vcpu); +else +kvm_mmu_free_roots (vcpu, vcpu->arch.mmu,KVM_MMU_ROOT_CURRENT );Shouldn't matter since it's legacy shadow paging, but @mmu should be used instead of
vcpu->arch.mmuvcpu->arch.mmu.In kvm/next actually there's no mmu parameter to
load_pdptrs, so it's okay to keepvcpu->arch.mmu.Quoted 5 lines
To avoid a dependency on the previous patch, I think it makes sense to have this be:
if (!tdp_enabled && memcmp(mmu->pdptrs, pdpte, sizeof(mmu->pdptrs))) kvm_mmu_free_roots(vcpu, mmu, KVM_MMU_ROOT_CURRENT);Quoted 8 lines
Quoted 4 lines
before the
memcpy().Then we can decide independently if skipping the
KVM_REQ_LOAD_MMU_PGDif the PDPTRs are unchanged with respect to the MMU is safe.Do you disagree that there's already an invariant that the PDPTRs can only be dirty if KVM_REQ_LOAD_MMU_PGD---and therefore a previous change to the PDPTRs would have triggered
KVM_REQ_LOAD_MMU_PGD?What I think is moot, because commit
24cd19a28cb7("KVM: X86: Updatemmu->pdptrsonly when it is changed") breaks nested VMs with EPT in L0 and PAE shadow paging in L2. Reproducing is trivial, just disable EPT in L1 and run a VM. I haven't investigating how it breaks things, because why it's broken is secondary for me.
Quoted 5 lines
My primary concern is that we would even consider optimizing the PDPTR logic without a mountain of evidence that any patch is correct for all scenarios. We had to add an entire
ioctl()just to get PDPTRs functional. This apparently wasn't validated against a simple use case, let alone against things like migration with nested VMs, multliple L2s, etc...
I did validate the *SREGS2* against all the cases I could (like migration, EPT/NPT disabled/etc. I even started testing SMM to see how it affects PDPTRs, and patched seabios to use PAE paging. I still could have missed something.
But note that qemu still doesn't use that ioctl (patch stuck in review).
Best regards,
Maxim Levitsky
Quoted 1 line
Paolo Bonzini
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Quoted 8 lines · Maxim Levitsky
Quoted 3 lines
This apparently wasn't validated against a simple use case, let alone against things like migration with nested VMs, multliple L2s, etc...
I did validate the *SREGS2* against all the cases I could (like migration, EPT/NPT disabled/etc. I even started testing SMM to see how it affects PDPTRs, and patched seabios to use PAE paging. I still could have missed something.
Don't worry, I think Sean was talking about patch 16 and specifically digging at me (who deserved it completely).
Paolo
Sean Christopherson
maintainerRe: [PATCH 17/15] KVM: X86: Ensure pae_root to be reconstructed for shadow paging if the guest PDPTEs is changed
Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Maxim Levitsky
Quoted 3 lines
This apparently wasn't validated against a simple use case, let alone against things like migration with nested VMs, multliple L2s, etc...
Quoted 7 lines
Quoted 4 lines
I did validate the *SREGS2* against all the cases I could (like migration, EPT/NPT disabled/etc. I even started testing SMM to see how it affects PDPTRs, and patched seabios to use PAE paging. I still could have missed something.
Don't worry, I think Sean was talking about patch 16 and specifically digging at me (who deserved it completely).
Yes, patch 16. My goal wasn't to dig at anyone, I just wanted to dramatically emphasize how ridiculousy fragile and complex the PDPTR crud is due to the number of edge cases.
Hello,all
Ping
Thanks
Lai
Quoted 54 lines · Lai Jiangshan
From: Lai Jiangshan <[email protected]>
The patchset was started when I read the code of
nested_svm_load_cr3()and found that it marks CR3 available other than dirty when changingvcpu->arch.cr3. I thought its caller has ensured that vmcs.GUEST_CR3 will be or already be set to @cr3 so that it doesn't need to be marked dirty. And later I found that it is not true and it must be a bug in a rare case before I realized that all the code just (ab)usesvcpu->arch.regs_availforVCPU_EXREG_CR3and there is not such bug of usingregs_availhere.
(The above finding becomes a low meaningpatch_15rather than a fix)The unhappyness of the reading code made me do some cleanup for
regs_availandregs_dirtyandkvm_register_xxx()functions in the hope that the code become clearer with less misunderstanding.Major focus was on
VCPU_EXREG_CR3andVCPU_EXREG_PDPTR. They are ensured to be marked the correct tags (available or dirty), and the value is ensured to be synced to architecture before run if it is marked dirty.When cleaning
VCPU_EXREG_PDPTR, I also checked if the corresponding cr0/cr4 pdptr bits are all intercepted when !tdp_enabled, and I think it is not clear enough, soX86_CR4_PDPTR_BITSis added as self-comments in the code.Lai Jiangshan (15):
KVM: X86: Ensure the dirty PDPTEs to be loaded
KVM: VMX: MarkVCPU_EXREG_PDPTRavailable inept_save_pdptrs()
KVM: SVM: Always clear available ofVCPU_EXREG_PDPTRinsvm_vcpu_run()
KVM: VMX: Add and useX86_CR4_TLB_BITSwhen !enable_ept
KVM: VMX: Add and useX86_CR4_PDPTR_BITSwhen !enable_ept
KVM: X86: Move CR0pdptr_bitsinto header file asX86_CR0_PDPTR_BITS
KVM: SVM: Remove outdate comment insvm_load_mmu_pgd()
KVM: SVM: Remove useless check insvm_load_mmu_pgd()
KVM: SVM: Remove the unneeded code to mark available for CR3
KVM: X86: Mark CR3 dirty whenvcpu->arch.cr3is changed
KVM: VMX: Update vmcs.GUEST_CR3 only when the guest CR3 is dirty
KVM: VMX: Reset the bits that are meaningful to be reset in
vmx_register_cache_reset()
KVM: SVM: Add and usesvm_register_cache_reset()
KVM: X86: Removekvm_register_clear_available()
KVM: nVMX: Always write vmcs.GUEST_CR3 during nested VM-Exitarch/x86/kvm/kvm_cache_regs.h | 13 ++++++------ arch/x86/kvm/svm/nested.c | 1 - arch/x86/kvm/svm/svm.c | 17 ++++++++-------- arch/x86/kvm/svm/svm.h | 26 ++++++++++++++++++++++++ arch/x86/kvm/vmx/nested.c | 30 ++++++++++++++++++---------- arch/x86/kvm/vmx/vmx.c | 12 +++++++----- arch/x86/kvm/vmx/vmx.h | 37 +++++++++++++++++++++++++---------- arch/x86/kvm/x86.c | 13 ++++++------ 8 files changed, 101 insertions(+), 48 deletions(-)