KVM: SVM: A fix and cleanups for VMCB intercepts
Sean Christopherson
maintainerv2 cover KVM: SVM: A fix and cleanups for VMCB intercepts
Fix a likely-benign bug where KVM fails to mark vmcb01 intercepts as dirty after recalculating intercepts while L2 is active, then do a bunch of related cleanup, e.g. to split recalc_intercepts() into nested vs. non-nested functionality.
v2:
- Fix the aforementioned bug.
- Split recalc_intercepts() instead of simply renaming it.
- Move the new WARN in nested_vmcb02_recalc_intercepts() to its own patch.
- Use less weird local variables even if they aren't consistent with the existing code...
- ... and then change some names in the existing code to provide consistency.
v1: https://lkml.kernel.org/r/20260112182022.771276-1-yosry.ahmed%40linux.dev
Sean Christopherson (6):
KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts
KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested
parts
KVM: nSVM: Directly (re)calc vmcb02 intercepts from
nested_vmcb02_prepare_control()
KVM: nSVM: Use intuitive local variables in
nested_vmcb02_recalc_intercepts()
KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state
KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing
vmcb02
Yosry Ahmed (2):
KVM: nSVM: WARN and abort vmcb02 intercepts recalc if vmcb02 isn't
active
KVM: nSVM: Use vmcb12_is_intercept() in
nested_sync_control_from_vmcb02()
arch/x86/kvm/svm/nested.c | 88 +++++++++++++++++++-------------------- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 6 +-- arch/x86/kvm/svm/svm.h | 28 +++++++++---- 4 files changed, 67 insertions(+), 57 deletions(-)
base-commit: 183bb0ce8c77b0fd1fb25874112bc8751a461e49
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Sean Christopherson
maintainerPATCH v2 1/8 KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts
When reacting to an intercept update, explicitly mark vmcb01's intercepts dirty, as KVM always initially operates on vmcb01, and nested_svm_vmexit() isn't guaranteed to mark VMCB_INTERCEPTS as dirty. I.e. if L2 is active, KVM will modify the intercepts for L1, but might not mark them as dirty before the next VMRUN of L1.
--- arch/x86/kvm/svm/nested.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +3 -1
@@ -128,11 +128,13 @@ void recalc_intercepts(struct vcpu_svm *svm)struct vmcb_ctrl_area_cached *g;unsigned int i; -vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_INTERCEPTS );if (!is_guest_mode (&svm->vcpu))return ; +vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); + c = &svm->vmcb->control; h = &svm->vmcb01.ptr->control; g = &svm->nested.ctl;
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Re: [PATCH v2 1/8] KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts
Quoted 9 lines · Sean Christopherson
When reacting to an intercept update, explicitly mark vmcb01's intercepts dirty, as KVM always initially operates on vmcb01, and
nested_svm_vmexit()isn't guaranteed to markVMCB_INTERCEPTSas dirty. I.e. if L2 is active, KVM will modify the intercepts for L1, but might not mark them as dirty before the next VMRUN of L1.Fixes 116a0a23676e ("KVM: SVM: Add clean-bit for intercetps, tsc-offset and pause filter count")Signed-off-by Sean Christopherson <[email protected]>
Quoted 26 lines
--- arch/x86/kvm/svm/nested.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)arch/x86/kvm/svm/nested.c +3
-1@@ -128,11 +128,13 @@ void recalc_intercepts(struct vcpu_svm *svm)struct vmcb_ctrl_area_cached *g;unsigned int i; -vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_INTERCEPTS );if (!is_guest_mode (&svm->vcpu))return ; +vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); + c = &svm->vmcb->control; h = &svm->vmcb01.ptr->control; g = &svm->nested.ctl;Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Sean Christopherson
maintainerPATCH v2 2/8 KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested parts
In reply to Sean ChristophersonExtract the non-nested aspects of recalc_intercepts() into a separate helper, svm_mark_intercepts_dirty(), to make it clear that the call isn't *just* recalculating (vmcb02's) intercepts, and to not bury non-nested code in nested.c.
As suggested by Yosry, opportunistically prepend "nested_vmbc02_" to recalc_intercepts() so that it's obvious the function specifically deals with recomputing intercepts for L2.
No functional change intended.
--- arch/x86/kvm/svm/nested.c | 9 ++------- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 4 ++-- arch/x86/kvm/svm/svm.h | 26 ++++++++++++++++++++------ 4 files changed, 25 insertions(+), 16 deletions(-)
arch/x86/kvm/svm/nested.c +2 -7
@@ -122,17 +122,12 @@ static bool nested_vmcb_needs_vls_intercept(struct vcpu_svm *svm)return false; } -void recalc_intercepts (struct vcpu_svm *svm) +void nested_vmcb02_recalc_intercepts (struct vcpu_svm *svm) {struct vmcb_control_area *c, *h;struct vmcb_ctrl_area_cached *g;unsigned int i; -vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_INTERCEPTS ); - -if (!is_guest_mode (&svm->vcpu)) -return ; -vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); c = &svm->vmcb->control;
@@ -918,7 +913,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,* Merge guest and host intercepts - must be called with vcpu in * guest-mode to take effect. */ -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static void nested_svm_copy_common_state (struct vmcb *from_vmcb,struct vmcb *to_vmcb)
arch/x86/kvm/svm/sev.c +1 -1
@@ -4639,7 +4639,7 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm, bool init_event)if (!sev_vcpu_has_debug_swap (svm)) {vmcb_set_intercept (&vmcb->control,INTERCEPT_DR7_READ );vmcb_set_intercept (&vmcb->control,INTERCEPT_DR7_WRITE ); -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }else {/* * Disable #DB intercept iff DebugSwap is enabled. KVM doesn't
arch/x86/kvm/svm/svm.c +2 -2
@@ -635,7 +635,7 @@ static void set_dr_intercepts(struct vcpu_svm *svm)vmcb_set_intercept (&vmcb->control,INTERCEPT_DR7_READ );vmcb_set_intercept (&vmcb->control,INTERCEPT_DR7_WRITE ); -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static void clr_dr_intercepts (struct vcpu_svm *svm)
@@ -644,7 +644,7 @@ static void clr_dr_intercepts(struct vcpu_svm *svm) vmcb->control.intercepts[INTERCEPT_DR ] =0 ; -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static bool msr_write_intercepted (struct kvm_vcpu *vcpu,u32 msr)
arch/x86/kvm/svm/svm.h +20 -6
@@ -357,8 +357,6 @@ struct svm_cpu_data { DECLARE_PER_CPU(struct svm_cpu_data, svm_data); -void recalc_intercepts (struct vcpu_svm *svm); - static __always_inline struct kvm_svm *to_kvm_svm(struct kvm *kvm) { return container_of(kvm, struct kvm_svm, kvm);
@@ -486,6 +484,22 @@ static inline bool vmcb12_is_intercept(struct vmcb_ctrl_area_cached *control, u3return __vmcb_is_intercept ((unsigned long *)&control->intercepts, bit); } +void nested_vmcb02_recalc_intercepts (struct vcpu_svm *svm); + +static inline void svm_mark_intercepts_dirty (struct vcpu_svm *svm) +{ +vmcb_mark_dirty (svm->vmcb01.ptr,VMCB_INTERCEPTS ); + +/* +* If L2 is active, recalculate the intercepts for vmcb02 to account +* for the changes made to vmcb01. All intercept configuration is done +* for vmcb01 and then propagated to vmcb02 to combine KVM's intercepts +* with L1's intercepts (from the vmcb12 snapshot). +*/ +if (is_guest_mode (&svm->vcpu)) +nested_vmcb02_recalc_intercepts (svm); +} +static inline void set_exception_intercept (struct vcpu_svm *svm,u32 bit) {struct vmcb *vmcb = svm->vmcb01.ptr;
@@ -493,7 +507,7 @@ static inline void set_exception_intercept(struct vcpu_svm *svm, u32 bit)WARN_ON_ONCE (bit >=32 );vmcb_set_intercept (&vmcb->control,INTERCEPT_EXCEPTION_OFFSET + bit); -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static inline void clr_exception_intercept (struct vcpu_svm *svm,u32 bit)
@@ -503,7 +517,7 @@ static inline void clr_exception_intercept(struct vcpu_svm *svm, u32 bit)WARN_ON_ONCE (bit >=32 );vmcb_clr_intercept (&vmcb->control,INTERCEPT_EXCEPTION_OFFSET + bit); -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static inline void svm_set_intercept (struct vcpu_svm *svm,int bit)
@@ -512,7 +526,7 @@ static inline void svm_set_intercept(struct vcpu_svm *svm, int bit)vmcb_set_intercept (&vmcb->control, bit); -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static inline void svm_clr_intercept (struct vcpu_svm *svm,int bit)
@@ -521,7 +535,7 @@ static inline void svm_clr_intercept(struct vcpu_svm *svm, int bit)vmcb_clr_intercept (&vmcb->control, bit); -recalc_intercepts (svm); +svm_mark_intercepts_dirty (svm); }static inline bool svm_is_intercept (struct vcpu_svm *svm,int bit)
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Re: [PATCH v2 2/8] KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested parts
Quoted 5 lines · Sean Christopherson
Extract the non-nested aspects of
recalc_intercepts()into a separate helper,svm_mark_intercepts_dirty(), to make it clear that the call isn't *just* recalculating (vmcb02's) intercepts, and to not bury non-nested code innested.c.
Quoted 8 lines
As suggested by Yosry, opportunistically prepend "
nested_vmbc02_" torecalc_intercepts()so that it's obvious the function specifically deals with recomputing intercepts for L2.No functional change intended.
Cc Yosry Ahmed <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v2 3/8 KVM: nSVM: WARN and abort vmcb02 intercepts recalc if vmcb02 isn't active
In reply to Sean ChristophersonFrom: Yosry Ahmed <[email protected]>
WARN and bail early from nested_vmcb02_recalc_intercepts() if vmcb02 isn't the active/current VMCB, as recalculating intercepts for vmcb01 using logic intended for merging vmcb12 and vmcb01 intercepts can yield unexpected and unwanted results.
In addition to hardening against general bugs, this will provide additional safeguards "if" nested_vmcb02_recalc_intercepts() is invoked directly from nested_vmcb02_prepare_control().
--- arch/x86/kvm/svm/nested.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/nested.c +3 -0
@@ -128,6 +128,9 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)struct vmcb_ctrl_area_cached *g;unsigned int i; +if (WARN_ON_ONCE (svm->vmcb != svm->nested.vmcb02.ptr)) +return ; +vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); c = &svm->vmcb->control;
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Sean Christopherson
maintainerPATCH v2 4/8 KVM: nSVM: Directly (re)calc vmcb02 intercepts from nested_vmcb02_prepare_control()
In reply to Sean ChristophersonNow that nested_vmcb02_recalc_intercepts() provides guardrails against it being incorrectly called without vmcb02 active, invoke it directly from nested_vmcb02_recalc_intercepts() instead of bouncing through svm_mark_intercepts_dirty(), which unnecessarily marks vmcb01 as dirty.
--- arch/x86/kvm/svm/nested.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
@@ -916,7 +916,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,* Merge guest and host intercepts - must be called with vcpu in * guest-mode to take effect. */ -svm_mark_intercepts_dirty (svm); +nested_vmcb02_recalc_intercepts (svm); }static void nested_svm_copy_common_state (struct vmcb *from_vmcb,struct vmcb *to_vmcb)
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Re: [PATCH v2 4/8] KVM: nSVM: Directly (re)calc vmcb02 intercepts from nested_vmcb02_prepare_control()
Quoted 6 lines · Sean Christopherson
Now that
nested_vmcb02_recalc_intercepts()provides guardrails against it being incorrectly called without vmcb02 active, invoke it directly fromnested_vmcb02_recalc_intercepts()instead of bouncing throughsvm_mark_intercepts_dirty(), which unnecessarily marks vmcb01 as dirty.Signed-off-by Sean Christopherson <[email protected]>
Quoted 20 lines
--- arch/x86/kvm/svm/nested.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/svm/nested.c +1
-1@@ -916,7 +916,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,* Merge guest and host intercepts - must be called with vcpu in * guest-mode to take effect. */ -svm_mark_intercepts_dirty (svm); +nested_vmcb02_recalc_intercepts (svm); }static void nested_svm_copy_common_state (struct vmcb *from_vmcb,struct vmcb *to_vmcb)Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Sean Christopherson
maintainerPATCH v2 5/8 KVM: nSVM: Use intuitive local variables in nested_vmcb02_recalc_intercepts()
In reply to Sean ChristophersonNow that nested_vmcb02_recalc_intercepts() is explicitly scoped to deal with *only* recalculating vmcb02 intercepts, rename its local variables to use more intuivite names. The current "c", "h", and "g" local variables, for the current VMCB, vmcb01, and (cached) vmcb12 respectively, are short and sweet, but don't do much to help unfamiliar readers understand what the code is doing.
Use vmcb12_ctrl/vmcb01/vmcb02/vmcb12_ctrl in lieu of c/h/g to make it clear the function is updating intercepts in vmcb02 based on the intercepts in vmcb01 and (cached) vmcb12.
Opportunistically change the existing WARN_ON to a WARN_ON_ONCE so that a KVM bug doesn't unintentionally DoS the host.
No functional change intended.
--- arch/x86/kvm/svm/nested.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-)
arch/x86/kvm/svm/nested.c +16 -19
@@ -124,23 +124,20 @@ static bool nested_vmcb_needs_vls_intercept(struct vcpu_svm *svm)void nested_vmcb02_recalc_intercepts (struct vcpu_svm *svm) { -struct vmcb_control_area *c, *h; -struct vmcb_ctrl_area_cached *g; +struct vmcb_ctrl_area_cached *vmcb12_ctrl = &svm->nested.ctl; +struct vmcb *vmcb02 = svm->nested.vmcb02.ptr; +struct vmcb *vmcb01 = svm->vmcb01.ptr;unsigned int i; -if (WARN_ON_ONCE (svm->vmcb != svm->nested.vmcb02.ptr)) +if (WARN_ON_ONCE (svm->vmcb != vmcb02))return ; -vmcb_mark_dirty (svm->vmcb,VMCB_INTERCEPTS ); - - c = &svm->vmcb->control; - h = &svm->vmcb01.ptr->control; - g = &svm->nested.ctl; +vmcb_mark_dirty (vmcb02,VMCB_INTERCEPTS );for (i =0 ; i <MAX_INTERCEPT ; i++) - c->intercepts[i] = h->intercepts[i]; + vmcb02->control.intercepts[i] = vmcb01->control.intercepts[i]; -if (g->int_ctl &V_INTR_MASKING_MASK ) { +if (vmcb12_ctrl->int_ctl &V_INTR_MASKING_MASK ) {/* * If L2 is active and V_INTR_MASKING is enabled in vmcb12, * disable intercept of CR8 writes as L2's CR8 does not affect
@@ -151,9 +148,9 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)* the effective RFLAGS.IF for L1 interrupts will never be set * while L2 is running (L2's RFLAGS.IF doesn't affect L1 IRQs). */ -vmcb_clr_intercept (c,INTERCEPT_CR8_WRITE ); -if (!(svm->vmcb01.ptr->save.rflags &X86_EFLAGS_IF )) -vmcb_clr_intercept (c,INTERCEPT_VINTR ); +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_CR8_WRITE ); +if (!(vmcb01->save.rflags &X86_EFLAGS_IF )) +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_VINTR ); }/*
@@ -161,14 +158,14 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)* flush feature is enabled. */ if (!nested_svm_l2_tlb_flush_enabled (&svm->vcpu)) -vmcb_clr_intercept (c,INTERCEPT_VMMCALL ); +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_VMMCALL );for (i =0 ; i <MAX_INTERCEPT ; i++) - c->intercepts[i] |= g->intercepts[i]; + vmcb02->control.intercepts[i] |= vmcb12_ctrl->intercepts[i];/* If SMI is not intercepted, ignore guest SMI intercept as well */ if (!intercept_smi) -vmcb_clr_intercept (c,INTERCEPT_SMI ); +vmcb_clr_intercept (&vmcb02->control,INTERCEPT_SMI );if (nested_vmcb_needs_vls_intercept (svm)) {/*
@@ -176,10 +173,10 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)* we must intercept these instructions to correctly * emulate them in case L1 doesn't intercept them. */ -vmcb_set_intercept (c,INTERCEPT_VMLOAD ); -vmcb_set_intercept (c,INTERCEPT_VMSAVE ); +vmcb_set_intercept (&vmcb02->control,INTERCEPT_VMLOAD ); +vmcb_set_intercept (&vmcb02->control,INTERCEPT_VMSAVE ); }else { -WARN_ON (!(c->virt_ext &VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK )); +WARN_ON_ONCE (!(vmcb02->control.virt_ext &VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK )); } }
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Sean Christopherson
maintainerPATCH v2 6/8 KVM: nSVM: Use vmcb12_is_intercept() in nested_sync_control_from_vmcb02()
In reply to Sean ChristophersonFrom: Yosry Ahmed <[email protected]>
Use vmcb12_is_intercept() instead of open-coding the intercept check.
No functional change intended.
--- arch/x86/kvm/svm/nested.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
@@ -530,7 +530,7 @@ void nested_sync_control_from_vmcb02(struct vcpu_svm *svm)* int_ctl (because it was never recognized while L2 was running). */ if (svm_is_intercept (svm,INTERCEPT_VINTR ) && - !test_bit (INTERCEPT_VINTR , (unsigned long *)svm->nested.ctl.intercepts)) + !vmcb12_is_intercept (&svm->nested.ctl,INTERCEPT_VINTR )) mask &= ~V_IRQ_MASK ;if (nested_vgif_enabled (svm))
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Sean Christopherson
maintainerPATCH v2 7/8 KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state
In reply to Sean ChristophersonMove "bus_lock_rip" from "vmcb_ctrl_area_cached" to "svm_nested_state" as "last_bus_lock_rip" to more accurately reflect what it tracks, and because it is NOT a cached vmcb12 control field. The misplaced field isn't all that apparent in the current code base, as KVM uses "svm->nested.ctl" broadly, but the bad placement becomes glaringly obvious if "svm->nested.ctl" is captured as a local "vmcb12_ctrl" variable.
No functional change intended.
--- arch/x86/kvm/svm/nested.c | 8 ++++---- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/nested.c +4 -4
@@ -806,7 +806,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,* L1 re-enters L2, the same instruction will trigger a VM-Exit and the * entire cycle start over. */ -if (vmcb02->save.rip && (svm->nested.ctl.bus_lock_rip == vmcb02->save.rip)) +if (vmcb02->save.rip && (svm->nested.last_bus_lock_rip == vmcb02->save.rip)) vmcb02->control.bus_lock_counter =1 ;else vmcb02->control.bus_lock_counter =0 ;
@@ -1191,11 +1191,11 @@ int nested_svm_vmexit(struct vcpu_svm *svm) }/* -* Invalidate bus_lock_rip unless KVM is still waiting for the guest -* to make forward progress before re-enabling bus lock detection. +* Invalidate last_bus_lock_rip unless KVM is still waiting for the +* guest to make forward progress before re-enabling bus lock detection. */ if (!vmcb02->control.bus_lock_counter) - svm->nested.ctl.bus_lock_rip =INVALID_GPA ; + svm->nested.last_bus_lock_rip =INVALID_GPA ;nested_svm_copy_common_state (svm->nested.vmcb02.ptr, svm->vmcb01.ptr);
arch/x86/kvm/svm/svm.c +1 -1
@@ -3223,7 +3223,7 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu) vcpu->arch.complete_userspace_io = complete_userspace_buslock;if (is_guest_mode (vcpu)) - svm->nested.ctl.bus_lock_rip = vcpu->arch.cui_linear_rip; + svm->nested.last_bus_lock_rip = vcpu->arch.cui_linear_rip;return 0 ; }
arch/x86/kvm/svm/svm.h +1 -1
@@ -173,7 +173,6 @@ struct vmcb_ctrl_area_cached {u64 nested_cr3;u64 virt_ext;u32 clean; -u64 bus_lock_rip;union {#if IS_ENABLED (CONFIG_HYPERV ) ||IS_ENABLED (CONFIG_KVM_HYPERV )struct hv_vmcb_enlightenments hv_enlightenments;
@@ -188,6 +187,7 @@ struct svm_nested_state {u64 vm_cr_msr;u64 vmcb12_gpa;u64 last_vmcb12_gpa; +u64 last_bus_lock_rip;/* * The MSR permissions map used for vmcb02, which is the merge result
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Re: [PATCH v2 7/8] KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state
Quoted 10 lines · Sean Christopherson
Move "
bus_lock_rip" from "vmcb_ctrl_area_cached" to "svm_nested_state" as "last_bus_lock_rip" to more accurately reflect what it tracks, and because it is NOT a cached vmcb12 control field. The misplaced field isn't all that apparent in the current code base, as KVM uses "svm->nested.ctl" broadly, but the bad placement becomes glaringly obvious if "svm->nested.ctl" is captured as a local "vmcb12_ctrl" variable.No functional change intended.
Signed-off-by Sean Christopherson <[email protected]>
Sean Christopherson
maintainerPATCH v2 8/8 KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing vmcb02
In reply to Sean ChristophersonGrab svm->nested.ctl as vmcb12_ctrl when preparing the vmcb02 controls to make it more obvious that much of the data is coming from vmcb12 (or rather, a snapshot of vmcb12 at the time of L1's VMRUN).
Opportunistically reorder the variable definitions to create a pretty reverse fir tree.
No functional change intended.
--- arch/x86/kvm/svm/nested.c | 39 +++++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 20 deletions(-)
arch/x86/kvm/svm/nested.c +19 -20
@@ -745,11 +745,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,u32 int_ctl_vmcb01_bits =V_INTR_MASKING_MASK ;u32 int_ctl_vmcb12_bits =V_TPR_MASK |V_IRQ_INJECTION_BITS_MASK ; -struct kvm_vcpu *vcpu = &svm->vcpu; -struct vmcb *vmcb01 = svm->vmcb01.ptr; +struct vmcb_ctrl_area_cached *vmcb12_ctrl = &svm->nested.ctl;struct vmcb *vmcb02 = svm->nested.vmcb02.ptr; -u32 pause_count12; -u32 pause_thresh12; +struct vmcb *vmcb01 = svm->vmcb01.ptr; +struct kvm_vcpu *vcpu = &svm->vcpu; +u32 pause_count12, pause_thresh12;nested_svm_transition_tlb_flush (vcpu);
@@ -762,7 +762,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,*/ if (guest_cpu_cap_has (vcpu,X86_FEATURE_VGIF ) && - (svm->nested.ctl.int_ctl &V_GIF_ENABLE_MASK )) + (vmcb12_ctrl->int_ctl &V_GIF_ENABLE_MASK )) int_ctl_vmcb12_bits |= (V_GIF_MASK |V_GIF_ENABLE_MASK );else int_ctl_vmcb01_bits |= (V_GIF_MASK |V_GIF_ENABLE_MASK );
@@ -820,10 +820,9 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,if (nested_npt_enabled (svm))nested_svm_init_mmu_context (vcpu); - vcpu->arch.tsc_offset =kvm_calc_nested_tsc_offset ( - vcpu->arch.l1_tsc_offset, - svm->nested.ctl.tsc_offset, - svm->tsc_ratio_msr); + vcpu->arch.tsc_offset =kvm_calc_nested_tsc_offset (vcpu->arch.l1_tsc_offset, + vmcb12_ctrl->tsc_offset, + svm->tsc_ratio_msr); vmcb02->control.tsc_offset = vcpu->arch.tsc_offset;
@@ -832,13 +831,13 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,nested_svm_update_tsc_ratio_msr (vcpu); vmcb02->control.int_ctl = - (svm->nested.ctl.int_ctl & int_ctl_vmcb12_bits) | + (vmcb12_ctrl->int_ctl & int_ctl_vmcb12_bits) | (vmcb01->control.int_ctl & int_ctl_vmcb01_bits); - vmcb02->control.int_vector = svm->nested.ctl.int_vector; - vmcb02->control.int_state = svm->nested.ctl.int_state; - vmcb02->control.event_inj = svm->nested.ctl.event_inj; - vmcb02->control.event_inj_err = svm->nested.ctl.event_inj_err; + vmcb02->control.int_vector = vmcb12_ctrl->int_vector; + vmcb02->control.int_state = vmcb12_ctrl->int_state; + vmcb02->control.event_inj = vmcb12_ctrl->event_inj; + vmcb02->control.event_inj_err = vmcb12_ctrl->event_inj_err;/* * next_rip is consumed on VMRUN as the return address pushed on the
@@ -849,7 +848,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,* prior to injecting the event). */ if (guest_cpu_cap_has (vcpu,X86_FEATURE_NRIPS )) - vmcb02->control.next_rip = svm->nested.ctl.next_rip; + vmcb02->control.next_rip = vmcb12_ctrl->next_rip;else if (boot_cpu_has (X86_FEATURE_NRIPS )) vmcb02->control.next_rip = vmcb12_rip;
@@ -859,7 +858,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, svm->soft_int_csbase = vmcb12_csbase; svm->soft_int_old_rip = vmcb12_rip;if (guest_cpu_cap_has (vcpu,X86_FEATURE_NRIPS )) - svm->soft_int_next_rip = svm->nested.ctl.next_rip; + svm->soft_int_next_rip = vmcb12_ctrl->next_rip;else svm->soft_int_next_rip = vmcb12_rip; }
@@ -870,11 +869,11 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, vmcb02->control.virt_ext |=VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK ;if (guest_cpu_cap_has (vcpu,X86_FEATURE_PAUSEFILTER )) - pause_count12 = svm->nested.ctl.pause_filter_count; + pause_count12 = vmcb12_ctrl->pause_filter_count;else pause_count12 =0 ;if (guest_cpu_cap_has (vcpu,X86_FEATURE_PFTHRESHOLD )) - pause_thresh12 = svm->nested.ctl.pause_filter_thresh; + pause_thresh12 = vmcb12_ctrl->pause_filter_thresh;else pause_thresh12 =0 ;if (kvm_pause_in_guest (svm->vcpu.kvm)) {
@@ -888,7 +887,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm, vmcb02->control.pause_filter_thresh = vmcb01->control.pause_filter_thresh;/* ... but ensure filtering is disabled if so requested. */ -if (vmcb12_is_intercept (&svm->nested.ctl,INTERCEPT_PAUSE )) { +if (vmcb12_is_intercept (vmcb12_ctrl,INTERCEPT_PAUSE )) {if (!pause_count12) vmcb02->control.pause_filter_count =0 ;if (!pause_thresh12)
@@ -905,7 +904,7 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm,* L2 is the "guest"). */ if (guest_cpu_cap_has (vcpu,X86_FEATURE_ERAPS )) - vmcb02->control.erap_ctl = (svm->nested.ctl.erap_ctl & + vmcb02->control.erap_ctl = (vmcb12_ctrl->erap_ctl &ERAP_CONTROL_ALLOW_LARGER_RAP ) |ERAP_CONTROL_CLEAR_RAP ;
Signature · 1 line
2.53.0.345.g96ddfc5eaa-goog
Re: [PATCH v2 8/8] KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing vmcb02
Quoted 11 lines · Sean Christopherson
Grab
svm->nested.ctlasvmcb12_ctrlwhen preparing the vmcb02 controls to make it more obvious that much of the data is coming from vmcb12 (or rather, a snapshot of vmcb12 at the time of L1's VMRUN).Opportunistically reorder the variable definitions to create a pretty reverse fir tree.
No functional change intended.
Cc Yosry Ahmed <[email protected]>Signed-off-by Sean Christopherson <[email protected]>
Quoted 13 lines · Sean Christopherson
Fix a likely-benign bug where KVM fails to mark vmcb01 intercepts as dirty after recalculating intercepts while L2 is active, then do a bunch of related cleanup, e.g. to split
recalc_intercepts()into nested vs. non-nested functionality.v2:
- Fix the aforementioned bug.
- Splitrecalc_intercepts()instead of simply renaming it.
- Move the new WARN innested_vmcb02_recalc_intercepts()to its own patch.
- Use less weird local variables even if they aren't consistent with the existing code...
- ... and then change some names in the existing code to provide consistency.
Quoted 1 line
[...]
Applied to kvm-x86 nested, thanks!
[1/8] KVM: SVM: Explicitly mark vmcb01 dirty after modifying VMCB intercepts
https://github.com/kvm-x86/linux/commit/d5bde6113aed [2/8] KVM: SVM: Separate recalc_intercepts() into nested vs. non-nested parts https://github.com/kvm-x86/linux/commit/0b97f929831a [3/8] KVM: nSVM: WARN and abort vmcb02 intercepts recalc if vmcb02 isn't active https://github.com/kvm-x86/linux/commit/a367b6e10372 [4/8] KVM: nSVM: Directly (re)calc vmcb02 intercepts from nested_vmcb02_prepare_control() https://github.com/kvm-x86/linux/commit/4a80c4bc1f10 [5/8] KVM: nSVM: Use intuitive local variables in nested_vmcb02_recalc_intercepts() https://github.com/kvm-x86/linux/commit/586160b75091 [6/8] KVM: nSVM: Use vmcb12_is_intercept() in nested_sync_control_from_vmcb02() https://github.com/kvm-x86/linux/commit/ef09eebc5736 [7/8] KVM: nSVM: Move vmcb_ctrl_area_cached.bus_lock_rip to svm_nested_state https://github.com/kvm-x86/linux/commit/af75470944f4 [8/8] KVM: nSVM: Capture svm->nested.ctl as vmcb12_ctrl when preparing vmcb02 https://github.com/kvm-x86/linux/commit/56bfbe68f78e