KVM: SVM: Stop toggling virtual VMSAVE/VMLOAD on intercept recalc
Subsystems (3)
PATCH 1/1 KVM: SVM: Stop toggling virtual VMSAVE/VMLOAD on intercept recalc
From: Yosry Ahmed <[email protected]>
[ Upstream commit 55780d8a1dcc93d2c4b33c565ada88df12c9f206 ]
Virtual VMSAVE/VMLOAD enablement (i.e. VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK) is set/cleared by svm_recalc_instruction_intercepts() when the intercepts are cleared/set. This is unnecessary because the bit is meaningless when intercepts are set and KVM emulates the instructions. Initialize the bit in vmcb01 base on vls, and keep it unchanged.
This is similar-ish to how vGIF is handled. It is enabled in init_vmcb() if vgif=1 and remains unchanged when the STGI intercept is enabled (e.g. for NMI windows).
This fixes a bug in svm_recalc_instruction_intercepts(). The intercepts for VMSAVE/VMLOAD are always toggled in vmcb01, but VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK is toggled in the current VMCB, which could be vmcb02 instead of vmcb01 if L2 is active.
Virtual VMSAVE/VMLOAD enablement in vmcb02 is separately controlled by nested_vmcb02_prepare_control() based on the vCPU features and VMCB12, and if intercepts are needed they are set by recalc_intercepts().
The bug is benign though. Not toggling the bit for vmcb01 is harmless because it's useless anyway. For vmcb02:
- The bit could be incorrectly cleared when intercepts are set in vmcb01. This is harmless because VMSAVE/VMLOAD will be emulated by KVM anyway.
- The bit could be incorrectly set when the intercepts are cleared in vmcb01. However, if the bit was originally clear in vmcb02, then recalc_intercepts() will enable in the intercepts in vmcb02 anyway and VMSAVE/VMLOAD will be emulated by KVM.
[ yosry: The bug is not benign. Since 5a1a726e68ff, a vCPU restored into
L2 (e.g. on live migration) recalculates intercepts with vmcb02 loaded, leaving vmcb01 with neither VMLOAD/VMSAVE intercepts nor virtual VMLOAD/VMSAVE, so L1's VMLOAD/VMSAVE corrupt host memory. ]
--- arch/x86/kvm/svm/svm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/svm.c +8 -2
@@ -981,10 +981,14 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)svm_set_intercept (svm,INTERCEPT_RDTSCP ); } +/* +* No need to toggle VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK here, it is +* always set if vls is enabled. If the intercepts are set, the bit is +* meaningless anyway. +*/ if (guest_cpuid_is_intel_compatible (vcpu)) {svm_set_intercept (svm,INTERCEPT_VMLOAD );svm_set_intercept (svm,INTERCEPT_VMSAVE ); - svm->vmcb->control.virt_ext &= ~VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK ; }else {/* * If hardware supports Virtual VMLOAD VMSAVE then enable it
@@ -993,7 +997,6 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)if (vls) {svm_clr_intercept (svm,INTERCEPT_VMLOAD );svm_clr_intercept (svm,INTERCEPT_VMSAVE ); - svm->vmcb->control.virt_ext |=VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK ; } } }
@@ -1137,6 +1140,9 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) svm->vmcb->control.int_ctl |=V_GIF_ENABLE_MASK ; } +if (vls) + svm->vmcb->control.virt_ext |=VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK ; +if (vcpu->kvm->arch.bus_lock_detection_enabled)svm_set_intercept (svm,INTERCEPT_BUSLOCK );
Signature · 1 line
2.56.0.212.g129ede20d9-goog
Quoted 43 lines · Yosry Ahmed
From: Yosry Ahmed <[email protected]>
[ Upstream commit
55780d8a1dcc93d2c4b33c565ada88df12c9f206]Virtual VMSAVE/VMLOAD enablement (i.e.
VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK) is set/cleared bysvm_recalc_instruction_intercepts()when the intercepts are cleared/set. This is unnecessary because the bit is meaningless when intercepts are set and KVM emulates the instructions. Initialize the bit in vmcb01 base on vls, and keep it unchanged.This is similar-ish to how vGIF is handled. It is enabled in
init_vmcb()if vgif=1 and remains unchanged when the STGI intercept is enabled (e.g. for NMI windows).This fixes a bug in
svm_recalc_instruction_intercepts(). The intercepts for VMSAVE/VMLOAD are always toggled in vmcb01, butVIRTUAL_VMLOAD_VMSAVE_ENABLE_MASKis toggled in the current VMCB, which could be vmcb02 instead of vmcb01 if L2 is active.Virtual VMSAVE/VMLOAD enablement in vmcb02 is separately controlled by
nested_vmcb02_prepare_control()based on the vCPU features and VMCB12, and if intercepts are needed they are set byrecalc_intercepts().The bug is benign though. Not toggling the bit for vmcb01 is harmless because it's useless anyway. For vmcb02:
- The bit could be incorrectly cleared when intercepts are set in vmcb01. This is harmless because VMSAVE/VMLOAD will be emulated by KVM anyway.
- The bit could be incorrectly set when the intercepts are cleared in vmcb01. However, if the bit was originally clear in vmcb02, then
recalc_intercepts()will enable in the intercepts in vmcb02 anyway and VMSAVE/VMLOAD will be emulated by KVM.Signed-off-by Yosry Ahmed <[email protected]>Signed-off-by Sean Christopherson <[email protected]>Fixes 5a1a726e68ff ("KVM: x86: Use KVM_REQ_RECALC_INTERCEPTS to react to CPUID updates")Reported-by Nikola Ciprich <[email protected]>Tested-by Nikola Ciprich <[email protected]>
Quoted 6 lines
[ yosry: The bug is not benign. Since
5a1a726e68ff, a vCPU restored into
L2 (e.g. on live migration) recalculates intercepts with vmcb02 loaded, leaving vmcb01 with neither VMLOAD/VMSAVE intercepts nor virtual VMLOAD/VMSAVE, so L1's VMLOAD/VMSAVE corrupt host memory. ]Signed-off-by Yosry Ahmed <[email protected]>---
Quoted 3 lines
Virtual VMSAVE/VMLOAD enablement (i.e.
VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK) is set/cleared bysvm_recalc_instruction_intercepts()when the intercepts are cleared/set.
Queued for 6.18, thanks.
Signature · 2 lines
Thanks, Sasha