Few nSVM bugfixes
cover Few nSVM bugfixes
This patch series contains few nested SVM fixes from testing I did this weekend.
Patch #1 fixes issue where we were setting the GIF (global interrupt flag) on first nested VMexit, after migration thus making the nested guest crash from unexpected interrupts.
Patch #2 is my observation that we never setup nesed msr bitmap on nested state load after migration.
Patch #3 was 'migrated' ;-) from my other patch series to make it smaller, which is about more strict checks when we about to return to a nested guest, from SMM.
Best regards,
Maxim Levitsky
Maxim Levitsky (3):
SVM: nSVM: correctly restore GIF on vmexit from nesting after
migration
SVM: nSVM: setup nested msr permission bitmap on nested state load
KVM: nSVM: more strict SMM checks when returning to nested guest
arch/x86/kvm/svm/nested.c | 7 ++++++- arch/x86/kvm/svm/svm.c | 29 ++++++++++++++++++----------- 2 files changed, 24 insertions(+), 12 deletions(-)
Signature · 1 line
2.26.2
PATCH 1/3 SVM: nSVM: correctly restore GIF on vmexit from nesting after migration
Currently code in svm_set_nested_state copies the current vmcb control area to L1 control area (hsave->control), under assumption that it mostly reflects the defaults that kvm choose, and later qemu overrides these defaults with L2 state using standard KVM interfaces, like KVM_SET_REGS.
However nested GIF (which is AMD specific thing) is by default is true, and it is copied to hsave area as such.
This alone is not a big deal since on VMexit, GIF is always set to false, regardless of what it was on VM entry.
However in nested_svm_vmexit we were first were setting GIF to false, but then we overwrite this with value from the hsave area.
Now on normal vm entry this is not a problem, since GIF is false prior to normal vm entry,
and this is the value that copied to hsave, and then restored, but this is not always the case when the nested state is loaded as explained above.
Anyway to fix this issue, move svm_set_gif after we restore the L1 control state in nested_svm_vmexit, so that even with wrong GIF in the saved L1 control area, we still clear GIF as the spec says.
All of this is only relevant when GIF virtualization is enabled, (otherwise nested GIF doesn't reside in the vmcb).
--- arch/x86/kvm/svm/nested.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +3 -1
@@ -586,7 +586,6 @@ int nested_svm_vmexit(struct vcpu_svm *svm) svm->vcpu.arch.mp_state =KVM_MP_STATE_RUNNABLE ;/* Give the current vmcb to the guest */ -svm_set_gif (svm, false); nested_vmcb->save.es = vmcb->save.es; nested_vmcb->save.cs = vmcb->save.cs;
@@ -632,6 +631,9 @@ int nested_svm_vmexit(struct vcpu_svm *svm)/* Restore the original control entries */ copy_vmcb_control_area (&vmcb->control, &hsave->control); +/* On vmexit the GIF is set to false */ +svm_set_gif (svm, false); + svm->vmcb->control.tsc_offset = svm->vcpu.arch.tsc_offset = svm->vcpu.arch.l1_tsc_offset;
Signature · 1 line
2.26.2
Paolo Bonzini
maintainerRe: [PATCH 1/3] SVM: nSVM: correctly restore GIF on vmexit from nesting after migration
Quoted 2 lines · Maxim Levitsky
However in
nested_svm_vmexitwe were first were setting GIF to false, but then we overwrite this with value from the hsave area.
Do you mean we are overwriting the resulting intercepts with values from the hsave area? If so, I can rewrite the commit message but the patch is good!
Paolo
PATCH 2/3 SVM: nSVM: setup nested msr permission bitmap on nested state load
In reply to Maxim LevitskyThis code was missing and was forcing the L2 run with L1's msr permission bitmap
--- arch/x86/kvm/svm/nested.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/nested.c +3 -0
@@ -1134,6 +1134,9 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,load_nested_vmcb_control (svm, &ctl);nested_prepare_vmcb_control (svm); +if (!nested_svm_vmrun_msrpm (svm)) +return -EINVAL ; + out_set_gif:svm_set_gif (svm, !!(kvm_state->flags &KVM_STATE_NESTED_GIF_SET ));return 0 ;
Signature · 1 line
2.26.2
PATCH 3/3 KVM: nSVM: more strict SMM checks when returning to nested guest
In reply to Maxim Levitsky* check that guest is 64 bit guest, otherwise the SVM related fields in the smm state area are not defined
* If the SMM area indicates that SMM interrupted a running guest, check that EFER.SVME which is also saved in this area is set, otherwise the guest might have tampered with SMM save area, and so indicate emulation failure which should triple fault the guest.
* Check that that guest CPUID supports SVM (due to the same issue as above)
--- arch/x86/kvm/svm/svm.c | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)
arch/x86/kvm/svm/svm.c +18 -11
@@ -3900,21 +3900,28 @@ static int svm_pre_enter_smm(struct kvm_vcpu *vcpu, char *smstate)static int svm_pre_leave_smm (struct kvm_vcpu *vcpu,const char *smstate) {struct vcpu_svm *svm =to_svm (vcpu); -struct vmcb *nested_vmcb;struct kvm_host_map map; -u64 guest; -u64 vmcb;int ret =0 ; - guest =GET_SMSTATE (u64, smstate,0x7ed8 ); - vmcb =GET_SMSTATE (u64, smstate,0x7ee0 ); +if (guest_cpuid_has (vcpu,X86_FEATURE_LM )) { +u64 saved_efer =GET_SMSTATE (u64, smstate,0x7ed0 ); +u64 guest =GET_SMSTATE (u64, smstate,0x7ed8 ); +u64 vmcb =GET_SMSTATE (u64, smstate,0x7ee0 ); -if (guest) { -if (kvm_vcpu_map (&svm->vcpu,gpa_to_gfn (vmcb), &map) == -EINVAL ) -return 1 ; - nested_vmcb = map.hva; - ret =enter_svm_guest_mode (svm, vmcb, nested_vmcb); -kvm_vcpu_unmap (&svm->vcpu, &map, true); +if (guest) { +if (!guest_cpuid_has (vcpu,X86_FEATURE_SVM )) +return 1 ; + +if (!(saved_efer &&EFER_SVME )) +return 1 ; + +if (kvm_vcpu_map (&svm->vcpu, +gpa_to_gfn (vmcb), &map) == -EINVAL ) +return 1 ; + + ret =enter_svm_guest_mode (svm, vmcb, map.hva); +kvm_vcpu_unmap (&svm->vcpu, &map, true); + } }return ret;
Signature · 1 line
2.26.2
Re: [PATCH 3/3] KVM: nSVM: more strict SMM checks when returning to nested guest
Hi Maxim,
url: https://github.com/0day-ci/linux/commits/Maxim-Levitsky/Few-nSVM-bugfixes/20200828-003025 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next config: x86_64-randconfig-m001-20200827 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
If you fix the issue, kindly add following tag as appropriate
smatch warnings:arch/x86/kvm/svm/svm.c:3915 svm_pre_leave_smm() warn: should this be a bitwise op?
# https://github.com/0day-ci/linux/commit/e2317f8eb1f0e9f731ddbe66ab175be19f3bdaf1
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Maxim-Levitsky/Few-nSVM-bugfixes/20200828-003025 git checkout e2317f8eb1f0e9f731ddbe66ab175be19f3bdaf1
vim +3915 arch/x86/kvm/svm/svm.c
ed19321fb657121 arch/x86/kvm/svm.c Sean Christopherson 2019-04-02 3900 static int svm_pre_leave_smm(struct kvm_vcpu *vcpu, const char *smstate)
0234bf885236a41 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3901 {
05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3902 struct vcpu_svm *svm = to_svm(vcpu);
8c5fbf1a7231078 arch/x86/kvm/svm.c KarimAllah Ahmed 2019-01-31 3903 struct kvm_host_map map;
59cd9bc5b03f0ba arch/x86/kvm/svm/svm.c Vitaly Kuznetsov 2020-07-10 3904 int ret = 0;
05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3905
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3906 if (guest_cpuid_has(vcpu, X86_FEATURE_LM)) {
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3907 u64 saved_efer = GET_SMSTATE(u64, smstate, 0x7ed0);
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3908 u64 guest = GET_SMSTATE(u64, smstate, 0x7ed8);
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3909 u64 vmcb = GET_SMSTATE(u64, smstate, 0x7ee0);
05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3910
ed19321fb657121 arch/x86/kvm/svm.c Sean Christopherson 2019-04-02 3911 if (guest) {
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3912 if (!guest_cpuid_has(vcpu, X86_FEATURE_SVM))
9ec19493fb86d6d arch/x86/kvm/svm.c Sean Christopherson 2019-04-02 3913 return 1;
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3914
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 @3915 if (!(saved_efer && EFER_SVME))
^^It looks like bitwise AND was intended.
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3916 return 1; e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3917 e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3918 if (kvm_vcpu_map(&svm->vcpu, e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3919 gpa_to_gfn(vmcb), &map) == -EINVAL) e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3920 return 1; e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3921 e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3922 ret = enter_svm_guest_mode(svm, vmcb, map.hva); 69c9dfa24bb7bac arch/x86/kvm/svm/svm.c Paolo Bonzini 2020-05-13 3923 kvm_vcpu_unmap(&svm->vcpu, &map, true); 05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3924 } e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3925 } 59cd9bc5b03f0ba arch/x86/kvm/svm/svm.c Vitaly Kuznetsov 2020-07-10 3926 59cd9bc5b03f0ba arch/x86/kvm/svm/svm.c Vitaly Kuznetsov 2020-07-10 3927 return ret; 0234bf885236a41 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3928 }
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Re: [PATCH 3/3] KVM: nSVM: more strict SMM checks when returning to nested guest
Quoted 37 lines · Dan Carpenter
Hi Maxim,
url: https://github.com/0day-ci/linux/commits/Maxim-Levitsky/Few-nSVM-bugfixes/20200828-003025 base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next config: x86_64-randconfig-m001-20200827 (attached as .config) compiler: gcc-9 (Debian 9.3.0-15) 9.3.0If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>Reported-by Dan Carpenter <[email protected]>smatch warnings:
arch/x86/kvm/svm/svm.c:3915svm_pre_leave_smm()warn: should this be a bitwise op?# https://github.com/0day-ci/linux/commit/e2317f8eb1f0e9f731ddbe66ab175be19f3bdaf1
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Maxim-Levitsky/Few-nSVM-bugfixes/20200828-003025 git checkoute2317f8eb1f0e9f731ddbe66ab175be19f3bdaf1
vim +3915arch/x86/kvm/svm/svm.ced19321fb657121 arch/x86/kvm/svm.c Sean Christopherson 2019-04-02 3900 static int svm_pre_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) 0234bf885236a41 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3901 { 05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3902 struct vcpu_svm *svm = to_svm(vcpu); 8c5fbf1a7231078 arch/x86/kvm/svm.c KarimAllah Ahmed 2019-01-31 3903 struct kvm_host_map map; 59cd9bc5b03f0ba arch/x86/kvm/svm/svm.c Vitaly Kuznetsov 2020-07-10 3904 int ret = 0; 05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3905 e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3906 if (guest_cpuid_has(vcpu, X86_FEATURE_LM)) { e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3907 u64 saved_efer = GET_SMSTATE(u64, smstate, 0x7ed0); e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3908 u64 guest = GET_SMSTATE(u64, smstate, 0x7ed8); e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3909 u64 vmcb = GET_SMSTATE(u64, smstate, 0x7ee0); 05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3910 ed19321fb657121 arch/x86/kvm/svm.c Sean Christopherson 2019-04-02 3911 if (guest) { e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3912 if (!guest_cpuid_has(vcpu, X86_FEATURE_SVM)) 9ec19493fb86d6d arch/x86/kvm/svm.c Sean Christopherson 2019-04-02 3913 return 1; e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3914 e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 @3915 if (!(saved_efer && EFER_SVME)) ^^
Quoted 1 line
It looks like bitwise AND was intended.
Oops. Thanks!
Best regards,
Maxim Levitskky
Quoted 18 lines
e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3916 return 1; e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3917 e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3918 if (kvm_vcpu_map(&svm->vcpu, e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3919 gpa_to_gfn(vmcb), &map) == -EINVAL) e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3920 return 1; e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3921 e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3922 ret = enter_svm_guest_mode(svm, vmcb, map.hva); 69c9dfa24bb7bac arch/x86/kvm/svm/svm.c Paolo Bonzini 2020-05-13 3923 kvm_vcpu_unmap(&svm->vcpu, &map, true); 05cade71cf3b925 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3924 } e2317f8eb1f0e9f arch/x86/kvm/svm/svm.c Maxim Levitsky 2020-08-27 3925 } 59cd9bc5b03f0ba arch/x86/kvm/svm/svm.c Vitaly Kuznetsov 2020-07-10 3926 59cd9bc5b03f0ba arch/x86/kvm/svm/svm.c Vitaly Kuznetsov 2020-07-10 3927 return ret; 0234bf885236a41 arch/x86/kvm/svm.c Ladi Prosek 2017-10-11 3928 }---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]
Quoted 23 lines · Maxim Levitsky
This patch series contains few nested SVM fixes from testing I did this weekend.
Patch #1 fixes issue where we were setting the GIF (global interrupt flag) on first nested VMexit, after migration thus making the nested guest crash from unexpected interrupts.
Patch #2 is my observation that we never setup nesed msr bitmap on nested state load after migration.
Patch #3 was 'migrated' ;-) from my other patch series to make it smaller, which is about more strict checks when we about to return to a nested guest, from SMM.
Best regards,
Maxim LevitskyMaxim Levitsky (3):
SVM: nSVM: correctly restore GIF on vmexit from nesting after
migration
SVM: nSVM: setup nested msr permission bitmap on nested state load
KVM: nSVM: more strict SMM checks when returning to nested guest
Quoted 4 lines
arch/x86/kvm/svm/nested.c | 7 ++++++- arch/x86/kvm/svm/svm.c | 29 ++++++++++++++++++----------- 2 files changed, 24 insertions(+), 12 deletions(-)
Queued, thanks.
Paolo