KVM: nVMX: guest reset fixes
Subsystems (2)
cover KVM: nVMX: guest reset fixes
Few patches were recently marked for stable@ but commits are not backportable as-is and require a few tweaks. Here is 4.19 stable backport.
Jan Kiszka (1):
KVM: nVMX: Clear pending KVM_REQ_GET_VMCS12_PAGES when leaving nested
Paolo Bonzini (1):
KVM: nVMX: do not use dangling shadow VMCS after guest reset
arch/x86/kvm/vmx.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
Signature · 1 line
2.20.1
PATCH 1/2 KVM: nVMX: do not use dangling shadow VMCS after guest reset
From: Paolo Bonzini <[email protected]>
[ Upstream commit 88dddc11a8d6b09201b4db9d255b3394d9bc9e57 ]
If a KVM guest is reset while running a nested guest, free_nested will disable the shadow VMCS execution control in the vmcs01. However, on the next KVM_RUN vmx_vcpu_run would nevertheless try to sync the VMCS12 to the shadow VMCS which has since been freed.
This causes a vmptrld of a NULL pointer on my machime, but Jan reports the host to hang altogether. Let's see how much this trivial patch fixes.
--- arch/x86/kvm/vmx.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx.c +7 -1
@@ -8457,6 +8457,7 @@ static void vmx_disable_shadow_vmcs(struct vcpu_vmx *vmx) {vmcs_clear_bits (SECONDARY_VM_EXEC_CONTROL ,SECONDARY_EXEC_SHADOW_VMCS );vmcs_write64 (VMCS_LINK_POINTER ,-1ull ); + vmx->nested.sync_shadow_vmcs = false; }static inline void nested_release_vmcs12 (struct vcpu_vmx *vmx)
@@ -8468,7 +8469,6 @@ static inline void nested_release_vmcs12(struct vcpu_vmx *vmx)/* copy to memory all shadowed fields in case they were modified */ copy_shadow_to_vmcs12 (vmx); - vmx->nested.sync_shadow_vmcs = false;vmx_disable_shadow_vmcs (vmx); } vmx->nested.posted_intr_nv =-1 ;
@@ -8668,6 +8668,9 @@ static void copy_shadow_to_vmcs12(struct vcpu_vmx *vmx)u64 field_value;struct vmcs *shadow_vmcs = vmx->vmcs01.shadow_vmcs; +if (WARN_ON (!shadow_vmcs)) +return ; +preempt_disable ();vmcs_load (shadow_vmcs);
@@ -8706,6 +8709,9 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)u64 field_value =0 ;struct vmcs *shadow_vmcs = vmx->vmcs01.shadow_vmcs; +if (WARN_ON (!shadow_vmcs)) +return ; +vmcs_load (shadow_vmcs);for (q =0 ; q <ARRAY_SIZE (fields); q++) {
Signature · 1 line
2.20.1
Re: [PATCH stable-4.19 1/2] KVM: nVMX: do not use dangling shadow VMCS after guest reset
Quoted 10 lines · Vitaly Kuznetsov <[email protected]> 于2019年7月25日周四 下午3:29写道:
From: Paolo Bonzini <[email protected]>
[ Upstream commit
88dddc11a8d6b09201b4db9d255b3394d9bc9e57]If a KVM guest is reset while running a nested guest,
free_nestedwill disable the shadow VMCS execution control in the vmcs01. However, on the nextKVM_RUNvmx_vcpu_runwould nevertheless try to sync the VMCS12 to the shadow VMCS which has since been freed.
Quoted 7 lines
This causes a vmptrld of a NULL pointer on my machime, but Jan reports the host to hang altogether. Let's see how much this trivial patch fixes.
Reported-by Jan Kiszka <[email protected]>Cc Liran Alon <[email protected]>Signed-off-by Paolo Bonzini <[email protected]>
Hi all,
Do we need to backport the fix also to stable 4.14? It applies cleanly and compiles fine.
Regards,
Jack Wang
Paolo Bonzini
maintainerRe: [PATCH stable-4.19 1/2] KVM: nVMX: do not use dangling shadow VMCS after guest reset
Quoted 19 lines · Jack Wang
Quoted 10 lines · Vitaly Kuznetsov <[email protected]> 于2019年7月25日周四 下午3:29写道:
From: Paolo Bonzini <[email protected]>
[ Upstream commit
88dddc11a8d6b09201b4db9d255b3394d9bc9e57]If a KVM guest is reset while running a nested guest,
free_nestedwill disable the shadow VMCS execution control in the vmcs01. However, on the nextKVM_RUNvmx_vcpu_runwould nevertheless try to sync the VMCS12 to the shadow VMCS which has since been freed.Quoted 7 lines
This causes a vmptrld of a NULL pointer on my machime, but Jan reports the host to hang altogether. Let's see how much this trivial patch fixes.
Reported-by Jan Kiszka <[email protected]>Cc Liran Alon <[email protected]>Signed-off-by Paolo Bonzini <[email protected]>
Quoted 4 lines
Hi all,
Do we need to backport the fix also to stable 4.14? It applies cleanly and compiles fine.
The reproducer required newer kernels that support KVM_GET_NESTED_STATE and KVM_SET_NESTED_STATE, so it would be hard to test it. However, the patch itself should be safe.
Paolo
Re: [PATCH stable-4.19 1/2] KVM: nVMX: do not use dangling shadow VMCS after guest reset
Quoted 23 lines · Paolo Bonzini <[email protected]> 于2019年7月29日周一 上午11:10写道:
Quoted 21 lines · Jack Wang
Quoted 10 lines · Vitaly Kuznetsov <[email protected]> 于2019年7月25日周四 下午3:29写道:
From: Paolo Bonzini <[email protected]>
[ Upstream commit
88dddc11a8d6b09201b4db9d255b3394d9bc9e57]If a KVM guest is reset while running a nested guest,
free_nestedwill disable the shadow VMCS execution control in the vmcs01. However, on the nextKVM_RUNvmx_vcpu_runwould nevertheless try to sync the VMCS12 to the shadow VMCS which has since been freed.Quoted 7 lines
This causes a vmptrld of a NULL pointer on my machime, but Jan reports the host to hang altogether. Let's see how much this trivial patch fixes.
Reported-by Jan Kiszka <[email protected]>Cc Liran Alon <[email protected]>Signed-off-by Paolo Bonzini <[email protected]>Hi all,
Quoted 8 lines
Quoted 2 lines
Do we need to backport the fix also to stable 4.14? It applies cleanly and compiles fine.
The reproducer required newer kernels that support
KVM_GET_NESTED_STATEandKVM_SET_NESTED_STATE, so it would be hard to test it. However, the patch itself should be safe.Paolo
Thanks Paolo for confirmation. I'm asking because we had one incident in our production with 4.14.129 kernel,
System is Skylake Gold cpu, first kvm errors, host hung afterwards
kernel: [1186161.091160] kvm: vmptrld (null)/6bfc00000000 failed
kernel: [1186161.091537] kvm: vmclear fail: (null)/6bfc00000000
kernel: [1186186.490300] watchdog: BUG: soft lockup - CPU#54 stuck for 23s! [qemu:16639]
Hi Sasha, hi Greg,
Would be great if you can pick this patch also to 4.14 kernel.
Best regards,
Jack Wang
Re: [PATCH stable-4.19 1/2] KVM: nVMX: do not use dangling shadow VMCS after guest reset
Quoted 4 lines · Jack Wang
Thanks Paolo for confirmation. I'm asking because we had one incident in our production with 4.14.129 kernel,
System is Skylake Gold cpu, first kvm errors, host hung afterwards
Quoted 8 lines
kernel: [1186161.091160] kvm: vmptrld (null)/6bfc00000000 failed
kernel: [1186161.091537] kvm: vmclear fail: (null)/6bfc00000000
kernel: [1186186.490300] watchdog: BUG: soft lockup - CPU#54 stuck for 23s! [qemu:16639]Hi Sasha, hi Greg,
Would be great if you can pick this patch also to 4.14 kernel.
Greg Kroah-Hartman
maintainerRe: [PATCH stable-4.19 1/2] KVM: nVMX: do not use dangling shadow VMCS after guest reset
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Jack Wang
Thanks Paolo for confirmation. I'm asking because we had one incident in our production with 4.14.129 kernel,
System is Skylake Gold cpu, first kvm errors, host hung afterwardskernel: [1186161.091160] kvm: vmptrld (null)/6bfc00000000 failed
kernel: [1186161.091537] kvm: vmclear fail: (null)/6bfc00000000
kernel: [1186186.490300] watchdog: BUG: soft lockup - CPU#54 stuck for 23s! [qemu:16639]
Quoted 5 lines
Quoted 3 lines
Hi Sasha, hi Greg,
Would be great if you can pick this patch also to 4.14 kernel.
Acked-by Paolo Bonzini <[email protected]>
Ok, now queued up, thanks.
greg k-h
PATCH 2/2 KVM: nVMX: Clear pending KVM_REQ_GET_VMCS12_PAGES when leaving nested
In reply to Vitaly KuznetsovFrom: Jan Kiszka <[email protected]>
[ Upstream commit cf64527bb33f6cec2ed50f89182fc4688d0056b6 ]
Letting this pend may cause nested_get_vmcs12_pages to run against an invalid state, corrupting the effective vmcs of L1.
This was triggerable in QEMU after a guest corruption in L2, followed by a L1 reset.
--- arch/x86/kvm/vmx.c | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/kvm/vmx.c +2 -0
@@ -8490,6 +8490,8 @@ static void free_nested(struct vcpu_vmx *vmx)if (!vmx->nested.vmxon && !vmx->nested.smm.vmxon)return ; +kvm_clear_request (KVM_REQ_GET_VMCS12_PAGES , &vmx->vcpu); +hrtimer_cancel (&vmx->nested.preemption_timer); vmx->nested.vmxon = false; vmx->nested.smm.vmxon = false;
Signature · 1 line
2.20.1
Quoted 12 lines · Vitaly Kuznetsov
Few patches were recently marked for stable@ but commits are not backportable as-is and require a few tweaks. Here is 4.19 stable backport.
Jan Kiszka (1):
KVM: nVMX: Clear pendingKVM_REQ_GET_VMCS12_PAGESwhen leaving nestedPaolo Bonzini (1):
KVM: nVMX: do not use dangling shadow VMCS after guest resetarch/x86/kvm/vmx.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
Thanks Vitaly for helping out.
Paolo
Quoted 11 lines · Vitaly Kuznetsov
Few patches were recently marked for stable@ but commits are not backportable as-is and require a few tweaks. Here is 4.19 stable backport.
Jan Kiszka (1):
KVM: nVMX: Clear pendingKVM_REQ_GET_VMCS12_PAGESwhen leaving nestedPaolo Bonzini (1):
KVM: nVMX: do not use dangling shadow VMCS after guest resetarch/x86/kvm/vmx.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
All now applied, thanks!
greg k-h