KVM: nVMX: Fix migration of nested guests when eVMCS is in use
cover KVM: nVMX: Fix migration of nested guests when eVMCS is in use
Win10 guests with WSL2 enabled sometimes crash on migration when enlightened VMCS was used. The condition seems to be induced by the situation when L2->L1 exit is caused immediately after migration and before L2 gets a chance to run (e.g. when there's an interrupt pending). The issue was introduced by commit f2c7ef3ba955 ("KVM: nSVM: cancel KVM_REQ_GET_NESTED_STATE_PAGES on nested vmexit") and the first patch of the series addresses the immediate issue. The eVMCS mapping restoration path, however, seems to be fragile and the rest of the series tries to make it more future proof by including eVMCS GPA in the migration data.
Vitaly Kuznetsov (4):
KVM: nVMX: Always make an attempt to map eVMCS after migration
KVM: nVMX: Properly pad 'struct kvm_vmx_nested_state_hdr'
KVM: nVMX: Introduce __nested_vmx_handle_enlightened_vmptrld()
KVM: nVMX: Map enlightened VMCS upon restore when possible
arch/x86/include/uapi/asm/kvm.h | 4 ++ arch/x86/kvm/vmx/nested.c | 82 +++++++++++++++++++++++---------- 2 files changed, 61 insertions(+), 25 deletions(-)
Signature · 1 line
2.30.2
PATCH 1/4 KVM: nVMX: Always make an attempt to map eVMCS after migration
When enlightened VMCS is in use and nested state is migrated with vmx_get_nested_state()/vmx_set_nested_state() KVM can't map evmcs page right away: evmcs gpa is not 'struct kvm_vmx_nested_state_hdr' and we can't read it from VP assist page because userspace may decide to restore HV_X64_MSR_VP_ASSIST_PAGE after restoring nested state (and QEMU, for example, does exactly that). To make sure eVMCS is mapped /vmx_set_nested_state() raises KVM_REQ_GET_NESTED_STATE_PAGES request.
Commit f2c7ef3ba955 ("KVM: nSVM: cancel KVM_REQ_GET_NESTED_STATE_PAGES on nested vmexit") added KVM_REQ_GET_NESTED_STATE_PAGES clearing to nested_vmx_vmexit() to make sure MSR permission bitmap is not switched when an immediate exit from L2 to L1 happens right after migration (caused by a pending event, for example). Unfortunately, in the exact same situation we still need to have eVMCS mapped so nested_sync_vmcs12_to_shadow() reflects changes in VMCS12 to eVMCS.
As a band-aid, restore nested_get_evmcs_page() when clearing KVM_REQ_GET_NESTED_STATE_PAGES in nested_vmx_vmexit(). The 'fix' is far from being ideal as we can't easily propagate possible failures and even if we could, this is most likely already too late to do so. The whole 'KVM_REQ_GET_NESTED_STATE_PAGES' idea for mapping eVMCS after migration seems to be fragile as we diverge too much from the 'native' path when vmptr loading happens on vmx_set_nested_state().
--- arch/x86/kvm/vmx/nested.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx/nested.c +19 -10
@@ -3098,15 +3098,8 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)nested_vmx_handle_enlightened_vmptrld (vcpu, false);if (evmptrld_status ==EVMPTRLD_VMFAIL || - evmptrld_status ==EVMPTRLD_ERROR ) { -pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , - __func__); - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; + evmptrld_status ==EVMPTRLD_ERROR )return false; - } }return true;
@@ -3194,8 +3187,16 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)static bool vmx_get_nested_state_pages (struct kvm_vcpu *vcpu) { -if (!nested_get_evmcs_page (vcpu)) +if (!nested_get_evmcs_page (vcpu)) { +pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , + __func__); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror = +KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return false; + }if (is_guest_mode (vcpu) && !nested_get_vmcs12_pages (vcpu))return false;
@@ -4422,7 +4423,15 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,/* trying to cancel vmlaunch/vmresume is a bug */ WARN_ON_ONCE (vmx->nested.nested_run_pending); -kvm_clear_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +/* +* KVM_REQ_GET_NESTED_STATE_PAGES is also used to map +* Enlightened VMCS after migration and we still need to +* do that when something is forcing L2->L1 exit prior to +* the first L2 run. +*/ + (void )nested_get_evmcs_page (vcpu); + }/* Service the TLB flush request for L2 before switching to L1. */ if (kvm_check_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu))
Signature · 1 line
2.30.2
Re: [PATCH 1/4] KVM: nVMX: Always make an attempt to map eVMCS after migration
Quoted 57 lines · Vitaly Kuznetsov
When enlightened VMCS is in use and nested state is migrated with vmx_get_nested_state()/vmx_set_nested_state() KVM can't map evmcs page right away: evmcs gpa is not 'struct
kvm_vmx_nested_state_hdr' and we can't read it from VP assist page because userspace may decide to restoreHV_X64_MSR_VP_ASSIST_PAGEafter restoring nested state (and QEMU, for example, does exactly that). To make sure eVMCS is mapped /vmx_set_nested_state() raisesKVM_REQ_GET_NESTED_STATE_PAGESrequest.Commit
f2c7ef3ba955("KVM: nSVM: cancelKVM_REQ_GET_NESTED_STATE_PAGESon nested vmexit") addedKVM_REQ_GET_NESTED_STATE_PAGESclearing tonested_vmx_vmexit()to make sure MSR permission bitmap is not switched when an immediate exit from L2 to L1 happens right after migration (caused by a pending event, for example). Unfortunately, in the exact same situation we still need to have eVMCS mapped sonested_sync_vmcs12_to_shadow()reflects changes in VMCS12 to eVMCS.As a band-aid, restore
nested_get_evmcs_page()when clearingKVM_REQ_GET_NESTED_STATE_PAGESinnested_vmx_vmexit(). The 'fix' is far from being ideal as we can't easily propagate possible failures and even if we could, this is most likely already too late to do so. The whole 'KVM_REQ_GET_NESTED_STATE_PAGES' idea for mapping eVMCS after migration seems to be fragile as we diverge too much from the 'native' path when vmptr loading happens onvmx_set_nested_state().Fixes f2c7ef3ba955 ("KVM: nSVM: cancel KVM_REQ_GET_NESTED_STATE_PAGES on nested vmexit")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)arch/x86/kvm/vmx/nested.c +1
-9@@ -3098,15 +3098,8 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)nested_vmx_handle_enlightened_vmptrld (vcpu, false);if (evmptrld_status ==EVMPTRLD_VMFAIL || - evmptrld_status ==EVMPTRLD_ERROR ) { -pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , - __func__); - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; + evmptrld_status ==EVMPTRLD_ERROR )return false; - } }return true;@@ -3194,8 +3187,16 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)static bool vmx_get_nested_state_pages (struct kvm_vcpu *vcpu) { -if (!nested_get_evmcs_page (vcpu))
arch/x86/kvm/vmx/nested.c static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) · line 3199
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-0+if (!nested_get_evmcs_page (vcpu)) { +pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , + __func__); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror = +KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return false; + }
Hi!
Any reason to move the debug prints out of nested_get_evmcs_page?
Quoted 7 lines
arch/x86/kvm/vmx/nested.c +0
-0if (is_guest_mode (vcpu) && !nested_get_vmcs12_pages (vcpu))return false;@@ -4422,7 +4423,15 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,/* trying to cancel vmlaunch/vmresume is a bug */ WARN_ON_ONCE (vmx->nested.nested_run_pending);
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4434
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-1-kvm_clear_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +/* +* KVM_REQ_GET_NESTED_STATE_PAGES is also used to map +* Enlightened VMCS after migration and we still need to +* do that when something is forcing L2->L1 exit prior to +* the first L2 run. +*/ + (void )nested_get_evmcs_page (vcpu); + }
Yes this is a band-aid, but it has to be done I agree.
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4437
Quoted 3 lines
arch/x86/kvm/vmx/nested.c +0
-0/* Service the TLB flush request for L2 before switching to L1. */ if (kvm_check_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu))
I also tested this and it survives a bit better (used to crash instantly
after a single migration cycle, but the guest still crashes after around ~20 iterations of my regular nested migration test).
Blues screen shows that stop code is HYPERVISOR ERROR and nothing else.
I tested both this patch alone and all 4 patches.
Without evmcs, the same VM with same host kernel and qemu survived an overnight test and passed about 1800 migration iterations.
(my synthetic migration test doesn't yet work on Intel, I need to investigate why)
For reference this is the VM that you gave me to test, kvm/queue kernel, with merged mainline in it,
and mostly latest qemu (updated about a week ago or so)
qemu: 3791642c8d60029adf9b00bcb4e34d7d8a1aea4d
kernel: 9f242010c3b46e63bc62f08fff42cef992d3801b and
then merge v5.12 from mainline.
Best regards,
Maxim Levitsky
Re: [PATCH 1/4] KVM: nVMX: Always make an attempt to map eVMCS after migration
Quoted 71 lines · Maxim Levitsky
Quoted 57 lines · Vitaly Kuznetsov
When enlightened VMCS is in use and nested state is migrated with vmx_get_nested_state()/vmx_set_nested_state() KVM can't map evmcs page right away: evmcs gpa is not 'struct
kvm_vmx_nested_state_hdr' and we can't read it from VP assist page because userspace may decide to restoreHV_X64_MSR_VP_ASSIST_PAGEafter restoring nested state (and QEMU, for example, does exactly that). To make sure eVMCS is mapped /vmx_set_nested_state() raisesKVM_REQ_GET_NESTED_STATE_PAGESrequest.Commit
f2c7ef3ba955("KVM: nSVM: cancelKVM_REQ_GET_NESTED_STATE_PAGESon nested vmexit") addedKVM_REQ_GET_NESTED_STATE_PAGESclearing tonested_vmx_vmexit()to make sure MSR permission bitmap is not switched when an immediate exit from L2 to L1 happens right after migration (caused by a pending event, for example). Unfortunately, in the exact same situation we still need to have eVMCS mapped sonested_sync_vmcs12_to_shadow()reflects changes in VMCS12 to eVMCS.As a band-aid, restore
nested_get_evmcs_page()when clearingKVM_REQ_GET_NESTED_STATE_PAGESinnested_vmx_vmexit(). The 'fix' is far from being ideal as we can't easily propagate possible failures and even if we could, this is most likely already too late to do so. The whole 'KVM_REQ_GET_NESTED_STATE_PAGES' idea for mapping eVMCS after migration seems to be fragile as we diverge too much from the 'native' path when vmptr loading happens onvmx_set_nested_state().Fixes f2c7ef3ba955 ("KVM: nSVM: cancel KVM_REQ_GET_NESTED_STATE_PAGES on nested vmexit")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)arch/x86/kvm/vmx/nested.c +1
-9@@ -3098,15 +3098,8 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)nested_vmx_handle_enlightened_vmptrld (vcpu, false);if (evmptrld_status ==EVMPTRLD_VMFAIL || - evmptrld_status ==EVMPTRLD_ERROR ) { -pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , - __func__); - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; + evmptrld_status ==EVMPTRLD_ERROR )return false; - } }return true;@@ -3194,8 +3187,16 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)static bool vmx_get_nested_state_pages (struct kvm_vcpu *vcpu) { -if (!nested_get_evmcs_page (vcpu))arch/x86/kvm/vmx/nested.c static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) · line 3199
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-0+if (!nested_get_evmcs_page (vcpu)) { +pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , + __func__); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror = +KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return false; + }Hi!
Quoted 2 lines
Any reason to move the debug prints out of
nested_get_evmcs_page?
Debug print could've probably stayed or could've been dropped completely -- I don't really believe it's going to help anyone. Debugging such issues without instrumentation/tracing seems to be hard-to-impossible...
Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/vmx/nested.c +0
-0if (is_guest_mode (vcpu) && !nested_get_vmcs12_pages (vcpu))return false;@@ -4422,7 +4423,15 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,/* trying to cancel vmlaunch/vmresume is a bug */ WARN_ON_ONCE (vmx->nested.nested_run_pending);
Quoted 12 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4434
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-1-kvm_clear_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +/* +* KVM_REQ_GET_NESTED_STATE_PAGES is also used to map +* Enlightened VMCS after migration and we still need to +* do that when something is forcing L2->L1 exit prior to +* the first L2 run. +*/ + (void )nested_get_evmcs_page (vcpu); + }Yes this is a band-aid, but it has to be done I agree.
To restore the status quo, yes.
Quoted 11 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4437
Quoted 3 lines
arch/x86/kvm/vmx/nested.c +0
-0/* Service the TLB flush request for L2 before switching to L1. */ if (kvm_check_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu))I also tested this and it survives a bit better (used to crash instantly
after a single migration cycle, but the guest still crashes after around ~20 iterations of my regular nested migration test).
Quoted 8 lines
Blues screen shows that stop code is HYPERVISOR ERROR and nothing else.
I tested both this patch alone and all 4 patches.
Without evmcs, the same VM with same host kernel and qemu survived an overnight test and passed about 1800 migration iterations.
(my synthetic migration test doesn't yet work on Intel, I need to investigate why)
It would be great to compare on Intel to be 100% sure the issue is eVMCS related, Hyper-V may be behaving quite differently on AMD.
Quoted 7 lines
For reference this is the VM that you gave me to test, kvm/queue kernel, with merged mainline in it,
and mostly latest qemu (updated about a week ago or so)qemu:
3791642c8d60029adf9b00bcb4e34d7d8a1aea4d
kernel:9f242010c3b46e63bc62f08fff42cef992d3801band
then merge v5.12 from mainline.
Thanks for testing! I'll try to come up with a selftest for this issue, maybe it'll help us discovering others)
Signature · 1 line
Vitaly
Re: [PATCH 1/4] KVM: nVMX: Always make an attempt to map eVMCS after migration
Quoted 117 lines · Vitaly Kuznetsov
Quoted 71 lines · Maxim Levitsky
Quoted 57 lines · Vitaly Kuznetsov
When enlightened VMCS is in use and nested state is migrated with vmx_get_nested_state()/vmx_set_nested_state() KVM can't map evmcs page right away: evmcs gpa is not 'struct
kvm_vmx_nested_state_hdr' and we can't read it from VP assist page because userspace may decide to restoreHV_X64_MSR_VP_ASSIST_PAGEafter restoring nested state (and QEMU, for example, does exactly that). To make sure eVMCS is mapped /vmx_set_nested_state() raisesKVM_REQ_GET_NESTED_STATE_PAGESrequest.Commit
f2c7ef3ba955("KVM: nSVM: cancelKVM_REQ_GET_NESTED_STATE_PAGESon nested vmexit") addedKVM_REQ_GET_NESTED_STATE_PAGESclearing tonested_vmx_vmexit()to make sure MSR permission bitmap is not switched when an immediate exit from L2 to L1 happens right after migration (caused by a pending event, for example). Unfortunately, in the exact same situation we still need to have eVMCS mapped sonested_sync_vmcs12_to_shadow()reflects changes in VMCS12 to eVMCS.As a band-aid, restore
nested_get_evmcs_page()when clearingKVM_REQ_GET_NESTED_STATE_PAGESinnested_vmx_vmexit(). The 'fix' is far from being ideal as we can't easily propagate possible failures and even if we could, this is most likely already too late to do so. The whole 'KVM_REQ_GET_NESTED_STATE_PAGES' idea for mapping eVMCS after migration seems to be fragile as we diverge too much from the 'native' path when vmptr loading happens onvmx_set_nested_state().Fixes f2c7ef3ba955 ("KVM: nSVM: cancel KVM_REQ_GET_NESTED_STATE_PAGES on nested vmexit")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)arch/x86/kvm/vmx/nested.c +1
-9@@ -3098,15 +3098,8 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)nested_vmx_handle_enlightened_vmptrld (vcpu, false);if (evmptrld_status ==EVMPTRLD_VMFAIL || - evmptrld_status ==EVMPTRLD_ERROR ) { -pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , - __func__); - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; + evmptrld_status ==EVMPTRLD_ERROR )return false; - } }return true;@@ -3194,8 +3187,16 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)static bool vmx_get_nested_state_pages (struct kvm_vcpu *vcpu) { -if (!nested_get_evmcs_page (vcpu))arch/x86/kvm/vmx/nested.c static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) · line 3199
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-0+if (!nested_get_evmcs_page (vcpu)) { +pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , + __func__); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror = +KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return false; + }Hi!
Quoted 2 lines
Any reason to move the debug prints out of
nested_get_evmcs_page?Debug print could've probably stayed or could've been dropped completely -- I don't really believe it's going to help anyone. Debugging such issues without instrumentation/tracing seems to be hard-to-impossible...
Quoted 7 lines
Quoted 7 lines
arch/x86/kvm/vmx/nested.c +0
-0if (is_guest_mode (vcpu) && !nested_get_vmcs12_pages (vcpu))return false;@@ -4422,7 +4423,15 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,/* trying to cancel vmlaunch/vmresume is a bug */ WARN_ON_ONCE (vmx->nested.nested_run_pending);Quoted 12 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4434
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-1-kvm_clear_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +/* +* KVM_REQ_GET_NESTED_STATE_PAGES is also used to map +* Enlightened VMCS after migration and we still need to +* do that when something is forcing L2->L1 exit prior to +* the first L2 run. +*/ + (void )nested_get_evmcs_page (vcpu); + }Yes this is a band-aid, but it has to be done I agree.
To restore the status quo, yes.
Quoted 14 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4437
Quoted 3 lines
arch/x86/kvm/vmx/nested.c +0
-0/* Service the TLB flush request for L2 before switching to L1. */ if (kvm_check_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu))I also tested this and it survives a bit better (used to crash instantly
after a single migration cycle, but the guest still crashes after around ~20 iterations of my regular nested migration test).Blues screen shows that stop code is HYPERVISOR ERROR and nothing else.
I tested both this patch alone and all 4 patches.
Quoted 7 lines
Quoted 4 lines
Without evmcs, the same VM with same host kernel and qemu survived an overnight test and passed about 1800 migration iterations.
(my synthetic migration test doesn't yet work on Intel, I need to investigate why)It would be great to compare on Intel to be 100% sure the issue is eVMCS related, Hyper-V may be behaving quite differently on AMD.
Hi!
I tested this on my Intel machine with and without eVMCS, without changing any other parameters, running the same VM from a snapshot.
As I said without eVMCS the test survived overnight stress of ~1800 migrations. With eVMCs, it fails pretty much on first try.
With those patches, it fails after about 20 iterations.
Best regards,
Maxim Levitsky
Quoted 12 lines
Quoted 8 lines
For reference this is the VM that you gave me to test, kvm/queue kernel, with merged mainline in it,
and mostly latest qemu (updated about a week ago or so)qemu:
3791642c8d60029adf9b00bcb4e34d7d8a1aea4d
kernel:9f242010c3b46e63bc62f08fff42cef992d3801band
then merge v5.12 from mainline.Thanks for testing! I'll try to come up with a selftest for this issue, maybe it'll help us discovering others)
Re: [PATCH 1/4] KVM: nVMX: Always make an attempt to map eVMCS after migration
Quoted 125 lines · Maxim Levitsky
Quoted 117 lines · Vitaly Kuznetsov
Quoted 71 lines · Maxim Levitsky
Quoted 57 lines · Vitaly Kuznetsov
When enlightened VMCS is in use and nested state is migrated with vmx_get_nested_state()/vmx_set_nested_state() KVM can't map evmcs page right away: evmcs gpa is not 'struct
kvm_vmx_nested_state_hdr' and we can't read it from VP assist page because userspace may decide to restoreHV_X64_MSR_VP_ASSIST_PAGEafter restoring nested state (and QEMU, for example, does exactly that). To make sure eVMCS is mapped /vmx_set_nested_state() raisesKVM_REQ_GET_NESTED_STATE_PAGESrequest.Commit
f2c7ef3ba955("KVM: nSVM: cancelKVM_REQ_GET_NESTED_STATE_PAGESon nested vmexit") addedKVM_REQ_GET_NESTED_STATE_PAGESclearing tonested_vmx_vmexit()to make sure MSR permission bitmap is not switched when an immediate exit from L2 to L1 happens right after migration (caused by a pending event, for example). Unfortunately, in the exact same situation we still need to have eVMCS mapped sonested_sync_vmcs12_to_shadow()reflects changes in VMCS12 to eVMCS.As a band-aid, restore
nested_get_evmcs_page()when clearingKVM_REQ_GET_NESTED_STATE_PAGESinnested_vmx_vmexit(). The 'fix' is far from being ideal as we can't easily propagate possible failures and even if we could, this is most likely already too late to do so. The whole 'KVM_REQ_GET_NESTED_STATE_PAGES' idea for mapping eVMCS after migration seems to be fragile as we diverge too much from the 'native' path when vmptr loading happens onvmx_set_nested_state().Fixes f2c7ef3ba955 ("KVM: nSVM: cancel KVM_REQ_GET_NESTED_STATE_PAGES on nested vmexit")Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)arch/x86/kvm/vmx/nested.c +1
-9@@ -3098,15 +3098,8 @@ static bool nested_get_evmcs_page(struct kvm_vcpu *vcpu)nested_vmx_handle_enlightened_vmptrld (vcpu, false);if (evmptrld_status ==EVMPTRLD_VMFAIL || - evmptrld_status ==EVMPTRLD_ERROR ) { -pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , - __func__); - vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; - vcpu->run->internal.suberror = -KVM_INTERNAL_ERROR_EMULATION ; - vcpu->run->internal.ndata =0 ; + evmptrld_status ==EVMPTRLD_ERROR )return false; - } }return true;@@ -3194,8 +3187,16 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)static bool vmx_get_nested_state_pages (struct kvm_vcpu *vcpu) { -if (!nested_get_evmcs_page (vcpu))arch/x86/kvm/vmx/nested.c static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) · line 3199
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-0+if (!nested_get_evmcs_page (vcpu)) { +pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , + __func__); + vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ; + vcpu->run->internal.suberror = +KVM_INTERNAL_ERROR_EMULATION ; + vcpu->run->internal.ndata =0 ; +return false; + }Hi!
Quoted 2 lines
Any reason to move the debug prints out of
nested_get_evmcs_page?Debug print could've probably stayed or could've been dropped completely -- I don't really believe it's going to help anyone. Debugging such issues without instrumentation/tracing seems to be hard-to-impossible...
Quoted 7 lines
Quoted 7 lines
arch/x86/kvm/vmx/nested.c +0
-0if (is_guest_mode (vcpu) && !nested_get_vmcs12_pages (vcpu))return false;@@ -4422,7 +4423,15 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,/* trying to cancel vmlaunch/vmresume is a bug */ WARN_ON_ONCE (vmx->nested.nested_run_pending);Quoted 12 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4434
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +9
-1-kvm_clear_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (kvm_check_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu)) { +/* +* KVM_REQ_GET_NESTED_STATE_PAGES is also used to map +* Enlightened VMCS after migration and we still need to +* do that when something is forcing L2->L1 exit prior to +* the first L2 run. +*/ + (void )nested_get_evmcs_page (vcpu); + }Yes this is a band-aid, but it has to be done I agree.
To restore the status quo, yes.
Quoted 14 lines
arch/x86/kvm/vmx/nested.c void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, · line 4437
Quoted 3 lines
arch/x86/kvm/vmx/nested.c +0
-0/* Service the TLB flush request for L2 before switching to L1. */ if (kvm_check_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu))I also tested this and it survives a bit better (used to crash instantly
after a single migration cycle, but the guest still crashes after around ~20 iterations of my regular nested migration test).Blues screen shows that stop code is HYPERVISOR ERROR and nothing else.
I tested both this patch alone and all 4 patches.
Quoted 7 lines
Quoted 4 lines
Without evmcs, the same VM with same host kernel and qemu survived an overnight test and passed about 1800 migration iterations.
(my synthetic migration test doesn't yet work on Intel, I need to investigate why)It would be great to compare on Intel to be 100% sure the issue is eVMCS related, Hyper-V may be behaving quite differently on AMD.
Quoted 9 lines
Hi!
I tested this on my Intel machine with and without eVMCS, without changing any other parameters, running the same VM from a snapshot.
As I said without eVMCS the test survived overnight stress of ~1800 migrations. With eVMCs, it fails pretty much on first try.
With those patches, it fails after about 20 iterations.
Ah, sorry, misunderstood your 'synthetic migration test doesn't yet work on Intel' :-)
Signature · 1 line
Vitaly
PATCH 2/4 KVM: nVMX: Properly pad 'struct kvm_vmx_nested_state_hdr'
In reply to Vitaly KuznetsovEliminate the probably unwanted hole in 'struct kvm_vmx_nested_state_hdr':
Pre-patch:
struct kvm_vmx_nested_state_hdr {
__u64 vmxon_pa; /* 0 8 */
__u64 vmcs12_pa; /* 8 8 */ struct {
__u16 flags; /* 16 2 */
} smm; /* 16 2 */
/* XXX 2 bytes hole, try to pack */
__u32 flags; /* 20 4 */
__u64 preemption_timer_deadline; /* 24 8 */
};
Post-patch:
struct kvm_vmx_nested_state_hdr {
__u64 vmxon_pa; /* 0 8 */
__u64 vmcs12_pa; /* 8 8 */ struct {
__u16 flags; /* 16 2 */
} smm; /* 16 2 */
__u16 pad; /* 18 2 */
__u32 flags; /* 20 4 */
__u64 preemption_timer_deadline; /* 24 8 */
};
--- arch/x86/include/uapi/asm/kvm.h | 2 ++ 1 file changed, 2 insertions(+)
arch/x86/include/uapi/asm/kvm.h +2 -0
@@ -437,6 +437,8 @@ struct kvm_vmx_nested_state_hdr {__u16 flags; } smm; +__u16 pad; +__u32 flags;__u64 preemption_timer_deadline; };
Signature · 1 line
2.30.2
Re: [PATCH 2/4] KVM: nVMX: Properly pad 'struct kvm_vmx_nested_state_hdr'
Quoted 38 lines · Vitaly Kuznetsov
Eliminate the probably unwanted hole in 'struct
kvm_vmx_nested_state_hdr':Pre-patch:
structkvm_vmx_nested_state_hdr{__u64 vmxon_pa; /* 0 8 */ __u64 vmcs12_pa; /* 8 8 */struct {
__u16flags; /* 16 2 */
} smm; /* 16 2 *//* XXX 2 bytes hole, try to pack */
__u32flags; /* 20 4 */
__u64preemption_timer_deadline; /* 24 8 */
};Post-patch:
structkvm_vmx_nested_state_hdr{__u64 vmxon_pa; /* 0 8 */ __u64 vmcs12_pa; /* 8 8 */struct {
__u16flags; /* 16 2 */
} smm; /* 16 2 */
__u16pad; /* 18 2 */
__u32flags; /* 20 4 */
__u64preemption_timer_deadline; /* 24 8 */
};Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 2 ++ 1 file changed, 2 insertions(+)arch/x86/include/uapi/asm/kvm.h +0
-0
arch/x86/include/uapi/asm/kvm.h struct kvm_vmx_nested_state_hdr { · line 444
Quoted 8 lines
arch/x86/include/uapi/asm/kvm.h +2
-0@@ -437,6 +437,8 @@ struct kvm_vmx_nested_state_hdr {__u16 flags; } smm; +__u16 pad; +__u32 flags;__u64 preemption_timer_deadline; };
Looks good to me.
I wonder if we can enable the -Wpadded GCC warning to warn about such cases. Probably can't be enabled for the whole kernel but maybe we can enable it for KVM codebase at least, like we did with -Werror.
From GCC manual:
"-Wpadded
Warn if padding is included in a structure, either to align an element of the structure or to align the whole structure. Sometimes when this happens it is possible to rearrange the fields of the structure
to reduce the padding and so make the structure smaller."
Best regards,
Maxim Levitsky
Sean Christopherson
maintainerRe: [PATCH 2/4] KVM: nVMX: Properly pad 'struct kvm_vmx_nested_state_hdr'
Quoted 51 lines · Maxim Levitsky
Quoted 38 lines · Vitaly Kuznetsov
Eliminate the probably unwanted hole in 'struct
kvm_vmx_nested_state_hdr':Pre-patch:
structkvm_vmx_nested_state_hdr{__u64 vmxon_pa; /* 0 8 */ __u64 vmcs12_pa; /* 8 8 */struct {
__u16flags; /* 16 2 */
} smm; /* 16 2 *//* XXX 2 bytes hole, try to pack */
__u32flags; /* 20 4 */
__u64preemption_timer_deadline; /* 24 8 */
};Post-patch:
structkvm_vmx_nested_state_hdr{__u64 vmxon_pa; /* 0 8 */ __u64 vmcs12_pa; /* 8 8 */struct {
__u16flags; /* 16 2 */
} smm; /* 16 2 */
__u16pad; /* 18 2 */
__u32flags; /* 20 4 */
__u64preemption_timer_deadline; /* 24 8 */
};Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 2 ++ 1 file changed, 2 insertions(+)arch/x86/include/uapi/asm/kvm.h +0
-0arch/x86/include/uapi/asm/kvm.h struct kvm_vmx_nested_state_hdr { · line 444
Quoted 8 lines
arch/x86/include/uapi/asm/kvm.h +2
-0@@ -437,6 +437,8 @@ struct kvm_vmx_nested_state_hdr {__u16 flags; } smm; +__u16 pad; +__u32 flags;__u64 preemption_timer_deadline; };Looks good to me.
Quoted 3 lines
I wonder if we can enable the -Wpadded GCC warning to warn about such cases. Probably can't be enabled for the whole kernel but maybe we can enable it for KVM codebase at least, like we did with -Werror.
It'll never work, there are far, far too many structs throughout the kernel and KVM that have implicit padding. And for kernel-internal structs, that's perfectly ok and even desirable since the kernel generally shouldn't make assumptions about the layouts of its structs, i.e. it's a good thing the compiler pads structs so that accesses are optimally aligned.
The padding behavior is only problematic for structs that are exposed to userspace, because if userspace pads differently then we've got problems. But even then, building the kernel with -Wpadded wouldn't prevent userspace from using a broken/goofy compiler that inserts unusual padding and misinterprets the intended layout.
AFAIK, the C standard only expicitly disallows padding at the beginning of a struct, i.e. the kernel's ABI is heavily reliant on existing compiler convention. The only way to ensure exact layouts without relying on compiler convention would be to tagged structs as packed, but "packed" also causes the compiler to generate sub-optimal code since "packed" has strict requirements, and so the kernel relies on sane compiler padding to provide a stable ABI.
PATCH 3/4 KVM: nVMX: Introduce __nested_vmx_handle_enlightened_vmptrld()
In reply to Vitaly KuznetsovAs a preparation to mapping eVMCS from vmx_set_nested_state() split the actual eVMCS mappign from aquiring eVMCS GPA.
No functional change intended.
--- arch/x86/kvm/vmx/nested.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-)
arch/x86/kvm/vmx/nested.c +17 -9
@@ -1972,18 +1972,11 @@ static int copy_vmcs12_to_enlightened(struct vcpu_vmx *vmx)* This is an equivalent of the nested hypervisor executing the vmptrld * instruction. */ -static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld ( -struct kvm_vcpu *vcpu,bool from_launch) +static enum nested_evmptrld_status __nested_vmx_handle_enlightened_vmptrld ( +struct kvm_vcpu *vcpu,u64 evmcs_gpa,bool from_launch) {struct vcpu_vmx *vmx =to_vmx (vcpu);bool evmcs_gpa_changed = false; -u64 evmcs_gpa; - -if (likely (!vmx->nested.enlightened_vmcs_enabled)) -return EVMPTRLD_DISABLED ; - -if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) -return EVMPTRLD_DISABLED ;if (unlikely (!vmx->nested.hv_evmcs || evmcs_gpa != vmx->nested.hv_evmcs_vmptr)) {
@@ -2055,6 +2048,21 @@ static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld(return EVMPTRLD_SUCCEEDED ; } +static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld ( +struct kvm_vcpu *vcpu,bool from_launch) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +u64 evmcs_gpa; + +if (likely (!vmx->nested.enlightened_vmcs_enabled)) +return EVMPTRLD_DISABLED ; + +if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) +return EVMPTRLD_DISABLED ; + +return __nested_vmx_handle_enlightened_vmptrld (vcpu, evmcs_gpa, from_launch); +} +void nested_sync_vmcs12_to_shadow (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
Signature · 1 line
2.30.2
Re: [PATCH 3/4] KVM: nVMX: Introduce __nested_vmx_handle_enlightened_vmptrld()
Quoted 47 lines · Vitaly Kuznetsov
As a preparation to mapping eVMCS from
vmx_set_nested_state()split the actual eVMCS mappign from aquiring eVMCS GPA.No functional change intended.
Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/kvm/vmx/nested.c | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-)arch/x86/kvm/vmx/nested.c +10
-9@@ -1972,18 +1972,11 @@ static int copy_vmcs12_to_enlightened(struct vcpu_vmx *vmx)* This is an equivalent of the nested hypervisor executing the vmptrld * instruction. */ -static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld ( -struct kvm_vcpu *vcpu,bool from_launch) +static enum nested_evmptrld_status __nested_vmx_handle_enlightened_vmptrld ( +struct kvm_vcpu *vcpu,u64 evmcs_gpa,bool from_launch) {struct vcpu_vmx *vmx =to_vmx (vcpu);bool evmcs_gpa_changed = false; -u64 evmcs_gpa; - -if (likely (!vmx->nested.enlightened_vmcs_enabled)) -return EVMPTRLD_DISABLED ; - -if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) -return EVMPTRLD_DISABLED ;if (unlikely (!vmx->nested.hv_evmcs || evmcs_gpa != vmx->nested.hv_evmcs_vmptr)) {@@ -2055,6 +2048,21 @@ static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld(return EVMPTRLD_SUCCEEDED ; } +static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld ( +struct kvm_vcpu *vcpu,bool from_launch) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +u64 evmcs_gpa; + +if (likely (!vmx->nested.enlightened_vmcs_enabled)) +return EVMPTRLD_DISABLED ;
arch/x86/kvm/vmx/nested.c static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld( · line 2068
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +7
-0+ +if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) +return EVMPTRLD_DISABLED ; + +return __nested_vmx_handle_enlightened_vmptrld (vcpu, evmcs_gpa, from_launch); +} +void nested_sync_vmcs12_to_shadow (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
Best regards,
Maxim Levitsky
PATCH 4/4 KVM: nVMX: Map enlightened VMCS upon restore when possible
In reply to Vitaly KuznetsovIt now looks like a bad idea to not restore eVMCS mapping directly from vmx_set_nested_state(). The restoration path now depends on whether KVM will continue executing L2 (vmx_get_nested_state_pages()) or will have to exit to L1 (nested_vmx_vmexit()), this complicates error propagation and diverges too much from the 'native' path when 'nested.current_vmptr' is set directly from vmx_get_nested_state_pages().
The existing solution postponing eVMCS mapping also seems to be fragile. In multiple places the code checks whether 'vmx->nested.hv_evmcs' is not NULL to distinguish between eVMCS and non-eVMCS cases. All these checks are 'incomplete' as we have a weird 'eVMCS is in use but not yet mapped' state.
Also, in case vmx_get_nested_state() is called right after vmx_set_nested_state() without executing the guest first, the resulting state is going to be incorrect as 'KVM_STATE_NESTED_EVMCS' flag will be missing.
Fix all these issues by making eVMCS restoration path closer to its 'native' sibling by putting eVMCS GPA to 'struct kvm_vmx_nested_state_hdr'. To avoid ABI incompatibility, do not introduce a new flag and keep the original eVMCS mapping path through KVM_REQ_GET_NESTED_STATE_PAGES in place. To distinguish between 'new' and 'old' formats consider eVMCS GPA == 0 as an unset GPA (thus forcing KVM_REQ_GET_NESTED_STATE_PAGES path). While technically possible, it seems to be an extremely unlikely case.
--- arch/x86/include/uapi/asm/kvm.h | 2 ++ arch/x86/kvm/vmx/nested.c | 27 +++++++++++++++++++++------ 2 files changed, 23 insertions(+), 6 deletions(-)
arch/x86/include/uapi/asm/kvm.h +2 -0
@@ -441,6 +441,8 @@ struct kvm_vmx_nested_state_hdr {__u32 flags;__u64 preemption_timer_deadline; + +__u64 evmcs_pa; };struct kvm_svm_nested_state_data {
arch/x86/kvm/vmx/nested.c +21 -6
@@ -6019,6 +6019,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu, .hdr.vmx.vmxon_pa = -1ull, .hdr.vmx.vmcs12_pa = -1ull, .hdr.vmx.preemption_timer_deadline = 0, + .hdr.vmx.evmcs_pa = -1ull, };struct kvm_vmx_nested_state_data __user *user_vmx_nested_state = &user_kvm_nested_state->data.vmx[0 ];
@@ -6037,8 +6038,10 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,if (vmx_has_valid_vmcs12 (vcpu)) { kvm_state.size +=sizeof (user_vmx_nested_state->vmcs12); -if (vmx->nested.hv_evmcs) +if (vmx->nested.hv_evmcs) { kvm_state.flags |=KVM_STATE_NESTED_EVMCS ; + kvm_state.hdr.vmx.evmcs_pa = vmx->nested.hv_evmcs_vmptr; + } if (is_guest_mode(vcpu) && nested_cpu_has_shadow_vmcs(vmcs12) &&
@@ -6230,13 +6233,25 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,set_current_vmptr (vmx, kvm_state->hdr.vmx.vmcs12_pa); }else if (kvm_state->flags &KVM_STATE_NESTED_EVMCS ) { +u64 evmcs_gpa = kvm_state->hdr.vmx.evmcs_pa; +/* -* nested_vmx_handle_enlightened_vmptrld() cannot be called -* directly from here as HV_X64_MSR_VP_ASSIST_PAGE may not be -* restored yet. EVMCS will be mapped from -* nested_get_vmcs12_pages(). +* EVMCS GPA == 0 most likely indicates that the migration data is +* coming from an older KVM which doesn't support 'evmcs_pa' in +* 'struct kvm_vmx_nested_state_hdr'. */ -kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (evmcs_gpa && (evmcs_gpa !=-1ull ) && + (__nested_vmx_handle_enlightened_vmptrld (vcpu, evmcs_gpa, false) != +EVMPTRLD_SUCCEEDED )) { +return -EINVAL ; + }else if (!evmcs_gpa) { +/* +* EVMCS GPA can't be acquired from VP assist page here because +* HV_X64_MSR_VP_ASSIST_PAGE may not be restored yet. +* EVMCS will be mapped from nested_get_evmcs_page(). +*/ +kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); + } }else {return -EINVAL ; }
Signature · 1 line
2.30.2
Paolo Bonzini
maintainerRe: [PATCH 4/4] KVM: nVMX: Map enlightened VMCS upon restore when possible
Quoted 13 lines · Vitaly Kuznetsov
It now looks like a bad idea to not restore eVMCS mapping directly from
vmx_set_nested_state(). The restoration path now depends on whether KVM will continue executing L2 (vmx_get_nested_state_pages()) or will have to exit to L1 (nested_vmx_vmexit()), this complicates error propagation and diverges too much from the 'native' path when 'nested.current_vmptr' is set directly fromvmx_get_nested_state_pages().The existing solution postponing eVMCS mapping also seems to be fragile. In multiple places the code checks whether '
vmx->nested.hv_evmcs' is not NULL to distinguish between eVMCS and non-eVMCS cases. All these checks are 'incomplete' as we have a weird 'eVMCS is in use but not yet mapped' state.
Quoted 8 lines
Also, in case
vmx_get_nested_state()is called right aftervmx_set_nested_state()without executing the guest first, the resulting state is going to be incorrect as 'KVM_STATE_NESTED_EVMCS' flag will be missing.Fix all these issues by making eVMCS restoration path closer to its 'native' sibling by putting eVMCS GPA to 'struct
kvm_vmx_nested_state_hdr'. To avoid ABI incompatibility, do not introduce a new flag and keep the
I'm not sure what is the disadvantage of not having a new flag.
Having two different paths with subtly different side effects however seems really worse for maintenance. We are already discussing in another thread how to get rid of the check_nested_events side effects; that might possibly even remove the need for patch 1, so it's at least worth pursuing more than adding this second path.
I have queued patch 1, but I'd rather have a kvm selftest for it. It doesn't seem impossible to have one...
Paolo
Quoted 8 lines
original eVMCS mapping path through
KVM_REQ_GET_NESTED_STATE_PAGESin place. To distinguish between 'new' and 'old' formats consider eVMCS GPA == 0 as an unset GPA (thus forcingKVM_REQ_GET_NESTED_STATE_PAGESpath). While technically possible, it seems to be an extremely unlikely case.Signed-off-by Vitaly Kuznetsov<[email protected]>
Re: [PATCH 4/4] KVM: nVMX: Map enlightened VMCS upon restore when possible
Quoted 19 lines · Paolo Bonzini
Quoted 18 lines · Vitaly Kuznetsov
It now looks like a bad idea to not restore eVMCS mapping directly from
vmx_set_nested_state(). The restoration path now depends on whether KVM will continue executing L2 (vmx_get_nested_state_pages()) or will have to exit to L1 (nested_vmx_vmexit()), this complicates error propagation and diverges too much from the 'native' path when 'nested.current_vmptr' is set directly fromvmx_get_nested_state_pages().The existing solution postponing eVMCS mapping also seems to be fragile. In multiple places the code checks whether '
vmx->nested.hv_evmcs' is not NULL to distinguish between eVMCS and non-eVMCS cases. All these checks are 'incomplete' as we have a weird 'eVMCS is in use but not yet mapped' state.Also, in case
vmx_get_nested_state()is called right aftervmx_set_nested_state()without executing the guest first, the resulting state is going to be incorrect as 'KVM_STATE_NESTED_EVMCS' flag will be missing.
Quoted 6 lines
Quoted 3 lines
Fix all these issues by making eVMCS restoration path closer to its 'native' sibling by putting eVMCS GPA to 'struct
kvm_vmx_nested_state_hdr'. To avoid ABI incompatibility, do not introduce a new flag and keep theI'm not sure what is the disadvantage of not having a new flag.
Adding a new flag would make us backwards-incompatible both ways:
1) Migrating 'new' state to an older KVM will fail the
if (kvm_state->hdr.vmx.flags & ~KVM_STATE_VMX_PREEMPTION_TIMER_DEADLINE) return -EINVAL;
check.
2) Migrating 'old' state to 'new' KVM would make us support the old path ('KVM_REQ_GET_NESTED_STATE_PAGES') so the flag will still be 'optional'.
Quoted 5 lines
Having two different paths with subtly different side effects however seems really worse for maintenance. We are already discussing in another thread how to get rid of the
check_nested_eventsside effects; that might possibly even remove the need for patch 1, so it's at least worth pursuing more than adding this second path.
I have to admit I don't fully like this solution either :-( In case we make sure KVM_REQ_GET_NESTED_STATE_PAGES always gets handled the fix can be omitted indeed, however, I still dislike the divergence and the fact that 'if (vmx->nested.hv_evmcs)' checks scattered across the code are not fully valid. E.g. how do we fix immediate KVM_GET_NESTED_STATE after KVM_SET_NESTED_STATE without executing the vCPU problem?
Quoted 3 lines
I have queued patch 1, but I'd rather have a kvm selftest for it. It doesn't seem impossible to have one...
Thank you, the band-aid solves a real problem. Let me try to come up with a selftest for it.
Quoted 12 lines
Paolo
Quoted 8 lines
original eVMCS mapping path through
KVM_REQ_GET_NESTED_STATE_PAGESin place. To distinguish between 'new' and 'old' formats consider eVMCS GPA == 0 as an unset GPA (thus forcingKVM_REQ_GET_NESTED_STATE_PAGESpath). While technically possible, it seems to be an extremely unlikely case.Signed-off-by Vitaly Kuznetsov<[email protected]>
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerRe: [PATCH 4/4] KVM: nVMX: Map enlightened VMCS upon restore when possible
Quoted 4 lines · Vitaly Kuznetsov
I still dislike the divergence and the fact
that 'if (vmx->nested.hv_evmcs)' checks scattered across the code are not fully valid. E.g. how do we fix immediateKVM_GET_NESTED_STATEafterKVM_SET_NESTED_STATEwithout executing the vCPU problem?
You obviously have thought about this more than I did, but if you can write a testcase for that as well, I can take a look.
Thanks,
Paolo
Re: [PATCH 4/4] KVM: nVMX: Map enlightened VMCS upon restore when possible
In reply to Vitaly KuznetsovQuoted 91 lines · Vitaly Kuznetsov
It now looks like a bad idea to not restore eVMCS mapping directly from
vmx_set_nested_state(). The restoration path now depends on whether KVM will continue executing L2 (vmx_get_nested_state_pages()) or will have to exit to L1 (nested_vmx_vmexit()), this complicates error propagation and diverges too much from the 'native' path when 'nested.current_vmptr' is set directly fromvmx_get_nested_state_pages().The existing solution postponing eVMCS mapping also seems to be fragile. In multiple places the code checks whether '
vmx->nested.hv_evmcs' is not NULL to distinguish between eVMCS and non-eVMCS cases. All these checks are 'incomplete' as we have a weird 'eVMCS is in use but not yet mapped' state.Also, in case
vmx_get_nested_state()is called right aftervmx_set_nested_state()without executing the guest first, the resulting state is going to be incorrect as 'KVM_STATE_NESTED_EVMCS' flag will be missing.Fix all these issues by making eVMCS restoration path closer to its 'native' sibling by putting eVMCS GPA to 'struct
kvm_vmx_nested_state_hdr'. To avoid ABI incompatibility, do not introduce a new flag and keep the original eVMCS mapping path throughKVM_REQ_GET_NESTED_STATE_PAGESin place. To distinguish between 'new' and 'old' formats consider eVMCS GPA == 0 as an unset GPA (thus forcingKVM_REQ_GET_NESTED_STATE_PAGESpath). While technically possible, it seems to be an extremely unlikely case.Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 2 ++ arch/x86/kvm/vmx/nested.c | 27 +++++++++++++++++++++------ 2 files changed, 23 insertions(+), 6 deletions(-)arch/x86/include/uapi/asm/kvm.h +2
-0@@ -441,6 +441,8 @@ struct kvm_vmx_nested_state_hdr {__u32 flags;__u64 preemption_timer_deadline; + +__u64 evmcs_pa; };struct kvm_svm_nested_state_data {arch/x86/kvm/vmx/nested.c +14
-6@@ -6019,6 +6019,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu, .hdr.vmx.vmxon_pa = -1ull, .hdr.vmx.vmcs12_pa = -1ull, .hdr.vmx.preemption_timer_deadline = 0, + .hdr.vmx.evmcs_pa = -1ull, };struct kvm_vmx_nested_state_data __user *user_vmx_nested_state = &user_kvm_nested_state->data.vmx[0 ];@@ -6037,8 +6038,10 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,if (vmx_has_valid_vmcs12 (vcpu)) { kvm_state.size +=sizeof (user_vmx_nested_state->vmcs12); -if (vmx->nested.hv_evmcs) +if (vmx->nested.hv_evmcs) { kvm_state.flags |=KVM_STATE_NESTED_EVMCS ; + kvm_state.hdr.vmx.evmcs_pa = vmx->nested.hv_evmcs_vmptr; + } if (is_guest_mode(vcpu) && nested_cpu_has_shadow_vmcs(vmcs12) &&@@ -6230,13 +6233,25 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,set_current_vmptr (vmx, kvm_state->hdr.vmx.vmcs12_pa); }else if (kvm_state->flags &KVM_STATE_NESTED_EVMCS ) { +u64 evmcs_gpa = kvm_state->hdr.vmx.evmcs_pa; +/* -* nested_vmx_handle_enlightened_vmptrld() cannot be called -* directly from here as HV_X64_MSR_VP_ASSIST_PAGE may not be -* restored yet. EVMCS will be mapped from -* nested_get_vmcs12_pages(). +* EVMCS GPA == 0 most likely indicates that the migration data is +* coming from an older KVM which doesn't support 'evmcs_pa' in +* 'struct kvm_vmx_nested_state_hdr'. */ -kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (evmcs_gpa && (evmcs_gpa !=-1ull ) && + (__nested_vmx_handle_enlightened_vmptrld (vcpu, evmcs_gpa, false) != +EVMPTRLD_SUCCEEDED )) { +return -EINVAL ; + }else if (!evmcs_gpa) {
arch/x86/kvm/vmx/nested.c static int vmx_set_nested_state(struct kvm_vcpu *vcpu, · line 6257
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +7
-0+/* +* EVMCS GPA can't be acquired from VP assist page here because +* HV_X64_MSR_VP_ASSIST_PAGE may not be restored yet. +* EVMCS will be mapped from nested_get_evmcs_page(). +*/ +kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); + } }else {return -EINVAL ; }
Hi everyone!
Let me expalin my concern about this patch and also ask if I understand this correctly.
In a nutshell if I understand this correctly, we are not allowed to access any guest memory while setting the nested state.
Now, if I understand correctly as well, the reason for the above,
is that the userspace is allowed to set the nested state first, then fiddle with the KVM memslots, maybe even update the guest memory and only later do the KVM_RUN ioctl,
And so this is the major reason why the KVM_REQ_GET_NESTED_STATE_PAGES request exists in the first place.
If that is correct I assume that we either have to keep loading the EVMCS page on KVM_REQ_GET_NESTED_STATE_PAGES request, or we want to include the EVMCS itself in the migration state in addition to its physical address, similar to how we treat the VMCS12 and the VMCB12.
I personally tinkered with qemu to try and reproduce this situation and in my tests I wasn't able to make it update the memory map after the load of the nested state but prior to KVM_RUN but neither I wasn't able to prove that this can't happen.
In addition to that I don't know how qemu behaves when it does guest ram post-copy because so far I haven't tried to tinker with it.
Finally other userspace hypervisors exist, and they might rely on assumption as well.
Looking forward for any comments,
Best regards,
Maxim Levitsky
Re: [PATCH 4/4] KVM: nVMX: Map enlightened VMCS upon restore when possible
Quoted 105 lines · Maxim Levitsky
Quoted 91 lines · Vitaly Kuznetsov
It now looks like a bad idea to not restore eVMCS mapping directly from
vmx_set_nested_state(). The restoration path now depends on whether KVM will continue executing L2 (vmx_get_nested_state_pages()) or will have to exit to L1 (nested_vmx_vmexit()), this complicates error propagation and diverges too much from the 'native' path when 'nested.current_vmptr' is set directly fromvmx_get_nested_state_pages().The existing solution postponing eVMCS mapping also seems to be fragile. In multiple places the code checks whether '
vmx->nested.hv_evmcs' is not NULL to distinguish between eVMCS and non-eVMCS cases. All these checks are 'incomplete' as we have a weird 'eVMCS is in use but not yet mapped' state.Also, in case
vmx_get_nested_state()is called right aftervmx_set_nested_state()without executing the guest first, the resulting state is going to be incorrect as 'KVM_STATE_NESTED_EVMCS' flag will be missing.Fix all these issues by making eVMCS restoration path closer to its 'native' sibling by putting eVMCS GPA to 'struct
kvm_vmx_nested_state_hdr'. To avoid ABI incompatibility, do not introduce a new flag and keep the original eVMCS mapping path throughKVM_REQ_GET_NESTED_STATE_PAGESin place. To distinguish between 'new' and 'old' formats consider eVMCS GPA == 0 as an unset GPA (thus forcingKVM_REQ_GET_NESTED_STATE_PAGESpath). While technically possible, it seems to be an extremely unlikely case.Signed-off-by Vitaly Kuznetsov <[email protected]>--- arch/x86/include/uapi/asm/kvm.h | 2 ++ arch/x86/kvm/vmx/nested.c | 27 +++++++++++++++++++++------ 2 files changed, 23 insertions(+), 6 deletions(-)arch/x86/include/uapi/asm/kvm.h +2
-0@@ -441,6 +441,8 @@ struct kvm_vmx_nested_state_hdr {__u32 flags;__u64 preemption_timer_deadline; + +__u64 evmcs_pa; };struct kvm_svm_nested_state_data {arch/x86/kvm/vmx/nested.c +14
-6@@ -6019,6 +6019,7 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu, .hdr.vmx.vmxon_pa = -1ull, .hdr.vmx.vmcs12_pa = -1ull, .hdr.vmx.preemption_timer_deadline = 0, + .hdr.vmx.evmcs_pa = -1ull, };struct kvm_vmx_nested_state_data __user *user_vmx_nested_state = &user_kvm_nested_state->data.vmx[0 ];@@ -6037,8 +6038,10 @@ static int vmx_get_nested_state(struct kvm_vcpu *vcpu,if (vmx_has_valid_vmcs12 (vcpu)) { kvm_state.size +=sizeof (user_vmx_nested_state->vmcs12); -if (vmx->nested.hv_evmcs) +if (vmx->nested.hv_evmcs) { kvm_state.flags |=KVM_STATE_NESTED_EVMCS ; + kvm_state.hdr.vmx.evmcs_pa = vmx->nested.hv_evmcs_vmptr; + } if (is_guest_mode(vcpu) && nested_cpu_has_shadow_vmcs(vmcs12) &&@@ -6230,13 +6233,25 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu,set_current_vmptr (vmx, kvm_state->hdr.vmx.vmcs12_pa); }else if (kvm_state->flags &KVM_STATE_NESTED_EVMCS ) { +u64 evmcs_gpa = kvm_state->hdr.vmx.evmcs_pa; +/* -* nested_vmx_handle_enlightened_vmptrld() cannot be called -* directly from here as HV_X64_MSR_VP_ASSIST_PAGE may not be -* restored yet. EVMCS will be mapped from -* nested_get_vmcs12_pages(). +* EVMCS GPA == 0 most likely indicates that the migration data is +* coming from an older KVM which doesn't support 'evmcs_pa' in +* 'struct kvm_vmx_nested_state_hdr'. */ -kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); +if (evmcs_gpa && (evmcs_gpa !=-1ull ) && + (__nested_vmx_handle_enlightened_vmptrld (vcpu, evmcs_gpa, false) != +EVMPTRLD_SUCCEEDED )) { +return -EINVAL ; + }else if (!evmcs_gpa) {arch/x86/kvm/vmx/nested.c static int vmx_set_nested_state(struct kvm_vcpu *vcpu, · line 6257
Quoted 10 lines
arch/x86/kvm/vmx/nested.c +7
-0+/* +* EVMCS GPA can't be acquired from VP assist page here because +* HV_X64_MSR_VP_ASSIST_PAGE may not be restored yet. +* EVMCS will be mapped from nested_get_evmcs_page(). +*/ +kvm_make_request (KVM_REQ_GET_NESTED_STATE_PAGES , vcpu); + } }else {return -EINVAL ; }Hi everyone!
Quoted 8 lines
Let me expalin my concern about this patch and also ask if I understand this correctly.
In a nutshell if I understand this correctly, we are not allowed to access any guest memory while setting the nested state.
Now, if I understand correctly as well, the reason for the above,
is that the userspace is allowed to set the nested state first, then fiddle with the KVM memslots, maybe even update the guest memory and only later do theKVM_RUNioctl,
Currently, userspace is free to restore the guest in any order indeed. I've probably missed post-copy but even the fact that guest MSRs can be restored after restoring nested state doesn't make our life easier.
Quoted 8 lines
And so this is the major reason why the
KVM_REQ_GET_NESTED_STATE_PAGESrequest exists in the first place.If that is correct I assume that we either have to keep loading the EVMCS page on
KVM_REQ_GET_NESTED_STATE_PAGESrequest, or we want to include the EVMCS itself in the migration state in addition to its physical address, similar to how we treat the VMCS12 and the VMCB12.
Keeping eVMCS load from KVM_REQ_GET_NESTED_STATE_PAGES is OK I believe (or at least I still don't see a reason for us to carry a copy in the migration data). What I still don't like is the transient state after vmx_set_nested_state():
- vmx->nested.current_vmptr is -1ull because no 'real' vmptrld was done (we skip set_current_vmptr() when KVM_STATE_NESTED_EVMCS)
- vmx->nested.hv_evmcs/vmx->nested.hv_evmcs_vmptr are also NULL because we haven't performed nested_vmx_handle_enlightened_vmptrld() yet.
I know of at least one real problem with this state: in case vmx_get_nested_state() happens before KVM_RUN the resulting state won't have KVM_STATE_NESTED_EVMCS flag and this is incorrect. Take a look at the check in nested_vmx_fail() for example:
if (vmx->nested.current_vmptr == -1ull && !vmx->nested.hv_evmcs)
return nested_vmx_failInvalid(vcpu);this also seems off (I'm not sure it matters in any context but still).
Quoted 5 lines
I personally tinkered with qemu to try and reproduce this situation and in my tests I wasn't able to make it update the memory map after the load of the nested state but prior to
KVM_RUNbut neither I wasn't able to prove that this can't happen.
Userspace has multiple ways to mess with the state of course, in KVM we only need to make sure we don't crash :-) On migration, well behaving userspace is supposed to restore exactly what it got though. The restoration sequence may vary.
Quoted 13 lines
In addition to that I don't know how qemu behaves when it does guest ram post-copy because so far I haven't tried to tinker with it.
Finally other userspace hypervisors exist, and they might rely on assumption as well.
Looking forward for any comments,
Best regards,
Maxim Levitsky
Signature · 1 line
Vitaly
Paolo Bonzini
maintainerQuoted 4 lines · Vitaly Kuznetsov
Win10 guests with WSL2 enabled sometimes crash on migration when enlightened VMCS was used. The condition seems to be induced by the situation when
L2->L1exit is caused immediately after migration and before L2 gets a chance to run (e.g. when there's an interrupt pending).
Interesting, I think it gets to nested_vmx_vmexit before
if (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu)) {
if (unlikely(!kvm_x86_ops.nested_ops->get_nested_state_pages(vcpu))) {
r = 0;
goto out;
}
}due to the infamous calls to check_nested_events that are scattered through KVM?
Paolo
Quoted 15 lines · Paolo Bonzini
Quoted 4 lines · Vitaly Kuznetsov
Win10 guests with WSL2 enabled sometimes crash on migration when enlightened VMCS was used. The condition seems to be induced by the situation when
L2->L1exit is caused immediately after migration and before L2 gets a chance to run (e.g. when there's an interrupt pending).Interesting, I think it gets to
nested_vmx_vmexitbeforeif (kvm_check_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu)) { if (unlikely(!kvm_x86_ops.nested_ops->get_nested_state_pages(vcpu))) { r = 0; goto out; } }
Quoted 2 lines
due to the infamous calls to
check_nested_eventsthat are scattered through KVM?
Yea,
vcpu_run() -> kvm_vcpu_running() -> vmx_check_nested_events() if I remember it correctly.
Signature · 1 line
Vitaly