KVM: SVM: GIF and EFER.SVME are independent
v2 cover KVM: SVM: GIF and EFER.SVME are independent
Clearing EFER.SVME is not architected to set GIF, so GIF may be clear even when EFER.SVME is clear.
This is covered in the discussion at
https://lore.kernel.org/all/[email protected]/.
v2:
- Allow KVM_SET_NESTED_STATE to clear GIF when SVME==0
v1: https://lore.kernel.org/kvm/[email protected]/
Jim Mattson (2):
KVM: SVM: Allow KVM_SET_NESTED_STATE to clear GIF when SVME==0
KVM: SVM: Don't set GIF when clearing EFER.SVME
arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/svm.c | 1 - 2 files changed, 2 insertions(+), 3 deletions(-)
Signature · 1 line
2.51.0.740.g6adb054d12-goog
PATCH v2 1/2 KVM: SVM: Allow KVM_SET_NESTED_STATE to clear GIF when SVME==0
GIF==0 together with EFER.SVME==0 is a valid architectural state. Don't return -EINVAL for KVM_SET_NESTED_STATE when this combination is specified.
--- arch/x86/kvm/svm/nested.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/nested.c +2 -2
@@ -1798,8 +1798,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,* EFER.SVME, but EFER.SVME still has to be 1 for VMRUN to succeed. */ if (!(vcpu->arch.efer &EFER_SVME )) { -/* GIF=1 and no guest mode are required if SVME=0. */ -if (kvm_state->flags !=KVM_STATE_NESTED_GIF_SET ) +/* GUEST_MODE must be clear when SVME==0 */ +if (kvm_state->flags &KVM_STATE_NESTED_GUEST_MODE )return -EINVAL ; }
Signature · 1 line
2.51.0.740.g6adb054d12-goog
Re: [PATCH v2 1/2] KVM: SVM: Allow KVM_SET_NESTED_STATE to clear GIF when SVME==0
Quoted 6 lines · Jim Mattson
GIF==0 together with EFER.SVME==0 is a valid architectural state. Don't return
-EINVALforKVM_SET_NESTED_STATEwhen this combination is specified.Fixes cc440cdad5b7 ("KVM: nSVM: implement KVM_GET_NESTED_STATE and KVM_SET_NESTED_STATE")Signed-off-by Jim Mattson <[email protected]>
Quoted 22 lines
--- arch/x86/kvm/svm/nested.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/svm/nested.c +2
-2@@ -1798,8 +1798,8 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu,* EFER.SVME, but EFER.SVME still has to be 1 for VMRUN to succeed. */ if (!(vcpu->arch.efer &EFER_SVME )) { -/* GIF=1 and no guest mode are required if SVME=0. */ -if (kvm_state->flags !=KVM_STATE_NESTED_GIF_SET ) +/* GUEST_MODE must be clear when SVME==0 */ +if (kvm_state->flags &KVM_STATE_NESTED_GUEST_MODE )return -EINVAL ; }Signature · 1 line
2.51.0.740.g6adb054d12-goog
PATCH v2 2/2 KVM: SVM: Don't set GIF when clearing EFER.SVME
In reply to Jim MattsonClearing EFER.SVME is not architected to set GIF. Don't set GIF when emulating a change to EFER that clears EFER.SVME.
--- arch/x86/kvm/svm/svm.c | 1 - 1 file changed, 1 deletion(-)
arch/x86/kvm/svm/svm.c +0 -1
@@ -215,7 +215,6 @@ int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)if ((old_efer &EFER_SVME ) != (efer &EFER_SVME )) {if (!(efer &EFER_SVME )) {svm_leave_nested (vcpu); -svm_set_gif (svm, true);/* #GP intercept is still needed for vmware backdoor */ if (!enable_vmware_backdoor)clr_exception_intercept (svm,GP_VECTOR );
Signature · 1 line
2.51.0.740.g6adb054d12-goog
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.
arch/x86/kvm/svm/nested.c +2 -0
@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb); +svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }
Quoted 24 lines
Don't set GIF when emulating a change to EFER that clears EFER.SVME.
Fixes c513f484c558 ("KVM: nSVM: leave guest mode when clearing EFER.SVME")Reviewed-by Yosry Ahmed <[email protected]>Signed-off-by Jim Mattson <[email protected]>--- arch/x86/kvm/svm/svm.c | 1 - 1 file changed, 1 deletion(-)arch/x86/kvm/svm/svm.c +0
-1@@ -215,7 +215,6 @@ int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)if ((old_efer &EFER_SVME ) != (efer &EFER_SVME )) {if (!(efer &EFER_SVME )) {svm_leave_nested (vcpu); -svm_set_gif (svm, true);/* #GP intercept is still needed for vmware backdoor */ if (!enable_vmware_backdoor)clr_exception_intercept (svm,GP_VECTOR );Signature · 1 line
2.51.0.740.g6adb054d12-goog
Re: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 21 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0
arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 8 lines
arch/x86/kvm/svm/nested.c +2
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb); +svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }
This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 24 lines · Jim Mattson
Quoted 23 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb);
Quoted 9 lines
arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 6 lines
arch/x86/kvm/svm/nested.c +2
-0+svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Regardless of what KVM does, we're defining hardware behavior, i.e. keeping GIF unchanged defines behavior just as much as setting GIF. The only way to truly avoid defining behavior would be to terminate the VM and completely prevent userspace from accessing its state.
Re: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 36 lines · Sean Christopherson
Quoted 24 lines · Jim Mattson
Quoted 23 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb);Quoted 9 lines
arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 6 lines
arch/x86/kvm/svm/nested.c +2
-0+svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Quoted 4 lines
Regardless of what KVM does, we're defining hardware behavior, i.e. keeping GIF unchanged defines behavior just as much as setting GIF. The only way to truly avoid defining behavior would be to terminate the VM and completely prevent userspace from accessing its state.
This can't be the only instance of "undefined behavior" that KVM deals with. What about, say, misaligned accesses to xAPIC memory?
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 37 lines · Jim Mattson
Quoted 36 lines · Sean Christopherson
Quoted 33 lines · Jim Mattson
Quoted 21 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 8 lines
arch/x86/kvm/svm/nested.c +2
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb); +svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Quoted 7 lines
Quoted 4 lines
Regardless of what KVM does, we're defining hardware behavior, i.e. keeping GIF unchanged defines behavior just as much as setting GIF. The only way to truly avoid defining behavior would be to terminate the VM and completely prevent userspace from accessing its state.
This can't be the only instance of "undefined behavior" that KVM deals with.
Oh, for sure. But unsurprisingly, people only care about cases that actually matter in practice. E.g. the other one that comes to mind is SHUTDOWN on AMD:
/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
* KVM_MP_STATE_SHUTDOWN and can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
Quoted 1 line
What about, say, misaligned accesses to xAPIC memory?
Drops all accesses (doesn't even set the destination on reads).
Re: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 50 lines · Sean Christopherson
Quoted 37 lines · Jim Mattson
Quoted 36 lines · Sean Christopherson
Quoted 33 lines · Jim Mattson
Quoted 21 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 8 lines
arch/x86/kvm/svm/nested.c +2
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb); +svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Quoted 7 lines
Quoted 4 lines
Regardless of what KVM does, we're defining hardware behavior, i.e. keeping GIF unchanged defines behavior just as much as setting GIF. The only way to truly avoid defining behavior would be to terminate the VM and completely prevent userspace from accessing its state.
This can't be the only instance of "undefined behavior" that KVM deals with.
Oh, for sure. But unsurprisingly, people only care about cases that actually matter in practice. E.g. the other one that comes to mind is SHUTDOWN on AMD:
Quoted 8 lines
/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
*KVM_MP_STATE_SHUTDOWNand can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
The behavior of SHUTDOWN while GIF==0 is clearly architected:
"If the processor enters the shutdown state (due to a triple fault for instance) while GIF is clear, it can only be restarted by means of a RESET."
Doesn't setting GIF in svm_leave_nested() violate this specification?
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 60 lines · Jim Mattson
Quoted 50 lines · Sean Christopherson
Quoted 37 lines · Jim Mattson
Quoted 36 lines · Sean Christopherson
Quoted 33 lines · Jim Mattson
Quoted 21 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 8 lines
arch/x86/kvm/svm/nested.c +2
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb); +svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Quoted 7 lines
Quoted 4 lines
Regardless of what KVM does, we're defining hardware behavior, i.e. keeping GIF unchanged defines behavior just as much as setting GIF. The only way to truly avoid defining behavior would be to terminate the VM and completely prevent userspace from accessing its state.
This can't be the only instance of "undefined behavior" that KVM deals with.
Oh, for sure. But unsurprisingly, people only care about cases that actually matter in practice. E.g. the other one that comes to mind is SHUTDOWN on AMD:
Quoted 8 lines
/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
*KVM_MP_STATE_SHUTDOWNand can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*
Quoted 7 lines
The behavior of SHUTDOWN while GIF==0 is clearly architected:
"If the processor enters the shutdown state (due to a triple fault for instance) while GIF is clear, it can only be restarted by means of a RESET."
Doesn't setting GIF in
svm_leave_nested()violate this specification?
Probably? But SHUTDOWN also makes the VMCB undefined, so KVM is caught between a rock and a hard place. And when using vGIF, I don't see how KVM can do the right thing, because the state of GIF at the time of SHUTDOWN is unknown.
And FWIW, if userspace does RESET the guest (which KVM can't detect with 100% accuracy), GIF=1 on RESET, so it's kinda sorta right :-)
Re: [PATCH v2 2/2] KVM: SVM: Don't set GIF when clearing EFER.SVME
Quoted 67 lines · Sean Christopherson
Quoted 66 lines · Jim Mattson
Quoted 50 lines · Sean Christopherson
Quoted 37 lines · Jim Mattson
Quoted 36 lines · Sean Christopherson
Quoted 33 lines · Jim Mattson
Quoted 21 lines · Sean Christopherson
Quoted 1 line · Jim Mattson
Clearing EFER.SVME is not architected to set GIF.
But it's also not architected to leave GIF set when the guest is running, which was the basic gist of the Fixes commit. I suspect that forcing GIF=1 was intentional, e.g. so that the guest doesn't end up with GIF=0 after stuffing the vCPU into SMM mode, which might actually be invalid.
I think what we actually want is to to set GIF when force-leaving nested. The only path where it's not obvious that's "safe" is toggling SMM in
kvm_vcpu_ioctl_x86_set_vcpu_events(). In every other path, setting GIF is either correct/desirable, or irrelevant because the caller immediately and unconditionally sets/clears GIF.arch/x86/kvm/svm/nested.c +0
-0arch/x86/kvm/svm/nested.c void svm_leave_nested(struct kvm_vcpu *vcpu) · line 1374
Quoted 8 lines
arch/x86/kvm/svm/nested.c +2
-0@@ -1367,6 +1367,8 @@ void svm_leave_nested(struct kvm_vcpu *vcpu)nested_svm_uninit_mmu_context (vcpu);vmcb_mark_all_dirty (svm->vmcb); +svm_set_gif (svm, true); +if (kvm_apicv_activated (vcpu->kvm))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); }This seems dangerously close to KVM making up "hardware" behavior, but I'm okay with that if you are.
Quoted 7 lines
Quoted 4 lines
Regardless of what KVM does, we're defining hardware behavior, i.e. keeping GIF unchanged defines behavior just as much as setting GIF. The only way to truly avoid defining behavior would be to terminate the VM and completely prevent userspace from accessing its state.
This can't be the only instance of "undefined behavior" that KVM deals with.
Oh, for sure. But unsurprisingly, people only care about cases that actually matter in practice. E.g. the other one that comes to mind is SHUTDOWN on AMD:
Quoted 8 lines
/*
* VMCB is undefined after a SHUTDOWN intercept. INIT the vCPU to put
* the VMCB in a known good state. Unfortuately, KVM doesn't have
*KVM_MP_STATE_SHUTDOWNand can't add it without potentially breaking
* userspace. At a platform view, INIT is acceptable behavior as
* there exist bare metal platforms that automatically INIT the CPU
* in response to shutdown.
*The behavior of SHUTDOWN while GIF==0 is clearly architected:
"If the processor enters the shutdown state (due to a triple fault for instance) while GIF is clear, it can only be restarted by means of a RESET."
Quoted 8 lines
Quoted 1 line
Doesn't setting GIF in
svm_leave_nested()violate this specification?Probably? But SHUTDOWN also makes the VMCB undefined, so KVM is caught between a rock and a hard place. And when using vGIF, I don't see how KVM can do the right thing, because the state of GIF at the time of SHUTDOWN is unknown.
And FWIW, if userspace does RESET the guest (which KVM can't detect with 100% accuracy), GIF=1 on RESET, so it's kinda sorta right :-)
This thread kinda ended on a cliffhanger.
Sean, is the plan to apply Jim's patch + your diff to svm_set_gif() in svm_leave_nested()? Or are you waiting for Jim to send a v2?