KVM: x86: More EFER.LMA vs. CR4.PAE fixes
Sean Christopherson
maintainerv2 cover KVM: x86: More EFER.LMA vs. CR4.PAE fixes
Revert a change I made years ago to allow userspace to stuff EFER before configuring guest CPUID, as configuring CPUID and feature MSRs before anything else is now a hard requirement (though I suspect I was just a bit overzealous at the time). The biggest problem with that change, other than creating what should be impossible state, is that it also bypassed this check:
if (is_paging(vcpu) && (vcpu->arch.efer & EFER_LME) != (efer & EFER_LME)) return 1;
which was kinda sorta intentional? I assume the idea was that userspace could stuff CR0 vs. EFER in any order. Except that in doing so, userspace could get impossible state in CR0+CR4+EFER (in the aggregate, not in any one individual register). KVM doesn't advertise MSR_EFER as an MSR that needs to be saved/restored, and AFAICT no VMM manually does that, they all just use KVM_{G,S}ET_SREGS{,2} (which is what we want).
So, hit the "I'm feeling lucky!" button and simply revert the misguided commit.
Patch 2 plugs a hole where userspace can get at the "wants_to_run" logic in svm_set_efer() via KVM_SYNC_X86_SREGS, which is "fine" for now, but would let a misbehaving userspace trigger the KVM_BUG_ON() added in patch 3.
Patch 3 is more hardening. Rather than pend a triple fault, which is flawed because (a) KVM still yanks the vCPU out of L2 without going through a normal nested VM-Exit and (b) there's a window where KVM can do stuff, e.g. init the MMU, before the triple fault request is handled, immediately synthesize a #GP. The only reason I didn't go with a #GP the first time around is that I didn't want to bleed EFER_SVME information into the common set_efer(), KVM effectively doesn't allow kvm_x86_call(set_efer)() to inject faults, and the vendor version is used for many flows, whereas set_efer() is strictly used for WRMSR emulation.
Patch 4 is also more hardening, to try and detect+prevent EFER.LMA from going stale.
Finally, update the EFER selftest to match the new #GP behavior.
v2:
- Disallowing clearing SVME via KVM_SYNC_X86_SREGS if L2 is active. [Sashiko]
v1: https://lore.kernel.org/all/[email protected]
Sean Christopherson (6):
Revert "KVM: x86: Skip EFER vs. guest CPUID checks for host-initiated
writes"
KVM: x86: Reject clearing EFER.SVME via KVM_SYNC_X86_SREGS if L2 is
active
KVM: nSVM: Synthesize #GP instead of SHUTDOWN if L2 attempts to clear
EFER.SVME
KVM: x86: Recompute EFER.LMA instead of inheriting the current value
on EFER writes
KVM: selftests: Update nSVM EFER test to expect #GP when clearing SVME
in L2
KVM: selftests: Expect failure when trying to set EFER that violates
vCPU model
arch/x86/kvm/msrs.c | 53 ++++++----- arch/x86/kvm/regs.c | 4 + arch/x86/kvm/svm/svm.c | 15 +-- .../selftests/kvm/x86/svm_nested_efer_test.c | 95 ++++++++++--------- 4 files changed, 92 insertions(+), 75 deletions(-)
base-commit: 7e60162b16f9da0d2fa83da5c1723b6853a0ac1d
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Sean Christopherson
maintainerPATCH v2 1/6 Revert "KVM: x86: Skip EFER vs. guest CPUID checks for host-initiated writes"
Revert the misguided-in-hindsight decision to allow userspace to violate architectural consistency when setting EFER as it has since been solidified that userspace is required to configure guest CPUID prior to setting MSRs, and skipping the CR0.PG vs. EFER.LME check in allows userspace to load an end state of EFER.LMA=1 && EFER.LME=0, which is supposed to be impossible and trips a variety of KVM_BUG_ON()s in the MMU.
This reverts commit 11988499e62b310f3bf6f6d0a807a06d3f9ccc96.
--- arch/x86/kvm/msrs.c | 30 +++++++++--------------------- 1 file changed, 9 insertions(+), 21 deletions(-)
arch/x86/kvm/msrs.c +9 -21
@@ -580,8 +580,11 @@ static int do_get_feature_msr(struct kvm_vcpu *vcpu, unsigned index, u64 *data) kvm_get_feature_msr); } -static bool __kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer) +bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer) { +if (efer & ~kvm_caps.supported_efer_bits) +return false; +if (efer &EFER_AUTOIBRS && !guest_cpu_cap_has (vcpu,X86_FEATURE_AUTOIBRS ))return false;
@@ -610,33 +613,18 @@ static bool __kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer)return true; } - -bool kvm_valid_efer (struct kvm_vcpu *vcpu,u64 efer) -{ -if (efer & ~kvm_caps.supported_efer_bits) -return false; - -return __kvm_valid_efer (vcpu, efer); -}EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_valid_efer); -static int set_efer (struct kvm_vcpu *vcpu,struct msr_data *msr_info) +static int set_efer (struct kvm_vcpu *vcpu,u64 efer) {u64 old_efer = vcpu->arch.efer; -u64 efer = msr_info->data;int r; -if (efer & ~kvm_caps.supported_efer_bits) +if (!kvm_valid_efer (vcpu, efer))return 1 ; -if (!msr_info->host_initiated) { -if (!__kvm_valid_efer (vcpu, efer)) -return 1 ; - -if (is_paging (vcpu) && - (vcpu->arch.efer &EFER_LME ) != (efer &EFER_LME )) -return 1 ; - } +if (is_paging (vcpu) && (vcpu->arch.efer &EFER_LME ) != (efer &EFER_LME )) +return 1 ; efer &= ~EFER_LMA ; efer |= vcpu->arch.efer &EFER_LMA ;
@@ -1547,7 +1535,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info)wrmsrq (MSR_IA32_FLUSH_CMD ,L1D_FLUSH );break ;case MSR_EFER : -return set_efer (vcpu, msr_info); +return set_efer (vcpu, msr_info->data);case MSR_K7_HWCR : {/* * Allow McStatusWrEn and TscFreqSel. (Linux guests from v3.2
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Sean Christopherson
maintainerPATCH v2 2/6 KVM: x86: Reject clearing EFER.SVME via KVM_SYNC_X86_SREGS if L2 is active
In reply to Sean ChristophersonDisallow clearing EFER.SVME via KVM_SYNC_X86_SREGS if L2 is active so that svm_set_efer() can treat yanking the vCPU out of L2 during KVM_RUN as a KVM bug. Ideally, kvm_is_valid_sregs() itself would reject attempts to clear SVME while L2 is active, but the risk of breaking userspace, e.g. across save/restore, is too high. On the other hand, yanking the vCPU out of L2 during save/restore is already "broken" as doing so would queue a triple fault for L1.
--- arch/x86/kvm/regs.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/x86/kvm/regs.c +4 -0
@@ -738,6 +738,10 @@ int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu)if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_SREGS ) {struct kvm_sregs sregs = vcpu->run->s.regs.sregs; +if (is_guest_mode (vcpu) && + (vcpu->arch.efer &EFER_SVME ) && !(sregs.efer &EFER_SVME )) +return -EINVAL ; +if (__set_sregs (vcpu, &sregs))return -EINVAL ;
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Re: [PATCH v2 2/6] KVM: x86: Reject clearing EFER.SVME via KVM_SYNC_X86_SREGS if L2 is active
Quoted 6 lines · Sean Christopherson
Disallow clearing EFER.SVME via
KVM_SYNC_X86_SREGSif L2 is active so thatsvm_set_efer()can treat yanking the vCPU out of L2 duringKVM_RUNas a KVM bug. Ideally,kvm_is_valid_sregs()itself would reject attempts to clear SVME while L2 is active, but the risk of breaking userspace, e.g. across save/restore, is too high.
I am confused. This says we shouldn't disallow clearing SVME while L2 is active because we may break userspace..
Quoted 3 lines
On the other hand, yanking the vCPU out of L2
during save/restore is already "broken" as doing so would queue a triple fault for L1.
..and this says it's already broken.
I suspect you in the first paragraph were referring to userspace restoring a vCPU not in guest mode on top of a vCPU in guest mode or similar? Might be worth clarifying what exactly we don't wanna break.
Quoted 12 lines
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/regs.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/regs.c +0
-0
arch/x86/kvm/regs.c int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu) · line 743
Quoted 6 lines
arch/x86/kvm/regs.c +3
-0@@ -738,6 +738,10 @@ int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu)if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_SREGS ) {struct kvm_sregs sregs = vcpu->run->s.regs.sregs; +if (is_guest_mode (vcpu) && + (vcpu->arch.efer &EFER_SVME ) && !(sregs.efer &EFER_SVME )) +return -EINVAL ;
Do we need to check (vcpu->arch.efer & EFER_SVME)?
If that's not set, we already lost, right?
IIUC, we can't be in the middle of save/restore here. Is this intended to check for the situation where userspace cleared SVME while L2 is running? Or maybe L1 cleared EFER_SVME from under L2 but the shutdown didn't happen yet?
I suppose it makes sense for the sync to only fail if it's actually changing something. Mostly just thinking out loud.
Quoted 8 lines
arch/x86/kvm/regs.c +1
-1+if (__set_sregs (vcpu, &sregs))return -EINVAL ; --2.56.0.307.g5fe7bd680c-goog
Sean Christopherson
maintainerRe: [PATCH v2 2/6] KVM: x86: Reject clearing EFER.SVME via KVM_SYNC_X86_SREGS if L2 is active
Quoted 8 lines · Yosry Ahmed
Quoted 6 lines · Sean Christopherson
Disallow clearing EFER.SVME via
KVM_SYNC_X86_SREGSif L2 is active so thatsvm_set_efer()can treat yanking the vCPU out of L2 duringKVM_RUNas a KVM bug. Ideally,kvm_is_valid_sregs()itself would reject attempts to clear SVME while L2 is active, but the risk of breaking userspace, e.g. across save/restore, is too high.
Quoted 8 lines
I am confused. This says we shouldn't disallow clearing SVME while L2 is active because we may break userspace..
Quoted 3 lines
On the other hand, yanking the vCPU out of L2
during save/restore is already "broken" as doing so would queue a triple fault for L1...and this says it's already broken.
Gah, the "save/restore" part is a goof. KVM_SYNC_X86_SREGS is already broken. That should read:
On the other hand, yanking the vCPU out of L2 via KVM_SYNC_X86_SREGS is already "broken" as doing so would queue a triple fault for L1 (because "wants_to_run" will be true).
Quoted 19 lines
I suspect you in the first paragraph were referring to userspace restoring a vCPU not in guest mode on top of a vCPU in guest mode or similar? Might be worth clarifying what exactly we don't wanna break.
Quoted 15 lines
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/regs.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/regs.c +0
-0@@ -738,6 +738,10 @@ int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu)if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_SREGS ) {struct kvm_sregs sregs = vcpu->run->s.regs.sregs;
Quoted 7 lines
arch/x86/kvm/regs.c int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu) · line 743
Quoted 3 lines
arch/x86/kvm/regs.c +3
-0+if (is_guest_mode (vcpu) && + (vcpu->arch.efer &EFER_SVME ) && !(sregs.efer &EFER_SVME )) +return -EINVAL ;Do we need to check (
vcpu->arch.efer&EFER_SVME)?If that's not set, we already lost, right?
This is common code, i.e. runs for both VMX and SVM. The vCPU could be in L2 via nVMX. This could check guest_cpu_cap_has(vcpu, X86_FEATURE_SVM), but I worry that would be more confusing overall.
Quoted 4 lines
IIUC, we can't be in the middle of save/restore here. Is this intended to check for the situation where userspace cleared SVME while L2 is running? Or maybe L1 cleared
EFER_SVMEfrom under L2 but the shutdown didn't happen yet?
It's intended to prevent syzkaller from abusing KVM_SYNC_X86_SREGS to trip the KVM_BUG_ON() I want to add in the next patch.
Quoted 2 lines
I suppose it makes sense for the sync to only fail if it's actually changing something. Mostly just thinking out loud.
Re: [PATCH v2 2/6] KVM: x86: Reject clearing EFER.SVME via KVM_SYNC_X86_SREGS if L2 is active
Quoted 17 lines · Sean Christopherson
Quoted 15 lines · Yosry Ahmed
Quoted 6 lines · Sean Christopherson
Disallow clearing EFER.SVME via
KVM_SYNC_X86_SREGSif L2 is active so thatsvm_set_efer()can treat yanking the vCPU out of L2 duringKVM_RUNas a KVM bug. Ideally,kvm_is_valid_sregs()itself would reject attempts to clear SVME while L2 is active, but the risk of breaking userspace, e.g. across save/restore, is too high.I am confused. This says we shouldn't disallow clearing SVME while L2 is active because we may break userspace..
Quoted 3 lines
On the other hand, yanking the vCPU out of L2
during save/restore is already "broken" as doing so would queue a triple fault for L1.
Quoted 8 lines
Quoted 1 line
..and this says it's already broken.
Gah, the "save/restore" part is a goof.
KVM_SYNC_X86_SREGSis already broken. That should read:On the other hand, yanking the vCPU out of L2 via
KVM_SYNC_X86_SREGSis already "broken" as doing so would queue a triple fault for L1 (because "wants_to_run" will be true).
Much clearer :)
Quoted 26 lines
Quoted 26 lines
I suspect you in the first paragraph were referring to userspace restoring a vCPU not in guest mode on top of a vCPU in guest mode or similar? Might be worth clarifying what exactly we don't wanna break.
Quoted 12 lines
Signed-off-by Sean Christopherson <[email protected]>--- arch/x86/kvm/regs.c | 4 ++++ 1 file changed, 4 insertions(+)arch/x86/kvm/regs.c +0
-0arch/x86/kvm/regs.c int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu) · line 743
Quoted 6 lines
arch/x86/kvm/regs.c +3
-0@@ -738,6 +738,10 @@ int kvm_run_sync_regs_from_user(struct kvm_vcpu *vcpu)if (vcpu->run->kvm_dirty_regs &KVM_SYNC_X86_SREGS ) {struct kvm_sregs sregs = vcpu->run->s.regs.sregs; +if (is_guest_mode (vcpu) && + (vcpu->arch.efer &EFER_SVME ) && !(sregs.efer &EFER_SVME )) +return -EINVAL ;Do we need to check (
vcpu->arch.efer&EFER_SVME)?
Quoted 5 lines
Quoted 1 line
If that's not set, we already lost, right?
This is common code, i.e. runs for both VMX and SVM. The vCPU could be in L2 via nVMX. This could check
guest_cpu_cap_has(vcpu, X86_FEATURE_SVM), but I worry that would be more confusing overall.
Duh, I missed this, sorry for the noise.
Quoted 7 lines
Quoted 4 lines
IIUC, we can't be in the middle of save/restore here. Is this intended to check for the situation where userspace cleared SVME while L2 is running? Or maybe L1 cleared
EFER_SVMEfrom under L2 but the shutdown didn't happen yet?It's intended to prevent syzkaller from abusing
KVM_SYNC_X86_SREGSto trip theKVM_BUG_ON()I want to add in the next patch.
lol
Sean Christopherson
maintainerPATCH v2 3/6 KVM: nSVM: Synthesize #GP instead of SHUTDOWN if L2 attempts to clear EFER.SVME
In reply to Sean ChristophersonIf L2 attempts to clear EFER.SVME (because L1 neglected to intercept writes to EFER), synthesize a #GP instead of a triple-fault SHUTDOWN for L1, as yanking the vCPU out of L2 and mixing the new (invalid) EFER with L2's current state is unnecessarily risky for KVM since the TRIPLE_FAULT request isn't fully handled until KVM attempts to enter the guest, and nothing prevents userspace from ignoring the SHUTDOWN exit and continue running the guest.
Apply the restriction to both guest and host (a.k.a. userspace) writes, as unlike kvm_x86_ops.set_efer() => svm_set_efer(), set_efer() is used *only* for targeted MSR writes, e.g. not for KVM_SET_SREGS{,2}, and KVM does NOT advertise EFER as an MSR that needs to be saved (because it's handled via sregs). I.e. there is far less chance of breaking userspace that manually restores EFER with SVME=0 while L2 is active, because in all likelihood no such VMM exists.
Keep the sanity check in svm_set_efer() that handles the KVM_RUN case, but escalate it to a KVM_BUG_ON() + -EIO, as it should now be unreachable.
--- arch/x86/kvm/msrs.c | 12 ++++++++++++ arch/x86/kvm/svm/svm.c | 15 +++++---------- 2 files changed, 17 insertions(+), 10 deletions(-)
arch/x86/kvm/msrs.c +12 -0
@@ -626,6 +626,18 @@ static int set_efer(struct kvm_vcpu *vcpu, u64 efer)if (is_paging (vcpu) && (vcpu->arch.efer &EFER_LME ) != (efer &EFER_LME ))return 1 ; +/* +* Architecturally (per the APM), clearing EFER.SVME while a guest is +* running yields undefined behavior, i.e. KVM can do literally +* anything. Synthesize a #GP as that is the safest option for KVM, +* as mixing the current vCPU state (from L2) with L1 and the new EFER +* (if KVM simply forces the vCPU out of L2) may yield an unexpected +* end result. +*/ +if (is_guest_mode (vcpu) && + (vcpu->arch.efer &EFER_SVME ) && !(efer &EFER_SVME )) +return 1 ; + efer &= ~EFER_LMA ; efer |= vcpu->arch.efer &EFER_LMA ;
arch/x86/kvm/svm/svm.c +5 -10
@@ -227,17 +227,12 @@ int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)if ((old_efer &EFER_SVME ) != (efer &EFER_SVME )) {if (!(efer &EFER_SVME )) {/* -* Architecturally, clearing EFER.SVME while a guest is -* running yields undefined behavior, i.e. KVM can do -* literally anything. Force the vCPU back into L1 as -* that is the safest option for KVM, but synthesize a -* triple fault (for L1!) so that KVM at least doesn't -* run random L2 code in the context of L1. Do so if -* and only if the vCPU is actively running, e.g. to -* avoid positives if userspace is stuffing state. +* KVM is supposed to synthesize #GP if L2 attempts to +* clear SVME. */ -if (is_guest_mode (vcpu) && vcpu->wants_to_run) -kvm_make_request (KVM_REQ_TRIPLE_FAULT , vcpu); +if (KVM_BUG_ON (is_guest_mode (vcpu) && + vcpu->wants_to_run, vcpu->kvm)) +return -EIO ;svm_leave_nested (vcpu);/* #GP intercept is still needed for vmware backdoor */
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Sean Christopherson
maintainerPATCH v2 4/6 KVM: x86: Recompute EFER.LMA instead of inheriting the current value on EFER writes
In reply to Sean ChristophersonRecompute EFER.LMA on bare writes to EFER (via WRMSR or KVM_SET_MSRS) instead of carrying forward the current EFER.LMA. While it _should_ be impossible for EFER.LMA to end up in an inconsistent state on such a write, KVM has had a rash of exactly those bugs, mostly because SVM is quite lax when it comes to consistenty checks, which in turn bleeds into KVM's nested SVM implementation. Retaining the existing EFER is also rather confusing, as understanding why keeping EFER.LMA is correct requires a lot of a priori knowledge of both x86-64 and KVM.
Keep the core concept of the existing logic as a sanity check: WARN if the computed EFER.LMA would be toggled, because as described above, that _should_ be impossible. Don't return an error if the sanity check fails, e.g. don't inject a #GP, as setting EFER.LMA based on the current state provides architecturally consistency.
--- arch/x86/kvm/msrs.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
arch/x86/kvm/msrs.c +11 -2
@@ -638,8 +638,17 @@ static int set_efer(struct kvm_vcpu *vcpu, u64 efer) (vcpu->arch.efer & EFER_SVME) && !(efer & EFER_SVME))return 1 ; - efer &= ~EFER_LMA ; - efer |= vcpu->arch.efer &EFER_LMA ; +if ((efer &EFER_LME ) &&is_paging (vcpu) &&is_pae (vcpu)) + efer |=EFER_LMA ; +else + efer &= ~EFER_LMA ; + +/* +* Toggling EFER.LMA on a bare write to EFER should be impossible, as +* it means KVM allowed architecturally invalid state at some point in +* the past, i.e. allowed EFER.LMA=1 && (!CR0.PG || !CR4.PAE). +*/ +WARN_ON_ONCE ((efer &EFER_LMA ) != (vcpu->arch.efer &EFER_LMA )); r =kvm_x86_call (set_efer)(vcpu, efer);if (r) {
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v2 4/6] KVM: x86: Recompute EFER.LMA instead of inheriting the current value on EFER writes
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] kvm: x86: guest- and userspace-reachable WARN_ON_ONCE() in set_efer() - [High] kvm: x86: missing…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] kvm: x86: guest- and userspace-reachable WARN_ON_ONCE() in set_efer()
- [High] kvm: x86: missing MMU context reset in set_efer() when EFER_LMA changes
- [High] KVM: nSVM: guest-reachable WARN_ON_ONCE in set_efer() due to stale EFER.LMA --
commit d17bdc446bf3834407b9fa79da8f4a4f619465b7
Author: Sean Christopherson <[email protected]>
KVM: x86: Recompute EFER.LMA instead of inheriting the current value on EFER writes
This commit recomputes EFER.LMA from EFER.LME, CR0.PG, and CR4.PAE on bare writes to MSR_EFER instead of inheriting the existing EFER.LMA bit from vcpu->arch.efer, and adds a WARN_ON_ONCE() if EFER.LMA changes.
Quoted 11 lines
arch/x86/kvm/msrs.c +1
-2@@ -638,8 +638,17 @@ static int set_efer(struct kvm_vcpu *vcpu, u64 efer) (vcpu->arch.efer & EFER_SVME) && !(efer & EFER_SVME))return 1 ; - efer &= ~EFER_LMA ; - efer |= vcpu->arch.efer &EFER_LMA ; +if ((efer &EFER_LME ) &&is_paging (vcpu) &&is_pae (vcpu))
arch/x86/kvm/msrs.c static int set_efer(struct kvm_vcpu *vcpu, u64 efer) · line 651
Quoted 10 lines
arch/x86/kvm/msrs.c +10
-0+ efer |=EFER_LMA ; + else + efer &= ~EFER_LMA; + +/* +* Toggling EFER.LMA on a bare write to EFER should be impossible, as +* it means KVM allowed architecturally invalid state at some point in +* the past, i.e. allowed EFER.LMA=1 && (!CR0.PG || !CR4.PAE). +*/ +WARN_ON_ONCE ((efer &EFER_LMA ) != (vcpu->arch.efer &EFER_LMA ));
[Severity: High]
Can an unprivileged guest or host userspace trigger this WARN_ON_ONCE() in arch/x86/kvm/msrs.c:set_efer() by leaving vcpu->arch.efer with a stale EFER_LMA across nested SVM transitions?
On AMD SVM, VMRUN ignores EFER.LMA in the VMCB save area and computes it from (EFER.LME && CR0.PG), so arch/x86/kvm/svm/nested.c:nested_vmcb_check_save() does not require save->efer's EFER_LMA to match (EFER.LME && CR0.PG).
In arch/x86/kvm/svm/nested.c:nested_vmcb02_prepare_save(), KVM only clears EFER_LMA when EFER_LME is 0, and relies on svm_set_cr0() to update EFER_LMA when EFER_LME is 1:
arch/x86/kvm/svm/nested.c:nested_vmcb02_prepare_save() {
/* SVM ignores EFER.LMA if EFER.LME=0 (instead of failing VMRUN). */
if (!(svm->nested.save.efer & EFER_LME))
svm->nested.save.efer &= ~EFER_LMA;
svm_set_efer(vcpu, svm->nested.save.efer);
svm_set_cr0(vcpu, svm->nested.save.cr0);
svm_set_cr4(vcpu, svm->nested.save.cr4);
}However, arch/x86/kvm/svm/svm.c:svm_set_cr0() only updates EFER_LMA when CR0.PG changes relative to the previous vcpu->arch.cr0 (is_paging(vcpu)):
arch/x86/kvm/svm/svm.c:svm_set_cr0() {
if (vcpu->arch.efer & EFER_LME) {
if (!is_paging(vcpu) && (cr0 & X86_CR0_PG)) {
vcpu->arch.efer |= EFER_LMA;
if (!vcpu->arch.guest_state_protected)
svm->vmcb->save.efer |= EFER_LMA | EFER_LME;
}
if (is_paging(vcpu) && !(cr0 & X86_CR0_PG)) {
vcpu->arch.efer &= ~EFER_LMA;
if (!vcpu->arch.guest_state_protected)
svm->vmcb->save.efer &= ~(EFER_LMA | EFER_LME);
}
}
}This allows vcpu->arch.efer's EFER_LMA to become out of sync with (EFER.LME && CR0.PG && CR4.PAE) in several ways:
1. Guest VMRUN (nested_svm_vmrun() -> enter_svm_guest_mode() -> nested_vmcb02_prepare_save()): L1 must have CR0.PG=1 to execute VMRUN, so is_paging(vcpu) is already true when nested_vmcb02_prepare_save() runs. If L1 sets vmcb12->save with EFER_LME=1, EFER_LMA=0, CR0.PG=1, CR0.PE=1, and CR4.PAE=1, svm_set_efer() loads EFER_LMA=0 into vcpu->arch.efer (and also clears EFER_LME in vmcb02->save.efer when !npt_enabled), and svm_set_cr0() sees !is_paging(vcpu) == false and never sets EFER_LMA. L2 is left with EFER_LME=1, EFER_LMA=0, CR0.PG=1, CR4.PAE=1 (which also truncates L2's 64-bit RSP and RIP to 32 bits and configures the MMU for 32-bit PAE), and a subsequent WRMSR(MSR_EFER) in L2 (or KVM_SET_MSRS from userspace) computes EFER_LMA=1 and trips the WARN_ON_ONCE() in set_efer().
2. Guest RSM into L2 (emulator_leave_smm() -> svm_leave_smm() -> enter_svm_guest_mode() -> nested_vmcb02_prepare_save() -> rsm_load_state_64()): emulator_leave_smm() clears CR0.PG before calling svm_leave_smm(), so is_paging(vcpu) is false. If vmcb12->save in guest memory has EFER_LME=1, EFER_LMA=1, and CR0.PG=0, nested_vmcb02_prepare_save() does not clear EFER_LMA and svm_set_cr0() sees is_paging(vcpu) == false and leaves EFER_LMA=1 in vcpu->arch.efer while CR0.PG=0. Immediately after svm_leave_smm() returns, arch/x86/kvm/smm.c:rsm_load_state_64() calls __kvm_emulate_msr_write(vcpu, MSR_EFER, smstate->efer & ~EFER_LMA) -> set_efer(), which computes EFER_LMA=0 and triggers the WARN_ON_ONCE().
3. KVM_SET_NESTED_STATE or SMM HSAVE restoration followed by nested #VMEXIT (svm_set_nested_state() / svm_leave_smm() -> nested_svm_vmexit()):
arch/x86/kvm/svm/nested.c:svm_set_nested_state() requires CR0.PG=1 in L1's saved host state (vmcb01->save) and rejects (EFER_LMA && !EFER_LME), but allows EFER_LME=1 with EFER_LMA=0:
arch/x86/kvm/svm/nested.c:svm_set_nested_state() {
if (!(save->cr0 & X86_CR0_PG) ||
!(save->cr0 & X86_CR0_PE) ||
(save->rflags & X86_EFLAGS_VM) ||
((save->efer & EFER_LMA) && !(save->efer & EFER_LME)) ||
!nested_vmcb_check_save(vcpu, &save_cached, false))
goto out_free;
}When exiting from a paged L2 (is_paging(vcpu) == true), arch/x86/kvm/svm/nested.c:nested_svm_vmexit() calls svm_set_efer(vcpu, vmcb01->save.efer) followed by svm_set_cr0(vcpu, vmcb01->save.cr0 | X86_CR0_PE), leaving L1 with EFER_LME=1, EFER_LMA=0, CR0.PG=1, CR4.PAE=1 and triggering the WARN_ON_ONCE() on the next MSR_EFER write.
4. SEV-SNP AP creation (sev_snp_ap_creation() -> KVM_REQ_UPDATE_PROTECTED_GUEST_STATE -> kvm_vcpu_reset()): svm_set_cr0() sets EFER_LMA without checking is_pae(vcpu) while vcpu->arch.cr4 still has CR4.PAE=0, allowing a subsequent trapped EFER write via efer_trap() to trigger the WARN_ON_ONCE().
arch/x86/kvm/msrs.c static int set_efer(struct kvm_vcpu *vcpu, u64 efer) · line 654
Quoted 3 lines
arch/x86/kvm/msrs.c +0
-0r =kvm_x86_call (set_efer)(vcpu, efer);if (r) {
[Severity: High]
When set_efer() toggles EFER_LMA to restore architectural consistency, doesn't it also need to reset the MMU context if EFER_LMA changed?
At the end of arch/x86/kvm/msrs.c:set_efer(), KVM only calls kvm_mmu_reset_context(vcpu) when bits in KVM_MMU_EFER_ROLE_BITS change:
arch/x86/kvm/msrs.c:set_efer() {
...
if ((efer ^ old_efer) & KVM_MMU_EFER_ROLE_BITS)
kvm_mmu_reset_context(vcpu);
}However, KVM_MMU_EFER_ROLE_BITS in arch/x86/kvm/mmu.h is defined as (EFER_LME | EFER_NX) and does not include EFER_LMA, even though arch/x86/kvm/mmu/mmu.c:kvm_calc_cpu_role() computes role.base.level, role.ext.efer_lma, role.ext.cr4_pke, and role.ext.cr4_la57 from ____is_efer_lma(regs).
Because set_efer() rejects any write that modifies EFER_LME while is_paging(vcpu) is true, ((efer ^ old_efer) & EFER_LME) is always 0 when paging is enabled. If EFER_LMA is toggled by set_efer() while EFER_NX is unchanged, ((efer ^ old_efer) & KVM_MMU_EFER_ROLE_BITS) evaluates to 0 and kvm_mmu_reset_context(vcpu) is skipped, leaving the vCPU's MMU role and root level stale relative to the updated vcpu->arch.efer.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Sean Christopherson
maintainerRe: [PATCH v2 4/6] KVM: x86: Recompute EFER.LMA instead of inheriting the current value on EFER writes
Quoted 3 lines · On Tue, Oct 27, 2026, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] kvm: x86: guest- and userspace-reachableWARN_ON_ONCE()inset_efer()
- [High] KVM: nSVM: guest-reachableWARN_ON_ONCEinset_efer()due to stale EFER.LMA
As explained in v1, I don't think any of these are actually reachable, and if they are, they are pre-existing bugs that need to be fixed.
Quoted 2 lines
- [High] kvm: x86: missing MMU context reset in
set_efer()whenEFER_LMAchanges --
This is a legit flaw though, though also pre-existing. I'll tack on yet another patch in v3.
Sean Christopherson
maintainerPATCH v2 5/6 KVM: selftests: Update nSVM EFER test to expect #GP when clearing SVME in L2
In reply to Sean ChristophersonAdjust the nSVM EFER test to expect a #GP on WRMSR if L2 attempts to clear EFER.SVME to match KVM's new behavior (synthesizing #GP is safer for KVM as it doesn't require yanking the vCPU out of L2, and is fully synchronous).
--- .../selftests/kvm/x86/svm_nested_efer_test.c | 75 ++++++++++++------- 1 file changed, 46 insertions(+), 29 deletions(-)
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c +46 -29
@@ -15,22 +15,34 @@ static bool l2_ran;static void l2_clear_efer_svme (void ) {u64 efer =rdmsr (MSR_EFER ); +u8 vector;/* generic_svm_setup() initializes EFER_SVME set for L2 */ GUEST_ASSERT (efer &EFER_SVME ); -wrmsr (MSR_EFER , efer & ~EFER_SVME ); -/* Unreachable, L1 should be shutdown */ -GUEST_ASSERT (0 ); + vector =wrmsr_safe (MSR_EFER , efer & ~EFER_SVME ); +__GUEST_ASSERT (vector ==GP_VECTOR , +"Expected #GP when clearing EFER.SVME in L2, got %s" , +ex_str (vector)); + + efer =rdmsr (MSR_EFER ); +GUEST_ASSERT (efer &EFER_SVME ); + + l2_ran = true; +vmmcall (); }static void l1_clear_efer_svme (struct svm_test_data *svm) { +struct vmcb *vmcb = svm->vmcb; +generic_svm_setup (svm, l2_clear_efer_svme); -run_guest (svm->vmcb, svm->vmcb_gpa); +run_guest (vmcb, svm->vmcb_gpa); -/* Unreachable, L1 should be shutdown */ -GUEST_ASSERT (0 ); +GUEST_ASSERT_EQ (vmcb->control.exit_code,SVM_EXIT_VMMCALL ); +GUEST_ASSERT (l2_ran); +GUEST_ASSERT (vmcb->save.efer &EFER_SVME ); +GUEST_DONE (); }static void l2_lmsle (void )
@@ -130,31 +142,10 @@ static void test_enumeration(void)ksft_test_result_pass ("KVM enumerates EFER_LMSLE_MBZ=%d\n" , lmsle_mbz); } -static void test_clear_efer_svme (void ) +static void test_efer_in_guest (struct kvm_vcpu *vcpu) { -struct kvm_vcpu *vcpu; -struct kvm_vm *vm; - - vcpu =create_l1_vcpu (&vm, l1_clear_efer_svme); - -vcpu_run (vcpu); -TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_SHUTDOWN ); - -kvm_vm_free (vm); -ksft_test_result_pass ("L2 clearing EFER.SVME shuts down L1\n" ); -} - -static void test_lmsle (bool lmsle_mbz) -{ -struct kvm_vcpu *vcpu; -struct kvm_vm *vm;struct ucall uc; - vcpu =create_l1_vcpu (&vm, l1_lmsle); - -vcpu_set_or_clear_cpuid_feature (vcpu,X86_FEATURE_EFER_LMSLE_MBZ , - lmsle_mbz); -vcpu_run (vcpu);TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO );
@@ -166,6 +157,32 @@ static void test_lmsle(bool lmsle_mbz)default :TEST_FAIL ("Unexpected ucall: %lu" , uc.cmd); } +} + +static void test_clear_efer_svme (void ) +{ +struct kvm_vcpu *vcpu; +struct kvm_vm *vm; + + vcpu =create_l1_vcpu (&vm, l1_clear_efer_svme); + +test_efer_in_guest (vcpu); + +kvm_vm_free (vm); +ksft_test_result_pass ("L2 clearing EFER.SVME gets #GP\n" ); +} + +static void test_lmsle (bool lmsle_mbz) +{ +struct kvm_vcpu *vcpu; +struct kvm_vm *vm; + + vcpu =create_l1_vcpu (&vm, l1_lmsle); + +vcpu_set_or_clear_cpuid_feature (vcpu,X86_FEATURE_EFER_LMSLE_MBZ , + lmsle_mbz); + +test_efer_in_guest (vcpu);kvm_vm_free (vm);ksft_test_result_pass ("Guest EFER_LMSLE_MBZ=%d\n" , lmsle_mbz);
@@ -248,7 +265,7 @@ int main(int argc, char *argv[])test_clear_efer_svme ();test_lmsle (true); }else { -ksft_test_result_skip ("L2 clearing EFER.SVME shuts down L1\n" ); +ksft_test_result_skip ("L2 clearing EFER.SVME gets #GP\n" );ksft_test_result_skip ("Guest EFER_LMSLE_MBZ=1\n" ); }
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
Sean Christopherson
maintainerPATCH v2 6/6 KVM: selftests: Expect failure when trying to set EFER that violates vCPU model
In reply to Sean ChristophersonAdjust the EFER.LMSLE testcases to expect failure, not success, when attempting to stuff EFER.LMSLE=1 even when the guest CPUID model is configured with EFER_LMSLE_MBZ enabled, i.e. when it should be impossible to set EFER.LMSLE=1. KVM's ABI is that userspace *must* set CPUID and feature MSRs before restoring (or stuffing) vCPU state, i.e. before writing normal MSRs. The previous KVM behavior of allowing usersepace to stuff garbage was a KVM bug (that has since been fixed).
--- .../selftests/kvm/x86/svm_nested_efer_test.c | 20 ++++++------------- 1 file changed, 6 insertions(+), 14 deletions(-)
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c +6 -14
@@ -193,23 +193,20 @@ static void test_host_initiated_lmsle(void)struct kvm_vcpu *vcpu;struct kvm_vm *vm;u64 efer; +int r; vm =vm_create_with_one_vcpu (&vcpu,NULL );vcpu_set_cpuid_feature (vcpu,X86_FEATURE_EFER_LMSLE_MBZ ); -/* -* EFER_LMSLE_MBZ is a guest CPUID consistency check, not a host -* capability, i.e. must not be enforced against host-initiated writes, -* so that userspace can set MSRs before it sets guest CPUID. -*/ efer =vcpu_get_msr (vcpu,MSR_EFER );TEST_ASSERT (!(efer &EFER_LMSLE ),"EFER.LMSLE unexpectedly set" ); -vcpu_set_msr (vcpu,MSR_EFER , efer |EFER_LMSLE ); -TEST_ASSERT_EQ (vcpu_get_msr (vcpu,MSR_EFER ), efer |EFER_LMSLE ); + r =_vcpu_set_msr (vcpu,MSR_EFER , efer |EFER_LMSLE ); +TEST_ASSERT_EQ (r,0 ); +TEST_ASSERT_EQ (vcpu_get_msr (vcpu,MSR_EFER ), efer);kvm_vm_free (vm); -ksft_test_result_pass ("Host-initiated EFER.LMSLE=1 is allowed\n" ); +ksft_test_result_pass ("KVM_SET_MSRS rejects EFER.LMSLE=1 with EFER_LMSLE_MBZ\n" ); }static void test_sregs_lmsle (void )
@@ -222,11 +219,6 @@ static void test_sregs_lmsle(void) vm =vm_create_with_one_vcpu (&vcpu,NULL );vcpu_set_cpuid_feature (vcpu,X86_FEATURE_EFER_LMSLE_MBZ ); -/* -* Unlike KVM_SET_MSRS, KVM_SET_SREGS runs the full set of guest CPUID -* checks, i.e. rejects EFER.LMSLE even though it's host-initiated. -*/ -vcpu_sregs_get (vcpu, &sregs);TEST_ASSERT (!(sregs.efer &EFER_LMSLE ),"EFER.LMSLE unexpectedly set" ); sregs.efer |=EFER_LMSLE ;
@@ -275,7 +267,7 @@ int main(int argc, char *argv[])test_sregs_lmsle (); }else {ksft_test_result_skip ("Guest EFER_LMSLE_MBZ=0\n" ); -ksft_test_result_skip ("Host-initiated EFER.LMSLE=1 is allowed\n" ); +ksft_test_result_skip ("KVM_SET_MSRS rejects EFER.LMSLE=1 with EFER_LMSLE_MBZ\n" );ksft_test_result_skip ("KVM_SET_SREGS rejects EFER.LMSLE=1\n" ); }
Signature · 1 line
2.56.0.307.g5fe7bd680c-goog
[email protected]
AI reviewRe: [PATCH v2 6/6] KVM: selftests: Expect failure when trying to set EFER that violates vCPU model
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] KVM: selftests: uninitialized sregs in test_sregs_lmsle() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] KVM: selftests: uninitialized sregs in test_sregs_lmsle()
--
commit 6277431d999ca1b09f23758ba28993921a931827
Author: Sean Christopherson <[email protected]>
KVM: selftests: Expect failure when trying to set EFER that violates vCPU model
This commit updates test_host_initiated_lmsle() in svm_nested_efer_test.c to expect KVM_SET_MSRS to reject EFER.LMSLE=1 when EFER_LMSLE_MBZ is enabled in guest CPUID, and cleans up related comments in the test.
Quoted 4 lines
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c +0
-0
[ ... ]
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c static void test_sregs_lmsle(void) · line 222
Quoted 10 lines
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c +0
-5@@ -222,11 +219,6 @@ static void test_sregs_lmsle(void) vm =vm_create_with_one_vcpu (&vcpu,NULL );vcpu_set_cpuid_feature (vcpu,X86_FEATURE_EFER_LMSLE_MBZ ); -/* -* Unlike KVM_SET_MSRS, KVM_SET_SREGS runs the full set of guest CPUID -* checks, i.e. rejects EFER.LMSLE even though it's host-initiated. -*/ -vcpu_sregs_get (vcpu, &sregs);TEST_ASSERT (!(sregs.efer &EFER_LMSLE ),"EFER.LMSLE unexpectedly set" );
[Severity: Medium]
Was removing vcpu_sregs_get(vcpu, &sregs) in test_sregs_lmsle() unintentional?
Without that call, the stack variable sregs is left uninitialized when TEST_ASSERT() reads sregs.efer and when sregs.efer |= EFER_LMSLE is passed to _vcpu_sregs_set(vcpu, &sregs):
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c:test_sregs_lmsle() {
struct kvm_vcpu *vcpu;
struct kvm_sregs sregs;
struct kvm_vm *vm;
int rc;
vm = vm_create_with_one_vcpu(&vcpu, NULL);
vcpu_set_cpuid_feature(vcpu, X86_FEATURE_EFER_LMSLE_MBZ);
TEST_ASSERT(!(sregs.efer & EFER_LMSLE), "EFER.LMSLE unexpectedly set");
sregs.efer |= EFER_LMSLE;
rc = _vcpu_sregs_set(vcpu, &sregs);
...
}In addition to reading uninitialized stack memory on the assertion, passing an uninitialized struct kvm_sregs to KVM_SET_SREGS means KVM may reject the ioctl due to garbage in other fields rather than specifically testing rejection of EFER.LMSLE=1.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Sean Christopherson
maintainerRe: [PATCH v2 6/6] KVM: selftests: Expect failure when trying to set EFER that violates vCPU model
Quoted 4 lines · On Tue, Oct 27, 2026, [email protected] wrote:
Quoted 4 lines
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c +0
-0@@ -222,11 +219,6 @@ static void test_sregs_lmsle(void) vm =vm_create_with_one_vcpu (&vcpu,NULL );vcpu_set_cpuid_feature (vcpu,X86_FEATURE_EFER_LMSLE_MBZ );
Quoted 9 lines
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c static void test_sregs_lmsle(void) · line 222
Quoted 6 lines
tools/testing/selftests/kvm/x86/svm_nested_efer_test.c +0
-5-/* -* Unlike KVM_SET_MSRS, KVM_SET_SREGS runs the full set of guest CPUID -* checks, i.e. rejects EFER.LMSLE even though it's host-initiated. -*/ -vcpu_sregs_get (vcpu, &sregs);TEST_ASSERT (!(sregs.efer &EFER_LMSLE ),"EFER.LMSLE unexpectedly set" );[Severity: Medium]
Was removingvcpu_sregs_get(vcpu, &sregs)intest_sregs_lmsle()unintentional?
No, I intended to delete just the comment, and unfortunately the test managed to pass on multiple systems (I'm guessing my build zeroed the stack?).