KVM: nVMX: Don't let L2's LMSW modify CR0 bits owned by L1
v2 cover KVM: nVMX: Don't let L2's LMSW modify CR0 bits owned by L1
When L2's LMSW exits to L0 but not to L1, KVM emulates it with kvm_lmsw(), which ignores vmcs12's CR0 guest/host mask, so L2 can change CR0 bits that L1 owns.
v2:
- Patch 2: order the CR0 store before FNOP and check L2's whole CR0.
v1: https://lore.kernel.org/all/[email protected]
Mahmoud Abumandour (2):
KVM: nVMX: Don't let L2's LMSW modify CR0 bits owned by L1
KVM: selftests: Verify that L2's LMSW leaves L1-owned CR0 bits alone
arch/x86/kvm/vmx/vmx.c | 15 ++++++++ .../kvm/x86/nested_exceptions_test.c | 36 +++++++++++++++++++ 2 files changed, 51 insertions(+)
base-commit: dea4f6380f4ec96e8f3b0aaf2343764bd0d25ef4
Signature · 1 line
2.25.1
PATCH v2 1/2 KVM: nVMX: Don't let L2's LMSW modify CR0 bits owned by L1
When L2's LMSW exits to L0 but not to L1, emulate it through handle_set_cr0(), as is done for MOV to CR0. kvm_lmsw() ignores vmcs12's CR0 guest/host mask, so L2 could change CR0 bits that L1 owns, e.g. clear TS, and read their effective values instead of the read shadow.
--- arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
arch/x86/kvm/vmx/vmx.c +15 -0
@@ -5748,6 +5748,21 @@ static int handle_cr(struct kvm_vcpu *vcpu)case 3 :/* lmsw */ val = (exit_qualification >>LMSW_SOURCE_DATA_SHIFT ) &0x0f ;trace_kvm_cr_write (0 , (kvm_read_cr0_bits (vcpu, ~0xful ) | val)); +if (is_guest_mode (vcpu)) { +struct vmcs12 *vmcs12 =get_vmcs12 (vcpu); +unsigned long mask = vmcs12->cr0_guest_host_mask; +unsigned long cr0; + +/* +* LMSW leaves the bits L1 owns unmodified, as does a MOV +* to CR0 that writes their read shadow values. Emulate +* it as that MOV: L2's view of CR0 with LMSW's bits. +*/ + cr0 = (kvm_read_cr0 (vcpu) & ~mask) | + (vmcs12->cr0_read_shadow & mask); + err =handle_set_cr0 (vcpu, (cr0 & ~0x0eul ) | val); +return kvm_complete_insn_gp (vcpu, err); + }kvm_lmsw (vcpu, val);return kvm_skip_emulated_instruction (vcpu);
Signature · 1 line
2.25.1
Sean Christopherson
maintainerQuoted 21 lines · Mahmoud Abumandour
When L2's LMSW exits to L0 but not to L1, emulate it through
handle_set_cr0(), as is done for MOV to CR0.kvm_lmsw()ignores vmcs12's CR0 guest/host mask, so L2 could change CR0 bits that L1 owns, e.g. clear TS, and read their effective values instead of the read shadow.Fixes eeadf9e7558c ("KVM: nVMX: Handling of CR0 and CR4 modifying instructions")Signed-off-by Mahmoud Abumandour <[email protected]>--- arch/x86/kvm/vmx/vmx.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)arch/x86/kvm/vmx/vmx.c +2
-0@@ -5748,6 +5748,21 @@ static int handle_cr(struct kvm_vcpu *vcpu)case 3 :/* lmsw */ val = (exit_qualification >>LMSW_SOURCE_DATA_SHIFT ) &0x0f ;trace_kvm_cr_write (0 , (kvm_read_cr0_bits (vcpu, ~0xful ) | val)); +if (is_guest_mode (vcpu)) { +struct vmcs12 *vmcs12 =get_vmcs12 (vcpu);
arch/x86/kvm/vmx/vmx.c static int handle_cr(struct kvm_vcpu *vcpu) · line 5762
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10
-0+unsigned long mask = vmcs12->cr0_guest_host_mask; +unsigned long cr0; + +/* +* LMSW leaves the bits L1 owns unmodified, as does a MOV +* to CR0 that writes their read shadow values. Emulate +* it as that MOV: L2's view of CR0 with LMSW's bits. +*/ + cr0 = (kvm_read_cr0 (vcpu) & ~mask) | + (vmcs12->cr0_read_shadow & mask);
This is quite confusing. This *looks* like it incorrectly handles CR0.PE for the case where PE=1 in guest_cr0 and the guest/host mask, but PE=0 in cr0_read_shadow. In that insane scenario, LMSW with PE=0 won't be intercepted by L1, but needs to preserve PE=1 in the "hardware" value while also keeping PE=0 in the read shadow.
The above initially preseve CR0.PE=1, but then drops it from the effective @val passed to handle_set_cr0(). Which works in the end handle_set_cr0() preserves L1-owned bits using guest_cr0, but piecing that all together is confusing because at first (and second, and third...) glance, it's very easy to think that *this*
code is taking responsibility for enforcing the cr0_guest_host_mask, but the
ownership is super weirdly split across this code and handle_set_cr0().
I also don't like that this duplicates logic for kvm_lmsw() and handle_set_cr0().
Unless I'm missing something, we can handle all of this in handle_set_cr0() simply by ensuring handle_set_cr0() (and handle_set_cr4() for completeness) preserve bits that are L1-owned in the read shadow. handle_set_cr{0,4} currently squeak by without explicitly doing so, because straight writes via MOV CR will generate a nested VM-Exit to L1 if the incoming value toggles any of the shadowed bits. I.e. it's only LMSW that can kinda sorta toggle bits in violation of the intercept controls.
So, as a prep patch, this:
x86/kvm/vmx/vmx.c +13 -17
@@ -5629,22 +5629,19 @@ static int handle_set_cr0(struct kvm_vcpu *vcpu, unsigned long val) {if (is_guest_mode (vcpu)) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu); -unsigned long orig_val = val; +u64 mask = vmcs12->cr0_guest_host_mask;/* -* We get here when L2 changed cr0 in a way that did not change -* any of L1's shadowed bits (see nested_vmx_exit_handled_cr), -* but did change L0 shadowed bits. So we first calculate the -* effective cr0 value that L1 would like to write into the -* hardware. It consists of the L2-owned bits from the new -* value combined with the L1-owned bits from L1's guest_cr0. +* L2 is writing a CR0 value that isn't modifying any of L1's +* shadowed bits (see nested_vmx_exit_handled_cr()), but *is* +* modifying L0 shadowed bits. Set the "hardware" CR0 and the +* read shadow using the L2-owned bits from the new value, +* combined with the L1-owned bits. */ - val = (val & ~vmcs12->cr0_guest_host_mask) | - (vmcs12->guest_cr0 & vmcs12->cr0_guest_host_mask); - -if (kvm_set_cr0 (vcpu, val)) + val &= ~mask; +if (kvm_set_cr0 (vcpu, val | (vmcs12->guest_cr0 & mask)))return 1 ; -vmcs_writel (CR0_READ_SHADOW , orig_val); +vmcs_writel (CR0_READ_SHADOW , val | (vmcs12->cr0_read_shadow & mask));return 0 ; }else {return kvm_set_cr0 (vcpu, val);
@@ -5655,14 +5652,13 @@ static int handle_set_cr4(struct kvm_vcpu *vcpu, unsigned long val) {if (is_guest_mode (vcpu)) {struct vmcs12 *vmcs12 =get_vmcs12 (vcpu); -unsigned long orig_val = val; +u64 mask = vmcs12->cr4_guest_host_mask;/* analogously to handle_set_cr0 */ - val = (val & ~vmcs12->cr4_guest_host_mask) | - (vmcs12->guest_cr4 & vmcs12->cr4_guest_host_mask); -if (kvm_set_cr4 (vcpu, val)) + val &= ~mask; +if (kvm_set_cr4 (vcpu, val | (vmcs12->guest_cr4 & mask)))return 1 ; -vmcs_writel (CR4_READ_SHADOW , orig_val); +vmcs_writel (CR4_READ_SHADOW , val | (vmcs12->cr4_read_shadow & mask));return 0 ; }else return kvm_set_cr4 (vcpu, val);
and then the fix becomes:
x86/kvm/vmx/vmx.c +4 -2
@@ -5743,8 +5743,10 @@ static int handle_cr(struct kvm_vcpu *vcpu)break ;case 3 :/* lmsw */ val = (exit_qualification >>LMSW_SOURCE_DATA_SHIFT ) &0x0f ; -trace_kvm_cr_write (0 , (kvm_read_cr0_bits (vcpu, ~0xful ) | val)); -kvm_lmsw (vcpu, val); + val |=kvm_read_cr0_bits (vcpu, ~0x0eul ); + +trace_kvm_cr_write (cr, val); + (void )handle_set_cr0 (vcpu, val);return kvm_skip_emulated_instruction (vcpu);default :
arch/x86/kvm/vmx/vmx.c static int handle_cr(struct kvm_vcpu *vcpu) · line 5764
Quoted 2 lines
arch/x86/kvm/vmx/vmx.c +2
-0+ err =handle_set_cr0 (vcpu, (cr0 & ~0x0eul ) | val); +return kvm_complete_insn_gp (vcpu, err);
This needs to (a) be a separate patch and (b) apply to L1 as well (though I'm not
even sure if you/AI realize this is a bug fix). Emulation of LMSW originally
suppressed the return value, because outside of VMX operation, LMSW can't #GP on a bad value. LMSW can't set reserved bits (for the CPU models KVM supports), and it can't toggle any bits that trigger additional checks, e.g. can't try to clear CR0.PG while in 64-bit mode.
But with VMX, must-be-1 and must-be-0 bits become a thing, and so LMSW could #GP if it tried to set/clear a bit in violation of those masks.
Ugh, never mind, screw this. KVM doesn't support arbitrary MSR_IA32_VMX_CR0_FIXED1 values, i.e. doesn't let userspace require that arbitrary bits always be '0', and instead relies on common CR0 reserved bit checks. KVM (very, very stupidly) does allow userspace to expand the set of must-be-1 (MSR_IA32_VMX_CR0_FIXED0) bits), and so *in theory* LMSW could fault when clearing EM, MP, or TS. Of course, no sane VMM sets those as must-be-1 bits. And to make this a complete trainwreck, KVM doesn't update its own guest/host masks, i.e. it's own intercepts, to ensure it enforces the must-be-1 bits.
E.g. if userspace made CR0.TS a must-be-1 bit, then the guest could violate that so long as it doesn't toggle any other bits that would trigger a VM-Exit.
I see absolutely zero reason to try and fix KVM. Letting userspace make CR0.TS a must-be-1 bit is beyond stupid. KVM already requires CR0 and CR4 must-be-1 masks to be a superset of KVM's "defaults", so userspace can only make things more restrictive.
So, rather than synthesize #GPs, I think we should prevent nVMX MSRs from inducing #GPs in the first place, and then WARN.
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +1
-0+ }kvm_lmsw (vcpu, val);return kvm_skip_emulated_instruction (vcpu);Signature · 1 line
2.25.1
Sean Christopherson
maintainerQuoted 44 lines · Sean Christopherson
On Thu, Oct 29, 2026, Mahmoud Abumandour wrote:
x86/kvm/vmx/vmx.c +4
-2@@ -5743,8 +5743,10 @@ static int handle_cr(struct kvm_vcpu *vcpu)break ;case 3 :/* lmsw */ val = (exit_qualification >>LMSW_SOURCE_DATA_SHIFT ) &0x0f ; -trace_kvm_cr_write (0 , (kvm_read_cr0_bits (vcpu, ~0xful ) | val)); -kvm_lmsw (vcpu, val); + val |=kvm_read_cr0_bits (vcpu, ~0x0eul ); + +trace_kvm_cr_write (cr, val); + (void )handle_set_cr0 (vcpu, val);return kvm_skip_emulated_instruction (vcpu);default :Review
Quoted 3 lines
(quoted diff) +2
-0+ err = handle_set_cr0(vcpu, (cr0 & ~0x0eul) | val); + return kvm_complete_insn_gp(vcpu, err);This needs to (a) be a separate patch and (b) apply to L1 as well (though I'm not
even sure if you/AI realize this is a bug fix). Emulation of LMSW originally
suppressed the return value, because outside of VMX operation, LMSW can't #GP on a bad value. LMSW can't set reserved bits (for the CPU models KVM supports), and it can't toggle any bits that trigger additional checks, e.g. can't try to clear CR0.PG while in 64-bit mode.But with VMX, must-be-1 and must-be-0 bits become a thing, and so LMSW could #GP if it tried to set/clear a bit in violation of those masks.
Ugh, never mind, screw this. KVM doesn't support arbitrary
MSR_IA32_VMX_CR0_FIXED1values, i.e. doesn't let userspace require that arbitrary bits always be '0', and instead relies on common CR0 reserved bit checks. KVM (very, very stupidly) does allow userspace to expand the set of must-be-1 (MSR_IA32_VMX_CR0_FIXED0) bits), and so *in theory* LMSW could fault when clearing EM, MP, or TS. Of course, no sane VMM sets those as must-be-1 bits. And to make this a complete trainwreck, KVM doesn't update its own guest/host masks, i.e. it's own intercepts, to ensure it enforces the must-be-1 bits.E.g. if userspace made CR0.TS a must-be-1 bit, then the guest could violate that so long as it doesn't toggle any other bits that would trigger a VM-Exit.
Quoted 7 lines
I see absolutely zero reason to try and fix KVM. Letting userspace make CR0.TS a must-be-1 bit is beyond stupid. KVM already requires CR0 and CR4 must-be-1 masks to be a superset of KVM's "defaults", so userspace can only make things more restrictive.
So, rather than synthesize #GPs, I think we should prevent nVMX MSRs from inducing #GPs in the first place, and then WARN.
Argh, that's not going to work. Thanks to KVM's massive "attack" surface, a.k.a. save/restore APIs, it's possible for userspace to shove a value into vcpu->arch.cr0 that's in violation of must-be-1 bits. I.e. userspace can trigger the WARN.
So as much as I want to WARN, in practice no real world use case will induce a #GP on LMSW (or hell, even trigger an exit on LMSW), so I'll go with your patch of synthesizing a #GP (which is better than silently failing).
PATCH v2 2/2 KVM: selftests: Verify that L2's LMSW leaves L1-owned CR0 bits alone
In reply to Mahmoud AbumandourVerify that an LMSW in L2 that exits to L0 only leaves an L1-owned CR0.TS set, and that L2 reads an L1-owned CR0.NE from the read shadow.
--- .../kvm/x86/nested_exceptions_test.c | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+)
tools/testing/selftests/kvm/x86/nested_exceptions_test.c +36 -0
@@ -109,6 +109,40 @@ static void l1_svm_code(struct svm_test_data *svm)GUEST_DONE (); } +static u64 l2_cr0; + +static void l2_lmsw_test (void ) +{ +/* Toggle CR0.MP, which KVM intercepts and L1 doesn't. */ + asmvolatile ("lmsw %w0" : :"r" (get_cr0 () ^X86_CR0_MP )); + l2_cr0 =get_cr0 (); + asmvolatile ("fnop" :::"memory" ); +GUEST_FAIL ("L2's FNOP didn't take #NM" ); +} + +/* +* L1 owns CR0.TS (set) and CR0.NE, both with a read shadow of 0. L2's LMSW +* must leave both alone: FNOP takes #NM and L2 reads both as 0. +*/ +static void vmx_run_l2_lmsw (void ) +{ +u64 cr0 =vmread (GUEST_CR0 ); + +vmwrite (CR0_GUEST_HOST_MASK ,X86_CR0_TS |X86_CR0_NE ); +vmwrite (CR0_READ_SHADOW , cr0 & ~(X86_CR0_TS |X86_CR0_NE )); +vmwrite (GUEST_CR0 , cr0 |X86_CR0_TS ); +vmwrite (EXCEPTION_BITMAP ,BIT (NM_VECTOR )); +vmwrite (GUEST_RIP , (u64 )l2_lmsw_test); +vmresume (); + +GUEST_ASSERT_EQ (vmread (VM_EXIT_REASON ),EXIT_REASON_EXCEPTION_NMI ); +GUEST_ASSERT_EQ (vmread (VM_EXIT_INTR_INFO ) &0xff ,NM_VECTOR ); +GUEST_ASSERT_EQ (l2_cr0, (cr0 & ~(X86_CR0_TS |X86_CR0_NE )) ^X86_CR0_MP ); + +vmwrite (CR0_GUEST_HOST_MASK ,0 ); +vmwrite (GUEST_CR0 , cr0); +} +static void vmx_run_l2 (void *l2_code,int vector,u32 error_code) {vmwrite (GUEST_RIP , (u64 )l2_code);
@@ -149,6 +183,8 @@ static void l1_vmx_code(struct vmx_pages *vmx)vmwrite (EXCEPTION_BITMAP ,INTERCEPT_SS_DF );vmx_run_l2 (l2_ss_injected_df_test,DF_VECTOR ,DF_ERROR_CODE ); +vmx_run_l2_lmsw (); +vmwrite (EXCEPTION_BITMAP ,INTERCEPT_SS );vmx_run_l2 (l2_ss_injected_tf_test,FAKE_TRIPLE_FAULT_VECTOR ,0 );GUEST_ASSERT_EQ (vmread (VM_EXIT_REASON ),EXIT_REASON_TRIPLE_FAULT );
Signature · 1 line
2.25.1
Sean Christopherson
maintainerRe: [PATCH v2 2/2] KVM: selftests: Verify that L2's LMSW leaves L1-owned CR0 bits alone
Quoted 14 lines · Mahmoud Abumandour
Verify that an LMSW in L2 that exits to L0 only leaves an L1-owned CR0.TS set, and that L2 reads an L1-owned CR0.NE from the read shadow.
Assisted-by LLMSigned-off-by Mahmoud Abumandour <[email protected]>--- .../kvm/x86/nested_exceptions_test.c | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+)tools/testing/selftests/kvm/x86/nested_exceptions_test.c +0
-0
tools/testing/selftests/kvm/x86/nested_exceptions_test.c static void l1_svm_code(struct svm_test_data *svm) · line 117
Quoted 9 lines
tools/testing/selftests/kvm/x86/nested_exceptions_test.c +6
-0@@ -109,6 +109,40 @@ static void l1_svm_code(struct svm_test_data *svm)GUEST_DONE (); } +static u64 l2_cr0; + +static void l2_lmsw_test (void ) +{ +/* Toggle CR0.MP, which KVM intercepts and L1 doesn't. */ + asmvolatile ("lmsw %w0" : :"r" (get_cr0 () ^X86_CR0_MP ));
We should add lmsw(), and also test MOV CR0 because it's easy to do so.
tools/testing/selftests/kvm/x86/nested_exceptions_test.c static void l1_svm_code(struct svm_test_data *svm) · line 120
Quoted 3 lines
tools/testing/selftests/kvm/x86/nested_exceptions_test.c +3
-0+ l2_cr0 =get_cr0 (); + asmvolatile ("fnop" :::"memory" ); +GUEST_FAIL ("L2's FNOP didn't take #NM" );
*sigh*
Again, I have no idea if you/AI even realize the careful dance that's going on here. I *wanted* to say this should simply have L2 assert on the expected CR0 value, but doing so results in false passes because trying to report the assert means making a ucall(), ucall_alloc() does a memset(), and the compiler likes to generate SIMD instructions for memset(). I.e. trying to report a failure triggers the #NM that L1 expects.
So the code is fine, but this needs a big fat pile of comments and the changelog needs to explain what's going on as well.
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_exceptions_test.c +10
-0+} + +/* +* L1 owns CR0.TS (set) and CR0.NE, both with a read shadow of 0. L2's LMSW +* must leave both alone: FNOP takes #NM and L2 reads both as 0. +*/ +static void vmx_run_l2_lmsw (void ) +{ +u64 cr0 =vmread (GUEST_CR0 ); +
tools/testing/selftests/kvm/x86/nested_exceptions_test.c static void l1_svm_code(struct svm_test_data *svm) · line 140
Quoted 10 lines
tools/testing/selftests/kvm/x86/nested_exceptions_test.c +10
-0+vmwrite (CR0_GUEST_HOST_MASK ,X86_CR0_TS |X86_CR0_NE ); +vmwrite (CR0_READ_SHADOW , cr0 & ~(X86_CR0_TS |X86_CR0_NE )); +vmwrite (GUEST_CR0 , cr0 |X86_CR0_TS ); +vmwrite (EXCEPTION_BITMAP ,BIT (NM_VECTOR )); +vmwrite (GUEST_RIP , (u64 )l2_lmsw_test); +vmresume (); + +GUEST_ASSERT_EQ (vmread (VM_EXIT_REASON ),EXIT_REASON_EXCEPTION_NMI ); +GUEST_ASSERT_EQ (vmread (VM_EXIT_INTR_INFO ) &0xff ,NM_VECTOR ); +GUEST_ASSERT_EQ (l2_cr0, (cr0 & ~(X86_CR0_TS |X86_CR0_NE )) ^X86_CR0_MP );
Just use vmx_run_l2(), most of this is copy+pasted from there anyways.
Quoted 21 lines
tools/testing/selftests/kvm/x86/nested_exceptions_test.c +7
-0+ +vmwrite (CR0_GUEST_HOST_MASK ,0 ); +vmwrite (GUEST_CR0 , cr0); +} +static void vmx_run_l2 (void *l2_code,int vector,u32 error_code) {vmwrite (GUEST_RIP , (u64 )l2_code);@@ -149,6 +183,8 @@ static void l1_vmx_code(struct vmx_pages *vmx)vmwrite (EXCEPTION_BITMAP ,INTERCEPT_SS_DF );vmx_run_l2 (l2_ss_injected_df_test,DF_VECTOR ,DF_ERROR_CODE ); +vmx_run_l2_lmsw (); +vmwrite (EXCEPTION_BITMAP ,INTERCEPT_SS );vmx_run_l2 (l2_ss_injected_tf_test,FAKE_TRIPLE_FAULT_VECTOR ,0 );GUEST_ASSERT_EQ (vmread (VM_EXIT_REASON ),EXIT_REASON_TRIPLE_FAULT );Signature · 1 line
2.25.1
Sean Christopherson
maintainerQuoted 6 lines · Mahmoud Abumandour
When L2's LMSW exits to L0 but not to L1, KVM emulates it with
kvm_lmsw(), which ignores vmcs12's CR0 guest/host mask, so L2 can change CR0 bits that L1 owns.v2:
- Patch 2: order the CR0 store before FNOP and check L2's whole CR0.
I'll send a v3, I already spent an absurd amount of time figuring all of this out. I really hate x86 sometimes.
Quoted 11 lines · Sean Christopherson
Quoted 6 lines · Mahmoud Abumandour
When L2's LMSW exits to L0 but not to L1, KVM emulates it with
kvm_lmsw(), which ignores vmcs12's CR0 guest/host mask, so L2 can change CR0 bits that L1 owns.v2:
- Patch 2: order the CR0 store before FNOP and check L2's whole CR0.I'll send a v3, I already spent an absurd amount of time figuring all of this out. I really hate x86 sometimes.
You should redirect that emotion to KVM APIs!
Sean Christopherson
maintainerQuoted 7 lines · Jim Mattson
Quoted 6 lines · Sean Christopherson
Quoted 4 lines · Mahmoud Abumandour
When L2's LMSW exits to L0 but not to L1, KVM emulates it with
kvm_lmsw(), which ignores vmcs12's CR0 guest/host mask, so L2 can change CR0 bits that L1 owns.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
v2:
- Patch 2: order the CR0 store before FNOP and check L2's whole CR0.I'll send a v3, I already spent an absurd amount of time figuring all of this out. I really hate x86 sometimes.
You should redirect that emotion to KVM APIs!
Heh, saying I hate KVM's APIs sometimes would be a lie. I hate them all the time.