KVM: x86: Handle reserved CR4 bit interception in VMX
cover KVM: x86: Handle reserved CR4 bit interception in VMX
This series stems from Lai's RFC patches to intercept LA57 and let the guest own FSGSBASE[*]. Discussion and inspection revealed that KVM does not handle the case where LA57 is supported in hardware but not exposed to the guest. This is actually true for all CR4 bits, but LA57 is currently the only bit that can be reserved and also owned by the guest. I have a unit test for this that I'll post separately.
Intercepting LA57 was by far the easiest fix for the immedidate bug, and is likely the right change in the long term as there's no justification for letting the guest own LA57.
The middle three patches adjust VMX's CR4 guest/host mask to intercept reserved bits. This required reworking CPUID updates to also refresh said mask at the correct time.
The last past is Lai's, which let's the guest own FSGSBASE. This depends on the reserved bit handling being in place.
Ran everything through unit tests, and ran the kernel's FSGSBASE selftests in a VM.
[*] https://lkml.kernel.org/r/[email protected]
Lai Jiangshan (2):
KVM: x86: Intercept LA57 to inject #GP fault when it's reserved
KVM: x86: Let the guest own CR4.FSGSBASE
Sean Christopherson (3):
KVM: x86: Invoke vendor's vcpu_after_set_cpuid() after all common
updates
KVM: x86: Move call to update_exception_bitmap() into VMX code
KVM: VMX: Intercept guest reserved CR4 bits to inject #GP fault
arch/x86/kvm/cpuid.c | 6 +++--- arch/x86/kvm/kvm_cache_regs.h | 2 +- arch/x86/kvm/vmx/vmx.c | 18 +++++++++++++----- 3 files changed, 17 insertions(+), 9 deletions(-)
Signature · 1 line
2.28.0
PATCH 1/5 KVM: x86: Intercept LA57 to inject #GP fault when it's reserved
From: Lai Jiangshan <[email protected]>
Unconditionally intercept changes to CR4.LA57 so that KVM correctly injects a #GP fault if the guest attempts to set CR4.LA57 when it's supported in hardware but not exposed to the guest.
Long term, KVM needs to properly handle CR4 bits that can be under guest control but also may be reserved from the guest's perspective. But, KVM currently sets the CR4 guest/host mask only during vCPU creation, and reworking flows to change that will take a bit of elbow grease.
Even if/when generic support for intercepting reserved bits exists, it's probably not worth letting the guest set CR4.LA57 directly. LA57 can't be toggled while long mode is enabled, thus it's all but guaranteed to be set once (maybe twice, e.g. by BIOS and kernel) during boot and never touched again. On the flip side, letting the guest own CR4.LA57 may incur extra VMREADs. In other words, this temporary "hack" is probably also the right long term fix.
--- arch/x86/kvm/kvm_cache_regs.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/kvm_cache_regs.h +1 -1
@@ -7,7 +7,7 @@#define KVM_POSSIBLE_CR0_GUEST_BITS X86_CR0_TS#define KVM_POSSIBLE_CR4_GUEST_BITS \ (X86_CR4_PVI | X86_CR4_DE | X86_CR4_PCE | X86_CR4_OSFXSR \ - | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_PGE | X86_CR4_TSD) + | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD)#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\
Signature · 1 line
2.28.0
PATCH 2/5 KVM: x86: Invoke vendor's vcpu_after_set_cpuid() after all common updates
In reply to Sean ChristophersonMove the call to kvm_x86_ops.vcpu_after_set_cpuid() to the very end of kvm_vcpu_after_set_cpuid() to allow the vendor implementation to react to changes made by the common code. In the near future, this will be used by VMX to update its CR4 guest/host masks to account for reserved bits. In the long term, SGX support will update the allowed XCR0 mask for enclaves based on the vCPU's allowed XCR0.
vcpu_after_set_cpuid() (nee kvm_update_cpuid()) was originally added by commit 2acf923e38fb ("KVM: VMX: Enable XSAVE/XRSTOR for guest"), and was called separately after kvm_x86_ops.vcpu_after_set_cpuid() (nee kvm_x86_ops->cpuid_update()). There is no indication that the placement of the common code updates after the vendor updates was anything more than a "new function at the end" decision.
Inspection of the current code reveals no dependency on kvm_x86_ops' vcpu_after_set_cpuid() in kvm_vcpu_after_set_cpuid() or any of its helpers. The bulk of the common code depends only on the guest's CPUID configuration, kvm_mmu_reset_context() does not consume dynamic vendor state, and there are no collisions between kvm_pmu_refresh() and VMX's update of PT state.
--- arch/x86/kvm/cpuid.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +3 -2
@@ -121,8 +121,6 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)struct kvm_lapic *apic = vcpu->arch.apic;struct kvm_cpuid_entry2 *best; - kvm_x86_ops.vcpu_after_set_cpuid (vcpu); - best =kvm_find_cpuid_entry (vcpu,1 ,0 );if (best && apic) {if (cpuid_entry_has (best,X86_FEATURE_TSC_DEADLINE_TIMER ))
@@ -146,6 +144,9 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)kvm_pmu_refresh (vcpu); vcpu->arch.cr4_guest_rsvd_bits =__cr4_reserved_bits (guest_cpuid_has, vcpu); + +/* Invoke the vendor callback only after the above state is updated. */ + kvm_x86_ops.vcpu_after_set_cpuid (vcpu); kvm_x86_ops.update_exception_bitmap (vcpu); }
Signature · 1 line
2.28.0
PATCH 3/5 KVM: x86: Move call to update_exception_bitmap() into VMX code
In reply to Sean ChristophersonNow that vcpu_after_set_cpuid() and update_exception_bitmap() are called back-to-back, subsume the exception bitmap update into the common CPUID update. Drop the SVM invocation entirely as SVM's exception bitmap doesn't vary with respect to guest CPUID.
--- arch/x86/kvm/cpuid.c | 1 - arch/x86/kvm/vmx/vmx.c | 3 +++ 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +0 -1
@@ -147,7 +147,6 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)/* Invoke the vendor callback only after the above state is updated. */ kvm_x86_ops.vcpu_after_set_cpuid (vcpu); - kvm_x86_ops.update_exception_bitmap (vcpu); }static int is_efer_nx (void )
arch/x86/kvm/vmx/vmx.c +3 -0
@@ -7232,6 +7232,9 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)vmx_set_guest_uret_msr (vmx, msr, enabled ?0 :TSX_CTRL_RTM_DISABLE ); } } + +/* Refresh #PF interception to account for MAXPHYADDR changes. */ +update_exception_bitmap (vcpu); }static __init void vmx_set_cpu_caps (void )
Signature · 1 line
2.28.0
PATCH 4/5 KVM: VMX: Intercept guest reserved CR4 bits to inject #GP fault
In reply to Sean ChristophersonIntercept CR4 bits that are guest reserved so that KVM correctly injects a #GP fault if the guest attempts to set a reserved bit. If a feature is supported by the CPU but is not exposed to the guest, and its associated CR4 bit is not intercepted by KVM by default, then KVM will fail to inject a #GP if the guest sets the CR4 bit without triggering an exit, e.g. by toggling only the bit in question.
Note, KVM doesn't give the guest direct access to any CR4 bits that are also dependent on guest CPUID. Yet.
--- arch/x86/kvm/vmx/vmx.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
arch/x86/kvm/vmx/vmx.c +10 -5
@@ -4037,13 +4037,16 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx)void set_cr4_guest_host_mask (struct vcpu_vmx *vmx) { - vmx->vcpu.arch.cr4_guest_owned_bits =KVM_POSSIBLE_CR4_GUEST_BITS ; +struct kvm_vcpu *vcpu = &vmx->vcpu; + + vcpu->arch.cr4_guest_owned_bits =KVM_POSSIBLE_CR4_GUEST_BITS & + ~vcpu->arch.cr4_guest_rsvd_bits;if (!enable_ept) - vmx->vcpu.arch.cr4_guest_owned_bits &= ~X86_CR4_PGE ; + vcpu->arch.cr4_guest_owned_bits &= ~X86_CR4_PGE ;if (is_guest_mode (&vmx->vcpu)) - vmx->vcpu.arch.cr4_guest_owned_bits &= - ~get_vmcs12 (&vmx->vcpu)->cr4_guest_host_mask; -vmcs_writel (CR4_GUEST_HOST_MASK , ~vmx->vcpu.arch.cr4_guest_owned_bits); + vcpu->arch.cr4_guest_owned_bits &= + ~get_vmcs12 (vcpu)->cr4_guest_host_mask; +vmcs_writel (CR4_GUEST_HOST_MASK , ~vcpu->arch.cr4_guest_owned_bits); }u32 vmx_pin_based_exec_ctrl (struct vcpu_vmx *vmx)
@@ -7233,6 +7236,8 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) } } +set_cr4_guest_host_mask (vmx); +/* Refresh #PF interception to account for MAXPHYADDR changes. */ update_exception_bitmap (vcpu); }
Signature · 1 line
2.28.0
PATCH 5/5 KVM: x86: Let the guest own CR4.FSGSBASE
In reply to Sean ChristophersonFrom: Lai Jiangshan <[email protected]>
Add FSGSBASE to the set of possible guest-owned CR4 bits, i.e. let the guest own it on VMX. KVM never queries the guest's CR4.FSGSBASE value, thus there is no reason to force VM-Exit on FSGSBASE being toggled.
Note, because FSGSBASE is conditionally available, this is dependent on recent changes to intercept reserved CR4 bits and to update the CR4 guest/host mask in response to guest CPUID changes.
--- arch/x86/kvm/kvm_cache_regs.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/kvm_cache_regs.h +1 -1
@@ -7,7 +7,7 @@#define KVM_POSSIBLE_CR0_GUEST_BITS X86_CR0_TS#define KVM_POSSIBLE_CR4_GUEST_BITS \ (X86_CR4_PVI | X86_CR4_DE | X86_CR4_PCE | X86_CR4_OSFXSR \ - | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD) + | X86_CR4_OSXMMEXCPT | X86_CR4_PGE | X86_CR4_TSD | X86_CR4_FSGSBASE)#define BUILD_KVM_GPR_ACCESSORS (lname, uname) \ static __always_inline unsigned long kvm_##lname##_read(struct kvm_vcpu *vcpu)\
Signature · 1 line
2.28.0
Quoted 33 lines · Sean Christopherson
This series stems from Lai's RFC patches to intercept LA57 and let the guest own FSGSBASE[*]. Discussion and inspection revealed that KVM does not handle the case where LA57 is supported in hardware but not exposed to the guest. This is actually true for all CR4 bits, but LA57 is currently the only bit that can be reserved and also owned by the guest. I have a unit test for this that I'll post separately.
Intercepting LA57 was by far the easiest fix for the immedidate bug, and is likely the right change in the long term as there's no justification for letting the guest own LA57.
The middle three patches adjust VMX's CR4 guest/host mask to intercept reserved bits. This required reworking CPUID updates to also refresh said mask at the correct time.
The last past is Lai's, which let's the guest own FSGSBASE. This depends on the reserved bit handling being in place.
Ran everything through unit tests, and ran the kernel's FSGSBASE selftests in a VM.
[*] https://lkml.kernel.org/r/[email protected]
Lai Jiangshan (2):
KVM: x86: Intercept LA57 to inject #GP fault when it's reserved
KVM: x86: Let the guest own CR4.FSGSBASESean Christopherson (3):
KVM: x86: Invoke vendor'svcpu_after_set_cpuid()after all common
updates
KVM: x86: Move call toupdate_exception_bitmap()into VMX code
KVM: VMX: Intercept guest reserved CR4 bits to inject #GP fault
Quoted 5 lines
arch/x86/kvm/cpuid.c | 6 +++--- arch/x86/kvm/kvm_cache_regs.h | 2 +- arch/x86/kvm/vmx/vmx.c | 18 +++++++++++++----- 3 files changed, 17 insertions(+), 9 deletions(-)
Queued, thanks.
Paolo
Quoted 33 lines · Sean Christopherson
This series stems from Lai's RFC patches to intercept LA57 and let the guest own FSGSBASE[*]. Discussion and inspection revealed that KVM does not handle the case where LA57 is supported in hardware but not exposed to the guest. This is actually true for all CR4 bits, but LA57 is currently the only bit that can be reserved and also owned by the guest. I have a unit test for this that I'll post separately.
Intercepting LA57 was by far the easiest fix for the immedidate bug, and is likely the right change in the long term as there's no justification for letting the guest own LA57.
The middle three patches adjust VMX's CR4 guest/host mask to intercept reserved bits. This required reworking CPUID updates to also refresh said mask at the correct time.
The last past is Lai's, which let's the guest own FSGSBASE. This depends on the reserved bit handling being in place.
Ran everything through unit tests, and ran the kernel's FSGSBASE selftests in a VM.
[*] https://lkml.kernel.org/r/[email protected]
Lai Jiangshan (2):
KVM: x86: Intercept LA57 to inject #GP fault when it's reserved
KVM: x86: Let the guest own CR4.FSGSBASESean Christopherson (3):
KVM: x86: Invoke vendor'svcpu_after_set_cpuid()after all common
updates
KVM: x86: Move call toupdate_exception_bitmap()into VMX code
KVM: VMX: Intercept guest reserved CR4 bits to inject #GP fault
Quoted 5 lines
arch/x86/kvm/cpuid.c | 6 +++--- arch/x86/kvm/kvm_cache_regs.h | 2 +- arch/x86/kvm/vmx/vmx.c | 18 +++++++++++++----- 3 files changed, 17 insertions(+), 9 deletions(-)
Queued, thanks.
Paolo