Skip to contentSkip to conversation

KVM: x86: Legal GPA fixes and cleanups

PATCH In review Sean Christopherson 3 participants 21 messages Feb 4, 2021

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

cover KVM: x86: Legal GPA fixes and cleanups

Add helpers to consolidate the GPA reserved bits checks that are scattered all over KVM, and fix a few bugs in the process.

The original motivation was simply to get rid of all the different open coded variations of the checks (there were a lot), but this snowballed into a more ambitious cleanup when I realized common helpers are more or less required to correctly handle repurposed GPA bits, e.g. SEV's C-bit.

The last two patches (use nested VM-Enter failure tracepoints in SVM) aren't directly related to the GPA checks, but the conflicts would be rather messy, so I included them here.

Note, the SEV C-bit changes are technically bug fixes, but getting them in stable kernels would require backporting this entire pile. IMO, it's not worth the effort given that it's extremely unlikely anyone will encounter the bugs in anything but synthetic negative tests.

Based on kvm/queue, commit 3f87cb8253c3 ("KVM: X86: Expose bus lock debug exception to guest").

Sean Christopherson (12):
KVM: x86: Set so called 'reserved CR3 bits in LM mask' at vCPU reset
KVM: nSVM: Don't strip host's C-bit from guest's CR3 when reading
PDPTRs
KVM: x86: Add a helper to check for a legal GPA
KVM: x86: Add a helper to handle legal GPA with an alignment
requirement
KVM: VMX: Use GPA legality helpers to replace open coded equivalents
KVM: nSVM: Use common GPA helper to check for illegal CR3
KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode
KVM: x86: Use reserved_gpa_bits to calculate reserved PxE bits
KVM: x86/mmu: Add helper to generate mask of reserved HPA bits
KVM: x86: Add helper to consolidate "raw" reserved GPA mask
calculations
KVM: x86: Move nVMX's consistency check macro to common code
KVM: nSVM: Trace VM-Enter consistency check failures

 arch/x86/include/asm/kvm_host.h |   2 +-
 arch/x86/kvm/cpuid.c            |  20 +++++-
 arch/x86/kvm/cpuid.h            |  24 +++++--
 arch/x86/kvm/mmu/mmu.c          | 110 ++++++++++++++++----------------
 arch/x86/kvm/mtrr.c             |  12 ++--
 arch/x86/kvm/svm/nested.c       |  35 +++++-----
 arch/x86/kvm/svm/svm.c          |   2 +-
 arch/x86/kvm/vmx/nested.c       |  34 +++-------
 arch/x86/kvm/vmx/vmx.c          |   2 +-
 arch/x86/kvm/x86.c              |  11 ++--
 arch/x86/kvm/x86.h              |   8 +++
 11 files changed, 140 insertions(+), 120 deletions(-)
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#1 lore raw

PATCH 1/12 KVM: x86: Set so called 'reserved CR3 bits in LM mask' at vCPU reset

Set cr3_lm_rsvd_bits, which is effectively an invalid GPA mask, at vCPU reset. The reserved bits check needs to be done even if userspace never configures the guest's CPUID model.

Fixes 0107973a80ad ("KVM: x86: Introduce cr3_lm_rsvd_bits in kvm_vcpu_arch")
---
 arch/x86/kvm/x86.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/x86.c +1 -0
index 667d0042d0b7..e6fbf2f574a6 100644
@@ -10091,6 +10091,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	fx_init(vcpu);
 
 	vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
+	vcpu->arch.cr3_lm_rsvd_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
 
 	vcpu->arch.pat = MSR_IA32_CR_PAT_DEFAULT;
 
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#2 lore raw

PATCH 2/12 KVM: nSVM: Don't strip host's C-bit from guest's CR3 when reading PDPTRs

In reply to Sean Christopherson

Don't clear the SME C-bit when reading a guest PDPTR, as the GPA (CR3) is in the guest domain.

Barring a bizarre paravirtual use case, this is likely a benign bug. SME is not emulated by KVM, loading SEV guest PDPTRs is doomed as KVM can't use the correct key to read guest memory, and setting guest MAXPHYADDR higher than the host, i.e. overlapping the C-bit, would cause faults in the guest.

Note, for SEV guests, stripping the C-bit is technically aligned with CPU behavior, but for KVM it's the greater of two evils. Because KVM doesn't have access to the guest's encryption key, ignoring the C-bit would at best result in KVM reading garbage. By keeping the C-bit, KVM will fail its read (unless userspace creates a memslot with the C-bit set). The guest will still undoubtedly die, as KVM will use '0' for the PDPTR value, but that's preferable to interpreting encrypted data as a PDPTR.

Fixes d0ec49d4de90 ("kvm/x86/svm: Support Secure Memory Encryption within KVM")
Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
---
 arch/x86/kvm/svm/nested.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
index 1ffb28cfe39d..70c72fe61e02 100644
@@ -58,7 +58,7 @@ static u64 nested_svm_get_tdp_pdptr(struct kvm_vcpu *vcpu, int index)
 	u64 pdpte;
 	int ret;
 
-	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(__sme_clr(cr3)), &pdpte,
+	ret = kvm_vcpu_read_guest_page(vcpu, gpa_to_gfn(cr3), &pdpte,
 				       offset_in_page(cr3) + index * 8, 8);
 	if (ret)
 		return 0;
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#3 lore raw

PATCH 3/12 KVM: x86: Add a helper to check for a legal GPA

In reply to Sean Christopherson

Add a helper to check for a legal GPA, and use it to consolidate code in existing, related helpers. Future patches will extend usage to VMX and SVM code, properly handle exceptions to the maxphyaddr rule, and add more helpers.

No functional change intended.

---
 arch/x86/kvm/cpuid.h | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
arch/x86/kvm/cpuid.h +11 -6
index dc921d76e42e..674d61079f2d 100644
@@ -36,9 +36,19 @@ static inline int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
 	return vcpu->arch.maxphyaddr;
 }
 
+static inline bool kvm_vcpu_is_legal_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)
+{
+	return !(gpa >> cpuid_maxphyaddr(vcpu));
+}
+
 static inline bool kvm_vcpu_is_illegal_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)
 {
-	return (gpa >= BIT_ULL(cpuid_maxphyaddr(vcpu)));
+	return !kvm_vcpu_is_legal_gpa(vcpu, gpa);
+}
+
+static inline bool page_address_valid(struct kvm_vcpu *vcpu, gpa_t gpa)
+{
+	return PAGE_ALIGNED(gpa) && kvm_vcpu_is_legal_gpa(vcpu, gpa);
 }
 
 struct cpuid_reg {
@@ -324,11 +334,6 @@ static __always_inline void kvm_cpu_cap_check_and_set(unsigned int x86_feature)
 		kvm_cpu_cap_set(x86_feature);
 }
 
-static inline bool page_address_valid(struct kvm_vcpu *vcpu, gpa_t gpa)
-{
-	return PAGE_ALIGNED(gpa) && !(gpa >> cpuid_maxphyaddr(vcpu));
-}
-
 static __always_inline bool guest_pv_has(struct kvm_vcpu *vcpu,
 					 unsigned int kvm_feature)
 {
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#4 lore raw

PATCH 4/12 KVM: x86: Add a helper to handle legal GPA with an alignment requirement

In reply to Sean Christopherson

Add a helper to genericize checking for a legal GPA that also must conform to an arbitrary alignment, and use it in the existing page_address_valid(). Future patches will replace open coded variants in VMX and SVM.

---
 arch/x86/kvm/cpuid.h | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.h +7 -1
index 674d61079f2d..a9d55ab51e3c 100644
@@ -46,9 +46,15 @@ static inline bool kvm_vcpu_is_illegal_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)
 	return !kvm_vcpu_is_legal_gpa(vcpu, gpa);
 }
 
+static inline bool kvm_vcpu_is_legal_aligned_gpa(struct kvm_vcpu *vcpu,
+						 gpa_t gpa, gpa_t alignment)
+{
+	return IS_ALIGNED(gpa, alignment) && kvm_vcpu_is_legal_gpa(vcpu, gpa);
+}
+
 static inline bool page_address_valid(struct kvm_vcpu *vcpu, gpa_t gpa)
 {
-	return PAGE_ALIGNED(gpa) && kvm_vcpu_is_legal_gpa(vcpu, gpa);
+	return kvm_vcpu_is_legal_aligned_gpa(vcpu, gpa, PAGE_SIZE);
 }
 
 struct cpuid_reg {
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#5 lore raw

PATCH 5/12 KVM: VMX: Use GPA legality helpers to replace open coded equivalents

In reply to Sean Christopherson

Replace a variety of open coded GPA checks with the recently introduced common helpers.

No functional change intended.

---
 arch/x86/kvm/vmx/nested.c | 26 +++++++-------------------
 arch/x86/kvm/vmx/vmx.c    |  2 +-
 2 files changed, 8 insertions(+), 20 deletions(-)
arch/x86/kvm/vmx/nested.c +7 -19
index b14fc19ceb36..b25ce704a2aa 100644
@@ -775,8 +775,7 @@ static int nested_vmx_check_apicv_controls(struct kvm_vcpu *vcpu,
 	   (CC(!nested_cpu_has_vid(vmcs12)) ||
 	    CC(!nested_exit_intr_ack_set(vcpu)) ||
 	    CC((vmcs12->posted_intr_nv & 0xff00)) ||
-	    CC((vmcs12->posted_intr_desc_addr & 0x3f)) ||
-	    CC((vmcs12->posted_intr_desc_addr >> cpuid_maxphyaddr(vcpu)))))
+	    CC(!kvm_vcpu_is_legal_aligned_gpa(vcpu, vmcs12->posted_intr_desc_addr, 64))))
 		return -EINVAL;
 
 	/* tpr shadow is needed by all apicv features. */
@@ -789,13 +788,11 @@ static int nested_vmx_check_apicv_controls(struct kvm_vcpu *vcpu,
 static int nested_vmx_check_msr_switch(struct kvm_vcpu *vcpu,
 				       u32 count, u64 addr)
 {
-	int maxphyaddr;
-
 	if (count == 0)
 		return 0;
-	maxphyaddr = cpuid_maxphyaddr(vcpu);
-	if (!IS_ALIGNED(addr, 16) || addr >> maxphyaddr ||
-	    (addr + count * sizeof(struct vmx_msr_entry) - 1) >> maxphyaddr)
+
+	if (!kvm_vcpu_is_legal_aligned_gpa(vcpu, addr, 16) ||
+	    !kvm_vcpu_is_legal_gpa(vcpu, (addr + count * sizeof(struct vmx_msr_entry) - 1)))
 		return -EINVAL;
 
 	return 0;
@@ -1093,14 +1090,6 @@ static void prepare_vmx_msr_autostore_list(struct kvm_vcpu *vcpu,
 	}
 }
 
-static bool nested_cr3_valid(struct kvm_vcpu *vcpu, unsigned long val)
-{
-	unsigned long invalid_mask;
-
-	invalid_mask = (~0ULL) << cpuid_maxphyaddr(vcpu);
-	return (val & invalid_mask) == 0;
-}
-
 /*
  * Returns true if the MMU needs to be sync'd on nested VM-Enter/VM-Exit.
  * tl;dr: the MMU needs a sync if L0 is using shadow paging and L1 didn't
@@ -1152,7 +1141,7 @@ static bool nested_vmx_transition_mmu_sync(struct kvm_vcpu *vcpu)
 static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool nested_ept,
 			       enum vm_entry_failure_code *entry_failure_code)
 {
-	if (CC(!nested_cr3_valid(vcpu, cr3))) {
+	if (CC(kvm_vcpu_is_illegal_gpa(vcpu, cr3))) {
 		*entry_failure_code = ENTRY_FAIL_DEFAULT;
 		return -EINVAL;
 	}
@@ -2666,7 +2655,6 @@ static int nested_vmx_check_nmi_controls(struct vmcs12 *vmcs12)
 static bool nested_vmx_check_eptp(struct kvm_vcpu *vcpu, u64 new_eptp)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
-	int maxphyaddr = cpuid_maxphyaddr(vcpu);
 
 	/* Check for memory type validity */
 	switch (new_eptp & VMX_EPTP_MT_MASK) {
@@ -2697,7 +2685,7 @@ static bool nested_vmx_check_eptp(struct kvm_vcpu *vcpu, u64 new_eptp)
 	}
 
 	/* Reserved bits should not be set */
-	if (CC(new_eptp >> maxphyaddr || ((new_eptp >> 7) & 0x1f)))
+	if (CC(kvm_vcpu_is_illegal_gpa(vcpu, new_eptp) || ((new_eptp >> 7) & 0x1f)))
 		return false;
 
 	/* AD, if set, should be supported */
@@ -2881,7 +2869,7 @@ static int nested_vmx_check_host_state(struct kvm_vcpu *vcpu,
 
 	if (CC(!nested_host_cr0_valid(vcpu, vmcs12->host_cr0)) ||
 	    CC(!nested_host_cr4_valid(vcpu, vmcs12->host_cr4)) ||
-	    CC(!nested_cr3_valid(vcpu, vmcs12->host_cr3)))
+	    CC(kvm_vcpu_is_illegal_gpa(vcpu, vmcs12->host_cr3)))
 		return -EINVAL;
 
 	if (CC(is_noncanonical_address(vmcs12->host_ia32_sysenter_esp, vcpu)) ||
arch/x86/kvm/vmx/vmx.c +1 -1
index beb5a912014d..cbeb0748f25f 100644
@@ -1114,7 +1114,7 @@ static inline bool pt_can_write_msr(struct vcpu_vmx *vmx)
 static inline bool pt_output_base_valid(struct kvm_vcpu *vcpu, u64 base)
 {
 	/* The base must be 128-byte aligned and a legal physical address. */
-	return !kvm_vcpu_is_illegal_gpa(vcpu, base) && !(base & 0x7f);
+	return kvm_vcpu_is_legal_aligned_gpa(vcpu, base, 128);
 }
 
 static inline void pt_load_msr(struct pt_ctx *ctx, u32 addr_range)
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#6 lore raw

PATCH 6/12 KVM: nSVM: Use common GPA helper to check for illegal CR3

In reply to Sean Christopherson

Replace an open coded check for an invalid CR3 with its equivalent helper.

No functional change intended.

---
 arch/x86/kvm/svm/nested.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +1 -1
index 70c72fe61e02..ac662964cee5 100644
@@ -367,7 +367,7 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm)
 static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,
 			       bool nested_npt)
 {
-	if (cr3 & rsvd_bits(cpuid_maxphyaddr(vcpu), 63))
+	if (kvm_vcpu_is_illegal_gpa(vcpu, cr3))
 		return -EINVAL;
 
 	if (!nested_npt && is_pae_paging(vcpu) &&
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#7 lore raw

PATCH 7/12 KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

In reply to Sean Christopherson

Rename cr3_lm_rsvd_bits to reserved_gpa_bits, and use it for all GPA legality checks. AMD's APM states:

If the C-bit is an address bit, this bit is masked from the guest physical address when it is translated through the nested page tables.

Thus, any access that can conceivably be run through NPT should ignore the C-bit when checking for validity.

For features that KVM emulates in software, e.g. MTRRs, there is no clear direction in the APM for how the C-bit should be handled. For such cases, follow the SME behavior inasmuch as possible, since SEV is is essentially a VM-specific variant of SME. For SME, the APM states:

In this case the upper physical address bits are treated as reserved when the feature is enabled except where otherwise indicated.

Collecting the various relavant SME snippets in the APM and cross- referencing the omissions with Linux kernel code, this leaves MTTRs and APIC_BASE as the only flows that KVM emulates that should _not_ ignore the C-bit.

Note, this means the reserved bit checks in the page tables are technically broken. This will be remedied in a future patch.

Although the page table checks are technically broken, in practice, it's all but guaranteed to be irrelevant. NPT is required for SEV, i.e. shadowing page tables isn't needed in the common case. Theoretically, the checks could be in play for nested NPT, but it's extremely unlikely that anyone is running nested VMs on SEV, as doing so would require L1 to expose sensitive data to L0, e.g. the entire VMCB. And if anyone is running nested VMs, L0 can't read the guest's encrypted memory, i.e. L1 would need to put its NPT in shared memory, in which case the C-bit will never be set. Or, L1 could use shadow paging, but again, if L0 needs to read page tables, e.g. to load PDPTRs, the memory can't be encrypted if L1 has any expectation of L0 doing the right thing.

Cc Tom Lendacky <[email protected]>
Cc Brijesh Singh <[email protected]>
---
 arch/x86/include/asm/kvm_host.h | 2 +-
 arch/x86/kvm/cpuid.c            | 2 +-
 arch/x86/kvm/cpuid.h            | 2 +-
 arch/x86/kvm/svm/nested.c       | 2 +-
 arch/x86/kvm/svm/svm.c          | 2 +-
 arch/x86/kvm/x86.c              | 7 +++----
 6 files changed, 8 insertions(+), 9 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 915f716e78e6..1653d49a66ff 100644
@@ -654,7 +654,7 @@ struct kvm_vcpu_arch {
 	int cpuid_nent;
 	struct kvm_cpuid_entry2 *cpuid_entries;
 
-	unsigned long cr3_lm_rsvd_bits;
+	u64 reserved_gpa_bits;
 	int maxphyaddr;
 	int max_tdp_level;
 
arch/x86/kvm/cpuid.c +1 -1
index 944f518ca91b..7bd1331c1bbc 100644
@@ -194,7 +194,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
-	vcpu->arch.cr3_lm_rsvd_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
+	vcpu->arch.reserved_gpa_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
 
 	/* Invoke the vendor callback only after the above state is updated. */
 	static_call(kvm_x86_vcpu_after_set_cpuid)(vcpu);
arch/x86/kvm/cpuid.h +1 -1
index a9d55ab51e3c..f673f45bdf52 100644
@@ -38,7 +38,7 @@ static inline int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
 
 static inline bool kvm_vcpu_is_legal_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)
 {
-	return !(gpa >> cpuid_maxphyaddr(vcpu));
+	return !(gpa & vcpu->arch.reserved_gpa_bits);
 }
 
 static inline bool kvm_vcpu_is_illegal_gpa(struct kvm_vcpu *vcpu, gpa_t gpa)
arch/x86/kvm/svm/nested.c +1 -1
index ac662964cee5..add3cd4295e1 100644
@@ -241,7 +241,7 @@ static bool nested_vmcb_check_cr3_cr4(struct vcpu_svm *svm,
 	 */
 	if ((save->efer & EFER_LME) && (save->cr0 & X86_CR0_PG)) {
 		if (!(save->cr4 & X86_CR4_PAE) || !(save->cr0 & X86_CR0_PE) ||
-		    (save->cr3 & vcpu->arch.cr3_lm_rsvd_bits))
+		    kvm_vcpu_is_illegal_gpa(vcpu, save->cr3))
 			return false;
 	}
 
arch/x86/kvm/svm/svm.c +1 -1
index f53e6377a933..50ad5a3bf880 100644
@@ -4079,7 +4079,7 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	if (sev_guest(vcpu->kvm)) {
 		best = kvm_find_cpuid_entry(vcpu, 0x8000001F, 0);
 		if (best)
-			vcpu->arch.cr3_lm_rsvd_bits &= ~(1UL << (best->ebx & 0x3f));
+			vcpu->arch.reserved_gpa_bits &= ~(1UL << (best->ebx & 0x3f));
 	}
 
 	if (!kvm_vcpu_apicv_active(vcpu))
arch/x86/kvm/x86.c +3 -4
index e6fbf2f574a6..1da7ed093650 100644
@@ -1082,8 +1082,7 @@ int kvm_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)
 		return 0;
 	}
 
-	if (is_long_mode(vcpu) &&
-	    (cr3 & vcpu->arch.cr3_lm_rsvd_bits))
+	if (is_long_mode(vcpu) && kvm_vcpu_is_illegal_gpa(vcpu, cr3))
 		return 1;
 	else if (is_pae_paging(vcpu) &&
 		 !load_pdptrs(vcpu, vcpu->arch.walk_mmu, cr3))
@@ -9712,7 +9711,7 @@ static bool kvm_is_valid_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs)
 		 */
 		if (!(sregs->cr4 & X86_CR4_PAE) || !(sregs->efer & EFER_LMA))
 			return false;
-		if (sregs->cr3 & vcpu->arch.cr3_lm_rsvd_bits)
+		if (kvm_vcpu_is_illegal_gpa(vcpu, sregs->cr3))
 			return false;
 	} else {
 		/*
@@ -10091,7 +10090,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	fx_init(vcpu);
 
 	vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
-	vcpu->arch.cr3_lm_rsvd_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
+	vcpu->arch.reserved_gpa_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
 
 	vcpu->arch.pat = MSR_IA32_CR_PAT_DEFAULT;
 
Signature · 1 line
2.30.0.365.g02bc693789-goog
#8 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 3 lines · Sean Christopherson


-       unsigned long cr3_lm_rsvd_bits;
+       u64 reserved_gpa_bits;

LAM defines bits above the GFN in CR3:
https://software.intel.com/content/www/us/en/develop/download/intel-architecture-instruction-set-extensions-programming-reference.html

KVM doesn't support this today of course, but it might be confusing to try to combine the two concepts.

author maintainer
#9 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 10 lines · Rick P
Quoted 3 lines · Sean Christopherson


-       unsigned long cr3_lm_rsvd_bits;
+       u64 reserved_gpa_bits;

LAM defines bits above the GFN in CR3:
https://software.intel.com/content/www/us/en/develop/download/intel-architecture-instruction-set-extensions-programming-reference.html

KVM doesn't support this today of course, but it might be confusing to try to combine the two concepts.

Ah, took me a few minutes, but I see what you're saying. LAM will introduce bits that are repurposed for CR3, but not generic GPAs. And, the behavior is based on CPU support, so it'd make sense to have a mask cached in vcpu->arch as opposed to constantly generating it on the fly.

Definitely agree that having a separate cr3_lm_rsvd_bits or whatever is the right way to go when LAM comes along. Not sure it's worth keeping a duplicate field in the meantime, though it would avoid a small amount of thrash.

Paolo, any thoughts?

maintainer
#10 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 8 lines · Sean Christopherson

Ah, took me a few minutes, but I see what you're saying. LAM will introduce bits that are repurposed for CR3, but not generic GPAs. And, the behavior is based on CPU support, so it'd make sense to have a mask cached in vcpu->arch as opposed to constantly generating it on the fly.

Definitely agree that having a separate cr3_lm_rsvd_bits or whatever is the right way to go when LAM comes along. Not sure it's worth keeping a duplicate field in the meantime, though it would avoid a small amount of thrash.

We don't even know if the cr3_lm_rsvd_bits would be a field in vcpu->arch, or rather computed on the fly. So renaming the field in vcpu->arch seems like the simplest thing to do now.

Paolo

#11 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 16 lines · Paolo Bonzini
Quoted 8 lines · Sean Christopherson

Ah, took me a few minutes, but I see what you're saying.  LAM will introduce
bits that are repurposed for CR3, but not generic GPAs.  And, the behavior is
based on CPU support, so it'd make sense to have a mask cached in vcpu->arch
as opposed to constantly generating it on the fly.

Quoted 6 lines

Definitely agree that having a separate cr3_lm_rsvd_bits or whatever is the
right way to go when LAM comes along.  Not sure it's worth keeping a duplicate
field in the meantime, though it would avoid a small amount of thrash.

Quoted 3 lines

We don't even know if the cr3_lm_rsvd_bits would be a field in vcpu->arch, or rather computed on the fly.  So renaming the field in vcpu->arch seems like the simplest thing to do now.

Fair enough. But just to clarify, I meant that I thought the code would be more confusing to use illegal gpa bit checks for checking cr3. It seems they are only incidentally the same value. Alternatively there could be something like a is_rsvd_cr3_bits() helper that just uses reserved_gpa_bits for now. Probably put the comment in the wrong place. It's a minor point in any case.

author maintainer
#12 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 17 lines · Rick P
Quoted 16 lines · Paolo Bonzini
Quoted 14 lines · Sean Christopherson

Ah, took me a few minutes, but I see what you're saying.  LAM will introduce
bits that are repurposed for CR3, but not generic GPAs.  And, the behavior is
based on CPU support, so it'd make sense to have a mask cached in vcpu->arch
as opposed to constantly generating it on the fly.

Definitely agree that having a separate cr3_lm_rsvd_bits or whatever is the
right way to go when LAM comes along.  Not sure it's worth keeping a duplicate
field in the meantime, though it would avoid a small amount of thrash.

Quoted 7 lines
Quoted 3 lines

We don't even know if the cr3_lm_rsvd_bits would be a field in vcpu->arch, or rather computed on the fly.  So renaming the field in vcpu->arch seems like the simplest thing to do now.

Fair enough. But just to clarify, I meant that I thought the code would be more confusing to use illegal gpa bit checks for checking cr3. It seems they are only incidentally the same value.

Hmm, yeah, bits 63:52 are incidental. Bits 52:M are not, though. If/when we need to special case CR3, I would like to take a similar approach to __reset_rsvds_bits_mask(), where the high reserved bits start from reserved_gpa_bits and mask off the bits that can't be encoded into a PxE.

Quoted 3 lines

Alternatively there could be something like a is_rsvd_cr3_bits() helper that just uses reserved_gpa_bits for now. Probably put the comment in the wrong place. It's a minor point in any case.

That thought crossed my mind, too. Maybe kvm_vcpu_is_illegal_cr3() to match the gpa helpers?

maintainer
#13 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 5 lines · Sean Christopherson
Quoted 3 lines

Alternatively there could be something like a is_rsvd_cr3_bits() helper that just uses reserved_gpa_bits for now. Probably put the comment in the wrong place. It's a minor point in any case.

That thought crossed my mind, too. Maybe kvm_vcpu_is_illegal_cr3() to match the gpa helpers?

Yes, that's certainly a good name but it doesn't have to be done now. Or at least, if you do it, somebody is guaranteed to send a patch after one month because the wrapper is useless. :)

Paolo

author maintainer
#14 lore raw

Re: [PATCH 07/12] KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode

Quoted 10 lines · Paolo Bonzini
Quoted 5 lines · Sean Christopherson
Quoted 3 lines

Alternatively there could be something like a is_rsvd_cr3_bits() helper that just uses reserved_gpa_bits for now. Probably put the comment in the wrong place. It's a minor point in any case.

That thought crossed my mind, too. Maybe kvm_vcpu_is_illegal_cr3() to match the gpa helpers?

Yes, that's certainly a good name but it doesn't have to be done now. Or at least, if you do it, somebody is guaranteed to send a patch after one month because the wrapper is useless. :)

LOL, I can see myself doing that...

author maintainer
#15 lore raw

PATCH 8/12 KVM: x86: Use reserved_gpa_bits to calculate reserved PxE bits

In reply to Sean Christopherson

Use reserved_gpa_bits, which accounts for exceptions to the maxphyaddr rule, e.g. SEV's C-bit, for the page {table,directory,etc...} entry (PxE) reserved bits checks. For SEV, the C-bit is ignored by hardware when walking pages tables, e.g. the APM states:

Note that while the guest may choose to set the C-bit explicitly on instruction pages and page table addresses, the value of this bit is a don't-care in such situations as hardware always performs these as private accesses.

Such behavior is expected to hold true for other features that repurpose GPA bits, e.g. KVM could theoretically emulate SME or MKTME, which both allow non-zero repurposed bits in the page tables. Conceptually, KVM should apply reserved GPA checks universally, and any features that do not adhere to the basic rule should be explicitly handled, i.e. if a GPA bit is repurposed but not allowed in page tables for whatever reason.

Refactor __reset_rsvds_bits_mask() to take the pre-generated reserved bits mask, and opportunistically clean up its code, e.g. to align lines and comments.

Practically speaking, this is change is a likely a glorified nop given the current KVM code base. SEV's C-bit is the only repurposed GPA bit, and KVM doesn't support shadowing encrypted page tables (which is theoretically possible via SEV debug APIs).

---
 arch/x86/kvm/cpuid.c   |  10 ++--
 arch/x86/kvm/mmu/mmu.c | 104 ++++++++++++++++++++---------------------
 arch/x86/kvm/x86.c     |   3 +-
 3 files changed, 58 insertions(+), 59 deletions(-)
arch/x86/kvm/cpuid.c +7 -3
index 7bd1331c1bbc..d313b1804278 100644
@@ -188,16 +188,20 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	kvm_update_pv_runtime(vcpu);
 
 	vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
-	kvm_mmu_reset_context(vcpu);
+	vcpu->arch.reserved_gpa_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
 
 	kvm_pmu_refresh(vcpu);
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
-	vcpu->arch.reserved_gpa_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
-
 	/* Invoke the vendor callback only after the above state is updated. */
 	static_call(kvm_x86_vcpu_after_set_cpuid)(vcpu);
+
+	/*
+	 * Except for the MMU, which needs to be reset after any vendor
+	 * specific adjustments to the reserved GPA bits.
+	 */
+	kvm_mmu_reset_context(vcpu);
 }
 
 static int is_efer_nx(void)
arch/x86/kvm/mmu/mmu.c +50 -54

184 lines not on this page · raw message

arch/x86/kvm/x86.c +1 -2
index 1da7ed093650..82a70511c0d3 100644
@@ -761,8 +761,7 @@ static int kvm_read_nested_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn,
 
 static inline u64 pdptr_rsvd_bits(struct kvm_vcpu *vcpu)
 {
-	return rsvd_bits(cpuid_maxphyaddr(vcpu), 63) | rsvd_bits(5, 8) |
-	       rsvd_bits(1, 2);
+	return vcpu->arch.reserved_gpa_bits | rsvd_bits(5, 8) | rsvd_bits(1, 2);
 }
 
 /*
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#16 lore raw

PATCH 9/12 KVM: x86/mmu: Add helper to generate mask of reserved HPA bits

In reply to Sean Christopherson

Add a helper to generate the mask of reserved PA bits in the host.

No functional change intended.

---
 arch/x86/kvm/mmu/mmu.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)
arch/x86/kvm/mmu/mmu.c +9 -5
index d462db3bc742..86af58294272 100644
@@ -4123,6 +4123,11 @@ static void reset_rsvds_bits_mask_ept(struct kvm_vcpu *vcpu,
 				    vcpu->arch.reserved_gpa_bits, execonly);
 }
 
+static inline u64 reserved_hpa_bits(void)
+{
+	return rsvd_bits(shadow_phys_bits, 63);
+}
+
 /*
  * the page table on host is the shadow page table for the page
  * table in guest or amd nested guest, its mmu features completely
@@ -4142,7 +4147,7 @@ reset_shadow_zero_bits_mask(struct kvm_vcpu *vcpu, struct kvm_mmu *context)
 	 */
 	shadow_zero_check = &context->shadow_zero_check;
 	__reset_rsvds_bits_mask(vcpu, shadow_zero_check,
-				rsvd_bits(shadow_phys_bits, 63),
+				reserved_hpa_bits(),
 				context->shadow_root_level, uses_nx,
 				guest_cpuid_has(vcpu, X86_FEATURE_GBPAGES),
 				is_pse(vcpu), true);
@@ -4179,14 +4184,13 @@ reset_tdp_shadow_zero_bits_mask(struct kvm_vcpu *vcpu,
 
 	if (boot_cpu_is_amd())
 		__reset_rsvds_bits_mask(vcpu, shadow_zero_check,
-					rsvd_bits(shadow_phys_bits, 63),
+					reserved_hpa_bits(),
 					context->shadow_root_level, false,
 					boot_cpu_has(X86_FEATURE_GBPAGES),
 					true, true);
 	else
 		__reset_rsvds_bits_mask_ept(shadow_zero_check,
-					    rsvd_bits(shadow_phys_bits, 63),
-					    false);
+					    reserved_hpa_bits(), false);
 
 	if (!shadow_me_mask)
 		return;
@@ -4206,7 +4210,7 @@ reset_ept_shadow_zero_bits_mask(struct kvm_vcpu *vcpu,
 				struct kvm_mmu *context, bool execonly)
 {
 	__reset_rsvds_bits_mask_ept(&context->shadow_zero_check,
-				    rsvd_bits(shadow_phys_bits, 63), execonly);
+				    reserved_hpa_bits(), execonly);
 }
 
 #define BYTE_MASK(access) \
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#17 lore raw

PATCH 10/12 KVM: x86: Add helper to consolidate "raw" reserved GPA mask calculations

In reply to Sean Christopherson

Add a helper to generate the mask of reserved GPA bits _without_ any adjustments for repurposed bits, and use it to replace a variety of open coded variants in the MTRR and APIC_BASE flows.

No functional change intended.

---
 arch/x86/kvm/cpuid.c | 12 +++++++++++-
 arch/x86/kvm/cpuid.h |  1 +
 arch/x86/kvm/mtrr.c  | 12 ++++++------
 arch/x86/kvm/x86.c   |  4 ++--
 4 files changed, 20 insertions(+), 9 deletions(-)
arch/x86/kvm/cpuid.c +11 -1
index d313b1804278..dd9406450696 100644
@@ -188,7 +188,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	kvm_update_pv_runtime(vcpu);
 
 	vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
-	vcpu->arch.reserved_gpa_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
+	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
 	vcpu->arch.cr4_guest_rsvd_bits =
@@ -242,6 +242,16 @@ int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu)
 	return 36;
 }
 
+/*
+ * This "raw" version returns the reserved GPA bits without any adjustments for
+ * encryption technologies that usurp bits.  The raw mask should be used if and
+ * only if hardware does _not_ strip the usurped bits, e.g. in virtual MTRRs.
+ */
+u64 kvm_vcpu_reserved_gpa_bits_raw(struct kvm_vcpu *vcpu)
+{
+	return rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
+}
+
 /* when an old userspace process fills a new kernel module */
 int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu,
 			     struct kvm_cpuid *cpuid,
arch/x86/kvm/cpuid.h +1 -0
index f673f45bdf52..2a0c5064497f 100644
@@ -30,6 +30,7 @@ bool kvm_cpuid(struct kvm_vcpu *vcpu, u32 *eax, u32 *ebx,
 	       u32 *ecx, u32 *edx, bool exact_only);
 
 int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu);
+u64 kvm_vcpu_reserved_gpa_bits_raw(struct kvm_vcpu *vcpu);
 
 static inline int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
 {
arch/x86/kvm/mtrr.c +6 -6
index f472fdb6ae7e..a8502e02f479 100644
@@ -75,7 +75,7 @@ bool kvm_mtrr_valid(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 	/* variable MTRRs */
 	WARN_ON(!(msr >= 0x200 && msr < 0x200 + 2 * KVM_NR_VAR_MTRR));
 
-	mask = (~0ULL) << cpuid_maxphyaddr(vcpu);
+	mask = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 	if ((msr & 1) == 0) {
 		/* MTRR base */
 		if (!valid_mtrr_type(data & 0xff))
@@ -351,14 +351,14 @@ static void set_var_mtrr_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 	if (var_mtrr_range_is_valid(cur))
 		list_del(&mtrr_state->var_ranges[index].node);
 
-	/* Extend the mask with all 1 bits to the left, since those
-	 * bits must implicitly be 0.  The bits are then cleared
-	 * when reading them.
+	/*
+	 * Set all illegal GPA bits in the mask, since those bits must
+	 * implicitly be 0.  The bits are then cleared when reading them.
 	 */
 	if (!is_mtrr_mask)
 		cur->base = data;
 	else
-		cur->mask = data | (-1LL << cpuid_maxphyaddr(vcpu));
+		cur->mask = data | kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	/* add it to the list if it's enabled. */
 	if (var_mtrr_range_is_valid(cur)) {
@@ -426,7 +426,7 @@ int kvm_mtrr_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 		else
 			*pdata = vcpu->arch.mtrr_state.var_ranges[index].mask;
 
-		*pdata &= (1ULL << cpuid_maxphyaddr(vcpu)) - 1;
+		*pdata &= ~kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 	}
 
 	return 0;
arch/x86/kvm/x86.c +2 -2
index 82a70511c0d3..28fea7ff7a86 100644
@@ -408,7 +408,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	enum lapic_mode old_mode = kvm_get_apic_mode(vcpu);
 	enum lapic_mode new_mode = kvm_apic_mode(msr_info->data);
-	u64 reserved_bits = ((~0ULL) << cpuid_maxphyaddr(vcpu)) | 0x2ff |
+	u64 reserved_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu) | 0x2ff |
 		(guest_cpuid_has(vcpu, X86_FEATURE_X2APIC) ? 0 : X2APIC_ENABLE);
 
 	if ((msr_info->data & reserved_bits) != 0 || new_mode == LAPIC_MODE_INVALID)
@@ -10089,7 +10089,7 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
 	fx_init(vcpu);
 
 	vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
-	vcpu->arch.reserved_gpa_bits = rsvd_bits(cpuid_maxphyaddr(vcpu), 63);
+	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	vcpu->arch.pat = MSR_IA32_CR_PAT_DEFAULT;
 
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#18 lore raw

PATCH 11/12 KVM: x86: Move nVMX's consistency check macro to common code

In reply to Sean Christopherson

Move KVM's CC() macro to x86.h so that it can be reused by nSVM. Debugging VM-Enter is as painful on SVM as it is on VMX.

Rename the more visible macro to KVM_NESTED_VMENTER_CONSISTENCY_CHECK to avoid any collisions with the uber-concise "CC".

---
 arch/x86/kvm/vmx/nested.c | 8 +-------
 arch/x86/kvm/x86.h        | 8 ++++++++
 2 files changed, 9 insertions(+), 7 deletions(-)
arch/x86/kvm/vmx/nested.c +1 -7
index b25ce704a2aa..dbca1687ae8e 100644
@@ -21,13 +21,7 @@ module_param_named(enable_shadow_vmcs, enable_shadow_vmcs, bool, S_IRUGO);
 static bool __read_mostly nested_early_check = 0;
 module_param(nested_early_check, bool, S_IRUGO);
 
-#define CC(consistency_check)						\
-({									\
-	bool failed = (consistency_check);				\
-	if (failed)							\
-		trace_kvm_nested_vmenter_failed(#consistency_check, 0);	\
-	failed;								\
-})
+#define CC KVM_NESTED_VMENTER_CONSISTENCY_CHECK
 
 /*
  * Hyper-V requires all of these, so mark them as supported even though
arch/x86/kvm/x86.h +8 -0
index 4f875f8d93b3..a14da36a30ed 100644
@@ -8,6 +8,14 @@
 #include "kvm_cache_regs.h"
 #include "kvm_emulate.h"
 
+#define KVM_NESTED_VMENTER_CONSISTENCY_CHECK(consistency_check)		\
+({									\
+	bool failed = (consistency_check);				\
+	if (failed)							\
+		trace_kvm_nested_vmenter_failed(#consistency_check, 0);	\
+	failed;								\
+})
+
 #define KVM_DEFAULT_PLE_GAP		128
 #define KVM_VMX_DEFAULT_PLE_WINDOW	4096
 #define KVM_DEFAULT_PLE_WINDOW_GROW	2
Signature · 1 line
2.30.0.365.g02bc693789-goog
author maintainer
#19 lore raw

PATCH 12/12 KVM: nSVM: Trace VM-Enter consistency check failures

In reply to Sean Christopherson

Use trace_kvm_nested_vmenter_failed() and its macro magic to trace consistency check failures on nested VMRUN. Tracing such failures by running the buggy VMM as a KVM guest is often the only way to get a precise explanation of why VMRUN failed.

---
 arch/x86/kvm/svm/nested.c | 33 +++++++++++++++++++--------------
 1 file changed, 19 insertions(+), 14 deletions(-)
arch/x86/kvm/svm/nested.c +19 -14
index add3cd4295e1..16fea02471a7 100644
@@ -29,6 +29,8 @@
 #include "lapic.h"
 #include "svm.h"
 
+#define CC KVM_NESTED_VMENTER_CONSISTENCY_CHECK
+
 static void nested_svm_inject_npf_exit(struct kvm_vcpu *vcpu,
 				       struct x86_exception *fault)
 {
@@ -216,14 +218,13 @@ static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)
 
 static bool nested_vmcb_check_controls(struct vmcb_control_area *control)
 {
-	if ((vmcb_is_intercept(control, INTERCEPT_VMRUN)) == 0)
+	if (CC(!vmcb_is_intercept(control, INTERCEPT_VMRUN)))
 		return false;
 
-	if (control->asid == 0)
+	if (CC(control->asid == 0))
 		return false;
 
-	if ((control->nested_ctl & SVM_NESTED_CTL_NP_ENABLE) &&
-	    !npt_enabled)
+	if (CC((control->nested_ctl & SVM_NESTED_CTL_NP_ENABLE) && !npt_enabled))
 		return false;
 
 	return true;
@@ -240,32 +241,36 @@ static bool nested_vmcb_check_cr3_cr4(struct vcpu_svm *svm,
 	 * CR0.PG && EFER.LME.
 	 */
 	if ((save->efer & EFER_LME) && (save->cr0 & X86_CR0_PG)) {
-		if (!(save->cr4 & X86_CR4_PAE) || !(save->cr0 & X86_CR0_PE) ||
-		    kvm_vcpu_is_illegal_gpa(vcpu, save->cr3))
+		if (CC(!(save->cr4 & X86_CR4_PAE)) ||
+		    CC(!(save->cr0 & X86_CR0_PE)) ||
+		    CC(kvm_vcpu_is_illegal_gpa(vcpu, save->cr3)))
 			return false;
 	}
 
-	return kvm_is_valid_cr4(&svm->vcpu, save->cr4);
+	if (CC(!kvm_is_valid_cr4(vcpu, save->cr4)))
+		return false;
+
+	return true;
 }
 
 /* Common checks that apply to both L1 and L2 state.  */
 static bool nested_vmcb_valid_sregs(struct vcpu_svm *svm,
 				    struct vmcb_save_area *save)
 {
-	if (!(save->efer & EFER_SVME))
+	if (CC(!(save->efer & EFER_SVME)))
 		return false;
 
-	if (((save->cr0 & X86_CR0_CD) == 0 && (save->cr0 & X86_CR0_NW)) ||
-	    (save->cr0 & ~0xffffffffULL))
+	if (CC((save->cr0 & X86_CR0_CD) == 0 && (save->cr0 & X86_CR0_NW)) ||
+	    CC(save->cr0 & ~0xffffffffULL))
 		return false;
 
-	if (!kvm_dr6_valid(save->dr6) || !kvm_dr7_valid(save->dr7))
+	if (CC(!kvm_dr6_valid(save->dr6)) || CC(!kvm_dr7_valid(save->dr7)))
 		return false;
 
 	if (!nested_vmcb_check_cr3_cr4(svm, save))
 		return false;
 
-	if (!kvm_valid_efer(&svm->vcpu, save->efer))
+	if (CC(!kvm_valid_efer(&svm->vcpu, save->efer)))
 		return false;
 
 	return true;
@@ -367,12 +372,12 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm)
 static int nested_svm_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3,
 			       bool nested_npt)
 {
-	if (kvm_vcpu_is_illegal_gpa(vcpu, cr3))
+	if (CC(kvm_vcpu_is_illegal_gpa(vcpu, cr3)))
 		return -EINVAL;
 
 	if (!nested_npt && is_pae_paging(vcpu) &&
 	    (cr3 != kvm_read_cr3(vcpu) || pdptrs_changed(vcpu))) {
-		if (!load_pdptrs(vcpu, vcpu->arch.walk_mmu, cr3))
+		if (CC(!load_pdptrs(vcpu, vcpu->arch.walk_mmu, cr3)))
 			return -EINVAL;
 	}
 
Signature · 1 line
2.30.0.365.g02bc693789-goog
maintainer
#20 lore raw
In reply to Sean Christopherson
Quoted 12 lines · Sean Christopherson

Add helpers to consolidate the GPA reserved bits checks that are scattered all over KVM, and fix a few bugs in the process.

The original motivation was simply to get rid of all the different open coded variations of the checks (there were a lot), but this snowballed into a more ambitious cleanup when I realized common helpers are more or less required to correctly handle repurposed GPA bits, e.g. SEV's C-bit.

The last two patches (use nested VM-Enter failure tracepoints in SVM) aren't directly related to the GPA checks, but the conflicts would be rather messy, so I included them here.

Quoted 7 lines

Note, the SEV C-bit changes are technically bug fixes, but getting them in stable kernels would require backporting this entire pile. IMO, it's not worth the effort given that it's extremely unlikely anyone will encounter the bugs in anything but synthetic negative tests.

Based on kvm/queue, commit 3f87cb8253c3 ("KVM: X86: Expose bus lock debug exception to guest").

Queued 1 for 5.11 and 2-10 for 5.12; the VMCB01/VMCB02 patches are unlikely to make it in 5.12 so 11-12 won't be in kvm/next anytime soon---but you don't have to care about them anyway.

Paolo

Quoted 30 lines

Sean Christopherson (12):
KVM: x86: Set so called 'reserved CR3 bits in LM mask' at vCPU reset
KVM: nSVM: Don't strip host's C-bit from guest's CR3 when reading
PDPTRs
KVM: x86: Add a helper to check for a legal GPA
KVM: x86: Add a helper to handle legal GPA with an alignment
requirement
KVM: VMX: Use GPA legality helpers to replace open coded equivalents
KVM: nSVM: Use common GPA helper to check for illegal CR3
KVM: x86: SEV: Treat C-bit as legal GPA bit regardless of vCPU mode
KVM: x86: Use reserved_gpa_bits to calculate reserved PxE bits
KVM: x86/mmu: Add helper to generate mask of reserved HPA bits
KVM: x86: Add helper to consolidate "raw" reserved GPA mask
calculations
KVM: x86: Move nVMX's consistency check macro to common code
KVM: nSVM: Trace VM-Enter consistency check failures

  arch/x86/include/asm/kvm_host.h |   2 +-
  arch/x86/kvm/cpuid.c            |  20 +++++-
  arch/x86/kvm/cpuid.h            |  24 +++++--
  arch/x86/kvm/mmu/mmu.c          | 110 ++++++++++++++++----------------
  arch/x86/kvm/mtrr.c             |  12 ++--
  arch/x86/kvm/svm/nested.c       |  35 +++++-----
  arch/x86/kvm/svm/svm.c          |   2 +-
  arch/x86/kvm/vmx/nested.c       |  34 +++-------
  arch/x86/kvm/vmx/vmx.c          |   2 +-
  arch/x86/kvm/x86.c              |  11 ++--
  arch/x86/kvm/x86.h              |   8 +++
  11 files changed, 140 insertions(+), 120 deletions(-)