Skip to contentSkip to conversation

KVM: x86: CET vs. nVMX fix and hardening

PATCH Applied Sean Christopherson 3 participants 17 messages Jan 28 – Feb 4

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

v2 cover KVM: x86: CET vs. nVMX fix and hardening

Fix a bug where KVM will clear IBT and SHSTK bits after nested VMX MSRs have been configured, e.g. if the kernel is built with CONFIG_X86_CET=y but CONFIG_X86_KERNEL_IBT=n. The late clearing results in kvm-intel.ko refusing to load as the CPU compatible checks generate their VMCS configs with IBT=n and SHSTK=n, ultimately causing a mismatch on the CET entry and exit controls.

Patch 2 hardens against similar bugs in the future by adding a flag and WARNs to yell if KVM sets or clear feature flags outside of the dedicated flow.

Patch 3 adds (very, very) long overdue printing of the mistmatching offsets in the VMCS configs.

Chao, I didn't include any of your Reviewed-by's, as every patch changed quite a bit from v1.

v2:
- Isolate kvm_setup_xss_caps() from kvm_finalize_cpu_caps(). [Xiaoyao]
- Fix the pr_cont() printing. [Chao]

v1: https://lore.kernel.org/all/[email protected]

Sean Christopherson (3):
KVM: x86: Explicitly configure supported XSS from

    {svm,vmx}_set_cpu_caps()

KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps
KVM: VMX: Print out "bad" offsets+value on VMCS config mismatch

 arch/x86/kvm/cpuid.c   | 10 ++++++++--
 arch/x86/kvm/cpuid.h   | 12 +++++++++++-
 arch/x86/kvm/svm/svm.c |  6 +++++-
 arch/x86/kvm/vmx/vmx.c | 23 +++++++++++++++++++++--
 arch/x86/kvm/x86.c     | 30 +++++++++++++++++-------------
 arch/x86/kvm/x86.h     |  2 ++
 6 files changed, 64 insertions(+), 19 deletions(-)

base-commit: e81f7c908e1664233974b9f20beead78cde6343a

Signature · 1 line
2.52.0.457.g6b5491de43-goog
author maintainer
#1 lore raw

PATCH v2 1/3 KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1. The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

Note, kvm_caps.supported_{xcr0,xss} are unconditionally initialized by kvm_x86_vendor_init(), before calling into vendor code, and not referenced between ops->hardware_setup() and their current/old location.

Fixes 69cc3e886582 ("KVM: x86: Add XSS support for CET_KERNEL and CET_USER")
Cc Mathias Krause <[email protected]>
Cc John Allen <[email protected]>
Cc Rick Edgecombe <[email protected]>
Cc Chao Gao <[email protected]>
Cc Binbin Wu <[email protected]>
Cc Xiaoyao Li <[email protected]>
---
 arch/x86/kvm/svm/svm.c |  2 ++
 arch/x86/kvm/vmx/vmx.c |  2 ++
 arch/x86/kvm/x86.c     | 30 +++++++++++++++++-------------
 arch/x86/kvm/x86.h     |  2 ++
 4 files changed, 23 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/svm.c +2 -0
index 7803d2781144..c00a696dacfc 100644
@@ -5387,6 +5387,8 @@ static __init void svm_set_cpu_caps(void)
 	 */
 	kvm_cpu_cap_clear(X86_FEATURE_BUS_LOCK_DETECT);
 	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
+
+	kvm_setup_xss_caps();
 }
 
 static __init int svm_hardware_setup(void)
arch/x86/kvm/vmx/vmx.c +2 -0
index 27acafd03381..9f85c3829890 100644
@@ -8230,6 +8230,8 @@ static __init void vmx_set_cpu_caps(void)
 		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
 		kvm_cpu_cap_clear(X86_FEATURE_IBT);
 	}
+
+	kvm_setup_xss_caps();
 }
 
 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
arch/x86/kvm/x86.c +17 -13
index 8acfdfc583a1..cac1d6a67b49 100644
@@ -9965,6 +9965,23 @@ static struct notifier_block pvclock_gtod_notifier = {
 };
 #endif
 
+void kvm_setup_xss_caps(void)
+{
+	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
+		kvm_caps.supported_xss = 0;
+
+	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
+	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+
+	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
+		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
+		kvm_cpu_cap_clear(X86_FEATURE_IBT);
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+	}
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_setup_xss_caps);
+
 static inline void kvm_ops_update(struct kvm_x86_init_ops *ops)
 {
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
@@ -10138,19 +10155,6 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (!tdp_enabled)
 		kvm_caps.supported_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
-	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
-		kvm_caps.supported_xss = 0;
-
-	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
-	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-
-	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
-		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
-		kvm_cpu_cap_clear(X86_FEATURE_IBT);
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-	}
-
 	if (kvm_caps.has_tsc_control) {
 		/*
 		 * Make sure the user can only configure tsc_khz values that
arch/x86/kvm/x86.h +2 -0
index 70e81f008030..94d4f07aaaa0 100644
@@ -483,6 +483,8 @@ extern struct kvm_host_values kvm_host;
 extern bool enable_pmu;
 extern bool enable_mediated_pmu;
 
+void kvm_setup_xss_caps(void);
+
 /*
  * Get a filtered version of KVM's supported XCR0 that strips out dynamic
  * features for which the current process doesn't (yet) have permission to use.
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Reviewed-by on v2 1/3
#2 lore raw

Re: [PATCH v2 1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

Quoted 10 lines · Sean Christopherson

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1. The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

Quoted 3 lines

Note, kvm_caps.supported_{xcr0,xss} are unconditionally initialized by kvm_x86_vendor_init(), before calling into vendor code, and not referenced between ops->hardware_setup() and their current/old location.

I'm thinking whether to move the initialization of supported_xss from kvm_x86_vendor_init() to kvm_setup_xss_caps(). Anyway it can be a separate patch, if we agree to make the change.

For this fixing patch,

Quoted 103 lines
Fixes 69cc3e886582 ("KVM: x86: Add XSS support for CET_KERNEL and CET_USER")
Cc Mathias Krause <[email protected]>
Cc John Allen <[email protected]>
Cc Rick Edgecombe <[email protected]>
Cc Chao Gao <[email protected]>
Cc Binbin Wu <[email protected]>
Cc Xiaoyao Li <[email protected]>
---
  arch/x86/kvm/svm/svm.c |  2 ++
  arch/x86/kvm/vmx/vmx.c |  2 ++
  arch/x86/kvm/x86.c     | 30 +++++++++++++++++-------------
  arch/x86/kvm/x86.h     |  2 ++
  4 files changed, 23 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/svm.c +2 -0
index 7803d2781144..c00a696dacfc 100644
@@ -5387,6 +5387,8 @@ static __init void svm_set_cpu_caps(void)
  	 */
  	kvm_cpu_cap_clear(X86_FEATURE_BUS_LOCK_DETECT);
  	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
+
+	kvm_setup_xss_caps();
  }
 
  static __init int svm_hardware_setup(void)
arch/x86/kvm/vmx/vmx.c +2 -0
index 27acafd03381..9f85c3829890 100644
@@ -8230,6 +8230,8 @@ static __init void vmx_set_cpu_caps(void)
  		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
  		kvm_cpu_cap_clear(X86_FEATURE_IBT);
  	}
+
+	kvm_setup_xss_caps();
  }
 
  static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
arch/x86/kvm/x86.c +17 -13
index 8acfdfc583a1..cac1d6a67b49 100644
@@ -9965,6 +9965,23 @@ static struct notifier_block pvclock_gtod_notifier = {
  };
  #endif
 
+void kvm_setup_xss_caps(void)
+{
+	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
+		kvm_caps.supported_xss = 0;
+
+	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
+	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+
+	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
+		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
+		kvm_cpu_cap_clear(X86_FEATURE_IBT);
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+	}
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_setup_xss_caps);
+
  static inline void kvm_ops_update(struct kvm_x86_init_ops *ops)
  {
  	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
@@ -10138,19 +10155,6 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
  	if (!tdp_enabled)
  		kvm_caps.supported_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
-	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
-		kvm_caps.supported_xss = 0;
-
-	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
-	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-
-	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
-		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
-		kvm_cpu_cap_clear(X86_FEATURE_IBT);
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-	}
-
  	if (kvm_caps.has_tsc_control) {
  		/*
  		 * Make sure the user can only configure tsc_khz values that
arch/x86/kvm/x86.h +2 -0
index 70e81f008030..94d4f07aaaa0 100644
@@ -483,6 +483,8 @@ extern struct kvm_host_values kvm_host;
  extern bool enable_pmu;
  extern bool enable_mediated_pmu;
 
+void kvm_setup_xss_caps(void);
+
  /*
   * Get a filtered version of KVM's supported XCR0 that strips out dynamic
   * features for which the current process doesn't (yet) have permission to use.
author maintainer
#3 lore raw

Re: [PATCH v2 1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

Quoted 11 lines · Xiaoyao Li
Quoted 10 lines · Sean Christopherson

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1. The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

Quoted 7 lines
Quoted 3 lines

Note, kvm_caps.supported_{xcr0,xss} are unconditionally initialized by kvm_x86_vendor_init(), before calling into vendor code, and not referenced between ops->hardware_setup() and their current/old location.

I'm thinking whether to move the initialization of supported_xss from kvm_x86_vendor_init() to kvm_setup_xss_caps(). Anyway it can be a separate patch, if we agree to make the change.

Hmm, I definitely don't want to do that, because then we'll end up with asymmetric initialization of kvm_caps.*, and I don't want to move that initialization under {svm,vmx}_set_cpu_caps() because it's pretty much guaranteed to lead to different ordering issues.

One idea would be to call the new API kvm_finalize_xss_caps() instead of kvm_setup_xss_caps(), but I'm not sure I like that idea. Or maybe kvm_constrain_xss_caps()? That feels awkward and unintuitive though.

All in all, I agree that having a "setup" API rely on prior initialization is a bit wonky, but I don't love any of the alternatives either. :-/

#4 lore raw

Re: [PATCH v2 1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

In reply to Sean Christopherson
Quoted 9 lines · Sean Christopherson

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1. The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

A bit confuse about the description.

Before this patch:

kvm_x86_vendor_init

| vmx_hardware_setup
|   nested_vmx_hardware_setup
|     nested_vmx_setup_ctls_msrs
| ...
| for_each_online_cpu(cpu)
|   smp_call_function_single(cpu, kvm_x86_check_cpu_compat, &r, 1)
|                                 | kvm_x86_check_processor_compatibility
|                                 |   kvm_x86_call(check_processor_compatibility)()
|                                 |     vmx_check_processor_compatibility
|                                 |       setup_vmcs_config
|                                 |         nested_vmx_setup_ctls_msrs
| ...
| //late clearing of SHSTK and IBT

If we don't consider CPU hotplug case, both the setup of reference VMCS and the local config are before the late clearing of SHSTK and IBT. They should be consistent.

So you are referring the mismatch situation during CPU hotplug?

But if it's hotplug case, it shouldn't make kvm-intel.ko unloadable?

Quoted 108 lines

Note, kvm_caps.supported_{xcr0,xss} are unconditionally initialized by kvm_x86_vendor_init(), before calling into vendor code, and not referenced between ops->hardware_setup() and their current/old location.

Fixes 69cc3e886582 ("KVM: x86: Add XSS support for CET_KERNEL and CET_USER")
Cc Mathias Krause <[email protected]>
Cc John Allen <[email protected]>
Cc Rick Edgecombe <[email protected]>
Cc Chao Gao <[email protected]>
Cc Binbin Wu <[email protected]>
Cc Xiaoyao Li <[email protected]>
---
 arch/x86/kvm/svm/svm.c |  2 ++
 arch/x86/kvm/vmx/vmx.c |  2 ++
 arch/x86/kvm/x86.c     | 30 +++++++++++++++++-------------
 arch/x86/kvm/x86.h     |  2 ++
 4 files changed, 23 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/svm.c +2 -0
index 7803d2781144..c00a696dacfc 100644
@@ -5387,6 +5387,8 @@ static __init void svm_set_cpu_caps(void)
 	 */
 	kvm_cpu_cap_clear(X86_FEATURE_BUS_LOCK_DETECT);
 	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
+
+	kvm_setup_xss_caps();
 }
 
 static __init int svm_hardware_setup(void)
arch/x86/kvm/vmx/vmx.c +2 -0
index 27acafd03381..9f85c3829890 100644
@@ -8230,6 +8230,8 @@ static __init void vmx_set_cpu_caps(void)
 		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
 		kvm_cpu_cap_clear(X86_FEATURE_IBT);
 	}
+
+	kvm_setup_xss_caps();
 }
 
 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
arch/x86/kvm/x86.c +17 -13
index 8acfdfc583a1..cac1d6a67b49 100644
@@ -9965,6 +9965,23 @@ static struct notifier_block pvclock_gtod_notifier = {
 };
 #endif
 
+void kvm_setup_xss_caps(void)
+{
+	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
+		kvm_caps.supported_xss = 0;
+
+	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
+	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+
+	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
+		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
+		kvm_cpu_cap_clear(X86_FEATURE_IBT);
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+	}
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_setup_xss_caps);
+
 static inline void kvm_ops_update(struct kvm_x86_init_ops *ops)
 {
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
@@ -10138,19 +10155,6 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (!tdp_enabled)
 		kvm_caps.supported_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
-	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
-		kvm_caps.supported_xss = 0;
-
-	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
-	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-
-	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
-		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
-		kvm_cpu_cap_clear(X86_FEATURE_IBT);
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-	}
-
 	if (kvm_caps.has_tsc_control) {
 		/*
 		 * Make sure the user can only configure tsc_khz values that
arch/x86/kvm/x86.h +2 -0
index 70e81f008030..94d4f07aaaa0 100644
@@ -483,6 +483,8 @@ extern struct kvm_host_values kvm_host;
 extern bool enable_pmu;
 extern bool enable_mediated_pmu;
 
+void kvm_setup_xss_caps(void);
+
 /*
  * Get a filtered version of KVM's supported XCR0 that strips out dynamic
  * features for which the current process doesn't (yet) have permission to use.
#5 lore raw

Re: [PATCH v2 1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

Quoted 32 lines · Binbin Wu
Quoted 9 lines · Sean Christopherson

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1. The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

A bit confuse about the description.

Before this patch:

kvm_x86_vendor_init

| vmx_hardware_setup
|   nested_vmx_hardware_setup
|     nested_vmx_setup_ctls_msrs
| ...
| for_each_online_cpu(cpu)
|   smp_call_function_single(cpu, kvm_x86_check_cpu_compat, &r, 1)
|                                 | kvm_x86_check_processor_compatibility
|                                 |   kvm_x86_call(check_processor_compatibility)()
|                                 |     vmx_check_processor_compatibility
|                                 |       setup_vmcs_config
|                                 |         nested_vmx_setup_ctls_msrs
| ...
| //late clearing of SHSTK and IBT
Quoted 5 lines

If we don't consider CPU hotplug case, both the setup of reference VMCS and the local config are before the late clearing of SHSTK and IBT. They should be consistent.

So you are referring the mismatch situation during CPU hotplug?

I guess it's triggered the path

kvm_init()

    kvm_init_virtualization()
      kvm_enable_virtualization()
        cpuhp_setup_state()
          kvm_online_cpu()
            ...

(note, it requires enable_virt_at_load to be true)

which is after
vmx_init()

    kvm_x86_vendor_init()
#6 lore raw

Re: [PATCH v2 1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

Quoted 48 lines · Xiaoyao Li
Quoted 32 lines · Binbin Wu
Quoted 9 lines · Sean Christopherson

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1.  The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

A bit confuse about the description.

Before this patch:

kvm_x86_vendor_init

| vmx_hardware_setup
|   nested_vmx_hardware_setup
|     nested_vmx_setup_ctls_msrs
| ...
| for_each_online_cpu(cpu)
|   smp_call_function_single(cpu, kvm_x86_check_cpu_compat, &r, 1)
|                                 | kvm_x86_check_processor_compatibility
|                                 |   kvm_x86_call(check_processor_compatibility)()
|                                 |     vmx_check_processor_compatibility
|                                 |       setup_vmcs_config
|                                 |         nested_vmx_setup_ctls_msrs
| ...
| //late clearing of SHSTK and IBT
Quoted 5 lines

If we don't consider CPU hotplug case, both the setup of reference VMCS and the local config are before the late clearing of SHSTK and IBT. They should be consistent.

So you are referring the mismatch situation during CPU hotplug?

I guess it's triggered the path

  kvm_init()
    kvm_init_virtualization()
      kvm_enable_virtualization()
        cpuhp_setup_state()
          kvm_online_cpu()
            ...

Quoted 6 lines

  (note, it requires enable_virt_at_load to be true)

which is after
  vmx_init()
    kvm_x86_vendor_init()

Oh, right.
Forgot about that by default KVM enables virtualization when KVM is loaded, which trigger the cpuhp framework to do per-CPU enabling.

Thanks!

Reviewed-by on v2 1/3
#7 lore raw

Re: [PATCH v2 1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

In reply to Sean Christopherson
Quoted 10 lines · Sean Christopherson

Explicitly configure KVM's supported XSS as part of each vendor's setup flow to fix a bug where clearing SHSTK and IBT in kvm_cpu_caps, e.g. due to lack of CET XFEATURE support, makes kvm-intel.ko unloadable when nested VMX is enabled, i.e. when nested=1. The late clearing results in nested_vmx_setup_{entry,exit}_ctls() clearing VM_{ENTRY,EXIT}_LOAD_CET_STATE when nested_vmx_setup_ctls_msrs() runs during the CPU compatibility checks, ultimately leading to a mismatched VMCS config due to the reference config having the CET bits set, but every CPU's "local" config having the bits cleared.

Quoted 4 lines

Note, kvm_caps.supported_{xcr0,xss} are unconditionally initialized by kvm_x86_vendor_init(), before calling into vendor code, and not referenced between ops->hardware_setup() and their current/old location.

Quoted 103 lines
Fixes 69cc3e886582 ("KVM: x86: Add XSS support for CET_KERNEL and CET_USER")
Cc Mathias Krause <[email protected]>
Cc John Allen <[email protected]>
Cc Rick Edgecombe <[email protected]>
Cc Chao Gao <[email protected]>
Cc Binbin Wu <[email protected]>
Cc Xiaoyao Li <[email protected]>
---
 arch/x86/kvm/svm/svm.c |  2 ++
 arch/x86/kvm/vmx/vmx.c |  2 ++
 arch/x86/kvm/x86.c     | 30 +++++++++++++++++-------------
 arch/x86/kvm/x86.h     |  2 ++
 4 files changed, 23 insertions(+), 13 deletions(-)
arch/x86/kvm/svm/svm.c +2 -0
index 7803d2781144..c00a696dacfc 100644
@@ -5387,6 +5387,8 @@ static __init void svm_set_cpu_caps(void)
 	 */
 	kvm_cpu_cap_clear(X86_FEATURE_BUS_LOCK_DETECT);
 	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
+
+	kvm_setup_xss_caps();
 }
 
 static __init int svm_hardware_setup(void)
arch/x86/kvm/vmx/vmx.c +2 -0
index 27acafd03381..9f85c3829890 100644
@@ -8230,6 +8230,8 @@ static __init void vmx_set_cpu_caps(void)
 		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
 		kvm_cpu_cap_clear(X86_FEATURE_IBT);
 	}
+
+	kvm_setup_xss_caps();
 }
 
 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
arch/x86/kvm/x86.c +17 -13
index 8acfdfc583a1..cac1d6a67b49 100644
@@ -9965,6 +9965,23 @@ static struct notifier_block pvclock_gtod_notifier = {
 };
 #endif
 
+void kvm_setup_xss_caps(void)
+{
+	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
+		kvm_caps.supported_xss = 0;
+
+	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
+	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+
+	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
+		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
+		kvm_cpu_cap_clear(X86_FEATURE_IBT);
+		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
+	}
+}
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_setup_xss_caps);
+
 static inline void kvm_ops_update(struct kvm_x86_init_ops *ops)
 {
 	memcpy(&kvm_x86_ops, ops->runtime_ops, sizeof(kvm_x86_ops));
@@ -10138,19 +10155,6 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (!tdp_enabled)
 		kvm_caps.supported_quirks &= ~KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
-	if (!kvm_cpu_cap_has(X86_FEATURE_XSAVES))
-		kvm_caps.supported_xss = 0;
-
-	if (!kvm_cpu_cap_has(X86_FEATURE_SHSTK) &&
-	    !kvm_cpu_cap_has(X86_FEATURE_IBT))
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-
-	if ((kvm_caps.supported_xss & XFEATURE_MASK_CET_ALL) != XFEATURE_MASK_CET_ALL) {
-		kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
-		kvm_cpu_cap_clear(X86_FEATURE_IBT);
-		kvm_caps.supported_xss &= ~XFEATURE_MASK_CET_ALL;
-	}
-
 	if (kvm_caps.has_tsc_control) {
 		/*
 		 * Make sure the user can only configure tsc_khz values that
arch/x86/kvm/x86.h +2 -0
index 70e81f008030..94d4f07aaaa0 100644
@@ -483,6 +483,8 @@ extern struct kvm_host_values kvm_host;
 extern bool enable_pmu;
 extern bool enable_mediated_pmu;
 
+void kvm_setup_xss_caps(void);
+
 /*
  * Get a filtered version of KVM's supported XCR0 that strips out dynamic
  * features for which the current process doesn't (yet) have permission to use.
author maintainer
#8 lore raw

PATCH v2 2/3 KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps

In reply to Sean Christopherson

Add a flag to track when KVM is actively configuring its CPU caps, and WARN if a cap is set or cleared if KVM isn't in its configuration stage. Modifying CPU caps after {svm,vmx}_set_cpu_caps() can be fatal to KVM, as vendor setup code expects the CPU caps to be frozen at that point, e.g. will do additional configuration based on the caps.

Rename kvm_set_cpu_caps() to kvm_initialize_cpu_caps() to pair with the new "finalize", and to make it more obvious that KVM's CPU caps aren't fully configured within the function.

---
 arch/x86/kvm/cpuid.c   | 10 ++++++++--
 arch/x86/kvm/cpuid.h   | 12 +++++++++++-
 arch/x86/kvm/svm/svm.c |  4 +++-
 arch/x86/kvm/vmx/vmx.c |  4 +++-
 4 files changed, 25 insertions(+), 5 deletions(-)
arch/x86/kvm/cpuid.c +8 -2
index 575244af9c9f..7fe4e58a6ebf 100644
@@ -36,6 +36,9 @@
 u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_cpu_caps);
 
+bool kvm_is_configuring_cpu_caps __read_mostly;
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_is_configuring_cpu_caps);
+
 struct cpuid_xstate_sizes {
 	u32 eax;
 	u32 ebx;
@@ -826,10 +829,13 @@ do {									\
 /* DS is defined by ptrace-abi.h on 32-bit builds. */
 #undef DS
 
-void kvm_set_cpu_caps(void)
+void kvm_initialize_cpu_caps(void)
 {
 	memset(kvm_cpu_caps, 0, sizeof(kvm_cpu_caps));
 
+	WARN_ON_ONCE(kvm_is_configuring_cpu_caps);
+	kvm_is_configuring_cpu_caps = true;
+
 	BUILD_BUG_ON(sizeof(kvm_cpu_caps) - (NKVMCAPINTS * sizeof(*kvm_cpu_caps)) >
 		     sizeof(boot_cpu_data.x86_capability));
 
@@ -1289,7 +1295,7 @@ void kvm_set_cpu_caps(void)
 		kvm_cpu_cap_clear(X86_FEATURE_RDPID);
 	}
 }
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cpu_caps);
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_initialize_cpu_caps);
 
 #undef F
 #undef SCATTERED_F
arch/x86/kvm/cpuid.h +11 -1
index d3f5ae15a7ca..039b8e6f40ba 100644
@@ -8,7 +8,15 @@
 #include <uapi/asm/kvm_para.h>
 
 extern u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
-void kvm_set_cpu_caps(void);
+extern bool kvm_is_configuring_cpu_caps __read_mostly;
+
+void kvm_initialize_cpu_caps(void);
+
+static inline void kvm_finalize_cpu_caps(void)
+{
+	WARN_ON_ONCE(!kvm_is_configuring_cpu_caps);
+	kvm_is_configuring_cpu_caps = false;
+}
 
 void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu);
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2(struct kvm_cpuid_entry2 *entries,
@@ -188,6 +196,7 @@ static __always_inline void kvm_cpu_cap_clear(unsigned int x86_feature)
 {
 	unsigned int x86_leaf = __feature_leaf(x86_feature);
 
+	WARN_ON_ONCE(!kvm_is_configuring_cpu_caps);
 	kvm_cpu_caps[x86_leaf] &= ~__feature_bit(x86_feature);
 }
 
@@ -195,6 +204,7 @@ static __always_inline void kvm_cpu_cap_set(unsigned int x86_feature)
 {
 	unsigned int x86_leaf = __feature_leaf(x86_feature);
 
+	WARN_ON_ONCE(!kvm_is_configuring_cpu_caps);
 	kvm_cpu_caps[x86_leaf] |= __feature_bit(x86_feature);
 }
 
arch/x86/kvm/svm/svm.c +3 -1
index c00a696dacfc..5f0136dbdde6 100644
@@ -5305,7 +5305,7 @@ static __init void svm_adjust_mmio_mask(void)
 
 static __init void svm_set_cpu_caps(void)
 {
-	kvm_set_cpu_caps();
+	kvm_initialize_cpu_caps();
 
 	kvm_caps.supported_perf_cap = 0;
 
@@ -5389,6 +5389,8 @@ static __init void svm_set_cpu_caps(void)
 	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
 
 	kvm_setup_xss_caps();
+
+	kvm_finalize_cpu_caps();
 }
 
 static __init int svm_hardware_setup(void)
arch/x86/kvm/vmx/vmx.c +3 -1
index 9f85c3829890..93ec1e6181e4 100644
@@ -8173,7 +8173,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 
 static __init void vmx_set_cpu_caps(void)
 {
-	kvm_set_cpu_caps();
+	kvm_initialize_cpu_caps();
 
 	/* CPUID 0x1 */
 	if (nested)
@@ -8232,6 +8232,8 @@ static __init void vmx_set_cpu_caps(void)
 	}
 
 	kvm_setup_xss_caps();
+
+	kvm_finalize_cpu_caps();
 }
 
 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Reviewed-by on v2 2/3
#9 lore raw

Re: [PATCH v2 2/3] KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps

Quoted 23 lines · Sean Christopherson

Add a flag to track when KVM is actively configuring its CPU caps, and WARN if a cap is set or cleared if KVM isn't in its configuration stage. Modifying CPU caps after {svm,vmx}_set_cpu_caps() can be fatal to KVM, as vendor setup code expects the CPU caps to be frozen at that point, e.g. will do additional configuration based on the caps.

Rename kvm_set_cpu_caps() to kvm_initialize_cpu_caps() to pair with the new "finalize", and to make it more obvious that KVM's CPU caps aren't fully configured within the function.

---
  arch/x86/kvm/cpuid.c   | 10 ++++++++--
  arch/x86/kvm/cpuid.h   | 12 +++++++++++-
  arch/x86/kvm/svm/svm.c |  4 +++-
  arch/x86/kvm/vmx/vmx.c |  4 +++-
  4 files changed, 25 insertions(+), 5 deletions(-)
arch/x86/kvm/cpuid.c +0 -0
index 575244af9c9f..7fe4e58a6ebf 100644

arch/x86/kvm/cpuid.c line 39

Quoted 4 lines
arch/x86/kvm/cpuid.c +1 -0
@@ -36,6 +36,9 @@
  u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
  EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_cpu_caps);
 
+bool kvm_is_configuring_cpu_caps __read_mostly;

I prefer the name, kvm_cpu_caps_finalized. But not strongly, so

author maintainer
#10 lore raw

Re: [PATCH v2 2/3] KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps

Quoted 19 lines · Xiaoyao Li
Quoted 18 lines · Sean Christopherson

Add a flag to track when KVM is actively configuring its CPU caps, and WARN if a cap is set or cleared if KVM isn't in its configuration stage. Modifying CPU caps after {svm,vmx}_set_cpu_caps() can be fatal to KVM, as vendor setup code expects the CPU caps to be frozen at that point, e.g. will do additional configuration based on the caps.

Rename kvm_set_cpu_caps() to kvm_initialize_cpu_caps() to pair with the new "finalize", and to make it more obvious that KVM's CPU caps aren't fully configured within the function.

---
  arch/x86/kvm/cpuid.c   | 10 ++++++++--
  arch/x86/kvm/cpuid.h   | 12 +++++++++++-
  arch/x86/kvm/svm/svm.c |  4 +++-
  arch/x86/kvm/vmx/vmx.c |  4 +++-
  4 files changed, 25 insertions(+), 5 deletions(-)
Quoted 10 lines

arch/x86/kvm/cpuid.c line 38

Quoted 8 lines
arch/x86/kvm/cpuid.c +1 -0
index 575244af9c9f..7fe4e58a6ebf 100644
@@ -36,6 +36,9 @@
  u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
  EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_cpu_caps);
+bool kvm_is_configuring_cpu_caps __read_mostly;

I prefer the name, kvm_cpu_caps_finalized. But not strongly, so

"finalized" reads too much like the helper queries if the caps are already finalized, i.e. like an accessor.

#11 lore raw

Re: [PATCH v2 2/3] KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps

Quoted 31 lines · Sean Christopherson
Quoted 19 lines · Xiaoyao Li
Quoted 18 lines · Sean Christopherson

Add a flag to track when KVM is actively configuring its CPU caps, and WARN if a cap is set or cleared if KVM isn't in its configuration stage. Modifying CPU caps after {svm,vmx}_set_cpu_caps() can be fatal to KVM, as vendor setup code expects the CPU caps to be frozen at that point, e.g. will do additional configuration based on the caps.

Rename kvm_set_cpu_caps() to kvm_initialize_cpu_caps() to pair with the new "finalize", and to make it more obvious that KVM's CPU caps aren't fully configured within the function.

---
   arch/x86/kvm/cpuid.c   | 10 ++++++++--
   arch/x86/kvm/cpuid.h   | 12 +++++++++++-
   arch/x86/kvm/svm/svm.c |  4 +++-
   arch/x86/kvm/vmx/vmx.c |  4 +++-
   4 files changed, 25 insertions(+), 5 deletions(-)
Quoted 10 lines

arch/x86/kvm/cpuid.c line 38

Quoted 8 lines
arch/x86/kvm/cpuid.c +1 -0
index 575244af9c9f..7fe4e58a6ebf 100644
@@ -36,6 +36,9 @@
   u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
   EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_cpu_caps);
+bool kvm_is_configuring_cpu_caps __read_mostly;

I prefer the name, kvm_cpu_caps_finalized. But not strongly, so

Quoted 2 lines

"finalized" reads too much like the helper queries if the caps are already finalized, i.e. like an accessor.

And after a second thought, I find my preference is not good. Because it only tells the end of allowed stage while kvm_is_configuring_cpu_caps defines both the start and end.

So withdraw my preference.

Reviewed-by on v2 2/3
#12 lore raw

Re: [PATCH v2 2/3] KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps

In reply to Sean Christopherson
Quoted 10 lines · Sean Christopherson

Add a flag to track when KVM is actively configuring its CPU caps, and WARN if a cap is set or cleared if KVM isn't in its configuration stage. Modifying CPU caps after {svm,vmx}_set_cpu_caps() can be fatal to KVM, as vendor setup code expects the CPU caps to be frozen at that point, e.g. will do additional configuration based on the caps.

Rename kvm_set_cpu_caps() to kvm_initialize_cpu_caps() to pair with the new "finalize", and to make it more obvious that KVM's CPU caps aren't fully configured within the function.

Quoted 127 lines
---
 arch/x86/kvm/cpuid.c   | 10 ++++++++--
 arch/x86/kvm/cpuid.h   | 12 +++++++++++-
 arch/x86/kvm/svm/svm.c |  4 +++-
 arch/x86/kvm/vmx/vmx.c |  4 +++-
 4 files changed, 25 insertions(+), 5 deletions(-)
arch/x86/kvm/cpuid.c +8 -2
index 575244af9c9f..7fe4e58a6ebf 100644
@@ -36,6 +36,9 @@
 u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
 EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_cpu_caps);
 
+bool kvm_is_configuring_cpu_caps __read_mostly;
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_is_configuring_cpu_caps);
+
 struct cpuid_xstate_sizes {
 	u32 eax;
 	u32 ebx;
@@ -826,10 +829,13 @@ do {									\
 /* DS is defined by ptrace-abi.h on 32-bit builds. */
 #undef DS
 
-void kvm_set_cpu_caps(void)
+void kvm_initialize_cpu_caps(void)
 {
 	memset(kvm_cpu_caps, 0, sizeof(kvm_cpu_caps));
 
+	WARN_ON_ONCE(kvm_is_configuring_cpu_caps);
+	kvm_is_configuring_cpu_caps = true;
+
 	BUILD_BUG_ON(sizeof(kvm_cpu_caps) - (NKVMCAPINTS * sizeof(*kvm_cpu_caps)) >
 		     sizeof(boot_cpu_data.x86_capability));
 
@@ -1289,7 +1295,7 @@ void kvm_set_cpu_caps(void)
 		kvm_cpu_cap_clear(X86_FEATURE_RDPID);
 	}
 }
-EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_set_cpu_caps);
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_initialize_cpu_caps);
 
 #undef F
 #undef SCATTERED_F
arch/x86/kvm/cpuid.h +11 -1
index d3f5ae15a7ca..039b8e6f40ba 100644
@@ -8,7 +8,15 @@
 #include <uapi/asm/kvm_para.h>
 
 extern u32 kvm_cpu_caps[NR_KVM_CPU_CAPS] __read_mostly;
-void kvm_set_cpu_caps(void);
+extern bool kvm_is_configuring_cpu_caps __read_mostly;
+
+void kvm_initialize_cpu_caps(void);
+
+static inline void kvm_finalize_cpu_caps(void)
+{
+	WARN_ON_ONCE(!kvm_is_configuring_cpu_caps);
+	kvm_is_configuring_cpu_caps = false;
+}
 
 void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu);
 struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2(struct kvm_cpuid_entry2 *entries,
@@ -188,6 +196,7 @@ static __always_inline void kvm_cpu_cap_clear(unsigned int x86_feature)
 {
 	unsigned int x86_leaf = __feature_leaf(x86_feature);
 
+	WARN_ON_ONCE(!kvm_is_configuring_cpu_caps);
 	kvm_cpu_caps[x86_leaf] &= ~__feature_bit(x86_feature);
 }
 
@@ -195,6 +204,7 @@ static __always_inline void kvm_cpu_cap_set(unsigned int x86_feature)
 {
 	unsigned int x86_leaf = __feature_leaf(x86_feature);
 
+	WARN_ON_ONCE(!kvm_is_configuring_cpu_caps);
 	kvm_cpu_caps[x86_leaf] |= __feature_bit(x86_feature);
 }
 
arch/x86/kvm/svm/svm.c +3 -1
index c00a696dacfc..5f0136dbdde6 100644
@@ -5305,7 +5305,7 @@ static __init void svm_adjust_mmio_mask(void)
 
 static __init void svm_set_cpu_caps(void)
 {
-	kvm_set_cpu_caps();
+	kvm_initialize_cpu_caps();
 
 	kvm_caps.supported_perf_cap = 0;
 
@@ -5389,6 +5389,8 @@ static __init void svm_set_cpu_caps(void)
 	kvm_cpu_cap_clear(X86_FEATURE_MSR_IMM);
 
 	kvm_setup_xss_caps();
+
+	kvm_finalize_cpu_caps();
 }
 
 static __init int svm_hardware_setup(void)
arch/x86/kvm/vmx/vmx.c +3 -1
index 9f85c3829890..93ec1e6181e4 100644
@@ -8173,7 +8173,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 
 static __init void vmx_set_cpu_caps(void)
 {
-	kvm_set_cpu_caps();
+	kvm_initialize_cpu_caps();
 
 	/* CPUID 0x1 */
 	if (nested)
@@ -8232,6 +8232,8 @@ static __init void vmx_set_cpu_caps(void)
 	}
 
 	kvm_setup_xss_caps();
+
+	kvm_finalize_cpu_caps();
 }
 
 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu,
author maintainer
#13 lore raw

PATCH v2 3/3 KVM: VMX: Print out "bad" offsets+value on VMCS config mismatch

In reply to Sean Christopherson

When kvm-intel.ko refuses to load due to a mismatched VMCS config, print all mismatching offsets+values to make it easier to debug goofs during development, and it to make it at least feasible to triage failures that occur during production. E.g. if a physical core is flaky or is running with the "wrong" microcode patch loaded, then a CPU can get a legitimate mismatch even without KVM bugs.

Print the mismatches as 32-bit values as a compromise between hand coding every field (to provide precise information) and printing individual bytes (requires more effort to deduce the mismatch bit(s)). All fields in the VMCS config are either 32-bit or 64-bit values, i.e. in many cases, printing 32-bit values will be 100% precise, and in the others it's close enough, especially when considering that MSR values are split into EDX:EAX anyways.

E.g. on mismatch CET entry/exit controls, KVM will print:

kvm_intel: VMCS config on CPU 0 doesn't match reference config:

    Offset 76 REF = 0x107fffff, CPU0 = 0x007fffff, mismatch = 0x10000000
    Offset 84 REF = 0x0010f3ff, CPU0 = 0x0000f3ff, mismatch = 0x00100000

Opportunistically tweak the wording on the initial error message to say "mismatch" instead of "inconsistent", as the VMCS config itself isn't inconsistent, and the wording conflates the cross-CPU compatibility check with the error_on_inconsistent_vmcs_config knob that treats inconsistent VMCS configurations as errors (e.g. if a CPU supports CET entry controls but no CET exit controls).

---
 arch/x86/kvm/vmx/vmx.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +16 -1
index 93ec1e6181e4..11bb4b933227 100644
@@ -2962,8 +2962,23 @@ int vmx_check_processor_compat(void)
 	}
 	if (nested)
 		nested_vmx_setup_ctls_msrs(&vmcs_conf, vmx_cap.ept);
+
 	if (memcmp(&vmcs_config, &vmcs_conf, sizeof(struct vmcs_config))) {
-		pr_err("Inconsistent VMCS config on CPU %d\n", cpu);
+		u32 *gold = (void *)&vmcs_config;
+		u32 *mine = (void *)&vmcs_conf;
+		int i;
+
+		BUILD_BUG_ON(sizeof(struct vmcs_config) % sizeof(u32));
+
+		pr_err("VMCS config on CPU %d doesn't match reference config:", cpu);
+		for (i = 0; i < sizeof(struct vmcs_config) / sizeof(u32); i++) {
+			if (gold[i] == mine[i])
+				continue;
+
+			pr_cont("\n  Offset %u REF = 0x%08x, CPU%u = 0x%08x, mismatch = 0x%08x",
+				i * (int)sizeof(u32), gold[i], cpu, mine[i], gold[i] ^ mine[i]);
+		}
+		pr_cont("\n");
 		return -EIO;
 	}
 	return 0;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Reviewed-by on v2 3/3
#14 lore raw

Re: [PATCH v2 3/3] KVM: VMX: Print out "bad" offsets+value on VMCS config mismatch

Quoted 3 lines · Sean Christopherson

When kvm-intel.ko refuses to load due to a mismatched VMCS config, print all mismatching offsets+values to make it easier to debug goofs during development, and it to make it at least feasible to triage failures that

                   ^

typo "it"?

Quoted 25 lines

occur during production. E.g. if a physical core is flaky or is running with the "wrong" microcode patch loaded, then a CPU can get a legitimate mismatch even without KVM bugs.

Print the mismatches as 32-bit values as a compromise between hand coding every field (to provide precise information) and printing individual bytes (requires more effort to deduce the mismatch bit(s)). All fields in the VMCS config are either 32-bit or 64-bit values, i.e. in many cases, printing 32-bit values will be 100% precise, and in the others it's close enough, especially when considering that MSR values are split into EDX:EAX anyways.

E.g. on mismatch CET entry/exit controls, KVM will print:

   kvm_intel: VMCS config on CPU 0 doesn't match reference config:
     Offset 76 REF = 0x107fffff, CPU0 = 0x007fffff, mismatch = 0x10000000
     Offset 84 REF = 0x0010f3ff, CPU0 = 0x0000f3ff, mismatch = 0x00100000

Opportunistically tweak the wording on the initial error message to say "mismatch" instead of "inconsistent", as the VMCS config itself isn't inconsistent, and the wording conflates the cross-CPU compatibility check with the error_on_inconsistent_vmcs_config knob that treats inconsistent VMCS configurations as errors (e.g. if a CPU supports CET entry controls but no CET exit controls).

Quoted 2 lines
Quoted 33 lines
---
  arch/x86/kvm/vmx/vmx.c | 17 ++++++++++++++++-
  1 file changed, 16 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +16 -1
index 93ec1e6181e4..11bb4b933227 100644
@@ -2962,8 +2962,23 @@ int vmx_check_processor_compat(void)
  	}
  	if (nested)
  		nested_vmx_setup_ctls_msrs(&vmcs_conf, vmx_cap.ept);
+
  	if (memcmp(&vmcs_config, &vmcs_conf, sizeof(struct vmcs_config))) {
-		pr_err("Inconsistent VMCS config on CPU %d\n", cpu);
+		u32 *gold = (void *)&vmcs_config;
+		u32 *mine = (void *)&vmcs_conf;
+		int i;
+
+		BUILD_BUG_ON(sizeof(struct vmcs_config) % sizeof(u32));
+
+		pr_err("VMCS config on CPU %d doesn't match reference config:", cpu);
+		for (i = 0; i < sizeof(struct vmcs_config) / sizeof(u32); i++) {
+			if (gold[i] == mine[i])
+				continue;
+
+			pr_cont("\n  Offset %u REF = 0x%08x, CPU%u = 0x%08x, mismatch = 0x%08x",
+				i * (int)sizeof(u32), gold[i], cpu, mine[i], gold[i] ^ mine[i]);
+		}
+		pr_cont("\n");
  		return -EIO;
  	}
  	return 0;
Reviewed-by on v2 3/3
#15 lore raw

Re: [PATCH v2 3/3] KVM: VMX: Print out "bad" offsets+value on VMCS config mismatch

In reply to Sean Christopherson
Quoted 21 lines · Sean Christopherson

When kvm-intel.ko refuses to load due to a mismatched VMCS config, print all mismatching offsets+values to make it easier to debug goofs during development, and it to make it at least feasible to triage failures that occur during production. E.g. if a physical core is flaky or is running with the "wrong" microcode patch loaded, then a CPU can get a legitimate mismatch even without KVM bugs.

Print the mismatches as 32-bit values as a compromise between hand coding every field (to provide precise information) and printing individual bytes (requires more effort to deduce the mismatch bit(s)). All fields in the VMCS config are either 32-bit or 64-bit values, i.e. in many cases, printing 32-bit values will be 100% precise, and in the others it's close enough, especially when considering that MSR values are split into EDX:EAX anyways.

E.g. on mismatch CET entry/exit controls, KVM will print:

kvm_intel: VMCS config on CPU 0 doesn't match reference config:

    Offset 76 REF = 0x107fffff, CPU0 = 0x007fffff, mismatch = 0x10000000
    Offset 84 REF = 0x0010f3ff, CPU0 = 0x0000f3ff, mismatch = 0x00100000
Quoted 7 lines

Opportunistically tweak the wording on the initial error message to say "mismatch" instead of "inconsistent", as the VMCS config itself isn't inconsistent, and the wording conflates the cross-CPU compatibility check with the error_on_inconsistent_vmcs_config knob that treats inconsistent VMCS configurations as errors (e.g. if a CPU supports CET entry controls but no CET exit controls).

author maintainer Applied
#16 lore raw
In reply to Sean Christopherson
Quoted 8 lines · Sean Christopherson

Fix a bug where KVM will clear IBT and SHSTK bits after nested VMX MSRs have been configured, e.g. if the kernel is built with CONFIG_X86_CET=y but CONFIG_X86_KERNEL_IBT=n. The late clearing results in kvm-intel.ko refusing to load as the CPU compatible checks generate their VMCS configs with IBT=n and SHSTK=n, ultimately causing a mismatch on the CET entry and exit controls.

[...]

This got a bit messy, as I want to get the immediate fix into 6.19 (hopefully I wasn't too late), and at that point there was no reason to shove the VMCS patch in with the kvm_cpu_caps hardening.

Applied patch 1 to "fixes", patch 2 to "misc", and patch 3 to "vmx.

[1/3] KVM: x86: Explicitly configure supported XSS from {svm,vmx}_set_cpu_caps()

      https://github.com/kvm-x86/linux/commit/f8ade833b733
[2/3] KVM: x86: Harden against unexpected adjustments to kvm_cpu_caps
      https://github.com/kvm-x86/linux/commit/3f2757dbf32a
[3/3] KVM: VMX: Print out "bad" offsets+value on VMCS config mismatch
      https://github.com/kvm-x86/linux/commit/c0d6b8bbbced

--
https://github.com/kvm-x86/linux/tree/next