Skip to contentSkip to conversation

KVM: x86: Introduce has_protected_pmu state for TDX VMs

PATCH Superseded Vishal Annapurve 2 participants 3 messages May 7 – May 13

Subsystems (4)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 KVM: x86: Introduce has_protected_pmu state for TDX VMs

PMU state for TDX VMs is virtualized by TDX Module [1]. Host has following toggles to control the PMU functionality exposed to TDX VMs:
1) Configure TD_PARAMS to allow guests to use performance monitoring.
2) Restrict the TD to a subset of the PEBS counters if supported.
3) Limit the TD to setup a certain perfmon events using basic/enhanced event filtering.

KVM will need to be enlightened to support these toggles. Introduce has_protected_pmu state to track the pmu state for such scenarios and explicitly set the has_protected_pmu flag for TDX VMs.

If pmu state is protected:
1) Disable KVM's PMU virtualization framework as additional enlightenment is needed within KVM to control/manage the visibility of PMU state to such VMs.
2) Disallow userspace VMM from enabling PMU virtualization state using KVM_CAP_PMU_CAPABILITY.

[1] Section 15.2: https://cdrdv2.intel.com/v1/dl/getContent/733575

---
v3 -> v4:
- Allow userspace to disable pmu virtualization even if has_protected_pmu is set.
- Addressed feedback around enable_pmu initialization in kvm_arch_init_vm().

v3: https://lore.kernel.org/kvm/[email protected]/

 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/vmx/tdx.c          | 6 ++++++
 arch/x86/kvm/x86.c              | 9 ++++++++-
 3 files changed, 15 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +1 -0
index c470e40a00aa..8371dcaaed1a 100644
@@ -1422,6 +1422,7 @@ struct kvm_arch {
 	bool has_private_mem;
 	bool has_protected_state;
 	bool has_protected_eoi;
+	bool has_protected_pmu;
 	bool pre_fault_allowed;
 	struct hlist_head *mmu_page_hash;
 	struct list_head active_mmu_pages;
arch/x86/kvm/vmx/tdx.c +6 -0
index 1e47c194af53..eb4b4518e6f0 100644
@@ -638,6 +638,12 @@ int tdx_vm_init(struct kvm *kvm)
 	kvm->arch.has_private_mem = true;
 	kvm->arch.disabled_quirks |= KVM_X86_QUIRK_IGNORE_GUEST_PAT;
 
+	/*
+	 * PMU support is provided by the TDX-Module (if enabled for the VM).
+	 * From KVM's perspective, the VM doesn't have a virtual PMU.
+	 */
+	kvm->arch.has_protected_pmu = true;
+
 	/*
 	 * Because guest TD is protected, VMM can't parse the instruction in TD.
 	 * Instead, guest uses MMIO hypercall.  For unmodified device driver,
arch/x86/kvm/x86.c +8 -1
index 0a1b63c63d1a..c248abb0e2ab 100644
@@ -6909,6 +6909,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		if (!enable_pmu || (cap->args[0] & ~KVM_CAP_PMU_VALID_MASK))
 			break;
 
+		if (kvm->arch.has_protected_pmu) {
+			WARN_ON_ONCE(kvm->arch.enable_pmu);
+			if (cap->args[0] == KVM_PMU_CAP_DISABLE)
+				r = 0;
+			break;
+		}
+
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
@@ -13375,7 +13382,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz;
 	kvm->arch.apic_bus_cycle_ns = APIC_BUS_CYCLE_NS_DEFAULT;
 	kvm->arch.guest_can_read_msr_platform_info = true;
-	kvm->arch.enable_pmu = enable_pmu;
+	kvm->arch.enable_pmu = enable_pmu && !kvm->arch.has_protected_pmu;
 
 #if IS_ENABLED(CONFIG_HYPERV)
 	spin_lock_init(&kvm->arch.hv_root_tdp_lock);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
#1 lore raw

arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6916

Quoted 13 lines · Vishal Annapurve
arch/x86/kvm/x86.c +5 -0
index 0a1b63c63d1a..c248abb0e2ab 100644
@@ -6909,6 +6909,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		if (!enable_pmu || (cap->args[0] & ~KVM_CAP_PMU_VALID_MASK))
 			break;
 
+		if (kvm->arch.has_protected_pmu) {
+			WARN_ON_ONCE(kvm->arch.enable_pmu);
+			if (cap->args[0] == KVM_PMU_CAP_DISABLE)
+				r = 0;
+			break;

Sashiko: 3, Sean: 0

: Does this code bypass standard lifecycle validation?
:
: For standard VMs, KVM strictly enforces that PMU capabilities can only be : configured before any vCPUs are created, returning -EINVAL otherwise. The newly : added block for TDX VMs short-circuits this logic, returning 0 unconditionally : when disabling the PMU, regardless of whether vCPUs have been created. : Could this introduce a UAPI inconsistency where the same userspace sequence : could fail on standard VMs but silently succeed on TDX VMs, potentially : hiding VMM initialization order bugs?

That potential issue crossed my mind when first trying to figure out how to allow KVM_PMU_CAP_DISABLE for protected PMUs, but then SQUIRREL!!!

I think my original reaction is the way to go: add extra validation for protected PMUs (reject everything except CAP_DISABLE), but otherwise let the normal flow do it's thing.

Completely untested, but I think this would do what we want?

	case KVM_CAP_PMU_CAPABILITY:
		r = -EINVAL;
		if (!enable_pmu || (cap->args[0] & ~KVM_CAP_PMU_VALID_MASK))
			break;

		if (kvm->arch.has_protected_pmu &&
		    cap->args[0] != KVM_PMU_CAP_DISABLE)
			break;

		mutex_lock(&kvm->lock);
		if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) {
			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
			r = 0;
		}
		mutex_unlock(&kvm->lock);
		break;
#2 lore raw
Quoted 28 lines · Sean Christopherson

arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6916

Quoted 13 lines · Vishal Annapurve
arch/x86/kvm/x86.c +5 -0
index 0a1b63c63d1a..c248abb0e2ab 100644
@@ -6909,6 +6909,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
              if (!enable_pmu || (cap->args[0] & ~KVM_CAP_PMU_VALID_MASK))
                      break;
 
+             if (kvm->arch.has_protected_pmu) {
+                     WARN_ON_ONCE(kvm->arch.enable_pmu);
+                     if (cap->args[0] == KVM_PMU_CAP_DISABLE)
+                             r = 0;
+                     break;

Sashiko: 3, Sean: 0

: Does this code bypass standard lifecycle validation?
:
: For standard VMs, KVM strictly enforces that PMU capabilities can only be : configured before any vCPUs are created, returning -EINVAL otherwise. The newly : added block for TDX VMs short-circuits this logic, returning 0 unconditionally : when disabling the PMU, regardless of whether vCPUs have been created. : Could this introduce a UAPI inconsistency where the same userspace sequence : could fail on standard VMs but silently succeed on TDX VMs, potentially : hiding VMM initialization order bugs?

Quoted 8 lines

That potential issue crossed my mind when first trying to figure out how to allow KVM_PMU_CAP_DISABLE for protected PMUs, but then SQUIRREL!!!

I think my original reaction is the way to go: add extra validation for protected PMUs (reject everything except CAP_DISABLE), but otherwise let the normal flow do it's thing.

Completely untested, but I think this would do what we want?

SGTM. Will include this suggestion in the next version.

Quoted 17 lines
        case KVM_CAP_PMU_CAPABILITY:
                r = -EINVAL;
                if (!enable_pmu || (cap->args[0] & ~KVM_CAP_PMU_VALID_MASK))
                        break;

                if (kvm->arch.has_protected_pmu &&
                    cap->args[0] != KVM_PMU_CAP_DISABLE)
                        break;

                mutex_lock(&kvm->lock);
                if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) {
                        kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
                        r = 0;
                }
                mutex_unlock(&kvm->lock);
                break;