KVM: x86: Introduce has_protected_pmu state for TDX VMs
Subsystems (4)
PATCH v3 1/1 KVM: x86: Introduce has_protected_pmu state for TDX VMs
PMU state for TDX VMs is virtualized by TDX Module [1]. Host has following toggles to control the PMU functionality exposed to TDX VMs:
1) Configure TD_PARAMS to allow guests to use performance monitoring.
2) Restrict the TD to a subset of the PEBS counters if supported.
3) Limit the TD to setup a certain perfmon events using basic/enhanced event filtering.
KVM will need to be enlightened to support these toggles. Introduce has_protected_pmu state to track the pmu state for such scenarios and explicitly set the has_protected_pmu flag for TDX VMs.
If pmu state is protected:
1) Disable KVM's PMU virtualization framework as additional enlightenment is needed within KVM to control/manage the visibility of PMU state to such VMs.
2) Disallow userspace VMM from toggling PMU virtualization state using KVM_CAP_PMU_CAPABILITY.
[1] Section 15.2: https://cdrdv2.intel.com/v1/dl/getContent/733575
---
v2 -> v3:
- Squashed two patches into a single patch as per feedback from Sean.
- Dropped the cover letter.
v2: https://lore.kernel.org/kvm/[email protected]/#t
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/tdx.c | 6 ++++++ arch/x86/kvm/x86.c | 8 ++++++-- 3 files changed, 13 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1422,6 +1422,7 @@ struct kvm_arch {bool has_private_mem;bool has_protected_state;bool has_protected_eoi; +bool has_protected_pmu;bool pre_fault_allowed;struct hlist_head *mmu_page_hash;struct list_head active_mmu_pages;
arch/x86/kvm/vmx/tdx.c +6 -0
@@ -638,6 +638,12 @@ int tdx_vm_init(struct kvm *kvm) kvm->arch.has_private_mem = true; kvm->arch.disabled_quirks |=KVM_X86_QUIRK_IGNORE_GUEST_PAT ; +/* +* PMU support is provided by the TDX-Module (if enabled for the VM). +* From KVM's perspective, the VM doesn't have a virtual PMU. +*/ + kvm->arch.has_protected_pmu = true; +/* * Because guest TD is protected, VMM can't parse the instruction in TD. * Instead, guest uses MMIO hypercall. For unmodified device driver,
arch/x86/kvm/x86.c +6 -2
@@ -6910,7 +6910,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;mutex_lock (&kvm->lock); -if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) { +if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu && + !kvm->arch.has_protected_pmu) { kvm->arch.enable_pmu = !(cap->args[0 ] &KVM_PMU_CAP_DISABLE ); r =0 ; }
@@ -13375,7 +13376,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.apic_bus_cycle_ns =APIC_BUS_CYCLE_NS_DEFAULT ; kvm->arch.guest_can_read_msr_platform_info = true; - kvm->arch.enable_pmu = enable_pmu; +if (kvm->arch.has_protected_pmu) + kvm->arch.enable_pmu = false; +else + kvm->arch.enable_pmu = enable_pmu;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerQuoted 27 lines · Vishal Annapurve
PMU state for TDX VMs is virtualized by TDX Module [1]. Host has following toggles to control the PMU functionality exposed to TDX VMs:
1) ConfigureTD_PARAMSto allow guests to use performance monitoring.
2) Restrict the TD to a subset of the PEBS counters if supported.
3) Limit the TD to setup a certain perfmon events using basic/enhanced event filtering.KVM will need to be enlightened to support these toggles. Introduce
has_protected_pmustate to track the pmu state for such scenarios and explicitly set thehas_protected_pmuflag for TDX VMs.If pmu state is protected:
1) Disable KVM's PMU virtualization framework as additional enlightenment is needed within KVM to control/manage the visibility of PMU state to such VMs.
2) Disallow userspace VMM from toggling PMU virtualization state usingKVM_CAP_PMU_CAPABILITY.[1] Section 15.2: https://cdrdv2.intel.com/v1/dl/getContent/733575
Suggested-by Sean Christopherson <[email protected]>Signed-off-by Vishal Annapurve <[email protected]>---
v2 -> v3:
- Squashed two patches into a single patch as per feedback from Sean.
- Dropped the cover letter.
...
arch/x86/kvm/x86.c int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) · line 13382
Quoted 9 lines
arch/x86/kvm/x86.c +4
-1@@ -13375,7 +13376,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.apic_bus_cycle_ns =APIC_BUS_CYCLE_NS_DEFAULT ; kvm->arch.guest_can_read_msr_platform_info = true; - kvm->arch.enable_pmu = enable_pmu; +if (kvm->arch.has_protected_pmu) + kvm->arch.enable_pmu = false; +else + kvm->arch.enable_pmu = enable_pmu;
Once more, with feeling :-)
kvm->arch.enable_pmu = enable_pmu && !kvm->arch.has_protected_pmu;
No need for a v4, I'll fixup when applying (assuming this is indeed the directin we take).
Quoted 6 lines
arch/x86/kvm/x86.c +0
-0#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Sean Christopherson
maintainerarch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6914
Quoted 11 lines · Vishal Annapurve
arch/x86/kvm/x86.c +2
-1@@ -6910,7 +6910,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;mutex_lock (&kvm->lock); -if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) { +if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu && + !kvm->arch.has_protected_pmu) {
An offlist Sashiko review pointed out that this will prevent doing KVM_PMU_CAP_DISABLE on a TDX VM, which could prove problematic if userspace is already explicitly disabling vPMU support for TDX. I was thinking that wouldn't be a problem since KVM already disallows KVM_PMU_CAP_DISABLE if enable_pmu=false, but in that case KVM doesn't advertise KVM_CAP_PMU_VALID_MASK:
r = enable_pmu ? KVM_CAP_PMU_VALID_MASK : 0;
And unless I'm misreading QEMU code, this will indeed be a problem.
Argh. I was going to suggest something sightly fancier, but we rather stupidly didn't provide a flag for ENABLE, and so we can't feed has_protected_pmu into the valid mask. Lame.
The other super minor detail is that kvm->arch.has_protected_pmu isn't guarded by kvm->lock, and I really hope we never get to that point (e.g. for SNP).
So instead of disallowing KVM_CAP_PMU_CAPABILITY entirely, this?
x86/kvm/x86.c +7 -0
@@ -6911,6 +6911,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,if (!enable_pmu || (cap->args[0 ] & ~KVM_CAP_PMU_VALID_MASK ))break ; +if (kvm->arch.has_protected_pmu) { +WARN_ON_ONCE (kvm->arch.enable_pmu); +if (cap->args[0 ] ==KVM_PMU_CAP_DISABLE ) + r =0 ; +break ; + } +mutex_lock (&kvm->lock);if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) { kvm->arch.enable_pmu = !(cap->args[0 ] &KVM_PMU_CAP_DISABLE );
Quoted 18 lines
arch/x86/kvm/x86.c +4
-1kvm->arch.enable_pmu = !(cap->args[0 ] &KVM_PMU_CAP_DISABLE ); r =0 ; }@@ -13375,7 +13376,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.apic_bus_cycle_ns =APIC_BUS_CYCLE_NS_DEFAULT ; kvm->arch.guest_can_read_msr_platform_info = true; - kvm->arch.enable_pmu = enable_pmu; +if (kvm->arch.has_protected_pmu) + kvm->arch.enable_pmu = false; +else + kvm->arch.enable_pmu = enable_pmu;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock);Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Quoted 42 lines · Sean Christopherson
arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6914
Quoted 11 lines · Vishal Annapurve
arch/x86/kvm/x86.c +2
-1@@ -6910,7 +6910,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,break ;mutex_lock (&kvm->lock); -if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) { +if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu && + !kvm->arch.has_protected_pmu) {An offlist Sashiko review pointed out that this will prevent doing
KVM_PMU_CAP_DISABLEon a TDX VM, which could prove problematic if userspace is already explicitly disabling vPMU support for TDX. I was thinking that wouldn't be a problem since KVM already disallowsKVM_PMU_CAP_DISABLEif enable_pmu=false, but in that case KVM doesn't advertiseKVM_CAP_PMU_VALID_MASK:r = enable_pmu ? KVM_CAP_PMU_VALID_MASK : 0;And unless I'm misreading QEMU code, this will indeed be a problem.
Argh. I was going to suggest something sightly fancier, but we rather stupidly didn't provide a flag for ENABLE, and so we can't feed
has_protected_pmuinto the valid mask. Lame.The other super minor detail is that
kvm->arch.has_protected_pmuisn't guarded bykvm->lock, and I really hope we never get to that point (e.g. for SNP).So instead of disallowing
KVM_CAP_PMU_CAPABILITYentirely, this?x86/kvm/x86.c +0
-0@@ -6911,6 +6911,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,if (!enable_pmu || (cap->args[0 ] & ~KVM_CAP_PMU_VALID_MASK ))break ;
x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6923
Quoted 10 lines
x86/kvm/x86.c +7
-0+if (kvm->arch.has_protected_pmu) { +WARN_ON_ONCE (kvm->arch.enable_pmu); +if (cap->args[0 ] ==KVM_PMU_CAP_DISABLE ) + r =0 ; +break ; + } +mutex_lock (&kvm->lock);if (!kvm->created_vcpus && !kvm->arch.created_mediated_pmu) { kvm->arch.enable_pmu = !(cap->args[0 ] &KVM_PMU_CAP_DISABLE );
Incorporated this in v4 [1].
Quoted 19 lines
Quoted 19 lines
arch/x86/kvm/x86.c +4
-2kvm->arch.enable_pmu = !(cap->args[0 ] &KVM_PMU_CAP_DISABLE ); r =0 ; }@@ -13375,7 +13376,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; kvm->arch.apic_bus_cycle_ns =APIC_BUS_CYCLE_NS_DEFAULT ; kvm->arch.guest_can_read_msr_platform_info = true; - kvm->arch.enable_pmu = enable_pmu; +if (kvm->arch.has_protected_pmu) + kvm->arch.enable_pmu = false; +else + kvm->arch.enable_pmu = enable_pmu;#if IS_ENABLED (CONFIG_HYPERV )spin_lock_init (&kvm->arch.hv_root_tdp_lock); --2.54.0.563.g4f69b47b94-goog