Skip to contentSkip to conversation

KVM: Add support for hardware-switched mediated PMU

PATCH In review Sandipan Das 3 participants 20 messages Aug 10 – Aug 11

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover KVM: Add support for hardware-switched mediated PMU

Mediated PMU currently saves and restores the guest PMU state in software on every world switch. On Zen 5 and later processors, it is possible to offload this to the hardware, which automatically saves and restores the guest counter state to and from a vendor save area (VMCB) during world switches. This feature is called PMC virtualization and detailed information about it can be found in Volume 2 of the AMD64 Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched and hardware-switched modes is measured with the x86/vmexit test from kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test sw-switched hw-switched gain
------------------------------------------------------------------------

cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%

------------------------------------------------------------------------

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid (+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the kernel-handled port I/O paths (~+50%), and the control-register toggles (~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by ~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all within run-to-run noise.

Design
------
* The entity responsible for switching the guest PMU state is tracked per-VM via a new set of mediated PMU capability flags in kvm_arch. In software-switched mode the state resides in struct kvm_pmu; in hardware-switched mode it resides in the vendor save area.
* Event filtering, instruction emulation, and intercepted RDPMC still need the state in struct kvm_pmu, so new vendor state-sync PMU ops synchronize specific MSRs between struct kvm_pmu and the vendor save area on demand.
* On SVM, the feature depends on VNMI for guest PMI delivery and is controlled by the new "vpmc" kvm_amd module parameter, enabled by default when the host supports it and mediated PMU is enabled.

Limitations
-----------
* Not supported for SEV-ES and SEV-SNP guests: the guest PMU state resides in an encrypted VMSA that is inaccessible to the event filtering logic.
* With VNMI and no AVIC, overflow interrupts are always delivered as NMIs because the APIC is emulated and the LVTPC is ignored. This shows up as expected behavioral differences in the x86/pmu test of kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature bit, the VMCB save-area fields, and the SVM implementation. Patch 7 adds nested (nSVM) support to use the feature with L2 guests. Patch 8 explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test requires patching to accept overflow interrupts as NMIs (limitation because of VNMI ignoring the LVTPC). It is recommended to apply the following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d

             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/[email protected]/

Changes in v1:
* Introduce mediated PMU capabilities for hardware PMU virtualization features instead of perf PMU capabilities.
* Make VNMI a hard requirement since AVIC can be inhibited.
* Introduce new PMU ops to access VMCB PMU state instead of relying on host-initiated {get,set}_msr() ops.
* Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
* Implement support for nested SVM.
* Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
KVM: x86/pmu: Add mediated PMU capability flags
KVM: x86/pmu: Add PMU ops for vendor state sync
KVM: x86/pmu: Add support for hardware-switched PMU x86/cpufeatures: Add PerfCtrVirt feature bit
KVM: SVM: Add VMCB fields for PMC virtualization
KVM: SVM: Add support for hardware-switched PMU
KVM: nSVM: Add support for hardware-switched PMU
KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)
Signature · 1 line
2.53.0
#1 lore raw

PATCH 1/8 KVM: x86/pmu: Add mediated PMU capability flags

When mediated PMU is enabled, the guest PMU state is currently saved and restored in software. On capable processors, this can be offloaded to hardware. Hence, introduce the notion of VM-scoped mediated PMU capabilities to help KVM determine the available hardware assists.

When set, the KVM_MEDIATED_PMU_CAP_HW_SWITCHED flag indicates that hardware is responsible for switching the guest PMU state. When software-switched, the guest PMU state continues to reside in struct kvm_pmu. However, when hardware-switched, the guest PMU state resides in a vendor save area.

While KVM_MEDIATED_PMU_CAP_HW_FILTERED remains unused currently, when set, it indicates that hardware is also responsible for assisting in guest event filtering for further reduction of overhead resulting from intercepting event selector writes.

The capabilities are expected to be set in the vm_init() call. All capabilities are cleared if kvm_arch_vcpu_precreate() fails to create mediated PMU.

---
 arch/x86/include/asm/kvm_host.h | 11 +++++++++++
 arch/x86/kvm/pmu.h              | 20 ++++++++++++++++++++
 arch/x86/kvm/x86.c              |  2 ++
 3 files changed, 33 insertions(+)
arch/x86/include/asm/kvm_host.h +11 -0
index 283847619ff8..338df11e2a4d 100644
@@ -715,6 +715,16 @@ enum kvm_only_cpuid_leafs {
 	NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS,
 };
 
+/*
+ * Track how the guest PMU state is saved and restored, as this can be done
+ * either by software or, if capable, by hardware. When hardware manages the
+ * guest PMU state, the state resides in a vendor save area.
+ */
+#define KVM_MEDIATED_PMU_CAP_HW_SWITCHED	BIT(0)
+#define KVM_MEDIATED_PMU_CAP_HW_FILTERED	BIT(1)
+#define KVM_MEDIATED_PMU_CAP_VALID		(KVM_MEDIATED_PMU_CAP_HW_SWITCHED	| \
+						 KVM_MEDIATED_PMU_CAP_HW_FILTERED)
+
 struct kvm_vcpu_arch {
 	/*
 	 * rip and regs accesses must go through
@@ -1284,6 +1294,7 @@ struct kvm_arch {
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
 	bool created_mediated_pmu;
+	u32 mediated_pmu_caps;
 
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +20 -0
index 090c9bbb74f4..1aa46d3e2339 100644
@@ -88,6 +88,26 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
 	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
 }
 
+static inline bool kvm_vcpu_has_mediated_pmu_caps(struct kvm_vcpu *vcpu, u32 caps)
+{
+	return kvm_vcpu_has_mediated_pmu(vcpu) &&
+	       !!(vcpu->kvm->arch.mediated_pmu_caps & caps);
+}
+
+#define kvm_set_mediated_pmu_caps(kvm, caps)				\
+do {									\
+	BUILD_BUG_ON(!__builtin_constant_p(caps) ||			\
+		     ((caps) & ~KVM_MEDIATED_PMU_CAP_VALID));		\
+	(kvm)->arch.mediated_pmu_caps |= (caps);			\
+} while (0)
+
+#define kvm_clr_mediated_pmu_caps(kvm, caps)				\
+do {									\
+	BUILD_BUG_ON(!__builtin_constant_p(caps) ||			\
+		     ((caps) & ~KVM_MEDIATED_PMU_CAP_VALID));		\
+	(kvm)->arch.mediated_pmu_caps &= ~(caps);			\
+} while (0)
+
 /*
  * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
  * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
arch/x86/kvm/x86.c +2 -0
index d94b59140c45..4287982638b6 100644
@@ -9326,6 +9326,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
 		if (irqchip_in_kernel(kvm)) {
 			r = perf_create_mediated_pmu();
 			if (r) {
+				kvm->arch.mediated_pmu_caps = 0;
 				pr_warn_ratelimited(PERF_MEDIATED_PMU_MSG);
 				return r;
 			}
@@ -9334,6 +9335,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
 			kvm->arch.enable_pmu = false;
 		}
 	}
+
 	return 0;
 }
 
Signature · 1 line
2.53.0
#3 lore raw

PATCH 2/8 KVM: x86/pmu: Add PMU ops for vendor state sync

In reply to Sandipan Das

When hardware-switched mediated PMU is enabled, the guest PMU state resides in a vendor save area. Because of this, after every VMEXIT, the state inside struct kvm_pmu can go out of sync with the vendor save area. Since KVM expects struct kvm_pmu to have the correct guest PMU state during event filtering and instruction emulation, introduce new PMU ops to opportunistically synchronize the state of PMU MSRs between the two backing stores.

The new ops are directional. get_vendor_state() copies data from the vendor save area to the corresponding member of struct kvm_pmu while set_vendor_state() does the opposite.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 3 +++
 arch/x86/kvm/pmu.c                     | 5 ++++-
 arch/x86/kvm/pmu.h                     | 2 ++
 3 files changed, 9 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm-x86-pmu-ops.h +3 -0
index 4a223c2793e3..4a617162d4c0 100644
@@ -29,6 +29,9 @@ KVM_X86_PMU_OP_OPTIONAL_RET0(pmc_is_disabled_in_current_mode)
 KVM_X86_PMU_OP_OPTIONAL(write_global_ctrl)
 KVM_X86_PMU_OP(mediated_load)
 KVM_X86_PMU_OP(mediated_put)
+
+KVM_X86_PMU_OP_OPTIONAL_RET0(get_vendor_state)
+KVM_X86_PMU_OP_OPTIONAL_RET0(set_vendor_state)
 #endif
 
 #undef KVM_X86_PMU_OP
arch/x86/kvm/pmu.c +4 -1
index a7d60c8785cd..5ede3ee968e3 100644
@@ -166,8 +166,11 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 	    !pmu_ops->is_mediated_pmu_supported(&kvm_host_pmu))
 		enable_mediated_pmu = false;
 
-	if (!enable_mediated_pmu)
+	if (!enable_mediated_pmu) {
 		pmu_ops->write_global_ctrl = NULL;
+		pmu_ops->get_vendor_state = NULL;
+		pmu_ops->set_vendor_state = NULL;
+	}
 
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
arch/x86/kvm/pmu.h +2 -0
index 1aa46d3e2339..5d6f6863ad40 100644
@@ -42,6 +42,8 @@ struct kvm_pmu_ops {
 	void (*mediated_load)(struct kvm_vcpu *vcpu);
 	void (*mediated_put)(struct kvm_vcpu *vcpu);
 	void (*write_global_ctrl)(u64 global_ctrl);
+	int (*get_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);
+	int (*set_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
Signature · 1 line
2.53.0
#5 lore raw

PATCH 3/8 KVM: x86/pmu: Add support for hardware-switched PMU

In reply to Sandipan Das

Implement the hardware-switched mode of mediated PMU. The key design differences from the software-switched mode are listed below.
* Hardware handles context-switching of the guest PMU state.
* The guest PMU state resides in a vendor save area (such as VMCB or VMCS) instead of struct kvm_pmu.

When enabled, the load and put functionality of mediated PMU simplifies to just scheduling the active host events in and out.

Event filtering and instruction emulation require the ability to change the guest PMU state in software. Since struct kvm_pmu is not guaranteed to always have the correct state, make use of the new vendor state sync ops to access MSR states directly from a vendor save area. These are also used to read the latest counter value when RDPMC is intercepted for legacy guests with fewer counters.

The software-switched mode continues to be the default for hardware lacking the features to support this capability.

---
 arch/x86/kvm/pmu.c           | 97 +++++++++++++++++++++++++++++-------
 arch/x86/kvm/pmu.h           |  7 +++
 arch/x86/kvm/svm/pmu.c       |  2 +
 arch/x86/kvm/svm/svm.c       |  3 +-
 arch/x86/kvm/vmx/pmu_intel.c |  2 +
 arch/x86/kvm/vmx/vmx.c       |  3 +-
 6 files changed, 95 insertions(+), 19 deletions(-)
arch/x86/kvm/pmu.c +80 -17
index 5ede3ee968e3..4c918aa6d6f3 100644
@@ -130,6 +130,24 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
+static void kvm_pmu_get_vendor_state(struct kvm_vcpu *vcpu, u32 msr)
+{
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
+	if (kvm_pmu_call(get_vendor_state)(vcpu, msr))
+		kvm_pmu_warn_vendor_state(msr);
+}
+
+static void kvm_pmu_set_vendor_state(struct kvm_vcpu *vcpu, u32 msr)
+{
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
+	if (kvm_pmu_call(set_vendor_state)(vcpu, msr))
+		kvm_pmu_warn_vendor_state(msr);
+}
+
 void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 {
 	bool is_intel = boot_cpu_data.x86_vendor == X86_VENDOR_INTEL;
@@ -200,6 +218,27 @@ void kvm_handle_guest_mediated_pmi(void)
 	kvm_make_request(KVM_REQ_PMI, vcpu);
 }
 
+static __always_inline u32 fixed_counter_msr(u32 idx)
+{
+	return kvm_pmu_ops.FIXED_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
+}
+
+static __always_inline u32 gp_counter_msr(u32 idx)
+{
+	return kvm_pmu_ops.GP_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
+}
+
+static __always_inline u32 gp_eventsel_msr(u32 idx)
+{
+	return kvm_pmu_ops.GP_EVENTSEL_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
+}
+
+static __always_inline u32 pmc_counter_msr(struct kvm_pmc *pmc)
+{
+	return pmc_is_gp(pmc) ? gp_counter_msr(pmc->idx) :
+				fixed_counter_msr(pmc->idx - KVM_FIXED_PMC_BASE_IDX);
+}
+
 static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
@@ -553,18 +592,23 @@ static void kvm_mediated_pmu_refresh_event_filter(struct kvm_pmc *pmc)
 {
 	bool allowed = pmc_is_locally_enabled(pmc) && pmc_is_event_allowed(pmc);
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
+	struct kvm_vcpu *vcpu = pmc->vcpu;
 
 	if (pmc_is_gp(pmc)) {
 		pmc->eventsel_hw &= ~ARCH_PERFMON_EVENTSEL_ENABLE;
 		if (allowed)
 			pmc->eventsel_hw |= pmc->eventsel &
 					    ARCH_PERFMON_EVENTSEL_ENABLE;
+
+		kvm_pmu_set_vendor_state(vcpu, gp_eventsel_msr(pmc->idx));
 	} else {
 		u64 mask = intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX, 0xf);
 
 		pmu->fixed_ctr_ctrl_hw &= ~mask;
 		if (allowed)
 			pmu->fixed_ctr_ctrl_hw |= pmu->fixed_ctr_ctrl & mask;
+
+		kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.FIXED_COUNTER_CTRL);
 	}
 }
 
@@ -772,6 +816,8 @@ int kvm_pmu_rdpmc(struct kvm_vcpu *vcpu, unsigned idx, u64 *data)
 	    kvm_is_cr0_bit_set(vcpu, X86_CR0_PE))
 		return 1;
 
+	kvm_pmu_get_vendor_state(vcpu, pmc_counter_msr(pmc));
+
 	*data = pmc_read_counter(pmc) & mask;
 	return 0;
 }
@@ -855,10 +901,12 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	switch (msr) {
 	case MSR_CORE_PERF_GLOBAL_STATUS:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
+		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 		msr_info->data = pmu->global_status;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 	case MSR_CORE_PERF_GLOBAL_CTRL:
+		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
 		msr_info->data = pmu->global_ctrl;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
@@ -898,6 +946,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1;
 
 		pmu->global_status = data;
+		kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 		data &= ~pmu->global_ctrl_rsvd;
@@ -917,6 +966,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		 */
 		if (kvm_vcpu_has_mediated_pmu(vcpu))
 			kvm_pmu_call(write_global_ctrl)(data);
+
+		kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
 		/*
@@ -927,12 +978,18 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 1;
 		fallthrough;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
-		if (!msr_info->host_initiated)
+		if (!msr_info->host_initiated) {
+			kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 			pmu->global_status &= ~data;
+			kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+		}
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
-		if (!msr_info->host_initiated)
+		if (!msr_info->host_initiated) {
+			kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 			pmu->global_status |= data & ~pmu->global_status_rsvd;
+			kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+		}
 		break;
 	default:
 		kvm_pmu_mark_pmc_in_use(vcpu, msr_info->index);
@@ -1020,6 +1077,8 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (kvm_vcpu_has_mediated_pmu(vcpu))
 		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
 
+	kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
+
 	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
 	bitmap_set(pmu->all_valid_pmc_idx, KVM_FIXED_PMC_BASE_IDX,
 		   pmu->nr_arch_fixed_counters);
@@ -1142,6 +1201,9 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
 	if (bitmap_empty(event_pmcs, X86_PMC_IDX_MAX))
 		return;
 
+	kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
+	kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+
 	if (!kvm_pmu_has_perf_global_ctrl(pmu))
 		bitmap_copy(bitmap, event_pmcs, X86_PMC_IDX_MAX);
 	else if (!bitmap_and(bitmap, event_pmcs,
@@ -1150,11 +1212,17 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		kvm_pmu_get_vendor_state(vcpu, pmc_counter_msr(pmc));
+
 		if (!pmc_is_event_allowed(pmc) || !cpl_is_matched(pmc))
 			continue;
 
 		kvm_pmu_incr_counter(pmc);
+		kvm_pmu_set_vendor_state(vcpu, pmc_counter_msr(pmc));
 	}
+
+	kvm_pmu_set_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
+
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
 }
 
@@ -1313,21 +1381,6 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	return r;
 }
 
-static __always_inline u32 fixed_counter_msr(u32 idx)
-{
-	return kvm_pmu_ops.FIXED_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
-}
-
-static __always_inline u32 gp_counter_msr(u32 idx)
-{
-	return kvm_pmu_ops.GP_COUNTER_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
-}
-
-static __always_inline u32 gp_eventsel_msr(u32 idx)
-{
-	return kvm_pmu_ops.GP_EVENTSEL_BASE + idx * kvm_pmu_ops.MSR_STRIDE;
-}
-
 static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
@@ -1364,6 +1417,10 @@ void kvm_mediated_pmu_load(struct kvm_vcpu *vcpu)
 
 	perf_load_guest_context();
 
+	/* Guest PMU state is restored by hardware */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		return;
+
 	/*
 	 * Explicitly clear PERF_GLOBAL_CTRL, as "loading" the guest's context
 	 * disables all individual counters (if any were enabled), but doesn't
@@ -1423,6 +1480,12 @@ void kvm_mediated_pmu_put(struct kvm_vcpu *vcpu)
 
 	lockdep_assert_irqs_disabled();
 
+	/* Guest PMU state is saved by hardware */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED)) {
+		perf_put_guest_context();
+		return;
+	}
+
 	/*
 	 * Defer handling of PERF_GLOBAL_CTRL to vendor code.  On Intel, it's
 	 * atomically cleared on VM-Exit, i.e. doesn't need to be clear here.
arch/x86/kvm/pmu.h +7 -0
index 5d6f6863ad40..42d33072ece9 100644
@@ -50,8 +50,10 @@ struct kvm_pmu_ops {
 	const int MIN_NR_GP_COUNTERS;
 
 	const u32 PERF_GLOBAL_CTRL;
+	const u32 PERF_GLOBAL_STATUS;
 	const u32 GP_EVENTSEL_BASE;
 	const u32 GP_COUNTER_BASE;
+	const u32 FIXED_COUNTER_CTRL;
 	const u32 FIXED_COUNTER_BASE;
 	const u32 MSR_STRIDE;
 };
@@ -110,6 +112,11 @@ do {									\
 	(kvm)->arch.mediated_pmu_caps &= ~(caps);			\
 } while (0)
 
+static inline void kvm_pmu_warn_vendor_state(u32 msr)
+{
+	WARN_ONCE(1, "accessing unsupported vendor save slot for MSR 0x%x\n", msr);
+}
+
 /*
  * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
  * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
arch/x86/kvm/svm/pmu.c +2 -0
index c18286545a7a..5dccf8776368 100644
@@ -321,8 +321,10 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.MIN_NR_GP_COUNTERS = AMD64_NUM_COUNTERS,
 
 	.PERF_GLOBAL_CTRL = MSR_AMD64_PERF_CNTR_GLOBAL_CTL,
+	.PERF_GLOBAL_STATUS = MSR_AMD64_PERF_CNTR_GLOBAL_STATUS,
 	.GP_EVENTSEL_BASE = MSR_F15H_PERF_CTL0,
 	.GP_COUNTER_BASE = MSR_F15H_PERF_CTR0,
+	.FIXED_COUNTER_CTRL = 0,
 	.FIXED_COUNTER_BASE = 0,
 	.MSR_STRIDE = 2,
 };
arch/x86/kvm/svm/svm.c +2 -1
index 9d607b98bd06..2e39b5e752cd 100644
@@ -4618,7 +4618,8 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
 
 	kvm_clear_available_registers(vcpu, SVM_REGS_LAZY_LOAD_SET);
 
-	if (!msr_write_intercepted(svm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
+	if (!kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
+	    !msr_write_intercepted(svm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
 		rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, vcpu_to_pmu(vcpu)->global_ctrl);
 
 	trace_kvm_exit(vcpu, KVM_ISA_SVM);
arch/x86/kvm/vmx/pmu_intel.c +2 -0
index bfa8612fb450..d0723814410e 100644
@@ -853,8 +853,10 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.MIN_NR_GP_COUNTERS = 1,
 
 	.PERF_GLOBAL_CTRL = MSR_CORE_PERF_GLOBAL_CTRL,
+	.PERF_GLOBAL_STATUS = MSR_CORE_PERF_GLOBAL_STATUS,
 	.GP_EVENTSEL_BASE = MSR_P6_EVNTSEL0,
 	.GP_COUNTER_BASE = MSR_IA32_PMC0,
+	.FIXED_COUNTER_CTRL = MSR_CORE_PERF_FIXED_CTR_CTRL,
 	.FIXED_COUNTER_BASE = MSR_CORE_PERF_FIXED_CTR0,
 	.MSR_STRIDE = 1,
 };
arch/x86/kvm/vmx/vmx.c +2 -1
index e3bfe6aca1a0..7d8c032c1897 100644
@@ -7383,7 +7383,8 @@ static void vmx_refresh_guest_perf_global_control(struct kvm_vcpu *vcpu)
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
 
-	if (msr_write_intercepted(vmx, MSR_CORE_PERF_GLOBAL_CTRL))
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) ||
+	    msr_write_intercepted(vmx, MSR_CORE_PERF_GLOBAL_CTRL))
 		return;
 
 	if (!cpu_has_save_perf_global_ctrl()) {
Signature · 1 line
2.53.0
#7 lore raw

PATCH 4/8 x86/cpufeatures: Add PerfCtrVirt feature bit

In reply to Sandipan Das

Define a feature flag for bit 8 of CPUID leaf 0x8000000A EDX which indicates support for PMC virtualization. Hypervisors can use this feature to offload the guest PMU context switch to hardware. Hence, this is a required host capability for enabling hardware-switched mediated PMU in KVM.

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/cpufeatures.h +1 -0
index 30bb8fdf2a8b..b551ba381b6e 100644
@@ -384,6 +384,7 @@
 #define X86_FEATURE_VMCBCLEAN		(15*32+ 5) /* "vmcb_clean" VMCB clean bits support */
 #define X86_FEATURE_FLUSHBYASID		(15*32+ 6) /* "flushbyasid" Flush-by-ASID support */
 #define X86_FEATURE_DECODEASSISTS	(15*32+ 7) /* "decodeassists" Decode Assists support */
+#define X86_FEATURE_PERFCTR_VIRT	(15*32+ 8) /* PMC virtualization support */
 #define X86_FEATURE_PAUSEFILTER		(15*32+10) /* "pausefilter" Filtered pause intercept */
 #define X86_FEATURE_PFTHRESHOLD		(15*32+12) /* "pfthreshold" Pause filter threshold */
 #define X86_FEATURE_AVIC		(15*32+13) /* "avic" Virtual Interrupt Controller */
Signature · 1 line
2.53.0
#9 lore raw

PATCH 5/8 KVM: SVM: Add VMCB fields for PMC virtualization

In reply to Sandipan Das

When PMC virtualization (X86_FEATURE_PERFCTR_VIRT) is supported and the feature is enabled, additional save slots are available in the VMCB for the following MSRs.
* Performance Counter Global Control (MSR 0xc0000301) (Swap Type C)
* Performance Counter Global Status (MSR 0xc0000300) (Swap Type A)
* Performance Event Select (MSR 0xc0010200..0xc001020a) (Swap Type C)
* Performance Event Counter (MSR 0xc0010201..0xc001020b) (Swap Type C)

Define the additional VMCB fields that will be used by hardware to save and restore the guest PMU state.

---
 arch/x86/include/asm/svm.h | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
arch/x86/include/asm/svm.h +10 -2
index aa63431ba92c..73f17af22d6c 100644
@@ -338,7 +338,12 @@ struct vmcb_save_area {
 	u8 cpl;
 	u8 reserved_0xcc[4];
 	u64 efer;
-	u8 reserved_0xd8[112];
+	u8 reserved_0xd8[8];
+	struct {
+		u64 perf_ctl;
+		u64 perf_ctr;
+	} __packed pmc[6];
+	u8 reserved_0x140[8];
 	u64 cr4;
 	u64 cr3;
 	u64 cr0;
@@ -346,7 +351,9 @@ struct vmcb_save_area {
 	u64 dr6;
 	u64 rflags;
 	u64 rip;
-	u8 reserved_0x180[88];
+	u8 reserved_0x180[72];
+	u64 perf_cntr_global_status;
+	u64 perf_cntr_global_control;
 	u64 rsp;
 	u64 s_cet;
 	u64 ssp;
@@ -575,6 +582,7 @@ static inline void __unused_size_checks(void)
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0xa0);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0xcc);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0xd8);
+	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x140);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x180);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x248);
 	BUILD_BUG_RESERVED_OFFSET(vmcb_save_area, 0x298);
Signature · 1 line
2.53.0
#10 lore raw

PATCH 6/8 KVM: SVM: Add support for hardware-switched PMU

In reply to Sandipan Das

Zen 5 processors introduced the ability to offload guest PMU context switches to hardware using a feature called PMC virtualization. Use the feature, detected using X86_FEATURE_PERFCTR_VIRT, to implement the new hardware-switched mode of mediated PMU.

The feature is enabled by setting bit 3 of misc_ctl2 residing in the VMCB control area. Since the guest PMU state now resides in the VMCB save area, implement the vendor state sync PMU ops for use during event filtering, instruction emulation and RDPMC interception to access the latest PMU state.

The vendor state sync calls always use MSR_F15H_PERF_{CTL,CTR}x indices because of how GP_EVENTSEL_BASE and GP_COUNTER_BASE are set in the SVM kvm_pmu_ops. For legacy models, where X86_FEATURE_PERFCTR_CORE is not set, these are translated to the equivalent MSR_K7_{EVNTSEL,PERFCTR}x indices, as get_gp_pmc_amd() would otherwise return NULL.

The underlying implementation depends on the availability of either VNMI or AVIC for guest PMI delivery. Synthesized overflows such as those resulting from incrementing counters due to instruction emulation are still injected through the KVM_REQ_PMI processing path. Since AVIC can be inhibited, make VNMI the hard requirement.

When both VNMI and AVIC are enabled, AVIC gets preference in hardware. The main advantage of using AVIC is that it lets the guest change the delivery mode in the LVTPC. Unlike AVIC, VNMI ignores the LVTPC, as the APIC is emulated, and always presents PMIs as NMIs.

PMC virtualization does not use a VMCB clean bit. Hardware reloads the guest PMU state from the VMCB save area on every VMRUN. Hence, any software updates to the VMCB do not require calling vmcb_mark_dirty().

The "vpmc" vendor module parameter is introduced to toggle the feature. Compared to software-switched mediated PMU, enabling the feature reduces the world switch overhead. Hence it is enabled by default when the host supports X86_FEATURE_PERFCTR_VIRT and also has mediated PMU enabled (kvm_amd.enable_mediated_pmu=1).

---
 arch/x86/include/asm/svm.h |   1 +
 arch/x86/kvm/svm/pmu.c     | 115 +++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/svm/svm.c     |  51 ++++++++++++++++
 arch/x86/kvm/svm/svm.h     |   1 +
 4 files changed, 168 insertions(+)
arch/x86/include/asm/svm.h +1 -0
index 73f17af22d6c..e1c8a333e86f 100644
@@ -247,6 +247,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {
 
 #define SVM_MISC2_ENABLE_V_LBR	BIT_ULL(0)
 #define SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE	BIT_ULL(1)
+#define SVM_MISC2_ENABLE_V_PMC	BIT_ULL(3)
 
 #define SVM_TSC_RATIO_RSVD	0xffffff0000000000ULL
 #define SVM_TSC_RATIO_MIN	0x0000000000000001ULL
arch/x86/kvm/svm/pmu.c +115 -0

155 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +51 -0

87 lines not on this page · raw message

arch/x86/kvm/svm/svm.h +1 -0
index 66b44b54608e..eee26f6ce10c 100644
@@ -53,6 +53,7 @@ extern int vgif;
 extern bool intercept_smi;
 extern bool vnmi;
 extern int lbrv;
+extern bool vpmc;
 
 extern int tsc_aux_uret_slot __ro_after_init;
 
Signature · 1 line
2.53.0
#12 lore raw

PATCH 7/8 KVM: nSVM: Add support for hardware-switched PMU

In reply to Sandipan Das

Add nested support for PMC virtualization and advertise X86_FEATURE_PERFCTR_VIRT so that a L1 hypervisor can use it for its L2 guests. As long as the L0 hypervisor uses hardware-switched mediated PMU for L1, it is used while L1 runs L2 guests as well, irrespective of what L1 does.

Since hardware runs L2 from vmcb02, not vmcb12, enable PMC virtualization in vmcb02 whenever it is enabled in vmcb01, so that hardware switches the PMU state using the vmcb02 save area. The state to switch depends on whether L1 enables PMC virtualization for L2.

An L1 hypervisor enables PMC virtualization for L2 by setting bit 3 of misc_ctl2 in vmcb12. Following commit 84dc9fd0354d ("KVM: nSVM: Cache all used fields from VMCB12"), cache the PMU-related fields to avoid TOCTOU hazards. Copy the cached L2 PMU state into vmcb02 for L2 VMRUNs, and back into vmcb12 on nested VMEXITs, so that L1 observes the latest L2 PMU state.

When L1 does not enable PMC virtualization for L2, the PMU state to switch is the one in the vmcb01 save area. Copy it into vmcb02 for L2 VMRUNs, and back into vmcb01 on nested VMEXITs, so that the latest PMU state is preserved for L1.

The counter reprogramming done on nested transitions writes into the vmcb02 save area, so run it after copying the PMU state into vmcb02; otherwise the copy overwrites it.

---
 arch/x86/kvm/cpuid.c      |  1 +
 arch/x86/kvm/svm/nested.c | 45 ++++++++++++++++++++++++++++++++++++---
 arch/x86/kvm/svm/svm.c    |  4 ++++
 arch/x86/kvm/svm/svm.h    | 18 ++++++++++++++++
 4 files changed, 65 insertions(+), 3 deletions(-)
arch/x86/kvm/cpuid.c +1 -0
index ddb022cb203a..ce4f984bc6e1 100644
@@ -1217,6 +1217,7 @@ void kvm_initialize_cpu_caps(void)
 		VENDOR_F(VGIF),
 		VENDOR_F(VNMI),
 		VENDOR_F(SVME_ADDR_CHK),
+		VENDOR_F(PERFCTR_VIRT),
 	);
 
 	kvm_cpu_cap_init(CPUID_8000_001F_EAX,
arch/x86/kvm/svm/nested.c +42 -3

117 lines not on this page · raw message

arch/x86/kvm/svm/svm.c +4 -0
index 079b1a56e995..2023d38bba63 100644
@@ -5583,6 +5583,10 @@ static __init void svm_set_cpu_caps(void)
 		if (vnmi)
 			kvm_cpu_cap_set(X86_FEATURE_VNMI);
 
+		/* Nested AVIC is not supported, so VNMI must be enabled */
+		if (vpmc && vnmi)
+			kvm_cpu_cap_set(X86_FEATURE_PERFCTR_VIRT);
+
 		/* Nested VM can receive #VMEXIT instead of triggering #GP */
 		kvm_cpu_cap_set(X86_FEATURE_SVME_ADDR_CHK);
 	}
arch/x86/kvm/svm/svm.h +18 -0
index eee26f6ce10c..3db6b73e8163 100644
@@ -175,6 +175,12 @@ struct vmcb_save_area_cached {
 	u64 br_to;
 	u64 last_excp_from;
 	u64 last_excp_to;
+	struct {
+		u64 perf_ctl;
+		u64 perf_ctr;
+	} pmc[6];
+	u64 perf_cntr_global_status;
+	u64 perf_cntr_global_control;
 };
 
 struct vmcb_ctrl_area_cached {
@@ -806,6 +812,18 @@ do {								\
 	(to)->last_excp_to	= (from)->last_excp_to;		\
 } while (0)
 
+#define svm_copy_pmcs(to, from)							\
+do {										\
+	int i;									\
+										\
+	(to)->perf_cntr_global_control = (from)->perf_cntr_global_control;	\
+	(to)->perf_cntr_global_status  = (from)->perf_cntr_global_status;	\
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {			\
+		(to)->pmc[i].perf_ctl = (from)->pmc[i].perf_ctl;		\
+		(to)->pmc[i].perf_ctr = (from)->pmc[i].perf_ctr;		\
+	}									\
+} while (0)
+
 void svm_vcpu_free_msrpm(void *msrpm);
 void svm_enable_lbrv(struct kvm_vcpu *vcpu);
 void svm_update_lbrv(struct kvm_vcpu *vcpu);
Signature · 1 line
2.53.0
#14 lore raw

PATCH 8/8 KVM: SEV: Disallow the use of hardware-switched PMU

In reply to Sandipan Das

For hardware-switched mediated PMU, the guest PMU state resides in the VMCB save area instead of struct kvm_pmu. To inspect or modify that state in software, KVM uses the vendor state sync PMU ops.

When SEV-ES or SEV-SNP is used, these ops cannot work as the guest PMU state resides in an encrypted VMSA that KVM can neither read nor write. Clear KVM_MEDIATED_PMU_CAP_HW_SWITCHED for such guests so that they fall back to the software-switched mode.

---
 arch/x86/kvm/svm/sev.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/sev.c +3 -0
index fcb41dfde4c0..8e8cd4feb16a 100644
@@ -534,6 +534,9 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
 	if (snp_active)
 		sev->vmsa_features |= SVM_SEV_FEAT_SNP_ACTIVE;
 
+	if (es_active)
+		kvm_clr_mediated_pmu_caps(kvm, KVM_MEDIATED_PMU_CAP_HW_SWITCHED);
+
 	ret = sev_asid_new(sev, vm_type);
 	if (ret)
 		goto e_no_asid;
Signature · 1 line
2.53.0
#8 lore raw

Re: [PATCH 4/8] x86/cpufeatures: Add PerfCtrVirt feature bit

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The patch modifies `arch/x86/include/asm/cpufeatures.h` but fails to update its synchronized copy…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The patch modifies arch/x86/include/asm/cpufeatures.h but fails to update its synchronized copy in tools/arch/x86/include/asm/cpufeatures.h. --

--- Patch [4]: [PATCH 4/8] x86/cpufeatures: Add PerfCtrVirt feature bit --- commit d0af0746d5778097e8c3fa7f7763e5d0b909732c
Author: Sandipan Das <[email protected]>

x86/cpufeatures: Add PerfCtrVirt feature bit

This commit defines a feature flag for bit 8 of CPUID leaf 0x8000000A EDX which indicates support for PMC virtualization. Hypervisors can use this feature to offload the guest PMU context switch to hardware. This enables hardware-switched mediated PMU in KVM.

arch/x86/include/asm/cpufeatures.h line 377

Quoted 9 lines
arch/x86/include/asm/cpufeatures.h +1 -0
index 1b4a48bff18f7..d6281870efe02 100644
@@ -374,6 +374,7 @@
 #define X86_FEATURE_VMCBCLEAN		(15*32+ 5) /* "vmcb_clean" VMCB clean bits support */
 #define X86_FEATURE_FLUSHBYASID		(15*32+ 6) /* "flushbyasid" Flush-by-ASID support */
 #define X86_FEATURE_DECODEASSISTS	(15*32+ 7) /* "decodeassists" Decode Assists support */
+#define X86_FEATURE_PERFCTR_VIRT	(15*32+ 8) /* PMC virtualization support */

[Severity: Low]
Should this addition also be mirrored in the tools directory?

When arch/x86/include/asm/cpufeatures.h is modified, it usually requires a synchronized update in tools/arch/x86/include/asm/cpufeatures.h.

Does omitting this update trigger a kernel ABI header differences warning from check-headers.sh during the tools/perf build?

Quoted 3 lines
arch/x86/include/asm/cpufeatures.h +0 -0
 #define X86_FEATURE_PAUSEFILTER		(15*32+10) /* "pausefilter" Filtered pause intercept */
 #define X86_FEATURE_PFTHRESHOLD		(15*32+12) /* "pfthreshold" Pause filter threshold */
 #define X86_FEATURE_AVIC		(15*32+13) /* "avic" Virtual Interrupt Controller */
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#4 lore raw

Re: [PATCH 2/8] KVM: x86/pmu: Add PMU ops for vendor state sync

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The newly introduced PMU ops (get_vendor_state, set_vendor_state) are never invoked by KVM nor…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The newly introduced PMU ops (get_vendor_state, set_vendor_state) are never invoked by KVM nor implemented by any vendor PMU, rendering them dead code. --

--- Patch [2]: [PATCH 2/8] KVM: x86/pmu: Add PMU ops for vendor state sync --- commit 2b7ba0292c6d4a2eb66eaca07b256dee5f7af75a
Author: Sandipan Das <[email protected]>

KVM: x86/pmu: Add PMU ops for vendor state sync

This commit introduces new PMU operations get_vendor_state and set_vendor_state to KVM's PMU architecture to allow synchronizing guest PMU state with a vendor save area. It updates the capabilities initialization to nullify these operations if the mediated PMU is not enabled.

arch/x86/kvm/pmu.h struct kvm_pmu_ops { · line 46

Quoted 10 lines
arch/x86/kvm/pmu.h +2 -0
index f60856c48462b..2b1a8d6433734 100644
@@ -42,6 +42,8 @@ struct kvm_pmu_ops {
 	void (*mediated_load)(struct kvm_vcpu *vcpu);
 	void (*mediated_put)(struct kvm_vcpu *vcpu);
 	void (*write_global_ctrl)(u64 global_ctrl);
+	int (*get_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);
+	int (*set_vendor_state)(struct kvm_vcpu *vcpu, u32 msr);

[Severity: Low]
Are these new PMU operations actually invoked anywhere in KVM or implemented by any vendor PMU in this patch? It appears they might be dead code, as there are no callers or vendor implementations (e.g., for AMD or Intel) provided, despite the commit message mentioning opportunistically synchronizing the state.

Quoted 3 lines
arch/x86/kvm/pmu.h +0 -0
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#2 lore raw

Re: [PATCH 1/8] KVM: x86/pmu: Add mediated PMU capability flags

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Transient failure in `perf_create_mediated_pmu` permanently clears `mediated_pmu_caps`, corrupting…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Transient failure in perf_create_mediated_pmu permanently clears mediated_pmu_caps, corrupting VM capability state on retry.
- [Medium] kvm_vcpu_has_mediated_pmu_caps evaluates to true if *any* capability matches, rather than *all*.
--

--- Patch [1]: [PATCH 1/8] KVM: x86/pmu: Add mediated PMU capability flags --- commit b775f5357bd23699130ac68abbdd7cd26849e026
Author: Sandipan Das <[email protected]>

KVM: x86/pmu: Add mediated PMU capability flags

This commit introduces VM-scoped mediated PMU capability flags to help KVM determine available hardware assists for saving and restoring guest PMU state. It defines flags for hardware-switched and hardware-filtered PMU states, and manages these capabilities during VM initialization.

Quoted 4 lines
arch/x86/kvm/pmu.h +0 -0
index a5821d7c87f93..f60856c48462b 100644

[ ... ]

arch/x86/kvm/pmu.h static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu) · line 94

Quoted 8 lines
arch/x86/kvm/pmu.h +4 -0
@@ -88,6 +88,26 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
 	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
 }
 
+static inline bool kvm_vcpu_has_mediated_pmu_caps(struct kvm_vcpu *vcpu, u32 caps)
+{
+	return kvm_vcpu_has_mediated_pmu(vcpu) &&
+	       !!(vcpu->kvm->arch.mediated_pmu_caps & caps);

[Severity: Medium]
When a caller checks for multiple capabilities simultaneously, will this return true if only one of the bits matches, rather than all requested bits?

If a future caller checks kvm_vcpu_has_mediated_pmu_caps() with multiple flags, it appears it would return true even if only a subset of those flags are present in mediated_pmu_caps.

[ ... ]

Quoted 4 lines
arch/x86/kvm/x86.c +0 -0
index afcac1042947a..9f0e072ff466d 100644

[ ... ]

arch/x86/kvm/x86.c int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) · line 12762

Quoted 5 lines
arch/x86/kvm/x86.c +1 -0
@@ -12759,6 +12759,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id)
 		if (irqchip_in_kernel(kvm)) {
 			r = perf_create_mediated_pmu();
 			if (r) {
+				kvm->arch.mediated_pmu_caps = 0;

[Severity: High]
If perf_create_mediated_pmu() returns a transient error like -EBUSY, is it safe to permanently clear the VM-wide mediated_pmu_caps here?

If userspace receives the error, resolves the host perf conflict, and retries the KVM_CREATE_VCPU ioctl, the retry might succeed but the VM's capabilities will have been irreversibly wiped. Does this lead to a silent downgrade to software switching for the VM on a successful retry?

Quoted 3 lines
arch/x86/kvm/x86.c +0 -0
 				pr_warn_ratelimited(PERF_MEDIATED_PMU_MSG);
 				return r;
 			}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#6 lore raw

Re: [PATCH 3/8] KVM: x86/pmu: Add support for hardware-switched PMU

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Hardware-switched PMU state (PMCs, event selectors) is not synchronized with the vendor save area…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Hardware-switched PMU state (PMCs, event selectors) is not synchronized with the vendor save area during MSR access, VM reset, and event emulation, causing data loss across live migration and incorrect event filtering. --

--- Patch [3]: [PATCH 3/8] KVM: x86/pmu: Add support for hardware-switched PMU --- commit 388311da6f37e672745e8b6b16a96e9812fb8ddd
Author: Sandipan Das <[email protected]>

KVM: x86/pmu: Add support for hardware-switched PMU

This patch introduces the hardware-switched mode for the mediated PMU. It transitions the guest PMU state to reside in a vendor save area instead of the software kvm_pmu struct. The load and put functionality is simplified to schedule host events in and out.

Quoted 5 lines
arch/x86/kvm/pmu.c +0 -0
@@ -844,10 +890,12 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	switch (msr) {

arch/x86/kvm/pmu.c int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 900

Quoted 10 lines
arch/x86/kvm/pmu.c +2 -0
 	case MSR_CORE_PERF_GLOBAL_STATUS:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
+		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_STATUS);
 		msr_info->data = pmu->global_status;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 	case MSR_CORE_PERF_GLOBAL_CTRL:
+		kvm_pmu_get_vendor_state(vcpu, kvm_pmu_ops.PERF_GLOBAL_CTRL);
 		msr_info->data = pmu->global_ctrl;
 		break;

[Severity: High]
Does this miss synchronizing the PMCs and Event Selectors in the default switch cases for kvm_pmu_get_msr() and kvm_pmu_set_msr()?

When userspace reads or writes the guest PMU state via KVM_GET_MSRS and KVM_SET_MSRS, the default block falls through to vendor code:

	default:
		return kvm_pmu_call(get_msr)(vcpu, msr_info);

Without calling kvm_pmu_get_vendor_state() or kvm_pmu_set_vendor_state() for these MSRs, KVM might return stale software values or fail to commit restored values to the hardware save area. Since kvm_mediated_pmu_load() skips loading software state into hardware for the hardware-switched mode, could this cause data loss across live migration?

[ ... ]

arch/x86/kvm/pmu.c static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, · line 1212

Quoted 12 lines
arch/x86/kvm/pmu.c +3 -0
@@ -1140,11 +1202,17 @@ static void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		kvm_pmu_get_vendor_state(vcpu, pmc_counter_msr(pmc));
+
 		if (!pmc_is_event_allowed(pmc) || !cpl_is_matched(pmc))
 			continue;
 
 		kvm_pmu_incr_counter(pmc);
+		kvm_pmu_set_vendor_state(vcpu, pmc_counter_msr(pmc));
 	}

[Severity: High]
Are we evaluating the event filter with stale event selector state here?

While we fetch the latest counter values via kvm_pmu_get_vendor_state(), pmc_is_event_allowed() and cpl_is_matched() still rely on pmc->eventsel and pmu->fixed_ctr_ctrl. Without syncing the event selector or fixed counter control from the vendor save area, might these checks use outdated software state?

Additionally, how does KVM handle VM resets and instruction emulation tracking with hardware-switched PMUs?

For resets, kvm_pmu_reset() zeroes out the PMCs and event selectors in the software structures:

	pmc_stop_counter(pmc);
	pmc->counter = 0;
	pmc->emulated_counter = 0;

	if (pmc_is_gp(pmc)) {
		pmc->eventsel = 0;
		pmc->eventsel_hw = 0;

It doesn't seem to push this reset state down to the vendor save area.

For emulation, MSR writes bypass KVM interception in the hardware-switched mode. Since KVM_REQ_PMU is never triggered, it appears that kvm_pmu_recalc_pmc_emulation() won't run, leaving pmc_counting_instructions and pmc_counting_branches empty. Could this cause KVM to silently fail to increment counters during instruction and branch emulation?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#15 lore raw

Re: [PATCH 8/8] KVM: SEV: Disallow the use of hardware-switched PMU

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The patch correctly disables hardware-switched PMU for newly initialized SEV-ES guests to fix a…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The patch correctly disables hardware-switched PMU for newly initialized SEV-ES guests to fix a bug where KVM attempts to interact with PMU state in the unencrypted VMCB save area instead of the guest's encrypted VMSA. However, this fix fails to apply to SEV-ES VMs created via migration or mirroring. --

--- Patch [8]: [PATCH 8/8] KVM: SEV: Disallow the use of hardware-switched PMU --- commit 0e4fd883c1122d7a244f0bb8754669e617c9a34a
Author: Sandipan Das <[email protected]>

KVM: SEV: Disallow the use of hardware-switched PMU

For hardware-switched mediated PMU, the guest PMU state resides in the VMCB save area. When SEV-ES or SEV-SNP is used, KVM cannot access the encrypted VMSA, so this patch clears KVM_MEDIATED_PMU_CAP_HW_SWITCHED for such guests to force software-switched mode.

arch/x86/kvm/svm/sev.c static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp, · line 539

Quoted 11 lines
arch/x86/kvm/svm/sev.c +3 -0
index 944aaea6501f2..c5df6da8d030f 100644
@@ -534,6 +534,9 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
 	if (snp_active)
 		sev->vmsa_features |= SVM_SEV_FEAT_SNP_ACTIVE;
 
+	if (es_active)
+		kvm_clr_mediated_pmu_caps(kvm, KVM_MEDIATED_PMU_CAP_HW_SWITCHED);
+

[Severity: High]
Does this clearing also need to happen when an SEV-ES VM is created via migration or mirroring?

Looking at sev_migrate_from(), it copies the es_active state to the destination VM but does not appear to clear the hardware-switched PMU capability:

arch/x86/kvm/svm/sev.c:sev_migrate_from() {
	...
	dst->es_active = src->es_active;
	dst->vmsa_features = src->vmsa_features;
	...
}

Similarly, in sev_vm_copy_enc_context_from(), the mirror_sev->es_active flag is set to true without clearing the PMU capability:

arch/x86/kvm/svm/sev.c:sev_vm_copy_enc_context_from() {
	...
	mirror_sev->es_active = source_sev->es_active;
	...
}

Since KVM unconditionally sets KVM_MEDIATED_PMU_CAP_HW_SWITCHED during VM creation, could failing to clear it on the destination VMs leave the hardware PMU feature incorrectly enabled?

If a user migrates or mirrors an SEV-ES VM, might KVM attempt to read/write PMU state from the unencrypted VMCB save area instead of the guest's encrypted VMSA, reintroducing the issue this patch aims to fix?

Quoted 3 lines
arch/x86/kvm/svm/sev.c +0 -0
 	ret = sev_asid_new(sev, vm_type);
 	if (ret)
 		goto e_no_asid;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
#13 lore raw

Re: [PATCH 7/8] KVM: nSVM: Add support for hardware-switched PMU

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Critical] L1 hypervisor can bypass KVM's PMU event filter for L2 guests. - [High] KVM's software PMU…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] L1 hypervisor can bypass KVM's PMU event filter for L2 guests.
- [High] KVM's software PMU state is not context-switched between L1 and L2, allowing L2 MSR writes to permanently corrupt L1's PMU state and leak state.
- [Medium] KVM silently breaks PMU functionality for L1 and L2 if L1 enables V_PMC but L0 does not use hardware-switched mediated PMU. --

--- Patch [7]: [PATCH 7/8] KVM: nSVM: Add support for hardware-switched PMU --- commit 93d914b77cd285073e93f03a803ccbca4c41f03f
Author: Sandipan Das <[email protected]>

KVM: nSVM: Add support for hardware-switched PMU

This commit adds nested support for PMC virtualization. It advertises X86_FEATURE_PERFCTR_VIRT to allow an L1 hypervisor to use it for L2 guests, switching PMU state using vmcb02's save area.

Quoted 4 lines
arch/x86/kvm/svm/nested.c +0 -0
index 3e6c671a8dc26..666f8ca38a928 100644

[ ... ]

Quoted 4 lines
arch/x86/kvm/svm/nested.c +0 -0
@@ -741,6 +742,17 @@ static bool nested_vmcb12_has_lbrv(struct kvm_vcpu *vcpu)
 		(to_svm(vcpu)->nested.ctl.misc_ctl2 & SVM_MISC2_ENABLE_V_LBR);
 }
 

arch/x86/kvm/svm/nested.c static bool nested_vmcb12_has_lbrv(struct kvm_vcpu *vcpu) · line 754

Quoted 10 lines
arch/x86/kvm/svm/nested.c +10 -0
+static bool nested_vmcb12_has_vpmc(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Since nested AVIC is not supported, L2 PMIs can only be delivered
+	 * via VNMI, so make it a hard requirement.
+	 */
+	return guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_VIRT) &&
+		nested_vnmi_enabled(to_svm(vcpu)) &&
+		(to_svm(vcpu)->nested.ctl.misc_ctl2 & SVM_MISC2_ENABLE_V_PMC);
+}

[Severity: Medium]
Will this break PMU functionality if L0 does not use hardware-switched mediated PMU?

If nested_vmcb12_has_vpmc() returns true because the guest CPUID supports PERFCTR_VIRT and V_PMC is set in vmcb12, KVM skips the software PMU context switch in enter_svm_guest_mode() and nested_svm_vmexit().

However, if KVM uses the legacy software PMU (KVM_MEDIATED_PMU_CAP_HW_SWITCHED is not set), SVM_MISC2_ENABLE_V_PMC is never set in vmcb02->control.misc_ctl2. Does this result in neither hardware nor software PMU context switching occurring for both L1 and L2?

[ ... ]

arch/x86/kvm/svm/nested.c int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, bool from_vmrun) · line 1090

Quoted 7 lines
arch/x86/kvm/svm/nested.c +3 -0
@@ -1065,6 +1085,9 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa, bool from_vmrun)
 	nested_vmcb02_prepare_control(svm);
 	nested_vmcb02_prepare_save(svm);
 
+	if (!nested_vmcb12_has_vpmc(vcpu))
+		svm_pmu_handle_nested_transition(svm);
+

[Severity: Critical]
By skipping svm_pmu_handle_nested_transition() here, are we omitting the PMU event filter validation for L2?

The PMU configuration is copied directly from L1's cache into vmcb02 earlier in nested_vmcb02_prepare_save(). Without calling the transition handler, kvm_mediated_pmu_refresh_event_filter() is bypassed. Could this allow L1 to count restricted host events by placing forbidden selectors into vmcb12, since hardware will load these unvalidated perf_ctl values on VMRUN?

[ ... ]

arch/x86/kvm/svm/nested.c void nested_svm_vmexit(struct vcpu_svm *svm) · line 1365

Quoted 10 lines
arch/x86/kvm/svm/nested.c +3 -1
@@ -1329,7 +1358,9 @@ void nested_svm_vmexit(struct vcpu_svm *svm)
 
 	/* Exit Guest-Mode */
 	leave_guest_mode(vcpu);
-	svm_pmu_handle_nested_transition(svm);
+
+	if (!nested_vmcb12_has_vpmc(vcpu))
+		svm_pmu_handle_nested_transition(svm);
 
 	svm->nested.vmcb12_gpa = 0;

[Severity: High]
Since svm_pmu_handle_nested_transition() is skipped here, does KVM fail to restore L1's software PMU state on nested exit?

Because EVNTSEL MSRs are deliberately excluded from MSR merging, L2 writes to EVNTSEL MSRs will cause an L0 intercept. This intercept updates KVM's shared software PMU state. If we do not context switch this software state back to L1's values upon exiting L2, could L2 MSR writes permanently corrupt L1's PMU tracking state and leak information?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
#11 lore raw

Re: [PATCH 6/8] KVM: SVM: Add support for hardware-switched PMU

In reply to Sandipan Das
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider: - [High] Emulated guest reads and writes to PMU MSRs return stale data or are lost because the vendor state…

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] Emulated guest reads and writes to PMU MSRs return stale data or are lost because the vendor state sync is skipped for guest-initiated accesses.
- [High] PMC virtualization is never enabled for the BSP (Boot Strap Processor), leaving its PMU completely broken.
- [High] Nested virtualization (L2) PMU is broken because SVM_MISC2_ENABLE_V_PMC is never propagated to vmcb02.
- [High] Calling wrmsrq() inside the noinstr function svm_vcpu_enter_exit() invokes instrumentable tracepoint code while RCU is not watching.
- [High] The noinstr function svm_vcpu_enter_exit() calls non-__always_inline PMU capability helpers, risking instrumentation in the VMRUN path. --

--- Patch [6]: [PATCH 6/8] KVM: SVM: Add support for hardware-switched PMU --- commit d7b31e3949e70dae0c322517e2056c7007227bb4
Author: Sandipan Das <[email protected]>
Subject: KVM: SVM: Add support for hardware-switched PMU

This patch enables PMC virtualization on Zen 5 processors to offload guest PMU context switches to hardware. It introduces vendor state sync operations and a module parameter to toggle the feature, integrating it with the VMCB save area.

Quoted 4 lines
arch/x86/kvm/svm/pmu.c +0 -0
index 5dccf87763686..7d3a151b6cdee 100644

[ ... ]

arch/x86/kvm/svm/pmu.c static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 226

Quoted 11 lines
arch/x86/kvm/svm/pmu.c +4 -0
@@ -134,6 +217,10 @@ static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	/* MSR_PERFCTRn */
 	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
 	if (pmc) {
+		if (msr_info->host_initiated &&
+		    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
+		    amd_get_vendor_state(vcpu, msr))
+			kvm_pmu_warn_vendor_state(msr);
 		msr_info->data = pmc_read_counter(pmc);
 		return 0;
 	}

[Severity: High]
Does gating the vendor state sync behind msr_info->host_initiated break guest-initiated emulated reads?

When KVM emulates an instruction (e.g., forced emulation or MMIO) that accesses a PMU counter, host_initiated is false. For reads, this skips the vendor state sync and returns the stale value in pmc->counter.

arch/x86/kvm/svm/pmu.c static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 253

Quoted 10 lines
arch/x86/kvm/svm/pmu.c +4 -0
@@ -158,6 +245,10 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_COUNTER);
 	if (pmc) {
 		pmc_write_counter(pmc, data);
+		if (msr_info->host_initiated &&
+		    kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED) &&
+		    amd_set_vendor_state(vcpu, msr))
+			kvm_pmu_warn_vendor_state(msr);
 		return 0;
 	}

[Severity: High]
Does gating the vendor state sync behind msr_info->host_initiated break guest-initiated emulated writes?

For emulated writes, KVM will update pmc->counter but skip updating the VMCB save area because host_initiated is false. Since pmc_write_counter() does not trigger a PMU refresh for mediated PMUs, the VMCB save area is never updated. Hardware natively reloads the stale PMU state from the VMCB on the next VMRUN, silently discarding the guest's emulated write.

Quoted 4 lines
arch/x86/kvm/svm/svm.c +0 -0
index ac3d898d21bda..2fa5d85d3c40e 100644

[ ... ]

arch/x86/kvm/svm/svm.c static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event) · line 1280

Quoted 8 lines
arch/x86/kvm/svm/svm.c +3 -0
@@ -1271,6 +1274,9 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
 
 	svm_hv_init_vmcb(vmcb);
 
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		control->misc_ctl2 |= SVM_MISC2_ENABLE_V_PMC;
+
 	kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu);

[Severity: High]
Will PMC virtualization remain disabled for the Boot Strap Processor (BSP)?

During early VCPU creation (svm_vcpu_reset), init_vmcb() is called before userspace sets the CPUID via KVM_SET_CPUID2. At this point, the PMU version is 0, so kvm_vcpu_has_mediated_pmu_caps() returns false and SVM_MISC2_ENABLE_V_PMC is not set.

Unlike APs (which receive an INIT signal that triggers a subsequent init_vmcb() after CPUID is set), the BSP does not undergo another reset. During runtime, KVM skips manual PMU context switching because the capability evaluates to true, but hardware also skips it due to the missing V_PMC bit in the BSP's VMCB, effectively breaking the PMU for the BSP.

[Severity: High]
Is nested virtualization (L2) PMU broken because SVM_MISC2_ENABLE_V_PMC is never propagated to vmcb02?

When KVM prepares vmcb02 to run an L2 guest in nested_vmcb02_prepare_control(), it initializes misc_ctl2 but completely omits setting SVM_MISC2_ENABLE_V_PMC. KVM will skip manual loading of the PMU state because kvm_vcpu_has_mediated_pmu_caps() returns true when L2 is running, but hardware will also skip it because vmcb02 lacks the V_PMC enable bit.

[ ... ]

Quoted 4 lines
arch/x86/kvm/svm/svm.c +0 -0
@@ -4456,6 +4486,14 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, unsigned enter_fl
 
 	amd_clear_divider();
 

arch/x86/kvm/svm/svm.c static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu, unsigned enter_fl · line 4498

Quoted 10 lines
arch/x86/kvm/svm/svm.c +8 -0
+	/*
+	 * On #VMEXIT, PerfCntrGlobalCtl goes back to its reset state since
+	 * its save slot is of Swap Type C. All enable bits are set but PMC
+	 * virtualization requires them to be cleared before VMRUN.
+	 */
+	if (kvm_vcpu_has_mediated_pmu_caps(vcpu, KVM_MEDIATED_PMU_CAP_HW_SWITCHED))
+		wrmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
+
 	if (is_sev_es_guest(vcpu))
 		__svm_sev_es_vcpu_run(svm, enter_flags,

[Severity: High]
Does calling kvm_vcpu_has_mediated_pmu_caps() from a noinstr function risk instrumentation in the VMRUN path?

Functions executed in a noinstr context must be explicitly marked noinstr or __always_inline. kvm_vcpu_has_mediated_pmu_caps() and its dependency kvm_vcpu_has_mediated_pmu() are only declared as static inline. If the compiler compiles them out-of-line, the noinstr protections are stripped, which can lead to objtool warnings and potential kernel panics if tracing or instrumentation triggers during this VMRUN entry path.

[Severity: High]
Can calling wrmsrq() inside the noinstr function svm_vcpu_enter_exit() invoke instrumentable tracepoint code while RCU is not watching?

The wrmsrq() macro wraps native_write_msr(), which conditionally calls do_trace_write_msr() if the write_msr tracepoint is enabled. Tracepoints utilize RCU to safely iterate their probe lists. Executing this inside a noinstr section where RCU is not watching causes lockdep/RCU splats (RCU used illegally from extended quiescent state) and potential system panics when tracing is active.

Should this use a non-instrumented wrapper such as native_wrmsrl() or __wrmsr()?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
#16 lore raw
In reply to Sandipan Das
Quoted 75 lines · Sandipan Das

Mediated PMU currently saves and restores the guest PMU state in software on every world switch. On Zen 5 and later processors, it is possible to offload this to the hardware, which automatically saves and restores the guest counter state to and from a vendor save area (VMCB) during world switches. This feature is called PMC virtualization and detailed information about it can be found in Volume 2 of the AMD64 Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched and hardware-switched modes is measured with the x86/vmexit test from kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test sw-switched hw-switched gain
------------------------------------------------------------------------

cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%
Quoted 1 line

------------------------------------------------------------------------

Just curious what causes the gain difference between these test cases? Different vm-exit/vm-entry number in each test case? Suppose the cost for saving/restoring PMU MSRs on each vm-exit/vm-entry should be same or close.

Quoted 96 lines

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid (+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the kernel-handled port I/O paths (~+50%), and the control-register toggles (~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by ~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all within run-to-run noise.

Design
------
* The entity responsible for switching the guest PMU state is tracked per-VM via a new set of mediated PMU capability flags in kvm_arch. In software-switched mode the state resides in struct kvm_pmu; in hardware-switched mode it resides in the vendor save area.
* Event filtering, instruction emulation, and intercepted RDPMC still need the state in struct kvm_pmu, so new vendor state-sync PMU ops synchronize specific MSRs between struct kvm_pmu and the vendor save area on demand.
* On SVM, the feature depends on VNMI for guest PMI delivery and is controlled by the new "vpmc" kvm_amd module parameter, enabled by default when the host supports it and mediated PMU is enabled.

Limitations
-----------
* Not supported for SEV-ES and SEV-SNP guests: the guest PMU state resides in an encrypted VMSA that is inaccessible to the event filtering logic.
* With VNMI and no AVIC, overflow interrupts are always delivered as NMIs because the APIC is emulated and the LVTPC is ignored. This shows up as expected behavioral differences in the x86/pmu test of kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature bit, the VMCB save-area fields, and the SVM implementation. Patch 7 adds nested (nSVM) support to use the feature with L2 guests. Patch 8 explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test requires patching to accept overflow interrupts as NMIs (limitation because of VNMI ignoring the LVTPC). It is recommended to apply the following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d

             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/[email protected]/

Changes in v1:
* Introduce mediated PMU capabilities for hardware PMU virtualization features instead of perf PMU capabilities.
* Make VNMI a hard requirement since AVIC can be inhibited.
* Introduce new PMU ops to access VMCB PMU state instead of relying on host-initiated {get,set}_msr() ops.
* Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
* Implement support for nested SVM.
* Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
KVM: x86/pmu: Add mediated PMU capability flags
KVM: x86/pmu: Add PMU ops for vendor state sync
KVM: x86/pmu: Add support for hardware-switched PMU x86/cpufeatures: Add PerfCtrVirt feature bit
KVM: SVM: Add VMCB fields for PMC virtualization
KVM: SVM: Add support for hardware-switched PMU
KVM: nSVM: Add support for hardware-switched PMU
KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)
#17 lore raw
Quoted 79 lines · Dapeng
Quoted 75 lines · Sandipan Das

Mediated PMU currently saves and restores the guest PMU state in software on every world switch. On Zen 5 and later processors, it is possible to offload this to the hardware, which automatically saves and restores the guest counter state to and from a vendor save area (VMCB) during world switches. This feature is called PMC virtualization and detailed information about it can be found in Volume 2 of the AMD64 Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched and hardware-switched modes is measured with the x86/vmexit test from kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test sw-switched hw-switched gain
------------------------------------------------------------------------

cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%
Quoted 1 line

------------------------------------------------------------------------

Quoted 4 lines

Just curious what causes the gain difference between these test cases? Different vm-exit/vm-entry number in each test case? Suppose the cost for saving/restoring PMU MSRs on each vm-exit/vm-entry should be same or close.

In this case, the cost is actually lower when hardware handles save-restore of PMU MSRs. Hardware can use fastpaths and optimizations which an usual RDMSR/WRMSR sequence cannot.

Quoted 97 lines
Quoted 97 lines

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid (+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the kernel-handled port I/O paths (~+50%), and the control-register toggles (~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by ~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all within run-to-run noise.

Design
------
* The entity responsible for switching the guest PMU state is tracked per-VM via a new set of mediated PMU capability flags in kvm_arch. In software-switched mode the state resides in struct kvm_pmu; in hardware-switched mode it resides in the vendor save area.
* Event filtering, instruction emulation, and intercepted RDPMC still need the state in struct kvm_pmu, so new vendor state-sync PMU ops synchronize specific MSRs between struct kvm_pmu and the vendor save area on demand.
* On SVM, the feature depends on VNMI for guest PMI delivery and is controlled by the new "vpmc" kvm_amd module parameter, enabled by default when the host supports it and mediated PMU is enabled.

Limitations
-----------
* Not supported for SEV-ES and SEV-SNP guests: the guest PMU state resides in an encrypted VMSA that is inaccessible to the event filtering logic.
* With VNMI and no AVIC, overflow interrupts are always delivered as NMIs because the APIC is emulated and the LVTPC is ignored. This shows up as expected behavioral differences in the x86/pmu test of kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature bit, the VMCB save-area fields, and the SVM implementation. Patch 7 adds nested (nSVM) support to use the feature with L2 guests. Patch 8 explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test requires patching to accept overflow interrupts as NMIs (limitation because of VNMI ignoring the LVTPC). It is recommended to apply the following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d

             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/[email protected]/

Changes in v1:
* Introduce mediated PMU capabilities for hardware PMU virtualization features instead of perf PMU capabilities.
* Make VNMI a hard requirement since AVIC can be inhibited.
* Introduce new PMU ops to access VMCB PMU state instead of relying on host-initiated {get,set}_msr() ops.
* Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
* Implement support for nested SVM.
* Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
KVM: x86/pmu: Add mediated PMU capability flags
KVM: x86/pmu: Add PMU ops for vendor state sync
KVM: x86/pmu: Add support for hardware-switched PMU x86/cpufeatures: Add PerfCtrVirt feature bit
KVM: SVM: Add VMCB fields for PMC virtualization
KVM: SVM: Add support for hardware-switched PMU
KVM: nSVM: Add support for hardware-switched PMU
KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)
#18 lore raw
Quoted 78 lines · Sandipan Das
Quoted 77 lines · Dapeng
Quoted 76 lines · Sandipan Das

Mediated PMU currently saves and restores the guest PMU state in software on every world switch. On Zen 5 and later processors, it is possible to offload this to the hardware, which automatically saves and restores the guest counter state to and from a vendor save area (VMCB) during world switches. This feature is called PMC virtualization and detailed information about it can be found in Volume 2 of the AMD64 Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched and hardware-switched modes is measured with the x86/vmexit test from kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test sw-switched hw-switched gain
------------------------------------------------------------------------

cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%

------------------------------------------------------------------------

Quoted 7 lines
Quoted 4 lines

Just curious what causes the gain difference between these test cases? Different vm-exit/vm-entry number in each test case? Suppose the cost for saving/restoring PMU MSRs on each vm-exit/vm-entry should be same or close.

In this case, the cost is actually lower when hardware handles save-restore of PMU MSRs. Hardware can use fastpaths and optimizations which an usual RDMSR/WRMSR sequence cannot.

Thanks. My question is why the different test case show so much difference on the gains, e.g., cpuid has +51%, while invd has -15%. Per my understanding, the cost of saving/restoring PMU MSRs for most vm-exits (exclude the fastpath vm-exit) should be fixed even for different vm-exit reasons. Is it true?

Quoted 96 lines
Quoted 96 lines
Quoted 96 lines

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid (+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the kernel-handled port I/O paths (~+50%), and the control-register toggles (~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by ~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all within run-to-run noise.

Design
------
* The entity responsible for switching the guest PMU state is tracked per-VM via a new set of mediated PMU capability flags in kvm_arch. In software-switched mode the state resides in struct kvm_pmu; in hardware-switched mode it resides in the vendor save area.
* Event filtering, instruction emulation, and intercepted RDPMC still need the state in struct kvm_pmu, so new vendor state-sync PMU ops synchronize specific MSRs between struct kvm_pmu and the vendor save area on demand.
* On SVM, the feature depends on VNMI for guest PMI delivery and is controlled by the new "vpmc" kvm_amd module parameter, enabled by default when the host supports it and mediated PMU is enabled.

Limitations
-----------
* Not supported for SEV-ES and SEV-SNP guests: the guest PMU state resides in an encrypted VMSA that is inaccessible to the event filtering logic.
* With VNMI and no AVIC, overflow interrupts are always delivered as NMIs because the APIC is emulated and the LVTPC is ignored. This shows up as expected behavioral differences in the x86/pmu test of kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature bit, the VMCB save-area fields, and the SVM implementation. Patch 7 adds nested (nSVM) support to use the feature with L2 guests. Patch 8 explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test requires patching to accept overflow interrupts as NMIs (limitation because of VNMI ignoring the LVTPC). It is recommended to apply the following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d

             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/[email protected]/

Changes in v1:
* Introduce mediated PMU capabilities for hardware PMU virtualization features instead of perf PMU capabilities.
* Make VNMI a hard requirement since AVIC can be inhibited.
* Introduce new PMU ops to access VMCB PMU state instead of relying on host-initiated {get,set}_msr() ops.
* Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
* Implement support for nested SVM.
* Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
KVM: x86/pmu: Add mediated PMU capability flags
KVM: x86/pmu: Add PMU ops for vendor state sync
KVM: x86/pmu: Add support for hardware-switched PMU x86/cpufeatures: Add PerfCtrVirt feature bit
KVM: SVM: Add VMCB fields for PMC virtualization
KVM: SVM: Add support for hardware-switched PMU
KVM: nSVM: Add support for hardware-switched PMU
KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)
#19 lore raw
Quoted 88 lines · Dapeng
Quoted 78 lines · Sandipan Das
Quoted 77 lines · Dapeng
Quoted 76 lines · Sandipan Das

Mediated PMU currently saves and restores the guest PMU state in software on every world switch. On Zen 5 and later processors, it is possible to offload this to the hardware, which automatically saves and restores the guest counter state to and from a vendor save area (VMCB) during world switches. This feature is called PMC virtualization and detailed information about it can be found in Volume 2 of the AMD64 Architecture Programmer's Manual under Section 15.39.

This series extends the mediated PMU framework with a hardware-switched mode and adds the AMD (SVM) implementation, including nested support.

Performance
-----------
The difference in the world switch overhead between software-switched and hardware-switched modes is measured with the x86/vmexit test from kvm-unit-tests on a system with an AMD EPYC 9755 processor. The values are in cycles per operation (lower is better), averaged over 5 runs.

------------------------------------------------------------------------
test sw-switched hw-switched gain
------------------------------------------------------------------------

cpuid                                     5333         2592       +51.4%
vmcall                                   15858         7619       +52.0%
wbinvd                                    5256         2517       +52.1%
invd                                      1857         2147       -15.6%
mov_from_cr8                                18           18         0.0%
mov_to_cr8                                 157          156        +0.6%
inl_from_pmtimer                         25934        21176       +18.3%
inl_from_qemu                            25881        21156       +18.3%
inl_from_kernel                           5521         2772       +49.8%
outl_to_kernel                            5522         2778       +49.7%
mov_dr                                     114          114         0.0%
self_ipi_sti_nop                          1020         1015        +0.5%
  ipi self_ipi_sti_nop                     310          307        +1.0%
  eoi self_ipi_sti_nop                     242          241        +0.4%
self_ipi_sti_hlt                          2714         2704        +0.4%
  ipi self_ipi_sti_hlt                     310          307        +1.0%
  eoi self_ipi_sti_hlt                     242          240        +0.8%
self_ipi_tpr                              1272         1263        +0.7%
  ipi self_ipi_tpr                         309          307        +0.6%
  eoi self_ipi_tpr                         241          240        +0.4%
self_ipi_tpr_sti_nop                      1304         1295        +0.7%
  ipi self_ipi_tpr_sti_nop                 309          307        +0.6%
  eoi self_ipi_tpr_sti_nop                 242          241        +0.4%
self_ipi_tpr_sti_hlt                      3003         2991        +0.4%
  ipi self_ipi_tpr_sti_hlt                 309          307        +0.6%
  eoi self_ipi_tpr_sti_hlt                 242          240        +0.8%
x2apic_self_ipi_sti_nop                    899          895        +0.4%
  ipi x2apic_self_ipi_sti_nop              191          190        +0.5%
  eoi x2apic_self_ipi_sti_nop              242          241        +0.4%
x2apic_self_ipi_sti_hlt                   2592         2587        +0.2%
  ipi x2apic_self_ipi_sti_hlt              191          190        +0.5%
  eoi x2apic_self_ipi_sti_hlt              242          241        +0.4%
x2apic_self_ipi_tpr                       1159         1151        +0.7%
  ipi x2apic_self_ipi_tpr                  193          193         0.0%
  eoi x2apic_self_ipi_tpr                  241          240        +0.4%
x2apic_self_ipi_tpr_sti_nop               1189         1180        +0.8%
  ipi x2apic_self_ipi_tpr_sti_nop          193          193         0.0%
  eoi x2apic_self_ipi_tpr_sti_nop          242          241        +0.4%
x2apic_self_ipi_tpr_sti_hlt               2887         2875        +0.4%
  ipi x2apic_self_ipi_tpr_sti_hlt          194          192        +1.0%
  eoi x2apic_self_ipi_tpr_sti_hlt          242          239        +1.2%
ple_round_robin                             19           19         0.0%
wr_kernel_gs_base                           73           73         0.0%
wr_ibpb_msr                                563          557        +1.1%
wr_tsc_adjust_msr                         5352         2603       +51.4%
rd_tsc_adjust_msr                         5330         2586       +51.5%
toggle_cr0_wp                            11319         5810       +48.7%
toggle_cr4_pge                           11743         6242       +46.8%
mmio-no-eventfd:pci-mem                  31890        26835       +15.9%
mmio-wildcard-eventfd:pci-mem             6461         3666       +43.3%
mmio-datamatch-eventfd:pci-mem            6449         3671       +43.1%
portio-no-eventfd:pci-io                 25218        21455       +14.9%
portio-wildcard-eventfd:pci-io            5512         2774       +49.7%
portio-datamatch-eventfd:pci-io           5524         2780       +49.7%

------------------------------------------------------------------------

Quoted 7 lines
Quoted 4 lines

Just curious what causes the gain difference between these test cases? Different vm-exit/vm-entry number in each test case? Suppose the cost for saving/restoring PMU MSRs on each vm-exit/vm-entry should be same or close.

In this case, the cost is actually lower when hardware handles save-restore of PMU MSRs. Hardware can use fastpaths and optimizations which an usual RDMSR/WRMSR sequence cannot.

Quoted 6 lines

Thanks. My question is why the different test case show so much difference on the gains, e.g., cpuid has +51%, while invd has -15%. Per my understanding, the cost of saving/restoring PMU MSRs for most vm-exits (exclude the fastpath vm-exit) should be fixed even for different vm-exit reasons. Is it true?

I see. Yes, the absolute savings for the PMU context switch is roughly the same across all the test cases. I will clarify this in the next version.

Quoted 97 lines
Quoted 97 lines
Quoted 97 lines
Quoted 97 lines

The largest gains are for wbinvd (+52.1%), vmcall (+52.0%), and cpuid (+51.4%), followed by the TSC-adjust MSR accesses (~+51%), the kernel-handled port I/O paths (~+50%), and the control-register toggles (~+47-49%). The wildcard and datamatch MMIO / port-I/O paths improve by ~+43-50%, while the slower no-eventfd MMIO / port-I/O and pmtimer/qemu port-I/O paths improve by ~+15-18%.

The only meaningful regression is invd (-15.6%). The self-IPI and EOI subtests move by at most 1.2% and wr_kernel_gs_base is unchanged, all within run-to-run noise.

Design
------
* The entity responsible for switching the guest PMU state is tracked per-VM via a new set of mediated PMU capability flags in kvm_arch. In software-switched mode the state resides in struct kvm_pmu; in hardware-switched mode it resides in the vendor save area.
* Event filtering, instruction emulation, and intercepted RDPMC still need the state in struct kvm_pmu, so new vendor state-sync PMU ops synchronize specific MSRs between struct kvm_pmu and the vendor save area on demand.
* On SVM, the feature depends on VNMI for guest PMI delivery and is controlled by the new "vpmc" kvm_amd module parameter, enabled by default when the host supports it and mediated PMU is enabled.

Limitations
-----------
* Not supported for SEV-ES and SEV-SNP guests: the guest PMU state resides in an encrypted VMSA that is inaccessible to the event filtering logic.
* With VNMI and no AVIC, overflow interrupts are always delivered as NMIs because the APIC is emulated and the LVTPC is ignored. This shows up as expected behavioral differences in the x86/pmu test of kvm-unit-tests.

Overview
--------
Patches 1-3 extend the mediated PMU framework with hardware-switched
mode: the capability flags, the vendor state-sync ops, and the generic support. Patches 4-6 add the SVM enablement: the PerfCtrVirt feature bit, the VMCB save-area fields, and the SVM implementation. Patch 7 adds nested (nSVM) support to use the feature with L2 guests. Patch 8 explicitly disables the feature for SEV-ES and SEV-SNP guests.

Tested with kvm-unit-tests (KUT) and perf fuzzer. KUT's x86/pmu test requires patching to accept overflow interrupts as NMIs (limitation because of VNMI ignoring the LVTPC). It is recommended to apply the following patches before testing.
https://lore.kernel.org/kvm/3bfbf15c22652ba00cf4a16fe9e0a3bfe7071f97.1784096302.git.sandipan.das@amd.com/
https://lore.kernel.org/kvm/b075ec13e95167da1f2229626713c91d291bb75b.1784097178.git.sandipan.das@amd.com/
https://lore.kernel.org/all/106bc3a4660653cb7750f3ae07a4dbca42c2762c.1786356229.git.sandipan.das@amd.com/

base-commit: 6cf46b37b0356b3408f8ba949e10462cc71b533d

             (tag: kvm-x86-next-2026.08.07)

Previous versions can be found at:
rfc: https://lore.kernel.org/kvm/[email protected]/

Changes in v1:
* Introduce mediated PMU capabilities for hardware PMU virtualization features instead of perf PMU capabilities.
* Make VNMI a hard requirement since AVIC can be inhibited.
* Introduce new PMU ops to access VMCB PMU state instead of relying on host-initiated {get,set}_msr() ops.
* Make VMCB PMU state accessible via KVM_{GET,SET}_MSRS for migration.
* Implement support for nested SVM.
* Disable support for SEV-ES and SEV-SNP guests explicitly.

Sandipan Das (8):
KVM: x86/pmu: Add mediated PMU capability flags
KVM: x86/pmu: Add PMU ops for vendor state sync
KVM: x86/pmu: Add support for hardware-switched PMU x86/cpufeatures: Add PerfCtrVirt feature bit
KVM: SVM: Add VMCB fields for PMC virtualization
KVM: SVM: Add support for hardware-switched PMU
KVM: nSVM: Add support for hardware-switched PMU
KVM: SEV: Disallow the use of hardware-switched PMU

 arch/x86/include/asm/cpufeatures.h     |   1 +
 arch/x86/include/asm/kvm-x86-pmu-ops.h |   3 +
 arch/x86/include/asm/kvm_host.h        |  11 +++
 arch/x86/include/asm/svm.h             |  13 ++-
 arch/x86/kvm/cpuid.c                   |   1 +
 arch/x86/kvm/pmu.c                     | 102 +++++++++++++++++----
 arch/x86/kvm/pmu.h                     |  29 ++++++
 arch/x86/kvm/svm/nested.c              |  45 +++++++++-
 arch/x86/kvm/svm/pmu.c                 | 117 +++++++++++++++++++++++++
 arch/x86/kvm/svm/sev.c                 |   3 +
 arch/x86/kvm/svm/svm.c                 |  58 +++++++++++-
 arch/x86/kvm/svm/svm.h                 |  19 ++++
 arch/x86/kvm/vmx/pmu_intel.c           |   2 +
 arch/x86/kvm/vmx/vmx.c                 |   3 +-
 arch/x86/kvm/x86.c                     |   2 +
 15 files changed, 384 insertions(+), 25 deletions(-)