Skip to contentSkip to conversation

KVM: x86/pmu: Add mediated vPMU PerfMon v5 support

PATCH In review Zide Chen 4 participants 29 messages Nov 12 – Nov 16

Subsystems (6)
Documentation x86 KVM Kselftest +2 more subsystems
lore
Share & tools
lore

Zide Chen

author
#0 lore raw

v4 cover KVM: x86/pmu: Add mediated vPMU PerfMon v5 support

KVM currently caps the guest Intel architectural PMU version at 2 and relies on counter-count fields that cannot describe sparse or non-contiguous fixed counters.

Rework KVM's vPMU to track counters with bitmasks instead of counts, so mediated-PMU guests can use Intel PerfMon v5 and non-contiguous fixed counters, and lay the groundwork for non-contiguous GP counters (CPUID leaf 0x23). This series also adds support for new features introduced in PerfMon v3-v5.

- CPUID.0AH:ECX fixed-counter bitmap (v5): supersedes EDX[4:0] and lets KVM/guests represent non-contiguous fixed counters (e.g. some E-core server parts skip fixed counter 3).
- Support IA32_PERF_GLOBAL_STATUS_RESET (v4): inherits MSR 0x390 and its emulation from IA32_PERF_GLOBAL_OVF_CTRL; mark CondChgd (bit 63) reserved in IA32_PERF_GLOBAL_STATUS while allowing it to be cleared on STATUS_RESET, and document its mediated-PMU passthrough erratum in errata.rst.
- Add support for IA32_PERF_GLOBAL_STATUS_SET (v4): lets software set individual bits in the global status MSR.
- Add support for IA32_PERF_GLOBAL_INUSE (v4): reports which counters and PMI are currently claimed.
- Implement ANYTHREAD_DEPRECATION capability (v5).
- Clamp guest PMU version to KVM's host capabilities (pmu->version):
ensures KVM's internal state tracks the effective PMU version that KVM can actually emulate, matching how bit_width, mask_length, and nr_{gp,fixed}_counters are clamped.

Intel PerfMon v5 doesn't support non-contiguous GP counters; however, this series lays the groundwork for GP counter bitmap support anyway, for consistency with the host perf API interface, and prepares for PerfMon Masking support.

Freeze_PerfMon_On_PMI virtualization is not implemented because, as commit 3daa96d67274 ("perf/intel: Remove Perfmon-v4 counter_freezing support") states, the Freeze-on-PMI mechanism violates the perf counter independence.

Streamlined Freeze_LBRs_On_PMI is not implemented because LBR is not yet supported on mediated vPMU.

Mutual exclusion between Intel PT and mediated vPMU, to prevent host PT PMIs from asserting IA32_PERF_GLOBAL_STATUS.Trace_ToPA_PMI (bit 55) while guest PMU context is loaded, as analyzed by Yosry Ahmed, is left out of this series and will be addressed in a separate follow-up patch.

Tests are described in v2 cover letter.

Patch series summary:
patch 1-4: Code cleanups and bug fixes.
patch 5-10: Implement PMC bitmap support across perf and KVM. patch 11-19: Implement PerfMon v4-v5 new features, including PMU
version clamping.
patch 20: Advertise PerfMon version up to v5 on Intel hosts. patch 21: Support fixed counters bitmap in pmu_counters_test selftest.

Rebased on kvm-x86/next: commit cbe54fc3462b("Merge branch 'vmx'")

v4:
- New patches:
- Patch 1: Drop redundant counter MSR checks in amd_is_valid_msr().

     (Jim)

- Patch 12: Handle CondChgd (bit 63) asymmetry: mark bit 63 reserved in global_status_rsvd to reject writes to GLOBAL_STATUS{,_SET}, while allowing writes to STATUS_RESET. (Jim, Yosry)
- Patch 15: Clamp pmu->version to kvm_pmu_cap.version. (Sean, Yosry)
- Collected Reviewed-by tags, but did not carry Reviewed-by on patches 5, 9, and 13 due to code changes.
- All other changes are non-functional, including code improvements, coding style cleanups, and patch rearrangements.

v3:
- https://lore.kernel.org/kvm/[email protected]/
- Dropped v2 patch 4 ("Gate BUFFER_OVF reserved bit on guest DS"). (Jim)
- New patch 10 to document the IA32_PERF_GLOBAL_STATUS_RESET passthrough erratum in errata.rst.
- New patch 15 to mask only OS/USR bits when filtering fixed counters, preventing AnyThread leakage into hardware and preserving METRICS_CLEAR_EN.
- Split MSR 0x391 and MSR 0x392 emulation into separate patches.
- Always intercept IA32_PERF_GLOBAL_STATUS_SET and GLOBAL_INUSE to prevent hardware state divergence. (Jim)
- Reject bit 63 (CondChgd) writes to GLOBAL_STATUS_SET. (Jim)
- Keep the AnyThread bit guest-visible while removing it from the hardware-visible states. (Jim)
- Address other minor review comments from LKML, also some fixes suggested by AI tools.
- Collected Reviewed-by and Acked-by tags.

v2:
- https://lore.kernel.org/kvm/[email protected]/
- Dropped v1 11/15: KVM: x86/pmu: Emulate streamlined Freeze-LBR-on-PMI.
- New patch 4/15 to gate BUFFER_OVF reserved bit on guest DS feature.
- Some renames:
- all_valid_pmc_mask to pmc_exists;
- add _is to kvm_{gp,fixed}_pmc_supported();
- split kvm_for_each_set_pmc_idx() to

     kvm_for_each_{gp,fixed}_counter().

- Add missing AnyThr handling in IA32_FIXED_CTR_CTRL and advertise anythread_deprecated only if kvm_pmu_cap.version >= 5.
- Cap vPMU v3/v4 to v2, as AnyThread cannot be safely virtualized.
- Miscellaneous updates to changelogs and some minor fixes.

v1: https://lore.kernel.org/kvm/[email protected]/T/#u

Dapeng Mi (6):
KVM: x86/pmu: Drop nr_arch_{gp,fixed}_counters from kvm_pmu perf/x86: Plumb counter bitmap from x86_pmu to x86_pmu_cap
KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities perf/x86: Remove num_counters_{gp,fixed} from x86_pmu_capability
KVM: x86/pmu: Advertise PerfMon version 5 on Intel hosts
KVM: selftests: Support fixed counters bitmap in pmu_counters_test

Zide Chen (15):
KVM: SVM: Drop counter MSR checks in amd_is_valid_msr()
KVM: x86/pmu: Remove redundant Perf Global Status MSR bit definitions
KVM: x86/pmu: Rename all_valid_pmc_idx to pmc_exists
KVM: x86/pmu: Rename reserved_bits to eventsel_rsvd in kvm_pmu
KVM: x86/pmu: Use pmc_exists in PMC lookup helpers
KVM: x86/pmu: Expose kvm_host_pmu to vendor modules
KVM: x86/pmu: Document GLOBAL_STATUS_RESET passthrough erratum
KVM: x86/pmu: Mark CondChgd reserved in global_status_rsvd
KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_STATUS_SET MSR
KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_INUSE MSR
KVM: x86/pmu: Clamp pmu->version to kvm_pmu_cap.version
KVM: x86/pmu: Populate CPUID.0AH:ECX fixed-counter bitmap
KVM: x86/pmu: Factor out fixed counter control bit calculation
KVM: x86/pmu: Mask only OS/USR bits when filtering fixed counters
KVM: x86/pmu: Ignore AnyThread bit if CPUID.0AH:EDX[15] is set

 Documentation/virt/kvm/x86/errata.rst         |  10 +-
 arch/x86/events/core.c                        |   8 +-
 arch/x86/include/asm/kvm_host.h               |   9 +-
 arch/x86/include/asm/msr-index.h              |   9 +-
 arch/x86/include/asm/perf_event.h             |  13 +-
 arch/x86/kvm/cpuid.c                          |  24 ++-
 arch/x86/kvm/msrs.c                           |  12 +-
 arch/x86/kvm/pmu.c                            |  85 +++++---
 arch/x86/kvm/pmu.h                            |  54 ++++-
 arch/x86/kvm/svm/pmu.c                        |  43 ++--
 arch/x86/kvm/svm/svm.c                        |  19 +-
 arch/x86/kvm/vmx/nested.c                     |   4 +-
 arch/x86/kvm/vmx/pmu_intel.c                  | 192 +++++++++++++-----
 arch/x86/kvm/vmx/vmx.c                        |  28 ++-
 .../selftests/kvm/include/x86/processor.h     |   6 +-
 .../selftests/kvm/x86/pmu_counters_test.c     |  29 ++-
 16 files changed, 366 insertions(+), 179 deletions(-)
Signature · 1 line
2.56.1
author
#1 lore raw

PATCH v4 1/21 KVM: SVM: Drop counter MSR checks in amd_is_valid_msr()

amd_msr_idx_to_pmc() accurately validates individual counter MSRs by checking whether the corresponding counter is supported for the vCPU. Validating counter MSRs again in amd_is_valid_msr() is redundant.

Furthermore, validating counter MSRs using broad checks like "pmu->version > 0" or "PERFCTR_CORE" is inaccurate and misleading. It falsely validates unexposed counters, which then fail in amd_pmu_{get,set}_msr(), injecting an errant #GP instead of taking KVM's fallback (read 0, write ignored) and breaking migration for unadvertised counters.

Finally, the range check in the default case is dead code because AMD CPUs currently support at most 6 Core Perf counters. Even if future AMD CPUs support more than 6, counter MSRs would still be sufficiently validated by amd_msr_idx_to_pmc().

Drop the counter cases and dead range check from amd_is_valid_msr(), leaving counter validation solely to amd_msr_idx_to_pmc().

Fixes 4a2771895ca6 ("KVM: x86/svm/pmu: Add AMD PerfMonV2 support")

---
v4:
- New patch. (Jim)

---
 arch/x86/kvm/svm/pmu.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
arch/x86/kvm/svm/pmu.c +1 -8
index c18286545a7a..bf2f23c30afc 100644
@@ -106,23 +106,16 @@ static bool amd_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
 	switch (msr) {
-	case MSR_K7_EVNTSEL0 ... MSR_K7_PERFCTR3:
-		return pmu->version > 0;
-	case MSR_F15H_PERF_CTL0 ... MSR_F15H_PERF_CTR5:
-		return guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
 		return pmu->version > 1;
 	default:
-		if (msr > MSR_F15H_PERF_CTR5 &&
-		    msr < MSR_F15H_PERF_CTL0 + 2 * pmu->nr_arch_gp_counters)
-			return pmu->version > 1;
 		break;
 	}
 
-	return amd_msr_idx_to_pmc(vcpu, msr);
+	return false;
 }
 
 static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
Signature · 1 line
2.56.1
#2 lore raw

Re: [PATCH v4 01/21] KVM: SVM: Drop counter MSR checks in amd_is_valid_msr()

Quoted 16 lines · Zide Chen

amd_msr_idx_to_pmc() accurately validates individual counter MSRs by checking whether the corresponding counter is supported for the vCPU. Validating counter MSRs again in amd_is_valid_msr() is redundant.

Furthermore, validating counter MSRs using broad checks like "pmu->version > 0" or "PERFCTR_CORE" is inaccurate and misleading. It falsely validates unexposed counters, which then fail in amd_pmu_{get,set}_msr(), injecting an errant #GP instead of taking KVM's fallback (read 0, write ignored) and breaking migration for unadvertised counters.

Finally, the range check in the default case is dead code because AMD CPUs currently support at most 6 Core Perf counters. Even if future AMD CPUs support more than 6, counter MSRs would still be sufficiently validated by amd_msr_idx_to_pmc().

Quoted 6 lines

Drop the counter cases and dead range check from amd_is_valid_msr(), leaving counter validation solely to amd_msr_idx_to_pmc().

Fixes 4a2771895ca6 ("KVM: x86/svm/pmu: Add AMD PerfMonV2 support")

Should we also drop the get_gp_pmc(.., MSR_IA32_PERFCTR0), get_gp_pmc(.., MSR_P6_EVNTSEL0), and get_fixed_pmc() calls from the default case in intel_is_valid_msr()?

They are also covered by intel_msr_idx_to_pmc() IIUC.

Quoted 39 lines

---
v4:
- New patch. (Jim)

---
 arch/x86/kvm/svm/pmu.c | 9 +--------
 1 file changed, 1 insertion(+), 8 deletions(-)
arch/x86/kvm/svm/pmu.c +1 -8
index c18286545a7a..bf2f23c30afc 100644
@@ -106,23 +106,16 @@ static bool amd_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
 	switch (msr) {
-	case MSR_K7_EVNTSEL0 ... MSR_K7_PERFCTR3:
-		return pmu->version > 0;
-	case MSR_F15H_PERF_CTL0 ... MSR_F15H_PERF_CTR5:
-		return guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
 		return pmu->version > 1;
 	default:
-		if (msr > MSR_F15H_PERF_CTR5 &&
-		    msr < MSR_F15H_PERF_CTL0 + 2 * pmu->nr_arch_gp_counters)
-			return pmu->version > 1;
 		break;
 	}
 
-	return amd_msr_idx_to_pmc(vcpu, msr);
+	return false;
 }
 
 static int amd_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
Signature · 1 line
2.56.1
author
#3 lore raw

Re: [PATCH v4 01/21] KVM: SVM: Drop counter MSR checks in amd_is_valid_msr()

Quoted 24 lines · Yosry Ahmed
Quoted 16 lines · Zide Chen

amd_msr_idx_to_pmc() accurately validates individual counter MSRs by checking whether the corresponding counter is supported for the vCPU. Validating counter MSRs again in amd_is_valid_msr() is redundant.

Furthermore, validating counter MSRs using broad checks like "pmu->version > 0" or "PERFCTR_CORE" is inaccurate and misleading. It falsely validates unexposed counters, which then fail in amd_pmu_{get,set}_msr(), injecting an errant #GP instead of taking KVM's fallback (read 0, write ignored) and breaking migration for unadvertised counters.

Finally, the range check in the default case is dead code because AMD CPUs currently support at most 6 Core Perf counters. Even if future AMD CPUs support more than 6, counter MSRs would still be sufficiently validated by amd_msr_idx_to_pmc().

Quoted 6 lines

Drop the counter cases and dead range check from amd_is_valid_msr(), leaving counter validation solely to amd_msr_idx_to_pmc().

Fixes 4a2771895ca6 ("KVM: x86/svm/pmu: Add AMD PerfMonV2 support")
Quoted 5 lines

Should we also drop the get_gp_pmc(.., MSR_IA32_PERFCTR0), get_gp_pmc(.., MSR_P6_EVNTSEL0), and get_fixed_pmc() calls from the default case in intel_is_valid_msr()?

They are also covered by intel_msr_idx_to_pmc() IIUC.

It's a shame that I didn't take one step further to check the Intel code.

I completely agree that intel_is_valid_msr() should be simplified:

arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 4824e7eb98d3..77ce713101a4 100644
@@ -229,10 +229,7 @@ static bool intel_is_valid_msr(struct kvm_vcpu

*vcpu, u32 msr)

                        ((perf_capabilities & PERF_CAP_PEBS_FORMAT) > 3);
                break;
        default:
-               ret = get_gp_pmc(pmu, msr, MSR_IA32_PERFCTR0) ||
-                       get_gp_pmc(pmu, msr, MSR_P6_EVNTSEL0) ||
-                       get_fixed_pmc(pmu, msr) || get_fw_gp_pmc(pmu,
msr) ||
-                       intel_pmu_is_valid_lbr_msr(vcpu, msr);
+               ret = intel_pmu_is_valid_lbr_msr(vcpu, msr);
                break;
        }

Additionally, intel_msr_idx_to_pmc() currently misses get_fw_gp_pmc(), which leaves FW counter writes untracked in kvm_pmu_mark_pmc_in_use(). Moving get_fw_gp_pmc() into intel_msr_idx_to_pmc() fixes that bug and allows dropping all counter checks from intel_is_valid_msr():

@@ -247,6 +244,7 @@ static struct kvm_pmc *intel_msr_idx_to_pmc(struct kvm_vcpu *vcpu, u32 msr)

        pmc = get_fixed_pmc(pmu, msr);
        pmc = pmc ? pmc : get_gp_pmc(pmu, msr, MSR_P6_EVNTSEL0);
        pmc = pmc ? pmc : get_gp_pmc(pmu, msr, MSR_IA32_PERFCTR0);

+ pmc = pmc ? pmc : get_fw_gp_pmc(pmu, msr);

        return pmc;

}

This seems a little bit too complicated in intel_msr_idx_to_pmc(), especially once PMU v6 is introduced and more v6 specific MSRs are available. We may do some cleanup (not in this series though)

- Fold get_fw_gp_pmc() into get_gp_pmc() and hide the base MSR from the API signature to keep the callers clean.
- Move these helpers from pmu.h to pmu_intel.c, since AMD already uses its own helpers.

@@ -245,10 +294,9 @@ static struct kvm_pmc *intel_msr_idx_to_pmc(struct kvm_vcpu *vcpu, u32 msr)

        struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
        struct kvm_pmc *pmc;
-       pmc = get_fixed_pmc(pmu, msr);
-       pmc = pmc ? pmc : get_gp_pmc(pmu, msr, MSR_P6_EVNTSEL0);
-       pmc = pmc ? pmc : get_gp_pmc(pmu, msr, MSR_IA32_PERFCTR0);
-       pmc = pmc ? pmc : get_fw_gp_pmc(pmu, msr);
+       pmc = intel_get_fixed_pmc(pmu, msr);
+       pmc = pmc ? pmc : intel_get_gp_eventsel_pmc(pmu, msr);
+       pmc = pmc ? pmc : intel_get_gp_counter_pmc(pmu, msr);
        return pmc;

}

@@ -419,18 +467,17 @@ static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)

                msr_info->data = pmu->pebs_data_cfg;
                break;
        default:
-               if ((pmc = get_gp_pmc(pmu, msr, MSR_IA32_PERFCTR0)) ||
-                   (pmc = get_gp_pmc(pmu, msr, MSR_IA32_PMC0))) {
+               if ((pmc = intel_get_gp_counter_pmc(pmu, msr))) {
                        u64 val = pmc_read_counter(pmc);
                        msr_info->data =
                                val & pmu->counter_bitmask[KVM_PMC_GP];
author
#4 lore raw

PATCH v4 2/21 KVM: x86/pmu: Remove redundant Perf Global Status MSR bit definitions

In reply to Zide Chen

IA32_PERF_GLOBAL_STATUS and its SET/RESET counterparts share bit definitions with perf_event.h, which is the canonical home for Intel PMU bit definitions (e.g., PERFEVTSEL, FIXED_CTR_CTRL, GLOBAL_STATUS). Drop the duplicate definitions from msr-index.h and update KVM code to use the macros from perf_event.h.

Drop the comment as well, since the macros now match the MSR naming.

No functional change intended.

---
 arch/x86/include/asm/msr-index.h |  8 --------
 arch/x86/kvm/vmx/pmu_intel.c     | 11 ++---------
 arch/x86/kvm/vmx/vmx.c           |  2 +-
 3 files changed, 3 insertions(+), 18 deletions(-)
arch/x86/include/asm/msr-index.h +0 -8
index 3a8e51a0c9e8..30cd04c0ace9 100644
@@ -1245,14 +1245,6 @@
 
 #define MSR_PERF_METRICS		0x00000329
 
-/* PERF_GLOBAL_OVF_CTL bits */
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT	55
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI		(1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT)
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF_BIT		62
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF			(1ULL <<  MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF_BIT)
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD_BIT		63
-#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD			(1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD_BIT)
-
 /* Geode defined MSRs */
 #define MSR_GEODE_BUSCONT_CONF0		0x00001900
 
arch/x86/kvm/vmx/pmu_intel.c +2 -9
index 70a8c4816135..36d91215f42a 100644
@@ -579,17 +579,10 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 			 ((BIT_ULL(pmu->nr_arch_fixed_counters) - 1) << KVM_FIXED_PMC_BASE_IDX));
 	pmu->global_ctrl_rsvd = counter_rsvd;
 
-	/*
-	 * GLOBAL_STATUS and GLOBAL_OVF_CONTROL (a.k.a. GLOBAL_STATUS_RESET)
-	 * share reserved bit definitions.  The kernel just happens to use
-	 * OVF_CTRL for the names.
-	 */
 	pmu->global_status_rsvd = pmu->global_ctrl_rsvd
-			& ~(MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF |
-			    MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD);
+			& ~(GLOBAL_STATUS_BUFFER_OVF | GLOBAL_STATUS_COND_CHG);
 	if (vmx_pt_mode_is_host_guest())
-		pmu->global_status_rsvd &=
-				~MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI;
+		pmu->global_status_rsvd &= ~GLOBAL_STATUS_TRACE_TOPAPMI;
 
 	if (perf_capabilities & PERF_CAP_PEBS_FORMAT) {
 		if (perf_capabilities & PERF_CAP_PEBS_BASELINE) {
arch/x86/kvm/vmx/vmx.c +1 -1
index b341af4c6057..efde96e15533 100644
@@ -8477,7 +8477,7 @@ static unsigned int vmx_handle_intel_pt_intr(void)
 		return 0;
 
 	kvm_make_request(KVM_REQ_PMI, vcpu);
-	__set_bit(MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT,
+	__set_bit(GLOBAL_STATUS_TRACE_TOPAPMI_BIT,
 		  (unsigned long *)&vcpu->arch.pmu.global_status);
 	return 1;
 }
Signature · 1 line
2.56.1
author
#5 lore raw

PATCH v4 3/21 KVM: x86/pmu: Rename all_valid_pmc_idx to pmc_exists

In reply to Zide Chen

all_valid_pmc_idx is a bitmap of valid PMC indices, not an index itself; rename it to pmc_exists to better reflect what it holds and keep the name consistent with other fields in the same struct.

No functional change intended.

---
v2:
- Pick the name pmc_exists other than all_valid_pmc_mask. (Sean)

---
 arch/x86/include/asm/kvm_host.h |  2 +-
 arch/x86/kvm/pmu.c              | 10 +++++-----
 arch/x86/kvm/vmx/pmu_intel.c    |  2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
index e2eef7057bd3..72b8fbf432a2 100644
@@ -595,7 +595,7 @@ struct kvm_pmu {
 		DECLARE_BITMAP(reprogram_pmi, X86_PMC_IDX_MAX);
 		atomic64_t __reprogram_pmi;
 	};
-	DECLARE_BITMAP(all_valid_pmc_idx, X86_PMC_IDX_MAX);
+	DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
 	DECLARE_BITMAP(pmc_in_use, X86_PMC_IDX_MAX);
 
 	DECLARE_BITMAP(pmc_counting_instructions, X86_PMC_IDX_MAX);
arch/x86/kvm/pmu.c +5 -5
index eef2f792dc9e..83883191d124 100644
@@ -944,7 +944,7 @@ static void kvm_pmu_reset(struct kvm_vcpu *vcpu)
 	bitmap_zero(pmu->reprogram_pmi, X86_PMC_IDX_MAX);
 	bitmap_zero(pmu->pmc_has_mode_specific_enables, X86_PMC_IDX_MAX);
 
-	kvm_for_each_pmc(pmu, pmc, i, pmu->all_valid_pmc_idx) {
+	kvm_for_each_pmc(pmu, pmc, i, pmu->pmc_exists) {
 		pmc_stop_counter(pmc);
 		pmc->counter = 0;
 		pmc->emulated_counter = 0;
@@ -990,7 +990,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->fixed_ctr_ctrl_rsvd = ~0ull;
 	pmu->pebs_enable_rsvd = ~0ull;
 	pmu->pebs_data_cfg_rsvd = ~0ull;
-	bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
+	bitmap_zero(pmu->pmc_exists, X86_PMC_IDX_MAX);
 
 	if (!vcpu->kvm->arch.enable_pmu)
 		return;
@@ -1011,8 +1011,8 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (kvm_vcpu_has_mediated_pmu(vcpu))
 		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
 
-	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
-	bitmap_set(pmu->all_valid_pmc_idx, KVM_FIXED_PMC_BASE_IDX,
+	bitmap_set(pmu->pmc_exists, 0, pmu->nr_arch_gp_counters);
+	bitmap_set(pmu->pmc_exists, KVM_FIXED_PMC_BASE_IDX,
 		   pmu->nr_arch_fixed_counters);
 }
 
@@ -1034,7 +1034,7 @@ void kvm_pmu_cleanup(struct kvm_vcpu *vcpu)
 
 	pmu->need_cleanup = false;
 
-	bitmap_andnot(bitmask, pmu->all_valid_pmc_idx,
+	bitmap_andnot(bitmask, pmu->pmc_exists,
 		      pmu->pmc_in_use, X86_PMC_IDX_MAX);
 
 	kvm_for_each_pmc(pmu, pmc, i, bitmask) {
arch/x86/kvm/vmx/pmu_intel.c +1 -1
index 36d91215f42a..ee18899614a1 100644
@@ -560,7 +560,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 		lbr_desc->records.nr = 0;
 
 	if (lbr_desc->records.nr)
-		bitmap_set(pmu->all_valid_pmc_idx, INTEL_PMC_IDX_FIXED_VLBR, 1);
+		bitmap_set(pmu->pmc_exists, INTEL_PMC_IDX_FIXED_VLBR, 1);
 
 	if (pmu->version == 1)
 		return;
Signature · 1 line
2.56.1
author
#6 lore raw

PATCH v4 4/21 KVM: x86/pmu: Rename reserved_bits to eventsel_rsvd in kvm_pmu

In reply to Zide Chen

This field stores the bits that are reserved in guest IA32_PERFEVTSELx MSRs. Rename it to eventsel_rsvd to better reflect its purpose and to align with other field names in struct kvm_pmu.

Opportunistically, replace magic numbers with macros.

No functional change intended.

---
v4:
- Diff AMD eventsel_rsvd against generic baseline instead of overwriting from scratch. (Yosry)

---
 arch/x86/include/asm/kvm_host.h |  2 +-
 arch/x86/kvm/pmu.c              |  6 +++++-
 arch/x86/kvm/svm/pmu.c          | 11 ++++++++---
 arch/x86/kvm/vmx/pmu_intel.c    | 12 ++++++------
 4 files changed, 20 insertions(+), 11 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
index 72b8fbf432a2..b8875ed5502c 100644
@@ -581,7 +581,7 @@ struct kvm_pmu {
 	u64 counter_bitmask[2];
 	u64 global_ctrl_rsvd;
 	u64 global_status_rsvd;
-	u64 reserved_bits;
+	u64 eventsel_rsvd;
 	u64 raw_event_mask;
 	struct kvm_pmc gp_counters[KVM_MAX_NR_GP_COUNTERS];
 	struct kvm_pmc fixed_counters[KVM_MAX_NR_FIXED_COUNTERS];
arch/x86/kvm/pmu.c +5 -1
index 83883191d124..cbd63329cf13 100644
@@ -983,7 +983,11 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->nr_arch_fixed_counters = 0;
 	pmu->counter_bitmask[KVM_PMC_GP] = 0;
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
-	pmu->reserved_bits = 0xffffffff00200000ull;
+	/*
+	 * KVM is not able to emulate the AnyThread bit due to cross-VM
+	 * information leakage on SMT cores.
+	 */
+	pmu->eventsel_rsvd = GENMASK_ULL(63, 32) | ARCH_PERFMON_EVENTSEL_ANY;
 	pmu->raw_event_mask = X86_RAW_EVENT_MASK;
 	pmu->global_ctrl_rsvd = ~0ull;
 	pmu->global_status_rsvd = ~0ull;
arch/x86/kvm/svm/pmu.c +8 -3
index bf2f23c30afc..c1e401fabbde 100644
@@ -156,7 +156,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	/* MSR_EVNTSELn */
 	pmc = get_gp_pmc_amd(pmu, msr, PMU_TYPE_EVNTSEL);
 	if (pmc) {
-		data &= ~pmu->reserved_bits;
+		data &= ~pmu->eventsel_rsvd;
 		if (data != pmc->eventsel) {
 			pmc->eventsel = data;
 			pmc->eventsel_hw = (data & ~AMD64_EVENTSEL_HOSTONLY) |
@@ -207,9 +207,14 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 
 	pmu->counter_bitmask[KVM_PMC_GP] = BIT_ULL(48) - 1;
 
-	pmu->reserved_bits = 0xfffffff000280000ull;
+	/*
+	 * AMD PerfEvtSeln registers do not support PinControl, but support
+	 * a 12-bit EventSelect (bits 35:32 in addition to 7:0).
+	 */
+	pmu->eventsel_rsvd |= ARCH_PERFMON_EVENTSEL_PIN_CONTROL;
+	pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_EVENT;
 	if (guest_cpu_cap_has(vcpu, X86_FEATURE_SVM) && kvm_vcpu_has_mediated_pmu(vcpu))
-		pmu->reserved_bits &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;
+		pmu->eventsel_rsvd &= ~AMD64_EVENTSEL_HOST_GUEST_MASK;
 
 	pmu->raw_event_mask = AMD64_RAW_EVENT_MASK;
 	/* not applicable to AMD; but clean them to prevent any fall out */
arch/x86/kvm/vmx/pmu_intel.c +6 -6
index ee18899614a1..fadc466cd600 100644
@@ -388,7 +388,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	struct kvm_pmc *pmc;
 	u32 msr = msr_info->index;
 	u64 data = msr_info->data;
-	u64 reserved_bits, diff;
+	u64 eventsel_rsvd, diff;
 
 	switch (msr) {
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
@@ -436,11 +436,11 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			pmc_write_counter(pmc, data);
 			break;
 		} else if ((pmc = get_gp_pmc(pmu, msr, MSR_P6_EVNTSEL0))) {
-			reserved_bits = pmu->reserved_bits;
+			eventsel_rsvd = pmu->eventsel_rsvd;
 			if ((pmc->idx == 2) &&
 			    (pmu->raw_event_mask & HSW_IN_TX_CHECKPOINTED))
-				reserved_bits ^= HSW_IN_TX_CHECKPOINTED;
-			if (data & reserved_bits)
+				eventsel_rsvd ^= HSW_IN_TX_CHECKPOINTED;
+			if (data & eventsel_rsvd)
 				return 1;
 
 			if (data != pmc->eventsel) {
@@ -548,7 +548,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (entry &&
 	    (boot_cpu_has(X86_FEATURE_HLE) || boot_cpu_has(X86_FEATURE_RTM)) &&
 	    (entry->ebx & (X86_FEATURE_HLE|X86_FEATURE_RTM))) {
-		pmu->reserved_bits ^= HSW_IN_TX;
+		pmu->eventsel_rsvd ^= HSW_IN_TX;
 		pmu->raw_event_mask |= (HSW_IN_TX|HSW_IN_TX_CHECKPOINTED);
 	}
 
@@ -587,7 +587,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	if (perf_capabilities & PERF_CAP_PEBS_FORMAT) {
 		if (perf_capabilities & PERF_CAP_PEBS_BASELINE) {
 			pmu->pebs_enable_rsvd = counter_rsvd;
-			pmu->reserved_bits &= ~ICL_EVENTSEL_ADAPTIVE;
+			pmu->eventsel_rsvd &= ~ICL_EVENTSEL_ADAPTIVE;
 			pmu->pebs_data_cfg_rsvd = ~0xff00000full;
 			intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE);
 		} else {
Signature · 1 line
2.56.1
author
#7 lore raw

PATCH v4 5/21 KVM: x86/pmu: Use pmc_exists in PMC lookup helpers

In reply to Zide Chen

pmu->nr_arch_{gp,fixed}_counters cannot represent non-contiguous GP or fixed counters.

Introduce kvm_is_{gp,fixed}_pmc_supported() to validate counter indices via pmu->pmc_exists, and switch the PMC lookup helpers to use them in preparation for dropping nr_arch_{gp,fixed}_counters.

No functional change intended.

---
v4:
- Move kvm_for_each_{gp,fixed}_counter() and kvm_{gp,fixed}_pmc_mask() to the patch where they are first used. (Yosry)
- Move pmc_exists64 union to the patch where they are first used.
- Retitle to reflect that this patch now converts PMC lookup helpers.
- Do not carry over Jim's Reviewed-by due to above changes.

v2:
- Rename kvm_{gp,fixed}_pmc_supported() to kvm_is_{gp,fixed}_pmc_supported() and simplify them by removing the temporary bitmap. (Sean)
- Split kvm_for_each_set_pmc_idx() to kvm_for_each_gp_counter() and kvm_for_each_fixed_counter(). (Sean)

---
 arch/x86/kvm/pmu.h | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)
arch/x86/kvm/pmu.h +24 -6
index 090c9bbb74f4..fbab4f5c9181 100644
@@ -88,6 +88,18 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
 	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
 }
 
+static inline bool kvm_is_gp_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
+{
+	return idx < KVM_MAX_NR_GP_COUNTERS &&
+	       test_bit(idx, pmu->pmc_exists);
+}
+
+static inline bool kvm_is_fixed_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
+{
+	return idx < KVM_MAX_NR_FIXED_COUNTERS &&
+	       test_bit(KVM_FIXED_PMC_BASE_IDX + idx, pmu->pmc_exists);
+}
+
 /*
  * KVM tracks all counters in 64-bit bitmaps, with general purpose counters
  * mapped to bits 31:0 and fixed counters mapped to 63:32, e.g. fixed counter 0
@@ -104,11 +116,11 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
  */
 static inline struct kvm_pmc *kvm_pmc_idx_to_pmc(struct kvm_pmu *pmu, int idx)
 {
-	if (idx < pmu->nr_arch_gp_counters)
+	if (kvm_is_gp_pmc_supported(pmu, idx))
 		return &pmu->gp_counters[idx];
 
 	idx -= KVM_FIXED_PMC_BASE_IDX;
-	if (idx >= 0 && idx < pmu->nr_arch_fixed_counters)
+	if (kvm_is_fixed_pmc_supported(pmu, idx))
 		return &pmu->fixed_counters[idx];
 
 	return NULL;
@@ -168,9 +180,12 @@ static inline bool kvm_valid_perf_global_ctrl(struct kvm_pmu *pmu,
 static inline struct kvm_pmc *get_gp_pmc(struct kvm_pmu *pmu, u32 msr,
 					 u32 base)
 {
-	if (msr >= base && msr < base + pmu->nr_arch_gp_counters) {
+	if (msr >= base && msr < base + KVM_MAX_NR_GP_COUNTERS) {
 		u32 index = array_index_nospec(msr - base,
-					       pmu->nr_arch_gp_counters);
+					       KVM_MAX_NR_GP_COUNTERS);
+
+		if (!kvm_is_gp_pmc_supported(pmu, index))
+			return NULL;
 
 		return &pmu->gp_counters[index];
 	}
@@ -183,9 +198,12 @@ static inline struct kvm_pmc *get_fixed_pmc(struct kvm_pmu *pmu, u32 msr)
 {
 	int base = MSR_CORE_PERF_FIXED_CTR0;
 
-	if (msr >= base && msr < base + pmu->nr_arch_fixed_counters) {
+	if (msr >= base && msr < base + KVM_MAX_NR_FIXED_COUNTERS) {
 		u32 index = array_index_nospec(msr - base,
-					       pmu->nr_arch_fixed_counters);
+					       KVM_MAX_NR_FIXED_COUNTERS);
+
+		if (!kvm_is_fixed_pmc_supported(pmu, index))
+			return NULL;
 
 		return &pmu->fixed_counters[index];
 	}
Signature · 1 line
2.56.1
Reviewed-by on v4 5/21
#8 lore raw

Re: [PATCH v4 05/21] KVM: x86/pmu: Use pmc_exists in PMC lookup helpers

Quoted 7 lines · Zide Chen

pmu->nr_arch_{gp,fixed}_counters cannot represent non-contiguous GP or fixed counters.

Introduce kvm_is_{gp,fixed}_pmc_supported() to validate counter indices via pmu->pmc_exists, and switch the PMC lookup helpers to use them in preparation for dropping nr_arch_{gp,fixed}_counters.

Quoted 5 lines

No functional change intended.

author
#9 lore raw

PATCH v4 6/21 KVM: x86/pmu: Drop nr_arch_{gp,fixed}_counters from kvm_pmu

In reply to Zide Chen

From: Dapeng Mi <[email protected]>

Populate and use pmc_exists directly when walking guest-visible PMCs, instead of relying on pmu->nr_arch_{gp,fixed}_counters as intermediate state, in preparation for switching KVM's host PMU capabilities from counter counts to counter bitmaps.

Extend pmu->pmc_exists to a union so that the u64 alias (pmc_exists64) is available for mask arithmetic operations. Introduce kvm_{gp,fixed}_pmc_mask() to extract the respective counter bitmasks, and provide the kvm_for_each_{gp,fixed}_counter() iterator macros to replace open-coded counter index loops.

Consolidate PMU MSR intercept setup in VMX and SVM into single loops. In SVM, this also fixes legacy counter interception by re-enabling intercepts if a vCPU's counter count is narrowed.

---
v4:
- In intel_pmu_refresh(), directly clamp guest counters against kvm_pmu_cap.num_counters_{gp,fixed} using min_t() instead of temporary BIT_ULL() mask ANDing, avoiding intermediate code churn with the subsequent bitmask capability conversion.
- Move pmc_exists64 union, BUILD_BUG_ON(), kvm_{gp,fixed}_pmc_mask(), and kvm_for_each_{gp,fixed}_counter() to this patch where they are first used, and return u64 to avoid casts and 32-bit truncation hazards. (Yosry)
- Pass 'pmu' directly to kvm_for_each_{gp,fixed}_counter() using compound literals. (Yosry)
- In reprogram_fixed_counters(), directly index &pmu->fixed_counters[i] instead of calling get_fixed_pmc(). (Yosry)
- In intel_rdpmc_ecx_to_pmc(), rename 'num_counters' to 'max_nr_counters'. (Yosry)

v3:
- Consolidate PMU MSR intercept setup into single loops in VMX and SVM.
- Rename fixed_cntr_mask to fixed_mask in intel_pmu_refresh().
- Fix 32-bit zero-extension bug in pebs_enable_rsvd calculation by explicitly casting to u64. (Sashiko)

v2:
- Fix AMD legacy PMU MSR intercept handling for guest PMU narrowing reconfiguration by adding for_each_clear_bit_from().
- Re-word the stale interception changelog part.

---
 arch/x86/include/asm/kvm_host.h |  7 ++---
 arch/x86/kvm/pmu.c              | 24 +++++++----------
 arch/x86/kvm/pmu.h              | 23 ++++++++++++++++
 arch/x86/kvm/svm/pmu.c          | 23 +++++++---------
 arch/x86/kvm/svm/svm.c          | 13 ++++-----
 arch/x86/kvm/vmx/nested.c       |  4 +--
 arch/x86/kvm/vmx/pmu_intel.c    | 47 ++++++++++++++++++++-------------
 arch/x86/kvm/vmx/vmx.c          | 26 +++++++++---------
 8 files changed, 95 insertions(+), 72 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -3
index b8875ed5502c..8097d106d157 100644
@@ -570,8 +570,6 @@ struct kvm_pmc {
 
 struct kvm_pmu {
 	u8 version;
-	unsigned nr_arch_gp_counters;
-	unsigned nr_arch_fixed_counters;
 	unsigned available_event_types;
 	u64 fixed_ctr_ctrl;
 	u64 fixed_ctr_ctrl_hw;
@@ -595,7 +593,10 @@ struct kvm_pmu {
 		DECLARE_BITMAP(reprogram_pmi, X86_PMC_IDX_MAX);
 		atomic64_t __reprogram_pmi;
 	};
-	DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
+	union {
+		DECLARE_BITMAP(pmc_exists, X86_PMC_IDX_MAX);
+		u64 pmc_exists64;
+	};
 	DECLARE_BITMAP(pmc_in_use, X86_PMC_IDX_MAX);
 
 	DECLARE_BITMAP(pmc_counting_instructions, X86_PMC_IDX_MAX);
arch/x86/kvm/pmu.c +10 -14
index cbd63329cf13..484dfd2eba10 100644
@@ -787,8 +787,8 @@ static bool kvm_need_any_pmc_intercept(struct kvm_vcpu *vcpu)
 	 * KVM's capabilities are constrained based on KVM support, i.e. KVM's
 	 * capabilities themselves may be a subset of hardware capabilities.
 	 */
-	return pmu->nr_arch_gp_counters != kvm_host_pmu.num_counters_gp ||
-	       pmu->nr_arch_fixed_counters != kvm_host_pmu.num_counters_fixed;
+	return kvm_gp_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_gp) - 1 ||
+	       kvm_fixed_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_fixed) - 1;
 }
 
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu)
@@ -979,8 +979,6 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	kvm_pmu_reset(vcpu);
 
 	pmu->version = 0;
-	pmu->nr_arch_gp_counters = 0;
-	pmu->nr_arch_fixed_counters = 0;
 	pmu->counter_bitmask[KVM_PMC_GP] = 0;
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
 	/*
@@ -1008,22 +1006,20 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (pmu->nr_arch_gp_counters &&
+	if (kvm_gp_pmc_mask(pmu) &&
 	    (kvm_pmu_has_perf_global_ctrl(pmu) || kvm_vcpu_has_mediated_pmu(vcpu)))
-		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
+		pmu->global_ctrl = kvm_gp_pmc_mask(pmu);
 
 	if (kvm_vcpu_has_mediated_pmu(vcpu))
 		kvm_pmu_call(write_global_ctrl)(pmu->global_ctrl);
-
-	bitmap_set(pmu->pmc_exists, 0, pmu->nr_arch_gp_counters);
-	bitmap_set(pmu->pmc_exists, KVM_FIXED_PMC_BASE_IDX,
-		   pmu->nr_arch_fixed_counters);
 }
 
 void kvm_pmu_init(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
+	BUILD_BUG_ON(sizeof(pmu->pmc_exists) != sizeof(pmu->pmc_exists64));
+
 	memset(pmu, 0, sizeof(*pmu));
 	kvm_pmu_call(init)(vcpu);
 }
@@ -1334,14 +1330,14 @@ static void kvm_pmu_load_guest_pmcs(struct kvm_vcpu *vcpu)
 	 * is intercepted if hardware has counters that aren't visible to the
 	 * guest (KVM will inject #GP as appropriate).
 	 */
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	kvm_for_each_gp_counter(i, pmu) {
 		pmc = &pmu->gp_counters[i];
 
 		if (pmc->counter != rdpmc(i))
 			wrmsrq(gp_counter_msr(i), pmc->counter);
 		wrmsrq(gp_eventsel_msr(i), pmc->eventsel_hw);
 	}
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+	kvm_for_each_fixed_counter(i, pmu) {
 		pmc = &pmu->fixed_counters[i];
 
 		if (pmc->counter != rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i))
@@ -1391,7 +1387,7 @@ static void kvm_pmu_put_guest_pmcs(struct kvm_vcpu *vcpu)
 	 * Clear selectors and counters to ensure hardware doesn't count using
 	 * guest controls when the host (perf) restores its state.
 	 */
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	kvm_for_each_gp_counter(i, pmu) {
 		pmc = &pmu->gp_counters[i];
 
 		pmc->counter = rdpmc(i);
@@ -1401,7 +1397,7 @@ static void kvm_pmu_put_guest_pmcs(struct kvm_vcpu *vcpu)
 			wrmsrq(gp_eventsel_msr(i), 0);
 	}
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+	kvm_for_each_fixed_counter(i, pmu) {
 		pmc = &pmu->fixed_counters[i];
 
 		pmc->counter = rdpmc(INTEL_PMC_FIXED_RDPMC_BASE | i);
arch/x86/kvm/pmu.h +23 -0
index fbab4f5c9181..b312f80f5cda 100644
@@ -88,6 +88,18 @@ static inline bool kvm_vcpu_has_mediated_pmu(struct kvm_vcpu *vcpu)
 	return enable_mediated_pmu && vcpu_to_pmu(vcpu)->version;
 }
 
+static inline u64 kvm_gp_pmc_mask(struct kvm_pmu *pmu)
+{
+	return pmu->pmc_exists64 &
+	       GENMASK_ULL(KVM_MAX_NR_GP_COUNTERS - 1, 0);
+}
+
+static inline u64 kvm_fixed_pmc_mask(struct kvm_pmu *pmu)
+{
+	return (pmu->pmc_exists64 >> KVM_FIXED_PMC_BASE_IDX) &
+	       GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
+}
+
 static inline bool kvm_is_gp_pmc_supported(struct kvm_pmu *pmu, unsigned int idx)
 {
 	return idx < KVM_MAX_NR_GP_COUNTERS &&
@@ -132,6 +144,17 @@ static inline struct kvm_pmc *kvm_pmc_idx_to_pmc(struct kvm_pmu *pmu, int idx)
 			continue;				\
 		else						\
 
+/*
+ * Iterate over all general-purpose or fixed counters supported by @pmu.
+ */
+#define kvm_for_each_gp_counter(i, pmu)	\
+	for_each_set_bit((i), &(unsigned long){ kvm_gp_pmc_mask(pmu) }, KVM_MAX_NR_GP_COUNTERS)
+
+#define kvm_for_each_fixed_counter(i, pmu)				\
+	for_each_set_bit((i),						\
+			 &(unsigned long){ kvm_fixed_pmc_mask(pmu) },	\
+			 KVM_MAX_NR_FIXED_COUNTERS)
+
 static inline u64 pmc_bitmask(struct kvm_pmc *pmc)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
arch/x86/kvm/svm/pmu.c +10 -13
index c1e401fabbde..64215024e70e 100644
@@ -27,12 +27,11 @@ enum pmu_type {
 
 static struct kvm_pmc *amd_pmu_get_pmc(struct kvm_pmu *pmu, int pmc_idx)
 {
-	unsigned int num_counters = pmu->nr_arch_gp_counters;
-
-	if (pmc_idx >= num_counters)
+	if (!kvm_is_gp_pmc_supported(pmu, pmc_idx))
 		return NULL;
 
-	return &pmu->gp_counters[array_index_nospec(pmc_idx, num_counters)];
+	pmc_idx = array_index_nospec(pmc_idx, KVM_MAX_NR_AMD_GP_COUNTERS);
+	return &pmu->gp_counters[pmc_idx];
 }
 
 static inline struct kvm_pmc *get_gp_pmc_amd(struct kvm_pmu *pmu, u32 msr,
@@ -77,7 +76,7 @@ static int amd_check_rdpmc_early(struct kvm_vcpu *vcpu, unsigned int idx)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 
-	if (idx >= pmu->nr_arch_gp_counters)
+	if (!kvm_is_gp_pmc_supported(pmu, idx))
 		return -EINVAL;
 
 	return 0;
@@ -177,6 +176,7 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 
 static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 {
+	unsigned int nr_gp_counters = AMD64_NUM_COUNTERS;
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	union cpuid_0x80000022_ebx ebx;
 
@@ -190,18 +190,16 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 		BUILD_BUG_ON(x86_feature_cpuid(X86_FEATURE_PERFMON_V2).function != 0x80000022 ||
 			     x86_feature_cpuid(X86_FEATURE_PERFMON_V2).index);
 		ebx.full = kvm_find_cpuid_entry_index(vcpu, 0x80000022, 0)->ebx;
-		pmu->nr_arch_gp_counters = ebx.split.num_core_pmc;
+		nr_gp_counters = ebx.split.num_core_pmc;
 	} else if (guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE)) {
-		pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS_CORE;
-	} else {
-		pmu->nr_arch_gp_counters = AMD64_NUM_COUNTERS;
+		nr_gp_counters = AMD64_NUM_COUNTERS_CORE;
 	}
 
-	pmu->nr_arch_gp_counters = min_t(unsigned int, pmu->nr_arch_gp_counters,
-					 kvm_pmu_cap.num_counters_gp);
+	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
+			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
 
 	if (pmu->version > 1) {
-		pmu->global_ctrl_rsvd = ~(BIT_ULL(pmu->nr_arch_gp_counters) - 1);
+		pmu->global_ctrl_rsvd = ~pmu->pmc_exists64;
 		pmu->global_status_rsvd = pmu->global_ctrl_rsvd;
 	}
 
@@ -219,7 +217,6 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->raw_event_mask = AMD64_RAW_EVENT_MASK;
 	/* not applicable to AMD; but clean them to prevent any fall out */
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
-	pmu->nr_arch_fixed_counters = 0;
 }
 
 static void amd_pmu_init(struct kvm_vcpu *vcpu)
arch/x86/kvm/svm/svm.c +5 -8
index 136069dfcd44..e21abcedea70 100644
@@ -761,24 +761,21 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 {
 	bool intercept = !kvm_vcpu_has_mediated_pmu(vcpu);
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	int i;
 
 	if (!enable_mediated_pmu)
 		return;
 
 	/* Legacy counters are always available for AMD CPUs with a PMU. */
-	for (i = 0; i < min(pmu->nr_arch_gp_counters, AMD64_NUM_COUNTERS); i++)
+	for (i = 0; i < AMD64_NUM_COUNTERS; i++)
 		svm_set_intercept_for_msr(vcpu, MSR_K7_PERFCTR0 + i,
-					  MSR_TYPE_RW, intercept);
+					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept |= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++)
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++)
 		svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
-					  MSR_TYPE_RW, intercept);
-
-	for ( ; i < kvm_pmu_cap.num_counters_gp; i++)
-		svm_enable_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
-					     MSR_TYPE_RW);
+					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept = kvm_need_perf_global_ctrl_intercept(vcpu);
 	svm_set_intercept_for_msr(vcpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL,
arch/x86/kvm/vmx/nested.c +2 -2
index 9b0bfa2f854c..9b2559046971 100644
@@ -713,12 +713,12 @@ static void nested_vmx_merge_pmu_msr_bitmaps(struct kvm_vcpu *vcpu,
 	if (!kvm_vcpu_has_mediated_pmu(vcpu))
 		return;
 
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	kvm_for_each_gp_counter(i, pmu) {
 		nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_PERFCTR0 + i);
 		nested_vmx_merge_msr_bitmaps_rw(MSR_IA32_PMC0 + i);
 	}
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
+	kvm_for_each_fixed_counter(i, pmu)
 		nested_vmx_merge_msr_bitmaps_rw(MSR_CORE_PERF_FIXED_CTR0 + i);
 
 	nested_vmx_merge_msr_bitmaps_rw(MSR_CORE_PERF_GLOBAL_CTRL);
arch/x86/kvm/vmx/pmu_intel.c +29 -18

136 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.c +12 -14
index efde96e15533..944ecbf351cf 100644
@@ -4242,6 +4242,8 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 	bool has_mediated_pmu = kvm_vcpu_has_mediated_pmu(vcpu);
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	unsigned long fixed_mask = kvm_fixed_pmc_mask(pmu);
+	unsigned long gp_mask = kvm_gp_pmc_mask(pmu);
 	bool intercept = !has_mediated_pmu;
 	int i;
 
@@ -4262,25 +4264,21 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 
 	vm_exit_controls_changebit(vmx, vm_exit_controls_bits, has_mediated_pmu);
 
-	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+		bool pmc_intercept = intercept || !test_bit(i, &gp_mask);
+
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
-					  MSR_TYPE_RW, intercept);
+					  MSR_TYPE_RW, pmc_intercept);
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW,
-					  intercept || !fw_writes_is_enabled(vcpu));
-	}
-	for ( ; i < kvm_pmu_cap.num_counters_gp; i++) {
-		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
-					  MSR_TYPE_RW, true);
-		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i,
-					  MSR_TYPE_RW, true);
+					  pmc_intercept || !fw_writes_is_enabled(vcpu));
 	}
 
-	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
-		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
-					  MSR_TYPE_RW, intercept);
-	for ( ; i < kvm_pmu_cap.num_counters_fixed; i++)
+	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+		bool pmc_intercept = intercept || !test_bit(i, &fixed_mask);
+
 		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
-					  MSR_TYPE_RW, true);
+					  MSR_TYPE_RW, pmc_intercept);
+	}
 
 	intercept = kvm_need_perf_global_ctrl_intercept(vcpu);
 	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_STATUS,
Signature · 1 line
2.56.1
author
#10 lore raw

PATCH v4 7/21 KVM: x86/pmu: Expose kvm_host_pmu to vendor modules

In reply to Zide Chen

kvm_host_pmu holds the unadulterated host PMU capabilities, which can be used to compare against guest capabilities to determine whether certain MSRs should be intercepted by KVM.

Exposing it directly avoids the need to introduce per-field accessors, which would add boilerplate without hiding any implementation detail worth encapsulating.

Opportunistically, fix typo "Unadultered" in the comment.

---
 arch/x86/kvm/pmu.c | 5 +++--
 arch/x86/kvm/pmu.h | 1 +
 2 files changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/pmu.c +3 -2
index 484dfd2eba10..d203cbc7fd12 100644
@@ -27,8 +27,9 @@
 /* This is enough to filter the vast majority of currently defined events. */
 #define KVM_PMU_EVENT_FILTER_MAX_EVENTS 300
 
-/* Unadultered PMU capabilities of the host, i.e. of hardware. */
-static struct x86_pmu_capability __read_mostly kvm_host_pmu;
+/* Unadulterated PMU capabilities of the host, i.e. of hardware. */
+struct x86_pmu_capability __read_mostly kvm_host_pmu;
+EXPORT_SYMBOL_FOR_KVM_INTERNAL(kvm_host_pmu);
 
 /* KVM's PMU capabilities, i.e. the intersection of KVM and hardware support. */
 struct x86_pmu_capability __read_mostly kvm_pmu_cap;
arch/x86/kvm/pmu.h +1 -0
index b312f80f5cda..02a19aa7e7de 100644
@@ -252,6 +252,7 @@ static inline bool pmc_is_locally_enabled(struct kvm_pmc *pmc)
 	return !kvm_pmu_call(pmc_is_disabled_in_current_mode)(pmc);
 }
 
+extern struct x86_pmu_capability kvm_host_pmu;
 extern struct x86_pmu_capability kvm_pmu_cap;
 
 void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops);
Signature · 1 line
2.56.1
author
#11 lore raw

PATCH v4 8/21 perf/x86: Plumb counter bitmap from x86_pmu to x86_pmu_cap

In reply to Zide Chen

From: Dapeng Mi <[email protected]>

Intel PerfMon v5 introduced CPUID.0AH:ECX to support non-contiguous fixed counters and Architectural PerfMon Extension leaf (0x23) further supports non-contiguous general-purpose counters.

num_counters_{gp,fixed} indicates the total number of GP or fixed counters, but cannot represent non-contiguous counters.

Add cntr_mask and fixed_cntr_mask unions so that KVM can get the accurate counter availability directly from x86_pmu_cap. The u64 alias is convenient for mask arithmetic, while the bitmap form works with for_each_set_bit() and friends.

num_counters_{gp,fixed} in x86_pmu_capability will be removed once callers have been converted to the use of {,fixed_}cntr_mask.

---
 arch/x86/events/core.c            | 6 ++++--
 arch/x86/include/asm/perf_event.h | 8 ++++++++
 2 files changed, 12 insertions(+), 2 deletions(-)
arch/x86/events/core.c +4 -2
index 8b3ea0adb965..a9b86cecbc78 100644
@@ -3158,8 +3158,10 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	cap->version		= x86_pmu.version;
 	cap->num_counters_gp	= x86_pmu_num_counters(NULL);
 	cap->num_counters_fixed	= x86_pmu_num_counters_fixed(NULL);
-	cap->bit_width_gp	= cap->num_counters_gp ? x86_pmu.cntval_bits : 0;
-	cap->bit_width_fixed	= cap->num_counters_fixed ? x86_pmu.cntval_bits : 0;
+	cap->cntr_mask64	= x86_pmu.cntr_mask64;
+	cap->fixed_cntr_mask64	= x86_pmu.fixed_cntr_mask64;
+	cap->bit_width_gp	= cap->cntr_mask64 ? x86_pmu.cntval_bits : 0;
+	cap->bit_width_fixed	= cap->fixed_cntr_mask64 ? x86_pmu.cntval_bits : 0;
 	cap->events_mask	= (unsigned int)x86_pmu.events_maskl;
 	cap->events_mask_len	= x86_pmu.events_mask_len;
 	cap->pebs_ept		= x86_pmu.pebs_ept;
arch/x86/include/asm/perf_event.h +8 -0
index 1eb13673e889..190fda412a61 100644
@@ -302,6 +302,14 @@ struct x86_pmu_capability {
 	int		version;
 	int		num_counters_gp;
 	int		num_counters_fixed;
+	union {
+		u64	cntr_mask64;
+		DECLARE_BITMAP(cntr_mask, X86_PMC_IDX_MAX);
+	};
+	union {
+		u64	fixed_cntr_mask64;
+		DECLARE_BITMAP(fixed_cntr_mask, X86_PMC_IDX_MAX);
+	};
 	int		bit_width_gp;
 	int		bit_width_fixed;
 	unsigned int	events_mask;
Signature · 1 line
2.56.1
author
#12 lore raw

PATCH v4 9/21 KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities

In reply to Zide Chen

From: Dapeng Mi <[email protected]>

Intel platforms support non-contiguous fixed counters via CPUID.0AH:ECX starting with PerfMon v5, and support non-contiguous GP counters through the Architectural PerfMon Extension (CPUID leaf 23H).

struct x86_pmu_capability now exposes {,fixed_}cntr_mask64 bitmaps, which may contain sparse bits representing non-contiguous counters. Switch KVM's kvm_host_pmu and kvm_pmu_cap consumers over to the new bitmask fields, including iterating kvm_host_pmu bitmasks directly in VMX and SVM PMU MSR intercept setup.

CPUID.0AH:EAX[15:8] and CPUID.0AH:EDX[4:0] enumerate only contiguous counters. Derive these values from kvm_pmu_cap.{,fixed_}cntr_mask64 as the number of consecutive counters starting at index 0.

---
v4:
- Cleanly substitute kvm_pmu_cap.num_counters_{gp,fixed} with find_first_zero_bit() in intel_pmu_refresh() without throwaway code churn.
- Use find_first_zero_bit() instead of hweight64() for AMD CPUID 0x80000022 EBX[3:0] (NumCorePmc) and svm_set_cpu_caps() to enumerate contiguous counters. (Yosry)
- Do not carry over Jim's Reviewed-by from v3 due to reworking counter clamping in intel_pmu_refresh().

v3:
- Iterate kvm_host_pmu bitmasks directly in VMX and SVM intercept loops.
- Clamp nr_{gp,fixed}_counters to the number of contiguous counters supported by KVM in intel_pmu_refresh(). (Jim, Sashiko)
- Drop redundant !enable_mediated_pmu clamp on fixed_cntr_mask64. (Sashiko)

---
 arch/x86/kvm/cpuid.c         | 17 ++++++++++++++---
 arch/x86/kvm/msrs.c          | 12 ++++++------
 arch/x86/kvm/pmu.c           | 17 ++++++++---------
 arch/x86/kvm/svm/pmu.c       |  2 +-
 arch/x86/kvm/svm/svm.c       |  8 ++++----
 arch/x86/kvm/vmx/pmu_intel.c | 18 +++++++++++++++---
 arch/x86/kvm/vmx/vmx.c       |  4 ++--
 7 files changed, 50 insertions(+), 28 deletions(-)
arch/x86/kvm/cpuid.c +14 -3
index adafd6fa2c54..427871ec365f 100644
@@ -1535,10 +1535,18 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 		}
 
 		eax.split.version_id = kvm_pmu_cap.version;
-		eax.split.num_counters = kvm_pmu_cap.num_counters_gp;
+
+		/* Contiguous GP counters only. */
+		eax.split.num_counters =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		eax.split.bit_width = kvm_pmu_cap.bit_width_gp;
 		eax.split.mask_length = kvm_pmu_cap.events_mask_len;
-		edx.split.num_counters_fixed = kvm_pmu_cap.num_counters_fixed;
+
+		/* Contiguous fixed counters only. */
+		edx.split.num_counters_fixed =
+			find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+					    KVM_MAX_NR_FIXED_COUNTERS);
 		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
 
 		if (kvm_pmu_cap.version)
@@ -1903,7 +1911,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 
 		cpuid_entry_override(entry, CPUID_8000_0022_EAX);
 
-		ebx.split.num_core_pmc = kvm_pmu_cap.num_counters_gp;
+		/* Contiguous GP counters only. */
+		ebx.split.num_core_pmc =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		entry->ebx = ebx.full;
 		break;
 	}
arch/x86/kvm/msrs.c +6 -6
index e91650af44c2..01fb130a7d24 100644
@@ -2621,20 +2621,20 @@ static void kvm_probe_msr_to_save(u32 msr_index)
 		break;
 	case MSR_ARCH_PERFMON_PERFCTR0 ...
 	     MSR_ARCH_PERFMON_PERFCTR0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_PERFCTR0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_PERFCTR0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_EVENTSEL0 ...
 	     MSR_ARCH_PERFMON_EVENTSEL0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_EVENTSEL0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_EVENTSEL0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_FIXED_CTR0 ...
 	     MSR_ARCH_PERFMON_FIXED_CTR0 + KVM_MAX_NR_FIXED_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_FIXED_CTR0 >=
-		    kvm_pmu_cap.num_counters_fixed)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_FIXED_CTR0) &
+		      kvm_pmu_cap.fixed_cntr_mask64))
 			return;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
arch/x86/kvm/pmu.c +8 -9
index d203cbc7fd12..e1999a6c85d7 100644
@@ -20,7 +20,6 @@
 #include <asm/perf_event.h>
 #include <asm/cpu_device_id.h>
 #include "x86.h"
-#include "cpuid.h"
 #include "lapic.h"
 #include "pmu.h"
 
@@ -156,8 +155,8 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 		 * there are a non-zero number of counters, but fewer than what
 		 * is architecturally required.
 		 */
-		if (!kvm_host_pmu.num_counters_gp ||
-		    WARN_ON_ONCE(kvm_host_pmu.num_counters_gp < min_nr_gp_ctrs))
+		if (!kvm_host_pmu.cntr_mask64 ||
+		    WARN_ON_ONCE(hweight64(kvm_host_pmu.cntr_mask64) < min_nr_gp_ctrs))
 			enable_pmu = false;
 		else if (is_intel && !kvm_host_pmu.version)
 			enable_pmu = false;
@@ -177,10 +176,10 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 
 	memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu));
 	kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
-	kvm_pmu_cap.num_counters_gp = min(kvm_pmu_cap.num_counters_gp,
-					  pmu_ops->MAX_NR_GP_COUNTERS);
-	kvm_pmu_cap.num_counters_fixed = min(kvm_pmu_cap.num_counters_fixed,
-					     KVM_MAX_NR_FIXED_COUNTERS);
+	kvm_pmu_cap.cntr_mask64 &=
+		GENMASK_ULL(pmu_ops->MAX_NR_GP_COUNTERS - 1, 0);
+	kvm_pmu_cap.fixed_cntr_mask64 &=
+		GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
 
 	kvm_pmu_eventsel.INSTRUCTIONS_RETIRED =
 		perf_get_hw_event_config(PERF_COUNT_HW_INSTRUCTIONS);
@@ -788,8 +787,8 @@ static bool kvm_need_any_pmc_intercept(struct kvm_vcpu *vcpu)
 	 * KVM's capabilities are constrained based on KVM support, i.e. KVM's
 	 * capabilities themselves may be a subset of hardware capabilities.
 	 */
-	return kvm_gp_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_gp) - 1 ||
-	       kvm_fixed_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_fixed) - 1;
+	return kvm_gp_pmc_mask(pmu) != kvm_host_pmu.cntr_mask64 ||
+	       kvm_fixed_pmc_mask(pmu) != kvm_host_pmu.fixed_cntr_mask64;
 }
 
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu)
arch/x86/kvm/svm/pmu.c +1 -1
index 64215024e70e..5eedf9b35fce 100644
@@ -196,7 +196,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
-			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
+			    kvm_pmu_cap.cntr_mask64;
 
 	if (pmu->version > 1) {
 		pmu->global_ctrl_rsvd = ~pmu->pmc_exists64;
arch/x86/kvm/svm/svm.c +4 -4
index e21abcedea70..5b9c9d149486 100644
@@ -773,7 +773,7 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept |= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++)
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, AMD64_NUM_COUNTERS_CORE)
 		svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
@@ -5719,9 +5719,9 @@ static __init void svm_set_cpu_caps(void)
 		 * access to enough counters to virtualize "core" support,
 		 * otherwise limit vPMU support to the legacy number of counters.
 		 */
-		if (kvm_pmu_cap.num_counters_gp < AMD64_NUM_COUNTERS_CORE)
-			kvm_pmu_cap.num_counters_gp = min(AMD64_NUM_COUNTERS,
-							  kvm_pmu_cap.num_counters_gp);
+		if (find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					KVM_MAX_NR_GP_COUNTERS) < AMD64_NUM_COUNTERS_CORE)
+			kvm_pmu_cap.cntr_mask64 &= GENMASK_ULL(AMD64_NUM_COUNTERS - 1, 0);
 		else
 			kvm_cpu_cap_check_and_set(X86_FEATURE_PERFCTR_CORE);
 
arch/x86/kvm/vmx/pmu_intel.c +15 -3
index 68d23c7877a2..a240b74ac704 100644
@@ -493,7 +493,8 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 	 * have a known encoding for the associated general purpose event.
 	 */
 	eventsel = perf_get_hw_event_config(fixed_pmc_perf_ids[index]);
-	WARN_ON_ONCE(!eventsel && index < kvm_pmu_cap.num_counters_fixed);
+	WARN_ON_ONCE(!eventsel &&
+		     (kvm_pmu_cap.fixed_cntr_mask64 & BIT_ULL(index)));
 	return eventsel;
 }
 
@@ -548,12 +549,23 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				      kvm_pmu_cap.events_mask_len);
 	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
 
+	/*
+	 * CPUID.0AH:EDX[4:0] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous fixed counters supported by KVM.
+	 */
 	nr_fixed_counters = min_t(int, edx.split.num_counters_fixed,
-				  kvm_pmu_cap.num_counters_fixed);
+				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+						      KVM_MAX_NR_FIXED_COUNTERS));
+
 	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
 
+	/*
+	 * CPUID.0AH:EAX[15:8] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous GP counters supported by KVM.
+	 */
 	nr_gp_counters = min_t(int, eax.split.num_counters,
-			       kvm_pmu_cap.num_counters_gp);
+			       find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+						   KVM_MAX_NR_GP_COUNTERS));
 	pmu->pmc_exists64 = BIT_ULL(nr_gp_counters) - 1;
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
arch/x86/kvm/vmx/vmx.c +2 -2
index 944ecbf351cf..9db6107943ec 100644
@@ -4264,7 +4264,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 
 	vm_exit_controls_changebit(vmx, vm_exit_controls_bits, has_mediated_pmu);
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, INTEL_PMC_MAX_GENERIC) {
 		bool pmc_intercept = intercept || !test_bit(i, &gp_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
@@ -4273,7 +4273,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  pmc_intercept || !fw_writes_is_enabled(vcpu));
 	}
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+	for_each_set_bit(i, kvm_host_pmu.fixed_cntr_mask, INTEL_PMC_MAX_FIXED) {
 		bool pmc_intercept = intercept || !test_bit(i, &fixed_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
Signature · 1 line
2.56.1
#13 lore raw

Re: [PATCH v4 09/21] KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities

Quoted 247 lines · Zide Chen

From: Dapeng Mi <[email protected]>

Intel platforms support non-contiguous fixed counters via CPUID.0AH:ECX starting with PerfMon v5, and support non-contiguous GP counters through the Architectural PerfMon Extension (CPUID leaf 23H).

struct x86_pmu_capability now exposes {,fixed_}cntr_mask64 bitmaps, which may contain sparse bits representing non-contiguous counters. Switch KVM's kvm_host_pmu and kvm_pmu_cap consumers over to the new bitmask fields, including iterating kvm_host_pmu bitmasks directly in VMX and SVM PMU MSR intercept setup.

CPUID.0AH:EAX[15:8] and CPUID.0AH:EDX[4:0] enumerate only contiguous counters. Derive these values from kvm_pmu_cap.{,fixed_}cntr_mask64 as the number of consecutive counters starting at index 0.

---
v4:
- Cleanly substitute kvm_pmu_cap.num_counters_{gp,fixed} with find_first_zero_bit() in intel_pmu_refresh() without throwaway code churn.
- Use find_first_zero_bit() instead of hweight64() for AMD CPUID 0x80000022 EBX[3:0] (NumCorePmc) and svm_set_cpu_caps() to enumerate contiguous counters. (Yosry)
- Do not carry over Jim's Reviewed-by from v3 due to reworking counter clamping in intel_pmu_refresh().

v3:
- Iterate kvm_host_pmu bitmasks directly in VMX and SVM intercept loops.
- Clamp nr_{gp,fixed}_counters to the number of contiguous counters supported by KVM in intel_pmu_refresh(). (Jim, Sashiko)
- Drop redundant !enable_mediated_pmu clamp on fixed_cntr_mask64. (Sashiko)

---
 arch/x86/kvm/cpuid.c         | 17 ++++++++++++++---
 arch/x86/kvm/msrs.c          | 12 ++++++------
 arch/x86/kvm/pmu.c           | 17 ++++++++---------
 arch/x86/kvm/svm/pmu.c       |  2 +-
 arch/x86/kvm/svm/svm.c       |  8 ++++----
 arch/x86/kvm/vmx/pmu_intel.c | 18 +++++++++++++++---
 arch/x86/kvm/vmx/vmx.c       |  4 ++--
 7 files changed, 50 insertions(+), 28 deletions(-)
arch/x86/kvm/cpuid.c +14 -3
index adafd6fa2c54..427871ec365f 100644
@@ -1535,10 +1535,18 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 		}
 
 		eax.split.version_id = kvm_pmu_cap.version;
-		eax.split.num_counters = kvm_pmu_cap.num_counters_gp;
+
+		/* Contiguous GP counters only. */
+		eax.split.num_counters =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		eax.split.bit_width = kvm_pmu_cap.bit_width_gp;
 		eax.split.mask_length = kvm_pmu_cap.events_mask_len;
-		edx.split.num_counters_fixed = kvm_pmu_cap.num_counters_fixed;
+
+		/* Contiguous fixed counters only. */
+		edx.split.num_counters_fixed =
+			find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+					    KVM_MAX_NR_FIXED_COUNTERS);
 		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
 
 		if (kvm_pmu_cap.version)
@@ -1903,7 +1911,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 
 		cpuid_entry_override(entry, CPUID_8000_0022_EAX);
 
-		ebx.split.num_core_pmc = kvm_pmu_cap.num_counters_gp;
+		/* Contiguous GP counters only. */
+		ebx.split.num_core_pmc =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		entry->ebx = ebx.full;
 		break;
 	}
arch/x86/kvm/msrs.c +6 -6
index e91650af44c2..01fb130a7d24 100644
@@ -2621,20 +2621,20 @@ static void kvm_probe_msr_to_save(u32 msr_index)
 		break;
 	case MSR_ARCH_PERFMON_PERFCTR0 ...
 	     MSR_ARCH_PERFMON_PERFCTR0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_PERFCTR0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_PERFCTR0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_EVENTSEL0 ...
 	     MSR_ARCH_PERFMON_EVENTSEL0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_EVENTSEL0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_EVENTSEL0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_FIXED_CTR0 ...
 	     MSR_ARCH_PERFMON_FIXED_CTR0 + KVM_MAX_NR_FIXED_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_FIXED_CTR0 >=
-		    kvm_pmu_cap.num_counters_fixed)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_FIXED_CTR0) &
+		      kvm_pmu_cap.fixed_cntr_mask64))
 			return;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
arch/x86/kvm/pmu.c +8 -9
index d203cbc7fd12..e1999a6c85d7 100644
@@ -20,7 +20,6 @@
 #include <asm/perf_event.h>
 #include <asm/cpu_device_id.h>
 #include "x86.h"
-#include "cpuid.h"
 #include "lapic.h"
 #include "pmu.h"
 
@@ -156,8 +155,8 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 		 * there are a non-zero number of counters, but fewer than what
 		 * is architecturally required.
 		 */
-		if (!kvm_host_pmu.num_counters_gp ||
-		    WARN_ON_ONCE(kvm_host_pmu.num_counters_gp < min_nr_gp_ctrs))
+		if (!kvm_host_pmu.cntr_mask64 ||
+		    WARN_ON_ONCE(hweight64(kvm_host_pmu.cntr_mask64) < min_nr_gp_ctrs))
 			enable_pmu = false;
 		else if (is_intel && !kvm_host_pmu.version)
 			enable_pmu = false;
@@ -177,10 +176,10 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 
 	memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu));
 	kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
-	kvm_pmu_cap.num_counters_gp = min(kvm_pmu_cap.num_counters_gp,
-					  pmu_ops->MAX_NR_GP_COUNTERS);
-	kvm_pmu_cap.num_counters_fixed = min(kvm_pmu_cap.num_counters_fixed,
-					     KVM_MAX_NR_FIXED_COUNTERS);
+	kvm_pmu_cap.cntr_mask64 &=
+		GENMASK_ULL(pmu_ops->MAX_NR_GP_COUNTERS - 1, 0);
+	kvm_pmu_cap.fixed_cntr_mask64 &=
+		GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
 
 	kvm_pmu_eventsel.INSTRUCTIONS_RETIRED =
 		perf_get_hw_event_config(PERF_COUNT_HW_INSTRUCTIONS);
@@ -788,8 +787,8 @@ static bool kvm_need_any_pmc_intercept(struct kvm_vcpu *vcpu)
 	 * KVM's capabilities are constrained based on KVM support, i.e. KVM's
 	 * capabilities themselves may be a subset of hardware capabilities.
 	 */
-	return kvm_gp_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_gp) - 1 ||
-	       kvm_fixed_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_fixed) - 1;
+	return kvm_gp_pmc_mask(pmu) != kvm_host_pmu.cntr_mask64 ||
+	       kvm_fixed_pmc_mask(pmu) != kvm_host_pmu.fixed_cntr_mask64;
 }
 
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu)
arch/x86/kvm/svm/pmu.c +1 -1
index 64215024e70e..5eedf9b35fce 100644
@@ -196,7 +196,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
-			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
+			    kvm_pmu_cap.cntr_mask64;
 
 	if (pmu->version > 1) {
 		pmu->global_ctrl_rsvd = ~pmu->pmc_exists64;
arch/x86/kvm/svm/svm.c +4 -4
index e21abcedea70..5b9c9d149486 100644
@@ -773,7 +773,7 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept |= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++)
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, AMD64_NUM_COUNTERS_CORE)
 		svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
@@ -5719,9 +5719,9 @@ static __init void svm_set_cpu_caps(void)
 		 * access to enough counters to virtualize "core" support,
 		 * otherwise limit vPMU support to the legacy number of counters.
 		 */
-		if (kvm_pmu_cap.num_counters_gp < AMD64_NUM_COUNTERS_CORE)
-			kvm_pmu_cap.num_counters_gp = min(AMD64_NUM_COUNTERS,
-							  kvm_pmu_cap.num_counters_gp);
+		if (find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					KVM_MAX_NR_GP_COUNTERS) < AMD64_NUM_COUNTERS_CORE)
+			kvm_pmu_cap.cntr_mask64 &= GENMASK_ULL(AMD64_NUM_COUNTERS - 1, 0);
 		else
 			kvm_cpu_cap_check_and_set(X86_FEATURE_PERFCTR_CORE);
 
arch/x86/kvm/vmx/pmu_intel.c +15 -3
index 68d23c7877a2..a240b74ac704 100644
@@ -493,7 +493,8 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 	 * have a known encoding for the associated general purpose event.
 	 */
 	eventsel = perf_get_hw_event_config(fixed_pmc_perf_ids[index]);
-	WARN_ON_ONCE(!eventsel && index < kvm_pmu_cap.num_counters_fixed);
+	WARN_ON_ONCE(!eventsel &&
+		     (kvm_pmu_cap.fixed_cntr_mask64 & BIT_ULL(index)));
 	return eventsel;
 }
 
@@ -548,12 +549,23 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				      kvm_pmu_cap.events_mask_len);
 	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
 
+	/*
+	 * CPUID.0AH:EDX[4:0] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous fixed counters supported by KVM.
+	 */
 	nr_fixed_counters = min_t(int, edx.split.num_counters_fixed,
-				  kvm_pmu_cap.num_counters_fixed);
+				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+						      KVM_MAX_NR_FIXED_COUNTERS));
+
 	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
 
+	/*
+	 * CPUID.0AH:EAX[15:8] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous GP counters supported by KVM.
+	 */
 	nr_gp_counters = min_t(int, eax.split.num_counters,
-			       kvm_pmu_cap.num_counters_gp);
+			       find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+						   KVM_MAX_NR_GP_COUNTERS));
 	pmu->pmc_exists64 = BIT_ULL(nr_gp_counters) - 1;
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
arch/x86/kvm/vmx/vmx.c +0 -0
index 944ecbf351cf..9db6107943ec 100644

arch/x86/kvm/vmx/vmx.c static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu) · line 4267

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -4264,7 +4264,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 
 	vm_exit_controls_changebit(vmx, vm_exit_controls_bits, has_mediated_pmu);
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, INTEL_PMC_MAX_GENERIC) {

Nit: This seems a long standing bug, kvm_host_pmu instead of kvm_pmu_cap should be used here. Better put a stand-alone patch to fix this issue at the header of this patch series if possible.

Thanks.

Quoted 12 lines
arch/x86/kvm/vmx/vmx.c +1 -1
 		bool pmc_intercept = intercept || !test_bit(i, &gp_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
@@ -4273,7 +4273,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  pmc_intercept || !fw_writes_is_enabled(vcpu));
 	}
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+	for_each_set_bit(i, kvm_host_pmu.fixed_cntr_mask, INTEL_PMC_MAX_FIXED) {
 		bool pmc_intercept = intercept || !test_bit(i, &fixed_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
author
#14 lore raw

Re: [PATCH v4 09/21] KVM: x86/pmu: Switch to bitmask-based KVM PMU capabilities

Quoted 250 lines · Dapeng
Quoted 248 lines · Zide Chen

From: Dapeng Mi <[email protected]>

Intel platforms support non-contiguous fixed counters via CPUID.0AH:ECX starting with PerfMon v5, and support non-contiguous GP counters through the Architectural PerfMon Extension (CPUID leaf 23H).

struct x86_pmu_capability now exposes {,fixed_}cntr_mask64 bitmaps, which may contain sparse bits representing non-contiguous counters. Switch KVM's kvm_host_pmu and kvm_pmu_cap consumers over to the new bitmask fields, including iterating kvm_host_pmu bitmasks directly in VMX and SVM PMU MSR intercept setup.

CPUID.0AH:EAX[15:8] and CPUID.0AH:EDX[4:0] enumerate only contiguous counters. Derive these values from kvm_pmu_cap.{,fixed_}cntr_mask64 as the number of consecutive counters starting at index 0.

---
v4:
- Cleanly substitute kvm_pmu_cap.num_counters_{gp,fixed} with find_first_zero_bit() in intel_pmu_refresh() without throwaway code churn.
- Use find_first_zero_bit() instead of hweight64() for AMD CPUID 0x80000022 EBX[3:0] (NumCorePmc) and svm_set_cpu_caps() to enumerate contiguous counters. (Yosry)
- Do not carry over Jim's Reviewed-by from v3 due to reworking counter clamping in intel_pmu_refresh().

v3:
- Iterate kvm_host_pmu bitmasks directly in VMX and SVM intercept loops.
- Clamp nr_{gp,fixed}_counters to the number of contiguous counters supported by KVM in intel_pmu_refresh(). (Jim, Sashiko)
- Drop redundant !enable_mediated_pmu clamp on fixed_cntr_mask64. (Sashiko)

---
 arch/x86/kvm/cpuid.c         | 17 ++++++++++++++---
 arch/x86/kvm/msrs.c          | 12 ++++++------
 arch/x86/kvm/pmu.c           | 17 ++++++++---------
 arch/x86/kvm/svm/pmu.c       |  2 +-
 arch/x86/kvm/svm/svm.c       |  8 ++++----
 arch/x86/kvm/vmx/pmu_intel.c | 18 +++++++++++++++---
 arch/x86/kvm/vmx/vmx.c       |  4 ++--
 7 files changed, 50 insertions(+), 28 deletions(-)
arch/x86/kvm/cpuid.c +14 -3
index adafd6fa2c54..427871ec365f 100644
@@ -1535,10 +1535,18 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 		}
 
 		eax.split.version_id = kvm_pmu_cap.version;
-		eax.split.num_counters = kvm_pmu_cap.num_counters_gp;
+
+		/* Contiguous GP counters only. */
+		eax.split.num_counters =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		eax.split.bit_width = kvm_pmu_cap.bit_width_gp;
 		eax.split.mask_length = kvm_pmu_cap.events_mask_len;
-		edx.split.num_counters_fixed = kvm_pmu_cap.num_counters_fixed;
+
+		/* Contiguous fixed counters only. */
+		edx.split.num_counters_fixed =
+			find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+					    KVM_MAX_NR_FIXED_COUNTERS);
 		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
 
 		if (kvm_pmu_cap.version)
@@ -1903,7 +1911,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 
 		cpuid_entry_override(entry, CPUID_8000_0022_EAX);
 
-		ebx.split.num_core_pmc = kvm_pmu_cap.num_counters_gp;
+		/* Contiguous GP counters only. */
+		ebx.split.num_core_pmc =
+			find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					    KVM_MAX_NR_GP_COUNTERS);
 		entry->ebx = ebx.full;
 		break;
 	}
arch/x86/kvm/msrs.c +6 -6
index e91650af44c2..01fb130a7d24 100644
@@ -2621,20 +2621,20 @@ static void kvm_probe_msr_to_save(u32 msr_index)
 		break;
 	case MSR_ARCH_PERFMON_PERFCTR0 ...
 	     MSR_ARCH_PERFMON_PERFCTR0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_PERFCTR0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_PERFCTR0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_EVENTSEL0 ...
 	     MSR_ARCH_PERFMON_EVENTSEL0 + KVM_MAX_NR_GP_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_EVENTSEL0 >=
-		    kvm_pmu_cap.num_counters_gp)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_EVENTSEL0) &
+		      kvm_pmu_cap.cntr_mask64))
 			return;
 		break;
 	case MSR_ARCH_PERFMON_FIXED_CTR0 ...
 	     MSR_ARCH_PERFMON_FIXED_CTR0 + KVM_MAX_NR_FIXED_COUNTERS - 1:
-		if (msr_index - MSR_ARCH_PERFMON_FIXED_CTR0 >=
-		    kvm_pmu_cap.num_counters_fixed)
+		if (!(BIT_ULL(msr_index - MSR_ARCH_PERFMON_FIXED_CTR0) &
+		      kvm_pmu_cap.fixed_cntr_mask64))
 			return;
 		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_CTL:
arch/x86/kvm/pmu.c +8 -9
index d203cbc7fd12..e1999a6c85d7 100644
@@ -20,7 +20,6 @@
 #include <asm/perf_event.h>
 #include <asm/cpu_device_id.h>
 #include "x86.h"
-#include "cpuid.h"
 #include "lapic.h"
 #include "pmu.h"
 
@@ -156,8 +155,8 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 		 * there are a non-zero number of counters, but fewer than what
 		 * is architecturally required.
 		 */
-		if (!kvm_host_pmu.num_counters_gp ||
-		    WARN_ON_ONCE(kvm_host_pmu.num_counters_gp < min_nr_gp_ctrs))
+		if (!kvm_host_pmu.cntr_mask64 ||
+		    WARN_ON_ONCE(hweight64(kvm_host_pmu.cntr_mask64) < min_nr_gp_ctrs))
 			enable_pmu = false;
 		else if (is_intel && !kvm_host_pmu.version)
 			enable_pmu = false;
@@ -177,10 +176,10 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 
 	memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu));
 	kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
-	kvm_pmu_cap.num_counters_gp = min(kvm_pmu_cap.num_counters_gp,
-					  pmu_ops->MAX_NR_GP_COUNTERS);
-	kvm_pmu_cap.num_counters_fixed = min(kvm_pmu_cap.num_counters_fixed,
-					     KVM_MAX_NR_FIXED_COUNTERS);
+	kvm_pmu_cap.cntr_mask64 &=
+		GENMASK_ULL(pmu_ops->MAX_NR_GP_COUNTERS - 1, 0);
+	kvm_pmu_cap.fixed_cntr_mask64 &=
+		GENMASK_ULL(KVM_MAX_NR_FIXED_COUNTERS - 1, 0);
 
 	kvm_pmu_eventsel.INSTRUCTIONS_RETIRED =
 		perf_get_hw_event_config(PERF_COUNT_HW_INSTRUCTIONS);
@@ -788,8 +787,8 @@ static bool kvm_need_any_pmc_intercept(struct kvm_vcpu *vcpu)
 	 * KVM's capabilities are constrained based on KVM support, i.e. KVM's
 	 * capabilities themselves may be a subset of hardware capabilities.
 	 */
-	return kvm_gp_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_gp) - 1 ||
-	       kvm_fixed_pmc_mask(pmu) != BIT_ULL(kvm_host_pmu.num_counters_fixed) - 1;
+	return kvm_gp_pmc_mask(pmu) != kvm_host_pmu.cntr_mask64 ||
+	       kvm_fixed_pmc_mask(pmu) != kvm_host_pmu.fixed_cntr_mask64;
 }
 
 bool kvm_need_perf_global_ctrl_intercept(struct kvm_vcpu *vcpu)
arch/x86/kvm/svm/pmu.c +1 -1
index 64215024e70e..5eedf9b35fce 100644
@@ -196,7 +196,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	pmu->pmc_exists64 = (BIT_ULL(nr_gp_counters) - 1) &
-			    (BIT_ULL(kvm_pmu_cap.num_counters_gp) - 1);
+			    kvm_pmu_cap.cntr_mask64;
 
 	if (pmu->version > 1) {
 		pmu->global_ctrl_rsvd = ~pmu->pmc_exists64;
arch/x86/kvm/svm/svm.c +4 -4
index e21abcedea70..5b9c9d149486 100644
@@ -773,7 +773,7 @@ static void svm_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
 	intercept |= !guest_cpu_cap_has(vcpu, X86_FEATURE_PERFCTR_CORE);
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++)
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, AMD64_NUM_COUNTERS_CORE)
 		svm_set_intercept_for_msr(vcpu, MSR_F15H_PERF_CTR + 2 * i,
 					  MSR_TYPE_RW, intercept || !test_bit(i, &gp_mask));
 
@@ -5719,9 +5719,9 @@ static __init void svm_set_cpu_caps(void)
 		 * access to enough counters to virtualize "core" support,
 		 * otherwise limit vPMU support to the legacy number of counters.
 		 */
-		if (kvm_pmu_cap.num_counters_gp < AMD64_NUM_COUNTERS_CORE)
-			kvm_pmu_cap.num_counters_gp = min(AMD64_NUM_COUNTERS,
-							  kvm_pmu_cap.num_counters_gp);
+		if (find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+					KVM_MAX_NR_GP_COUNTERS) < AMD64_NUM_COUNTERS_CORE)
+			kvm_pmu_cap.cntr_mask64 &= GENMASK_ULL(AMD64_NUM_COUNTERS - 1, 0);
 		else
 			kvm_cpu_cap_check_and_set(X86_FEATURE_PERFCTR_CORE);
 
arch/x86/kvm/vmx/pmu_intel.c +15 -3
index 68d23c7877a2..a240b74ac704 100644
@@ -493,7 +493,8 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 	 * have a known encoding for the associated general purpose event.
 	 */
 	eventsel = perf_get_hw_event_config(fixed_pmc_perf_ids[index]);
-	WARN_ON_ONCE(!eventsel && index < kvm_pmu_cap.num_counters_fixed);
+	WARN_ON_ONCE(!eventsel &&
+		     (kvm_pmu_cap.fixed_cntr_mask64 & BIT_ULL(index)));
 	return eventsel;
 }
 
@@ -548,12 +549,23 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				      kvm_pmu_cap.events_mask_len);
 	pmu->available_event_types = ~entry->ebx & (BIT_ULL(eax.split.mask_length) - 1);
 
+	/*
+	 * CPUID.0AH:EDX[4:0] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous fixed counters supported by KVM.
+	 */
 	nr_fixed_counters = min_t(int, edx.split.num_counters_fixed,
-				  kvm_pmu_cap.num_counters_fixed);
+				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
+						      KVM_MAX_NR_FIXED_COUNTERS));
+
 	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
 
+	/*
+	 * CPUID.0AH:EAX[15:8] can only enumerate contiguous counters. Clamp the
+	 * value to the number of contiguous GP counters supported by KVM.
+	 */
 	nr_gp_counters = min_t(int, eax.split.num_counters,
-			       kvm_pmu_cap.num_counters_gp);
+			       find_first_zero_bit(kvm_pmu_cap.cntr_mask,
+						   KVM_MAX_NR_GP_COUNTERS));
 	pmu->pmc_exists64 = BIT_ULL(nr_gp_counters) - 1;
 
 	entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
arch/x86/kvm/vmx/vmx.c +0 -0
index 944ecbf351cf..9db6107943ec 100644
@@ -4264,7 +4264,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu) · line 4267

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1 -1
 	vm_exit_controls_changebit(vmx, vm_exit_controls_bits, has_mediated_pmu);
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+	for_each_set_bit(i, kvm_host_pmu.cntr_mask, INTEL_PMC_MAX_GENERIC) {

Nit: This seems a long standing bug, kvm_host_pmu instead of kvm_pmu_cap should be used here. Better put a stand-alone patch to fix this issue at the header of this patch series if possible.

Actually, it seems to me using kvm_pmu_cap here is not a bug in upstream. MSR bitmaps by default are all-intercepted, and guest PMCs are clamped to kvm_pmu_cap, so counters beyond kvm_pmu_cap can never be exposed to the guest.

Looking closer at the history, exposing kvm_host_pmu to vendor code in patch 6 was originally added in v2 when vmx.c had two loops and needed it to compute "host_only_gp_mask = kvm_host_pmu.cntr_mask64 & ~gp_mask".

In v3, patch 5 consolidated VMX and SVM into single loops, which eliminated the host_only mask calculation. Using kvm_host_pmu is intuitively "more" correct, but not a must.

kvm_pmu_cap.cntr_mask can be used in this patch:

- for_each_set_bit(i, kvm_host_pmu.cntr_mask, INTEL_PMC_MAX_GENERIC) {

+  for_each_set_bit(i, kvm_pmu_cap.cntr_mask, INTEL_PMC_MAX_GENERIC) {
       bool pmc_intercept = intercept || !test_bit(i, &gp_mask);

       vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
                          MSR_TYPE_RW, pmc_intercept);
       vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW,
                          pmc_intercept || !fw_writes_is_enabled(vcpu));

}

Thus, patch 6 ("KVM: x86/pmu: Expose kvm_host_pmu to vendor modules") could be dropped, keeping kvm_host_pmu encapsulated as static to pmu.c.

Quoted 16 lines

Thanks.

Quoted 12 lines
arch/x86/kvm/vmx/vmx.c +1 -1
 		bool pmc_intercept = intercept || !test_bit(i, &gp_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i,
@@ -4273,7 +4273,7 @@ static void vmx_recalc_pmu_msr_intercepts(struct kvm_vcpu *vcpu)
 					  pmc_intercept || !fw_writes_is_enabled(vcpu));
 	}
 
-	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+	for_each_set_bit(i, kvm_host_pmu.fixed_cntr_mask, INTEL_PMC_MAX_FIXED) {
 		bool pmc_intercept = intercept || !test_bit(i, &fixed_mask);
 
 		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i,
author
#15 lore raw

PATCH v4 10/21 perf/x86: Remove num_counters_{gp,fixed} from x86_pmu_capability

In reply to Zide Chen

From: Dapeng Mi <[email protected]>

Now that KVM has switched to bitmap-based PMU capabilities, num_counters_{gp,fixed} can be removed from x86_pmu_capability.

---
 arch/x86/events/core.c            | 2 --
 arch/x86/include/asm/perf_event.h | 2 --
 2 files changed, 4 deletions(-)
arch/x86/events/core.c +0 -2
index a9b86cecbc78..990d3be5a07f 100644
@@ -3156,8 +3156,6 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	 * base PMU holds the correct number of counters for P-cores.
 	 */
 	cap->version		= x86_pmu.version;
-	cap->num_counters_gp	= x86_pmu_num_counters(NULL);
-	cap->num_counters_fixed	= x86_pmu_num_counters_fixed(NULL);
 	cap->cntr_mask64	= x86_pmu.cntr_mask64;
 	cap->fixed_cntr_mask64	= x86_pmu.fixed_cntr_mask64;
 	cap->bit_width_gp	= cap->cntr_mask64 ? x86_pmu.cntval_bits : 0;
arch/x86/include/asm/perf_event.h +0 -2
index 190fda412a61..17b0bc7dfce7 100644
@@ -300,8 +300,6 @@ union cpuid_0x80000022_ebx {
 
 struct x86_pmu_capability {
 	int		version;
-	int		num_counters_gp;
-	int		num_counters_fixed;
 	union {
 		u64	cntr_mask64;
 		DECLARE_BITMAP(cntr_mask, X86_PMC_IDX_MAX);
Signature · 1 line
2.56.1
author
#16 lore raw

PATCH v4 11/21 KVM: x86/pmu: Document GLOBAL_STATUS_RESET passthrough erratum

In reply to Zide Chen

IA32_PERF_GLOBAL_STATUS_RESET, introduced in PerfMon version 4, inherits the MSR address and architecturally defined bits from IA32_PERF_GLOBAL_OVF_CTRL, while providing additional bits to clear new indicators in IA32_PERF_GLOBAL_STATUS.

From an emulation perspective, KVM handles it identically to the legacy OVF_CTRL MSR: reads return zero, and intercepted writes are guarded against pmu->global_status_rsvd. No code changes are needed in KVM to emulate this MSR.

However, to reduce VM-exits on performance-critical paths, KVM passes through this MSR in mediated PMU mode when all hardware counters are exposed to the guest. Consequently, writes to bits that are valid in hardware but unvirtualized by KVM will not trigger #GP (unlike when intercepted). This is safe because such unvirtualized bits are never set while the guest runs, so clearing them has no effect, and because the MSR is read-as-zero and write-to-clear, passing it through cannot leak host state.

Document this divergence in errata.rst.

---
v4:
- Clarify in errata.rst that passthrough occurs when all hardware counters are exposed to the guest, and that guest writes cannot clobber host state. (Yosry)

v3:
- New patch. (Jim)

---
 Documentation/virt/kvm/x86/errata.rst | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
Documentation/virt/kvm/x86/errata.rst +9 -1
index a9cf0e004651..70fd8a638ef7 100644
@@ -84,4 +84,12 @@ memory.
 
 Note, this erratum does not affect AMD CPUs, which fully virtualize CR0.CD in
 hardware, i.e. put the CPU caches into "no fill" mode when CR0.CD=1, even when
-running in the guest.
\ No newline at end of file
+running in the guest.
+
+IA32_PERF_GLOBAL_STATUS_RESET
+-----------------------------
+To reduce VM-exits on performance-critical paths, KVM passes through writes
+to IA32_PERF_GLOBAL_STATUS_RESET (0x390) in mediated PMU mode when all
+hardware counters are exposed to the guest, without injecting #GP on
+unvirtualized bits.  Because the MSR is read-as-zero and write-to-clear, this
+cannot leak host information or clobber host state.
Signature · 1 line
2.56.1
author
#17 lore raw

PATCH v4 12/21 KVM: x86/pmu: Mark CondChgd reserved in global_status_rsvd

In reply to Zide Chen

Bit 63 (CondChgd) of IA32_PERF_GLOBAL_STATUS indicates a change to IA32_MISC_ENABLE[7], which KVM intercepts and keeps read-only for guests.

In the Intel SDM, there is an asymmetry regarding bit 63: it is valid in IA32_PERF_GLOBAL_STATUS_{,RESET}, but it is architecturally reserved in IA32_PERF_GLOBAL_STATUS_SET. Consequently, KVM cannot restore bit 63 to hardware on vPMU context switch.

Fortunately, because KVM intercepts IA32_MISC_ENABLE and keeps it read-only for guests, neither hardware nor KVM will ever assert CondChgd for the guest. It is therefore safe to not virtualize CondChgd while maintaining architectural consistency.

Mark CondChgd reserved in pmu->global_status_rsvd so that userspace writes to IA32_PERF_GLOBAL_STATUS, and upcoming emulation of IA32_PERF_GLOBAL_STATUS_SET, reject bit 63.

Clearing CondChgd via IA32_PERF_GLOBAL_STATUS_RESET (0x390, a.k.a. GLOBAL_OVF_CTRL) is architecturally valid per the Intel SDM, so explicitly allow bit 63 to be cleared without #GP.

Additionally, in mediated PMU mode, defensively mask out reserved bits in intel_mediated_pmu_load() before writing to IA32_PERF_GLOBAL_STATUS_SET to prevent a host #GP.

---
v4:
- New prep patch to mark CondChgd reserved in global_status_rsvd (suggested by Yosry), addressing the STATUS_SET bit 63 asymmetry identified by Jim. (Jim, Yosry)

---
 arch/x86/kvm/pmu.c           |  8 +++++---
 arch/x86/kvm/vmx/pmu_intel.c | 11 +++++++----
 2 files changed, 12 insertions(+), 7 deletions(-)
arch/x86/kvm/pmu.c +5 -3
index e1999a6c85d7..7ba1a3b38184 100644
@@ -911,10 +911,12 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
 		/*
-		 * GLOBAL_OVF_CTRL, a.k.a. GLOBAL STATUS_RESET, clears bits in
-		 * GLOBAL_STATUS, and so the set of reserved bits is the same.
+		 * GLOBAL_OVF_CTRL, a.k.a. GLOBAL_STATUS_RESET, clears bits in
+		 * GLOBAL_STATUS.  CondChgd is not virtualized by KVM and is
+		 * reserved in global_status_rsvd, but clearing it via
+		 * GLOBAL_OVF_CTRL is architecturally valid per the SDM.
 		 */
-		if (data & pmu->global_status_rsvd)
+		if (data & (pmu->global_status_rsvd & ~GLOBAL_STATUS_COND_CHG))
 			return 1;
 		fallthrough;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
arch/x86/kvm/vmx/pmu_intel.c +7 -4
index a240b74ac704..4d05ec50f24f 100644
@@ -603,7 +603,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->global_ctrl_rsvd = counter_rsvd;
 
 	pmu->global_status_rsvd = pmu->global_ctrl_rsvd
-			& ~(GLOBAL_STATUS_BUFFER_OVF | GLOBAL_STATUS_COND_CHG);
+			& ~GLOBAL_STATUS_BUFFER_OVF;
 	if (vmx_pt_mode_is_host_guest())
 		pmu->global_status_rsvd &= ~GLOBAL_STATUS_TRACE_TOPAPMI;
 
@@ -818,14 +818,17 @@ static void intel_pmu_write_global_ctrl(u64 global_ctrl)
 static void intel_mediated_pmu_load(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
-	u64 global_status, toggle;
+	u64 global_status, toggle, set_bits;
 
 	rdmsrq(MSR_CORE_PERF_GLOBAL_STATUS, global_status);
 	toggle = pmu->global_status ^ global_status;
 	if (global_status & toggle)
 		wrmsrq(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle);
-	if (pmu->global_status & toggle)
-		wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
+
+	/* Defensively mask reserved bits to prevent a host #GP. */
+	set_bits = (pmu->global_status & toggle) & ~pmu->global_status_rsvd;
+	if (set_bits)
+		wrmsrq(MSR_CORE_PERF_GLOBAL_STATUS_SET, set_bits);
 
 	wrmsrq(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
 }
Signature · 1 line
2.56.1
author
#18 lore raw

PATCH v4 13/21 KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_STATUS_SET MSR

In reply to Zide Chen

Intel PerfMon v4 introduces IA32_PERF_GLOBAL_STATUS_SET (0x391) to allow software to set individual bits in the global status MSR. Reads of IA32_PERF_GLOBAL_STATUS_SET always return zero.

IA32_PERF_GLOBAL_STATUS_SET is not part of the VM state, so it doesn't need to be advertised to userspace, nor saved and restored during live migration.

Always intercept IA32_PERF_GLOBAL_STATUS_SET to prevent the guest from modifying status bits that are not virtualized by KVM.

Bit 63 (CondChgd) is architecturally reserved in STATUS_SET per the SDM. Reject writes to bit 63, injecting #GP for guest writes. While older processors had silicon errata that accepted writes without #GP (e.g. SKX95), strictly enforce the architectural specification for deterministic behavior across migrations.

---
v4:
- Split out CondChgd specific handling into a separate patch since it impacts MSR 0x38e and 0x390 as well.
- Move the defensive filtering code in pmu load to the new patch.
- Do not carry over Jim's Reviewed-by due to above changes.

v3:
- Split from combined v2 patch into standalone GLOBAL_STATUS_SET patch.
- Always intercept IA32_PERF_GLOBAL_STATUS_SET to prevent passthrough of unvirtualized status bits. (Jim)
- Reject writing bit 63 (CondChgd) to IA32_PERF_GLOBAL_STATUS_SET. (Jim)
- Filter out bit 63 before writing to hardware in intel_mediated_pmu_load().
- Store status set bits in a local variable in intel_mediated_pmu_load(). (Yosry)
- Move validation for MSR_CORE_PERF_GLOBAL_STATUS_SET to pmu_intel.c rather than common pmu.c to avoid exposing the Intel MSR on AMD.

---
 arch/x86/kvm/pmu.c           | 7 +++++++
 arch/x86/kvm/vmx/pmu_intel.c | 2 ++
 2 files changed, 9 insertions(+)
arch/x86/kvm/pmu.c +7 -0
index 7ba1a3b38184..bd5954e62080 100644
@@ -855,6 +855,7 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
+	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
 		msr_info->data = 0;
 		break;
 	default:
@@ -923,6 +924,12 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (!msr_info->host_initiated)
 			pmu->global_status &= ~data;
 		break;
+	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
+		if (data & pmu->global_status_rsvd)
+			return 1;
+		if (!msr_info->host_initiated)
+			pmu->global_status |= data;
+		break;
 	case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET:
 		if (!msr_info->host_initiated)
 			pmu->global_status |= data & ~pmu->global_status_rsvd;
arch/x86/kvm/vmx/pmu_intel.c +2 -0
index 4d05ec50f24f..c18d24b5c385 100644
@@ -196,6 +196,8 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 		return kvm_pmu_has_perf_global_ctrl(pmu);
+	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
+		return pmu->version >= 4;
 	case MSR_IA32_PEBS_ENABLE:
 		ret = vcpu_get_perf_capabilities(vcpu) & PERF_CAP_PEBS_FORMAT;
 		break;
Signature · 1 line
2.56.1
author
#19 lore raw

PATCH v4 14/21 KVM: x86/pmu: Emulate IA32_PERF_GLOBAL_INUSE MSR

In reply to Zide Chen

Intel PerfMon v4 introduces IA32_PERF_GLOBAL_INUSE (0x392) to track which counters and the PMI are currently claimed by software agents, allowing independent software agents to check counter availability without a shared scheduler arbitrating between them.

IA32_PERF_GLOBAL_INUSE is a read-only MSR, and any write attempt results in a #GP. The MSR is not part of the VM state, so it does not need to be advertised to userspace, nor saved and restored during live migration.

Always intercept IA32_PERF_GLOBAL_INUSE to prevent the guest from erroneously observing filtered fixed counters as idle.

---
v4:
- Pass 'pmu' directly to kvm_for_each_{gp,fixed}_counter().

v3:
- Split from combined v2 patch into standalone GLOBAL_INUSE patch.
- Always intercept MSR_CORE_PERF_GLOBAL_INUSE. (Jim)
- Use '!!' for explicit boolean conversion. (Jim)
- Relocate PERF_GLOBAL_INUSE_PMI_INUSE to perf_event.h.

---
 arch/x86/include/asm/msr-index.h  |  1 +
 arch/x86/include/asm/perf_event.h |  3 +++
 arch/x86/kvm/vmx/pmu_intel.c      | 34 +++++++++++++++++++++++++++++++
 3 files changed, 38 insertions(+)
arch/x86/include/asm/msr-index.h +1 -0
index 30cd04c0ace9..25dda50fbdc1 100644
@@ -1242,6 +1242,7 @@
 #define MSR_CORE_PERF_GLOBAL_CTRL	0x0000038f
 #define MSR_CORE_PERF_GLOBAL_OVF_CTRL	0x00000390
 #define MSR_CORE_PERF_GLOBAL_STATUS_SET	0x00000391
+#define MSR_CORE_PERF_GLOBAL_INUSE	0x00000392
 
 #define MSR_PERF_METRICS		0x00000329
 
arch/x86/include/asm/perf_event.h +3 -0
index 17b0bc7dfce7..2937e8a27891 100644
@@ -454,6 +454,9 @@ static inline bool is_topdown_idx(int idx)
 #define GLOBAL_STATUS_ARCH_PEBS_THRESHOLD	BIT_ULL(GLOBAL_STATUS_ARCH_PEBS_THRESHOLD_BIT)
 #define GLOBAL_STATUS_PERF_METRICS_OVF_BIT	48
 
+/* Intel IA32_PERF_GLOBAL_INUSE MSR */
+#define PERF_GLOBAL_INUSE_PMI_INUSE	BIT_ULL(63)
+
 #define GLOBAL_CTRL_EN_PERF_METRICS		BIT_ULL(48)
 /*
  * We model guest LBR event tracing as another fixed-mode PMC like BTS.
arch/x86/kvm/vmx/pmu_intel.c +34 -0
index c18d24b5c385..5eac64ba8832 100644
@@ -197,6 +197,7 @@ static bool intel_is_valid_msr(struct kvm_vcpu *vcpu, u32 msr)
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 		return kvm_pmu_has_perf_global_ctrl(pmu);
 	case MSR_CORE_PERF_GLOBAL_STATUS_SET:
+	case MSR_CORE_PERF_GLOBAL_INUSE:
 		return pmu->version >= 4;
 	case MSR_IA32_PEBS_ENABLE:
 		ret = vcpu_get_perf_capabilities(vcpu) & PERF_CAP_PEBS_FORMAT;
@@ -345,6 +346,36 @@ static bool intel_pmu_handle_lbr_msrs_access(struct kvm_vcpu *vcpu,
 	return true;
 }
 
+static u64 intel_pmu_get_global_inuse(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	bool pmi_inuse = false;
+	u64 eventsel, data = 0;
+	u32 fixed_ctrl;
+	int i;
+
+	kvm_for_each_gp_counter(i, pmu) {
+		eventsel = pmu->gp_counters[i].eventsel;
+
+		if (eventsel & ARCH_PERFMON_EVENTSEL_EVENT)
+			data |= BIT_ULL(i);
+		pmi_inuse |= !!(eventsel & ARCH_PERFMON_EVENTSEL_INT);
+	}
+	kvm_for_each_fixed_counter(i, pmu) {
+		fixed_ctrl = fixed_ctrl_field(pmu->fixed_ctr_ctrl, i);
+
+		if (fixed_ctrl & (INTEL_FIXED_0_KERNEL | INTEL_FIXED_0_USER))
+			data |= BIT_ULL(KVM_FIXED_PMC_BASE_IDX + i);
+		pmi_inuse |= !!(fixed_ctrl & INTEL_FIXED_0_ENABLE_PMI);
+	}
+	pmi_inuse |= pmu->pebs_enable;
+
+	if (pmi_inuse)
+		data |= PERF_GLOBAL_INUSE_PMI_INUSE;
+
+	return data;
+}
+
 static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
@@ -355,6 +386,9 @@ static int intel_pmu_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 		msr_info->data = pmu->fixed_ctr_ctrl;
 		break;
+	case MSR_CORE_PERF_GLOBAL_INUSE:
+		msr_info->data = intel_pmu_get_global_inuse(vcpu);
+		break;
 	case MSR_IA32_PEBS_ENABLE:
 		msr_info->data = pmu->pebs_enable;
 		break;
Signature · 1 line
2.56.1
author
#20 lore raw

PATCH v4 15/21 KVM: x86/pmu: Clamp pmu->version to kvm_pmu_cap.version

In reply to Zide Chen

KVM's policy of not policing guest CPUID allows userspace to configure an arbitrary PMU version without KVM rejecting the ioctl or mutating guest CPUID entries. As userspace is responsible for validating its configuration against KVM_GET_SUPPORTED_CPUID, userspace is entitled to "shoot itself in the foot" by presenting an unsupported virtual CPU model.

Clamp pmu->version to kvm_pmu_cap.version so that KVM's internal state tracks the effective PMU version that KVM supports, and avoid exposing or emulating features that KVM does not support (such as MSRs gated on higher PMU versions).

---
v4:
- New patch. (Sean, Yosry)

---
 arch/x86/kvm/vmx/pmu_intel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/vmx/pmu_intel.c +1 -1
index 5eac64ba8832..0be9956810ae 100644
@@ -574,7 +574,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	eax.full = entry->eax;
 	edx.full = entry->edx;
 
-	pmu->version = eax.split.version_id;
+	pmu->version = min_t(int, eax.split.version_id, kvm_pmu_cap.version);
 	if (!pmu->version)
 		return;
 
Signature · 1 line
2.56.1
author
#21 lore raw

PATCH v4 16/21 KVM: x86/pmu: Populate CPUID.0AH:ECX fixed-counter bitmap

In reply to Zide Chen

PerfMon v5 adds a fixed-counter support bitmap in CPUID.0AH:ECX. This is a superset of EDX[4:0] which indicates the contiguous counters starting from 0. For backward compatibility, it's recommended to use the following to determine if a fixed counter is supported:

FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i);

Populate CPUID.0AH:ECX with kvm_pmu_cap.fixed_cntr_mask64 when KVM supports PerfMon v5, and in intel_pmu_refresh(), incorporate the guest's ECX bits into fixed_mask for PerfMon v5+ guests.

---
v4:
- Check pmu->version directly instead of checking both kvm_pmu_cap and pmu version, as pmu->version is now clamped. (Yosry, Sean)
- Merge the blanket masking into the fixed_mask assignment. (Yosry)

v3:
- Remove '&' when populating entry->ecx. (Jim)

---
 arch/x86/kvm/cpuid.c         | 5 ++++-
 arch/x86/kvm/vmx/pmu_intel.c | 3 +++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/x86/kvm/cpuid.c +4 -1
index 427871ec365f..2a4530d000fc 100644
@@ -1554,7 +1554,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 
 		entry->eax = eax.full;
 		entry->ebx = kvm_pmu_cap.events_mask;
-		entry->ecx = 0;
+		if (kvm_pmu_cap.version >= 5)
+			entry->ecx = (u32)kvm_pmu_cap.fixed_cntr_mask64;
+		else
+			entry->ecx = 0;
 		entry->edx = edx.full;
 		break;
 	}
arch/x86/kvm/vmx/pmu_intel.c +3 -0
index 0be9956810ae..d259b66abac8 100644
@@ -593,7 +593,10 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				  find_first_zero_bit(kvm_pmu_cap.fixed_cntr_mask,
 						      KVM_MAX_NR_FIXED_COUNTERS));
 
+	/* FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i) */
 	fixed_mask = BIT_ULL(nr_fixed_counters) - 1;
+	if (pmu->version >= 5)
+		fixed_mask |= (entry->ecx & kvm_pmu_cap.fixed_cntr_mask64);
 
 	/*
 	 * CPUID.0AH:EAX[15:8] can only enumerate contiguous counters. Clamp the
Signature · 1 line
2.56.1
author
#22 lore raw

PATCH v4 17/21 KVM: x86/pmu: Factor out fixed counter control bit calculation

In reply to Zide Chen

Rename intel_pmu_enable_fixed_counter_bits() to intel_pmu_fixed_ctrl_bits() and have it return the computed fixed counter control mask instead of updating fixed_ctr_ctrl_rsvd directly.

Accumulate the desired fixed counter control bits during PMU setup and apply them once in intel_pmu_refresh().

Move the helper earlier in the file so it can also be reused for upcoming ANYTHREAD_DEPRECATION emulation.

No functional change intended.

---
v4:
- Pass 'pmu' directly to kvm_for_each_fixed_counter().

---
 arch/x86/kvm/vmx/pmu_intel.c | 30 ++++++++++++++++++------------
 1 file changed, 18 insertions(+), 12 deletions(-)
arch/x86/kvm/vmx/pmu_intel.c +18 -12
index d259b66abac8..65b6f274a8f6 100644
@@ -55,6 +55,17 @@ static struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu)
 
 #pragma GCC poison to_vmx
 
+static u64 intel_pmu_fixed_ctrl_bits(struct kvm_pmu *pmu, u64 bits)
+{
+	u64 fixed_ctrl_bits = 0;
+	int i;
+
+	kvm_for_each_fixed_counter(i, pmu)
+		fixed_ctrl_bits |= intel_fixed_bits_by_idx(i, bits);
+
+	return fixed_ctrl_bits;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	/*
@@ -534,20 +545,13 @@ static __always_inline u64 intel_get_fixed_pmc_eventsel(unsigned int index)
 	return eventsel;
 }
 
-static void intel_pmu_enable_fixed_counter_bits(struct kvm_pmu *pmu, u64 bits)
-{
-	int i;
-
-	kvm_for_each_fixed_counter(i, pmu)
-		pmu->fixed_ctr_ctrl_rsvd &= ~intel_fixed_bits_by_idx(i, bits);
-}
-
 static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct lbr_desc *lbr_desc = vcpu_to_lbr_desc(vcpu);
 	int nr_gp_counters, nr_fixed_counters;
 	struct kvm_cpuid_entry2 *entry;
+	u64 fixed_enable_bits = 0;
 	union cpuid10_eax eax;
 	union cpuid10_edx edx;
 	u64 perf_capabilities;
@@ -633,9 +637,8 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 					  kvm_pmu_cap.bit_width_fixed);
 	pmu->counter_bitmask[KVM_PMC_FIXED] = BIT_ULL(edx.split.bit_width_fixed) - 1;
 
-	intel_pmu_enable_fixed_counter_bits(pmu, INTEL_FIXED_0_KERNEL |
-						 INTEL_FIXED_0_USER |
-						 INTEL_FIXED_0_ENABLE_PMI);
+	fixed_enable_bits |= INTEL_FIXED_0_KERNEL | INTEL_FIXED_0_USER |
+			     INTEL_FIXED_0_ENABLE_PMI;
 
 	counter_rsvd = ~(kvm_gp_pmc_mask(pmu) |
 			 (kvm_fixed_pmc_mask(pmu) << KVM_FIXED_PMC_BASE_IDX));
@@ -651,11 +654,14 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 			pmu->pebs_enable_rsvd = counter_rsvd;
 			pmu->eventsel_rsvd &= ~ICL_EVENTSEL_ADAPTIVE;
 			pmu->pebs_data_cfg_rsvd = ~0xff00000full;
-			intel_pmu_enable_fixed_counter_bits(pmu, ICL_FIXED_0_ADAPTIVE);
+			fixed_enable_bits |= ICL_FIXED_0_ADAPTIVE;
 		} else {
 			pmu->pebs_enable_rsvd = ~kvm_gp_pmc_mask(pmu);
 		}
 	}
+
+	pmu->fixed_ctr_ctrl_rsvd &=
+		~intel_pmu_fixed_ctrl_bits(pmu, fixed_enable_bits);
 }
 
 static void intel_pmu_init(struct kvm_vcpu *vcpu)
Signature · 1 line
2.56.1
author
#23 lore raw

PATCH v4 18/21 KVM: x86/pmu: Mask only OS/USR bits when filtering fixed counters

In reply to Zide Chen

In preparation for supporting AnyThread deprecation and Fixed Counter 3, mask only INTEL_FIXED_0_KERNEL and INTEL_FIXED_0_USER when updating fixed_ctr_ctrl_hw in kvm_mediated_pmu_refresh_event_filter().

Masking the full 0xf nibble would cause two issues:

1. AnyThread for Fixed Counters 0-2: When a counter is allowed, copying the full nibble from guest-visible pmu->fixed_ctr_ctrl back into hardware-visible pmu->fixed_ctr_ctrl_hw would resurrect AnyThread in hardware, defeating SMT cross-VM isolation.

2. Metrics Clear for Fixed Counter 3: If an event filter denies Fixed Counter 3, clearing 0xf inadvertently strips METRICS_CLEAR_EN from hardware. If the guest sets METRICS_CLEAR_EN and executes native RDPMC, metrics will remain un-cleared, returning stale data.

This gates hardware counting without clobbering other control bits, matching general-purpose counter filtering which only masks the event ENABLE bit.

---
v3:
- New patch.

---
 arch/x86/kvm/pmu.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +3 -1
index bd5954e62080..39fc3f72b92d 100644
@@ -557,7 +557,9 @@ static void kvm_mediated_pmu_refresh_event_filter(struct kvm_pmc *pmc)
 			pmc->eventsel_hw |= pmc->eventsel &
 					    ARCH_PERFMON_EVENTSEL_ENABLE;
 	} else {
-		u64 mask = intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX, 0xf);
+		u64 mask = intel_fixed_bits_by_idx(pmc->idx - KVM_FIXED_PMC_BASE_IDX,
+						    INTEL_FIXED_0_KERNEL |
+						    INTEL_FIXED_0_USER);
 
 		pmu->fixed_ctr_ctrl_hw &= ~mask;
 		if (allowed)
Signature · 1 line
2.56.1
author
#24 lore raw

PATCH v4 19/21 KVM: x86/pmu: Ignore AnyThread bit if CPUID.0AH:EDX[15] is set

In reply to Zide Chen

Intel PerfMon v5 introduces the ANYTHREAD_DEPRECATION capability (CPUID.0AH:EDX[15]) to indicate that AnyThread counting is deprecated and that writes to the AnyThread bit in IA32_PERFEVTSELx and IA32_FIXED_CTR_CTRL are ignored.

Preserve the guest-written value in guest-visible state so that guest RDMSRs observe the AnyThread bit, but strip it before programming hardware to ensure cross-VM SMT isolation.

Continue to inject #GP when ANYTHREAD_DEPRECATION is not present in the guest CPUID, e.g. for PerfMon v3/v4 guests or when the capability is not configured for a PerfMon v5+ guest.

Advertise CPUID.0AH:EDX[15] only for PMU version 5+ guests, since the bit doesn't exist below version 5.

---
v3:
- Exclude Fixed Counter 3 bit 14 (METRICS_CLEAR_EN) from the AnyThread mask so it remains reserved in fixed_ctr_ctrl_rsvd. (Jim)
- Preserve AnyThread in guest-visible state (eventsel/fixed_ctr_ctrl) to match bare-metal RDMSR behavior. (Jim)
- Removed it from the hardware-visible counterparts (_hw) so that it doesn't need to explicitly remove this bit in intel_pmu_set_msr(). (Jim)
- Re-use intel_pmu_fixed_ctrl_bits() for AnyThread bits computation. (Sashiko)

---
 arch/x86/kvm/cpuid.c         |  2 +-
 arch/x86/kvm/vmx/pmu_intel.c | 28 +++++++++++++++++++++++++---
 2 files changed, 26 insertions(+), 4 deletions(-)
arch/x86/kvm/cpuid.c +1 -1
index 2a4530d000fc..36540e121807 100644
@@ -1549,7 +1549,7 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)
 					    KVM_MAX_NR_FIXED_COUNTERS);
 		edx.split.bit_width_fixed = kvm_pmu_cap.bit_width_fixed;
 
-		if (kvm_pmu_cap.version)
+		if (kvm_pmu_cap.version >= 5)
 			edx.split.anythread_deprecated = 1;
 
 		entry->eax = eax.full;
arch/x86/kvm/vmx/pmu_intel.c +25 -3
index 65b6f274a8f6..4824e7eb98d3 100644
@@ -66,6 +66,13 @@ static u64 intel_pmu_fixed_ctrl_bits(struct kvm_pmu *pmu, u64 bits)
 	return fixed_ctrl_bits;
 }
 
+static u64 intel_pmu_fixed_anythread_mask(struct kvm_pmu *pmu)
+{
+	/* Fixed counter 3 repurposes that bit for METRICS_CLEAR_EN. */
+	return intel_pmu_fixed_ctrl_bits(pmu, INTEL_FIXED_0_ANYTHREAD) &
+	       ~intel_fixed_bits_by_idx(3, INTEL_FIXED_3_METRICS_CLEAR);
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	/*
@@ -81,9 +88,9 @@ static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 	int i;
 
 	pmu->fixed_ctr_ctrl = data;
-	pmu->fixed_ctr_ctrl_hw = data;
+	pmu->fixed_ctr_ctrl_hw = data & ~intel_pmu_fixed_anythread_mask(pmu);
 	kvm_for_each_fixed_counter(i, pmu) {
-		u8 new_ctrl = fixed_ctrl_field(data, i);
+		u8 new_ctrl = fixed_ctrl_field(pmu->fixed_ctr_ctrl_hw, i);
 		u8 old_ctrl = fixed_ctrl_field(old_fixed_ctr_ctrl, i);
 
 		if (old_ctrl == new_ctrl)
@@ -496,7 +503,7 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 
 			if (data != pmc->eventsel) {
 				pmc->eventsel = data;
-				pmc->eventsel_hw = data;
+				pmc->eventsel_hw = data & ~ARCH_PERFMON_EVENTSEL_ANY;
 				kvm_pmu_request_counter_reprogram(pmc);
 			}
 			break;
@@ -662,6 +669,21 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 
 	pmu->fixed_ctr_ctrl_rsvd &=
 		~intel_pmu_fixed_ctrl_bits(pmu, fixed_enable_bits);
+
+	/*
+	 * AnyThread counting is not supported by KVM due to cross-VM
+	 * information leakage concerns on SMT cores. Therefore, AnyThread
+	 * remains unavailable for PerfMon v3/v4 guests, where AnyThread
+	 * deprecation is not enumerated.
+	 *
+	 * On PerfMon v5+ with ANYTHREAD_DEPRECATION, writes to AnyThread bits
+	 * are accepted and preserved on RDMSR, while counting behavior is
+	 * ignored.
+	 */
+	if (pmu->version >= 5 && edx.split.anythread_deprecated) {
+		pmu->eventsel_rsvd &= ~ARCH_PERFMON_EVENTSEL_ANY;
+		pmu->fixed_ctr_ctrl_rsvd &= ~intel_pmu_fixed_anythread_mask(pmu);
+	}
 }
 
 static void intel_pmu_init(struct kvm_vcpu *vcpu)
Signature · 1 line
2.56.1
author
#25 lore raw

PATCH v4 20/21 KVM: x86/pmu: Advertise PerfMon version 5 on Intel hosts

In reply to Zide Chen

From: Dapeng Mi <[email protected]>

KVM currently caps the guest PerfMon version at 2 for all Intel platforms. Now that KVM emulates the basic architectural PMU features introduced in PerfMon versions 3 through 5, raise the guest PerfMon version to 5. Features that require additional emulation support, e.g. architectural LBR will be enabled separately.

PerfMon versions 3 and 4 continue to be exposed as version 2 because KVM cannot safely virtualize AnyThread due to cross-VM information leakage. PerfMon v5 is supported because it provides the AnyThread-deprecated capability, allowing KVM to indicate that AnyThread is deprecated.

When enable_mediated_pmu is disabled or in the non-Intel paths, KVM retains the existing cap of version 2.

---
v3:
- Simplify version cap logic since mediated PMU already requires version >= 4. (Jim)
v2:
- Cap vPMU v3/v4 to v2, as AnyThread cannot be safely virtualized. (Sashiko)

---
 arch/x86/kvm/pmu.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +11 -1
index 39fc3f72b92d..7b4137fbcb4f 100644
@@ -175,7 +175,17 @@ void kvm_init_pmu_capability(struct kvm_pmu_ops *pmu_ops)
 	}
 
 	memcpy(&kvm_pmu_cap, &kvm_host_pmu, sizeof(kvm_host_pmu));
-	kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
+
+	/*
+	 * PMU v3/v4 has AnyThread support, which KVM cannot safely virtualize.
+	 * Only expose PMU v5+ for Intel mediated PMU, where AnyThread is
+	 * deprecated; otherwise, cap at PMU v2.
+	 */
+	if (is_intel && enable_mediated_pmu && kvm_pmu_cap.version >= 5)
+		kvm_pmu_cap.version = 5;
+	else
+		kvm_pmu_cap.version = min(kvm_pmu_cap.version, 2);
+
 	kvm_pmu_cap.cntr_mask64 &=
 		GENMASK_ULL(pmu_ops->MAX_NR_GP_COUNTERS - 1, 0);
 	kvm_pmu_cap.fixed_cntr_mask64 &=
Signature · 1 line
2.56.1
author
#26 lore raw

PATCH v4 21/21 KVM: selftests: Support fixed counters bitmap in pmu_counters_test

In reply to Zide Chen

From: Dapeng Mi <[email protected]>

On PerfMon v5, CPUID.0AH:ECX represents the fixed counter bitmask, while EDX[4:0] indicates the number of contiguous fixed counters starting from 0:

FxCtr[i]_is_supported := ECX[i] || (EDX[4:0] > i).

Fix test_fixed_counters() to derive supported fixed counters using the above formula instead of relying solely on EDX[4:0]. This allows the test to cover non-contiguous fixed counter configurations.

Update this_pmu_has() to take the PMU version into consideration. Without this fix, it could return incorrect results in test cases such as test_arch_events(), which configure a PMU version different from the one originally advertised by KVM without explicitly updating CPUID.0AH:{ECX,EDX} to match the new PMU version.

For example, after downgrading a guest from PMU v5 to PMU v2, this_pmu_has() would still consult CPUID.0AH:ECX while KVM ignores it, resulting in a mismatch in the reported fixed counter capabilities.

---
v3:
- use the BIT() macro instead of an open-coded left shift. (Sashiko)
- Simplified ternary condition in this_pmu_has(). (Sashiko)
- Also test legacy EDX contiguous counts with ECX=0 in test_fixed_counters(). (Sashiko)

v2:
- Fix this_pmu_has() to honor the guest's configured PMU version.
- Update the pmu_vm_create_with_vcpus() calls after rebasing.

---
 .../selftests/kvm/include/x86/processor.h     |  6 +++-
 .../selftests/kvm/x86/pmu_counters_test.c     | 29 +++++++++++++++----
 2 files changed, 29 insertions(+), 6 deletions(-)
tools/testing/selftests/kvm/include/x86/processor.h +5 -1
index ffec2291b1e1..21381db1c4ca 100644
@@ -836,6 +836,7 @@ static __always_inline bool this_cpu_has_p(struct kvm_x86_cpu_property property)
 
 static inline bool this_pmu_has(struct kvm_x86_pmu_feature feature)
 {
+	u8 pmu_version;
 	u32 nr_bits;
 
 	if (feature.f.reg == KVM_CPUID_EBX) {
@@ -845,7 +846,10 @@ static inline bool this_pmu_has(struct kvm_x86_pmu_feature feature)
 
 	GUEST_ASSERT(feature.f.reg == KVM_CPUID_ECX);
 	nr_bits = this_cpu_property(X86_PROPERTY_PMU_NR_FIXED_COUNTERS);
-	return nr_bits > feature.f.bit || this_cpu_has(feature.f);
+	pmu_version = this_cpu_property(X86_PROPERTY_PMU_VERSION);
+
+	return nr_bits > feature.f.bit ||
+	       (pmu_version >= 5 && this_cpu_has(feature.f));
 }
 
 static __always_inline u64 this_cpu_supported_xcr0(void)
tools/testing/selftests/kvm/x86/pmu_counters_test.c +24 -5
index c3e784e16348..61e6fe5b6ef4 100644
@@ -3,6 +3,7 @@
  * Copyright (C) 2023, Tencent, Inc.
  */
 #include <x86intrin.h>
+#include <linux/bitmap.h>
 
 #include "pmu.h"
 #include "processor.h"
@@ -631,25 +632,43 @@ static void __test_fixed_counters(struct kvm_vcpu *vcpu, u8 nr_fixed_counters,
 static void test_fixed_counters(u8 pmu_version, u64 perf_capabilities)
 {
 	u8 nr_fixed_counters = kvm_cpu_property(X86_PROPERTY_PMU_NR_FIXED_COUNTERS);
+	unsigned long fixed_subset;
 	struct kvm_vcpu **vcpus;
 	struct kvm_vm *vm;
+	u32 fixed_bitmap;
 	int i = 0;
-	u32 k;
 	u8 j;
 
 	pr_info("Testing %u fixed counters, PMU version %u, perf_caps = %lx\n",
 		nr_fixed_counters, pmu_version, perf_capabilities);
 
+	fixed_bitmap = BIT(nr_fixed_counters) - 1;
+	if (pmu_version >= 5)
+		fixed_bitmap |= kvm_cpu_property(X86_PROPERTY_PMU_FIXED_COUNTERS_BITMASK);
 
-	vm = pmu_vm_create_with_vcpus((nr_fixed_counters + 1) * BIT(nr_fixed_counters),
+	/* One vCPU per subset of fixed_bitmap, plus one per legacy EDX test. */
+	vm = pmu_vm_create_with_vcpus(BIT(__builtin_popcount(fixed_bitmap)) + nr_fixed_counters,
 				      guest_test_fixed_counters,
 				      pmu_version, perf_capabilities, &vcpus);
 
-	for (j = 0; j <= nr_fixed_counters; j++) {
-		for (k = 0; k <= (BIT(nr_fixed_counters) - 1); k++)
-			__test_fixed_counters(vcpus[i++], j, k);
+	for (fixed_subset = 0; fixed_subset <= fixed_bitmap; fixed_subset++) {
+		u32 nr_contiguous;
+
+		/*
+		 * The loop walks all values from 0 to fixed_bitmap, so skip any
+		 * value that is not a subset of fixed_bitmap.
+		 */
+		if (fixed_subset & ~fixed_bitmap)
+			continue;
+
+		nr_contiguous = find_first_zero_bit(&fixed_subset,
+						    MAX_NR_FIXED_COUNTERS);
+		__test_fixed_counters(vcpus[i++], nr_contiguous, fixed_subset);
 	}
 
+	for (j = 1; j <= nr_fixed_counters; j++)
+		__test_fixed_counters(vcpus[i++], j, 0);
+
 	pmu_vm_free(vm, vcpus);
 }
 
Signature · 1 line
2.56.1
#27 lore raw

Re: [PATCH v4 21/21] KVM: selftests: Support fixed counters bitmap in pmu_counters_test

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] KVM: selftests: missing EDX/ECX combinations in test_fixed_counters() --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] KVM: selftests: missing EDX/ECX combinations in test_fixed_counters()
--

commit 3d8b2ca900bb2a01f5f9cee15fc9a12bc40184ad
Author: Dapeng Mi <[email protected]>

KVM: selftests: Support fixed counters bitmap in pmu_counters_test

This commit updates this_pmu_has() to check the guest PMU version before consulting CPUID.0AH:ECX and updates test_fixed_counters() to test supported fixed counter bitmasks alongside CPUID.0AH:EDX[4:0].

Quoted 4 lines
tools/testing/selftests/kvm/x86/pmu_counters_test.c +0 -0
index c3e784e16348c..61e6fe5b6ef48 100644

[ ... ]

Quoted 38 lines
tools/testing/selftests/kvm/x86/pmu_counters_test.c +17 -5
@@ -631,25 +632,43 @@ static void __test_fixed_counters(struct kvm_vcpu *vcpu, u8 nr_fixed_counters,
 static void test_fixed_counters(u8 pmu_version, u64 perf_capabilities)
 {
 	u8 nr_fixed_counters = kvm_cpu_property(X86_PROPERTY_PMU_NR_FIXED_COUNTERS);
+	unsigned long fixed_subset;
 	struct kvm_vcpu **vcpus;
 	struct kvm_vm *vm;
+	u32 fixed_bitmap;
 	int i = 0;
-	u32 k;
 	u8 j;
 
 	pr_info("Testing %u fixed counters, PMU version %u, perf_caps = %lx\n",
 		nr_fixed_counters, pmu_version, perf_capabilities);
 
+	fixed_bitmap = BIT(nr_fixed_counters) - 1;
+	if (pmu_version >= 5)
+		fixed_bitmap |= kvm_cpu_property(X86_PROPERTY_PMU_FIXED_COUNTERS_BITMASK);
 
-	vm = pmu_vm_create_with_vcpus((nr_fixed_counters + 1) * BIT(nr_fixed_counters),
+	/* One vCPU per subset of fixed_bitmap, plus one per legacy EDX test. */
+	vm = pmu_vm_create_with_vcpus(BIT(__builtin_popcount(fixed_bitmap)) + nr_fixed_counters,
 				      guest_test_fixed_counters,
 				      pmu_version, perf_capabilities, &vcpus);
 
-	for (j = 0; j <= nr_fixed_counters; j++) {
-		for (k = 0; k <= (BIT(nr_fixed_counters) - 1); k++)
-			__test_fixed_counters(vcpus[i++], j, k);
+	for (fixed_subset = 0; fixed_subset <= fixed_bitmap; fixed_subset++) {
+		u32 nr_contiguous;
+
+		/*
+		 * The loop walks all values from 0 to fixed_bitmap, so skip any
+		 * value that is not a subset of fixed_bitmap.
+		 */
+		if (fixed_subset & ~fixed_bitmap)
+			continue;
+

tools/testing/selftests/kvm/x86/pmu_counters_test.c static void __test_fixed_counters(struct kvm_vcpu *vcpu, u8 nr_fixed_counters, · line 673

Quoted 10 lines
tools/testing/selftests/kvm/x86/pmu_counters_test.c +6 -0
+		nr_contiguous = find_first_zero_bit(&fixed_subset,
+						    MAX_NR_FIXED_COUNTERS);
+		__test_fixed_counters(vcpus[i++], nr_contiguous, fixed_subset);
 	}
 
+	for (j = 1; j <= nr_fixed_counters; j++)
+		__test_fixed_counters(vcpus[i++], j, 0);
+
 	pmu_vm_free(vm, vcpus);
 }

[Severity: Medium]
Does deriving nr_contiguous from fixed_subset drop test coverage for fixed counters enabled solely via CPUID.0AH:ECX when EDX[4:0] is 0?

Whenever bit 0 of fixed_subset is set, find_first_zero_bit() returns at least 1, and the second loop only tests supported_bitmask == 0 with j >= 1. As a result, fixed counter 0 (and any contiguous prefix of fixed_subset starting at bit 0) is never tested with nr_fixed_counters == 0 and ECX[i] set, nor does a guest with pmu_version < 5 get tested with EDX[4:0] == 0 and a non-zero contiguous ECX mask to verify that KVM ignores ECX.

In addition, gating X86_PROPERTY_PMU_FIXED_COUNTERS_BITMASK on pmu_version >= 5 skips non-contiguous host ECX bits when testing guest pmu_version < 5, so the test never verifies that KVM ignores those bits on older PMU versions. Keeping the outer loop over j in 0..nr_fixed_counters and the inner loop over subsets of an unconditionally constructed fixed_bitmap would cover all (EDX[4:0], ECX) combinations across all PMU versions.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=21
author
#28 lore raw

Re: [PATCH v4 21/21] KVM: selftests: Support fixed counters bitmap in pmu_counters_test

Quoted 75 lines · On 11/12/2026 3:00 PM, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] KVM: selftests: missing EDX/ECX combinations in test_fixed_counters()
--

commit 3d8b2ca900bb2a01f5f9cee15fc9a12bc40184ad
Author: Dapeng Mi <[email protected]>

KVM: selftests: Support fixed counters bitmap in pmu_counters_test

This commit updates this_pmu_has() to check the guest PMU version before consulting CPUID.0AH:ECX and updates test_fixed_counters() to test supported fixed counter bitmasks alongside CPUID.0AH:EDX[4:0].

Quoted 4 lines
tools/testing/selftests/kvm/x86/pmu_counters_test.c +0 -0
index c3e784e16348c..61e6fe5b6ef48 100644

[ ... ]

Quoted 38 lines
tools/testing/selftests/kvm/x86/pmu_counters_test.c +17 -5
@@ -631,25 +632,43 @@ static void __test_fixed_counters(struct kvm_vcpu *vcpu, u8 nr_fixed_counters,
 static void test_fixed_counters(u8 pmu_version, u64 perf_capabilities)
 {
 	u8 nr_fixed_counters = kvm_cpu_property(X86_PROPERTY_PMU_NR_FIXED_COUNTERS);
+	unsigned long fixed_subset;
 	struct kvm_vcpu **vcpus;
 	struct kvm_vm *vm;
+	u32 fixed_bitmap;
 	int i = 0;
-	u32 k;
 	u8 j;
 
 	pr_info("Testing %u fixed counters, PMU version %u, perf_caps = %lx\n",
 		nr_fixed_counters, pmu_version, perf_capabilities);
 
+	fixed_bitmap = BIT(nr_fixed_counters) - 1;
+	if (pmu_version >= 5)
+		fixed_bitmap |= kvm_cpu_property(X86_PROPERTY_PMU_FIXED_COUNTERS_BITMASK);
 
-	vm = pmu_vm_create_with_vcpus((nr_fixed_counters + 1) * BIT(nr_fixed_counters),
+	/* One vCPU per subset of fixed_bitmap, plus one per legacy EDX test. */
+	vm = pmu_vm_create_with_vcpus(BIT(__builtin_popcount(fixed_bitmap)) + nr_fixed_counters,
 				      guest_test_fixed_counters,
 				      pmu_version, perf_capabilities, &vcpus);
 
-	for (j = 0; j <= nr_fixed_counters; j++) {
-		for (k = 0; k <= (BIT(nr_fixed_counters) - 1); k++)
-			__test_fixed_counters(vcpus[i++], j, k);
+	for (fixed_subset = 0; fixed_subset <= fixed_bitmap; fixed_subset++) {
+		u32 nr_contiguous;
+
+		/*
+		 * The loop walks all values from 0 to fixed_bitmap, so skip any
+		 * value that is not a subset of fixed_bitmap.
+		 */
+		if (fixed_subset & ~fixed_bitmap)
+			continue;
+

tools/testing/selftests/kvm/x86/pmu_counters_test.c static void __test_fixed_counters(struct kvm_vcpu *vcpu, u8 nr_fixed_counters, · line 673

Quoted 10 lines
tools/testing/selftests/kvm/x86/pmu_counters_test.c +6 -0
+		nr_contiguous = find_first_zero_bit(&fixed_subset,
+						    MAX_NR_FIXED_COUNTERS);
+		__test_fixed_counters(vcpus[i++], nr_contiguous, fixed_subset);
 	}
 
+	for (j = 1; j <= nr_fixed_counters; j++)
+		__test_fixed_counters(vcpus[i++], j, 0);
+
 	pmu_vm_free(vm, vcpus);
 }

[Severity: Medium]
Does deriving nr_contiguous from fixed_subset drop test coverage for fixed counters enabled solely via CPUID.0AH:ECX when EDX[4:0] is 0?

Quoted 6 lines

Whenever bit 0 of fixed_subset is set, find_first_zero_bit() returns at least 1, and the second loop only tests supported_bitmask == 0 with j >= 1. As a result, fixed counter 0 (and any contiguous prefix of fixed_subset starting at bit 0) is never tested with nr_fixed_counters == 0 and ECX[i] set, nor does a guest with pmu_version < 5 get tested with EDX[4:0] == 0 and a non-zero contiguous ECX mask to verify that KVM ignores ECX.

The mentioned configuration is architecturally invalid and can be
ignored: if fixed counter 0 is supported (ECX[0] == 1), EDX[4:0] cannot be zero.

Furthermore, verifying that KVM ignores ECX on pmu_version < 5 with EDX[4:0] == 0 is already tested for all subsets where bit 0 is clear (e.g. subsets 2, 4, 6), where find_first_zero_bit() returns 0 and the guest explicitly asserts #GP.

Quoted 9 lines

In addition, gating X86_PROPERTY_PMU_FIXED_COUNTERS_BITMASK on pmu_version >= 5 skips non-contiguous host ECX bits when testing guest pmu_version < 5, so the test never verifies that KVM ignores those bits on older PMU versions. Keeping the outer loop over j in 0..nr_fixed_counters and the inner loop over subsets of an unconditionally constructed fixed_bitmap would cover all (EDX[4:0], ECX) combinations across all PMU versions.

The proposed nested loop produces massive duplication without covering any new counter configurations.

In contrast, the current single loop exercises all 2^N unique subsets of available counters exactly once with consistent EDX/ECX pairs, and the second loop explicitly exercises legacy EDX-only enumeration (ECX = 0).

Therefore, I prefer to keep the current implementation.