Skip to contentSkip to conversation

Mediated Passthrough vPMU 2.0 for x86

PATCH Superseded Mingwei Zhang 11 participants 116 messages May 6, 2024 – Jul 10, 2024

Subsystems (4)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover Mediated Passthrough vPMU 2.0 for x86

In this version, we added the mediated passthrough vPMU support for AMD. This is 1st version that comes up with a full x86 support on the vPMU new design.

Major changes:
- AMD support integration. Supporting guest PerfMon v1 and v2.
- Ensure !exclude_guest events only exist prior to mediate passthrough vPMU loaded. [sean]
- Update PMU MSR interception according to exposed counters and pmu version. [mingwei reported pmu_counters_test fails]
- Enforce RDPMC interception unless all counters exposed to guest. This removes a hack in RFCv1 where we pass through RDPMC and zero unexposed counters. [jim/sean]
- Combine the PMU context switch for both AMD and Intel.
- Because of RDPMC interception, update PMU context switch code by removing the "zeroing out" logic when restoring the guest context. [jim/sean: intercept rdpmc]

Minor changes:
- Flip enable_passthrough_pmu to false and change to a vendor param.
- Remove "Intercept full-width GP counter MSRs by checking with perf capabilities".
- Remove the write to pmc patch.
- Move host_perf_cap as an independent variable, will update after
https://lore.kernel.org/all/[email protected]/

TODOs:
- Simplify enabling code for mediated passthrough vPMU.
- Further optimization on PMU context switch.

On-going discussions:
- Final name of mediated passthrough vPMU.
- PMU context switch optimizations.

Testing:
- Testcases:
- selftest: pmu_counters_test
- selftest: pmu_event_filter_test
- kvm-unit-tests: pmu
- qemu based ubuntu 20.04 (guest kernel: 5.10 and 6.7.9)
- Platforms:
- genoa
- skylake
- icelake
- sapphirerapids
- emeraldrapids

Ongoing Issues:
- AMD platform [milan]:
- ./pmu_event_filter_test error:
- test_amd_deny_list: Branch instructions retired = 44 (expected 42)
- test_without_filter: Branch instructions retired = 44 (expected 42)
- test_member_allow_list: Branch instructions retired = 44 (expected 42)
- test_not_member_deny_list: Branch instructions retired = 44 (expected 42)
- Intel platform [skylake]:
- kvm-unit-tests/pmu fails with two errors:
- FAIL: Intel: TSX cycles: gp cntr-3 with a value of 0
- FAIL: Intel: full-width writes: TSX cycles: gp cntr-3 with a value of 0

Installation guidance:
- echo 0 > /proc/sys/kernel/nmi_watchdog
- modprobe kvm_{amd,intel} enable_passthrough_pmu=Y 2>/dev/null

v1: https://lore.kernel.org/all/[email protected]/

Dapeng Mi (3):
x86/msr: Introduce MSR_CORE_PERF_GLOBAL_STATUS_SET
KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS
KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU
MSRs

Kan Liang (3):
perf: Support get/put passthrough PMU interfaces
perf: Add generic exclude_guest support perf/x86/intel: Support PERF_PMU_CAP_PASSTHROUGH_VPMU

Manali Shukla (1):
KVM: x86/pmu/svm: Wire up PMU filtering functionality for passthrough
PMU

Mingwei Zhang (24):
perf: core/x86: Forbid PMI handler when guest own PMU
perf: core/x86: Plumb passthrough PMU capability from x86_pmu to

    x86_pmu_cap
  KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter
  KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs
  KVM: x86/pmu: Add a helper to check if passthrough PMU is enabled
  KVM: x86/pmu: Add host_perf_cap and initialize it in
    kvm_x86_vendor_init()

KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to
guest
KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough
allowed
KVM: x86/pmu: Create a function prototype to disable MSR interception
KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in
passthrough vPMU
KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()
KVM: x86/pmu: Add counter MSR and selector MSR index into struct
kvm_pmc
KVM: x86/pmu: Introduce PMU operation prototypes for save/restore PMU
context
KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU
KVM: x86/pmu: Make check_pmu_event_filter() an exported function
KVM: x86/pmu: Allow writing to event selector for GP counters if event
is allowed
KVM: x86/pmu: Allow writing to fixed counter selector if counter is
exposed
KVM: x86/pmu: Exclude existing vLBR logic from the passthrough PMU
KVM: x86/pmu: Introduce PMU operator to increment counter
KVM: x86/pmu: Introduce PMU operator for setting counter overflow
KVM: x86/pmu: Implement emulated counter increment for passthrough PMU
KVM: x86/pmu: Update pmc_{read,write}_counter() to disconnect perf API
KVM: x86/pmu: Disconnect counter reprogram logic from passthrough PMU
KVM: nVMX: Add nested virtualization support for passthrough PMU

Sandipan Das (11):
KVM: x86/pmu: Do not mask LVTPC when handling a PMI on AMD platforms x86/msr: Define PerfCntrGlobalStatusSet register
KVM: x86/pmu: Always set global enable bits in passthrough mode perf/x86/amd/core: Set passthrough capability for host
KVM: x86/pmu/svm: Set passthrough capability for vcpus
KVM: x86/pmu/svm: Set enable_passthrough_pmu module parameter
KVM: x86/pmu/svm: Allow RDPMC pass through when all counters exposed
to guest
KVM: x86/pmu/svm: Implement callback to disable MSR interception
KVM: x86/pmu/svm: Set GuestOnly bit and clear HostOnly bit when guest
write to event selectors
KVM: x86/pmu/svm: Add registers to direct access list
KVM: x86/pmu/svm: Implement handlers to save and restore context

Sean Christopherson (2):
KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at

    "RESET"

KVM: x86: Snapshot if a vCPU's vendor model is AMD vs. Intel
compatible

Xiong Zhang (10):
perf: core/x86: Register a new vector for KVM GUEST PMI
KVM: x86: Extract x86_set_kvm_irq_handler() function
KVM: x86/pmu: Register guest pmi handler for emulated PMU
perf: x86: Add x86 function to switch PMI handler
KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL
KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary
KVM: x86/pmu: Switch PMI handler at KVM context switch boundary
KVM: x86/pmu: Grab x86 core PMU for passthrough PMU VM
KVM: x86/pmu: Call perf_guest_enter() at PMU context switch
KVM: x86/pmu: Add support for PMU context switch at VM-exit/enter

 arch/x86/events/amd/core.c               |   3 +
 arch/x86/events/core.c                   |  41 ++++-
 arch/x86/events/intel/core.c             |   6 +
 arch/x86/events/perf_event.h             |   1 +
 arch/x86/include/asm/hardirq.h           |   1 +
 arch/x86/include/asm/idtentry.h          |   1 +
 arch/x86/include/asm/irq.h               |   2 +-
 arch/x86/include/asm/irq_vectors.h       |   5 +-
 arch/x86/include/asm/kvm-x86-pmu-ops.h   |   6 +
 arch/x86/include/asm/kvm_host.h          |  10 ++
 arch/x86/include/asm/msr-index.h         |   2 +
 arch/x86/include/asm/perf_event.h        |   4 +
 arch/x86/include/asm/vmx.h               |   1 +
 arch/x86/kernel/idt.c                    |   1 +
 arch/x86/kernel/irq.c                    |  36 ++++-
 arch/x86/kvm/cpuid.c                     |   4 +
 arch/x86/kvm/cpuid.h                     |  10 ++
 arch/x86/kvm/lapic.c                     |   3 +-
 arch/x86/kvm/mmu/mmu.c                   |   2 +-
 arch/x86/kvm/pmu.c                       | 168 ++++++++++++++++++-
 arch/x86/kvm/pmu.h                       |  47 ++++++
 arch/x86/kvm/svm/pmu.c                   | 112 ++++++++++++-
 arch/x86/kvm/svm/svm.c                   |  23 +++
 arch/x86/kvm/svm/svm.h                   |   2 +-
 arch/x86/kvm/vmx/capabilities.h          |   1 +
 arch/x86/kvm/vmx/nested.c                |  52 ++++++
 arch/x86/kvm/vmx/pmu_intel.c             | 192 ++++++++++++++++++++--
 arch/x86/kvm/vmx/vmx.c                   | 197 +++++++++++++++++++----
 arch/x86/kvm/vmx/vmx.h                   |   3 +-
 arch/x86/kvm/x86.c                       |  47 +++++-
 arch/x86/kvm/x86.h                       |   1 +
 include/linux/perf_event.h               |  18 +++
 kernel/events/core.c                     | 176 ++++++++++++++++++++
 tools/arch/x86/include/asm/irq_vectors.h |   3 +-
 34 files changed, 1120 insertions(+), 61 deletions(-)

base-commit: fec50db7033ea478773b159e0e2efb135270e3b7

Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#1 lore raw

PATCH v2 1/54 KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"

From: Sean Christopherson <[email protected]>

Set the enable bits for general purpose counters in IA32_PERF_GLOBAL_CTRL when refreshing the PMU to emulate the MSR's architecturally defined post-RESET behavior. Per Intel's SDM:

IA32_PERF_GLOBAL_CTRL: Sets bits n-1:0 and clears the upper bits.

and

Where "n" is the number of general-purpose counters available in the processor.

AMD also documents this behavior for PerfMonV2 CPUs in one of AMD's many PPRs.

Do not set any PERF_GLOBAL_CTRL bits if there are no general purpose counters, although a literal reading of the SDM would require the CPU to set either bits 63:0 or 31:0. The intent of the behavior is to globally enable all GP counters; honor the intent, if not the letter of the law.

Leaving PERF_GLOBAL_CTRL '0' effectively breaks PMU usage in guests that haven't been updated to work with PMUs that support PERF_GLOBAL_CTRL. This bug was recently exposed when KVM added supported for AMD's PerfMonV2, i.e. when KVM started exposing a vPMU with PERF_GLOBAL_CTRL to guest software that only knew how to program v1 PMUs (that don't support PERF_GLOBAL_CTRL).

Failure to emulate the post-RESET behavior results in such guests unknowingly leaving all general purpose counters globally disabled (the entire reason the post-RESET value sets the GP counter enable bits is to maintain backwards compatibility).

The bug has likely gone unnoticed because PERF_GLOBAL_CTRL has been supported on Intel CPUs for as long as KVM has existed, i.e. hardly anyone is running guest software that isn't aware of PERF_GLOBAL_CTRL on Intel PMUs. And because up until v6.0, KVM _did_ emulate the behavior for Intel CPUs, although the old behavior was likely dumb luck.

Because (a) that old code was also broken in its own way (the history of this code is a comedy of errors), and (b) PERF_GLOBAL_CTRL was documented as having a value of '0' post-RESET in all SDMs before March 2023.

Initial vPMU support in commit f5132b01386b ("KVM: Expose a version 2 architectural PMU to a guests") *almost* got it right (again likely by dumb luck), but for some reason only set the bits if the guest PMU was advertised as v1:

        if (pmu->version == 1) {
                pmu->global_ctrl = (1 << pmu->nr_arch_gp_counters) - 1;
                return;
        }

Commit f19a0c2c2e6a ("KVM: PMU emulation: GLOBAL_CTRL MSR should be enabled on reset") then tried to remedy that goof, presumably because guest PMUs were leaving PERF_GLOBAL_CTRL '0', i.e. weren't enabling counters.

        pmu->global_ctrl = ((1 << pmu->nr_arch_gp_counters) - 1) |
                (((1ull << pmu->nr_arch_fixed_counters) - 1) << X86_PMC_IDX_FIXED);
        pmu->global_ctrl_mask = ~pmu->global_ctrl;

That was KVM's behavior up until commit c49467a45fe0 ("KVM: x86/pmu:
Don't overwrite the pmu->global_ctrl when refreshing") removed *everything*. However, it did so based on the behavior defined by the SDM , which at the time stated that "Global Perf Counter Controls" is '0' at Power-Up and RESET.

But then the March 2023 SDM (325462-079US), stealthily changed its "IA-32 and Intel 64 Processor States Following Power-up, Reset, or INIT" table to say:

IA32_PERF_GLOBAL_CTRL: Sets bits n-1:0 and clears the upper bits.

Note, kvm_pmu_refresh() can be invoked multiple times, i.e. it's not a "pure" RESET flow. But it can only be called prior to the first KVM_RUN, i.e. the guest will only ever observe the final value.

Note #2, KVM has always cleared global_ctrl during refresh (see commit f5132b01386b ("KVM: Expose a version 2 architectural PMU to a guests")), i.e. there is no danger of breaking existing setups by clobbering a value set by userspace.

---
 arch/x86/kvm/pmu.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)
arch/x86/kvm/pmu.c +14 -2
index c397b28e3d1b..a593b03c9aed 100644
@@ -775,8 +775,20 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->pebs_data_cfg_mask = ~0ull;
 	bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
 
-	if (vcpu->kvm->arch.enable_pmu)
-		static_call(kvm_x86_pmu_refresh)(vcpu);
+	if (!vcpu->kvm->arch.enable_pmu)
+		return;
+
+	static_call(kvm_x86_pmu_refresh)(vcpu);
+
+	/*
+	 * At RESET, both Intel and AMD CPUs set all enable bits for general
+	 * purpose counters in IA32_PERF_GLOBAL_CTRL (so that software that
+	 * was written for v1 PMUs don't unknowingly leave GP counters disabled
+	 * in the global controls).  Emulate that behavior when refreshing the
+	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
+	 */
+	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
 }
 
 void kvm_pmu_init(struct kvm_vcpu *vcpu)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#2 lore raw

PATCH v2 2/54 KVM: x86: Snapshot if a vCPU's vendor model is AMD vs. Intel compatible

In reply to Mingwei Zhang

From: Sean Christopherson <[email protected]>

Add kvm_vcpu_arch.is_amd_compatible to cache if a vCPU's vendor model is compatible with AMD, i.e. if the vCPU vendor is AMD or Hygon, along with helpers to check if a vCPU is compatible AMD vs. Intel. To handle Intel vs. AMD behavior related to masking the LVTPC entry, KVM will need to check for vendor compatibility on every PMI injection, i.e. querying for AMD will soon be a moderately hot path.

Note! This subtly (or maybe not-so-subtly) makes "Intel compatible" KVM's default behavior, both if userspace omits (or never sets) CPUID 0x0 and if userspace sets a completely unknown vendor. One could argue that KVM should treat such vCPUs as not being compatible with Intel *or* AMD, but that would add useless complexity to KVM.

KVM needs to do *something* in the face of vendor specific behavior, and so unless KVM conjured up a magic third option, choosing to treat unknown vendors as neither Intel nor AMD means that checks on AMD compatibility would yield Intel behavior, and checks for Intel compatibility would yield AMD behavior. And that's far worse as it would effectively yield random behavior depending on whether KVM checked for AMD vs. Intel vs. !AMD vs. !Intel. And practically speaking, all x86 CPUs follow either Intel or AMD architecture, i.e. "supporting" an unknown third architecture adds no value.

Deliberately don't convert any of the existing guest_cpuid_is_intel() checks, as the Intel side of things is messier due to some flows explicitly checking for exactly vendor==Intel, versus some flows assuming anything that isn't "AMD compatible" gets Intel behavior. The Intel code will be cleaned up in the future.

Message-ID: <[email protected]>

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/cpuid.c            |  1 +
 arch/x86/kvm/cpuid.h            | 10 ++++++++++
 arch/x86/kvm/mmu/mmu.c          |  2 +-
 arch/x86/kvm/x86.c              |  2 +-
 5 files changed, 14 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 16e07a2eee19..6efd1497b026 100644
@@ -855,6 +855,7 @@ struct kvm_vcpu_arch {
 	int cpuid_nent;
 	struct kvm_cpuid_entry2 *cpuid_entries;
 	struct kvm_hypervisor_cpuid kvm_cpuid;
+	bool is_amd_compatible;
 
 	/*
 	 * FIXME: Drop this macro and use KVM_NR_GOVERNED_FEATURES directly
arch/x86/kvm/cpuid.c +1 -0
index bfc0bfcb2bc6..77352a4abd87 100644
@@ -376,6 +376,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 
 	kvm_update_pv_runtime(vcpu);
 
+	vcpu->arch.is_amd_compatible = guest_cpuid_is_amd_or_hygon(vcpu);
 	vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
arch/x86/kvm/cpuid.h +10 -0
index 856e3037e74f..23dbb9eb277c 100644
@@ -120,6 +120,16 @@ static inline bool guest_cpuid_is_intel(struct kvm_vcpu *vcpu)
 	return best && is_guest_vendor_intel(best->ebx, best->ecx, best->edx);
 }
 
+static inline bool guest_cpuid_is_amd_compatible(struct kvm_vcpu *vcpu)
+{
+	return vcpu->arch.is_amd_compatible;
+}
+
+static inline bool guest_cpuid_is_intel_compatible(struct kvm_vcpu *vcpu)
+{
+	return !guest_cpuid_is_amd_compatible(vcpu);
+}
+
 static inline int guest_cpuid_family(struct kvm_vcpu *vcpu)
 {
 	struct kvm_cpuid_entry2 *best;
arch/x86/kvm/mmu/mmu.c +1 -1
index 992e651540e8..bf4de6d7e39c 100644
@@ -4935,7 +4935,7 @@ static void reset_guest_rsvds_bits_mask(struct kvm_vcpu *vcpu,
 				context->cpu_role.base.level, is_efer_nx(context),
 				guest_can_use(vcpu, X86_FEATURE_GBPAGES),
 				is_cr4_pse(context),
-				guest_cpuid_is_amd_or_hygon(vcpu));
+				guest_cpuid_is_amd_compatible(vcpu));
 }
 
 static void __reset_rsvds_bits_mask_ept(struct rsvd_bits_validate *rsvd_check,
arch/x86/kvm/x86.c +1 -1
index 47d9f03b7778..ebcc12d1e1de 100644
@@ -3470,7 +3470,7 @@ static bool is_mci_status_msr(u32 msr)
 static bool can_set_mci_status(struct kvm_vcpu *vcpu)
 {
 	/* McStatusWrEn enabled? */
-	if (guest_cpuid_is_amd_or_hygon(vcpu))
+	if (guest_cpuid_is_amd_compatible(vcpu))
 		return !!(vcpu->arch.msr_hwcr & BIT_ULL(18));
 
 	return false;
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#3 lore raw

PATCH v2 3/54 KVM: x86/pmu: Do not mask LVTPC when handling a PMI on AMD platforms

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

On AMD and Hygon platforms, the local APIC does not automatically set the mask bit of the LVTPC register when handling a PMI and there is no need to clear it in the kernel's PMI handler.

For guests, the mask bit is currently set by kvm_apic_local_deliver() and unless it is cleared by the guest kernel's PMI handler, PMIs stop arriving and break use-cases like sampling with perf record.

This does not affect non-PerfMonV2 guests because PMIs are handled in the guest kernel by x86_pmu_handle_irq() which always clears the LVTPC mask bit irrespective of the vendor.

Before:

  $ perf record -e cycles:u true
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.001 MB perf.data (1 samples) ]

After:

  $ perf record -e cycles:u true
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.002 MB perf.data (19 samples) ]
Fixes a16eb25b09c0 ("KVM: x86: Mask LVTPC when handling a PMI")
[sean: use is_intel_compatible instead of !is_amd_or_hygon()]

Message-ID: <[email protected]>

---
 arch/x86/kvm/lapic.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/lapic.c +2 -1
index cf37586f0466..ebf41023be38 100644
@@ -2776,7 +2776,8 @@ int kvm_apic_local_deliver(struct kvm_lapic *apic, int lvt_type)
 		trig_mode = reg & APIC_LVT_LEVEL_TRIGGER;
 
 		r = __apic_accept_irq(apic, mode, vector, 1, trig_mode, NULL);
-		if (r && lvt_type == APIC_LVTPC)
+		if (r && lvt_type == APIC_LVTPC &&
+		    guest_cpuid_is_intel_compatible(apic->vcpu))
 			kvm_lapic_set_reg(apic, APIC_LVTPC, reg | APIC_LVT_MASKED);
 		return r;
 	}
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#4 lore raw

PATCH v2 4/54 x86/msr: Define PerfCntrGlobalStatusSet register

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Define PerfCntrGlobalStatusSet (MSR 0xc0000303) as it is required by passthrough PMU to set the overflow bits of PerfCntrGlobalStatus (MSR 0xc0000300).

When using passthrough PMU, it is necessary to restore the guest state of the overflow bits. Since PerfCntrGlobalStatus is read-only, this is done by writing to PerfCntrGlobalStatusSet instead.

The register is available on AMD processors where the PerfMonV2 feature bit of CPUID leaf 0x80000022 EAX is set.

---
 arch/x86/include/asm/msr-index.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/msr-index.h +1 -0
index 05956bd8bacf..ed9e3e8a57d6 100644
@@ -675,6 +675,7 @@
 #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS	0xc0000300
 #define MSR_AMD64_PERF_CNTR_GLOBAL_CTL		0xc0000301
 #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR	0xc0000302
+#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET	0xc0000303
 
 /* AMD Last Branch Record MSRs */
 #define MSR_AMD64_LBR_SELECT			0xc000010e
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#5 lore raw

PATCH v2 5/54 x86/msr: Introduce MSR_CORE_PERF_GLOBAL_STATUS_SET

In reply to Mingwei Zhang

From: Dapeng Mi <[email protected]>

Add additional PMU MSRs MSR_CORE_PERF_GLOBAL_STATUS_SET to allow passthrough PMU operation on the read-only MSR IA32_PERF_GLOBAL_STATUS.

---
 arch/x86/include/asm/msr-index.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/msr-index.h +1 -0
index ed9e3e8a57d6..a1281511807c 100644
@@ -1107,6 +1107,7 @@
 #define MSR_CORE_PERF_GLOBAL_STATUS	0x0000038e
 #define MSR_CORE_PERF_GLOBAL_CTRL	0x0000038f
 #define MSR_CORE_PERF_GLOBAL_OVF_CTRL	0x00000390
+#define MSR_CORE_PERF_GLOBAL_STATUS_SET 0x00000391
 
 #define MSR_PERF_METRICS		0x00000329
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#6 lore raw

PATCH v2 6/54 perf: Support get/put passthrough PMU interfaces

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Currently, the guest and host share the PMU resources when a guest is running. KVM has to create an extra virtual event to simulate the guest's event, which brings several issues, e.g., high overhead, not accuracy and etc.

A new passthrough PMU method is proposed to address the issue. It requires that the PMU resources can be fully occupied by the guest while it's running. Two new interfaces are implemented to fulfill the requirement. The hypervisor should invoke the interface while creating a guest which wants the passthrough PMU capability.

The PMU resources should only be temporarily occupied as a whole when a guest is running. When the guest is out, the PMU resources are still shared among different users.

The exclude_guest event modifier is used to guarantee the exclusive occupation of the PMU resources. When creating a guest, the hypervisor should check whether there are !exclude_guest events in the system. If yes, the creation should fail. Because some PMU resources have been occupied by other users.
If no, the PMU resources can be safely accessed by the guest directly. Perf guarantees that no new !exclude_guest events are created when a guest is running.

Only the passthrough PMU is affected, but not for other PMU e.g., uncore and SW PMU. The behavior of those PMUs are not changed. The guest enter/exit interfaces should only impact the supported PMUs. Add a new PERF_PMU_CAP_PASSTHROUGH_VPMU flag to indicate the PMUs that support the feature.

Add nr_include_guest_events to track the !exclude_guest system-wide event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU.

---
 include/linux/perf_event.h |  9 +++++
 kernel/events/core.c       | 67 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
include/linux/perf_event.h +9 -0
index d2a15c0c6f8a..dd4920bf3d1b 100644
@@ -291,6 +291,7 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_NO_EXCLUDE			0x0040
 #define PERF_PMU_CAP_AUX_OUTPUT			0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE		0x0100
+#define PERF_PMU_CAP_PASSTHROUGH_VPMU		0x0200
 
 struct perf_output_handle;
 
@@ -1731,6 +1732,8 @@ extern void perf_event_task_tick(void);
 extern int perf_event_account_interrupt(struct perf_event *event);
 extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
+extern int perf_get_mediated_pmu(void);
+extern void perf_put_mediated_pmu(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1817,6 +1820,12 @@ static inline u64 perf_event_pause(struct perf_event *event, bool reset)
 {
 	return 0;
 }
+static inline int perf_get_mediated_pmu(void)
+{
+	return 0;
+}
+
+static inline void perf_put_mediated_pmu(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +67 -0

96 lines not on this page · raw message

Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#7 lore raw

Re: [PATCH v2 06/54] perf: Support get/put passthrough PMU interfaces

Quoted 8 lines · Mingwei Zhang
kernel/events/core.c +8 -0
+int perf_get_mediated_pmu(void)
+{
+	int ret = 0;
+
+	mutex_lock(&perf_mediated_pmu_mutex);
+	if (refcount_inc_not_zero(&nr_mediated_pmu_vms))
+		goto end;
+

kernel/events/core.c u64 perf_event_pause(struct perf_event *event, bool reset) · line 5785

Quoted 10 lines
kernel/events/core.c +10 -0
+	if (atomic_read(&nr_include_guest_events)) {
+		ret = -EBUSY;
+		goto end;
+	}
+	refcount_set(&nr_mediated_pmu_vms, 1);
+end:
+	mutex_unlock(&perf_mediated_pmu_mutex);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);

Blergh... it seems I never got my perf guard patches merged :/, but could we please write this like:

int perf_get_mediated_pmu(void)

{
	guard(mutex)(&perf_mediated_pmu_mutex);
	if (refcount_inc_not_zero(&nr_mediated_pmu_vms))
		return 0;

	if (atomic_read(&nr_include_guest_events))
		return -EBUSY;

	refcount_set(&nr_mediated_pmu_vms, 1);
	return 0;
}

And avoid adding more unlock goto thingies?

#8 lore raw

Re: [PATCH v2 06/54] perf: Support get/put passthrough PMU interfaces

Quoted 38 lines · Peter Zijlstra
Quoted 8 lines · Mingwei Zhang
kernel/events/core.c +8 -0
+int perf_get_mediated_pmu(void)
+{
+	int ret = 0;
+
+	mutex_lock(&perf_mediated_pmu_mutex);
+	if (refcount_inc_not_zero(&nr_mediated_pmu_vms))
+		goto end;
+

kernel/events/core.c u64 perf_event_pause(struct perf_event *event, bool reset) · line 5785

Quoted 10 lines
kernel/events/core.c +10 -0
+	if (atomic_read(&nr_include_guest_events)) {
+		ret = -EBUSY;
+		goto end;
+	}
+	refcount_set(&nr_mediated_pmu_vms, 1);
+end:
+	mutex_unlock(&perf_mediated_pmu_mutex);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);

Blergh... it seems I never got my perf guard patches merged :/, but could we please write this like:

int perf_get_mediated_pmu(void)

{
	guard(mutex)(&perf_mediated_pmu_mutex);
	if (refcount_inc_not_zero(&nr_mediated_pmu_vms))
		return 0;

	if (atomic_read(&nr_include_guest_events))
		return -EBUSY;

	refcount_set(&nr_mediated_pmu_vms, 1);
	return 0;
}
Quoted 2 lines

And avoid adding more unlock goto thingies?
yes, this works. And code is cleaner.

thanks

maintainer
#9 lore raw

Re: [PATCH v2 06/54] perf: Support get/put passthrough PMU interfaces

In reply to Mingwei Zhang
Quoted 13 lines · Mingwei Zhang
kernel/events/core.c +13 -0
+int perf_get_mediated_pmu(void)
+{
+	int ret = 0;
+
+	mutex_lock(&perf_mediated_pmu_mutex);
+	if (refcount_inc_not_zero(&nr_mediated_pmu_vms))
+		goto end;
+
+	if (atomic_read(&nr_include_guest_events)) {
+		ret = -EBUSY;
+		goto end;
+	}
+	refcount_set(&nr_mediated_pmu_vms, 1);

kernel/events/core.c u64 perf_event_pause(struct perf_event *event, bool reset) · line 5790

Quoted 10 lines
kernel/events/core.c +10 -0
+end:
+	mutex_unlock(&perf_mediated_pmu_mutex);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);
+
+void perf_put_mediated_pmu(void)
+{
+	if (!refcount_dec_not_one(&nr_mediated_pmu_vms))
+		refcount_set(&nr_mediated_pmu_vms, 0);

I'm sorry, but this made the WTF'o'meter go 'ding'.

Isn't that simply refcount_dec() ?

Quoted 10 lines
kernel/events/core.c +3 -0
+}
+EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block
@@ -12086,11 +12140,24 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	if (err)
 		goto err_callchain_buffer;
 

kernel/events/core.c perf_event_alloc(struct perf_event_attr *attr, int cpu, · line 12152

Quoted 10 lines
kernel/events/core.c +10 -0
+	if (is_include_guest_event(event)) {
+		mutex_lock(&perf_mediated_pmu_mutex);
+		if (refcount_read(&nr_mediated_pmu_vms)) {
+			mutex_unlock(&perf_mediated_pmu_mutex);
+			err = -EACCES;
+			goto err_security_alloc;
+		}
+		atomic_inc(&nr_include_guest_events);
+		mutex_unlock(&perf_mediated_pmu_mutex);
+	}

Wouldn't all that be nicer with a helper function?

	if (is_include_guest_event() && !perf_get_guest_event())
		goto err_security_alloc;
Quoted 14 lines
kernel/events/core.c +3 -0
+
 	/* symmetric to unaccount_event() in _free_event() */
 	account_event(event);
 
 	return event;
 
+err_security_alloc:
+	security_perf_event_free(event);
 err_callchain_buffer:
 	if (!event->parent) {
 		if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#10 lore raw

Re: [PATCH v2 06/54] perf: Support get/put passthrough PMU interfaces

Quoted 28 lines · Peter Zijlstra
Quoted 13 lines · Mingwei Zhang
kernel/events/core.c +13 -0
+int perf_get_mediated_pmu(void)
+{
+	int ret = 0;
+
+	mutex_lock(&perf_mediated_pmu_mutex);
+	if (refcount_inc_not_zero(&nr_mediated_pmu_vms))
+		goto end;
+
+	if (atomic_read(&nr_include_guest_events)) {
+		ret = -EBUSY;
+		goto end;
+	}
+	refcount_set(&nr_mediated_pmu_vms, 1);

kernel/events/core.c u64 perf_event_pause(struct perf_event *event, bool reset) · line 5790

Quoted 10 lines
kernel/events/core.c +10 -0
+end:
+	mutex_unlock(&perf_mediated_pmu_mutex);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);
+
+void perf_put_mediated_pmu(void)
+{
+	if (!refcount_dec_not_one(&nr_mediated_pmu_vms))
+		refcount_set(&nr_mediated_pmu_vms, 0);

I'm sorry, but this made the WTF'o'meter go 'ding'.

Quoted 1 line

Isn't that simply refcount_dec() ?

when nr_mediated_pmu_vms is 1, refcount_dec(&nr_mediated_pmu_vms) has an error and call trace: refcount_t: decrement hit 0; leaking memory.

Similar when nr_mediated_pmu_vms is 0, refcount_inc(&nr_mediated_pmu_vms) has an error and call trace also: refcount_t: addition on 0; use_after_free.

it seems refcount_set() should be used to set 1 or 0 to refcount_t.

Quoted 11 lines
Quoted 11 lines
kernel/events/core.c +3 -0
+}
+EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block
@@ -12086,11 +12140,24 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	if (err)
 		goto err_callchain_buffer;
 
Quoted 12 lines

kernel/events/core.c perf_event_alloc(struct perf_event_attr *attr, int cpu, · line 12152

Quoted 10 lines
kernel/events/core.c +10 -0
+	if (is_include_guest_event(event)) {
+		mutex_lock(&perf_mediated_pmu_mutex);
+		if (refcount_read(&nr_mediated_pmu_vms)) {
+			mutex_unlock(&perf_mediated_pmu_mutex);
+			err = -EACCES;
+			goto err_security_alloc;
+		}
+		atomic_inc(&nr_include_guest_events);
+		mutex_unlock(&perf_mediated_pmu_mutex);
+	}

Wouldn't all that be nicer with a helper function?

yes, it is nicer.

thanks

Quoted 19 lines
	if (is_include_guest_event() && !perf_get_guest_event())
		goto err_security_alloc;
Quoted 14 lines
kernel/events/core.c +3 -0
+
 	/* symmetric to unaccount_event() in _free_event() */
 	account_event(event);
 
 	return event;
 
+err_security_alloc:
+	security_perf_event_free(event);
 err_callchain_buffer:
 	if (!event->parent) {
 		if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#11 lore raw

Re: [PATCH v2 06/54] perf: Support get/put passthrough PMU interfaces

Quoted 10 lines · Xiong Y
Quoted 9 lines · Peter Zijlstra

kernel/events/core.c u64 perf_event_pause(struct perf_event *event, bool reset) · line 5790

Quoted 4 lines · Mingwei Zhang
kernel/events/core.c +4 -0
+void perf_put_mediated_pmu(void)
+{
+	if (!refcount_dec_not_one(&nr_mediated_pmu_vms))
+		refcount_set(&nr_mediated_pmu_vms, 0);

I'm sorry, but this made the WTF'o'meter go 'ding'.

Quoted 8 lines
Quoted 1 line

Isn't that simply refcount_dec() ?

when nr_mediated_pmu_vms is 1, refcount_dec(&nr_mediated_pmu_vms) has an error and call trace: refcount_t: decrement hit 0; leaking memory.

Similar when nr_mediated_pmu_vms is 0, refcount_inc(&nr_mediated_pmu_vms) has an error and call trace also: refcount_t: addition on 0; use_after_free.

it seems refcount_set() should be used to set 1 or 0 to refcount_t.

Ah, yes, you need refcount_dec_and_test() in order to free. But if this is the case, then you simply shouldn't be using refcount.

#12 lore raw

PATCH v2 7/54 perf: Add generic exclude_guest support

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem for the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to a exclude_guest event when a guest is running (Intel way), or set the correspoinding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the passthrough vPMU introduced. A guest owns all the PMU counters when it's running. Host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwrite.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

Expose two interfaces to KVM. The KVM should notify the perf when entering/exiting a guest.

It's possible that a exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

---
 include/linux/perf_event.h |   4 ++
 kernel/events/core.c       | 104 +++++++++++++++++++++++++++++++++++++
 2 files changed, 108 insertions(+)
include/linux/perf_event.h +4 -0
index dd4920bf3d1b..acf16676401a 100644
@@ -1734,6 +1734,8 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 extern int perf_get_mediated_pmu(void);
 extern void perf_put_mediated_pmu(void);
+void perf_guest_enter(void);
+void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1826,6 +1828,8 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)			{ }
+static inline void perf_guest_enter(void)			{ }
+static inline void perf_guest_exit(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +104 -0

122 lines not on this page · raw message

Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#13 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 87 lines · Mingwei Zhang
kernel/events/core.c +83 -0
@@ -5791,6 +5801,100 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_sched_out_exclude_guest(struct perf_event_context *ctx)
+{
+	struct perf_event_pmu_context *pmu_ctx;
+
+	update_context_time(ctx);
+	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
+		struct perf_event *event, *tmp;
+		struct pmu *pmu = pmu_ctx->pmu;
+
+		if (!(pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+			continue;
+
+		perf_pmu_disable(pmu);
+
+		/*
+		 * All active events must be exclude_guest events.
+		 * See perf_get_mediated_pmu().
+		 * Unconditionally remove all active events.
+		 */
+		list_for_each_entry_safe(event, tmp, &pmu_ctx->pinned_active, active_list)
+			group_sched_out(event, pmu_ctx->ctx);
+
+		list_for_each_entry_safe(event, tmp, &pmu_ctx->flexible_active, active_list)
+			group_sched_out(event, pmu_ctx->ctx);
+
+		pmu_ctx->rotate_necessary = 0;
+
+		perf_pmu_enable(pmu);
+	}
+}
+
+/* When entering a guest, schedule out all exclude_guest events. */
+void perf_guest_enter(void)
+{
+	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+	lockdep_assert_irqs_disabled();
+
+	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+	if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest))) {
+		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+		return;
+	}
+
+	perf_sched_out_exclude_guest(&cpuctx->ctx);
+	if (cpuctx->task_ctx)
+		perf_sched_out_exclude_guest(cpuctx->task_ctx);
+
+	__this_cpu_write(perf_in_guest, true);
+
+	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}
+
+static void perf_sched_in_exclude_guest(struct perf_event_context *ctx)
+{
+	struct perf_event_pmu_context *pmu_ctx;
+
+	update_context_time(ctx);
+	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
+		struct pmu *pmu = pmu_ctx->pmu;
+
+		if (!(pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+			continue;
+
+		perf_pmu_disable(pmu);
+		pmu_groups_sched_in(ctx, &ctx->pinned_groups, pmu);
+		pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+		perf_pmu_enable(pmu);
+	}
+}
+
+void perf_guest_exit(void)
+{
+	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+	lockdep_assert_irqs_disabled();
+
+	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest))) {
+		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+		return;

kernel/events/core.c void perf_put_mediated_pmu(void) · line 5896

Quoted 10 lines
kernel/events/core.c +10 -0
+	}
+
+	__this_cpu_write(perf_in_guest, false);
+
+	perf_sched_in_exclude_guest(&cpuctx->ctx);
+	if (cpuctx->task_ctx)
+		perf_sched_in_exclude_guest(cpuctx->task_ctx);
+
+	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}

Bah, this is a ton of copy-paste from the normal scheduling code with random changes. Why ?

Why can't this use ctx_sched_{in,out}() ? Surely the whole CAP_PASSTHROUGHT thing is but a flag away.

#14 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 103 lines · Peter Zijlstra
Quoted 87 lines · Mingwei Zhang
(quoted diff) +83 -0
@@ -5791,6 +5801,100 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_sched_out_exclude_guest(struct perf_event_context *ctx)
+{
+	struct perf_event_pmu_context *pmu_ctx;
+
+	update_context_time(ctx);
+	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
+		struct perf_event *event, *tmp;
+		struct pmu *pmu = pmu_ctx->pmu;
+
+		if (!(pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+			continue;
+
+		perf_pmu_disable(pmu);
+
+		/*
+		 * All active events must be exclude_guest events.
+		 * See perf_get_mediated_pmu().
+		 * Unconditionally remove all active events.
+		 */
+		list_for_each_entry_safe(event, tmp, &pmu_ctx->pinned_active, active_list)
+			group_sched_out(event, pmu_ctx->ctx);
+
+		list_for_each_entry_safe(event, tmp, &pmu_ctx->flexible_active, active_list)
+			group_sched_out(event, pmu_ctx->ctx);
+
+		pmu_ctx->rotate_necessary = 0;
+
+		perf_pmu_enable(pmu);
+	}
+}
+
+/* When entering a guest, schedule out all exclude_guest events. */
+void perf_guest_enter(void)
+{
+	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+	lockdep_assert_irqs_disabled();
+
+	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+	if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest))) {
+		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+		return;
+	}
+
+	perf_sched_out_exclude_guest(&cpuctx->ctx);
+	if (cpuctx->task_ctx)
+		perf_sched_out_exclude_guest(cpuctx->task_ctx);
+
+	__this_cpu_write(perf_in_guest, true);
+
+	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}
+
+static void perf_sched_in_exclude_guest(struct perf_event_context *ctx)
+{
+	struct perf_event_pmu_context *pmu_ctx;
+
+	update_context_time(ctx);
+	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
+		struct pmu *pmu = pmu_ctx->pmu;
+
+		if (!(pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+			continue;
+
+		perf_pmu_disable(pmu);
+		pmu_groups_sched_in(ctx, &ctx->pinned_groups, pmu);
+		pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+		perf_pmu_enable(pmu);
+	}
+}
+
+void perf_guest_exit(void)
+{
+	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+	lockdep_assert_irqs_disabled();
+
+	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest))) {
+		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+		return;

Review void perf_put_mediated_pmu(void) · line 5896

Quoted 10 lines
(quoted diff) +10 -0
+	}
+
+	__this_cpu_write(perf_in_guest, false);
+
+	perf_sched_in_exclude_guest(&cpuctx->ctx);
+	if (cpuctx->task_ctx)
+		perf_sched_in_exclude_guest(cpuctx->task_ctx);
+
+	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}

Bah, this is a ton of copy-paste from the normal scheduling code with random changes. Why ?

Quoted 3 lines

Why can't this use ctx_sched_{in,out}() ? Surely the whole CAP_PASSTHROUGHT thing is but a flag away.

Not just a flag. The time has to be updated as well, since the ctx->time is shared among PMUs. Perf shouldn't stop it while other PMUs is still running.

A timeguest will be introduced to track the start time of a guest. The event->tstamp of an exclude_guest event should always keep ctx->timeguest while a guest is running.
When a guest is leaving, update the event->tstamp to now, so the guest time can be deducted.

The below patch demonstrate how the timeguest works. (It's an incomplete patch. Just to show the idea.)

include/linux/perf_event.h +1 -0
index 22d3e56682c9..2134e6886e22 100644
@@ -953,6 +953,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
kernel/events/core.c +0 -0
index 14fd881e3e1d..2aed56671a24 100644
@@ -690,12 +690,31 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin
*running += delta;
}

+static void perf_event_update_time_guest(struct perf_event *event) +{
+ /*
+ * If a guest is running, use the timestamp while entering the guest.
+ * If the guest is leaving, reset the event timestamp.
+ */
+ if (!__this_cpu_read(perf_in_guest))
+ event->tstamp = event->ctx->time;
+ else
+ event->tstamp = event->ctx->timeguest;
+}
+
static void perf_event_update_time(struct perf_event *event) {
- u64 now = perf_event_time(event);
+ u64 now;
+
+ /* Never count the time of an active guest into an exclude_guest event. */
+ if (event->ctx->timeguest &&
+ event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU)
+ return perf_event_update_time_guest(event);

+ now = perf_event_time(event);
__perf_update_times(event, now, &event->total_time_enabled,
&event->total_time_running);
+
event->tstamp = now;
}

@@ -3398,7 +3417,14 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
cpuctx->task_ctx = NULL;
}

- is_active ^= ctx->is_active; /* changed bits */
+ if (event_type & EVENT_GUEST) {
+ /*
+ * Schedule out all !exclude_guest events of PMU
+ * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+ */
+ is_active = EVENT_ALL;
+ } else
+ is_active ^= ctx->is_active; /* changed bits */

list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3998,7 +4024,20 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
WARN_ON_ONCE(cpuctx->task_ctx != ctx);
}

- is_active ^= ctx->is_active; /* changed bits */
+ if (event_type & EVENT_GUEST) {
+ /*
+ * Schedule in all !exclude_guest events of PMU
+ * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+ */
+ is_active = EVENT_ALL;
+
+ /*
+ * Update ctx time to set the new start time for
+ * the exclude_guest events.
+ */
+ update_context_time(ctx);
+ } else
+ is_active ^= ctx->is_active; /* changed bits */

/*
* First go through the list and put on any pinned groups

@@ -5894,14 +5933,18 @@ void perf_guest_enter(u32 guest_lvtpc)
}

perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
- ctx_sched_out(&cpuctx->ctx, EVENT_ALL | EVENT_GUEST);
+ ctx_sched_out(&cpuctx->ctx, EVENT_GUEST);
+ /* Set the guest start time */
+ cpuctx->ctx.timeguest = cpuctx->ctx.time;
perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
if (cpuctx->task_ctx) {
perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
- task_ctx_sched_out(cpuctx->task_ctx, EVENT_ALL | EVENT_GUEST);
+ task_ctx_sched_out(cpuctx->task_ctx, EVENT_GUEST);
+ cpuctx->task_ctx->timeguest = cpuctx->task_ctx->time;
perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
}

__this_cpu_write(perf_in_guest, true);
@@ -5925,14 +5968,17 @@ void perf_guest_exit(void)

__this_cpu_write(perf_in_guest, false);

perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
- ctx_sched_in(&cpuctx->ctx, EVENT_ALL | EVENT_GUEST);
+ ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
+ cpuctx->ctx.timeguest = 0;
perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
if (cpuctx->task_ctx) {
perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
- ctx_sched_in(cpuctx->task_ctx, EVENT_ALL | EVENT_GUEST);
+ ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
+ cpuctx->task_ctx->timeguest = 0;
perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
}

Thanks
Kan

maintainer
#15 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 5 lines · Kan
Quoted 3 lines · Peter Zijlstra

Bah, this is a ton of copy-paste from the normal scheduling code with random changes. Why ?

Quoted 7 lines
Quoted 3 lines

Why can't this use ctx_sched_{in,out}() ? Surely the whole CAP_PASSTHROUGHT thing is but a flag away.

Not just a flag. The time has to be updated as well, since the ctx->time is shared among PMUs. Perf shouldn't stop it while other PMUs is still running.

Obviously the original changelog didn't mention any of that.... :/

Quoted 31 lines

A timeguest will be introduced to track the start time of a guest. The event->tstamp of an exclude_guest event should always keep ctx->timeguest while a guest is running.
When a guest is leaving, update the event->tstamp to now, so the guest time can be deducted.

The below patch demonstrate how the timeguest works. (It's an incomplete patch. Just to show the idea.)

include/linux/perf_event.h +1 -0
index 22d3e56682c9..2134e6886e22 100644
@@ -953,6 +953,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
kernel/events/core.c +0 -0
index 14fd881e3e1d..2aed56671a24 100644
@@ -690,12 +690,31 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

kernel/events/core.c +1 -0
 		*running += delta;
 }
 
+static void perf_event_update_time_guest(struct perf_event *event)

kernel/events/core.c

Quoted 10 lines
kernel/events/core.c +10 -0
+{
+	/*
+	 * If a guest is running, use the timestamp while entering the guest.
+	 * If the guest is leaving, reset the event timestamp.
+	 */
+	if (!__this_cpu_read(perf_in_guest))
+		event->tstamp = event->ctx->time;
+	else
+		event->tstamp = event->ctx->timeguest;
+}

This conditional seems inverted, without a good reason. Also, in another thread you talk about some PMUs stopping time in a guest, while other PMUs would keep ticking. I don't think the above captures that.

kernel/events/core.c

Quoted 9 lines

static void perf_event_update_time(struct perf_event *event) {

kernel/events/core.c +6 -1
-	u64 now = perf_event_time(event);
+	u64 now;
+
+	/* Never count the time of an active guest into an exclude_guest event. */
+	if (event->ctx->timeguest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU)
+		return perf_event_update_time_guest(event);

Urgh, weird split. The PMU check is here. Please just inline the above here, this seems to be the sole caller anyway.

Quoted 9 lines
kernel/events/core.c +2 -0
+	now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
+
 	event->tstamp = now;
 }

kernel/events/core.c ctx_sched_out(struct perf_event_context *ctx, · line 3421

Quoted 6 lines
kernel/events/core.c +1 -1
@@ -3398,7 +3417,14 @@ ctx_sched_out(struct perf_event_context *ctx,
 enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {

Patch doesn't introduce EVENT_GUEST, lost a hunk somewhere?

Quoted 35 lines
kernel/events/core.c +7 -0
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+	} else
+		is_active ^= ctx->is_active; /* changed bits */
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
kernel/events/core.c +5 -3
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -5894,14 +5933,18 @@ void perf_guest_enter(u32 guest_lvtpc)
 	}
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
-	ctx_sched_out(&cpuctx->ctx, EVENT_ALL | EVENT_GUEST);
+	ctx_sched_out(&cpuctx->ctx, EVENT_GUEST);
+	/* Set the guest start time */
+	cpuctx->ctx.timeguest = cpuctx->ctx.time;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
-		task_ctx_sched_out(cpuctx->task_ctx, EVENT_ALL | EVENT_GUEST);
+		task_ctx_sched_out(cpuctx->task_ctx, EVENT_GUEST);
+		cpuctx->task_ctx->timeguest = cpuctx->task_ctx->time;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
 	__this_cpu_write(perf_in_guest, true);
@@ -5925,14 +5968,17 @@ void perf_guest_exit(void)
 
 	__this_cpu_write(perf_in_guest, false);
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
-	ctx_sched_in(&cpuctx->ctx, EVENT_ALL | EVENT_GUEST);

kernel/events/core.c void perf_guest_exit(void) · line 5980

Quoted 10 lines
kernel/events/core.c +4 -1
+	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
+	cpuctx->ctx.timeguest = 0;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
-		ctx_sched_in(cpuctx->task_ctx, EVENT_ALL | EVENT_GUEST);
+		ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
+		cpuctx->task_ctx->timeguest = 0;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}

I'm thinking EVENT_GUEST should cause the ->timeguest updates, no point in having them explicitly duplicated here, hmm?

#16 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 10 lines · Peter Zijlstra
Quoted 9 lines · Kan
Quoted 6 lines · Peter Zijlstra

Bah, this is a ton of copy-paste from the normal scheduling code with random changes. Why ?

Why can't this use ctx_sched_{in,out}() ? Surely the whole CAP_PASSTHROUGHT thing is but a flag away.

Quoted 5 lines
Quoted 3 lines

Not just a flag. The time has to be updated as well, since the ctx->time is shared among PMUs. Perf shouldn't stop it while other PMUs is still running.

Obviously the original changelog didn't mention any of that.... :/

Yes, the time issue was a newly found bug when we test the uncore PMUs.

Quoted 47 lines
Quoted 32 lines

A timeguest will be introduced to track the start time of a guest. The event->tstamp of an exclude_guest event should always keep ctx->timeguest while a guest is running.
When a guest is leaving, update the event->tstamp to now, so the guest time can be deducted.

The below patch demonstrate how the timeguest works. (It's an incomplete patch. Just to show the idea.)

include/linux/perf_event.h +1 -0
index 22d3e56682c9..2134e6886e22 100644
@@ -953,6 +953,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
kernel/events/core.c +0 -0
index 14fd881e3e1d..2aed56671a24 100644
@@ -690,12 +690,31 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

kernel/events/core.c +1 -0
 		*running += delta;
 }
 
+static void perf_event_update_time_guest(struct perf_event *event)

kernel/events/core.c

Quoted 10 lines
kernel/events/core.c +10 -0
+{
+	/*
+	 * If a guest is running, use the timestamp while entering the guest.
+	 * If the guest is leaving, reset the event timestamp.
+	 */
+	if (!__this_cpu_read(perf_in_guest))
+		event->tstamp = event->ctx->time;
+	else
+		event->tstamp = event->ctx->timeguest;
+}

This conditional seems inverted, without a good reason. Also, in another thread you talk about some PMUs stopping time in a guest, while other PMUs would keep ticking. I don't think the above captures that.

Quoted 13 lines

kernel/events/core.c

Quoted 9 lines

static void perf_event_update_time(struct perf_event *event) {

kernel/events/core.c +6 -1
-	u64 now = perf_event_time(event);
+	u64 now;
+
+	/* Never count the time of an active guest into an exclude_guest event. */
+	if (event->ctx->timeguest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU)
+		return perf_event_update_time_guest(event);

Urgh, weird split. The PMU check is here. Please just inline the above here, this seems to be the sole caller anyway.

Sure

Quoted 13 lines
Quoted 13 lines
kernel/events/core.c +2 -0
+	now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
+
 	event->tstamp = now;
 }
@@ -3398,7 +3417,14 @@ ctx_sched_out(struct perf_event_context *ctx,
 enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
Quoted 4 lines

kernel/events/core.c ctx_sched_out(struct perf_event_context *ctx, · line 3421

Quoted 2 lines
kernel/events/core.c +1 -1
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {

Patch doesn't introduce EVENT_GUEST, lost a hunk somewhere?

Sorry, there is a prerequisite patch to factor out the EVENT_CGROUP. I thought it will be complex and confusion to paste both. Some details are lost.
I will post both two patches at the end.

Quoted 34 lines
Quoted 34 lines
kernel/events/core.c +7 -0
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+	} else
+		is_active ^= ctx->is_active; /* changed bits */
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
kernel/events/core.c +5 -2
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -5894,14 +5933,18 @@ void perf_guest_enter(u32 guest_lvtpc)
 	}
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
-	ctx_sched_out(&cpuctx->ctx, EVENT_ALL | EVENT_GUEST);
+	ctx_sched_out(&cpuctx->ctx, EVENT_GUEST);
+	/* Set the guest start time */
+	cpuctx->ctx.timeguest = cpuctx->ctx.time;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
-		task_ctx_sched_out(cpuctx->task_ctx, EVENT_ALL | EVENT_GUEST);
+		task_ctx_sched_out(cpuctx->task_ctx, EVENT_GUEST);
+		cpuctx->task_ctx->timeguest = cpuctx->task_ctx->time;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
 	__this_cpu_write(perf_in_guest, true);
@@ -5925,14 +5968,17 @@ void perf_guest_exit(void)
 
 	__this_cpu_write(perf_in_guest, false);
 
Quoted 15 lines

kernel/events/core.c void perf_guest_exit(void) · line 5980

Quoted 12 lines
kernel/events/core.c +4 -2
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
-	ctx_sched_in(&cpuctx->ctx, EVENT_ALL | EVENT_GUEST);
+	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
+	cpuctx->ctx.timeguest = 0;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
-		ctx_sched_in(cpuctx->task_ctx, EVENT_ALL | EVENT_GUEST);
+		ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
+		cpuctx->task_ctx->timeguest = 0;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}

I'm thinking EVENT_GUEST should cause the ->timeguest updates, no point in having them explicitly duplicated here, hmm?

We have to add a EVENT_GUEST check and update the ->timeguest at the end of the ctx_sched_out/in functions after the pmu_ctx_sched_out/in(). Because the ->timeguest also be used to check if it's leaving the guest in the perf_event_update_time().

Since the EVENT_GUEST only be used by perf_guest_enter/exit(), I thought maybe it's better to move it to where it is used rather than the generic ctx_sched_out/in(). It will minimize the impact on the non-virtualization user.

Here are the two complete patches.

The first one is to factor out the EVENT_CGROUP.

From c508f2b0e11a2eea71fe3ff16d1d848359ede535 Mon Sep 17 00:00:00 2001
From: Kan Liang <[email protected]>
Date: Mon, 27 May 2024 06:58:29 -0700
Subject: [PATCH 1/2] perf: Skip pmu_ctx based on event_type

To optimize the cgroup context switch, the perf_event_pmu_context iteration skips the PMUs without cgroup events. A bool cgroup was introduced to indicate the case. It can work, but this way is hard to extend for other cases, e.g. skipping non-passthrough PMUs. It doesn't make sense to keep adding bool variables.

Pass the event_type instead of the specific bool variable. Check both the event_type and related pmu_ctx variables to decide whether skipping a PMU.

Event flags, e.g., EVENT_CGROUP, should be cleard in the ctx->is_active. Add EVENT_FLAGS to indicate such event flags.

No functional change.

---
 kernel/events/core.c | 70 +++++++++++++++++++++++---------------------
 1 file changed, 37 insertions(+), 33 deletions(-)
kernel/events/core.c +11 -1
index abd4027e3859..95d1d5a5addc 100644
@@ -376,6 +376,7 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
+	EVENT_FLAGS = EVENT_CGROUP,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -699,23 +700,32 @@ do {									\
 	___p;								\
 })
 
-static void perf_ctx_disable(struct perf_event_context *ctx, bool cgroup)
+static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
+			      enum event_type_t event_type)
+{
+	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
+		return true;
+
+	return false;
+}
+
+static void perf_ctx_disable(struct perf_event_context *ctx, enum

event_type_t event_type)
{
struct perf_event_pmu_context *pmu_ctx;

list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
- if (cgroup && !pmu_ctx->nr_cgroups)
+ if (perf_skip_pmu_ctx(pmu_ctx, event_type))
continue;
perf_pmu_disable(pmu_ctx->pmu);
}
}

-static void perf_ctx_enable(struct perf_event_context *ctx, bool cgroup) +static void perf_ctx_enable(struct perf_event_context *ctx, enum event_type_t event_type)
{
struct perf_event_pmu_context *pmu_ctx;

list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
- if (cgroup && !pmu_ctx->nr_cgroups)
+ if (perf_skip_pmu_ctx(pmu_ctx, event_type))
continue;
perf_pmu_enable(pmu_ctx->pmu);
}
@@ -877,7 +887,7 @@ static void perf_cgroup_switch(struct task_struct *task)
return;

perf_ctx_lock(cpuctx, cpuctx->task_ctx);
- perf_ctx_disable(&cpuctx->ctx, true);
+ perf_ctx_disable(&cpuctx->ctx, EVENT_CGROUP);

ctx_sched_out(&cpuctx->ctx, EVENT_ALL|EVENT_CGROUP);
/*
@@ -893,7 +903,7 @@ static void perf_cgroup_switch(struct task_struct *task)
*/
ctx_sched_in(&cpuctx->ctx, EVENT_ALL|EVENT_CGROUP);

- perf_ctx_enable(&cpuctx->ctx, true);
+ perf_ctx_enable(&cpuctx->ctx, EVENT_CGROUP);
perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
}

@@ -2729,9 +2739,9 @@ static void ctx_resched(struct perf_cpu_context *cpuctx,

event_type &= EVENT_ALL;

- perf_ctx_disable(&cpuctx->ctx, false);
+ perf_ctx_disable(&cpuctx->ctx, 0);
if (task_ctx) {
- perf_ctx_disable(task_ctx, false);
+ perf_ctx_disable(task_ctx, 0);
task_ctx_sched_out(task_ctx, event_type);
}

@@ -2749,9 +2759,9 @@ static void ctx_resched(struct perf_cpu_context *cpuctx,

perf_event_sched_in(cpuctx, task_ctx);

- perf_ctx_enable(&cpuctx->ctx, false);
+ perf_ctx_enable(&cpuctx->ctx, 0);
if (task_ctx)
- perf_ctx_enable(task_ctx, false);
+ perf_ctx_enable(task_ctx, 0);
}

void perf_pmu_resched(struct pmu *pmu)
@@ -3296,9 +3306,6 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context); struct perf_event_pmu_context *pmu_ctx;
int is_active = ctx->is_active;
- bool cgroup = event_type & EVENT_CGROUP;
-
- event_type &= ~EVENT_CGROUP;

lockdep_assert_held(&ctx->lock);

@@ -3333,7 +3340,7 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
barrier();
}

- ctx->is_active &= ~event_type;
+ ctx->is_active &= ~(event_type & ~EVENT_FLAGS);
if (!(ctx->is_active & EVENT_ALL))
ctx->is_active = 0;

@@ -3346,7 +3353,7 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
is_active ^= ctx->is_active; /* changed bits */

list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
- if (cgroup && !pmu_ctx->nr_cgroups)
+ if (perf_skip_pmu_ctx(pmu_ctx, event_type))
continue;
__pmu_ctx_sched_out(pmu_ctx, is_active);
}
@@ -3540,7 +3547,7 @@ perf_event_context_sched_out(struct task_struct *task, struct task_struct *next)
raw_spin_lock_nested(&next_ctx->lock, SINGLE_DEPTH_NESTING); if (context_equiv(ctx, next_ctx)) {

- perf_ctx_disable(ctx, false);
+ perf_ctx_disable(ctx, 0);

/* PMIs are disabled; ctx->nr_pending is stable. */ if (local_read(&ctx->nr_pending) ||
@@ -3560,7 +3567,7 @@ perf_event_context_sched_out(struct task_struct *task, struct task_struct *next)
perf_ctx_sched_task_cb(ctx, false); perf_event_swap_task_ctx_data(ctx, next_ctx);

- perf_ctx_enable(ctx, false);
+ perf_ctx_enable(ctx, 0);

/*
* RCU_INIT_POINTER here is safe because we've not
@@ -3584,13 +3591,13 @@ perf_event_context_sched_out(struct task_struct *task, struct task_struct *next)

if (do_switch) {
raw_spin_lock(&ctx->lock);
- perf_ctx_disable(ctx, false);
+ perf_ctx_disable(ctx, 0);

inside_switch:
perf_ctx_sched_task_cb(ctx, false); task_ctx_sched_out(ctx, EVENT_ALL);

- perf_ctx_enable(ctx, false);
+ perf_ctx_enable(ctx, 0);
raw_spin_unlock(&ctx->lock);
}
}
@@ -3887,12 +3894,12 @@ static void pmu_groups_sched_in(struct perf_event_context *ctx,

static void ctx_groups_sched_in(struct perf_event_context *ctx,
struct perf_event_groups *groups,
- bool cgroup)
+ enum event_type_t event_type)
{
struct perf_event_pmu_context *pmu_ctx;

list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
- if (cgroup && !pmu_ctx->nr_cgroups)
+ if (perf_skip_pmu_ctx(pmu_ctx, event_type))
continue;
pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
}
@@ -3909,9 +3916,6 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
{
struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context); int is_active = ctx->is_active;
- bool cgroup = event_type & EVENT_CGROUP;
-
- event_type &= ~EVENT_CGROUP;

lockdep_assert_held(&ctx->lock);

@@ -3929,7 +3933,7 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
barrier();
}

- ctx->is_active |= (event_type | EVENT_TIME);
+ ctx->is_active |= ((event_type & ~EVENT_FLAGS) | EVENT_TIME);
if (ctx->task) {
if (!is_active)
cpuctx->task_ctx = ctx;
@@ -3944,11 +3948,11 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
* in order to give them the best chance of going on. */
if (is_active & EVENT_PINNED)
- ctx_groups_sched_in(ctx, &ctx->pinned_groups, cgroup);
+ ctx_groups_sched_in(ctx, &ctx->pinned_groups, event_type);

/* Then walk through the lower prio flexible groups */ if (is_active & EVENT_FLEXIBLE)
- ctx_groups_sched_in(ctx, &ctx->flexible_groups, cgroup);
+ ctx_groups_sched_in(ctx, &ctx->flexible_groups, event_type);
}

static void perf_event_context_sched_in(struct task_struct *task)
@@ -3963,11 +3967,11 @@ static void perf_event_context_sched_in(struct task_struct *task)

if (cpuctx->task_ctx == ctx) {
perf_ctx_lock(cpuctx, ctx);
- perf_ctx_disable(ctx, false);
+ perf_ctx_disable(ctx, 0);

perf_ctx_sched_task_cb(ctx, true);

- perf_ctx_enable(ctx, false);
+ perf_ctx_enable(ctx, 0);
perf_ctx_unlock(cpuctx, ctx);
goto rcu_unlock;
}
@@ -3980,7 +3984,7 @@ static void perf_event_context_sched_in(struct task_struct *task)
if (!ctx->nr_events)
goto unlock;

- perf_ctx_disable(ctx, false);
+ perf_ctx_disable(ctx, 0);
/*
* We want to keep the following priority order:
* cpu pinned (that don't need to move), task pinned,
@@ -3990,7 +3994,7 @@ static void perf_event_context_sched_in(struct task_struct *task)
* events, no need to flip the cpuctx's events around. */
if (!RB_EMPTY_ROOT(&ctx->pinned_groups.tree)) {
- perf_ctx_disable(&cpuctx->ctx, false);
+ perf_ctx_disable(&cpuctx->ctx, 0);
ctx_sched_out(&cpuctx->ctx, EVENT_FLEXIBLE);
}

@@ -3999,9 +4003,9 @@ static void perf_event_context_sched_in(struct task_struct *task)
perf_ctx_sched_task_cb(cpuctx->task_ctx, true);

if (!RB_EMPTY_ROOT(&ctx->pinned_groups.tree))
- perf_ctx_enable(&cpuctx->ctx, false);
+ perf_ctx_enable(&cpuctx->ctx, 0);

- perf_ctx_enable(ctx, false);
+ perf_ctx_enable(ctx, 0);

unlock:
perf_ctx_unlock(cpuctx, ctx);

Here are the second patch to introduce EVENT_GUEST and perf_guest_exit/enter().

From c052e673dc3e0e7ebacdce23f2b0d50ec98401b3 Mon Sep 17 00:00:00 2001
From: Kan Liang <[email protected]>
Date: Tue, 11 Jun 2024 06:04:20 -0700
Subject: [PATCH 2/2] perf: Add generic exclude_guest support

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the introduced passthrough vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

Expose two interfaces to KVM. The KVM should notify the perf when entering/exiting a guest.

Introduce a new event type EVENT_CGUEST to indicate that perf should check and skip the PMUs which doesn't support the passthrough mode.

It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

The ctx->time is used to calculated the running/enabling time of an event, which is shared among PMUs. The ctx_sched_in/out() with EVENT_CGUEST doesn't stop the ctx->time. A timeguest is introduced to track the start time of a guest. For an exclude_guest event, the time in the guest mode is deducted.

---
 include/linux/perf_event.h |   5 ++
 kernel/events/core.c       | 119 +++++++++++++++++++++++++++++++++++--
 2 files changed, 120 insertions(+), 4 deletions(-)
include/linux/perf_event.h +1 -0
index dd4920bf3d1b..68c8b93c4e5c 100644
@@ -945,6 +945,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1734,6 +1735,8 @@ extern int perf_event_period(struct perf_event

*event, u64 value);
extern u64 perf_event_pause(struct perf_event *event, bool reset); extern int perf_get_mediated_pmu(void);
extern void perf_put_mediated_pmu(void);
+void perf_guest_enter(void);
+void perf_guest_exit(void);
#else /* !CONFIG_PERF_EVENTS: */
static inline void *
perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1826,6 +1829,8 @@ static inline int perf_get_mediated_pmu(void)
}

static inline void perf_put_mediated_pmu(void) { }
+static inline void perf_guest_enter(void) { }
+static inline void perf_guest_exit(void) { }
#endif

#if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)

kernel/events/core.c +3 -1
index 95d1d5a5addc..cd3a89672b14 100644
@@ -376,7 +376,8 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
-	EVENT_FLAGS = EVENT_CGROUP,
+	EVENT_GUEST = 0x20,
+	EVENT_FLAGS = EVENT_CGROUP | EVENT_GUEST,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -407,6 +408,7 @@ static atomic_t nr_include_guest_events __read_mostly;
 
 static atomic_t nr_mediated_pmu_vms;
 static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+static DEFINE_PER_CPU(bool, perf_in_guest);
 
 /* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
 static inline bool is_include_guest_event(struct perf_event *event)
@@ -651,10 +653,26 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

static void perf_event_update_time(struct perf_event *event) {
- u64 now = perf_event_time(event);
+ u64 now;
+
+ /* Never count the time of an active guest into an exclude_guest event. */
+ if (event->ctx->timeguest &&
+ event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+ /*
+ * If a guest is running, use the timestamp while entering the guest.
+ * If the guest is leaving, reset the event timestamp.
+ */
+ if (__this_cpu_read(perf_in_guest))
+ event->tstamp = event->ctx->timeguest;
+ else
+ event->tstamp = event->ctx->time;
+ return;
+ }

+ now = perf_event_time(event);
__perf_update_times(event, now, &event->total_time_enabled,
&event->total_time_running);
+
event->tstamp = now;
}

@@ -706,6 +724,10 @@ static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
return true;

+ if ((event_type & EVENT_GUEST) &&
+ !(pmu_ctx->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+ return true;
+
return false;
}

@@ -3350,7 +3372,14 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
cpuctx->task_ctx = NULL;
}

- is_active ^= ctx->is_active; /* changed bits */
+ if (event_type & EVENT_GUEST) {
+ /*
+ * Schedule out all !exclude_guest events of PMU
+ * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+ */
+ is_active = EVENT_ALL;
+ } else
+ is_active ^= ctx->is_active; /* changed bits */

list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3860,6 +3889,15 @@ static int merge_sched_in(struct perf_event *event, void *data)
if (!event_filter_match(event))
return 0;

+ /*
+ * Don't schedule in any exclude_guest events of PMU with
+ * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+ */
+ if (__this_cpu_read(perf_in_guest) &&
+ event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+ event->attr.exclude_guest)
+ return 0;
+
if (group_can_go_on(event, *can_add_hw)) {
if (!group_sched_in(event, ctx))
list_add_tail(&event->active_list, get_event_list(event));
@@ -3941,7 +3979,20 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
WARN_ON_ONCE(cpuctx->task_ctx != ctx);
}

- is_active ^= ctx->is_active; /* changed bits */
+ if (event_type & EVENT_GUEST) {
+ /*
+ * Schedule in all !exclude_guest events of PMU
+ * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+ */
+ is_active = EVENT_ALL;
+
+ /*
+ * Update ctx time to set the new start time for
+ * the exclude_guest events.
+ */
+ update_context_time(ctx);
+ } else
+ is_active ^= ctx->is_active; /* changed bits */

/*
* First go through the list and put on any pinned groups
@@ -5788,6 +5839,66 @@ void perf_put_mediated_pmu(void)
}
EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);

+/* When entering a guest, schedule out all exclude_guest events. */ +void perf_guest_enter(void)
+{
+ struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+ lockdep_assert_irqs_disabled();
+
+ perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+ if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest))) {
+ perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+ return;
+ }
+
+ perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
+ ctx_sched_out(&cpuctx->ctx, EVENT_GUEST);
+ /* Set the guest start time */
+ cpuctx->ctx.timeguest = cpuctx->ctx.time;
+ perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
+ if (cpuctx->task_ctx) {
+ perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
+ task_ctx_sched_out(cpuctx->task_ctx, EVENT_GUEST);
+ cpuctx->task_ctx->timeguest = cpuctx->task_ctx->time;
+ perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
+ }
+
+ __this_cpu_write(perf_in_guest, true);
+
+ perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}
+
+void perf_guest_exit(void)
+{
+ struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+ lockdep_assert_irqs_disabled();
+
+ perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+ if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest))) {
+ perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+ return;
+ }
+
+ __this_cpu_write(perf_in_guest, false);
+
+ perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
+ ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
+ cpuctx->ctx.timeguest = 0;
+ perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
+ if (cpuctx->task_ctx) {
+ perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
+ ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
+ cpuctx->task_ctx->timeguest = 0;
+ perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
+ }
+
+ perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}
+
/*
* Holding the top-level event's child_mutex means that any
* descendant process that has inherited this event will block

Thanks,
Kan

maintainer
#17 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 31 lines · Kan
include/linux/perf_event.h +1 -0
index dd4920bf3d1b..68c8b93c4e5c 100644
@@ -945,6 +945,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -651,10 +653,26 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

static void perf_event_update_time(struct perf_event *event) {

include/linux/perf_event.h +12 -1
-	u64 now = perf_event_time(event);
+	u64 now;
+
+	/* Never count the time of an active guest into an exclude_guest event. */
+	if (event->ctx->timeguest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		/*
+		 * If a guest is running, use the timestamp while entering the guest.
+		 * If the guest is leaving, reset the event timestamp.
+		 */
+		if (__this_cpu_read(perf_in_guest))
+			event->tstamp = event->ctx->timeguest;
+		else

include/linux/perf_event.h

Quoted 10 lines
include/linux/perf_event.h +5 -0
+			event->tstamp = event->ctx->time;
+		return;
+	}
 
+	now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
+
 	event->tstamp = now;
 }

So I really don't like this much, and AFAICT this is broken. At the very least this doesn't work right for cgroup events, because they have their own timeline.

Let me have a poke...

#18 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 36 lines · Peter Zijlstra
Quoted 35 lines · Kan
include/linux/perf_event.h +1 -0
index dd4920bf3d1b..68c8b93c4e5c 100644
@@ -945,6 +945,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -651,10 +653,26 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

static void perf_event_update_time(struct perf_event *event) {

include/linux/perf_event.h +15 -1
-	u64 now = perf_event_time(event);
+	u64 now;
+
+	/* Never count the time of an active guest into an exclude_guest event. */
+	if (event->ctx->timeguest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		/*
+		 * If a guest is running, use the timestamp while entering the guest.
+		 * If the guest is leaving, reset the event timestamp.
+		 */
+		if (__this_cpu_read(perf_in_guest))
+			event->tstamp = event->ctx->timeguest;
+		else
+			event->tstamp = event->ctx->time;
+		return;
+	}
Quoted 8 lines

include/linux/perf_event.h

Quoted 6 lines
include/linux/perf_event.h +2 -0
+	now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
+
 	event->tstamp = now;
 }

So I really don't like this much,

An alternative way I can imagine may maintain a dedicated timeline for the PASSTHROUGH PMUs. For that, we probably need two new timelines for the normal events and the cgroup events. That sounds too complex.

Quoted 3 lines

and AFAICT this is broken. At the very
least this doesn't work right for cgroup events, because they have their own timeline.

I think we just need a new start time for an event. So we may use perf_event_time() to replace the ctx->time for the cgroup events.

kernel/events/core.c +0 -0
index 019c237dd456..6c46699c6752 100644
@@ -665,7 +665,7 @@ static void perf_event_update_time(struct perf_event

*event)
if (__this_cpu_read(perf_in_guest))
event->tstamp = event->ctx->timeguest;
else
- event->tstamp = event->ctx->time;
+ event->tstamp = perf_event_time(event);
return;
}

Quoted 2 lines

Let me have a poke...

Sure.

Thanks,
Kan

maintainer
#19 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 46 lines · Kan
Quoted 36 lines · Peter Zijlstra
Quoted 35 lines · Kan
include/linux/perf_event.h +1 -0
index dd4920bf3d1b..68c8b93c4e5c 100644
@@ -945,6 +945,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -651,10 +653,26 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

static void perf_event_update_time(struct perf_event *event) {

include/linux/perf_event.h +15 -1
-	u64 now = perf_event_time(event);
+	u64 now;
+
+	/* Never count the time of an active guest into an exclude_guest event. */
+	if (event->ctx->timeguest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		/*
+		 * If a guest is running, use the timestamp while entering the guest.
+		 * If the guest is leaving, reset the event timestamp.
+		 */
+		if (__this_cpu_read(perf_in_guest))
+			event->tstamp = event->ctx->timeguest;
+		else
+			event->tstamp = event->ctx->time;
+		return;
+	}
Quoted 8 lines

include/linux/perf_event.h

Quoted 6 lines
include/linux/perf_event.h +2 -0
+	now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
+
 	event->tstamp = now;
 }

So I really don't like this much,

Quoted 3 lines

An alternative way I can imagine may maintain a dedicated timeline for the PASSTHROUGH PMUs. For that, we probably need two new timelines for the normal events and the cgroup events. That sounds too complex.

I'm afraid we might have to. Specifically, the below:

kernel/events/core.c static void perf_event_update_time(struct perf_event · line 671

Quoted 13 lines
kernel/events/core.c +1 -1
index 019c237dd456..6c46699c6752 100644
@@ -665,7 +665,7 @@ static void perf_event_update_time(struct perf_event
 *event)
 		if (__this_cpu_read(perf_in_guest))
 			event->tstamp = event->ctx->timeguest;
 		else
-			event->tstamp = event->ctx->time;
+			event->tstamp = perf_event_time(event);
 		return;
 	}

is still broken in that it (ab)uses event state to track time, and this goes sideways in case of event overcommit, because then ctx_sched_{out,in}() will not visit all events.

We've ran into that before. Time-keeping really should be per context or we'll get a ton of pain.

I've ended up with the (uncompiled) below. Yes, it is unfortunate, but aside from a few cleanups (we could introduce a struct time_ctx { u64 time, stamp, offset }; and fold a bunch of code, this is more or less the best we can do I'm afraid.

---

include/linux/perf_event.h +6 -1
@@ -947,7 +947,9 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
-	u64				timeguest;
+	u64				guest_time;
+	u64				guest_timestamp;
+	u64				guest_timeoffset;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1043,6 +1045,9 @@ struct perf_cgroup_info {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				guest_time;
+	u64				guest_timestamp;
+	u64				guest_timeoffset;
 	int				active;
 };
 
kernel/events/core.c +120 -57

347 lines not on this page · raw message

#20 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 71 lines · Peter Zijlstra
Quoted 46 lines · Kan
Quoted 36 lines · Peter Zijlstra
Quoted 35 lines · Kan
include/linux/perf_event.h +1 -0
index dd4920bf3d1b..68c8b93c4e5c 100644
@@ -945,6 +945,7 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -651,10 +653,26 @@ __perf_update_times(struct perf_event *event, u64

now, u64 *enabled, u64 *runnin

static void perf_event_update_time(struct perf_event *event) {

include/linux/perf_event.h +15 -1
-	u64 now = perf_event_time(event);
+	u64 now;
+
+	/* Never count the time of an active guest into an exclude_guest event. */
+	if (event->ctx->timeguest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		/*
+		 * If a guest is running, use the timestamp while entering the guest.
+		 * If the guest is leaving, reset the event timestamp.
+		 */
+		if (__this_cpu_read(perf_in_guest))
+			event->tstamp = event->ctx->timeguest;
+		else
+			event->tstamp = event->ctx->time;
+		return;
+	}
Quoted 8 lines

include/linux/perf_event.h

Quoted 6 lines
include/linux/perf_event.h +2 -0
+	now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
+
 	event->tstamp = now;
 }

So I really don't like this much,

Quoted 3 lines

An alternative way I can imagine may maintain a dedicated timeline for the PASSTHROUGH PMUs. For that, we probably need two new timelines for the normal events and the cgroup events. That sounds too complex.

I'm afraid we might have to. Specifically, the below:

kernel/events/core.c static void perf_event_update_time(struct perf_event · line 671

Quoted 13 lines
kernel/events/core.c +1 -1
index 019c237dd456..6c46699c6752 100644
@@ -665,7 +665,7 @@ static void perf_event_update_time(struct perf_event
 *event)
 		if (__this_cpu_read(perf_in_guest))
 			event->tstamp = event->ctx->timeguest;
 		else
-			event->tstamp = event->ctx->time;
+			event->tstamp = perf_event_time(event);
 		return;
 	}

is still broken in that it (ab)uses event state to track time, and this goes sideways in case of event overcommit, because then ctx_sched_{out,in}() will not visit all events.

Quoted 7 lines

We've ran into that before. Time-keeping really should be per context or we'll get a ton of pain.

I've ended up with the (uncompiled) below. Yes, it is unfortunate, but aside from a few cleanups (we could introduce a struct time_ctx { u64 time, stamp, offset }; and fold a bunch of code, this is more or less the best we can do I'm afraid.

Sure. I will try the below codes and implement the cleanup patch as well.

Thanks,
Kan

Quoted 392 lines

---

include/linux/perf_event.h +6 -1
@@ -947,7 +947,9 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
-	u64				timeguest;
+	u64				guest_time;
+	u64				guest_timestamp;
+	u64				guest_timeoffset;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1043,6 +1045,9 @@ struct perf_cgroup_info {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				guest_time;
+	u64				guest_timestamp;
+	u64				guest_timeoffset;
 	int				active;
 };
 
kernel/events/core.c +120 -57
@@ -638,26 +638,9 @@ __perf_update_times(struct perf_event *e
 
 static void perf_event_update_time(struct perf_event *event)
 {
-	u64 now;
-
-	/* Never count the time of an active guest into an exclude_guest event. */
-	if (event->ctx->timeguest &&
-	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
-		/*
-		 * If a guest is running, use the timestamp while entering the guest.
-		 * If the guest is leaving, reset the event timestamp.
-		 */
-		if (__this_cpu_read(perf_in_guest))
-			event->tstamp = event->ctx->timeguest;
-		else
-			event->tstamp = event->ctx->time;
-		return;
-	}
-
-	now = perf_event_time(event);
+	u64 now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
-
 	event->tstamp = now;
 }
 
@@ -780,19 +763,33 @@ static inline int is_cgroup_event(struct
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
+	u64 time;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time;
+	time = t->time;
+	if (event->attr.exclude_guest)
+		time -= t->guest_time;
+	return time;
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
 {
 	struct perf_cgroup_info *t;
+	u64 time, guest_time;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	if (!__load_acquire(&t->active))
-		return t->time;
-	now += READ_ONCE(t->timeoffset);
+	if (!__load_acquire(&t->active)) {
+		time = t->time;
+		if (event->attr.exclude_guest)
+			time -= t->guest_time;
+		return time;
+	}
+
+	time = now + READ_ONCE(t->timeoffset);
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest)) {
+		guest_time = now + READ_ONCE(t->guest_offset);
+		time -= guest_time;
+	}
 	return now;
 }
 
@@ -807,6 +804,17 @@ static inline void __update_cgrp_time(st
 	WRITE_ONCE(info->timeoffset, info->time - info->timestamp);
 }
 
+static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info, u64 now, bool adv)
+{
+	if (adv)
+		info->guest_time += now - info->guest_timestamp;
+	info->guest_timestamp = now;
+	/*
+	 * see update_context_time()
+	 */
+	WRITE_ONCE(info->guest_timeoffset, info->guest_time - info->guest_timestamp);
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -821,6 +829,8 @@ static inline void update_cgrp_time_from
 			info = this_cpu_ptr(cgrp->info);
 
 			__update_cgrp_time(info, now, true);
+			if (__this_cpu_read(perf_in_guest))
+				__update_cgrp_guest_time(info, now, true);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -1501,14 +1511,39 @@ static void __update_context_time(struct
 	WRITE_ONCE(ctx->timeoffset, ctx->time - ctx->timestamp);
 }
 
+static void __update_context_guest_time(struct perf_event_context *ctx, bool adv)
+{
+	u64 now = ctx->timestamp; /* must be called after __update_context_time(); */
+
+	lockdep_assert_held(&ctx->lock);
+
+	if (adv)
+		ctx->guest_time += now - ctx->guest_timestamp;
+	ctx->guest_timestamp = now;
+
+	/*
+	 * The above: time' = time + (now - timestamp), can be re-arranged
+	 * into: time` = now + (time - timestamp), which gives a single value
+	 * offset to compute future time without locks on.
+	 *
+	 * See perf_event_time_now(), which can be used from NMI context where
+	 * it's (obviously) not possible to acquire ctx->lock in order to read
+	 * both the above values in a consistent manner.
+	 */
+	WRITE_ONCE(ctx->guest_timeoffset, ctx->guest_time - ctx->guest_timestamp);
+}
+
 static void update_context_time(struct perf_event_context *ctx)
 {
 	__update_context_time(ctx, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_context_guest_time(ctx, true);
 }
 
 static u64 perf_event_time(struct perf_event *event)
 {
 	struct perf_event_context *ctx = event->ctx;
+	u64 time;
 
 	if (unlikely(!ctx))
 		return 0;
@@ -1516,12 +1551,17 @@ static u64 perf_event_time(struct perf_e
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time(event);
 
-	return ctx->time;
+	time = ctx->time;
+	if (event->attr.exclude_guest)
+		time -= ctx->guest_time;
+
+	return time;
 }
 
 static u64 perf_event_time_now(struct perf_event *event, u64 now)
 {
 	struct perf_event_context *ctx = event->ctx;
+	u64 time, guest_time;
 
 	if (unlikely(!ctx))
 		return 0;
@@ -1529,11 +1569,19 @@ static u64 perf_event_time_now(struct pe
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time_now(event, now);
 
-	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME))
-		return ctx->time;
+	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME)) {
+		time = ctx->time;
+		if (event->attr.exclude_guest)
+			time -= ctx->guest_time;
+		return time;
+	}
 
-	now += READ_ONCE(ctx->timeoffset);
-	return now;
+	time = now + READ_ONCE(ctx->timeoffset);
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest)) {
+		guest_time = now + READ_ONCE(ctx->guest_timeoffset);
+		time -= guest_time;
+	}
+	return time;
 }
 
 static enum event_type_t get_event_type(struct perf_event *event)
@@ -3340,9 +3388,14 @@ ctx_sched_out(struct perf_event_context
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		stop = !((ctx->is_active & event_type) & EVENT_ALL) &&
+		       ctx == &cpuctx->ctx;
+			
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3366,8 +3419,12 @@ ctx_sched_out(struct perf_event_context
 		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
 		 */
 		is_active = EVENT_ALL;
-	} else
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_guest_timestamp(cpuctx);
+		barrier();
+	} else {
 		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3866,10 +3923,15 @@ static inline void group_update_userpage
 		event_update_userpage(event);
 }
 
+struct merge_sched_data {
+	int can_add_hw;
+	enum event_type_t event_type;
+};
+
 static int merge_sched_in(struct perf_event *event, void *data)
 {
 	struct perf_event_context *ctx = event->ctx;
-	int *can_add_hw = data;
+	struct merge_sched_data *msd = data;
 
 	if (event->state <= PERF_EVENT_STATE_OFF)
 		return 0;
@@ -3881,18 +3943,18 @@ static int merge_sched_in(struct perf_ev
 	 * Don't schedule in any exclude_guest events of PMU with
 	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
 	 */
-	if (__this_cpu_read(perf_in_guest) &&
-	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
-	    event->attr.exclude_guest)
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest) &&
+	    (event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) &&
+	    !(msd->event_type & EVENT_GUEST))
 		return 0;
 
-	if (group_can_go_on(event, *can_add_hw)) {
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 	}
 
 	if (event->state == PERF_EVENT_STATE_INACTIVE) {
-		*can_add_hw = 0;
+		msd->can_add_hw = 0;
 		if (event->attr.pinned) {
 			perf_cgroup_event_disable(event, ctx);
 			perf_event_set_state(event, PERF_EVENT_STATE_ERROR);
@@ -3911,11 +3973,15 @@ static int merge_sched_in(struct perf_ev
 
 static void pmu_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				struct pmu *pmu)
+				struct pmu *pmu,
+				enum even_type_t event_type)
 {
-	int can_add_hw = 1;
+	struct merge_sched_data msd = {
+		.can_add_hw = 1,
+		.event_type = event_type,
+	};
 	visit_groups_merge(ctx, groups, smp_processor_id(), pmu,
-			   merge_sched_in, &can_add_hw);
+			   merge_sched_in, &msd);
 }
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
@@ -3927,14 +3993,14 @@ static void ctx_groups_sched_in(struct p
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
-		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
+		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu, event_type);
 	}
 }
 
 static void __pmu_ctx_sched_in(struct perf_event_context *ctx,
 			       struct pmu *pmu)
 {
-	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu, 0);
 }
 
 static void
@@ -3949,6 +4015,8 @@ ctx_sched_in(struct perf_event_context *
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
 		perf_cgroup_set_timestamp(cpuctx);
@@ -3979,8 +4047,11 @@ ctx_sched_in(struct perf_event_context *
 		 * the exclude_guest events.
 		 */
 		update_context_time(ctx);
-	} else
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
 		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	/*
 	 * First go through the list and put on any pinned groups
@@ -5832,25 +5903,20 @@ void perf_guest_enter(void)
 
 	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
 
-	if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest))) {
-		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
-		return;
-	}
+	if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest)))
+		goto unlock;
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_out(&cpuctx->ctx, EVENT_GUEST);
-	/* Set the guest start time */
-	cpuctx->ctx.timeguest = cpuctx->ctx.time;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
 		task_ctx_sched_out(cpuctx->task_ctx, EVENT_GUEST);
-		cpuctx->task_ctx->timeguest = cpuctx->task_ctx->time;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
 	__this_cpu_write(perf_in_guest, true);
-
+unlock:
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
@@ -5862,24 +5928,21 @@ void perf_guest_exit(void)
 
 	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
 
-	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest))) {
-		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
-		return;
-	}
-
-	__this_cpu_write(perf_in_guest, false);
+	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
+		goto unlock;
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
-	cpuctx->ctx.timeguest = 0;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
 		ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
-		cpuctx->task_ctx->timeguest = 0;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	__this_cpu_write(perf_in_guest, false);
+
+unlock:
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
#21 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 202 lines · Kan
Quoted 202 lines

---

include/linux/perf_event.h +6 -1
@@ -947,7 +947,9 @@ struct perf_event_context {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
-	u64				timeguest;
+	u64				guest_time;
+	u64				guest_timestamp;
+	u64				guest_timeoffset;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1043,6 +1045,9 @@ struct perf_cgroup_info {
 	u64				time;
 	u64				timestamp;
 	u64				timeoffset;
+	u64				guest_time;
+	u64				guest_timestamp;
+	u64				guest_timeoffset;
 	int				active;
 };
 
kernel/events/core.c +75 -27
@@ -638,26 +638,9 @@ __perf_update_times(struct perf_event *e
 
 static void perf_event_update_time(struct perf_event *event)
 {
-	u64 now;
-
-	/* Never count the time of an active guest into an exclude_guest event. */
-	if (event->ctx->timeguest &&
-	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
-		/*
-		 * If a guest is running, use the timestamp while entering the guest.
-		 * If the guest is leaving, reset the event timestamp.
-		 */
-		if (__this_cpu_read(perf_in_guest))
-			event->tstamp = event->ctx->timeguest;
-		else
-			event->tstamp = event->ctx->time;
-		return;
-	}
-
-	now = perf_event_time(event);
+	u64 now = perf_event_time(event);
 	__perf_update_times(event, now, &event->total_time_enabled,
 					&event->total_time_running);
-
 	event->tstamp = now;
 }
 
@@ -780,19 +763,33 @@ static inline int is_cgroup_event(struct
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
+	u64 time;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time;
+	time = t->time;
+	if (event->attr.exclude_guest)
+		time -= t->guest_time;
+	return time;
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
 {
 	struct perf_cgroup_info *t;
+	u64 time, guest_time;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	if (!__load_acquire(&t->active))
-		return t->time;
-	now += READ_ONCE(t->timeoffset);
+	if (!__load_acquire(&t->active)) {
+		time = t->time;
+		if (event->attr.exclude_guest)
+			time -= t->guest_time;
+		return time;
+	}
+
+	time = now + READ_ONCE(t->timeoffset);
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest)) {
+		guest_time = now + READ_ONCE(t->guest_offset);
+		time -= guest_time;
+	}
 	return now;
 }
 
@@ -807,6 +804,17 @@ static inline void __update_cgrp_time(st
 	WRITE_ONCE(info->timeoffset, info->time - info->timestamp);
 }
 
+static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info, u64 now, bool adv)
+{
+	if (adv)
+		info->guest_time += now - info->guest_timestamp;
+	info->guest_timestamp = now;
+	/*
+	 * see update_context_time()
+	 */
+	WRITE_ONCE(info->guest_timeoffset, info->guest_time - info->guest_timestamp);
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -821,6 +829,8 @@ static inline void update_cgrp_time_from
 			info = this_cpu_ptr(cgrp->info);
 
 			__update_cgrp_time(info, now, true);
+			if (__this_cpu_read(perf_in_guest))
+				__update_cgrp_guest_time(info, now, true);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -1501,14 +1511,39 @@ static void __update_context_time(struct
 	WRITE_ONCE(ctx->timeoffset, ctx->time - ctx->timestamp);
 }
 
+static void __update_context_guest_time(struct perf_event_context *ctx, bool adv)
+{
+	u64 now = ctx->timestamp; /* must be called after __update_context_time(); */
+
+	lockdep_assert_held(&ctx->lock);
+
+	if (adv)
+		ctx->guest_time += now - ctx->guest_timestamp;
+	ctx->guest_timestamp = now;
+
+	/*
+	 * The above: time' = time + (now - timestamp), can be re-arranged
+	 * into: time` = now + (time - timestamp), which gives a single value
+	 * offset to compute future time without locks on.
+	 *
+	 * See perf_event_time_now(), which can be used from NMI context where
+	 * it's (obviously) not possible to acquire ctx->lock in order to read
+	 * both the above values in a consistent manner.
+	 */
+	WRITE_ONCE(ctx->guest_timeoffset, ctx->guest_time - ctx->guest_timestamp);
+}
+
 static void update_context_time(struct perf_event_context *ctx)
 {
 	__update_context_time(ctx, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_context_guest_time(ctx, true);
 }
 
 static u64 perf_event_time(struct perf_event *event)
 {
 	struct perf_event_context *ctx = event->ctx;
+	u64 time;
 
 	if (unlikely(!ctx))
 		return 0;
@@ -1516,12 +1551,17 @@ static u64 perf_event_time(struct perf_e
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time(event);
 
-	return ctx->time;
+	time = ctx->time;
+	if (event->attr.exclude_guest)
+		time -= ctx->guest_time;
+
+	return time;
 }
 
 static u64 perf_event_time_now(struct perf_event *event, u64 now)
 {
 	struct perf_event_context *ctx = event->ctx;
+	u64 time, guest_time;
 
 	if (unlikely(!ctx))
 		return 0;
@@ -1529,11 +1569,19 @@ static u64 perf_event_time_now(struct pe
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time_now(event, now);
 
-	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME))
-		return ctx->time;
+	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME)) {
+		time = ctx->time;
+		if (event->attr.exclude_guest)
+			time -= ctx->guest_time;
+		return time;
+	}
 
-	now += READ_ONCE(ctx->timeoffset);
-	return now;
+	time = now + READ_ONCE(ctx->timeoffset);
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest)) {
+		guest_time = now + READ_ONCE(ctx->guest_timeoffset);
+		time -= guest_time;
+	}
+	return time;
 }
 
Quoted 14 lines
Quoted 4 lines
kernel/events/core.c +0 -0
 static enum event_type_t get_event_type(struct perf_event *event)
@@ -3340,9 +3388,14 @@ ctx_sched_out(struct perf_event_context
 	 * would only update time for the pinned events.
 	 */

kernel/events/core.c ctx_sched_out(struct perf_event_context · line 3398

Quoted 10 lines
kernel/events/core.c +6 -1
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		stop = !((ctx->is_active & event_type) & EVENT_ALL) &&
+		       ctx == &cpuctx->ctx;
+			
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);

For the event_type == EVENT_GUEST, the "stop" should always be the same as "ctx == &cpuctx->ctx". Because the ctx->is_active never set the EVENT_GUEST bit.
Why the stop is introduced?

Quoted 97 lines
Quoted 97 lines
kernel/events/core.c +25 -12
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3366,8 +3419,12 @@ ctx_sched_out(struct perf_event_context
 		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
 		 */
 		is_active = EVENT_ALL;
-	} else
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_guest_timestamp(cpuctx);
+		barrier();
+	} else {
 		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3866,10 +3923,15 @@ static inline void group_update_userpage
 		event_update_userpage(event);
 }
 
+struct merge_sched_data {
+	int can_add_hw;
+	enum event_type_t event_type;
+};
+
 static int merge_sched_in(struct perf_event *event, void *data)
 {
 	struct perf_event_context *ctx = event->ctx;
-	int *can_add_hw = data;
+	struct merge_sched_data *msd = data;
 
 	if (event->state <= PERF_EVENT_STATE_OFF)
 		return 0;
@@ -3881,18 +3943,18 @@ static int merge_sched_in(struct perf_ev
 	 * Don't schedule in any exclude_guest events of PMU with
 	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
 	 */
-	if (__this_cpu_read(perf_in_guest) &&
-	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
-	    event->attr.exclude_guest)
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest) &&
+	    (event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) &&
+	    !(msd->event_type & EVENT_GUEST))
 		return 0;
 
-	if (group_can_go_on(event, *can_add_hw)) {
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 	}
 
 	if (event->state == PERF_EVENT_STATE_INACTIVE) {
-		*can_add_hw = 0;
+		msd->can_add_hw = 0;
 		if (event->attr.pinned) {
 			perf_cgroup_event_disable(event, ctx);
 			perf_event_set_state(event, PERF_EVENT_STATE_ERROR);
@@ -3911,11 +3973,15 @@ static int merge_sched_in(struct perf_ev
 
 static void pmu_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				struct pmu *pmu)
+				struct pmu *pmu,
+				enum even_type_t event_type)
 {
-	int can_add_hw = 1;
+	struct merge_sched_data msd = {
+		.can_add_hw = 1,
+		.event_type = event_type,
+	};
 	visit_groups_merge(ctx, groups, smp_processor_id(), pmu,
-			   merge_sched_in, &can_add_hw);
+			   merge_sched_in, &msd);
 }
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
@@ -3927,14 +3993,14 @@ static void ctx_groups_sched_in(struct p
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
-		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
+		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu, event_type);
 	}
 }
 
 static void __pmu_ctx_sched_in(struct perf_event_context *ctx,
 			       struct pmu *pmu)
 {
-	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu, 0);
 }
 
 static void
@@ -3949,6 +4015,8 @@ ctx_sched_in(struct perf_event_context *
 		return;
 
Quoted 12 lines

kernel/events/core.c ctx_sched_in(struct perf_event_context * · line 4050

Quoted 12 lines
kernel/events/core.c +3 -1
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
 		perf_cgroup_set_timestamp(cpuctx);
@@ -3979,8 +4047,11 @@ ctx_sched_in(struct perf_event_context *
 		 * the exclude_guest events.
 		 */
 		update_context_time(ctx);
-	} else
+		update_cgrp_time_from_cpuctx(cpuctx, false);

In the above ctx_sched_out(), the cgrp_time is stopped and the cgrp has been set to inactive.
I think we need a perf_cgroup_set_timestamp(cpuctx) here to restart the cgrp_time, Right?

Also, I think the cgrp_time is different from the normal ctx->time. When a guest is running, there must be no cgroup. It's OK to disable the cgrp_time. If so, I don't think we need to track the guest_time for the cgrp.

Thanks,
Kan

Quoted 66 lines
Quoted 66 lines
kernel/events/core.c +11 -16
+		barrier();
+	} else {
 		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	/*
 	 * First go through the list and put on any pinned groups
@@ -5832,25 +5903,20 @@ void perf_guest_enter(void)
 
 	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
 
-	if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest))) {
-		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
-		return;
-	}
+	if (WARN_ON_ONCE(__this_cpu_read(perf_in_guest)))
+		goto unlock;
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_out(&cpuctx->ctx, EVENT_GUEST);
-	/* Set the guest start time */
-	cpuctx->ctx.timeguest = cpuctx->ctx.time;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
 		task_ctx_sched_out(cpuctx->task_ctx, EVENT_GUEST);
-		cpuctx->task_ctx->timeguest = cpuctx->task_ctx->time;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
 	__this_cpu_write(perf_in_guest, true);
-
+unlock:
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
@@ -5862,24 +5928,21 @@ void perf_guest_exit(void)
 
 	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
 
-	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest))) {
-		perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
-		return;
-	}
-
-	__this_cpu_write(perf_in_guest, false);
+	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
+		goto unlock;
 
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
-	cpuctx->ctx.timeguest = 0;
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
 	if (cpuctx->task_ctx) {
 		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
 		ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
-		cpuctx->task_ctx->timeguest = 0;
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	__this_cpu_write(perf_in_guest, false);
+
+unlock:
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
maintainer
#22 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 11 lines · Kan
Quoted 11 lines
Quoted 11 lines

static enum event_type_t get_event_type(struct perf_event *event)

kernel/events/core.c +5 -0
@@ -3340,9 +3388,14 @@ ctx_sched_out(struct perf_event_context
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		stop = !((ctx->is_active & event_type) & EVENT_ALL) &&
+		       ctx == &cpuctx->ctx;
+			
 		/* update (and stop) ctx time */
Quoted 8 lines
Quoted 3 lines

kernel/events/core.c ctx_sched_out(struct perf_event_context · line 3398

Quoted 3 lines
kernel/events/core.c +1 -1
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);

For the event_type == EVENT_GUEST, the "stop" should always be the same as "ctx == &cpuctx->ctx". Because the ctx->is_active never set the EVENT_GUEST bit.
Why the stop is introduced?

Because the ctx_sched_out() for vPMU should not stop time, only the 'normal' sched-out should stop time.

Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
kernel/events/core.c +2 -0
@@ -3949,6 +4015,8 @@ ctx_sched_in(struct perf_event_context *
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
 		perf_cgroup_set_timestamp(cpuctx);
@@ -3979,8 +4047,11 @@ ctx_sched_in(struct perf_event_context *
 		 * the exclude_guest events.
 		 */
 		update_context_time(ctx);
Quoted 8 lines
Quoted 2 lines

kernel/events/core.c ctx_sched_in(struct perf_event_context * · line 4050

Quoted 2 lines
kernel/events/core.c +1 -1
-	} else
+		update_cgrp_time_from_cpuctx(cpuctx, false);

In the above ctx_sched_out(), the cgrp_time is stopped and the cgrp has been set to inactive.
I think we need a perf_cgroup_set_timestamp(cpuctx) here to restart the cgrp_time, Right?

So the idea was to not stop time when we schedule out for the vPMU, as per the above.

Quoted 4 lines

Also, I think the cgrp_time is different from the normal ctx->time. When a guest is running, there must be no cgroup. It's OK to disable the cgrp_time. If so, I don't think we need to track the guest_time for the cgrp.

Uh, the vCPU thread is/can-be part of a cgroup, and different guests part of different cgroups. The CPU wide 'guest' time is all time spend in guets, but the cgroup view of things might differ, depending on how the guets are arranged in cgroups, no?

As such, we need per cgroup guest tracking.

#23 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 21 lines · Peter Zijlstra
Quoted 11 lines · Kan
Quoted 11 lines
Quoted 11 lines

static enum event_type_t get_event_type(struct perf_event *event)

kernel/events/core.c +5 -0
@@ -3340,9 +3388,14 @@ ctx_sched_out(struct perf_event_context
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		stop = !((ctx->is_active & event_type) & EVENT_ALL) &&
+		       ctx == &cpuctx->ctx;
+			
 		/* update (and stop) ctx time */
Quoted 8 lines
Quoted 3 lines

kernel/events/core.c ctx_sched_out(struct perf_event_context · line 3398

Quoted 3 lines
kernel/events/core.c +1 -1
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);

For the event_type == EVENT_GUEST, the "stop" should always be the same as "ctx == &cpuctx->ctx". Because the ctx->is_active never set the EVENT_GUEST bit.
Why the stop is introduced?

Quoted 1 line

Because the ctx_sched_out() for vPMU should not stop time,

But the implementation seems stop the time.

The ctx->is_active should be (EVENT_ALL | EVENT_TIME) for most of cases.

When a vPMU is scheduling in (invoke ctx_sched_out()), the event_type should only be EVENT_GUEST.

!((ctx->is_active & event_type) & EVENT_ALL) should be TRUE.

For a CPU context, ctx == &cpuctx->ctx is TRUE as well.

The update_cgrp_time_from_cpuctx(cpuctx, TRUE) stops the time by deactivate the cgroup, __store_release(&info->active, 0).

If an user try to read the cgroup events when a guest is running. The update_cgrp_time_from_event() doesn't update the cgrp time. So both time and counter are stopped.

Quoted 2 lines

only the
'normal' sched-out should stop time.

If the guest is the only case which we want to keep the time for, I think we may use a straightforward check as below.

	stop = !(event_type & EVENT_GUEST) && ctx == &cpuctx->ctx;
Quoted 32 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
kernel/events/core.c +2 -0
@@ -3949,6 +4015,8 @@ ctx_sched_in(struct perf_event_context *
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
 		perf_cgroup_set_timestamp(cpuctx);
@@ -3979,8 +4047,11 @@ ctx_sched_in(struct perf_event_context *
 		 * the exclude_guest events.
 		 */
 		update_context_time(ctx);
Quoted 8 lines
Quoted 2 lines

kernel/events/core.c ctx_sched_in(struct perf_event_context * · line 4050

Quoted 2 lines
kernel/events/core.c +1 -1
-	} else
+		update_cgrp_time_from_cpuctx(cpuctx, false);

In the above ctx_sched_out(), the cgrp_time is stopped and the cgrp has been set to inactive.
I think we need a perf_cgroup_set_timestamp(cpuctx) here to restart the cgrp_time, Right?

So the idea was to not stop time when we schedule out for the vPMU, as per the above.

Quoted 4 lines

Also, I think the cgrp_time is different from the normal ctx->time. When a guest is running, there must be no cgroup. It's OK to disable the cgrp_time. If so, I don't think we need to track the guest_time for the cgrp.

Quoted 6 lines

Uh, the vCPU thread is/can-be part of a cgroup, and different guests part of different cgroups. The CPU wide 'guest' time is all time spend in guets, but the cgroup view of things might differ, depending on how the guets are arranged in cgroups, no?

As such, we need per cgroup guest tracking.

Got it.

Thanks,
Kan

maintainer
#24 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 26 lines · Kan
Quoted 21 lines · Peter Zijlstra
Quoted 11 lines · Kan
Quoted 11 lines
Quoted 11 lines

static enum event_type_t get_event_type(struct perf_event *event)

kernel/events/core.c +5 -0
@@ -3340,9 +3388,14 @@ ctx_sched_out(struct perf_event_context
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		stop = !((ctx->is_active & event_type) & EVENT_ALL) &&
+		       ctx == &cpuctx->ctx;
+			
 		/* update (and stop) ctx time */
Quoted 8 lines
Quoted 3 lines

kernel/events/core.c ctx_sched_out(struct perf_event_context · line 3398

Quoted 3 lines
kernel/events/core.c +1 -1
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);

For the event_type == EVENT_GUEST, the "stop" should always be the same as "ctx == &cpuctx->ctx". Because the ctx->is_active never set the EVENT_GUEST bit.
Why the stop is introduced?

Quoted 1 line

Because the ctx_sched_out() for vPMU should not stop time,

Quoted 8 lines

But the implementation seems stop the time.

The ctx->is_active should be (EVENT_ALL | EVENT_TIME) for most of cases.

When a vPMU is scheduling in (invoke ctx_sched_out()), the event_type should only be EVENT_GUEST.

!((ctx->is_active & event_type) & EVENT_ALL) should be TRUE.

Hmm.. yeah, I think I might've gotten that wrong.

Quoted 7 lines
Quoted 2 lines

only the
'normal' sched-out should stop time.

If the guest is the only case which we want to keep the time for, I think we may use a straightforward check as below.

	stop = !(event_type & EVENT_GUEST) && ctx == &cpuctx->ctx;

So I think I was trying to get stop true when there weren't in fact events on, that is when we got in without EVENT_ALL bits, but perhaps that case isn't relevant.

#25 lore raw

Re: [PATCH v2 07/54] perf: Add generic exclude_guest support

Quoted 9 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines

only the
'normal' sched-out should stop time.

If the guest is the only case which we want to keep the time for, I think we may use a straightforward check as below.

	stop = !(event_type & EVENT_GUEST) && ctx == &cpuctx->ctx;

So I think I was trying to get stop true when there weren't in fact events on, that is when we got in without EVENT_ALL bits, but perhaps that case isn't relevant.

It should be irrelevant. Here I think we just need to make sure that the guest is not the reason for stopping the time if there is an active time. For the others, there is nothing changed.
The __this_cpu_read(perf_in_guest) check will guarantee that both guest time and host time sync when any updates occur in the guest mode.

Thanks,
Kan

#26 lore raw

PATCH v2 8/54 perf/x86/intel: Support PERF_PMU_CAP_PASSTHROUGH_VPMU

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Apply the PERF_PMU_CAP_PASSTHROUGH_VPMU for Intel core PMU. It only indicates that the perf side of core PMU is ready to support the passthrough vPMU. Besides the capability, the hypervisor should still need to check the PMU version and other capabilities to decide whether to enable the passthrough vPMU.

---
 arch/x86/events/intel/core.c | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/events/intel/core.c +5 -0
index 768d1414897f..4d8f907a9416 100644
@@ -4743,6 +4743,8 @@ static void intel_pmu_check_hybrid_pmus(struct x86_hybrid_pmu *pmu)
 	else
 		pmu->pmu.capabilities &= ~PERF_PMU_CAP_AUX_OUTPUT;
 
+	pmu->pmu.capabilities |= PERF_PMU_CAP_PASSTHROUGH_VPMU;
+
 	intel_pmu_check_event_constraints(pmu->event_constraints,
 					  pmu->num_counters,
 					  pmu->num_counters_fixed,
@@ -6242,6 +6244,9 @@ __init int intel_pmu_init(void)
 			pr_cont(" AnyThread deprecated, ");
 	}
 
+	/* The perf side of core PMU is ready to support the passthrough vPMU. */
+	x86_get_pmu(smp_processor_id())->capabilities |= PERF_PMU_CAP_PASSTHROUGH_VPMU;
+
 	/*
 	 * Install the hw-cache-events table:
 	 */
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#27 lore raw

PATCH v2 9/54 perf: core/x86: Register a new vector for KVM GUEST PMI

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Create a new vector in the host IDT for kvm guest PMI handling within passthrough PMU. In addition, add a function to allow kvm register the new vector's handler.

This is the preparation work to support passthrough PMU to handle kvm guest PMIs without interference from PMI handler of the host PMU.

---
 arch/x86/include/asm/hardirq.h           |  1 +
 arch/x86/include/asm/idtentry.h          |  1 +
 arch/x86/include/asm/irq.h               |  1 +
 arch/x86/include/asm/irq_vectors.h       |  5 +++-
 arch/x86/kernel/idt.c                    |  1 +
 arch/x86/kernel/irq.c                    | 29 ++++++++++++++++++++++++
 tools/arch/x86/include/asm/irq_vectors.h |  3 ++-
 7 files changed, 39 insertions(+), 2 deletions(-)
arch/x86/include/asm/hardirq.h +1 -0
index fbc7722b87d1..250e6db1cb5f 100644
@@ -19,6 +19,7 @@ typedef struct {
 	unsigned int kvm_posted_intr_ipis;
 	unsigned int kvm_posted_intr_wakeup_ipis;
 	unsigned int kvm_posted_intr_nested_ipis;
+	unsigned int kvm_guest_pmis;
 #endif
 	unsigned int x86_platform_ipis;	/* arch dependent */
 	unsigned int apic_perf_irqs;
arch/x86/include/asm/idtentry.h +1 -0
index 749c7411d2f1..4090aea47b76 100644
@@ -745,6 +745,7 @@ DECLARE_IDTENTRY_SYSVEC(IRQ_WORK_VECTOR,		sysvec_irq_work);
 DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_VECTOR,		sysvec_kvm_posted_intr_ipi);
 DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_WAKEUP_VECTOR,	sysvec_kvm_posted_intr_wakeup_ipi);
 DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR,	sysvec_kvm_posted_intr_nested_ipi);
+DECLARE_IDTENTRY_SYSVEC(KVM_GUEST_PMI_VECTOR,	        sysvec_kvm_guest_pmi_handler);
 #else
 # define fred_sysvec_kvm_posted_intr_ipi		NULL
 # define fred_sysvec_kvm_posted_intr_wakeup_ipi		NULL
arch/x86/include/asm/irq.h +1 -0
index 194dfff84cb1..2483f6ef5d4e 100644
@@ -31,6 +31,7 @@ extern void fixup_irqs(void);
 
 #if IS_ENABLED(CONFIG_KVM)
 extern void kvm_set_posted_intr_wakeup_handler(void (*handler)(void));
+void kvm_set_guest_pmi_handler(void (*handler)(void));
 #endif
 
 extern void (*x86_platform_ipi_callback)(void);
arch/x86/include/asm/irq_vectors.h +4 -1
index d18bfb238f66..e5f741bb1557 100644
@@ -77,7 +77,10 @@
  */
 #define IRQ_WORK_VECTOR			0xf6
 
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR		0xf5
+#endif
+
 #define DEFERRED_ERROR_VECTOR		0xf4
 
 /* Vector on which hypervisor callbacks will be delivered */
arch/x86/kernel/idt.c +1 -0
index fc37c8d83daf..c62368a3ba04 100644
@@ -157,6 +157,7 @@ static const __initconst struct idt_data apic_idts[] = {
 	INTG(POSTED_INTR_VECTOR,		asm_sysvec_kvm_posted_intr_ipi),
 	INTG(POSTED_INTR_WAKEUP_VECTOR,		asm_sysvec_kvm_posted_intr_wakeup_ipi),
 	INTG(POSTED_INTR_NESTED_VECTOR,		asm_sysvec_kvm_posted_intr_nested_ipi),
+	INTG(KVM_GUEST_PMI_VECTOR,		asm_sysvec_kvm_guest_pmi_handler),
 # endif
 # ifdef CONFIG_IRQ_WORK
 	INTG(IRQ_WORK_VECTOR,			asm_sysvec_irq_work),
arch/x86/kernel/irq.c +29 -0
index 35fde0107901..22c10e5c50af 100644
@@ -181,6 +181,13 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 		seq_printf(p, "%10u ",
 			   irq_stats(j)->kvm_posted_intr_wakeup_ipis);
 	seq_puts(p, "  Posted-interrupt wakeup event\n");
+
+	seq_printf(p, "%*s: ", prec, "VPMU");
+	for_each_online_cpu(j)
+		seq_printf(p, "%10u ",
+			   irq_stats(j)->kvm_guest_pmis);
+	seq_puts(p, " KVM GUEST PMI\n");
+
 #endif
 	return 0;
 }
@@ -293,6 +300,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi)
 #if IS_ENABLED(CONFIG_KVM)
 static void dummy_handler(void) {}
 static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
+static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
 
 void kvm_set_posted_intr_wakeup_handler(void (*handler)(void))
 {
@@ -305,6 +313,17 @@ void kvm_set_posted_intr_wakeup_handler(void (*handler)(void))
 }
 EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wakeup_handler);
 
+void kvm_set_guest_pmi_handler(void (*handler)(void))
+{
+	if (handler) {
+		kvm_guest_pmi_handler = handler;
+	} else {
+		kvm_guest_pmi_handler = dummy_handler;
+		synchronize_rcu();
+	}
+}
+EXPORT_SYMBOL_GPL(kvm_set_guest_pmi_handler);
+
 /*
  * Handler for POSTED_INTERRUPT_VECTOR.
  */
@@ -332,6 +351,16 @@ DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_kvm_posted_intr_nested_ipi)
 	apic_eoi();
 	inc_irq_stat(kvm_posted_intr_nested_ipis);
 }
+
+/*
+ * Handler for KVM_GUEST_PMI_VECTOR.
+ */
+DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_guest_pmi_handler)
+{
+	apic_eoi();
+	inc_irq_stat(kvm_guest_pmis);
+	kvm_guest_pmi_handler();
+}
 #endif
 
 
tools/arch/x86/include/asm/irq_vectors.h +2 -1
index 3f73ac3ed3a0..6df2a986805d 100644
@@ -83,8 +83,9 @@
 /* Vector on which hypervisor callbacks will be delivered */
 #define HYPERVISOR_CALLBACK_VECTOR	0xf3
 
-/* Vector for KVM to deliver posted interrupt IPI */
 #if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR		0xf5
+/* Vector for KVM to deliver posted interrupt IPI */
 #define POSTED_INTR_VECTOR		0xf2
 #define POSTED_INTR_WAKEUP_VECTOR	0xf1
 #define POSTED_INTR_NESTED_VECTOR	0xf0
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#28 lore raw

Re: [PATCH v2 09/54] perf: core/x86: Register a new vector for KVM GUEST PMI

arch/x86/kernel/irq.c void kvm_set_posted_intr_wakeup_handler(void (*handler)(void)) · line 325

Quoted 10 lines · Mingwei Zhang
arch/x86/kernel/irq.c +10 -0
+void kvm_set_guest_pmi_handler(void (*handler)(void))
+{
+	if (handler) {
+		kvm_guest_pmi_handler = handler;
+	} else {
+		kvm_guest_pmi_handler = dummy_handler;
+		synchronize_rcu();
+	}
+}
+EXPORT_SYMBOL_GPL(kvm_set_guest_pmi_handler);

Just for my edification, after synchronize_rcu() nobody should observe the old handler, but what guarantees there's not still one running?

I'm thinking the fact that these handlers run with IRQs disabled, and synchronize_rcu() also very much ensures all prior non-preempt sections are complete?

#29 lore raw

Re: [PATCH v2 09/54] perf: core/x86: Register a new vector for KVM GUEST PMI

Quoted 15 lines · Peter Zijlstra

arch/x86/kernel/irq.c void kvm_set_posted_intr_wakeup_handler(void (*handler)(void)) · line 325

Quoted 10 lines · Mingwei Zhang
arch/x86/kernel/irq.c +10 -0
+void kvm_set_guest_pmi_handler(void (*handler)(void))
+{
+	if (handler) {
+		kvm_guest_pmi_handler = handler;
+	} else {
+		kvm_guest_pmi_handler = dummy_handler;
+		synchronize_rcu();
+	}
+}
+EXPORT_SYMBOL_GPL(kvm_set_guest_pmi_handler);

Just for my edification, after synchronize_rcu() nobody should observe the old handler, but what guarantees there's not still one running?

Interrupts handler can be regarded as RCU read-side critical section, once synchronize_rcu returns, no one accessing the old handler lefts.

Quoted 4 lines

I'm thinking the fact that these handlers run with IRQs disabled, and synchronize_rcu() also very much ensures all prior non-preempt sections are complete?

Yes :)

Thanks,
Yanfei

Quoted 3 lines
#30 lore raw

PATCH v2 10/54 KVM: x86: Extract x86_set_kvm_irq_handler() function

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

KVM needs to register irq handler for POSTED_INTR_WAKEUP_VECTOR and KVM_GUEST_PMI_VECTOR, a common function x86_set_kvm_irq_handler() is extracted to reduce exports function and duplicated code.

---
 arch/x86/include/asm/irq.h |  3 +--
 arch/x86/kernel/irq.c      | 27 +++++++++++----------------
 arch/x86/kvm/vmx/vmx.c     |  4 ++--
 3 files changed, 14 insertions(+), 20 deletions(-)
arch/x86/include/asm/irq.h +1 -2
index 2483f6ef5d4e..050a247b69b4 100644
@@ -30,8 +30,7 @@ struct irq_desc;
 extern void fixup_irqs(void);
 
 #if IS_ENABLED(CONFIG_KVM)
-extern void kvm_set_posted_intr_wakeup_handler(void (*handler)(void));
-void kvm_set_guest_pmi_handler(void (*handler)(void));
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void));
 #endif
 
 extern void (*x86_platform_ipi_callback)(void);
arch/x86/kernel/irq.c +11 -16
index 22c10e5c50af..3ada69c50951 100644
@@ -302,27 +302,22 @@ static void dummy_handler(void) {}
 static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
 static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
 
-void kvm_set_posted_intr_wakeup_handler(void (*handler)(void))
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
-	if (handler)
+	if (!handler)
+		handler = dummy_handler;
+
+	if (vector == POSTED_INTR_WAKEUP_VECTOR)
 		kvm_posted_intr_wakeup_handler = handler;
-	else {
-		kvm_posted_intr_wakeup_handler = dummy_handler;
-		synchronize_rcu();
-	}
-}
-EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wakeup_handler);
-
-void kvm_set_guest_pmi_handler(void (*handler)(void))
-{
-	if (handler) {
+	else if (vector == KVM_GUEST_PMI_VECTOR)
 		kvm_guest_pmi_handler = handler;
-	} else {
-		kvm_guest_pmi_handler = dummy_handler;
+	else
+		WARN_ON_ONCE(1);
+
+	if (handler == dummy_handler)
 		synchronize_rcu();
-	}
 }
-EXPORT_SYMBOL_GPL(kvm_set_guest_pmi_handler);
+EXPORT_SYMBOL_GPL(x86_set_kvm_irq_handler);
 
 /*
  * Handler for POSTED_INTERRUPT_VECTOR.
arch/x86/kvm/vmx/vmx.c +2 -2
index c37a89eda90f..c2dc68a25a53 100644
@@ -8214,7 +8214,7 @@ static void vmx_migrate_timers(struct kvm_vcpu *vcpu)
 
 static void vmx_hardware_unsetup(void)
 {
-	kvm_set_posted_intr_wakeup_handler(NULL);
+	x86_set_kvm_irq_handler(POSTED_INTR_WAKEUP_VECTOR, NULL);
 
 	if (nested)
 		nested_vmx_hardware_unsetup();
@@ -8679,7 +8679,7 @@ static __init int hardware_setup(void)
 	if (r && nested)
 		nested_vmx_hardware_unsetup();
 
-	kvm_set_posted_intr_wakeup_handler(pi_wakeup_handler);
+	x86_set_kvm_irq_handler(POSTED_INTR_WAKEUP_VECTOR, pi_wakeup_handler);
 
 	return r;
 }
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#31 lore raw

Re: [PATCH v2 10/54] KVM: x86: Extract x86_set_kvm_irq_handler() function

Quoted 58 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

KVM needs to register irq handler for POSTED_INTR_WAKEUP_VECTOR and KVM_GUEST_PMI_VECTOR, a common function x86_set_kvm_irq_handler() is extracted to reduce exports function and duplicated code.

---
 arch/x86/include/asm/irq.h |  3 +--
 arch/x86/kernel/irq.c      | 27 +++++++++++----------------
 arch/x86/kvm/vmx/vmx.c     |  4 ++--
 3 files changed, 14 insertions(+), 20 deletions(-)
arch/x86/include/asm/irq.h +1 -2
index 2483f6ef5d4e..050a247b69b4 100644
@@ -30,8 +30,7 @@ struct irq_desc;
 extern void fixup_irqs(void);
 
 #if IS_ENABLED(CONFIG_KVM)
-extern void kvm_set_posted_intr_wakeup_handler(void (*handler)(void));
-void kvm_set_guest_pmi_handler(void (*handler)(void));
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void));
 #endif
 
 extern void (*x86_platform_ipi_callback)(void);
arch/x86/kernel/irq.c +6 -13
index 22c10e5c50af..3ada69c50951 100644
@@ -302,27 +302,22 @@ static void dummy_handler(void) {}
 static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
 static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
 
-void kvm_set_posted_intr_wakeup_handler(void (*handler)(void))
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
-	if (handler)
+	if (!handler)
+		handler = dummy_handler;
+
+	if (vector == POSTED_INTR_WAKEUP_VECTOR)
 		kvm_posted_intr_wakeup_handler = handler;
-	else {
-		kvm_posted_intr_wakeup_handler = dummy_handler;
-		synchronize_rcu();
-	}
-}
-EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wakeup_handler);
-
-void kvm_set_guest_pmi_handler(void (*handler)(void))
-{
-	if (handler) {
+	else if (vector == KVM_GUEST_PMI_VECTOR)
 		kvm_guest_pmi_handler = handler;
-	} else {

arch/x86/kernel/irq.c static void dummy_handler(void) {} · line 320

Quoted 10 lines
arch/x86/kernel/irq.c +5 -3
-		kvm_guest_pmi_handler = dummy_handler;
+	else
+		WARN_ON_ONCE(1);
+
+	if (handler == dummy_handler)
 		synchronize_rcu();
-	}
 }
-EXPORT_SYMBOL_GPL(kvm_set_guest_pmi_handler);
+EXPORT_SYMBOL_GPL(x86_set_kvm_irq_handler);

Can't you just squash this into the previous patch? I mean, what's the point of this back and forth?

Quoted 6 lines
arch/x86/kernel/irq.c +5 -0
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
+	if (!handler)
+		handler = dummy_handler;
+
+	if (vector == POSTED_INTR_WAKEUP_VECTOR)

arch/x86/kernel/irq.c static void dummy_handler(void) {} · line 320

Quoted 10 lines
arch/x86/kernel/irq.c +6 -0
 		kvm_posted_intr_wakeup_handler = handler;
+	else if (vector == KVM_GUEST_PMI_VECTOR)
 		kvm_guest_pmi_handler = handler;
+	else
+		WARN_ON_ONCE(1);
+
+	if (handler == dummy_handler)
 		synchronize_rcu();
 }
+EXPORT_SYMBOL_GPL(x86_set_kvm_irq_handler);

So what about:

x86_set_kvm_irq_handler(foo, handler1);
x86_set_kvm_irq_handler(foo, handler2);

?

I'm fairly sure you either want to enforce a NULL<->handler transition, or add some additional synchronize stuff.

Hmm?

#32 lore raw

Re: [PATCH v2 10/54] KVM: x86: Extract x86_set_kvm_irq_handler() function

Quoted 64 lines · Peter Zijlstra
Quoted 63 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

KVM needs to register irq handler for POSTED_INTR_WAKEUP_VECTOR and KVM_GUEST_PMI_VECTOR, a common function x86_set_kvm_irq_handler() is extracted to reduce exports function and duplicated code.

---
 arch/x86/include/asm/irq.h |  3 +--
 arch/x86/kernel/irq.c      | 27 +++++++++++----------------
 arch/x86/kvm/vmx/vmx.c     |  4 ++--
 3 files changed, 14 insertions(+), 20 deletions(-)
arch/x86/include/asm/irq.h +1 -2
index 2483f6ef5d4e..050a247b69b4 100644
@@ -30,8 +30,7 @@ struct irq_desc;
 extern void fixup_irqs(void);
 
 #if IS_ENABLED(CONFIG_KVM)
-extern void kvm_set_posted_intr_wakeup_handler(void (*handler)(void));
-void kvm_set_guest_pmi_handler(void (*handler)(void));
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void));
 #endif
 
 extern void (*x86_platform_ipi_callback)(void);
arch/x86/kernel/irq.c +10 -14
index 22c10e5c50af..3ada69c50951 100644
@@ -302,27 +302,22 @@ static void dummy_handler(void) {}
 static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
 static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
 
-void kvm_set_posted_intr_wakeup_handler(void (*handler)(void))
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
-	if (handler)
+	if (!handler)
+		handler = dummy_handler;
+
+	if (vector == POSTED_INTR_WAKEUP_VECTOR)
 		kvm_posted_intr_wakeup_handler = handler;
-	else {
-		kvm_posted_intr_wakeup_handler = dummy_handler;
-		synchronize_rcu();
-	}
-}
-EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wakeup_handler);
-
-void kvm_set_guest_pmi_handler(void (*handler)(void))
-{
-	if (handler) {
+	else if (vector == KVM_GUEST_PMI_VECTOR)
 		kvm_guest_pmi_handler = handler;
-	} else {
-		kvm_guest_pmi_handler = dummy_handler;
+	else
+		WARN_ON_ONCE(1);
+
+	if (handler == dummy_handler)
Quoted 8 lines

arch/x86/kernel/irq.c static void dummy_handler(void) {} · line 320

Quoted 5 lines
arch/x86/kernel/irq.c +1 -2
 		synchronize_rcu();
-	}
 }
-EXPORT_SYMBOL_GPL(kvm_set_guest_pmi_handler);
+EXPORT_SYMBOL_GPL(x86_set_kvm_irq_handler);

Can't you just squash this into the previous patch? I mean, what's the point of this back and forth?

Ok, I will put this before the previous patch, and let previous patch use this directly.

Quoted 23 lines
Quoted 7 lines
arch/x86/kernel/irq.c +5 -0
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
+	if (!handler)
+		handler = dummy_handler;
+
+	if (vector == POSTED_INTR_WAKEUP_VECTOR)

arch/x86/kernel/irq.c static void dummy_handler(void) {} · line 320

Quoted 10 lines
arch/x86/kernel/irq.c +6 -0
 		kvm_posted_intr_wakeup_handler = handler;
+	else if (vector == KVM_GUEST_PMI_VECTOR)
 		kvm_guest_pmi_handler = handler;
+	else
+		WARN_ON_ONCE(1);
+
+	if (handler == dummy_handler)
 		synchronize_rcu();
 }
+EXPORT_SYMBOL_GPL(x86_set_kvm_irq_handler);

So what about:

x86_set_kvm_irq_handler(foo, handler1);
x86_set_kvm_irq_handler(foo, handler2);

Quoted 6 lines

?

I'm fairly sure you either want to enforce a NULL<->handler transition, or add some additional synchronize stuff.

Hmm?

yes, x86_set_kvm_irq_handler() is called once for each vector at kvm/kvm_intel module_init() and module_exit(). so we should enforce a NULL<->handler transition.

thanks

Quoted 2 lines
#33 lore raw

PATCH v2 11/54 KVM: x86/pmu: Register guest pmi handler for emulated PMU

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add function to register/unregister PMI handler at KVM module initialization and destroy. This allows the host PMU with passthough capability enabled switch PMI handler at PMU context switch.

---
 arch/x86/kvm/x86.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/kvm/x86.c +12 -0
index ebcc12d1e1de..51b5a88222ef 100644
@@ -13887,6 +13887,16 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
 }
 EXPORT_SYMBOL_GPL(kvm_sev_es_string_io);
 
+static void kvm_handle_guest_pmi(void)
+{
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+	if (WARN_ON_ONCE(!vcpu))
+		return;
+
+	kvm_make_request(KVM_REQ_PMI, vcpu);
+}
+
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_entry);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio);
@@ -13921,12 +13931,14 @@ static int __init kvm_x86_init(void)
 {
 	kvm_mmu_x86_module_init();
 	mitigate_smt_rsb &= boot_cpu_has_bug(X86_BUG_SMT_RSB) && cpu_smt_possible();
+	x86_set_kvm_irq_handler(KVM_GUEST_PMI_VECTOR, kvm_handle_guest_pmi);
 	return 0;
 }
 module_init(kvm_x86_init);
 
 static void __exit kvm_x86_exit(void)
 {
+	x86_set_kvm_irq_handler(KVM_GUEST_PMI_VECTOR, NULL);
 	WARN_ON_ONCE(static_branch_unlikely(&kvm_has_noapic_vcpu));
 }
 module_exit(kvm_x86_exit);
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#34 lore raw

PATCH v2 12/54 perf: x86: Add x86 function to switch PMI handler

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add x86 specific function to switch PMI handler since passthrough PMU and host PMU use different interrupt vectors.

x86_perf_guest_enter() switch PMU vector from NMI to KVM_GUEST_PMI_VECTOR, and guest LVTPC_MASK value should be reflected onto HW to indicate whether guest has cleared LVTPC_MASK or not, so guest lvt_pc is passed as parameter.

x86_perf_guest_exit() switch PMU vector from KVM_GUEST_PMI_VECTOR to NMI.

---
 arch/x86/events/core.c            | 17 +++++++++++++++++
 arch/x86/include/asm/perf_event.h |  3 +++
 2 files changed, 20 insertions(+)
arch/x86/events/core.c +17 -0
index 09050641ce5d..8167f2230d3a 100644
@@ -701,6 +701,23 @@ struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
 }
 EXPORT_SYMBOL_GPL(perf_guest_get_msrs);
 
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_enter);
+
+void x86_perf_guest_exit(void)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_exit);
+
 /*
  * There may be PMI landing after enabled=0. The PMI hitting could be before or
  * after disable_all.
arch/x86/include/asm/perf_event.h +3 -0
index 3736b8a46c04..807ea9c98567 100644
@@ -577,6 +577,9 @@ static inline void perf_events_lapic_init(void)	{ }
 static inline void perf_check_microcode(void) { }
 #endif
 
+void x86_perf_guest_enter(u32 guest_lvtpc);
+void x86_perf_guest_exit(void);
+
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
 extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data);
 extern void x86_perf_get_lbr(struct x86_pmu_lbr *lbr);
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#35 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

Quoted 32 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add x86 specific function to switch PMI handler since passthrough PMU and host PMU use different interrupt vectors.

x86_perf_guest_enter() switch PMU vector from NMI to KVM_GUEST_PMI_VECTOR, and guest LVTPC_MASK value should be reflected onto HW to indicate whether guest has cleared LVTPC_MASK or not, so guest lvt_pc is passed as parameter.

x86_perf_guest_exit() switch PMU vector from KVM_GUEST_PMI_VECTOR to NMI.

---
 arch/x86/events/core.c            | 17 +++++++++++++++++
 arch/x86/include/asm/perf_event.h |  3 +++
 2 files changed, 20 insertions(+)
arch/x86/events/core.c +6 -0
index 09050641ce5d..8167f2230d3a 100644
@@ -701,6 +701,23 @@ struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
 }
 EXPORT_SYMBOL_GPL(perf_guest_get_msrs);
 
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 719

Quoted 10 lines
arch/x86/events/core.c +10 -0
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_enter);
+
+void x86_perf_guest_exit(void)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

Urgghh... because it makes sense for this bare APIC write to be exported ?!?

Can't this at the very least be hard tied to perf_guest_{enter,exit}() ?

#36 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

Quoted 38 lines · Peter Zijlstra
Quoted 37 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add x86 specific function to switch PMI handler since passthrough PMU and host PMU use different interrupt vectors.

x86_perf_guest_enter() switch PMU vector from NMI to KVM_GUEST_PMI_VECTOR, and guest LVTPC_MASK value should be reflected onto HW to indicate whether guest has cleared LVTPC_MASK or not, so guest lvt_pc is passed as parameter.

x86_perf_guest_exit() switch PMU vector from KVM_GUEST_PMI_VECTOR to NMI.

---
 arch/x86/events/core.c            | 17 +++++++++++++++++
 arch/x86/include/asm/perf_event.h |  3 +++
 2 files changed, 20 insertions(+)
arch/x86/events/core.c +11 -0
index 09050641ce5d..8167f2230d3a 100644
@@ -701,6 +701,23 @@ struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
 }
 EXPORT_SYMBOL_GPL(perf_guest_get_msrs);
 
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_enter);
+
+void x86_perf_guest_exit(void)
+{
Quoted 8 lines

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 719

Quoted 5 lines
arch/x86/events/core.c +5 -0
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

Urgghh... because it makes sense for this bare APIC write to be exported ?!?

Usually KVM doesn't access HW except vmx directly and requests other components to access HW to avoid confliction, APIC_LVTPC is managed by x86 perf driver, so I added two functions here and exported them.

Quoted 3 lines

Can't this at the very least be hard tied to perf_guest_{enter,exit}() ? perf_guest_{enter, exit} is called from this function in another commit, I

should merge that commit into this one according to your suggestion in other email.

thanks

maintainer
#37 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

Quoted 23 lines · Xiong Y
Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

arch/x86/events/core.c +11 -0
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_enter);
+
+void x86_perf_guest_exit(void)
+{
Quoted 8 lines

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 719

Quoted 5 lines
arch/x86/events/core.c +5 -0
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

Urgghh... because it makes sense for this bare APIC write to be exported ?!?

Quoted 3 lines

Usually KVM doesn't access HW except vmx directly and requests other components to access HW to avoid confliction, APIC_LVTPC is managed by x86 perf driver, so I added two functions here and exported them.

Yes, I understand how you got here. But as with everything you export, you should ask yourself, should I export this. The above x86_perf_guest_enter() function allows any module to write random LVTPC entries. That's not a good thing to export.

I utterly detest how KVM is a module and ends up exporting a ton of stuff that *really* should not be exported.

#38 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

Quoted 24 lines · Peter Zijlstra
Quoted 23 lines · Xiong Y
Quoted 22 lines · Peter Zijlstra
Quoted 8 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

arch/x86/events/core.c +6 -0
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 719

Quoted 10 lines
arch/x86/events/core.c +10 -0
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_enter);
+
+void x86_perf_guest_exit(void)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

Urgghh... because it makes sense for this bare APIC write to be exported ?!?

Quoted 8 lines
Quoted 3 lines

Usually KVM doesn't access HW except vmx directly and requests other components to access HW to avoid confliction, APIC_LVTPC is managed by x86 perf driver, so I added two functions here and exported them.

Yes, I understand how you got here. But as with everything you export, you should ask yourself, should I export this. The above x86_perf_guest_enter() function allows any module to write random LVTPC entries. That's not a good thing to export.

Totally agree with your concern. Here KVM need to switch PMI vector at PMU context switch, export isn't good, could you kindly give guideline to design or improve such interface?
I thought the following two method, but they are worse than this commit.
1. Perf register notification to KVM, but this makes perf depends on KVM.
2. KVM write APIC_LVTPC directly, but this needs x86 export apic_write().

thanks

Quoted 3 lines

I utterly detest how KVM is a module and ends up exporting a ton of stuff that *really* should not be exported.

#39 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

In reply to Mingwei Zhang
Quoted 23 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add x86 specific function to switch PMI handler since passthrough PMU and host PMU use different interrupt vectors.

x86_perf_guest_enter() switch PMU vector from NMI to KVM_GUEST_PMI_VECTOR, and guest LVTPC_MASK value should be reflected onto HW to indicate whether guest has cleared LVTPC_MASK or not, so guest lvt_pc is passed as parameter.

x86_perf_guest_exit() switch PMU vector from KVM_GUEST_PMI_VECTOR to NMI.

---
 arch/x86/events/core.c            | 17 +++++++++++++++++
 arch/x86/include/asm/perf_event.h |  3 +++
 2 files changed, 20 insertions(+)
arch/x86/events/core.c +0 -0
index 09050641ce5d..8167f2230d3a 100644

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 709

Quoted 9 lines
arch/x86/events/core.c +6 -0
@@ -701,6 +701,23 @@ struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
 }
 EXPORT_SYMBOL_GPL(perf_guest_get_msrs);
 
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));

If CONFIG_KVM is not defined, KVM_GUEST_PMI_VECTOR is not available and it causes compiling error.

#40 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

Quoted 28 lines · Zide
Quoted 26 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add x86 specific function to switch PMI handler since passthrough PMU and host PMU use different interrupt vectors.

x86_perf_guest_enter() switch PMU vector from NMI to KVM_GUEST_PMI_VECTOR, and guest LVTPC_MASK value should be reflected onto HW to indicate whether guest has cleared LVTPC_MASK or not, so guest lvt_pc is passed as parameter.

x86_perf_guest_exit() switch PMU vector from KVM_GUEST_PMI_VECTOR to NMI.

---
 arch/x86/events/core.c            | 17 +++++++++++++++++
 arch/x86/include/asm/perf_event.h |  3 +++
 2 files changed, 20 insertions(+)
arch/x86/events/core.c +0 -0
index 09050641ce5d..8167f2230d3a 100644
@@ -701,6 +701,23 @@ struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
 }
 EXPORT_SYMBOL_GPL(perf_guest_get_msrs);
 
Quoted 8 lines

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 709

Quoted 6 lines
arch/x86/events/core.c +6 -0
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));

If CONFIG_KVM is not defined, KVM_GUEST_PMI_VECTOR is not available and it causes compiling error.

Zide, thanks. If CONFIG_KVM is not defined, these two helpers would not need to be called, it can be defined as null.

#41 lore raw

Re: [PATCH v2 12/54] perf: x86: Add x86 function to switch PMI handler

In reply to Chen, Zide
Quoted 29 lines · Zide
Quoted 26 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add x86 specific function to switch PMI handler since passthrough PMU and host PMU use different interrupt vectors.

x86_perf_guest_enter() switch PMU vector from NMI to KVM_GUEST_PMI_VECTOR, and guest LVTPC_MASK value should be reflected onto HW to indicate whether guest has cleared LVTPC_MASK or not, so guest lvt_pc is passed as parameter.

x86_perf_guest_exit() switch PMU vector from KVM_GUEST_PMI_VECTOR to NMI.

---
 arch/x86/events/core.c            | 17 +++++++++++++++++
 arch/x86/include/asm/perf_event.h |  3 +++
 2 files changed, 20 insertions(+)
arch/x86/events/core.c +0 -0
index 09050641ce5d..8167f2230d3a 100644
@@ -701,6 +701,23 @@ struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data)
 }
 EXPORT_SYMBOL_GPL(perf_guest_get_msrs);
 
Quoted 9 lines

arch/x86/events/core.c struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr, void *data) · line 709

Quoted 6 lines
arch/x86/events/core.c +6 -0
+void x86_perf_guest_enter(u32 guest_lvtpc)
+{
+	lockdep_assert_irqs_disabled();
+
+	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			       (guest_lvtpc & APIC_LVT_MASKED));

If CONFIG_KVM is not defined, KVM_GUEST_PMI_VECTOR is not available and it causes compiling error.

That is a good discovery, thanks. hmm, we could put the whole function under IS_ENABLED(CONFIG_KVM) to avoid that.

Thanks.
-Mingwei

#42 lore raw

PATCH v2 13/54 perf: core/x86: Forbid PMI handler when guest own PMU

In reply to Mingwei Zhang

If a guest PMI is delivered after VM-exit, the KVM maskable interrupt will be held pending until EFLAGS.IF is set. In the meantime, if the logical processor receives an NMI for any reason at all, perf_event_nmi_handler() will be invoked. If there is any active perf event anywhere on the system, x86_pmu_handle_irq() will be invoked, and it will clear IA32_PERF_GLOBAL_STATUS. By the time KVM's PMI handler is invoked, it will be a mystery which counter(s) overflowed.

When LVTPC is using KVM PMI vecotr, PMU is owned by guest, Host NMI let x86_pmu_handle_irq() run, x86_pmu_handle_irq() restore PMU vector to NMI and clear IA32_PERF_GLOBAL_STATUS, this breaks guest vPMU passthrough environment.

So modify perf_event_nmi_handler() to check perf_guest_context_loaded, and if so, to simply return without calling x86_pmu_handle_irq().

---
 arch/x86/events/core.c     | 19 ++++++++++++++++++-
 include/linux/perf_event.h |  5 +++++
 kernel/events/core.c       |  5 +++++
 3 files changed, 28 insertions(+), 1 deletion(-)
arch/x86/events/core.c +18 -1
index 8167f2230d3a..c0f6e294fcad 100644
@@ -726,7 +726,7 @@ EXPORT_SYMBOL_GPL(x86_perf_guest_exit);
  * It will not be re-enabled in the NMI handler again, because enabled=0. After
  * handling the NMI, disable_all will be called, which will not change the
  * state either. If PMI hits after disable_all, the PMU is already disabled
- * before entering NMI handler. The NMI handler will not change the state
+ * before entering NMI handler. The NMI handler will no	change the state
  * either.
  *
  * So either situation is harmless.
@@ -1749,6 +1749,23 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
 	u64 finish_clock;
 	int ret;
 
+	/*
+	 * When guest pmu context is loaded this handler should be forbidden from
+	 * running, the reasons are:
+	 * 1. After x86_perf_guest_enter() is called, and before cpu enter into
+	 *    non-root mode, NMI could happen, but x86_pmu_handle_irq() restore PMU
+	 *    to use NMI vector, which destroy KVM PMI vector setting.
+	 * 2. When VM is running, host NMI other than PMI causes VM exit, KVM will
+	 *    call host NMI handler (vmx_vcpu_enter_exit()) first before KVM save
+	 *    guest PMU context (kvm_pmu_save_pmu_context()), as x86_pmu_handle_irq()
+	 *    clear global_status MSR which has guest status now, then this destroy
+	 *    guest PMU status.
+	 * 3. After VM exit, but before KVM save guest PMU context, host NMI other
+	 *    than PMI could happen, x86_pmu_handle_irq() clear global_status MSR
+	 *    which has guest status now, then this destroy guest PMU status.
+	 */
+	if (perf_is_guest_context_loaded())
+		return 0;
 	/*
 	 * All PMUs/events that share this PMI handler should make sure to
 	 * increment active_events for their events.
include/linux/perf_event.h +5 -0
index acf16676401a..5da7de42954e 100644
@@ -1736,6 +1736,7 @@ extern int perf_get_mediated_pmu(void);
 extern void perf_put_mediated_pmu(void);
 void perf_guest_enter(void);
 void perf_guest_exit(void);
+bool perf_is_guest_context_loaded(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1830,6 +1831,10 @@ static inline int perf_get_mediated_pmu(void)
 static inline void perf_put_mediated_pmu(void)			{ }
 static inline void perf_guest_enter(void)			{ }
 static inline void perf_guest_exit(void)			{ }
+static inline bool perf_is_guest_context_loaded(void)
+{
+	return false;
+}
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +5 -0
index 4c6daf5cc923..184d06c23391 100644
@@ -5895,6 +5895,11 @@ void perf_guest_exit(void)
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
+bool perf_is_guest_context_loaded(void)
+{
+	return __this_cpu_read(perf_in_guest);
+}
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#43 lore raw

Re: [PATCH v2 13/54] perf: core/x86: Forbid PMI handler when guest own PMU

Quoted 11 lines · Mingwei Zhang
arch/x86/events/core.c +7 -0
@@ -1749,6 +1749,23 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
 	u64 finish_clock;
 	int ret;
 
+	/*
+	 * When guest pmu context is loaded this handler should be forbidden from
+	 * running, the reasons are:
+	 * 1. After x86_perf_guest_enter() is called, and before cpu enter into
+	 *    non-root mode, NMI could happen, but x86_pmu_handle_irq() restore PMU
+	 *    to use NMI vector, which destroy KVM PMI vector setting.
+	 * 2. When VM is running, host NMI other than PMI causes VM exit, KVM will

arch/x86/events/core.c perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs) · line 1768

Quoted 10 lines
arch/x86/events/core.c +10 -0
+	 *    call host NMI handler (vmx_vcpu_enter_exit()) first before KVM save
+	 *    guest PMU context (kvm_pmu_save_pmu_context()), as x86_pmu_handle_irq()
+	 *    clear global_status MSR which has guest status now, then this destroy
+	 *    guest PMU status.
+	 * 3. After VM exit, but before KVM save guest PMU context, host NMI other
+	 *    than PMI could happen, x86_pmu_handle_irq() clear global_status MSR
+	 *    which has guest status now, then this destroy guest PMU status.
+	 */
+	if (perf_is_guest_context_loaded())
+		return 0;

A function call makes sense because? Also, isn't this naming at least a very little misleading? Specifically this is about passthrough, not guest context per se.

Quoted 45 lines
arch/x86/events/core.c +0 -0
 	/*
 	 * All PMUs/events that share this PMI handler should make sure to
 	 * increment active_events for their events.
include/linux/perf_event.h +5 -0
index acf16676401a..5da7de42954e 100644
@@ -1736,6 +1736,7 @@ extern int perf_get_mediated_pmu(void);
 extern void perf_put_mediated_pmu(void);
 void perf_guest_enter(void);
 void perf_guest_exit(void);
+bool perf_is_guest_context_loaded(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1830,6 +1831,10 @@ static inline int perf_get_mediated_pmu(void)
 static inline void perf_put_mediated_pmu(void)			{ }
 static inline void perf_guest_enter(void)			{ }
 static inline void perf_guest_exit(void)			{ }
+static inline bool perf_is_guest_context_loaded(void)
+{
+	return false;
+}
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +5 -0
index 4c6daf5cc923..184d06c23391 100644
@@ -5895,6 +5895,11 @@ void perf_guest_exit(void)
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
+bool perf_is_guest_context_loaded(void)
+{
+	return __this_cpu_read(perf_in_guest);
+}
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#44 lore raw

Re: [PATCH v2 13/54] perf: core/x86: Forbid PMI handler when guest own PMU

Quoted 17 lines · Peter Zijlstra
Quoted 15 lines · Mingwei Zhang
arch/x86/events/core.c +11 -0
@@ -1749,6 +1749,23 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
 	u64 finish_clock;
 	int ret;
 
+	/*
+	 * When guest pmu context is loaded this handler should be forbidden from
+	 * running, the reasons are:
+	 * 1. After x86_perf_guest_enter() is called, and before cpu enter into
+	 *    non-root mode, NMI could happen, but x86_pmu_handle_irq() restore PMU
+	 *    to use NMI vector, which destroy KVM PMI vector setting.
+	 * 2. When VM is running, host NMI other than PMI causes VM exit, KVM will
+	 *    call host NMI handler (vmx_vcpu_enter_exit()) first before KVM save
+	 *    guest PMU context (kvm_pmu_save_pmu_context()), as x86_pmu_handle_irq()
+	 *    clear global_status MSR which has guest status now, then this destroy
+	 *    guest PMU status.
Quoted 8 lines

arch/x86/events/core.c perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs) · line 1768

Quoted 6 lines
arch/x86/events/core.c +6 -0
+	 * 3. After VM exit, but before KVM save guest PMU context, host NMI other
+	 *    than PMI could happen, x86_pmu_handle_irq() clear global_status MSR
+	 *    which has guest status now, then this destroy guest PMU status.
+	 */
+	if (perf_is_guest_context_loaded())
+		return 0;

A function call makes sense because? Also, isn't this naming at least the purpose of function call is to re-use the per-cpu variable defined in

perf core, otherwise another per-cpu variable will be defined in arch/x86/event/core.c, whether function call or per-cpu variable depends on the interface between perf and KVM.

Quoted 49 lines

very little misleading? Specifically this is about passthrough, not guest context per se.

Quoted 45 lines
arch/x86/events/core.c +0 -0
 	/*
 	 * All PMUs/events that share this PMI handler should make sure to
 	 * increment active_events for their events.
include/linux/perf_event.h +5 -0
index acf16676401a..5da7de42954e 100644
@@ -1736,6 +1736,7 @@ extern int perf_get_mediated_pmu(void);
 extern void perf_put_mediated_pmu(void);
 void perf_guest_enter(void);
 void perf_guest_exit(void);
+bool perf_is_guest_context_loaded(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1830,6 +1831,10 @@ static inline int perf_get_mediated_pmu(void)
 static inline void perf_put_mediated_pmu(void)			{ }
 static inline void perf_guest_enter(void)			{ }
 static inline void perf_guest_exit(void)			{ }
+static inline bool perf_is_guest_context_loaded(void)
+{
+	return false;
+}
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +5 -0
index 4c6daf5cc923..184d06c23391 100644
@@ -5895,6 +5895,11 @@ void perf_guest_exit(void)
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
+bool perf_is_guest_context_loaded(void)
+{
+	return __this_cpu_read(perf_in_guest);
+}
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#45 lore raw

PATCH v2 14/54 perf: core/x86: Plumb passthrough PMU capability from x86_pmu to x86_pmu_cap

In reply to Mingwei Zhang

Plumb passthrough PMU capability to x86_pmu_cap in order to let any kernel entity such as KVM know that host PMU support passthrough PMU mode and has the implementation.

---
 arch/x86/events/core.c            | 1 +
 arch/x86/events/intel/core.c      | 1 +
 arch/x86/events/perf_event.h      | 1 +
 arch/x86/include/asm/perf_event.h | 1 +
 4 files changed, 4 insertions(+)
arch/x86/events/core.c +1 -0
index c0f6e294fcad..f5a043410614 100644
@@ -3023,6 +3023,7 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	cap->events_mask	= (unsigned int)x86_pmu.events_maskl;
 	cap->events_mask_len	= x86_pmu.events_mask_len;
 	cap->pebs_ept		= x86_pmu.pebs_ept;
+	cap->passthrough	= !!(x86_pmu.flags & PMU_FL_PASSTHROUGH);
 }
 EXPORT_SYMBOL_GPL(perf_get_x86_pmu_capability);
 
arch/x86/events/intel/core.c +1 -0
index 4d8f907a9416..62d327cb5424 100644
@@ -6246,6 +6246,7 @@ __init int intel_pmu_init(void)
 
 	/* The perf side of core PMU is ready to support the passthrough vPMU. */
 	x86_get_pmu(smp_processor_id())->capabilities |= PERF_PMU_CAP_PASSTHROUGH_VPMU;
+	x86_pmu.flags |= PMU_FL_PASSTHROUGH;
 
 	/*
 	 * Install the hw-cache-events table:
arch/x86/events/perf_event.h +1 -0
index fb56518356ec..bdf6d114d05a 100644
@@ -1020,6 +1020,7 @@ do {									\
 #define PMU_FL_MEM_LOADS_AUX	0x100 /* Require an auxiliary event for the complete memory info */
 #define PMU_FL_RETIRE_LATENCY	0x200 /* Support Retire Latency in PEBS */
 #define PMU_FL_BR_CNTR		0x400 /* Support branch counter logging */
+#define PMU_FL_PASSTHROUGH      0x800 /* Support passthrough mode */
 
 #define EVENT_VAR(_id)  event_attr_##_id
 #define EVENT_PTR(_id) &event_attr_##_id.attr.attr
arch/x86/include/asm/perf_event.h +1 -0
index 807ea9c98567..39a6379162bc 100644
@@ -258,6 +258,7 @@ struct x86_pmu_capability {
 	unsigned int	events_mask;
 	int		events_mask_len;
 	unsigned int	pebs_ept	:1;
+	unsigned int	passthrough	:1;
 };
 
 /*
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#46 lore raw

PATCH v2 15/54 KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

In reply to Mingwei Zhang

Introduce enable_passthrough_pmu as a RO KVM kernel module parameter. This variable is true only when the following conditions satisfies:
- set to true when module loaded.
- enable_pmu is true.
- is running on Intel CPU.
- supports PerfMon v4.
- host PMU supports passthrough mode.

The value is always read-only because passthrough PMU currently does not support features like LBR and PEBS, while emualted PMU does. This will end up with two different values for kvm_cap.supported_perf_cap, which is initialized at module load time. Maintaining two different perf capabilities will add complexity. Further, there is not enough motivation to support running two types of PMU implementations at the same time, although it is possible/feasible in reality.

Finally, always propagate enable_passthrough_pmu and perf_capabilities into kvm->arch for each KVM instance.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.h              | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c          |  7 +++++--
 arch/x86/kvm/x86.c              |  8 ++++++++
 arch/x86/kvm/x86.h              |  1 +
 5 files changed, 29 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 6efd1497b026..9851f0c8e91b 100644
@@ -1402,6 +1402,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
+	bool enable_passthrough_pmu;
 
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +14 -0
index 4d52b0b539ba..cf93be5e7359 100644
@@ -208,6 +208,20 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
+	/* Pass-through vPMU is only supported in Intel CPUs. */
+	if (!is_intel)
+		enable_passthrough_pmu = false;
+
+	/*
+	 * Pass-through vPMU requires at least PerfMon version 4 because the
+	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
+	 * for counter emulation as well as PMU context switch.  In addition, it
+	 * requires host PMU support on passthrough mode. Disable pass-through
+	 * vPMU if any condition fails.
+	 */
+	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)
+		enable_passthrough_pmu = false;
+
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
 		return;
arch/x86/kvm/vmx/vmx.c +5 -2
index c2dc68a25a53..af253cfa5c37 100644
@@ -144,6 +144,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
+module_param(enable_passthrough_pmu, bool, 0444);
+
 #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD)
 #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE
 #define KVM_VM_CR0_ALWAYS_ON				\
@@ -7874,13 +7876,14 @@ static u64 vmx_get_perf_capabilities(void)
 	if (boot_cpu_has(X86_FEATURE_PDCM))
 		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
 
-	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) {
+	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
+	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&lbr);
 		if (lbr.nr)
 			perf_cap |= host_perf_cap & PMU_CAP_LBR_FMT;
 	}
 
-	if (vmx_pebs_supported()) {
+	if (vmx_pebs_supported() && !enable_passthrough_pmu) {
 		perf_cap |= host_perf_cap & PERF_CAP_PEBS_MASK;
 		if ((perf_cap & PERF_CAP_PEBS_FORMAT) < 4)
 			perf_cap &= ~PERF_CAP_PEBS_BASELINE;
arch/x86/kvm/x86.c +8 -0
index 51b5a88222ef..4c289fcb34fe 100644
@@ -193,6 +193,10 @@ bool __read_mostly enable_pmu = true;
 EXPORT_SYMBOL_GPL(enable_pmu);
 module_param(enable_pmu, bool, 0444);
 
+/* Enable/disable mediated passthrough PMU virtualization */
+bool __read_mostly enable_passthrough_pmu;
+EXPORT_SYMBOL_GPL(enable_passthrough_pmu);
+
 bool __read_mostly eager_page_split = true;
 module_param(eager_page_split, bool, 0644);
 
@@ -6666,6 +6670,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
+			/* Disable passthrough PMU if enable_pmu is false. */
+			if (!kvm->arch.enable_pmu)
+				kvm->arch.enable_passthrough_pmu = false;
 			r = 0;
 		}
 		mutex_unlock(&kvm->lock);
@@ -12564,6 +12571,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz;
 	kvm->arch.guest_can_read_msr_platform_info = true;
 	kvm->arch.enable_pmu = enable_pmu;
+	kvm->arch.enable_passthrough_pmu = enable_passthrough_pmu;
 
 #if IS_ENABLED(CONFIG_HYPERV)
 	spin_lock_init(&kvm->arch.hv_root_tdp_lock);
arch/x86/kvm/x86.h +1 -0
index a8b71803777b..d5cc008e18f5 100644
@@ -330,6 +330,7 @@ extern u64 host_arch_capabilities;
 extern struct kvm_caps kvm_caps;
 
 extern bool enable_pmu;
+extern bool enable_passthrough_pmu;
 
 /*
  * Get a filtered version of KVM's supported XCR0 that strips out dynamic
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#47 lore raw

PATCH v2 16/54 KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs

In reply to Mingwei Zhang

Plumb through pass-through PMU setting from kvm->arch into kvm_pmu on each vcpu created. Note that enabling PMU is decided by VMM when it sets the CPUID bits exposed to guest VM. So plumb through the enabling for each pmu in intel_pmu_refresh().

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/pmu.c              |  1 +
 arch/x86/kvm/vmx/pmu_intel.c    | 12 +++++++++---
 3 files changed, 12 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 9851f0c8e91b..19b924c3bd85 100644
@@ -592,6 +592,8 @@ struct kvm_pmu {
 	 * redundant check before cleanup if guest don't use vPMU at all.
 	 */
 	u8 event_count;
+
+	bool passthrough;
 };
 
 struct kvm_pmu_ops;
arch/x86/kvm/pmu.c +1 -0
index a593b03c9aed..5768ea2935e9 100644
@@ -797,6 +797,7 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
 
 	memset(pmu, 0, sizeof(*pmu));
 	static_call(kvm_x86_pmu_init)(vcpu);
+	pmu->passthrough = false;
 	kvm_pmu_refresh(vcpu);
 }
 
arch/x86/kvm/vmx/pmu_intel.c +9 -3
index 12ade343a17e..0ed71f825e6b 100644
@@ -470,15 +470,21 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 		return;
 
 	entry = kvm_find_cpuid_entry(vcpu, 0xa);
-	if (!entry)
+	if (!entry || !vcpu->kvm->arch.enable_pmu) {
+		pmu->passthrough = false;
 		return;
-
+	}
 	eax.full = entry->eax;
 	edx.full = entry->edx;
 
 	pmu->version = eax.split.version_id;
-	if (!pmu->version)
+	if (!pmu->version) {
+		pmu->passthrough = false;
 		return;
+	}
+
+	pmu->passthrough = vcpu->kvm->arch.enable_passthrough_pmu &&
+			   lapic_in_kernel(vcpu);
 
 	pmu->nr_arch_gp_counters = min_t(int, eax.split.num_counters,
 					 kvm_pmu_cap.num_counters_gp);
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#48 lore raw

PATCH v2 17/54 KVM: x86/pmu: Always set global enable bits in passthrough mode

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

[removed the fixes tag]
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +1 -1
index 5768ea2935e9..e656f72fdace 100644
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
 }
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#49 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 29 lines · Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

[removed the fixes tag]
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index 5768ea2935e9..e656f72fdace 100644

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 790

Quoted 5 lines
arch/x86/kvm/pmu.c +1 -1
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)

The logic seems not correct. we could support perfmon version 1 for meidated vPMU (passthrough vPMU) as well in the future.  pmu->passthrough is ture doesn't guarantee GLOBAL_CTRL MSR always exists.

Quoted 3 lines
arch/x86/kvm/pmu.c +0 -0
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
 }
 
#50 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 27 lines · Dapeng
Quoted 24 lines · Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

[removed the fixes tag]
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 790

Quoted 10 lines
arch/x86/kvm/pmu.c +1 -1
index 5768ea2935e9..e656f72fdace 100644
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
       * in the global controls).  Emulate that behavior when refreshing the
       * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
       */
-     if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+     if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)

The logic seems not correct. we could support perfmon version 1 for meidated vPMU (passthrough vPMU) as well in the future. pmu->passthrough is ture doesn't guarantee GLOBAL_CTRL MSR always exists.

heh, the logic is correct here. However, I would say the code change may not reflect that clearly.

The if condition combines the handling of global ctrl registers for both the legacy vPMU and the mediated passthrough vPMU.

In legacy pmu, the logic should be this:

if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)

Because, since KVM emulates the MSR, if the global ctrl register does not exist, then there is no point resetting it to any value. However, if it does exist, there are non-zero number of GP counters, we should reset it to some value (all enabling bits are set for GP counters) according to SDM.

The logic for mediated passthrough PMU is different as follows:

if (pmu->passthrough && pmu->nr_arch_gp_counters)

Since mediated passthrough PMU requires PerfMon v4 in Intel (PerfMon v2 in AMD), once it is enabled (pmu->passthrough = true), then global ctrl _must_ exist phyiscally. Regardless of whether we expose it to the guest VM, at reset time, we need to ensure enabling bits for GP counters are set (behind the screen). This is critical for AMD, since most of the guests are usually in (AMD) PerfMon v1 in which global ctrl MSR is inaccessible, but does exist and is operating in HW.

Yes, if we eliminate that requirement (pmu->passthrough -> Perfmon v4 Intel / Perfmon v2 AMD), then this code will have to change. However, that is currently not in our RFCv2.

Thanks.
-Mingwei

Quoted 5 lines
Quoted 5 lines
arch/x86/kvm/pmu.c +0 -0
              pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
 }
 
#51 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 68 lines · Mingwei Zhang
Quoted 26 lines · Dapeng
Quoted 24 lines · Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

[removed the fixes tag]
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 13 lines

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 790

Quoted 10 lines
arch/x86/kvm/pmu.c +1 -1
index 5768ea2935e9..e656f72fdace 100644
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
       * in the global controls).  Emulate that behavior when refreshing the
       * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
       */
-     if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+     if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)

The logic seems not correct. we could support perfmon version 1 for meidated vPMU (passthrough vPMU) as well in the future. pmu->passthrough is ture doesn't guarantee GLOBAL_CTRL MSR always exists.

heh, the logic is correct here. However, I would say the code change may not reflect that clearly.

The if condition combines the handling of global ctrl registers for both the legacy vPMU and the mediated passthrough vPMU.

In legacy pmu, the logic should be this:

if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)

Because, since KVM emulates the MSR, if the global ctrl register does not exist, then there is no point resetting it to any value. However, if it does exist, there are non-zero number of GP counters, we should reset it to some value (all enabling bits are set for GP counters) according to SDM.

The logic for mediated passthrough PMU is different as follows:

if (pmu->passthrough && pmu->nr_arch_gp_counters)

Since mediated passthrough PMU requires PerfMon v4 in Intel (PerfMon v2 in AMD), once it is enabled (pmu->passthrough = true), then global ctrl _must_ exist phyiscally. Regardless of whether we expose it to the guest VM, at reset time, we need to ensure enabling bits for GP counters are set (behind the screen). This is critical for AMD, since most of the guests are usually in (AMD) PerfMon v1 in which global ctrl MSR is inaccessible, but does exist and is operating in HW.

Quoted 2 lines

Yes, if we eliminate that requirement (pmu->passthrough -> Perfmon v4 Intel / Perfmon v2 AMD), then this code will have to change. However,

Yeah, that's what I'm worrying about. We ever discussed to support mediated vPMU on HW below perfmon v4. When someone implements this, he may not notice this place needs to be changed as well, this introduces a potential bug and we should avoid this.

Quoted 15 lines

that is currently not in our RFCv2.

Thanks.
-Mingwei

Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/pmu.c +0 -0
              pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
 }
 
#52 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 12 lines · Dapeng Mi
Quoted 10 lines · Mingwei Zhang

if (pmu->passthrough && pmu->nr_arch_gp_counters)

Since mediated passthrough PMU requires PerfMon v4 in Intel (PerfMon v2 in AMD), once it is enabled (pmu->passthrough = true), then global ctrl _must_ exist phyiscally. Regardless of whether we expose it to the guest VM, at reset time, we need to ensure enabling bits for GP counters are set (behind the screen). This is critical for AMD, since most of the guests are usually in (AMD) PerfMon v1 in which global ctrl MSR is inaccessible, but does exist and is operating in HW.

Quoted 6 lines
Quoted 2 lines

Yes, if we eliminate that requirement (pmu->passthrough -> Perfmon v4 Intel / Perfmon v2 AMD), then this code will have to change. However,

Yeah, that's what I'm worrying about. We ever discussed to support mediated vPMU on HW below perfmon v4. When someone implements this, he may not notice this place needs to be changed as well, this introduces a potential bug and we should avoid this.

Just add a WARN on the PMU version. I haven't thought much about whether or not KVM should support mediated PMU for earlier hardware, but having a sanity check on the assumptions of this code is reasonable even if we don't _plan_ on supporting earlier hardware.

#53 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 14 lines · Sean Christopherson
Quoted 12 lines · Dapeng Mi
Quoted 10 lines · Mingwei Zhang

if (pmu->passthrough && pmu->nr_arch_gp_counters)

Since mediated passthrough PMU requires PerfMon v4 in Intel (PerfMon v2 in AMD), once it is enabled (pmu->passthrough = true), then global ctrl _must_ exist phyiscally. Regardless of whether we expose it to the guest VM, at reset time, we need to ensure enabling bits for GP counters are set (behind the screen). This is critical for AMD, since most of the guests are usually in (AMD) PerfMon v1 in which global ctrl MSR is inaccessible, but does exist and is operating in HW.

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

Yes, if we eliminate that requirement (pmu->passthrough -> Perfmon v4 Intel / Perfmon v2 AMD), then this code will have to change. However,

Yeah, that's what I'm worrying about. We ever discussed to support mediated vPMU on HW below perfmon v4. When someone implements this, he may not notice this place needs to be changed as well, this introduces a potential bug and we should avoid this.

I think you might have worried too much about future problems, but yes, things are under the radar. For Intel, this version constraint might be ok as Perfmon v4 is skylake, which is already pretty early.

For AMD, things are slightly different, PerfMon v2 in AMD requires Genoa, which is pretty new. So, this problem probably could be something for AMD if they want to extend the new vPMU design to Milan, but we will see how people think. So one potential (easy) extension for AMD is host PerfMon v1 + guest PerfMon v1 support for mediated passthrough vPMU.

Quoted 5 lines

Just add a WARN on the PMU version. I haven't thought much about whether or not KVM should support mediated PMU for earlier hardware, but having a sanity check on the assumptions of this code is reasonable even if we don't _plan_ on supporting earlier hardware.

Sure. That sounds pretty reasonable.

Thanks.
-Mingwei

#54 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 15 lines · Mingwei Zhang
Quoted 14 lines · Sean Christopherson
Quoted 13 lines · Dapeng Mi
Quoted 12 lines · Mingwei Zhang

if (pmu->passthrough && pmu->nr_arch_gp_counters)

Since mediated passthrough PMU requires PerfMon v4 in Intel (PerfMon v2 in AMD), once it is enabled (pmu->passthrough = true), then global ctrl _must_ exist phyiscally. Regardless of whether we expose it to the guest VM, at reset time, we need to ensure enabling bits for GP counters are set (behind the screen). This is critical for AMD, since most of the guests are usually in (AMD) PerfMon v1 in which global ctrl MSR is inaccessible, but does exist and is operating in HW.

Yes, if we eliminate that requirement (pmu->passthrough -> Perfmon v4 Intel / Perfmon v2 AMD), then this code will have to change. However,

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

Yeah, that's what I'm worrying about. We ever discussed to support mediated vPMU on HW below perfmon v4. When someone implements this, he may not notice this place needs to be changed as well, this introduces a potential bug and we should avoid this.

I think you might have worried too much about future problems, but yes, things are under the radar. For Intel, this version constraint might be ok as Perfmon v4 is skylake, which is already pretty early.

No, I don't think this is redundant worry since we did discuss this requirement before and it could need to be supported in the future. We need to consider the code's extensibility and not introduce potential issue.

Quoted 8 lines

For AMD, things are slightly different, PerfMon v2 in AMD requires Genoa, which is pretty new. So, this problem probably could be something for AMD if they want to extend the new vPMU design to Milan, but we will see how people think. So one potential (easy) extension for AMD is host PerfMon v1 + guest PerfMon v1 support for mediated passthrough vPMU.

Quoted 5 lines
Quoted 4 lines

Just add a WARN on the PMU version. I haven't thought much about whether or not KVM should support mediated PMU for earlier hardware, but having a sanity check on the assumptions of this code is reasonable even if we don't _plan_ on supporting earlier hardware.

Sure. That sounds pretty reasonable.

Good for me.

Quoted 4 lines

Thanks.
-Mingwei

#55 lore raw

Re: [PATCH v2 17/54] KVM: x86/pmu: Always set global enable bits in passthrough mode

In reply to Sean Christopherson
Quoted 14 lines · Sean Christopherson
Quoted 13 lines · Dapeng Mi
Quoted 12 lines · Mingwei Zhang

if (pmu->passthrough && pmu->nr_arch_gp_counters)

Since mediated passthrough PMU requires PerfMon v4 in Intel (PerfMon v2 in AMD), once it is enabled (pmu->passthrough = true), then global ctrl _must_ exist phyiscally. Regardless of whether we expose it to the guest VM, at reset time, we need to ensure enabling bits for GP counters are set (behind the screen). This is critical for AMD, since most of the guests are usually in (AMD) PerfMon v1 in which global ctrl MSR is inaccessible, but does exist and is operating in HW.

Yes, if we eliminate that requirement (pmu->passthrough -> Perfmon v4 Intel / Perfmon v2 AMD), then this code will have to change. However,

Quoted 8 lines
Quoted 4 lines

Yeah, that's what I'm worrying about. We ever discussed to support mediated vPMU on HW below perfmon v4. When someone implements this, he may not notice this place needs to be changed as well, this introduces a potential bug and we should avoid this.

Just add a WARN on the PMU version. I haven't thought much about whether or not KVM should support mediated PMU for earlier hardware, but having a sanity check on the assumptions of this code is reasonable even if we don't _plan_ on supporting earlier hardware.

I have no preference on whether supporting the old hardware (especially for Intel CPUs below v4 which has no GLOBAL_STATUS_SET MSR), but I think the key question is whether we want to totally drop the emulated vPMU after mediated passthrough vPMU becomes mature. If so, we may have to support the old platforms.

#56 lore raw

PATCH v2 18/54 KVM: x86/pmu: Add a helper to check if passthrough PMU is enabled

In reply to Mingwei Zhang

Add a helper to check if passthrough PMU is enabled for convenience as it is vendor neutral.

---
 arch/x86/kvm/pmu.h | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kvm/pmu.h +5 -0
index cf93be5e7359..56ba0772568c 100644
@@ -48,6 +48,11 @@ struct kvm_pmu_ops {
 
 void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops);
 
+static inline bool is_passthrough_pmu_enabled(struct kvm_vcpu *vcpu)
+{
+	return vcpu_to_pmu(vcpu)->passthrough;
+}
+
 static inline bool kvm_pmu_has_perf_global_ctrl(struct kvm_pmu *pmu)
 {
 	/*
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#57 lore raw

PATCH v2 19/54 KVM: x86/pmu: Add host_perf_cap and initialize it in kvm_x86_vendor_init()

In reply to Mingwei Zhang

Initialize host_perf_cap early in kvm_x86_vendor_init(). This helps KVM recognize the HW PMU capability against its guest VMs. This awareness directly decides the feasibility of passing through RDPMC and indirectly affects the performance in PMU context switch. Having the host PMU feature set cached in host_perf_cap saves a rdmsrl() to IA32_PERF_CAPABILITY MSR on each PMU context switch.

In addition, just opportunistically remove the host_perf_cap initialization in vmx_get_perf_capabilities() so the value is not dependent on module parameter "enable_pmu".

---
 arch/x86/kvm/pmu.h     | 1 +
 arch/x86/kvm/vmx/vmx.c | 4 ----
 arch/x86/kvm/x86.c     | 6 ++++++
 3 files changed, 7 insertions(+), 4 deletions(-)
arch/x86/kvm/pmu.h +1 -0
index 56ba0772568c..e041c8a23e2f 100644
@@ -295,4 +295,5 @@ bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
 extern struct kvm_pmu_ops intel_pmu_ops;
 extern struct kvm_pmu_ops amd_pmu_ops;
+extern u64 __read_mostly host_perf_cap;
 #endif /* __KVM_X86_PMU_H */
arch/x86/kvm/vmx/vmx.c +0 -4
index af253cfa5c37..a5024b7b0439 100644
@@ -7868,14 +7868,10 @@ static u64 vmx_get_perf_capabilities(void)
 {
 	u64 perf_cap = PMU_CAP_FW_WRITES;
 	struct x86_pmu_lbr lbr;
-	u64 host_perf_cap = 0;
 
 	if (!enable_pmu)
 		return 0;
 
-	if (boot_cpu_has(X86_FEATURE_PDCM))
-		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
-
 	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
 	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&lbr);
arch/x86/kvm/x86.c +6 -0
index 4c289fcb34fe..db395c00955f 100644
@@ -245,6 +245,9 @@ EXPORT_SYMBOL_GPL(host_xss);
 u64 __read_mostly host_arch_capabilities;
 EXPORT_SYMBOL_GPL(host_arch_capabilities);
 
+u64 __read_mostly host_perf_cap;
+EXPORT_SYMBOL_GPL(host_perf_cap);
+
 const struct _kvm_stats_desc kvm_vm_stats_desc[] = {
 	KVM_GENERIC_VM_STATS(),
 	STATS_DESC_COUNTER(VM, mmu_shadow_zapped),
@@ -9772,6 +9775,9 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
 		rdmsrl(MSR_IA32_ARCH_CAPABILITIES, host_arch_capabilities);
 
+	if (boot_cpu_has(X86_FEATURE_PDCM))
+		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
+
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#58 lore raw

PATCH v2 20/54 KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

In reply to Mingwei Zhang

Clear RDPMC_EXITING in vmcs when all counters on the host side are exposed to guest VM. This gives performance to passthrough PMU. However, when guest does not get all counters, intercept RDPMC to prevent access to unexposed counters. Make decision in vmx_vcpu_after_set_cpuid() when guest enables PMU and passthrough PMU is enabled.

---
 arch/x86/kvm/pmu.c     | 16 ++++++++++++++++
 arch/x86/kvm/pmu.h     |  1 +
 arch/x86/kvm/vmx/vmx.c |  5 +++++
 3 files changed, 22 insertions(+)
arch/x86/kvm/pmu.c +16 -0
index e656f72fdace..19104e16a986 100644
@@ -96,6 +96,22 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu) &&
+	    !enable_vmware_backdoor &&
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
+	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
+	    pmu->counter_bitmask[KVM_PMC_FIXED] == (((u64)1 << kvm_pmu_cap.bit_width_fixed)  - 1))
+		return true;
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(kvm_pmu_check_rdpmc_passthrough);
+
 static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
arch/x86/kvm/pmu.h +1 -0
index e041c8a23e2f..91941a0f6e47 100644
@@ -290,6 +290,7 @@ void kvm_pmu_cleanup(struct kvm_vcpu *vcpu);
 void kvm_pmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
arch/x86/kvm/vmx/vmx.c +5 -0
index a5024b7b0439..a18ba5ae5376 100644
@@ -7860,6 +7860,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);
+
 	/* Refresh #PF interception to account for MAXPHYADDR changes. */
 	vmx_update_exception_bitmap(vcpu);
 }
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#59 lore raw

Re: [PATCH v2 20/54] KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

arch/x86/kvm/vmx/vmx.c static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7867

Quoted 13 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +5 -0
index a5024b7b0439..a18ba5ae5376 100644
@@ -7860,6 +7860,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);
+

Seems it's cleaner to put the code inside vmx_set_perf_global_ctrl(), and change the name to reflect that it handles all the PMU related VMCS controls.

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 	/* Refresh #PF interception to account for MAXPHYADDR changes. */
 	vmx_update_exception_bitmap(vcpu);
 }
#60 lore raw

Re: [PATCH v2 20/54] KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

Quoted 11 lines · Zide
Quoted 8 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +0 -0
index a5024b7b0439..a18ba5ae5376 100644
@@ -7860,6 +7860,11 @@ static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
Quoted 9 lines

arch/x86/kvm/vmx/vmx.c static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7867

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +5 -0
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);
+

Seems it's cleaner to put the code inside vmx_set_perf_global_ctrl(), and change the name to reflect that it handles all the PMU related VMCS controls.

I prefer putting them separately, but I think I could put the above code into a different helper.

Thanks.
-Mingwei

Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 	/* Refresh #PF interception to account for MAXPHYADDR changes. */
 	vmx_update_exception_bitmap(vcpu);
 }
#61 lore raw

PATCH v2 21/54 KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS

In reply to Mingwei Zhang

From: Dapeng Mi <[email protected]>

Define macro PMU_CAP_PERF_METRICS to represent bit[15] of MSR_IA32_PERF_CAPABILITIES MSR. This bit is used to represent whether perf metrics feature is enabled.

---
 arch/x86/kvm/vmx/capabilities.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/vmx/capabilities.h +1 -0
index 41a4533f9989..d8317552b634 100644
@@ -22,6 +22,7 @@ extern int __read_mostly pt_mode;
 #define PT_MODE_HOST_GUEST	1
 
 #define PMU_CAP_FW_WRITES	(1ULL << 13)
+#define PMU_CAP_PERF_METRICS	BIT_ULL(15)
 #define PMU_CAP_LBR_FMT		0x3f
 
 struct nested_vmx_msrs {
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#62 lore raw

PATCH v2 22/54 KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough allowed

In reply to Mingwei Zhang

Introduce a vendor specific API to check if rdpmc passthrough allowed. RDPMC passthrough requires guest VM have the full ownership of all counters. These include general purpose counters and fixed counters and some vendor specific MSRs such as PERF_METRICS. Since PERF_METRICS MSR is Intel specific, putting the check into vendor specific code.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  1 +
 arch/x86/kvm/pmu.c                     |  1 +
 arch/x86/kvm/pmu.h                     |  1 +
 arch/x86/kvm/svm/pmu.c                 |  6 ++++++
 arch/x86/kvm/vmx/pmu_intel.c           | 16 ++++++++++++++++
 5 files changed, 25 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index f852b13aeefe..fd986d5146e4 100644
@@ -20,6 +20,7 @@ KVM_X86_PMU_OP(get_msr)
 KVM_X86_PMU_OP(set_msr)
 KVM_X86_PMU_OP(refresh)
 KVM_X86_PMU_OP(init)
+KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
arch/x86/kvm/pmu.c +1 -0
index 19104e16a986..3afefe4cf6e2 100644
@@ -102,6 +102,7 @@ bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)
 
 	if (is_passthrough_pmu_enabled(vcpu) &&
 	    !enable_vmware_backdoor &&
+	    static_call(kvm_x86_pmu_is_rdpmc_passthru_allowed)(vcpu) &&
 	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
 	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
 	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
arch/x86/kvm/pmu.h +1 -0
index 91941a0f6e47..e1af6d07b191 100644
@@ -40,6 +40,7 @@ struct kvm_pmu_ops {
 	void (*reset)(struct kvm_vcpu *vcpu);
 	void (*deliver_pmi)(struct kvm_vcpu *vcpu);
 	void (*cleanup)(struct kvm_vcpu *vcpu);
+	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
arch/x86/kvm/svm/pmu.c +6 -0
index dfcc38bd97d3..6b471b1ec9b8 100644
@@ -228,6 +228,11 @@ static void amd_pmu_init(struct kvm_vcpu *vcpu)
 	}
 }
 
+static bool amd_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
+{
+	return true;
+}
+
 struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = amd_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = amd_msr_idx_to_pmc,
@@ -237,6 +242,7 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.set_msr = amd_pmu_set_msr,
 	.refresh = amd_pmu_refresh,
 	.init = amd_pmu_init,
+	.is_rdpmc_passthru_allowed = amd_is_rdpmc_passthru_allowed,
 	.EVENTSEL_EVENT = AMD64_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_AMD_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = AMD64_NUM_COUNTERS,
arch/x86/kvm/vmx/pmu_intel.c +16 -0
index 0ed71f825e6b..ed79cbba1edc 100644
@@ -725,6 +725,21 @@ void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu)
 	}
 }
 
+static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Per Intel SDM vol. 2 for RDPMC, MSR_PERF_METRICS is accessible by
+	 * with type 0x2000 in ECX[31:16], while the index value in ECX[15:0] is
+	 * implementation specific. Therefore, if the host has this MSR, but
+	 * does not expose it to the guest, RDPMC has to be intercepted.
+	 */
+	if ((host_perf_cap & PMU_CAP_PERF_METRICS) &&
+	    !(vcpu_get_perf_capabilities(vcpu) & PMU_CAP_PERF_METRICS))
+		return false;
+
+	return true;
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -736,6 +751,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.reset = intel_pmu_reset,
 	.deliver_pmi = intel_pmu_deliver_pmi,
 	.cleanup = intel_pmu_cleanup,
+	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#63 lore raw

PATCH v2 23/54 KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

In PMU passthrough mode, there are three requirements to manage IA32_PERF_GLOBAL_CTRL:
- guest IA32_PERF_GLOBAL_CTRL MSR must be saved at vm exit.
- IA32_PERF_GLOBAL_CTRL MSR must be cleared at vm exit to avoid any counter of running within KVM runloop.
- guest IA32_PERF_GLOBAL_CTRL MSR must be restored at vm entry.

Introduce vmx_set_perf_global_ctrl() function to auto switching IA32_PERF_GLOBAL_CTR and invoke it after the VMM finishes setting up the CPUID bits.

---
 arch/x86/include/asm/vmx.h |   1 +
 arch/x86/kvm/vmx/vmx.c     | 117 +++++++++++++++++++++++++++++++------
 arch/x86/kvm/vmx/vmx.h     |   3 +-
 3 files changed, 103 insertions(+), 18 deletions(-)
arch/x86/include/asm/vmx.h +1 -0
index 4dba17363008..9b363e30e40c 100644
@@ -105,6 +105,7 @@
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
+#define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL      0x40000000
 
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
arch/x86/kvm/vmx/vmx.c +100 -17

147 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.h +2 -1
index 65786dbe7d60..1b56fac35656 100644
@@ -504,7 +504,8 @@ static inline u8 vmx_get_rvi(void)
 	       VM_EXIT_LOAD_IA32_EFER |					\
 	       VM_EXIT_CLEAR_BNDCFGS |					\
 	       VM_EXIT_PT_CONCEAL_PIP |					\
-	       VM_EXIT_CLEAR_IA32_RTIT_CTL)
+	       VM_EXIT_CLEAR_IA32_RTIT_CTL |                            \
+	       VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)
 
 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL			\
 	(PIN_BASED_EXT_INTR_MASK |					\
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#64 lore raw

PATCH v2 24/54 KVM: x86/pmu: Create a function prototype to disable MSR interception

In reply to Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/cpuid.c                   | 4 ++++
 arch/x86/kvm/pmu.c                     | 5 +++++
 arch/x86/kvm/pmu.h                     | 2 ++
 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index fd986d5146e4..1b7876dcb3c3 100644
@@ -24,6 +24,7 @@ KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
+KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/cpuid.c +4 -0
index 77352a4abd87..b577ba649feb 100644
@@ -381,6 +381,10 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_passthrough_pmu_msrs(vcpu);
+
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
arch/x86/kvm/pmu.c +5 -0
index 3afefe4cf6e2..bd94f2d67f5c 100644
@@ -1059,3 +1059,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	kfree(filter);
 	return r;
 }
+
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
+}
arch/x86/kvm/pmu.h +2 -0
index e1af6d07b191..63f876557716 100644
@@ -41,6 +41,7 @@ struct kvm_pmu_ops {
 	void (*deliver_pmi)(struct kvm_vcpu *vcpu);
 	void (*cleanup)(struct kvm_vcpu *vcpu);
 	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
+	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
@@ -292,6 +293,7 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
 bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#65 lore raw

Re: [PATCH v2 24/54] KVM: x86/pmu: Create a function prototype to disable MSR interception

Quoted 44 lines · Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/cpuid.c                   | 4 ++++
 arch/x86/kvm/pmu.c                     | 5 +++++
 arch/x86/kvm/pmu.h                     | 2 ++
 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index fd986d5146e4..1b7876dcb3c3 100644
@@ -24,6 +24,7 @@ KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
+KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/cpuid.c +4 -0
index 77352a4abd87..b577ba649feb 100644
@@ -381,6 +381,10 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_passthrough_pmu_msrs(vcpu);
+
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
arch/x86/kvm/pmu.c +0 -0
index 3afefe4cf6e2..bd94f2d67f5c 100644

arch/x86/kvm/pmu.c int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp) · line 1066

Quoted 8 lines
arch/x86/kvm/pmu.c +5 -0
@@ -1059,3 +1059,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	kfree(filter);
 	return r;
 }
+
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
+}

Don't quite understand why a separate callback is needed. It seems it's not messier if put this logic in the kvm_x86_vcpu_after_set_cpuid() callback.

#66 lore raw

Re: [PATCH v2 24/54] KVM: x86/pmu: Create a function prototype to disable MSR interception

Quoted 51 lines · Zide
Quoted 48 lines · Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/cpuid.c                   | 4 ++++
 arch/x86/kvm/pmu.c                     | 5 +++++
 arch/x86/kvm/pmu.h                     | 2 ++
 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index fd986d5146e4..1b7876dcb3c3 100644
@@ -24,6 +24,7 @@ KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
+KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/cpuid.c +4 -0
index 77352a4abd87..b577ba649feb 100644
@@ -381,6 +381,10 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_passthrough_pmu_msrs(vcpu);
+
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
arch/x86/kvm/pmu.c +1 -0
index 3afefe4cf6e2..bd94f2d67f5c 100644
@@ -1059,3 +1059,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	kfree(filter);
 	return r;
 }
+
Quoted 8 lines

arch/x86/kvm/pmu.c int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp) · line 1066

Quoted 4 lines
arch/x86/kvm/pmu.c +4 -0
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
+}

Don't quite understand why a separate callback is needed. It seems it's not messier if put this logic in the kvm_x86_vcpu_after_set_cpuid() callback.

One of the key point here is whether we _can_ intercept RDPMC. We have to intercept it if there is _any_ counters / MSRs that is accessible by rdmpc. In Intel CPU, the PERF_METRICS MSR is accessible from RDPMC. This MSR is a vendor specific one. So that's why we added another vendor API.

Thanks.
-Mingwei

#67 lore raw

PATCH v2 25/54 KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU MSRs

In reply to Mingwei Zhang

From: Dapeng Mi <[email protected]>

Event selectors for GP counters and fixed counters control MSR are intercepted for the purpose of security, i.e., preventing guest from using unallowed events to steal information or take advantages of any CPU errata.

Other than event selectors, disable PMU counter MSR interception specified in guest CPUID, counter MSR index outside of exported range will still be intercepted.

Global registers like global_ctrl will passthrough only if pmu version is greater than 1.

---
 arch/x86/kvm/cpuid.c         |  3 +--
 arch/x86/kvm/vmx/pmu_intel.c | 47 ++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +1 -2
index b577ba649feb..99e6cca67beb 100644
@@ -382,8 +382,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 
 	kvm_pmu_refresh(vcpu);
 
-	if (is_passthrough_pmu_enabled(vcpu))
-		kvm_pmu_passthrough_pmu_msrs(vcpu);
+	kvm_pmu_passthrough_pmu_msrs(vcpu);
 
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
arch/x86/kvm/vmx/pmu_intel.c +47 -0

59 lines not on this page · raw message

Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#68 lore raw

PATCH v2 26/54 KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

In reply to Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +2 -1
index bd94f2d67f5c..e9047051489e 100644
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);
 		}
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#69 lore raw

Re: [PATCH v2 26/54] KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

Quoted 16 lines · Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index bd94f2d67f5c..e9047051489e 100644

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 717

Quoted 6 lines
arch/x86/kvm/pmu.c +2 -1
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);

Since in [PATCH 44/54], reprogram_counters() is effectively skipped in the passthrough case, is this patch still needed?

Quoted 3 lines
arch/x86/kvm/pmu.c +0 -0
 		}
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
#70 lore raw

Re: [PATCH v2 26/54] KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

Quoted 13 lines · Zide
Quoted 11 lines · Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 13 lines

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 717

Quoted 11 lines
arch/x86/kvm/pmu.c +2 -1
index bd94f2d67f5c..e9047051489e 100644
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);

Since in [PATCH 44/54], reprogram_counters() is effectively skipped in the passthrough case, is this patch still needed?

Zide, reprogram_counters() and reprogram_counter() are two different helpers. Both they need to be skipped in passthrough mode.

Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/pmu.c +0 -0
 		}
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
#71 lore raw

Re: [PATCH v2 26/54] KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

Quoted 28 lines · Dapeng
Quoted 13 lines · Zide
Quoted 11 lines · Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 13 lines

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 717

Quoted 11 lines
arch/x86/kvm/pmu.c +2 -1
index bd94f2d67f5c..e9047051489e 100644
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);

Since in [PATCH 44/54], reprogram_counters() is effectively skipped in the passthrough case, is this patch still needed?

Quoted 2 lines

Zide, reprogram_counters() and reprogram_counter() are two different helpers. Both they need to be skipped in passthrough mode.

Yes, but this is talking about reprogram_counters() only. passthrough mode is being checked inside and outside the function call, which is redundant.

#72 lore raw

Re: [PATCH v2 26/54] KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

Quoted 28 lines · Zide
Quoted 26 lines · Dapeng
Quoted 25 lines · Zide
Quoted 16 lines · Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index bd94f2d67f5c..e9047051489e 100644

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 717

Quoted 6 lines
arch/x86/kvm/pmu.c +2 -1
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);

Since in [PATCH 44/54], reprogram_counters() is effectively skipped in the passthrough case, is this patch still needed?

Quoted 5 lines
Quoted 2 lines

Zide, reprogram_counters() and reprogram_counter() are two different helpers. Both they need to be skipped in passthrough mode.

Yes, but this is talking about reprogram_counters() only. passthrough mode is being checked inside and outside the function call, which is redundant.

Oh, yes. I don't need this patch then.

#73 lore raw

Re: [PATCH v2 26/54] KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

Quoted 30 lines · Dapeng
Quoted 28 lines · Zide
Quoted 26 lines · Dapeng
Quoted 25 lines · Zide
Quoted 16 lines · Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index bd94f2d67f5c..e9047051489e 100644

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 717

Quoted 6 lines
arch/x86/kvm/pmu.c +2 -1
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);

Since in [PATCH 44/54], reprogram_counters() is effectively skipped in the passthrough case, is this patch still needed?

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Zide, reprogram_counters() and reprogram_counter() are two different helpers. Both they need to be skipped in passthrough mode.

Yes, but this is talking about reprogram_counters() only. passthrough mode is being checked inside and outside the function call, which is redundant.

Oh, yes. I don't need this patch then.

Right. I am thinking about dropping [PATCH 44/54], since that one contains some redundant checking. I will see how should this be fixed in next version. Thanks for pointing it out though.

-Mingwei

#74 lore raw

PATCH v2 27/54 KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()

In reply to Mingwei Zhang

Reject PMU MSRs interception explicitly in vmx_get_passthrough_msr_slot() since interception of PMU MSRs are specially handled in intel_passthrough_pmu_msrs().

---
 arch/x86/kvm/vmx/vmx.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +8 -1
index c9de7d2623b8..62b5913abdd6 100644
@@ -164,7 +164,7 @@ module_param(enable_passthrough_pmu, bool, 0444);
 
 /*
  * List of MSRs that can be directly passed to the guest.
- * In addition to these x2apic, PT and LBR MSRs are handled specially.
+ * In addition to these x2apic, PMU, PT and LBR MSRs are handled specially.
  */
 static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {
 	MSR_IA32_SPEC_CTRL,
@@ -694,6 +694,13 @@ static int vmx_get_passthrough_msr_slot(u32 msr)
 	case MSR_LBR_CORE_FROM ... MSR_LBR_CORE_FROM + 8:
 	case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8:
 		/* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */
+	case MSR_IA32_PMC0 ... MSR_IA32_PMC0 + 7:
+	case MSR_IA32_PERFCTR0 ... MSR_IA32_PERFCTR0 + 7:
+	case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + 2:
+	case MSR_CORE_PERF_GLOBAL_STATUS:
+	case MSR_CORE_PERF_GLOBAL_CTRL:
+	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
+		/* PMU MSRs. These are handled in intel_passthrough_pmu_msrs() */
 		return -ENOENT;
 	}
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#75 lore raw

Re: [PATCH v2 27/54] KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()

Quoted 24 lines · Mingwei Zhang

Reject PMU MSRs interception explicitly in vmx_get_passthrough_msr_slot() since interception of PMU MSRs are specially handled in intel_passthrough_pmu_msrs().

---
 arch/x86/kvm/vmx/vmx.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
index c9de7d2623b8..62b5913abdd6 100644
@@ -164,7 +164,7 @@ module_param(enable_passthrough_pmu, bool, 0444);
 
 /*
  * List of MSRs that can be directly passed to the guest.
- * In addition to these x2apic, PT and LBR MSRs are handled specially.
+ * In addition to these x2apic, PMU, PT and LBR MSRs are handled specially.
  */
 static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {
 	MSR_IA32_SPEC_CTRL,

arch/x86/kvm/vmx/vmx.c static int vmx_get_passthrough_msr_slot(u32 msr) · line 699

Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +3 -0
@@ -694,6 +694,13 @@ static int vmx_get_passthrough_msr_slot(u32 msr)
 	case MSR_LBR_CORE_FROM ... MSR_LBR_CORE_FROM + 8:
 	case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8:
 		/* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */
+	case MSR_IA32_PMC0 ... MSR_IA32_PMC0 + 7:
+	case MSR_IA32_PERFCTR0 ... MSR_IA32_PERFCTR0 + 7:
+	case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + 2:

We'd better use helpers to get the maximum supported GP and fixed counter instead of magic numbers here. There are more GP and fixed counters in the future's Intel CPUs.

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +4 -0
+	case MSR_CORE_PERF_GLOBAL_STATUS:
+	case MSR_CORE_PERF_GLOBAL_CTRL:
+	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
+		/* PMU MSRs. These are handled in intel_passthrough_pmu_msrs() */
 		return -ENOENT;
 	}
 
#76 lore raw

PATCH v2 28/54 KVM: x86/pmu: Add counter MSR and selector MSR index into struct kvm_pmc

In reply to Mingwei Zhang

Add the MSR indices for both selector and counter in each kvm_pmc. Giving convenience to mediated passthrough vPMU in scenarios of querying MSR from a given pmc. Note that legacy vPMU does not need this because it never directly accesses PMU MSRs, instead each kvm_pmc is bound to a perf_event.

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/svm/pmu.c          | 13 +++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c    | 13 +++++++++++++
 3 files changed, 28 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
index 19b924c3bd85..8b4ea9bdcc74 100644
@@ -519,6 +519,8 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 msr_counter;
+	u64 msr_eventsel;
 	struct perf_event *perf_event;
 	struct kvm_vcpu *vcpu;
 	/*
arch/x86/kvm/svm/pmu.c +13 -0
index 6b471b1ec9b8..447657513729 100644
@@ -177,6 +177,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	union cpuid_0x80000022_ebx ebx;
+	int i;
 
 	pmu->version = 1;
 	if (guest_cpuid_has(vcpu, X86_FEATURE_PERFMON_V2)) {
@@ -210,6 +211,18 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
 	pmu->nr_arch_fixed_counters = 0;
 	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
+
+	if (guest_cpuid_has(vcpu, X86_FEATURE_PERFCTR_CORE)) {
+		for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+			pmu->gp_counters[i].msr_eventsel = MSR_F15H_PERF_CTL0 + 2*i;
+			pmu->gp_counters[i].msr_counter = MSR_F15H_PERF_CTR0 + 2*i;
+		}
+	} else {
+		for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+			pmu->gp_counters[i].msr_eventsel = MSR_K7_EVNTSEL0 + i;
+			pmu->gp_counters[i].msr_counter = MSR_K7_PERFCTR0 + i;
+		}
+	}
 }
 
 static void amd_pmu_init(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/pmu_intel.c +13 -0
index 8e8d1f2aa5e5..7852ba25a240 100644
@@ -562,6 +562,19 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				~((1ull << pmu->nr_arch_gp_counters) - 1);
 		}
 	}
+
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmu->gp_counters[i].msr_eventsel = MSR_P6_EVNTSEL0 + i;
+		if (fw_writes_is_enabled(vcpu))
+			pmu->gp_counters[i].msr_counter = MSR_IA32_PMC0 + i;
+		else
+			pmu->gp_counters[i].msr_counter = MSR_IA32_PERFCTR0 + i;
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmu->fixed_counters[i].msr_eventsel = MSR_CORE_PERF_FIXED_CTR_CTRL;
+		pmu->fixed_counters[i].msr_counter = MSR_CORE_PERF_FIXED_CTR0 + i;
+	}
 }
 
 static void intel_pmu_init(struct kvm_vcpu *vcpu)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#77 lore raw

PATCH v2 29/54 KVM: x86/pmu: Introduce PMU operation prototypes for save/restore PMU context

In reply to Mingwei Zhang

Plumb through kvm_pmu_ops with these two extra functions to allow pmu context switch.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  2 ++
 arch/x86/kvm/pmu.c                     | 14 ++++++++++++++
 arch/x86/kvm/pmu.h                     |  4 ++++
 3 files changed, 20 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +2 -0
index 1b7876dcb3c3..1a848ba6a7a7 100644
@@ -25,6 +25,8 @@ KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
 KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
+KVM_X86_PMU_OP_OPTIONAL(save_pmu_context)
+KVM_X86_PMU_OP_OPTIONAL(restore_pmu_context)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/pmu.c +14 -0
index e9047051489e..782b564bdf96 100644
@@ -1065,3 +1065,17 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 {
 	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
 }
+
+void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
+{
+	lockdep_assert_irqs_disabled();
+
+	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+}
+
+void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
+{
+	lockdep_assert_irqs_disabled();
+
+	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
+}
arch/x86/kvm/pmu.h +4 -0
index 63f876557716..8bd4b79e363f 100644
@@ -42,6 +42,8 @@ struct kvm_pmu_ops {
 	void (*cleanup)(struct kvm_vcpu *vcpu);
 	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
 	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);
+	void (*save_pmu_context)(struct kvm_vcpu *vcpu);
+	void (*restore_pmu_context)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
@@ -294,6 +296,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
 bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
 void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
+void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu);
+void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#78 lore raw

PATCH v2 30/54 KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU

In reply to Mingwei Zhang

Implement the save/restore of PMU state for pasthrough PMU in Intel. In passthrough mode, KVM owns exclusively the PMU HW when control flow goes to the scope of passthrough PMU. Thus, KVM needs to save the host PMU state and gains the full HW PMU ownership. On the contrary, host regains the ownership of PMU HW from KVM when control flow leaves the scope of passthrough PMU.

Implement PMU context switches for Intel CPUs and opptunistically use rdpmcl() instead of rdmsrl() when reading counters since the former has lower latency in Intel CPUs.

---
 arch/x86/kvm/pmu.c           | 46 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c | 41 +++++++++++++++++++++++++++++++-
 2 files changed, 86 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +46 -0
index 782b564bdf96..13197472e31d 100644
@@ -1068,14 +1068,60 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	u32 i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+
+	/*
+	 * Clear hardware selector MSR content and its counter to avoid
+	 * leakage and also avoid this guest GP counter get accidentally
+	 * enabled during host running when host enable global ctrl.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		rdmsrl(pmc->msr_counter, pmc->counter);
+		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+		if (pmc->eventsel)
+			wrmsrl(pmc->msr_eventsel, 0);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		rdmsrl(pmc->msr_counter, pmc->counter);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+	}
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	int i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
+
+	/*
+	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
+	 * in this case will be intercepted. Accessing to these counters and
+	 * selectors will cause #GP in the guest.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+	}
 }
arch/x86/kvm/vmx/pmu_intel.c +40 -1

59 lines not on this page · raw message

Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#79 lore raw

Re: [PATCH v2 30/54] KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU

Quoted 10 lines · Mingwei Zhang

Implement the save/restore of PMU state for pasthrough PMU in Intel. In passthrough mode, KVM owns exclusively the PMU HW when control flow goes to the scope of passthrough PMU. Thus, KVM needs to save the host PMU state and gains the full HW PMU ownership. On the contrary, host regains the ownership of PMU HW from KVM when control flow leaves the scope of passthrough PMU.

Implement PMU context switches for Intel CPUs and opptunistically use rdpmcl() instead of rdmsrl() when reading counters since the former has lower latency in Intel CPUs.

It looks rdpmcl() optimization is removed from this patch, right? The description is not identical with code.

Quoted 3 lines

The commit tags doesn't follow the rules in Linux document, we need to change it in next version.

https://docs.kernel.org/process/submitting-patches.html#:~:text=Co%2Ddeveloped%2Dby%3A%20states,work%20on%20a%20single%20patch.

Quoted 19 lines
---
 arch/x86/kvm/pmu.c           | 46 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c | 41 +++++++++++++++++++++++++++++++-
 2 files changed, 86 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +4 -0
index 782b564bdf96..13197472e31d 100644
@@ -1068,14 +1068,60 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	u32 i;
+
 	lockdep_assert_irqs_disabled();
 

arch/x86/kvm/pmu.c void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu) · line 1086

Quoted 10 lines
arch/x86/kvm/pmu.c +9 -0
 	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+
+	/*
+	 * Clear hardware selector MSR content and its counter to avoid
+	 * leakage and also avoid this guest GP counter get accidentally
+	 * enabled during host running when host enable global ctrl.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		rdmsrl(pmc->msr_counter, pmc->counter);

I understood we want to use common code to manipulate PMU MSRs as much as possible, but I don't think we should sacrifice performance. rdpmcl() has better performance than rdmsrl(). If AMD CPUs doesn't support rdpmc instruction, I think we should move this into vendor specific xxx_save/restore_pmu_context helpers().

Quoted 46 lines
arch/x86/kvm/pmu.c +33 -0
+		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+		if (pmc->eventsel)
+			wrmsrl(pmc->msr_eventsel, 0);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		rdmsrl(pmc->msr_counter, pmc->counter);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+	}
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	int i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
+
+	/*
+	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
+	 * in this case will be intercepted. Accessing to these counters and
+	 * selectors will cause #GP in the guest.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+	}
 }
arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 7852ba25a240..a23cf9ca224e 100644

arch/x86/kvm/vmx/pmu_intel.c static void intel_pmu_refresh(struct kvm_vcpu *vcpu) · line 575

Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +1 -1
@@ -572,7 +572,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
-		pmu->fixed_counters[i].msr_eventsel = MSR_CORE_PERF_FIXED_CTR_CTRL;
+		pmu->fixed_counters[i].msr_eventsel = 0;

Why to initialize msr_eventsel to 0 instead of the real MSR address here?

Quoted 56 lines
arch/x86/kvm/vmx/pmu_intel.c +39 -0
 		pmu->fixed_counters[i].msr_counter = MSR_CORE_PERF_FIXED_CTR0 + i;
 	}
 }
@@ -799,6 +799,43 @@ static void intel_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, MSR_TYPE_RW, msr_intercept);
 }
 
+static void intel_save_guest_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	/* Global ctrl register is already saved at VM-exit. */
+	rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, pmu->global_status);
+	/* Clear hardware MSR_CORE_PERF_GLOBAL_STATUS MSR, if non-zero. */
+	if (pmu->global_status)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status);
+
+	rdmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+	/*
+	 * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and
+	 * also avoid these guest fixed counters get accidentially enabled
+	 * during host running when host enable global ctrl.
+	 */
+	if (pmu->fixed_ctr_ctrl)
+		wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, 0);
+}
+
+static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	u64 global_status, toggle;
+
+	/* Clear host global_ctrl MSR if non-zero. */
+	wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, 0);
+	rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, global_status);
+	toggle = pmu->global_status ^ global_status;
+	if (global_status & toggle)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle);
+	if (pmu->global_status & toggle)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
+
+	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -812,6 +849,8 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.cleanup = intel_pmu_cleanup,
 	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
 	.passthrough_pmu_msrs = intel_passthrough_pmu_msrs,
+	.save_pmu_context = intel_save_guest_pmu_context,
+	.restore_pmu_context = intel_restore_guest_pmu_context,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
#80 lore raw

Re: [PATCH v2 30/54] KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU

Quoted 9 lines · Dapeng
Quoted 7 lines · Mingwei Zhang

Implement the save/restore of PMU state for pasthrough PMU in Intel. In passthrough mode, KVM owns exclusively the PMU HW when control flow goes to the scope of passthrough PMU. Thus, KVM needs to save the host PMU state and gains the full HW PMU ownership. On the contrary, host regains the ownership of PMU HW from KVM when control flow leaves the scope of passthrough PMU.

Quoted 6 lines
Quoted 3 lines

Implement PMU context switches for Intel CPUs and opptunistically use rdpmcl() instead of rdmsrl() when reading counters since the former has lower latency in Intel CPUs.

It looks rdpmcl() optimization is removed from this patch, right? The description is not identical with code.

That is correct. I was debugging this for a while and since we don't have rdpmcl_safe(), one of the bug cause rdpmc() to crash the kernel. Really don't like rdpmc(). But will add it back in next version.

Quoted 8 lines
Quoted 5 lines

The commit tags doesn't follow the rules in Linux document, we need to change it in next version.

Will do.

Quoted 149 lines

https://docs.kernel.org/process/submitting-patches.html#:~:text=Co%2Ddeveloped%2Dby%3A%20states,work%20on%20a%20single%20patch.

Quoted 19 lines
---
 arch/x86/kvm/pmu.c           | 46 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c | 41 +++++++++++++++++++++++++++++++-
 2 files changed, 86 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +4 -0
index 782b564bdf96..13197472e31d 100644
@@ -1068,14 +1068,60 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	u32 i;
+
 	lockdep_assert_irqs_disabled();
 

arch/x86/kvm/pmu.c void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu) · line 1086

Quoted 10 lines
arch/x86/kvm/pmu.c +9 -0
 	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+
+	/*
+	 * Clear hardware selector MSR content and its counter to avoid
+	 * leakage and also avoid this guest GP counter get accidentally
+	 * enabled during host running when host enable global ctrl.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		rdmsrl(pmc->msr_counter, pmc->counter);

I understood we want to use common code to manipulate PMU MSRs as much as possible, but I don't think we should sacrifice performance. rdpmcl() has better performance than rdmsrl(). If AMD CPUs doesn't support rdpmc instruction, I think we should move this into vendor specific xxx_save/restore_pmu_context helpers().

Quoted 46 lines
arch/x86/kvm/pmu.c +33 -0
+		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+		if (pmc->eventsel)
+			wrmsrl(pmc->msr_eventsel, 0);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		rdmsrl(pmc->msr_counter, pmc->counter);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+	}
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	int i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
+
+	/*
+	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
+	 * in this case will be intercepted. Accessing to these counters and
+	 * selectors will cause #GP in the guest.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+	}
 }
arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 7852ba25a240..a23cf9ca224e 100644

arch/x86/kvm/vmx/pmu_intel.c static void intel_pmu_refresh(struct kvm_vcpu *vcpu) · line 575

Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +1 -1
@@ -572,7 +572,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
-		pmu->fixed_counters[i].msr_eventsel = MSR_CORE_PERF_FIXED_CTR_CTRL;
+		pmu->fixed_counters[i].msr_eventsel = 0;

Why to initialize msr_eventsel to 0 instead of the real MSR address here?

Quoted 56 lines
arch/x86/kvm/vmx/pmu_intel.c +39 -0
 		pmu->fixed_counters[i].msr_counter = MSR_CORE_PERF_FIXED_CTR0 + i;
 	}
 }
@@ -799,6 +799,43 @@ static void intel_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, MSR_TYPE_RW, msr_intercept);
 }
 
+static void intel_save_guest_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	/* Global ctrl register is already saved at VM-exit. */
+	rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, pmu->global_status);
+	/* Clear hardware MSR_CORE_PERF_GLOBAL_STATUS MSR, if non-zero. */
+	if (pmu->global_status)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status);
+
+	rdmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+	/*
+	 * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and
+	 * also avoid these guest fixed counters get accidentially enabled
+	 * during host running when host enable global ctrl.
+	 */
+	if (pmu->fixed_ctr_ctrl)
+		wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, 0);
+}
+
+static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	u64 global_status, toggle;
+
+	/* Clear host global_ctrl MSR if non-zero. */
+	wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, 0);
+	rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, global_status);
+	toggle = pmu->global_status ^ global_status;
+	if (global_status & toggle)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle);
+	if (pmu->global_status & toggle)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
+
+	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -812,6 +849,8 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.cleanup = intel_pmu_cleanup,
 	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
 	.passthrough_pmu_msrs = intel_passthrough_pmu_msrs,
+	.save_pmu_context = intel_save_guest_pmu_context,
+	.restore_pmu_context = intel_restore_guest_pmu_context,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
#81 lore raw

PATCH v2 31/54 KVM: x86/pmu: Make check_pmu_event_filter() an exported function

In reply to Mingwei Zhang

Make check_pmu_event_filter() exported to usable by vendor modules like kvm_intel. This is because passthrough PMU intercept the guest writes to event selectors and directly do the event filter checking inside the vendor specific set_msr() instead of deferring to the KVM_REQ_PMU handler.

---
 arch/x86/kvm/pmu.c | 3 ++-
 arch/x86/kvm/pmu.h | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +2 -1
index 13197472e31d..0f587651a49e 100644
@@ -443,7 +443,7 @@ static bool is_fixed_event_allowed(struct kvm_x86_pmu_event_filter *filter,
 	return true;
 }
 
-static bool check_pmu_event_filter(struct kvm_pmc *pmc)
+bool check_pmu_event_filter(struct kvm_pmc *pmc)
 {
 	struct kvm_x86_pmu_event_filter *filter;
 	struct kvm *kvm = pmc->vcpu->kvm;
@@ -457,6 +457,7 @@ static bool check_pmu_event_filter(struct kvm_pmc *pmc)
 
 	return is_fixed_event_allowed(filter, pmc->idx);
 }
+EXPORT_SYMBOL_GPL(check_pmu_event_filter);
 
 static bool pmc_event_is_allowed(struct kvm_pmc *pmc)
 {
arch/x86/kvm/pmu.h +1 -0
index 8bd4b79e363f..9cde62f3988e 100644
@@ -298,6 +298,7 @@ bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
 void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu);
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu);
+bool check_pmu_event_filter(struct kvm_pmc *pmc);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#82 lore raw

PATCH v2 32/54 KVM: x86/pmu: Allow writing to event selector for GP counters if event is allowed

In reply to Mingwei Zhang

Only allow writing to event selector if event is allowed in filter. Since passthrough PMU implementation does the PMU context switch at VM Enter/Exit boudary, even if the value of event selector passes the checking, it cannot be written directly to HW since PMU HW is owned by the host PMU at the moment. Because of that, introduce eventsel_hw to cache that value which will be assigned into HW just before VM entry.

Note that regardless of whether an event value is allowed, the value will be cached in pmc->eventsel and guest VM can always read the cached value back. This implementation is consistent with the HW CPU design.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.c              |  5 ++---
 arch/x86/kvm/vmx/pmu_intel.c    | 13 ++++++++++++-
 3 files changed, 15 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 8b4ea9bdcc74..b396000b9440 100644
@@ -519,6 +519,7 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 eventsel_hw;
 	u64 msr_counter;
 	u64 msr_eventsel;
 	struct perf_event *perf_event;
arch/x86/kvm/pmu.c +2 -3
index 0f587651a49e..2ad71020a2c0 100644
@@ -1085,10 +1085,9 @@ void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
 		pmc = &pmu->gp_counters[i];
 		rdmsrl(pmc->msr_counter, pmc->counter);
-		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
 		if (pmc->counter)
 			wrmsrl(pmc->msr_counter, 0);
-		if (pmc->eventsel)
+		if (pmc->eventsel_hw)
 			wrmsrl(pmc->msr_eventsel, 0);
 	}
 
@@ -1118,7 +1117,7 @@ void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
 		pmc = &pmu->gp_counters[i];
 		wrmsrl(pmc->msr_counter, pmc->counter);
-		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel_hw);
 	}
 
 	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
arch/x86/kvm/vmx/pmu_intel.c +12 -1
index a23cf9ca224e..e706d107ff28 100644
@@ -399,7 +399,18 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			if (data & reserved_bits)
 				return 1;
 
-			if (data != pmc->eventsel) {
+			if (is_passthrough_pmu_enabled(vcpu)) {
+				pmc->eventsel = data;
+				if (!check_pmu_event_filter(pmc)) {
+					if (pmc->eventsel_hw &
+					    ARCH_PERFMON_EVENTSEL_ENABLE) {
+						pmc->eventsel_hw &= ~ARCH_PERFMON_EVENTSEL_ENABLE;
+						pmc->counter = 0;
+					}
+					return 0;
+				}
+				pmc->eventsel_hw = data;
+			} else if (data != pmc->eventsel) {
 				pmc->eventsel = data;
 				kvm_pmu_request_counter_reprogram(pmc);
 			}
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#83 lore raw

PATCH v2 33/54 KVM: x86/pmu: Allow writing to fixed counter selector if counter is exposed

In reply to Mingwei Zhang

Allow writing to fixed counter selector if counter is exposed. If this fixed counter is filtered out, this counter won't be enabled on HW.

Passthrough PMU implements the context switch at VM Enter/Exit boundary the guest value cannot be directly written to HW since the HW PMU is owned by the host. Introduce a new field fixed_ctr_ctrl_hw in kvm_pmu to cache the guest value. which will be assigne to HW at PMU context restore.

Since passthrough PMU intercept writes to fixed counter selector, there is no need to read the value at pmu context save, but still clear the fix counter ctrl MSR and counters when switching out to host PMU.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/vmx/pmu_intel.c    | 28 ++++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index b396000b9440..9857dda8b851 100644
@@ -546,6 +546,7 @@ struct kvm_pmu {
 	unsigned nr_arch_fixed_counters;
 	unsigned available_event_types;
 	u64 fixed_ctr_ctrl;
+	u64 fixed_ctr_ctrl_hw;
 	u64 fixed_ctr_ctrl_mask;
 	u64 global_ctrl;
 	u64 global_status;
arch/x86/kvm/vmx/pmu_intel.c +24 -4
index e706d107ff28..f0f99f5c21c5 100644
@@ -34,6 +34,25 @@
 
 #define MSR_PMC_FULL_WIDTH_BIT      (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)
 
+static void reprogram_fixed_counters_in_passthrough_pmu(struct kvm_pmu *pmu, u64 data)
+{
+	struct kvm_pmc *pmc;
+	u64 new_data = 0;
+	int i;
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = get_fixed_pmc(pmu, MSR_CORE_PERF_FIXED_CTR0 + i);
+		if (check_pmu_event_filter(pmc)) {
+			pmc->current_config = fixed_ctrl_field(data, i);
+			new_data |= (pmc->current_config << (i * 4));
+		} else {
+			pmc->counter = 0;
+		}
+	}
+	pmu->fixed_ctr_ctrl_hw = new_data;
+	pmu->fixed_ctr_ctrl = data;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	struct kvm_pmc *pmc;
@@ -351,7 +370,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (data & pmu->fixed_ctr_ctrl_mask)
 			return 1;
 
-		if (pmu->fixed_ctr_ctrl != data)
+		if (is_passthrough_pmu_enabled(vcpu))
+			reprogram_fixed_counters_in_passthrough_pmu(pmu, data);
+		else if (pmu->fixed_ctr_ctrl != data)
 			reprogram_fixed_counters(pmu, data);
 		break;
 	case MSR_IA32_PEBS_ENABLE:
@@ -820,13 +841,12 @@ static void intel_save_guest_pmu_context(struct kvm_vcpu *vcpu)
 	if (pmu->global_status)
 		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status);
 
-	rdmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
 	/*
 	 * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and
 	 * also avoid these guest fixed counters get accidentially enabled
 	 * during host running when host enable global ctrl.
 	 */
-	if (pmu->fixed_ctr_ctrl)
+	if (pmu->fixed_ctr_ctrl_hw)
 		wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, 0);
 }
 
@@ -844,7 +864,7 @@ static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
 	if (pmu->global_status & toggle)
 		wrmsrl(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
 
-	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
 }
 
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#84 lore raw

PATCH v2 34/54 KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

In PMU passthrough mode, use global_ctrl field in struct kvm_pmu as the cached value. This is convenient for KVM to set and get the value from the host side. In addition, load and save the value across VM enter/exit boundary in the following way:

- At VM exit, if processor supports GUEST_VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, read guest IA32_PERF_GLOBAL_CTRL GUEST_IA32_PERF_GLOBAL_CTRL VMCS field, else read it from VM-exit MSR-stroe array in VMCS. The value is then assigned to global_ctrl.

- At VM Entry, if processor supports GUEST_VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, read guest IA32_PERF_GLOBAL_CTRL from GUEST_IA32_PERF_GLOBAL_CTRL VMCS field, else read it from VM-entry MSR-load array in VMCS. The value is then assigned to global ctrl.

Implement the above logic into two helper functions and invoke them around VM Enter/exit boundary.

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/vmx/vmx.c          | 49 ++++++++++++++++++++++++++++++++-
 2 files changed, 50 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 9857dda8b851..54a56eda77f4 100644
@@ -598,6 +598,8 @@ struct kvm_pmu {
 	u8 event_count;
 
 	bool passthrough;
+	int global_ctrl_slot_in_autoload;
+	int global_ctrl_slot_in_autostore;
 };
 
 struct kvm_pmu_ops;
arch/x86/kvm/vmx/vmx.c +48 -1

91 lines not on this page · raw message

Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#85 lore raw

PATCH v2 35/54 KVM: x86/pmu: Exclude existing vLBR logic from the passthrough PMU

In reply to Mingwei Zhang

Excluding existing vLBR logic from the passthrough PMU because the it does not support LBR related MSRs. So to avoid any side effect, do not call vLBR related code in both vcpu_enter_guest() and pmi injection function.

---
 arch/x86/kvm/vmx/pmu_intel.c | 13 ++++++++-----
 arch/x86/kvm/vmx/vmx.c       |  2 +-
 2 files changed, 9 insertions(+), 6 deletions(-)
arch/x86/kvm/vmx/pmu_intel.c +8 -5
index f0f99f5c21c5..6db759147896 100644
@@ -660,13 +660,16 @@ static void intel_pmu_legacy_freezing_lbrs_on_pmi(struct kvm_vcpu *vcpu)
 
 static void intel_pmu_deliver_pmi(struct kvm_vcpu *vcpu)
 {
-	u8 version = vcpu_to_pmu(vcpu)->version;
+	u8 version;
 
-	if (!intel_pmu_lbr_is_enabled(vcpu))
-		return;
+	if (!is_passthrough_pmu_enabled(vcpu)) {
+		if (!intel_pmu_lbr_is_enabled(vcpu))
+			return;
 
-	if (version > 1 && version < 4)
-		intel_pmu_legacy_freezing_lbrs_on_pmi(vcpu);
+		version = vcpu_to_pmu(vcpu)->version;
+		if (version > 1 && version < 4)
+			intel_pmu_legacy_freezing_lbrs_on_pmi(vcpu);
+	}
 }
 
 static void vmx_update_intercept_for_lbr_msrs(struct kvm_vcpu *vcpu, bool set)
arch/x86/kvm/vmx/vmx.c +1 -1
index c86b768743a9..b6ed3ccdf1af 100644
@@ -7525,7 +7525,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu, bool force_immediate_exit)
 	pt_guest_enter(vmx);
 
 	atomic_switch_perf_msrs(vmx);
-	if (intel_pmu_lbr_is_enabled(vcpu))
+	if (!is_passthrough_pmu_enabled(&vmx->vcpu) && intel_pmu_lbr_is_enabled(vcpu))
 		vmx_passthrough_lbr_msrs(vcpu);
 
 	if (enable_preemption_timer)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#86 lore raw

PATCH v2 36/54 KVM: x86/pmu: Switch PMI handler at KVM context switch boundary

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Switch PMI handler at KVM context switch boundary because KVM uses a separate maskable interrupt vector other than the NMI handler for the host PMU to process its own PMIs. So invoke the perf API that allows registration of the PMI handler.

---
 arch/x86/kvm/pmu.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/pmu.c +4 -0
index 2ad71020a2c0..a12012a00c11 100644
@@ -1097,6 +1097,8 @@ void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 		if (pmc->counter)
 			wrmsrl(pmc->msr_counter, 0);
 	}
+
+	x86_perf_guest_exit();
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
@@ -1107,6 +1109,8 @@ void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 
 	lockdep_assert_irqs_disabled();
 
+	x86_perf_guest_enter(kvm_lapic_get_reg(vcpu->arch.apic, APIC_LVTPC));
+
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
 
 	/*
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#87 lore raw

Re: [PATCH v2 36/54] KVM: x86/pmu: Switch PMI handler at KVM context switch boundary

Quoted 27 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Switch PMI handler at KVM context switch boundary because KVM uses a separate maskable interrupt vector other than the NMI handler for the host PMU to process its own PMIs. So invoke the perf API that allows registration of the PMI handler.

---
 arch/x86/kvm/pmu.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/pmu.c +2 -0
index 2ad71020a2c0..a12012a00c11 100644
@@ -1097,6 +1097,8 @@ void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 		if (pmc->counter)
 			wrmsrl(pmc->msr_counter, 0);
 	}
+
+	x86_perf_guest_exit();
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)

arch/x86/kvm/pmu.c void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu) · line 1113

Quoted 5 lines
arch/x86/kvm/pmu.c +2 -0
@@ -1107,6 +1109,8 @@ void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 
 	lockdep_assert_irqs_disabled();
 
+	x86_perf_guest_enter(kvm_lapic_get_reg(vcpu->arch.apic, APIC_LVTPC));
+

Reading the LVTPC for a vCPU that does not have a struct kvm_lapic allocated leads to a NULL pointer dereference. I noticed this while trying to run a minimalistic guest like https://github.com/dpw/kvm-hello-world

Does this require a kvm_lapic_enabled() or similar check?

Quoted 3 lines
arch/x86/kvm/pmu.c +0 -0
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
 
 	/*
#88 lore raw

RE: [PATCH v2 36/54] KVM: x86/pmu: Switch PMI handler at KVM context switch boundary

Quoted 41 lines · Mingwei Zhang

arch/x86/kvm/pmu.c void kvm_pmu_save_pmu_context(struct kvm_vcpu

Quoted 18 lines

From: Xiong Zhang <[email protected]>

Switch PMI handler at KVM context switch boundary because KVM uses a separate maskable interrupt vector other than the NMI handler for the host PMU to process its own PMIs. So invoke the perf API that allows registration of the PMI handler.

---
 arch/x86/kvm/pmu.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/pmu.c index renamed from arch/x86/kvm/pmu.c +0 -0

2ad71020a2c0..a12012a00c11 100644

arch/x86/kvm/pmu.c +0 -0
@@ -1097,6 +1097,8 @@ void kvm_pmu_save_pmu_context(struct kvm_vcpu

*vcpu)

Quoted 5 lines
arch/x86/kvm/pmu.c +2 -0
 		if (pmc->counter)
 			wrmsrl(pmc->msr_counter, 0);
 	}
+
+	x86_perf_guest_exit();

arch/x86/kvm/pmu.c void kvm_pmu_save_pmu_context(struct kvm_vcpu

Quoted 10 lines
arch/x86/kvm/pmu.c +4 -0
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu) @@ -1107,6
+1109,8 @@ void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 
 	lockdep_assert_irqs_disabled();
 
+	x86_perf_guest_enter(kvm_lapic_get_reg(vcpu->arch.apic,
+APIC_LVTPC));
+

Reading the LVTPC for a vCPU that does not have a struct kvm_lapic allocated leads to a NULL pointer dereference. I noticed this while trying to run a minimalistic guest like https://github.com/dpw/kvm-hello-world

Quoted 2 lines

Does this require a kvm_lapic_enabled() or similar check?

Intel processor has lapci_in_kernel() checking in "[RFC PATCH v3 16/54] KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs".
+ pmu->passthrough = vcpu->kvm->arch.enable_passthrough_pmu &&
+ lapic_in_kernel(vcpu);

AMD processor seems need this checking also. we could move this checking into common place.

Thanks

Quoted 3 lines
Quoted 3 lines
arch/x86/kvm/pmu.c +0 -0
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
 
 	/*
#89 lore raw

Re: [PATCH v2 36/54] KVM: x86/pmu: Switch PMI handler at KVM context switch boundary

Quoted 48 lines · Xiong Y
Quoted 41 lines · Mingwei Zhang

arch/x86/kvm/pmu.c void kvm_pmu_save_pmu_context(struct kvm_vcpu

Quoted 18 lines

From: Xiong Zhang <[email protected]>

Switch PMI handler at KVM context switch boundary because KVM uses a separate maskable interrupt vector other than the NMI handler for the host PMU to process its own PMIs. So invoke the perf API that allows registration of the PMI handler.

---
 arch/x86/kvm/pmu.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kvm/pmu.c index renamed from arch/x86/kvm/pmu.c +0 -0

2ad71020a2c0..a12012a00c11 100644

arch/x86/kvm/pmu.c +0 -0
@@ -1097,6 +1097,8 @@ void kvm_pmu_save_pmu_context(struct kvm_vcpu

*vcpu)

Quoted 5 lines
arch/x86/kvm/pmu.c +2 -0
 		if (pmc->counter)
 			wrmsrl(pmc->msr_counter, 0);
 	}
+
+	x86_perf_guest_exit();

arch/x86/kvm/pmu.c void kvm_pmu_save_pmu_context(struct kvm_vcpu

Quoted 10 lines
arch/x86/kvm/pmu.c +4 -0
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu) @@ -1107,6
+1109,8 @@ void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 
 	lockdep_assert_irqs_disabled();
 
+	x86_perf_guest_enter(kvm_lapic_get_reg(vcpu->arch.apic,
+APIC_LVTPC));
+

Reading the LVTPC for a vCPU that does not have a struct kvm_lapic allocated leads to a NULL pointer dereference. I noticed this while trying to run a minimalistic guest like https://github.com/dpw/kvm-hello-world

Quoted 2 lines

Does this require a kvm_lapic_enabled() or similar check?

Intel processor has lapci_in_kernel() checking in "[RFC PATCH v3 16/54] KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs".

(quoted diff) +2 -0
+	pmu->passthrough = vcpu->kvm->arch.enable_passthrough_pmu &&
+			   lapic_in_kernel(vcpu);
Quoted 2 lines

AMD processor seems need this checking also. we could move this checking into common place.

Thanks. Adding that fixes the issue.

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
arch/x86/kvm/pmu.c +0 -0
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
 
 	/*
#90 lore raw

PATCH v2 37/54 KVM: x86/pmu: Grab x86 core PMU for passthrough PMU VM

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

When passthrough PMU is enabled by kvm and perf, KVM call perf_get_mediated_pmu() to exclusive own x86 core PMU at VM creation, KVM call perf_put_mediated_pmu() to return x86 core PMU to host perf at VM destroy.

When perf_get_mediated_pmu() fail, the host has system wide perf events without exclude_guest = 1 which must be disabled to enable VM with passthrough PMU.

Once VM with passthrough PMU starts, perf will refuse to create system wide perf event without exclude_guest = 1 until the vm is closed.

---
 arch/x86/kvm/x86.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +14 -1
index db395c00955f..3152587eca5b 100644
@@ -6674,8 +6674,11 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
 			/* Disable passthrough PMU if enable_pmu is false. */
-			if (!kvm->arch.enable_pmu)
+			if (!kvm->arch.enable_pmu) {
+				if (kvm->arch.enable_passthrough_pmu)
+					perf_put_mediated_pmu();
 				kvm->arch.enable_passthrough_pmu = false;
+			}
 			r = 0;
 		}
 		mutex_unlock(&kvm->lock);
@@ -12578,6 +12581,14 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.guest_can_read_msr_platform_info = true;
 	kvm->arch.enable_pmu = enable_pmu;
 	kvm->arch.enable_passthrough_pmu = enable_passthrough_pmu;
+	if (kvm->arch.enable_passthrough_pmu) {
+		ret = perf_get_mediated_pmu();
+		if (ret < 0) {
+			kvm_err("failed to enable mediated passthrough pmu, please disable system wide perf events\n");
+			goto out_uninit_mmu;
+		}
+	}
+
 
 #if IS_ENABLED(CONFIG_HYPERV)
 	spin_lock_init(&kvm->arch.hv_root_tdp_lock);
@@ -12726,6 +12737,8 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 		__x86_set_memory_region(kvm, TSS_PRIVATE_MEMSLOT, 0, 0);
 		mutex_unlock(&kvm->slots_lock);
 	}
+	if (kvm->arch.enable_passthrough_pmu)
+		perf_put_mediated_pmu();
 	kvm_unload_vcpu_mmus(kvm);
 	static_call_cond(kvm_x86_vm_destroy)(kvm);
 	kvm_free_msr_filter(srcu_dereference_check(kvm->arch.msr_filter, &kvm->srcu, 1));
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#91 lore raw

PATCH v2 38/54 KVM: x86/pmu: Call perf_guest_enter() at PMU context switch

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

perf subsystem should stop and restart all the perf events at the host level when entering and leaving passthrough PMU respectively. So invoke the perf API at PMU context switch functions.

---
 arch/x86/events/core.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/events/core.c +4 -0
index f5a043410614..6fe467bca809 100644
@@ -705,6 +705,8 @@ void x86_perf_guest_enter(u32 guest_lvtpc)
 {
 	lockdep_assert_irqs_disabled();
 
+	perf_guest_enter();
+
 	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
 			       (guest_lvtpc & APIC_LVT_MASKED));
 }
@@ -715,6 +717,8 @@ void x86_perf_guest_exit(void)
 	lockdep_assert_irqs_disabled();
 
 	apic_write(APIC_LVTPC, APIC_DM_NMI);
+
+	perf_guest_exit();
 }
 EXPORT_SYMBOL_GPL(x86_perf_guest_exit);
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
maintainer
#92 lore raw

Re: [PATCH v2 38/54] KVM: x86/pmu: Call perf_guest_enter() at PMU context switch

Quoted 26 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

perf subsystem should stop and restart all the perf events at the host level when entering and leaving passthrough PMU respectively. So invoke the perf API at PMU context switch functions.

---
 arch/x86/events/core.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/events/core.c +2 -0
index f5a043410614..6fe467bca809 100644
@@ -705,6 +705,8 @@ void x86_perf_guest_enter(u32 guest_lvtpc)
 {
 	lockdep_assert_irqs_disabled();
 
+	perf_guest_enter();
+
 	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
 			       (guest_lvtpc & APIC_LVT_MASKED));
 }

arch/x86/events/core.c void x86_perf_guest_exit(void) · line 723

Quoted 7 lines
arch/x86/events/core.c +2 -0
@@ -715,6 +717,8 @@ void x86_perf_guest_exit(void)
 	lockdep_assert_irqs_disabled();
 
 	apic_write(APIC_LVTPC, APIC_DM_NMI);
+
+	perf_guest_exit();
 }
 EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

*sigh*.. why does this patch exist? Please merge with the one that introduces these functions.

This is making review really hard.

#93 lore raw

Re: [PATCH v2 38/54] KVM: x86/pmu: Call perf_guest_enter() at PMU context switch

Quoted 37 lines · Peter Zijlstra
Quoted 26 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

perf subsystem should stop and restart all the perf events at the host level when entering and leaving passthrough PMU respectively. So invoke the perf API at PMU context switch functions.

---
 arch/x86/events/core.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/events/core.c +2 -0
index f5a043410614..6fe467bca809 100644
@@ -705,6 +705,8 @@ void x86_perf_guest_enter(u32 guest_lvtpc)
 {
 	lockdep_assert_irqs_disabled();
 
+	perf_guest_enter();
+
 	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
 			       (guest_lvtpc & APIC_LVT_MASKED));
 }

arch/x86/events/core.c void x86_perf_guest_exit(void) · line 723

Quoted 7 lines
arch/x86/events/core.c +2 -0
@@ -715,6 +717,8 @@ void x86_perf_guest_exit(void)
 	lockdep_assert_irqs_disabled();
 
 	apic_write(APIC_LVTPC, APIC_DM_NMI);
+
+	perf_guest_exit();
 }
 EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

*sigh*.. why does this patch exist? Please merge with the one that introduces these functions.

Quoted 1 line

This is making review really hard.

Sure. we would adjust the patches sequence. Thanks.

#94 lore raw

Re: [PATCH v2 38/54] KVM: x86/pmu: Call perf_guest_enter() at PMU context switch

In reply to Peter Zijlstra
Quoted 38 lines · Peter Zijlstra
Quoted 26 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

perf subsystem should stop and restart all the perf events at the host level when entering and leaving passthrough PMU respectively. So invoke the perf API at PMU context switch functions.

---
 arch/x86/events/core.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/events/core.c +2 -0
index f5a043410614..6fe467bca809 100644
@@ -705,6 +705,8 @@ void x86_perf_guest_enter(u32 guest_lvtpc)
 {
 	lockdep_assert_irqs_disabled();
 
+	perf_guest_enter();
+
 	apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
 			       (guest_lvtpc & APIC_LVT_MASKED));
 }

arch/x86/events/core.c void x86_perf_guest_exit(void) · line 723

Quoted 7 lines
arch/x86/events/core.c +2 -0
@@ -715,6 +717,8 @@ void x86_perf_guest_exit(void)
 	lockdep_assert_irqs_disabled();
 
 	apic_write(APIC_LVTPC, APIC_DM_NMI);
+
+	perf_guest_exit();
 }
 EXPORT_SYMBOL_GPL(x86_perf_guest_exit);

*sigh*.. why does this patch exist? Please merge with the one that introduces these functions.

Quoted 1 line

This is making review really hard.

Ah, right. This function should be added immediately after commit "perf:
x86: Add x86 function to switch PMI handler". It was just mind set of
development: "how can we call perf_guest_{enter,exit}() if KVM has not implemented anything?" So we defer the invocation until this moment :)

Will fix that in next version.

Thanks.
-Mingwei

#95 lore raw

PATCH v2 39/54 KVM: x86/pmu: Add support for PMU context switch at VM-exit/enter

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add correct PMU context switch at VM_entry/exit boundary.

---
 arch/x86/kvm/x86.c | 6 ++++++
 1 file changed, 6 insertions(+)
arch/x86/kvm/x86.c +6 -0
index 3152587eca5b..733c1acdc8c1 100644
@@ -11009,6 +11009,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 		set_debugreg(0, 7);
 	}
 
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_restore_pmu_context(vcpu);
+
 	guest_timing_enter_irqoff();
 
 	for (;;) {
@@ -11037,6 +11040,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 		++vcpu->stat.exits;
 	}
 
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_save_pmu_context(vcpu);
+
 	/*
 	 * Do this here before restoring debug registers on the host.  And
 	 * since we do this before handling the vmexit, a DR access vmexit
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#96 lore raw

PATCH v2 40/54 KVM: x86/pmu: Introduce PMU operator to increment counter

In reply to Mingwei Zhang

Introduce PMU operator to increment counter because in passthrough PMU there is no common backend implementation like host perf API. Having a PMU operator for counter increment and overflow checking will help hiding architectural differences.

So Introduce the operator function to make it convenient for passthrough PMU to synthesize a PMI.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  1 +
 arch/x86/kvm/pmu.h                     |  1 +
 arch/x86/kvm/vmx/pmu_intel.c           | 12 ++++++++++++
 3 files changed, 14 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index 1a848ba6a7a7..72ca78df8d2b 100644
@@ -27,6 +27,7 @@ KVM_X86_PMU_OP_OPTIONAL(cleanup)
 KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 KVM_X86_PMU_OP_OPTIONAL(save_pmu_context)
 KVM_X86_PMU_OP_OPTIONAL(restore_pmu_context)
+KVM_X86_PMU_OP_OPTIONAL(incr_counter)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/pmu.h +1 -0
index 9cde62f3988e..325f17673a00 100644
@@ -44,6 +44,7 @@ struct kvm_pmu_ops {
 	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);
 	void (*save_pmu_context)(struct kvm_vcpu *vcpu);
 	void (*restore_pmu_context)(struct kvm_vcpu *vcpu);
+	bool (*incr_counter)(struct kvm_pmc *pmc);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
arch/x86/kvm/vmx/pmu_intel.c +12 -0
index 6db759147896..485bbccf503a 100644
@@ -74,6 +74,17 @@ static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 	}
 }
 
+static bool intel_incr_counter(struct kvm_pmc *pmc)
+{
+	pmc->counter += 1;
+	pmc->counter &= pmc_bitmask(pmc);
+
+	if (!pmc->counter)
+		return true;
+
+	return false;
+}
+
 static struct kvm_pmc *intel_rdpmc_ecx_to_pmc(struct kvm_vcpu *vcpu,
 					    unsigned int idx, u64 *mask)
 {
@@ -885,6 +896,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.passthrough_pmu_msrs = intel_passthrough_pmu_msrs,
 	.save_pmu_context = intel_save_guest_pmu_context,
 	.restore_pmu_context = intel_restore_guest_pmu_context,
+	.incr_counter = intel_incr_counter,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#97 lore raw

PATCH v2 41/54 KVM: x86/pmu: Introduce PMU operator for setting counter overflow

In reply to Mingwei Zhang

Introduce PMU operator for setting counter overflow. When emulating counter increment, multiple counters could overflow at the same time, i.e., during the execution of the same instruction. In passthrough PMU, having an PMU operator provides convenience to update the PMU global status in one shot with details hidden behind the vendor specific implementation.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/pmu.h                     | 1 +
 arch/x86/kvm/vmx/pmu_intel.c           | 5 +++++
 3 files changed, 7 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index 72ca78df8d2b..bd5b118a5ce5 100644
@@ -28,6 +28,7 @@ KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 KVM_X86_PMU_OP_OPTIONAL(save_pmu_context)
 KVM_X86_PMU_OP_OPTIONAL(restore_pmu_context)
 KVM_X86_PMU_OP_OPTIONAL(incr_counter)
+KVM_X86_PMU_OP_OPTIONAL(set_overflow)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/pmu.h +1 -0
index 325f17673a00..78a7f0c5f3ba 100644
@@ -45,6 +45,7 @@ struct kvm_pmu_ops {
 	void (*save_pmu_context)(struct kvm_vcpu *vcpu);
 	void (*restore_pmu_context)(struct kvm_vcpu *vcpu);
 	bool (*incr_counter)(struct kvm_pmc *pmc);
+	void (*set_overflow)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
arch/x86/kvm/vmx/pmu_intel.c +5 -0
index 485bbccf503a..aac09eb9af0e 100644
@@ -881,6 +881,10 @@ static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
 	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
 }
 
+static void intel_set_overflow(struct kvm_vcpu *vcpu)
+{
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -897,6 +901,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.save_pmu_context = intel_save_guest_pmu_context,
 	.restore_pmu_context = intel_restore_guest_pmu_context,
 	.incr_counter = intel_incr_counter,
+	.set_overflow = intel_set_overflow,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#98 lore raw

PATCH v2 42/54 KVM: x86/pmu: Implement emulated counter increment for passthrough PMU

In reply to Mingwei Zhang

Implement emulated counter increment for passthrough PMU under KVM_REQ_PMU. Defer the counter increment to KVM_REQ_PMU handler because counter increment requests come from kvm_pmu_trigger_event() which can be triggered within the KVM_RUN inner loop or outside of the inner loop. This means the counter increment could happen before or after PMU context switch.

So process counter increment in one place makes the implementation simple.

---
 arch/x86/kvm/pmu.c | 50 +++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 47 insertions(+), 3 deletions(-)
arch/x86/kvm/pmu.c +47 -3
index a12012a00c11..06e70f74559d 100644
@@ -510,6 +510,18 @@ static int reprogram_counter(struct kvm_pmc *pmc)
 				     eventsel & ARCH_PERFMON_EVENTSEL_INT);
 }
 
+static void kvm_pmu_handle_event_in_passthrough_pmu(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	static_call_cond(kvm_x86_pmu_set_overflow)(vcpu);
+
+	if (atomic64_read(&pmu->__reprogram_pmi)) {
+		kvm_make_request(KVM_REQ_PMI, vcpu);
+		atomic64_set(&pmu->__reprogram_pmi, 0ull);
+	}
+}
+
 void kvm_pmu_handle_event(struct kvm_vcpu *vcpu)
 {
 	DECLARE_BITMAP(bitmap, X86_PMC_IDX_MAX);
@@ -517,6 +529,9 @@ void kvm_pmu_handle_event(struct kvm_vcpu *vcpu)
 	struct kvm_pmc *pmc;
 	int bit;
 
+	if (is_passthrough_pmu_enabled(vcpu))
+		return kvm_pmu_handle_event_in_passthrough_pmu(vcpu);
+
 	bitmap_copy(bitmap, pmu->reprogram_pmi, X86_PMC_IDX_MAX);
 
 	/*
@@ -848,6 +863,17 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu)
 	kvm_pmu_reset(vcpu);
 }
 
+static void kvm_passthrough_pmu_incr_counter(struct kvm_vcpu *vcpu, struct kvm_pmc *pmc)
+{
+	if (static_call(kvm_x86_pmu_incr_counter)(pmc)) {
+		__set_bit(pmc->idx, (unsigned long *)&pmc_to_pmu(pmc)->global_status);
+		kvm_make_request(KVM_REQ_PMU, vcpu);
+
+		if (pmc->eventsel & ARCH_PERFMON_EVENTSEL_INT)
+			set_bit(pmc->idx, (unsigned long *)&pmc_to_pmu(pmc)->reprogram_pmi);
+	}
+}
+
 static void kvm_pmu_incr_counter(struct kvm_pmc *pmc)
 {
 	pmc->emulated_counter++;
@@ -880,11 +906,13 @@ static inline bool cpl_is_matched(struct kvm_pmc *pmc)
 	return (static_call(kvm_x86_get_cpl)(pmc->vcpu) == 0) ? select_os : select_user;
 }
 
-void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
+static void __kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel,
+				    bool is_passthrough)
 {
 	DECLARE_BITMAP(bitmap, X86_PMC_IDX_MAX);
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	struct kvm_pmc *pmc;
+	bool is_pmc_allowed;
 	int i;
 
 	BUILD_BUG_ON(sizeof(pmu->global_ctrl) * BITS_PER_BYTE != X86_PMC_IDX_MAX);
@@ -896,6 +924,12 @@ void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
 		return;
 
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		if (is_passthrough)
+			is_pmc_allowed = pmc_speculative_in_use(pmc) &&
+					 check_pmu_event_filter(pmc);
+		else
+			is_pmc_allowed = pmc_event_is_allowed(pmc);
+
 		/*
 		 * Ignore checks for edge detect (all events currently emulated
 		 * but KVM are always rising edges), pin control (unsupported
@@ -911,12 +945,22 @@ void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
 		 * could ignoring them, so do the simple thing for now.
 		 */
 		if (((pmc->eventsel ^ eventsel) & AMD64_RAW_EVENT_MASK_NB) ||
-		    !pmc_event_is_allowed(pmc) || !cpl_is_matched(pmc))
+		    !is_pmc_allowed || !cpl_is_matched(pmc))
 			continue;
 
-		kvm_pmu_incr_counter(pmc);
+		if (is_passthrough)
+			kvm_passthrough_pmu_incr_counter(vcpu, pmc);
+		else
+			kvm_pmu_incr_counter(pmc);
 	}
 }
+
+void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
+{
+	bool is_passthrough = is_passthrough_pmu_enabled(vcpu);
+
+	__kvm_pmu_trigger_event(vcpu, eventsel, is_passthrough);
+}
 EXPORT_SYMBOL_GPL(kvm_pmu_trigger_event);
 
 static bool is_masked_filter_valid(const struct kvm_x86_pmu_event_filter *filter)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#99 lore raw

Re: [PATCH v2 42/54] KVM: x86/pmu: Implement emulated counter increment for passthrough PMU

arch/x86/kvm/pmu.c void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel) · line 932

Quoted 10 lines · Mingwei Zhang
arch/x86/kvm/pmu.c +6 -0
@@ -896,6 +924,12 @@ void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
 		return;
 
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		if (is_passthrough)
+			is_pmc_allowed = pmc_speculative_in_use(pmc) &&
+					 check_pmu_event_filter(pmc);
+		else
+			is_pmc_allowed = pmc_event_is_allowed(pmc);
+

Why don't need to check pmc_is_globally_enabled() in PMU passthrough case? Sorry if I missed something.

#100 lore raw

Re: [PATCH v2 42/54] KVM: x86/pmu: Implement emulated counter increment for passthrough PMU

Quoted 5 lines · Zide
Quoted 3 lines · Mingwei Zhang
(quoted diff) +0 -0
@@ -896,6 +924,12 @@ void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
 		return;
 
Quoted 9 lines

arch/x86/kvm/pmu.c void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel) · line 932

Quoted 7 lines
arch/x86/kvm/pmu.c +6 -0
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		if (is_passthrough)
+			is_pmc_allowed = pmc_speculative_in_use(pmc) &&
+					 check_pmu_event_filter(pmc);
+		else
+			is_pmc_allowed = pmc_event_is_allowed(pmc);
+

Why don't need to check pmc_is_globally_enabled() in PMU passthrough case? Sorry if I missed something.

Not sure if it's because the historical reason. Since pmu->global_ctrl would be updated in each vm-exit right now, we may not need to skip pmc_is_globally_enabled() anymore. Need Mingwei to confirm.

#101 lore raw

Re: [PATCH v2 42/54] KVM: x86/pmu: Implement emulated counter increment for passthrough PMU

Quoted 17 lines · Dapeng
Quoted 5 lines · Zide
Quoted 3 lines · Mingwei Zhang
(quoted diff) +0 -0
@@ -896,6 +924,12 @@ void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel)
 		return;
 
Quoted 9 lines

arch/x86/kvm/pmu.c void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel) · line 932

Quoted 7 lines
arch/x86/kvm/pmu.c +6 -0
 	kvm_for_each_pmc(pmu, pmc, i, bitmap) {
+		if (is_passthrough)
+			is_pmc_allowed = pmc_speculative_in_use(pmc) &&
+					 check_pmu_event_filter(pmc);
+		else
+			is_pmc_allowed = pmc_event_is_allowed(pmc);
+

Why don't need to check pmc_is_globally_enabled() in PMU passthrough case? Sorry if I missed something.

Quoted 4 lines

Not sure if it's because the historical reason. Since pmu->global_ctrl would be updated in each vm-exit right now, we may not need to skip pmc_is_globally_enabled() anymore. Need Mingwei to confirm.

yeah, this is a historical reason and how it becomes a bug. I will fix that in next version.

#102 lore raw

PATCH v2 43/54 KVM: x86/pmu: Update pmc_{read,write}_counter() to disconnect perf API

In reply to Mingwei Zhang

Update pmc_{read,write}_counter() to disconnect perf API because passthrough PMU does not use host PMU on backend. Because of that pmc->counter contains directly the actual value of the guest VM when set by the host (VMM) side.

---
 arch/x86/kvm/pmu.c | 5 +++++
 arch/x86/kvm/pmu.h | 4 ++++
 2 files changed, 9 insertions(+)
arch/x86/kvm/pmu.c +5 -0
index 06e70f74559d..12330d3f92f4 100644
@@ -322,6 +322,11 @@ static void pmc_update_sample_period(struct kvm_pmc *pmc)
 
 void pmc_write_counter(struct kvm_pmc *pmc, u64 val)
 {
+	if (pmc_to_pmu(pmc)->passthrough) {
+		pmc->counter = val;
+		return;
+	}
+
 	/*
 	 * Drop any unconsumed accumulated counts, the WRMSR is a write, not a
 	 * read-modify-write.  Adjust the counter value so that its value is
arch/x86/kvm/pmu.h +4 -0
index 78a7f0c5f3ba..7e006cb61296 100644
@@ -116,6 +116,10 @@ static inline u64 pmc_read_counter(struct kvm_pmc *pmc)
 {
 	u64 counter, enabled, running;
 
+	counter = pmc->counter;
+	if (pmc_to_pmu(pmc)->passthrough)
+		return counter & pmc_bitmask(pmc);
+
 	counter = pmc->counter + pmc->emulated_counter;
 
 	if (pmc->perf_event && !pmc->is_paused)
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#103 lore raw

PATCH v2 44/54 KVM: x86/pmu: Disconnect counter reprogram logic from passthrough PMU

In reply to Mingwei Zhang

Disconnect counter reprogram logic because passthrough PMU never use host PMU nor does it use perf API to do anything. Instead, when passthrough PMU is enabled, touching anywhere around counter reprogram part should be an error.

---
 arch/x86/kvm/pmu.c | 5 +++++
 arch/x86/kvm/pmu.h | 8 ++++++++
 2 files changed, 13 insertions(+)
arch/x86/kvm/pmu.c +5 -0
index 12330d3f92f4..da8b27f2b71d 100644
@@ -478,6 +478,11 @@ static int reprogram_counter(struct kvm_pmc *pmc)
 	bool emulate_overflow;
 	u8 fixed_ctr_ctrl;
 
+	if (pmu->passthrough) {
+		pr_warn_once("Passthrough PMU never reprogram counter\n");
+		return 0;
+	}
+
 	emulate_overflow = pmc_pause_counter(pmc);
 
 	if (!pmc_event_is_allowed(pmc))
arch/x86/kvm/pmu.h +8 -0
index 7e006cb61296..10553bc1ae1d 100644
@@ -256,6 +256,10 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 
 static inline void kvm_pmu_request_counter_reprogram(struct kvm_pmc *pmc)
 {
+	/* Passthrough PMU never reprogram counters via KVM_REQ_PMU. */
+	if (pmc_to_pmu(pmc)->passthrough)
+		return;
+
 	set_bit(pmc->idx, pmc_to_pmu(pmc)->reprogram_pmi);
 	kvm_make_request(KVM_REQ_PMU, pmc->vcpu);
 }
@@ -264,6 +268,10 @@ static inline void reprogram_counters(struct kvm_pmu *pmu, u64 diff)
 {
 	int bit;
 
+	/* Passthrough PMU never reprogram counters via KVM_REQ_PMU. */
+	if (pmu->passthrough)
+		return;
+
 	if (!diff)
 		return;
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#104 lore raw

PATCH v2 45/54 KVM: nVMX: Add nested virtualization support for passthrough PMU

In reply to Mingwei Zhang

Add nested virtualization support for passthrough PMU by combining the MSR interception bitmaps of vmcs01 and vmcs12. Readers may argue even without this patch, nested virtualization works for passthrough PMU because L1 will see Perfmon v2 and will have to use legacy vPMU implementation if it is Linux. However, any assumption made on L1 may be invalid, e.g., L1 may not even be Linux.

If both L0 and L1 pass through PMU MSRs, the correct behavior is to allow MSR access from L2 directly touch HW MSRs, since both L0 and L1 passthrough the access.

However, in current implementation, if without adding anything for nested, KVM always set MSR interception bits in vmcs02. This leads to the fact that L0 will emulate all MSR read/writes for L2, leading to errors, since the current passthrough vPMU never implements set_msr() and get_msr() for any counter access except counter accesses from the VMM side.

So fix the issue by setting up the correct MSR interception for PMU MSRs.

---
 arch/x86/kvm/vmx/nested.c | 52 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
arch/x86/kvm/vmx/nested.c +52 -0
index d05ddf751491..558032663228 100644
@@ -590,6 +590,55 @@ static inline void nested_vmx_set_intercept_for_msr(struct vcpu_vmx *vmx,
 						   msr_bitmap_l0, msr);
 }
 
+/* Pass PMU MSRs to nested VM if L0 and L1 are set to passthrough. */
+static void nested_vmx_set_passthru_pmu_intercept_for_msr(struct kvm_vcpu *vcpu,
+							  unsigned long *msr_bitmap_l1,
+							  unsigned long *msr_bitmap_l0)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct vcpu_vmx *vmx = to_vmx(vcpu);
+	int i;
+
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+						 msr_bitmap_l0,
+						 MSR_ARCH_PERFMON_EVENTSEL0 + i,
+						 MSR_TYPE_RW);
+		nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+						 msr_bitmap_l0,
+						 MSR_IA32_PERFCTR0 + i,
+						 MSR_TYPE_RW);
+		nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+						 msr_bitmap_l0,
+						 MSR_IA32_PMC0 + i,
+						 MSR_TYPE_RW);
+	}
+
+	for (i = 0; i < vcpu_to_pmu(vcpu)->nr_arch_fixed_counters; i++) {
+		nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+						 msr_bitmap_l0,
+						 MSR_CORE_PERF_FIXED_CTR0 + i,
+						 MSR_TYPE_RW);
+	}
+	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+					 msr_bitmap_l0,
+					 MSR_CORE_PERF_FIXED_CTR_CTRL,
+					 MSR_TYPE_RW);
+
+	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+					 msr_bitmap_l0,
+					 MSR_CORE_PERF_GLOBAL_STATUS,
+					 MSR_TYPE_RW);
+	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+					 msr_bitmap_l0,
+					 MSR_CORE_PERF_GLOBAL_CTRL,
+					 MSR_TYPE_RW);
+	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1,
+					 msr_bitmap_l0,
+					 MSR_CORE_PERF_GLOBAL_OVF_CTRL,
+					 MSR_TYPE_RW);
+}
+
 /*
  * Merge L0's and L1's MSR bitmap, return false to indicate that
  * we do not use the hardware.
@@ -691,6 +740,9 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,
 	nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0,
 					 MSR_IA32_FLUSH_CMD, MSR_TYPE_W);
 
+	if (is_passthrough_pmu_enabled(vcpu))
+		nested_vmx_set_passthru_pmu_intercept_for_msr(vcpu, msr_bitmap_l1, msr_bitmap_l0);
+
 	kvm_vcpu_unmap(vcpu, &vmx->nested.msr_bitmap_map, false);
 
 	vmx->nested.force_msr_bitmap_recalc = false;
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#105 lore raw

PATCH v2 46/54 perf/x86/amd/core: Set passthrough capability for host

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Apply the PERF_PMU_CAP_PASSTHROUGH_VPMU flag for version 2 and later implementations of the core PMU. Aside from having Global Control and Status registers, virtualizing the PMU using the passthrough model requires an interface to set or clear the overflow bits in the Global Status MSRs while restoring or saving the PMU context of a vCPU.

PerfMonV2-capable hardware has additional MSRs for this purpose namely, PerfCntrGlobalStatusSet and PerfCntrGlobalStatusClr, thereby making it suitable for use with passthrough PMU.

---
 arch/x86/events/amd/core.c | 3 +++
 1 file changed, 3 insertions(+)
arch/x86/events/amd/core.c +3 -0
index 985ef3b47919..7a91c991779c 100644
@@ -1395,6 +1395,9 @@ static int __init amd_core_pmu_init(void)
 
 		amd_pmu_global_cntr_mask = (1ULL << x86_pmu.num_counters) - 1;
 
+		x86_pmu.flags |= PMU_FL_PASSTHROUGH;
+		x86_get_pmu(smp_processor_id())->capabilities |= PERF_PMU_CAP_PASSTHROUGH_VPMU;
+
 		/* Update PMC handling functions */
 		x86_pmu.enable_all = amd_pmu_v2_enable_all;
 		x86_pmu.disable_all = amd_pmu_v2_disable_all;
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#106 lore raw

PATCH v2 47/54 KVM: x86/pmu/svm: Set passthrough capability for vcpus

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Pass on the passthrough PMU setting from kvm->arch into kvm_pmu for each vcpu. As long as the host supports PerfMonV2, the guest PMU version does not matter.

---
 arch/x86/kvm/svm/pmu.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/svm/pmu.c +1 -0
index 447657513729..385478103f65 100644
@@ -211,6 +211,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
 	pmu->nr_arch_fixed_counters = 0;
 	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
+	pmu->passthrough = vcpu->kvm->arch.enable_passthrough_pmu;
 
 	if (guest_cpuid_has(vcpu, X86_FEATURE_PERFCTR_CORE)) {
 		for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#107 lore raw

PATCH v2 48/54 KVM: x86/pmu/svm: Set enable_passthrough_pmu module parameter

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Since passthrough PMU can be also used on some AMD platforms, set the "enable_passthrough_pmu" KVM kernel module parameter to true when the following conditions are met.
- parameter is set to true when module loaded
- enable_pmu is true
- is running on and AMD CPU
- CPU supports PerfMonV2
- host PMU supports passthrough mode

---
 arch/x86/kvm/pmu.h     | 20 ++++++++++++--------
 arch/x86/kvm/svm/svm.c |  2 ++
 2 files changed, 14 insertions(+), 8 deletions(-)
arch/x86/kvm/pmu.h +12 -8
index 10553bc1ae1d..02ea90431033 100644
@@ -196,6 +196,7 @@ extern struct kvm_pmu_emulated_event_selectors kvm_pmu_eventsel;
 static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 {
 	bool is_intel = boot_cpu_data.x86_vendor == X86_VENDOR_INTEL;
+	bool is_amd = boot_cpu_data.x86_vendor == X86_VENDOR_AMD;
 	int min_nr_gp_ctrs = pmu_ops->MIN_NR_GP_COUNTERS;
 
 	/*
@@ -223,18 +224,21 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
-	/* Pass-through vPMU is only supported in Intel CPUs. */
-	if (!is_intel)
+	/* Pass-through vPMU is only supported in Intel and AMD CPUs. */
+	if (!is_intel && !is_amd)
 		enable_passthrough_pmu = false;
 
 	/*
-	 * Pass-through vPMU requires at least PerfMon version 4 because the
-	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
-	 * for counter emulation as well as PMU context switch.  In addition, it
-	 * requires host PMU support on passthrough mode. Disable pass-through
-	 * vPMU if any condition fails.
+	 * On Intel platforms, pass-through vPMU requires at least PerfMon
+	 * version 4 because the implementation requires the usage of
+	 * MSR_CORE_PERF_GLOBAL_STATUS_SET for counter emulation as well as
+	 * PMU context switch.  In addition, it requires host PMU support on
+	 * passthrough mode. Disable pass-through vPMU if any condition fails.
+	 *
+	 * On AMD platforms, pass-through vPMU requires at least PerfMonV2
+	 * because MSR_PERF_CNTR_GLOBAL_STATUS_SET is required.
 	 */
-	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)
+	if (!enable_pmu || !kvm_pmu_cap.passthrough || (is_intel && kvm_pmu_cap.version < 4) || (is_amd && kvm_pmu_cap.version < 2))
 		enable_passthrough_pmu = false;
 
 	if (!enable_pmu) {
arch/x86/kvm/svm/svm.c +2 -0
index d1a9f9951635..88648b3a9cdd 100644
@@ -238,6 +238,8 @@ module_param(intercept_smi, bool, 0444);
 bool vnmi = true;
 module_param(vnmi, bool, 0444);
 
+module_param(enable_passthrough_pmu, bool, 0444);
+
 static bool svm_gp_erratum_intercept = true;
 
 static u8 rsm_ins_bytes[] = "\x0f\xaa";
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#108 lore raw

PATCH v2 49/54 KVM: x86/pmu/svm: Allow RDPMC pass through when all counters exposed to guest

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

If passthrough PMU is enabled and all counters exposed to guest, clear the RDPMC interception bit in the VMCB Control Area (byte offset 0xc bit 15) to let RDPMC instructions proceed without VM-Exits. This improves the guest PMU performance in passthrough mode. If either condition is not satisfied, then intercept RDPMC and prevent guest accessing unexposed counters.

Note that On AMD platforms, passing through RDPMC will only allow guests to read the general-purpose counters. Details about the RDPMC interception bit can be found in Appendix B the "Layout of VMCB" from the AMD64 Architecture Programmer's Manual Volume 2.

---
 arch/x86/kvm/svm/svm.c | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kvm/svm/svm.c +5 -0
index 88648b3a9cdd..84dd1f560d0a 100644
@@ -1223,6 +1223,11 @@ static inline void init_vmcb_after_set_cpuid(struct kvm_vcpu *vcpu)
 		/* No need to intercept these MSRs */
 		set_msr_interception(vcpu, svm->msrpm, MSR_IA32_SYSENTER_EIP, 1, 1);
 		set_msr_interception(vcpu, svm->msrpm, MSR_IA32_SYSENTER_ESP, 1, 1);
+
+		if (kvm_pmu_check_rdpmc_passthrough(vcpu))
+			svm_clr_intercept(svm, INTERCEPT_RDPMC);
+		else
+			svm_set_intercept(svm, INTERCEPT_RDPMC);
 	}
 }
 
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#109 lore raw

PATCH v2 50/54 KVM: x86/pmu/svm: Implement callback to disable MSR interception

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Implement the AMD-specific callback for passthrough PMU that disables interception of PMU-related MSRs if the guest PMU counters qualify the requirement of passthrough. The PMU registers include the following.
- PerfCntrGlobalStatus (MSR 0xc0000300)
- PerfCntrGlobalCtl (MSR 0xc0000301)
- PerfCntrGlobalStatusClr (MSR 0xc0000302)
- PerfCntrGlobalStatusSet (MSR 0xc0000303)
- PERF_CTLx and PERF_CTRx pairs (MSRs 0xc0010200..0xc001020b)

Note that the passthrough/interception is invoked after each CPUID set. Since CPUID set can be done multiple times, do the intercept/clear of the bitmap explicitly for each counters as well as global registers.

---
 arch/x86/kvm/svm/pmu.c | 44 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
arch/x86/kvm/svm/pmu.c +44 -0
index 385478103f65..2ad62b8ac2c2 100644
@@ -247,6 +247,49 @@ static bool amd_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
 	return true;
 }
 
+static void amd_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct vcpu_svm *svm = to_svm(vcpu);
+	int msr_clear = !!(is_passthrough_pmu_enabled(vcpu));
+	int i;
+
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+		/*
+		 * PERF_CTLx registers require interception in order to clear
+		 * HostOnly bit and set GuestOnly bit. This is to prevent the
+		 * PERF_CTRx registers from counting before VM entry and after
+		 * VM exit.
+		 */
+		set_msr_interception(vcpu, svm->msrpm, MSR_F15H_PERF_CTL + 2 * i, 0, 0);
+
+		/*
+		 * Pass through counters exposed to the guest and intercept
+		 * counters that are unexposed. Do this explicitly since this
+		 * function may be set multiple times before vcpu runs.
+		 */
+		if (i >= pmu->nr_arch_gp_counters)
+			msr_clear = 0;
+		set_msr_interception(vcpu, svm->msrpm, MSR_F15H_PERF_CTR + 2 * i, msr_clear, msr_clear);
+	}
+
+	/*
+	 * In mediated passthrough vPMU, intercept global PMU MSRs when guest
+	 * PMU only owns a subset of counters provided in HW or its version is
+	 * less than 2.
+	 */
+	if (is_passthrough_pmu_enabled(vcpu) && pmu->version > 1 &&
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp)
+		msr_clear = 1;
+	else
+		msr_clear = 0;
+
+	set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL, msr_clear, msr_clear);
+	set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, msr_clear, msr_clear);
+	set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, msr_clear, msr_clear);
+	set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, msr_clear, msr_clear);
+}
+
 struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = amd_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = amd_msr_idx_to_pmc,
@@ -257,6 +300,7 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.refresh = amd_pmu_refresh,
 	.init = amd_pmu_init,
 	.is_rdpmc_passthru_allowed = amd_is_rdpmc_passthru_allowed,
+	.passthrough_pmu_msrs = amd_passthrough_pmu_msrs,
 	.EVENTSEL_EVENT = AMD64_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_AMD_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = AMD64_NUM_COUNTERS,
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#110 lore raw

PATCH v2 51/54 KVM: x86/pmu/svm: Set GuestOnly bit and clear HostOnly bit when guest write to event selectors

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

On AMD platforms, there is no way to restore PerfCntrGlobalCtl at VM-Entry or clear it at VM-Exit. Since the register states will be restored before entering and saved after exiting guest context, the counters can keep ticking and even overflow leading to chaos while still in host context.

To avoid this, the PERF_CTLx MSRs (event selectors) are always intercepted. KVM will always set the GuestOnly bit and clear the HostOnly bit so that the counters run only in guest context even if their enable bits are set. Intercepting these MSRs is also necessary for guest event filtering.

---
 arch/x86/kvm/svm/pmu.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/pmu.c +6 -1
index 2ad62b8ac2c2..bed0acfaf34d 100644
@@ -165,7 +165,12 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		data &= ~pmu->reserved_bits;
 		if (data != pmc->eventsel) {
 			pmc->eventsel = data;
-			kvm_pmu_request_counter_reprogram(pmc);
+			if (is_passthrough_pmu_enabled(vcpu)) {
+				data &= ~AMD64_EVENTSEL_HOSTONLY;
+				pmc->eventsel_hw = data | AMD64_EVENTSEL_GUESTONLY;
+			} else {
+				kvm_pmu_request_counter_reprogram(pmc);
+			}
 		}
 		return 0;
 	}
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#111 lore raw

PATCH v2 52/54 KVM: x86/pmu/svm: Add registers to direct access list

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Add all PMU-related MSRs to the list of possible direct access MSRs. Most of them will not be intercepted when using passthrough PMU.

---
 arch/x86/kvm/svm/svm.c | 16 ++++++++++++++++
 arch/x86/kvm/svm/svm.h |  2 +-
 2 files changed, 17 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +16 -0
index 84dd1f560d0a..ccc08c43f7fb 100644
@@ -140,6 +140,22 @@ static const struct svm_direct_access_msrs {
 	{ .index = X2APIC_MSR(APIC_TMICT),		.always = false },
 	{ .index = X2APIC_MSR(APIC_TMCCT),		.always = false },
 	{ .index = X2APIC_MSR(APIC_TDCR),		.always = false },
+	{ .index = MSR_F15H_PERF_CTL0,			.always = false },
+	{ .index = MSR_F15H_PERF_CTR0,			.always = false },
+	{ .index = MSR_F15H_PERF_CTL1,			.always = false },
+	{ .index = MSR_F15H_PERF_CTR1,			.always = false },
+	{ .index = MSR_F15H_PERF_CTL2,			.always = false },
+	{ .index = MSR_F15H_PERF_CTR2,			.always = false },
+	{ .index = MSR_F15H_PERF_CTL3,			.always = false },
+	{ .index = MSR_F15H_PERF_CTR3,			.always = false },
+	{ .index = MSR_F15H_PERF_CTL4,			.always = false },
+	{ .index = MSR_F15H_PERF_CTR4,			.always = false },
+	{ .index = MSR_F15H_PERF_CTL5,			.always = false },
+	{ .index = MSR_F15H_PERF_CTR5,			.always = false },
+	{ .index = MSR_AMD64_PERF_CNTR_GLOBAL_CTL,	.always = false },
+	{ .index = MSR_AMD64_PERF_CNTR_GLOBAL_STATUS,	.always = false },
+	{ .index = MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR,	.always = false },
+	{ .index = MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET,	.always = false },
 	{ .index = MSR_INVALID,				.always = false },
 };
 
arch/x86/kvm/svm/svm.h +1 -1
index 7f1fbd874c45..beb552a9ab05 100644
@@ -30,7 +30,7 @@
 #define	IOPM_SIZE PAGE_SIZE * 3
 #define	MSRPM_SIZE PAGE_SIZE * 2
 
-#define MAX_DIRECT_ACCESS_MSRS	47
+#define MAX_DIRECT_ACCESS_MSRS	63
 #define MSRPM_OFFSETS	32
 extern u32 msrpm_offsets[MSRPM_OFFSETS] __read_mostly;
 extern bool npt_enabled;
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#112 lore raw

PATCH v2 53/54 KVM: x86/pmu/svm: Implement handlers to save and restore context

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Implement the AMD-specific handlers to save and restore the state of PMU-related MSRs when using passthrough PMU.

---
 arch/x86/kvm/svm/pmu.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
arch/x86/kvm/svm/pmu.c +32 -0
index bed0acfaf34d..9629a172aa1b 100644
@@ -295,6 +295,36 @@ static void amd_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 	set_msr_interception(vcpu, svm->msrpm, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, msr_clear, msr_clear);
 }
 
+static void amd_save_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	rdmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, pmu->global_ctrl);
+	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
+	rdmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, pmu->global_status);
+
+	/* Clear global status bits if non-zero */
+	if (pmu->global_status)
+		wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, pmu->global_status);
+}
+
+static void amd_restore_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	u64 global_status;
+
+	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
+	rdmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, global_status);
+
+	/* Clear host global_status MSR if non-zero. */
+	if (global_status)
+		wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, global_status);
+
+	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, pmu->global_status);
+
+	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, pmu->global_ctrl);
+}
+
 struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = amd_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = amd_msr_idx_to_pmc,
@@ -306,6 +336,8 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.init = amd_pmu_init,
 	.is_rdpmc_passthru_allowed = amd_is_rdpmc_passthru_allowed,
 	.passthrough_pmu_msrs = amd_passthrough_pmu_msrs,
+	.save_pmu_context = amd_save_pmu_context,
+	.restore_pmu_context = amd_restore_pmu_context,
 	.EVENTSEL_EVENT = AMD64_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_AMD_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = AMD64_NUM_COUNTERS,
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
#113 lore raw

PATCH v2 54/54 KVM: x86/pmu/svm: Wire up PMU filtering functionality for passthrough PMU

In reply to Mingwei Zhang

From: Manali Shukla <[email protected]>

With the Passthrough PMU enabled, the PERF_CTLx MSRs (event selectors) are always intercepted and the event filter checking can be directly done inside amd_pmu_set_msr().

Add a check to allow writing to event selector for GP counters if and only if the event is allowed in filter.

---
 arch/x86/kvm/svm/pmu.c | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/x86/kvm/svm/pmu.c +9 -0
index 9629a172aa1b..cb6d3bfdd588 100644
@@ -166,6 +166,15 @@ static int amd_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (data != pmc->eventsel) {
 			pmc->eventsel = data;
 			if (is_passthrough_pmu_enabled(vcpu)) {
+				if (!check_pmu_event_filter(pmc)) {
+					/*
+					 * When guest request an invalid event,
+					 * stop the counter by clearing the
+					 * event selector MSR.
+					 */
+					pmc->eventsel_hw = 0;
+					return 0;
+				}
 				data &= ~AMD64_EVENTSEL_HOSTONLY;
 				pmc->eventsel_hw = data | AMD64_EVENTSEL_GUESTONLY;
 			} else {
Signature · 1 line
2.45.0.rc1.225.g2a3ae87e7f-goog
Tested-by on the v2 series
#114 lore raw
In reply to Mingwei Zhang
Quoted 190 lines

In this version, we added the mediated passthrough vPMU support for AMD. This is 1st version that comes up with a full x86 support on the vPMU new design.

Major changes:
- AMD support integration. Supporting guest PerfMon v1 and v2.
- Ensure !exclude_guest events only exist prior to mediate passthrough vPMU loaded. [sean]
- Update PMU MSR interception according to exposed counters and pmu version. [mingwei reported pmu_counters_test fails]
- Enforce RDPMC interception unless all counters exposed to guest. This removes a hack in RFCv1 where we pass through RDPMC and zero unexposed counters. [jim/sean]
- Combine the PMU context switch for both AMD and Intel.
- Because of RDPMC interception, update PMU context switch code by removing the "zeroing out" logic when restoring the guest context. [jim/sean: intercept rdpmc]

Minor changes:
- Flip enable_passthrough_pmu to false and change to a vendor param.
- Remove "Intercept full-width GP counter MSRs by checking with perf capabilities".
- Remove the write to pmc patch.
- Move host_perf_cap as an independent variable, will update after
https://lore.kernel.org/all/20240423221521.2923759-1-
[email protected]/

TODOs:
- Simplify enabling code for mediated passthrough vPMU.
- Further optimization on PMU context switch.

On-going discussions:
- Final name of mediated passthrough vPMU.
- PMU context switch optimizations.

Testing:
- Testcases:
- selftest: pmu_counters_test
- selftest: pmu_event_filter_test
- kvm-unit-tests: pmu
- qemu based ubuntu 20.04 (guest kernel: 5.10 and 6.7.9)
- Platforms:
- genoa
- skylake
- icelake
- sapphirerapids
- emeraldrapids

Ongoing Issues:
- AMD platform [milan]:
- ./pmu_event_filter_test error:
- test_amd_deny_list: Branch instructions retired = 44 (expected 42)
- test_without_filter: Branch instructions retired = 44 (expected 42)
- test_member_allow_list: Branch instructions retired = 44 (expected 42)
- test_not_member_deny_list: Branch instructions retired = 44 (expected
42)
- Intel platform [skylake]:
- kvm-unit-tests/pmu fails with two errors:
- FAIL: Intel: TSX cycles: gp cntr-3 with a value of 0
- FAIL: Intel: full-width writes: TSX cycles: gp cntr-3 with a value of 0

Installation guidance:
- echo 0 > /proc/sys/kernel/nmi_watchdog
- modprobe kvm_{amd,intel} enable_passthrough_pmu=Y 2>/dev/null

v1: https://lore.kernel.org/all/20240126085444.324918-1-
[email protected]/

Dapeng Mi (3):
x86/msr: Introduce MSR_CORE_PERF_GLOBAL_STATUS_SET
KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS
KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU
MSRs

Kan Liang (3):
perf: Support get/put passthrough PMU interfaces
perf: Add generic exclude_guest support perf/x86/intel: Support PERF_PMU_CAP_PASSTHROUGH_VPMU

Manali Shukla (1):
KVM: x86/pmu/svm: Wire up PMU filtering functionality for passthrough
PMU

Mingwei Zhang (24):
perf: core/x86: Forbid PMI handler when guest own PMU
perf: core/x86: Plumb passthrough PMU capability from x86_pmu to

    x86_pmu_cap
  KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter
  KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs
  KVM: x86/pmu: Add a helper to check if passthrough PMU is enabled
  KVM: x86/pmu: Add host_perf_cap and initialize it in
    kvm_x86_vendor_init()

KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to
guest
KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough
allowed
KVM: x86/pmu: Create a function prototype to disable MSR interception
KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in
passthrough vPMU
KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()
KVM: x86/pmu: Add counter MSR and selector MSR index into struct
kvm_pmc
KVM: x86/pmu: Introduce PMU operation prototypes for save/restore PMU
context
KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU
KVM: x86/pmu: Make check_pmu_event_filter() an exported function
KVM: x86/pmu: Allow writing to event selector for GP counters if event
is allowed
KVM: x86/pmu: Allow writing to fixed counter selector if counter is
exposed
KVM: x86/pmu: Exclude existing vLBR logic from the passthrough PMU
KVM: x86/pmu: Introduce PMU operator to increment counter
KVM: x86/pmu: Introduce PMU operator for setting counter overflow
KVM: x86/pmu: Implement emulated counter increment for passthrough
PMU
KVM: x86/pmu: Update pmc_{read,write}_counter() to disconnect perf API
KVM: x86/pmu: Disconnect counter reprogram logic from passthrough PMU
KVM: nVMX: Add nested virtualization support for passthrough PMU

Sandipan Das (11):
KVM: x86/pmu: Do not mask LVTPC when handling a PMI on AMD platforms x86/msr: Define PerfCntrGlobalStatusSet register
KVM: x86/pmu: Always set global enable bits in passthrough mode perf/x86/amd/core: Set passthrough capability for host
KVM: x86/pmu/svm: Set passthrough capability for vcpus
KVM: x86/pmu/svm: Set enable_passthrough_pmu module parameter
KVM: x86/pmu/svm: Allow RDPMC pass through when all counters exposed
to guest
KVM: x86/pmu/svm: Implement callback to disable MSR interception
KVM: x86/pmu/svm: Set GuestOnly bit and clear HostOnly bit when guest
write to event selectors
KVM: x86/pmu/svm: Add registers to direct access list
KVM: x86/pmu/svm: Implement handlers to save and restore context

Sean Christopherson (2):
KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at

    "RESET"

KVM: x86: Snapshot if a vCPU's vendor model is AMD vs. Intel
compatible

Xiong Zhang (10):
perf: core/x86: Register a new vector for KVM GUEST PMI
KVM: x86: Extract x86_set_kvm_irq_handler() function
KVM: x86/pmu: Register guest pmi handler for emulated PMU
perf: x86: Add x86 function to switch PMI handler
KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL
KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary
KVM: x86/pmu: Switch PMI handler at KVM context switch boundary
KVM: x86/pmu: Grab x86 core PMU for passthrough PMU VM
KVM: x86/pmu: Call perf_guest_enter() at PMU context switch
KVM: x86/pmu: Add support for PMU context switch at VM-exit/enter

 arch/x86/events/amd/core.c               |   3 +
 arch/x86/events/core.c                   |  41 ++++-
 arch/x86/events/intel/core.c             |   6 +
 arch/x86/events/perf_event.h             |   1 +
 arch/x86/include/asm/hardirq.h           |   1 +
 arch/x86/include/asm/idtentry.h          |   1 +
 arch/x86/include/asm/irq.h               |   2 +-
 arch/x86/include/asm/irq_vectors.h       |   5 +-
 arch/x86/include/asm/kvm-x86-pmu-ops.h   |   6 +
 arch/x86/include/asm/kvm_host.h          |  10 ++
 arch/x86/include/asm/msr-index.h         |   2 +
 arch/x86/include/asm/perf_event.h        |   4 +
 arch/x86/include/asm/vmx.h               |   1 +
 arch/x86/kernel/idt.c                    |   1 +
 arch/x86/kernel/irq.c                    |  36 ++++-
 arch/x86/kvm/cpuid.c                     |   4 +
 arch/x86/kvm/cpuid.h                     |  10 ++
 arch/x86/kvm/lapic.c                     |   3 +-
 arch/x86/kvm/mmu/mmu.c                   |   2 +-
 arch/x86/kvm/pmu.c                       | 168 ++++++++++++++++++-
 arch/x86/kvm/pmu.h                       |  47 ++++++
 arch/x86/kvm/svm/pmu.c                   | 112 ++++++++++++-
 arch/x86/kvm/svm/svm.c                   |  23 +++
 arch/x86/kvm/svm/svm.h                   |   2 +-
 arch/x86/kvm/vmx/capabilities.h          |   1 +
 arch/x86/kvm/vmx/nested.c                |  52 ++++++
 arch/x86/kvm/vmx/pmu_intel.c             | 192 ++++++++++++++++++++--
 arch/x86/kvm/vmx/vmx.c                   | 197 +++++++++++++++++++----
 arch/x86/kvm/vmx/vmx.h                   |   3 +-
 arch/x86/kvm/x86.c                       |  47 +++++-
 arch/x86/kvm/x86.h                       |   1 +
 include/linux/perf_event.h               |  18 +++
 kernel/events/core.c                     | 176 ++++++++++++++++++++
 tools/arch/x86/include/asm/irq_vectors.h |   3 +-
 34 files changed, 1120 insertions(+), 61 deletions(-)
Quoted 4 lines

base-commit: fec50db7033ea478773b159e0e2efb135270e3b7 --
2.45.0.rc1.225.g2a3ae87e7f-goog

Hi Mingwei,
Regarding the ongoing issue you mentioned on Intel Skylake platform, I tried to reproduce it .However, these two cases could PASS on my Skylake machine. Could you double check it with the latest kvm-unit-tests or share me your SKL CPU model?

CPU model on my SKL :

	 'Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz'.
Passthrough PMU status:
	$cat /sys/module/kvm_intel/parameters/enable_passthrough_pmu
	$Y

Kvm-unit-tests:
https://gitlab.com/kvm-unit-tests/kvm-unit-tests.git
Result:
PASS: Intel: TSX cycles: gp cntr-3 with a value of 37
PASS: Intel: full-width writes: TSX cycles: gp cntr-3 with a value of 36

Thanks and Best Regards,
Yongwei Ma

#115 lore raw
Quoted 202 lines · Yongwei
Quoted 190 lines

In this version, we added the mediated passthrough vPMU support for AMD. This is 1st version that comes up with a full x86 support on the vPMU new design.

Major changes:
- AMD support integration. Supporting guest PerfMon v1 and v2.
- Ensure !exclude_guest events only exist prior to mediate passthrough vPMU loaded. [sean]
- Update PMU MSR interception according to exposed counters and pmu version. [mingwei reported pmu_counters_test fails]
- Enforce RDPMC interception unless all counters exposed to guest. This removes a hack in RFCv1 where we pass through RDPMC and zero unexposed counters. [jim/sean]
- Combine the PMU context switch for both AMD and Intel.
- Because of RDPMC interception, update PMU context switch code by removing the "zeroing out" logic when restoring the guest context. [jim/sean: intercept rdpmc]

Minor changes:
- Flip enable_passthrough_pmu to false and change to a vendor param.
- Remove "Intercept full-width GP counter MSRs by checking with perf capabilities".
- Remove the write to pmc patch.
- Move host_perf_cap as an independent variable, will update after
https://lore.kernel.org/all/20240423221521.2923759-1-
[email protected]/

TODOs:
- Simplify enabling code for mediated passthrough vPMU.
- Further optimization on PMU context switch.

On-going discussions:
- Final name of mediated passthrough vPMU.
- PMU context switch optimizations.

Testing:
- Testcases:
- selftest: pmu_counters_test
- selftest: pmu_event_filter_test
- kvm-unit-tests: pmu
- qemu based ubuntu 20.04 (guest kernel: 5.10 and 6.7.9)
- Platforms:
- genoa
- skylake
- icelake
- sapphirerapids
- emeraldrapids

Ongoing Issues:
- AMD platform [milan]:
- ./pmu_event_filter_test error:
- test_amd_deny_list: Branch instructions retired = 44 (expected 42)
- test_without_filter: Branch instructions retired = 44 (expected 42)
- test_member_allow_list: Branch instructions retired = 44 (expected 42)
- test_not_member_deny_list: Branch instructions retired = 44 (expected
42)
- Intel platform [skylake]:
- kvm-unit-tests/pmu fails with two errors:
- FAIL: Intel: TSX cycles: gp cntr-3 with a value of 0
- FAIL: Intel: full-width writes: TSX cycles: gp cntr-3 with a value of 0

Installation guidance:
- echo 0 > /proc/sys/kernel/nmi_watchdog
- modprobe kvm_{amd,intel} enable_passthrough_pmu=Y 2>/dev/null

v1: https://lore.kernel.org/all/20240126085444.324918-1-
[email protected]/

Dapeng Mi (3):
x86/msr: Introduce MSR_CORE_PERF_GLOBAL_STATUS_SET
KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS
KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU
MSRs

Kan Liang (3):
perf: Support get/put passthrough PMU interfaces
perf: Add generic exclude_guest support perf/x86/intel: Support PERF_PMU_CAP_PASSTHROUGH_VPMU

Manali Shukla (1):
KVM: x86/pmu/svm: Wire up PMU filtering functionality for passthrough
PMU

Mingwei Zhang (24):
perf: core/x86: Forbid PMI handler when guest own PMU
perf: core/x86: Plumb passthrough PMU capability from x86_pmu to

    x86_pmu_cap
  KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter
  KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs
  KVM: x86/pmu: Add a helper to check if passthrough PMU is enabled
  KVM: x86/pmu: Add host_perf_cap and initialize it in
    kvm_x86_vendor_init()

KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to
guest
KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough
allowed
KVM: x86/pmu: Create a function prototype to disable MSR interception
KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in
passthrough vPMU
KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()
KVM: x86/pmu: Add counter MSR and selector MSR index into struct
kvm_pmc
KVM: x86/pmu: Introduce PMU operation prototypes for save/restore PMU
context
KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU
KVM: x86/pmu: Make check_pmu_event_filter() an exported function
KVM: x86/pmu: Allow writing to event selector for GP counters if event
is allowed
KVM: x86/pmu: Allow writing to fixed counter selector if counter is
exposed
KVM: x86/pmu: Exclude existing vLBR logic from the passthrough PMU
KVM: x86/pmu: Introduce PMU operator to increment counter
KVM: x86/pmu: Introduce PMU operator for setting counter overflow
KVM: x86/pmu: Implement emulated counter increment for passthrough
PMU
KVM: x86/pmu: Update pmc_{read,write}_counter() to disconnect perf API
KVM: x86/pmu: Disconnect counter reprogram logic from passthrough PMU
KVM: nVMX: Add nested virtualization support for passthrough PMU

Sandipan Das (11):
KVM: x86/pmu: Do not mask LVTPC when handling a PMI on AMD platforms x86/msr: Define PerfCntrGlobalStatusSet register
KVM: x86/pmu: Always set global enable bits in passthrough mode perf/x86/amd/core: Set passthrough capability for host
KVM: x86/pmu/svm: Set passthrough capability for vcpus
KVM: x86/pmu/svm: Set enable_passthrough_pmu module parameter
KVM: x86/pmu/svm: Allow RDPMC pass through when all counters exposed
to guest
KVM: x86/pmu/svm: Implement callback to disable MSR interception
KVM: x86/pmu/svm: Set GuestOnly bit and clear HostOnly bit when guest
write to event selectors
KVM: x86/pmu/svm: Add registers to direct access list
KVM: x86/pmu/svm: Implement handlers to save and restore context

Sean Christopherson (2):
KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at

    "RESET"

KVM: x86: Snapshot if a vCPU's vendor model is AMD vs. Intel
compatible

Xiong Zhang (10):
perf: core/x86: Register a new vector for KVM GUEST PMI
KVM: x86: Extract x86_set_kvm_irq_handler() function
KVM: x86/pmu: Register guest pmi handler for emulated PMU
perf: x86: Add x86 function to switch PMI handler
KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL
KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary
KVM: x86/pmu: Switch PMI handler at KVM context switch boundary
KVM: x86/pmu: Grab x86 core PMU for passthrough PMU VM
KVM: x86/pmu: Call perf_guest_enter() at PMU context switch
KVM: x86/pmu: Add support for PMU context switch at VM-exit/enter

 arch/x86/events/amd/core.c               |   3 +
 arch/x86/events/core.c                   |  41 ++++-
 arch/x86/events/intel/core.c             |   6 +
 arch/x86/events/perf_event.h             |   1 +
 arch/x86/include/asm/hardirq.h           |   1 +
 arch/x86/include/asm/idtentry.h          |   1 +
 arch/x86/include/asm/irq.h               |   2 +-
 arch/x86/include/asm/irq_vectors.h       |   5 +-
 arch/x86/include/asm/kvm-x86-pmu-ops.h   |   6 +
 arch/x86/include/asm/kvm_host.h          |  10 ++
 arch/x86/include/asm/msr-index.h         |   2 +
 arch/x86/include/asm/perf_event.h        |   4 +
 arch/x86/include/asm/vmx.h               |   1 +
 arch/x86/kernel/idt.c                    |   1 +
 arch/x86/kernel/irq.c                    |  36 ++++-
 arch/x86/kvm/cpuid.c                     |   4 +
 arch/x86/kvm/cpuid.h                     |  10 ++
 arch/x86/kvm/lapic.c                     |   3 +-
 arch/x86/kvm/mmu/mmu.c                   |   2 +-
 arch/x86/kvm/pmu.c                       | 168 ++++++++++++++++++-
 arch/x86/kvm/pmu.h                       |  47 ++++++
 arch/x86/kvm/svm/pmu.c                   | 112 ++++++++++++-
 arch/x86/kvm/svm/svm.c                   |  23 +++
 arch/x86/kvm/svm/svm.h                   |   2 +-
 arch/x86/kvm/vmx/capabilities.h          |   1 +
 arch/x86/kvm/vmx/nested.c                |  52 ++++++
 arch/x86/kvm/vmx/pmu_intel.c             | 192 ++++++++++++++++++++--
 arch/x86/kvm/vmx/vmx.c                   | 197 +++++++++++++++++++----
 arch/x86/kvm/vmx/vmx.h                   |   3 +-
 arch/x86/kvm/x86.c                       |  47 +++++-
 arch/x86/kvm/x86.h                       |   1 +
 include/linux/perf_event.h               |  18 +++
 kernel/events/core.c                     | 176 ++++++++++++++++++++
 tools/arch/x86/include/asm/irq_vectors.h |   3 +-
 34 files changed, 1120 insertions(+), 61 deletions(-)
Quoted 4 lines

base-commit: fec50db7033ea478773b159e0e2efb135270e3b7 --
2.45.0.rc1.225.g2a3ae87e7f-goog

Hi Mingwei,
Regarding the ongoing issue you mentioned on Intel Skylake platform, I tried to reproduce it .However, these two cases could PASS on my Skylake machine. Could you double check it with the latest kvm-unit-tests or share me your SKL CPU model?

CPU model on my SKL :

	 'Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz'.
Passthrough PMU status:
	$cat /sys/module/kvm_intel/parameters/enable_passthrough_pmu
	$Y
Quoted 6 lines

Kvm-unit-tests:
https://gitlab.com/kvm-unit-tests/kvm-unit-tests.git
Result:
PASS: Intel: TSX cycles: gp cntr-3 with a value of 37
PASS: Intel: full-width writes: TSX cycles: gp cntr-3 with a value of 36

That is good to see. We (me and my colleague) can reproduce the issue from my side. This might be related with the host setup instead of our code. Will figure it out.

Thanks.
-Mingwei

Quoted 4 lines

Thanks and Best Regards,
Yongwei Ma