Skip to contentSkip to conversation

Mediated Passthrough vPMU 3.0 for x86

RFC PATCH Superseded Mingwei Zhang 11 participants 183 messages Aug 1, 2024 – Jan 22, 2025 Page 1 of 2

Subsystems (7)
Documentation x86 KVM perf events +3 more subsystems

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Mediated Passthrough vPMU 3.0 for x86

This series contains perf interface improvements to address Peter's comments. In addition, fix several bugs for v2. This version is based on 6.10-rc4. The main changes are:

- Use atomics to replace refcounts to track the nr_mediated_pmu_vms.
- Use the generic ctx_sched_{in,out}() to switch PMU resources when a guest is entering and exiting.
- Add a new EVENT_GUEST flag to indicate the context switch case of entering and exiting a guest. Updates the generic ctx_sched_{in,out} to specifically handle this case, especially for time management.
- Switch PMI vector in perf_guest_{enter,exit}() as well. Add a new driver-specific interface to facilitate the switch.
- Remove the PMU_FL_PASSTHROUGH flag and uses the PASSTHROUGH pmu capability instead.
- Adjust commit sequence in PERF and KVM PMI interrupt functions.
- Use pmc_is_globally_enabled() check in emulated counter increment [1]
- Fix PMU context switch [2] by using rdpmc() instead of rdmsr().

AMD fixes:
- Add support for legacy PMU MSRs in MSR interception.
- Make MSR usage consistent if PerfMonV2 is available.
- Avoid enabling passthrough vPMU when local APIC is not in kernel.
- increment counters in emulation mode.

This series is organized in the following order:

Patches 1-3:
- Immediate bug fixes that can be applied to Linux tip.
- Note: will put immediate fixes ahead in the future. These patches might be duplicated with existing posts.
- Note: patches 1-2 are needed for AMD when host kernel enables preemption. Otherwise, guest will suffer from softlockup.

Patches 4-17:
- Perf side changes, infra changes in core pmu with API for KVM.

Patches 18-48:
- KVM mediated passthrough vPMU framework + Intel CPU implementation.

Patches 49-58:
- AMD CPU implementation for vPMU.

Reference (patches in v2):
[1] [PATCH v2 42/54] KVM: x86/pmu: Implement emulated counter increment for passthrough PMU
- https://lore.kernel.org/all/[email protected]/
[2] [PATCH v2 30/54] KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU
- https://lore.kernel.org/all/[email protected]/

V2: https://lore.kernel.org/all/[email protected]/

Dapeng Mi (3):
x86/msr: Introduce MSR_CORE_PERF_GLOBAL_STATUS_SET
KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS
KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU
MSRs

Kan Liang (8):
perf: Support get/put passthrough PMU interfaces
perf: Skip pmu_ctx based on event_type
perf: Clean up perf ctx time
perf: Add a EVENT_GUEST flag
perf: Add generic exclude_guest support
perf: Add switch_interrupt() interface perf/x86: Support switch_interrupt interface perf/x86/intel: Support PERF_PMU_CAP_PASSTHROUGH_VPMU

Manali Shukla (1):
KVM: x86/pmu/svm: Wire up PMU filtering functionality for passthrough
PMU

Mingwei Zhang (24):
perf/x86: Forbid PMI handler when guest own PMU
perf: core/x86: Plumb passthrough PMU capability from x86_pmu to

    x86_pmu_cap
  KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter
  KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs
  KVM: x86/pmu: Add a helper to check if passthrough PMU is enabled
  KVM: x86/pmu: Add host_perf_cap and initialize it in
    kvm_x86_vendor_init()

KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to
guest
KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough
allowed
KVM: x86/pmu: Create a function prototype to disable MSR interception
KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in
passthrough vPMU
KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()
KVM: x86/pmu: Add counter MSR and selector MSR index into struct
kvm_pmc
KVM: x86/pmu: Introduce PMU operation prototypes for save/restore PMU
context
KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU
KVM: x86/pmu: Make check_pmu_event_filter() an exported function
KVM: x86/pmu: Allow writing to event selector for GP counters if event
is allowed
KVM: x86/pmu: Allow writing to fixed counter selector if counter is
exposed
KVM: x86/pmu: Exclude existing vLBR logic from the passthrough PMU
KVM: x86/pmu: Introduce PMU operator to increment counter
KVM: x86/pmu: Introduce PMU operator for setting counter overflow
KVM: x86/pmu: Implement emulated counter increment for passthrough PMU
KVM: x86/pmu: Update pmc_{read,write}_counter() to disconnect perf API
KVM: x86/pmu: Disconnect counter reprogram logic from passthrough PMU
KVM: nVMX: Add nested virtualization support for passthrough PMU

Sandipan Das (12):
perf/x86: Do not set bit width for unavailable counters x86/msr: Define PerfCntrGlobalStatusSet register
KVM: x86/pmu: Always set global enable bits in passthrough mode
KVM: x86/pmu/svm: Set passthrough capability for vcpus
KVM: x86/pmu/svm: Set enable_passthrough_pmu module parameter
KVM: x86/pmu/svm: Allow RDPMC pass through when all counters exposed
to guest
KVM: x86/pmu/svm: Implement callback to disable MSR interception
KVM: x86/pmu/svm: Set GuestOnly bit and clear HostOnly bit when guest
write to event selectors
KVM: x86/pmu/svm: Add registers to direct access list
KVM: x86/pmu/svm: Implement handlers to save and restore context
KVM: x86/pmu/svm: Implement callback to increment counters perf/x86/amd: Support PERF_PMU_CAP_PASSTHROUGH_VPMU for AMD host

Sean Christopherson (2):
sched/core: Move preempt_model_*() helpers from sched.h to preempt.h sched/core: Drop spinlocks on contention iff kernel is preemptible

Xiong Zhang (8):
x86/irq: Factor out common code for installing kvm irq handler
perf: core/x86: Register a new vector for KVM GUEST PMI
KVM: x86/pmu: Register KVM_GUEST_PMI_VECTOR handler
KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL
KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary
KVM: x86/pmu: Notify perf core at KVM context switch boundary
KVM: x86/pmu: Grab x86 core PMU for passthrough PMU VM
KVM: x86/pmu: Add support for PMU context switch at VM-exit/enter

 .../admin-guide/kernel-parameters.txt         |   4 +-
 arch/x86/events/amd/core.c                    |   2 +
 arch/x86/events/core.c                        |  44 +-
 arch/x86/events/intel/core.c                  |   5 +
 arch/x86/include/asm/hardirq.h                |   1 +
 arch/x86/include/asm/idtentry.h               |   1 +
 arch/x86/include/asm/irq.h                    |   2 +-
 arch/x86/include/asm/irq_vectors.h            |   5 +-
 arch/x86/include/asm/kvm-x86-pmu-ops.h        |   6 +
 arch/x86/include/asm/kvm_host.h               |   9 +
 arch/x86/include/asm/msr-index.h              |   2 +
 arch/x86/include/asm/perf_event.h             |   1 +
 arch/x86/include/asm/vmx.h                    |   1 +
 arch/x86/kernel/idt.c                         |   1 +
 arch/x86/kernel/irq.c                         |  39 +-
 arch/x86/kvm/cpuid.c                          |   3 +
 arch/x86/kvm/pmu.c                            | 154 +++++-
 arch/x86/kvm/pmu.h                            |  49 ++
 arch/x86/kvm/svm/pmu.c                        | 136 +++++-
 arch/x86/kvm/svm/svm.c                        |  31 ++
 arch/x86/kvm/svm/svm.h                        |   2 +-
 arch/x86/kvm/vmx/capabilities.h               |   1 +
 arch/x86/kvm/vmx/nested.c                     |  52 +++
 arch/x86/kvm/vmx/pmu_intel.c                  | 192 +++++++-
 arch/x86/kvm/vmx/vmx.c                        | 197 ++++++--
 arch/x86/kvm/vmx/vmx.h                        |   3 +-
 arch/x86/kvm/x86.c                            |  45 ++
 arch/x86/kvm/x86.h                            |   1 +
 include/linux/perf_event.h                    |  38 +-
 include/linux/preempt.h                       |  41 ++
 include/linux/sched.h                         |  41 --
 include/linux/spinlock.h                      |  14 +-
 kernel/events/core.c                          | 441 ++++++++++++++----
 .../beauty/arch/x86/include/asm/irq_vectors.h |   5 +-
 34 files changed, 1373 insertions(+), 196 deletions(-)

base-commit: 6ba59ff4227927d3a8530fc2973b80e94b54d58f

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#1 lore raw

RFC PATCH v3 1/58 sched/core: Move preempt_model_*() helpers from sched.h to preempt.h

From: Sean Christopherson <[email protected]>

Move the declarations and inlined implementations of the preempt_model_*() helpers to preempt.h so that they can be referenced in spinlock.h without creating a potential circular dependency between spinlock.h and sched.h.

No functional change intended.

---
 include/linux/preempt.h | 41 +++++++++++++++++++++++++++++++++++++++++
 include/linux/sched.h   | 41 -----------------------------------------
 2 files changed, 41 insertions(+), 41 deletions(-)
include/linux/preempt.h +41 -0
index 7233e9cf1bab..ce76f1a45722 100644
@@ -481,4 +481,45 @@ DEFINE_LOCK_GUARD_0(preempt, preempt_disable(), preempt_enable())
 DEFINE_LOCK_GUARD_0(preempt_notrace, preempt_disable_notrace(), preempt_enable_notrace())
 DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+
+extern bool preempt_model_none(void);
+extern bool preempt_model_voluntary(void);
+extern bool preempt_model_full(void);
+
+#else
+
+static inline bool preempt_model_none(void)
+{
+	return IS_ENABLED(CONFIG_PREEMPT_NONE);
+}
+static inline bool preempt_model_voluntary(void)
+{
+	return IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY);
+}
+static inline bool preempt_model_full(void)
+{
+	return IS_ENABLED(CONFIG_PREEMPT);
+}
+
+#endif
+
+static inline bool preempt_model_rt(void)
+{
+	return IS_ENABLED(CONFIG_PREEMPT_RT);
+}
+
+/*
+ * Does the preemption model allow non-cooperative preemption?
+ *
+ * For !CONFIG_PREEMPT_DYNAMIC kernels this is an exact match with
+ * CONFIG_PREEMPTION; for CONFIG_PREEMPT_DYNAMIC this doesn't work as the
+ * kernel is *built* with CONFIG_PREEMPTION=y but may run with e.g. the
+ * PREEMPT_NONE model.
+ */
+static inline bool preempt_model_preemptible(void)
+{
+	return preempt_model_full() || preempt_model_rt();
+}
+
 #endif /* __LINUX_PREEMPT_H */
include/linux/sched.h +0 -41

47 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#2 lore raw

RFC PATCH v3 2/58 sched/core: Drop spinlocks on contention iff kernel is preemptible

In reply to Mingwei Zhang

From: Sean Christopherson <[email protected]>

Use preempt_model_preemptible() to detect a preemptible kernel when deciding whether or not to reschedule in order to drop a contended spinlock or rwlock. Because PREEMPT_DYNAMIC selects PREEMPTION, kernels built with PREEMPT_DYNAMIC=y will yield contended locks even if the live preemption model is "none" or "voluntary". In short, make kernels with dynamically selected models behave the same as kernels with statically selected models.

Somewhat counter-intuitively, NOT yielding a lock can provide better latency for the relevant tasks/processes. E.g. KVM x86's mmu_lock, a rwlock, is often contended between an invalidation event (takes mmu_lock for write) and a vCPU servicing a guest page fault (takes mmu_lock for read). For _some_ setups, letting the invalidation task complete even if there is mmu_lock contention provides lower latency for *all* tasks, i.e. the invalidation completes sooner *and* the vCPU services the guest page fault sooner.

But even KVM's mmu_lock behavior isn't uniform, e.g. the "best" behavior can vary depending on the host VMM, the guest workload, the number of vCPUs, the number of pCPUs in the host, why there is lock contention, etc.

In other words, simply deleting the CONFIG_PREEMPTION guard (or doing the opposite and removing contention yielding entirely) needs to come with a big pile of data proving that changing the status quo is a net positive.

Opportunistically document this side effect of preempt=full, as yielding contended spinlocks can have significant, user-visible impact.

Fixes c597bfddc9e9 ("sched: Provide Kconfig support for default dynamic preempt mode")
Cc Valentin Schneider <[email protected]>
Cc Peter Zijlstra (Intel) <[email protected]>
Cc Marco Elver <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc David Matlack <[email protected]>
Cc Friedrich Weber <[email protected]>
Cc Ankur Arora <[email protected]>
Cc Thomas Gleixner <[email protected]>
---
 Documentation/admin-guide/kernel-parameters.txt |  4 +++-
 include/linux/spinlock.h                        | 14 ++++++--------
 2 files changed, 9 insertions(+), 9 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +3 -1
index b600df82669d..ebb971a57d04 100644
@@ -4774,7 +4774,9 @@
 			none - Limited to cond_resched() calls
 			voluntary - Limited to cond_resched() and might_sleep() calls
 			full - Any section that isn't explicitly preempt disabled
-			       can be preempted anytime.
+			       can be preempted anytime.  Tasks will also yield
+			       contended spinlocks (if the critical section isn't
+			       explicitly preempt disabled beyond the lock itself).
 
 	print-fatal-signals=
 			[KNL] debug: print fatal signals
include/linux/spinlock.h +6 -8
index 3fcd20de6ca8..63dd8cf3c3c2 100644
@@ -462,11 +462,10 @@ static __always_inline int spin_is_contended(spinlock_t *lock)
  */
 static inline int spin_needbreak(spinlock_t *lock)
 {
-#ifdef CONFIG_PREEMPTION
+	if (!preempt_model_preemptible())
+		return 0;
+
 	return spin_is_contended(lock);
-#else
-	return 0;
-#endif
 }
 
 /*
@@ -479,11 +478,10 @@ static inline int spin_needbreak(spinlock_t *lock)
  */
 static inline int rwlock_needbreak(rwlock_t *lock)
 {
-#ifdef CONFIG_PREEMPTION
+	if (!preempt_model_preemptible())
+		return 0;
+
 	return rwlock_is_contended(lock);
-#else
-	return 0;
-#endif
 }
 
 /*
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#3 lore raw

RFC PATCH v3 3/58 perf/x86: Do not set bit width for unavailable counters

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Not all x86 processors have fixed counters. It may also be the case that a processor has only fixed counters and no general-purpose counters. Set the bit widths corresponding to each counter type only if such counters are available.

Fixes b3d9468a8bd2 ("perf, x86: Expose perf capability to other modules")
---
 arch/x86/events/core.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
arch/x86/events/core.c +7 -2
index 5b0dd07b1ef1..5bf78cd619bf 100644
@@ -2985,8 +2985,13 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	cap->version		= x86_pmu.version;
 	cap->num_counters_gp	= x86_pmu.num_counters;
 	cap->num_counters_fixed	= x86_pmu.num_counters_fixed;
-	cap->bit_width_gp	= x86_pmu.cntval_bits;
-	cap->bit_width_fixed	= x86_pmu.cntval_bits;
+
+	if (cap->num_counters_gp)
+		cap->bit_width_gp = x86_pmu.cntval_bits;
+
+	if (cap->num_counters_fixed)
+		cap->bit_width_fixed = x86_pmu.cntval_bits;
+
 	cap->events_mask	= (unsigned int)x86_pmu.events_maskl;
 	cap->events_mask_len	= x86_pmu.events_mask_len;
 	cap->pebs_ept		= x86_pmu.pebs_ept;
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#4 lore raw

RFC PATCH v3 4/58 x86/msr: Define PerfCntrGlobalStatusSet register

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Define PerfCntrGlobalStatusSet (MSR 0xc0000303) as it is required by passthrough PMU to set the overflow bits of PerfCntrGlobalStatus (MSR 0xc0000300).

When using passthrough PMU, it is necessary to restore the guest state of the overflow bits. Since PerfCntrGlobalStatus is read-only, this is done by writing to PerfCntrGlobalStatusSet instead.

The register is available on AMD processors where the PerfMonV2 feature bit of CPUID leaf 0x80000022 EAX is set.

---
 arch/x86/include/asm/msr-index.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/msr-index.h +1 -0
index e022e6eb766c..b9f8744b47e5 100644
@@ -681,6 +681,7 @@
 #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS	0xc0000300
 #define MSR_AMD64_PERF_CNTR_GLOBAL_CTL		0xc0000301
 #define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR	0xc0000302
+#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET	0xc0000303
 
 /* AMD Last Branch Record MSRs */
 #define MSR_AMD64_LBR_SELECT			0xc000010e
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#5 lore raw

RFC PATCH v3 5/58 x86/msr: Introduce MSR_CORE_PERF_GLOBAL_STATUS_SET

In reply to Mingwei Zhang

From: Dapeng Mi <[email protected]>

Add additional PMU MSRs MSR_CORE_PERF_GLOBAL_STATUS_SET to allow passthrough PMU operation on the read-only MSR IA32_PERF_GLOBAL_STATUS.

---
 arch/x86/include/asm/msr-index.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/include/asm/msr-index.h +1 -0
index b9f8744b47e5..1d7104713926 100644
@@ -1113,6 +1113,7 @@
 #define MSR_CORE_PERF_GLOBAL_STATUS	0x0000038e
 #define MSR_CORE_PERF_GLOBAL_CTRL	0x0000038f
 #define MSR_CORE_PERF_GLOBAL_OVF_CTRL	0x00000390
+#define MSR_CORE_PERF_GLOBAL_STATUS_SET 0x00000391
 
 #define MSR_PERF_METRICS		0x00000329
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#6 lore raw

RFC PATCH v3 6/58 perf: Support get/put passthrough PMU interfaces

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Currently, the guest and host share the PMU resources when a guest is running. KVM has to create an extra virtual event to simulate the guest's event, which brings several issues, e.g., high overhead, not accuracy and etc.

A new passthrough PMU method is proposed to address the issue. It requires that the PMU resources can be fully occupied by the guest while it's running. Two new interfaces are implemented to fulfill the requirement. The hypervisor should invoke the interface while creating a guest which wants the passthrough PMU capability.

The PMU resources should only be temporarily occupied as a whole when a guest is running. When the guest is out, the PMU resources are still shared among different users.

The exclude_guest event modifier is used to guarantee the exclusive occupation of the PMU resources. When creating a guest, the hypervisor should check whether there are !exclude_guest events in the system. If yes, the creation should fail. Because some PMU resources have been occupied by other users.
If no, the PMU resources can be safely accessed by the guest directly. Perf guarantees that no new !exclude_guest events are created when a guest is running.

Only the passthrough PMU is affected, but not for other PMU e.g., uncore and SW PMU. The behavior of those PMUs are not changed. The guest enter/exit interfaces should only impact the supported PMUs. Add a new PERF_PMU_CAP_PASSTHROUGH_VPMU flag to indicate the PMUs that support the feature.

Add nr_include_guest_events to track the !exclude_guest events of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU.

---
 include/linux/perf_event.h | 10 ++++++
 kernel/events/core.c       | 66 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
include/linux/perf_event.h +10 -0
index a5304ae8c654..45d1ea82aa21 100644
@@ -291,6 +291,7 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_NO_EXCLUDE			0x0040
 #define PERF_PMU_CAP_AUX_OUTPUT			0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE		0x0100
+#define PERF_PMU_CAP_PASSTHROUGH_VPMU		0x0200
 
 struct perf_output_handle;
 
@@ -1728,6 +1729,8 @@ extern void perf_event_task_tick(void);
 extern int perf_event_account_interrupt(struct perf_event *event);
 extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
+int perf_get_mediated_pmu(void);
+void perf_put_mediated_pmu(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1814,6 +1817,13 @@ static inline u64 perf_event_pause(struct perf_event *event, bool reset)
 {
 	return 0;
 }
+
+static inline int perf_get_mediated_pmu(void)
+{
+	return 0;
+}
+
+static inline void perf_put_mediated_pmu(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +66 -0

101 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#7 lore raw

Re: [RFC PATCH v3 06/58] perf: Support get/put passthrough PMU interfaces

Quoted 152 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Currently, the guest and host share the PMU resources when a guest is running. KVM has to create an extra virtual event to simulate the guest's event, which brings several issues, e.g., high overhead, not accuracy and etc.

A new passthrough PMU method is proposed to address the issue. It requires that the PMU resources can be fully occupied by the guest while it's running. Two new interfaces are implemented to fulfill the requirement. The hypervisor should invoke the interface while creating a guest which wants the passthrough PMU capability.

The PMU resources should only be temporarily occupied as a whole when a guest is running. When the guest is out, the PMU resources are still shared among different users.

The exclude_guest event modifier is used to guarantee the exclusive occupation of the PMU resources. When creating a guest, the hypervisor should check whether there are !exclude_guest events in the system. If yes, the creation should fail. Because some PMU resources have been occupied by other users.
If no, the PMU resources can be safely accessed by the guest directly. Perf guarantees that no new !exclude_guest events are created when a guest is running.

Only the passthrough PMU is affected, but not for other PMU e.g., uncore and SW PMU. The behavior of those PMUs are not changed. The guest enter/exit interfaces should only impact the supported PMUs. Add a new PERF_PMU_CAP_PASSTHROUGH_VPMU flag to indicate the PMUs that support the feature.

Add nr_include_guest_events to track the !exclude_guest events of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU.

---
 include/linux/perf_event.h | 10 ++++++
 kernel/events/core.c       | 66 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
include/linux/perf_event.h +10 -0
index a5304ae8c654..45d1ea82aa21 100644
@@ -291,6 +291,7 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_NO_EXCLUDE			0x0040
 #define PERF_PMU_CAP_AUX_OUTPUT			0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE		0x0100
+#define PERF_PMU_CAP_PASSTHROUGH_VPMU		0x0200
 
 struct perf_output_handle;
 
@@ -1728,6 +1729,8 @@ extern void perf_event_task_tick(void);
 extern int perf_event_account_interrupt(struct perf_event *event);
 extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
+int perf_get_mediated_pmu(void);
+void perf_put_mediated_pmu(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1814,6 +1817,13 @@ static inline u64 perf_event_pause(struct perf_event *event, bool reset)
 {
 	return 0;
 }
+
+static inline int perf_get_mediated_pmu(void)
+{
+	return 0;
+}
+
+static inline void perf_put_mediated_pmu(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +47 -0
index 8f908f077935..45868d276cde 100644
@@ -402,6 +402,20 @@ static atomic_t nr_bpf_events __read_mostly;
 static atomic_t nr_cgroup_events __read_mostly;
 static atomic_t nr_text_poke_events __read_mostly;
 static atomic_t nr_build_id_events __read_mostly;
+static atomic_t nr_include_guest_events __read_mostly;
+
+static atomic_t nr_mediated_pmu_vms;
+static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+
+/* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
+static inline bool is_include_guest_event(struct perf_event *event)
+{
+	if ((event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) &&
+	    !event->attr.exclude_guest)
+		return true;
+
+	return false;
+}
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
@@ -5212,6 +5226,9 @@ static void _free_event(struct perf_event *event)
 
 	unaccount_event(event);
 
+	if (is_include_guest_event(event))
+		atomic_dec(&nr_include_guest_events);
+
 	security_perf_event_free(event);
 
 	if (event->rb) {
@@ -5769,6 +5786,36 @@ u64 perf_event_pause(struct perf_event *event, bool reset)
 }
 EXPORT_SYMBOL_GPL(perf_event_pause);
 
+/*
+ * Currently invoked at VM creation to
+ * - Check whether there are existing !exclude_guest events of PMU with
+ *   PERF_PMU_CAP_PASSTHROUGH_VPMU
+ * - Set nr_mediated_pmu_vms to prevent !exclude_guest event creation on
+ *   PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU
+ *
+ * No impact for the PMU without PERF_PMU_CAP_PASSTHROUGH_VPMU. The perf
+ * still owns all the PMU resources.
+ */
+int perf_get_mediated_pmu(void)
+{
+	guard(mutex)(&perf_mediated_pmu_mutex);
+	if (atomic_inc_not_zero(&nr_mediated_pmu_vms))
+		return 0;
+
+	if (atomic_read(&nr_include_guest_events))
+		return -EBUSY;
+
+	atomic_inc(&nr_mediated_pmu_vms);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);
+
+void perf_put_mediated_pmu(void)
+{
+	atomic_dec(&nr_mediated_pmu_vms);
+}
+EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block

kernel/events/core.c static void account_event(struct perf_event *event) · line 11962

Quoted 9 lines
kernel/events/core.c +6 -0
@@ -11907,6 +11954,17 @@ static void account_event(struct perf_event *event)
 	account_pmu_sb_event(event);
 }
 
+static int perf_account_include_guest_event(void)
+{
+	guard(mutex)(&perf_mediated_pmu_mutex);
+
+	if (atomic_read(&nr_mediated_pmu_vms))
+		return -EACCES;

Kan, Namhyung posted a patchset
https://lore.kernel.org/all/[email protected]/
which would remove to set exclude_guest flag from perf tools by default. This may impact current mediated vPMU solution, but fortunately the patchset provides a fallback mechanism to add exclude_guest flag if kernel returns "EOPNOTSUPP".

So we'd better return "EOPNOTSUPP" instead of "EACCES" here. BTW, returning "EOPNOTSUPP" here looks more reasonable than "EACCES".

Quoted 28 lines
kernel/events/core.c +13 -0
+
+	atomic_inc(&nr_include_guest_events);
+	return 0;
+}
+
 /*
  * Allocate and initialize an event structure
  */
@@ -12114,11 +12172,19 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	if (err)
 		goto err_callchain_buffer;
 
+	if (is_include_guest_event(event)) {
+		err = perf_account_include_guest_event();
+		if (err)
+			goto err_security_alloc;
+	}
+
 	/* symmetric to unaccount_event() in _free_event() */
 	account_event(event);
 
 	return event;
 
+err_security_alloc:
+	security_perf_event_free(event);
 err_callchain_buffer:
 	if (!event->parent) {
 		if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)
#8 lore raw

Re: [RFC PATCH v3 06/58] perf: Support get/put passthrough PMU interfaces

Quoted 171 lines · Dapeng
Quoted 152 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Currently, the guest and host share the PMU resources when a guest is running. KVM has to create an extra virtual event to simulate the guest's event, which brings several issues, e.g., high overhead, not accuracy and etc.

A new passthrough PMU method is proposed to address the issue. It requires that the PMU resources can be fully occupied by the guest while it's running. Two new interfaces are implemented to fulfill the requirement. The hypervisor should invoke the interface while creating a guest which wants the passthrough PMU capability.

The PMU resources should only be temporarily occupied as a whole when a guest is running. When the guest is out, the PMU resources are still shared among different users.

The exclude_guest event modifier is used to guarantee the exclusive occupation of the PMU resources. When creating a guest, the hypervisor should check whether there are !exclude_guest events in the system. If yes, the creation should fail. Because some PMU resources have been occupied by other users.
If no, the PMU resources can be safely accessed by the guest directly. Perf guarantees that no new !exclude_guest events are created when a guest is running.

Only the passthrough PMU is affected, but not for other PMU e.g., uncore and SW PMU. The behavior of those PMUs are not changed. The guest enter/exit interfaces should only impact the supported PMUs. Add a new PERF_PMU_CAP_PASSTHROUGH_VPMU flag to indicate the PMUs that support the feature.

Add nr_include_guest_events to track the !exclude_guest events of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU.

---
 include/linux/perf_event.h | 10 ++++++
 kernel/events/core.c       | 66 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
include/linux/perf_event.h +10 -0
index a5304ae8c654..45d1ea82aa21 100644
@@ -291,6 +291,7 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_NO_EXCLUDE			0x0040
 #define PERF_PMU_CAP_AUX_OUTPUT			0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE		0x0100
+#define PERF_PMU_CAP_PASSTHROUGH_VPMU		0x0200
 
 struct perf_output_handle;
 
@@ -1728,6 +1729,8 @@ extern void perf_event_task_tick(void);
 extern int perf_event_account_interrupt(struct perf_event *event);
 extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
+int perf_get_mediated_pmu(void);
+void perf_put_mediated_pmu(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1814,6 +1817,13 @@ static inline u64 perf_event_pause(struct perf_event *event, bool reset)
 {
 	return 0;
 }
+
+static inline int perf_get_mediated_pmu(void)
+{
+	return 0;
+}
+
+static inline void perf_put_mediated_pmu(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +47 -0
index 8f908f077935..45868d276cde 100644
@@ -402,6 +402,20 @@ static atomic_t nr_bpf_events __read_mostly;
 static atomic_t nr_cgroup_events __read_mostly;
 static atomic_t nr_text_poke_events __read_mostly;
 static atomic_t nr_build_id_events __read_mostly;
+static atomic_t nr_include_guest_events __read_mostly;
+
+static atomic_t nr_mediated_pmu_vms;
+static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+
+/* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
+static inline bool is_include_guest_event(struct perf_event *event)
+{
+	if ((event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) &&
+	    !event->attr.exclude_guest)
+		return true;
+
+	return false;
+}
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
@@ -5212,6 +5226,9 @@ static void _free_event(struct perf_event *event)
 
 	unaccount_event(event);
 
+	if (is_include_guest_event(event))
+		atomic_dec(&nr_include_guest_events);
+
 	security_perf_event_free(event);
 
 	if (event->rb) {
@@ -5769,6 +5786,36 @@ u64 perf_event_pause(struct perf_event *event, bool reset)
 }
 EXPORT_SYMBOL_GPL(perf_event_pause);
 
+/*
+ * Currently invoked at VM creation to
+ * - Check whether there are existing !exclude_guest events of PMU with
+ *   PERF_PMU_CAP_PASSTHROUGH_VPMU
+ * - Set nr_mediated_pmu_vms to prevent !exclude_guest event creation on
+ *   PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU
+ *
+ * No impact for the PMU without PERF_PMU_CAP_PASSTHROUGH_VPMU. The perf
+ * still owns all the PMU resources.
+ */
+int perf_get_mediated_pmu(void)
+{
+	guard(mutex)(&perf_mediated_pmu_mutex);
+	if (atomic_inc_not_zero(&nr_mediated_pmu_vms))
+		return 0;
+
+	if (atomic_read(&nr_include_guest_events))
+		return -EBUSY;
+
+	atomic_inc(&nr_mediated_pmu_vms);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);
+
+void perf_put_mediated_pmu(void)
+{
+	atomic_dec(&nr_mediated_pmu_vms);
+}
+EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block

kernel/events/core.c static void account_event(struct perf_event *event) · line 11962

Quoted 9 lines
kernel/events/core.c +6 -0
@@ -11907,6 +11954,17 @@ static void account_event(struct perf_event *event)
 	account_pmu_sb_event(event);
 }
 
+static int perf_account_include_guest_event(void)
+{
+	guard(mutex)(&perf_mediated_pmu_mutex);
+
+	if (atomic_read(&nr_mediated_pmu_vms))
+		return -EACCES;

Kan, Namhyung posted a patchset
https://lore.kernel.org/all/[email protected]/
which would remove to set exclude_guest flag from perf tools by default. This may impact current mediated vPMU solution, but fortunately the patchset provides a fallback mechanism to add exclude_guest flag if kernel returns "EOPNOTSUPP".

Quoted 2 lines

So we'd better return "EOPNOTSUPP" instead of "EACCES" here. BTW, returning "EOPNOTSUPP" here looks more reasonable than "EACCES".

It seems the existing Apple M1 PMU has ready returned "EOPNOTSUPP" for the !exclude_guest. Yes, we should use the same error code.

Thanks,
Kan

Quoted 31 lines
Quoted 30 lines
kernel/events/core.c +13 -0
+
+	atomic_inc(&nr_include_guest_events);
+	return 0;
+}
+
 /*
  * Allocate and initialize an event structure
  */
@@ -12114,11 +12172,19 @@ perf_event_alloc(struct perf_event_attr *attr, int cpu,
 	if (err)
 		goto err_callchain_buffer;
 
+	if (is_include_guest_event(event)) {
+		err = perf_account_include_guest_event();
+		if (err)
+			goto err_security_alloc;
+	}
+
 	/* symmetric to unaccount_event() in _free_event() */
 	account_event(event);
 
 	return event;
 
+err_security_alloc:
+	security_perf_event_free(event);
 err_callchain_buffer:
 	if (!event->parent) {
 		if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)
maintainer
#9 lore raw

Re: [RFC PATCH v3 06/58] perf: Support get/put passthrough PMU interfaces

Hello,

Quoted 174 lines · Kan
Quoted 171 lines · Dapeng
Quoted 152 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Currently, the guest and host share the PMU resources when a guest is running. KVM has to create an extra virtual event to simulate the guest's event, which brings several issues, e.g., high overhead, not accuracy and etc.

A new passthrough PMU method is proposed to address the issue. It requires that the PMU resources can be fully occupied by the guest while it's running. Two new interfaces are implemented to fulfill the requirement. The hypervisor should invoke the interface while creating a guest which wants the passthrough PMU capability.

The PMU resources should only be temporarily occupied as a whole when a guest is running. When the guest is out, the PMU resources are still shared among different users.

The exclude_guest event modifier is used to guarantee the exclusive occupation of the PMU resources. When creating a guest, the hypervisor should check whether there are !exclude_guest events in the system. If yes, the creation should fail. Because some PMU resources have been occupied by other users.
If no, the PMU resources can be safely accessed by the guest directly. Perf guarantees that no new !exclude_guest events are created when a guest is running.

Only the passthrough PMU is affected, but not for other PMU e.g., uncore and SW PMU. The behavior of those PMUs are not changed. The guest enter/exit interfaces should only impact the supported PMUs. Add a new PERF_PMU_CAP_PASSTHROUGH_VPMU flag to indicate the PMUs that support the feature.

Add nr_include_guest_events to track the !exclude_guest events of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU.

---
 include/linux/perf_event.h | 10 ++++++
 kernel/events/core.c       | 66 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 76 insertions(+)
include/linux/perf_event.h +10 -0
index a5304ae8c654..45d1ea82aa21 100644
@@ -291,6 +291,7 @@ struct perf_event_pmu_context;
 #define PERF_PMU_CAP_NO_EXCLUDE			0x0040
 #define PERF_PMU_CAP_AUX_OUTPUT			0x0080
 #define PERF_PMU_CAP_EXTENDED_HW_TYPE		0x0100
+#define PERF_PMU_CAP_PASSTHROUGH_VPMU		0x0200
 
 struct perf_output_handle;
 
@@ -1728,6 +1729,8 @@ extern void perf_event_task_tick(void);
 extern int perf_event_account_interrupt(struct perf_event *event);
 extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
+int perf_get_mediated_pmu(void);
+void perf_put_mediated_pmu(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1814,6 +1817,13 @@ static inline u64 perf_event_pause(struct perf_event *event, bool reset)
 {
 	return 0;
 }
+
+static inline int perf_get_mediated_pmu(void)
+{
+	return 0;
+}
+
+static inline void perf_put_mediated_pmu(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +47 -0
index 8f908f077935..45868d276cde 100644
@@ -402,6 +402,20 @@ static atomic_t nr_bpf_events __read_mostly;
 static atomic_t nr_cgroup_events __read_mostly;
 static atomic_t nr_text_poke_events __read_mostly;
 static atomic_t nr_build_id_events __read_mostly;
+static atomic_t nr_include_guest_events __read_mostly;
+
+static atomic_t nr_mediated_pmu_vms;
+static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+
+/* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
+static inline bool is_include_guest_event(struct perf_event *event)
+{
+	if ((event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) &&
+	    !event->attr.exclude_guest)
+		return true;
+
+	return false;
+}
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
@@ -5212,6 +5226,9 @@ static void _free_event(struct perf_event *event)
 
 	unaccount_event(event);
 
+	if (is_include_guest_event(event))
+		atomic_dec(&nr_include_guest_events);
+
 	security_perf_event_free(event);
 
 	if (event->rb) {
@@ -5769,6 +5786,36 @@ u64 perf_event_pause(struct perf_event *event, bool reset)
 }
 EXPORT_SYMBOL_GPL(perf_event_pause);
 
+/*
+ * Currently invoked at VM creation to
+ * - Check whether there are existing !exclude_guest events of PMU with
+ *   PERF_PMU_CAP_PASSTHROUGH_VPMU
+ * - Set nr_mediated_pmu_vms to prevent !exclude_guest event creation on
+ *   PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU
+ *
+ * No impact for the PMU without PERF_PMU_CAP_PASSTHROUGH_VPMU. The perf
+ * still owns all the PMU resources.
+ */
+int perf_get_mediated_pmu(void)
+{
+	guard(mutex)(&perf_mediated_pmu_mutex);
+	if (atomic_inc_not_zero(&nr_mediated_pmu_vms))
+		return 0;
+
+	if (atomic_read(&nr_include_guest_events))
+		return -EBUSY;
+
+	atomic_inc(&nr_mediated_pmu_vms);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(perf_get_mediated_pmu);
+
+void perf_put_mediated_pmu(void)
+{
+	atomic_dec(&nr_mediated_pmu_vms);
+}
+EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
+
 /*
  * Holding the top-level event's child_mutex means that any
  * descendant process that has inherited this event will block

kernel/events/core.c static void account_event(struct perf_event *event) · line 11962

Quoted 9 lines
kernel/events/core.c +6 -0
@@ -11907,6 +11954,17 @@ static void account_event(struct perf_event *event)
 	account_pmu_sb_event(event);
 }
 
+static int perf_account_include_guest_event(void)
+{
+	guard(mutex)(&perf_mediated_pmu_mutex);
+
+	if (atomic_read(&nr_mediated_pmu_vms))
+		return -EACCES;

Kan, Namhyung posted a patchset
https://lore.kernel.org/all/[email protected]/
which would remove to set exclude_guest flag from perf tools by default. This may impact current mediated vPMU solution, but fortunately the patchset provides a fallback mechanism to add exclude_guest flag if kernel returns "EOPNOTSUPP".

Quoted 5 lines
Quoted 2 lines

So we'd better return "EOPNOTSUPP" instead of "EACCES" here. BTW, returning "EOPNOTSUPP" here looks more reasonable than "EACCES".

It seems the existing Apple M1 PMU has ready returned "EOPNOTSUPP" for the !exclude_guest. Yes, we should use the same error code.

Yep, it'd be much easier to handle if it returns the same error code.

Thanks,
Namhyung

#10 lore raw

RFC PATCH v3 7/58 perf: Skip pmu_ctx based on event_type

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

To optimize the cgroup context switch, the perf_event_pmu_context iteration skips the PMUs without cgroup events. A bool cgroup was introduced to indicate the case. It can work, but this way is hard to extend for other cases, e.g. skipping non-passthrough PMUs. It doesn't make sense to keep adding bool variables.

Pass the event_type instead of the specific bool variable. Check both the event_type and related pmu_ctx variables to decide whether skipping a PMU.

Event flags, e.g., EVENT_CGROUP, should be cleard in the ctx->is_active. Add EVENT_FLAGS to indicate such event flags.

No functional change.

---
 kernel/events/core.c | 70 +++++++++++++++++++++++---------------------
 1 file changed, 37 insertions(+), 33 deletions(-)
kernel/events/core.c +37 -33
index 45868d276cde..7cb51dbf897a 100644
@@ -376,6 +376,7 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
+	EVENT_FLAGS = EVENT_CGROUP,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -699,23 +700,32 @@ do {									\
 	___p;								\
 })
 
-static void perf_ctx_disable(struct perf_event_context *ctx, bool cgroup)
+static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
+			      enum event_type_t event_type)
+{
+	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
+		return true;
+
+	return false;
+}
+
+static void perf_ctx_disable(struct perf_event_context *ctx, enum event_type_t event_type)
 {
 	struct perf_event_pmu_context *pmu_ctx;
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
-		if (cgroup && !pmu_ctx->nr_cgroups)
+		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
 		perf_pmu_disable(pmu_ctx->pmu);
 	}
 }
 
-static void perf_ctx_enable(struct perf_event_context *ctx, bool cgroup)
+static void perf_ctx_enable(struct perf_event_context *ctx, enum event_type_t event_type)
 {
 	struct perf_event_pmu_context *pmu_ctx;
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
-		if (cgroup && !pmu_ctx->nr_cgroups)
+		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
 		perf_pmu_enable(pmu_ctx->pmu);
 	}
@@ -877,7 +887,7 @@ static void perf_cgroup_switch(struct task_struct *task)
 		return;
 
 	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
-	perf_ctx_disable(&cpuctx->ctx, true);
+	perf_ctx_disable(&cpuctx->ctx, EVENT_CGROUP);
 
 	ctx_sched_out(&cpuctx->ctx, EVENT_ALL|EVENT_CGROUP);
 	/*
@@ -893,7 +903,7 @@ static void perf_cgroup_switch(struct task_struct *task)
 	 */
 	ctx_sched_in(&cpuctx->ctx, EVENT_ALL|EVENT_CGROUP);
 
-	perf_ctx_enable(&cpuctx->ctx, true);
+	perf_ctx_enable(&cpuctx->ctx, EVENT_CGROUP);
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 }
 
@@ -2732,9 +2742,9 @@ static void ctx_resched(struct perf_cpu_context *cpuctx,
 
 	event_type &= EVENT_ALL;
 
-	perf_ctx_disable(&cpuctx->ctx, false);
+	perf_ctx_disable(&cpuctx->ctx, 0);
 	if (task_ctx) {
-		perf_ctx_disable(task_ctx, false);
+		perf_ctx_disable(task_ctx, 0);
 		task_ctx_sched_out(task_ctx, event_type);
 	}
 
@@ -2752,9 +2762,9 @@ static void ctx_resched(struct perf_cpu_context *cpuctx,
 
 	perf_event_sched_in(cpuctx, task_ctx);
 
-	perf_ctx_enable(&cpuctx->ctx, false);
+	perf_ctx_enable(&cpuctx->ctx, 0);
 	if (task_ctx)
-		perf_ctx_enable(task_ctx, false);
+		perf_ctx_enable(task_ctx, 0);
 }
 
 void perf_pmu_resched(struct pmu *pmu)
@@ -3299,9 +3309,6 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 	struct perf_event_pmu_context *pmu_ctx;
 	int is_active = ctx->is_active;
-	bool cgroup = event_type & EVENT_CGROUP;
-
-	event_type &= ~EVENT_CGROUP;
 
 	lockdep_assert_held(&ctx->lock);
 
@@ -3336,7 +3343,7 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 		barrier();
 	}
 
-	ctx->is_active &= ~event_type;
+	ctx->is_active &= ~(event_type & ~EVENT_FLAGS);
 	if (!(ctx->is_active & EVENT_ALL))
 		ctx->is_active = 0;
 
@@ -3349,7 +3356,7 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	is_active ^= ctx->is_active; /* changed bits */
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
-		if (cgroup && !pmu_ctx->nr_cgroups)
+		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
 		__pmu_ctx_sched_out(pmu_ctx, is_active);
 	}
@@ -3543,7 +3550,7 @@ perf_event_context_sched_out(struct task_struct *task, struct task_struct *next)
 		raw_spin_lock_nested(&next_ctx->lock, SINGLE_DEPTH_NESTING);
 		if (context_equiv(ctx, next_ctx)) {
 
-			perf_ctx_disable(ctx, false);
+			perf_ctx_disable(ctx, 0);
 
 			/* PMIs are disabled; ctx->nr_pending is stable. */
 			if (local_read(&ctx->nr_pending) ||
@@ -3563,7 +3570,7 @@ perf_event_context_sched_out(struct task_struct *task, struct task_struct *next)
 			perf_ctx_sched_task_cb(ctx, false);
 			perf_event_swap_task_ctx_data(ctx, next_ctx);
 
-			perf_ctx_enable(ctx, false);
+			perf_ctx_enable(ctx, 0);
 
 			/*
 			 * RCU_INIT_POINTER here is safe because we've not
@@ -3587,13 +3594,13 @@ perf_event_context_sched_out(struct task_struct *task, struct task_struct *next)
 
 	if (do_switch) {
 		raw_spin_lock(&ctx->lock);
-		perf_ctx_disable(ctx, false);
+		perf_ctx_disable(ctx, 0);
 
 inside_switch:
 		perf_ctx_sched_task_cb(ctx, false);
 		task_ctx_sched_out(ctx, EVENT_ALL);
 
-		perf_ctx_enable(ctx, false);
+		perf_ctx_enable(ctx, 0);
 		raw_spin_unlock(&ctx->lock);
 	}
 }
@@ -3890,12 +3897,12 @@ static void pmu_groups_sched_in(struct perf_event_context *ctx,
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				bool cgroup)
+				enum event_type_t event_type)
 {
 	struct perf_event_pmu_context *pmu_ctx;
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
-		if (cgroup && !pmu_ctx->nr_cgroups)
+		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
 		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
 	}
@@ -3912,9 +3919,6 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 	int is_active = ctx->is_active;
-	bool cgroup = event_type & EVENT_CGROUP;
-
-	event_type &= ~EVENT_CGROUP;
 
 	lockdep_assert_held(&ctx->lock);
 
@@ -3932,7 +3936,7 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 		barrier();
 	}
 
-	ctx->is_active |= (event_type | EVENT_TIME);
+	ctx->is_active |= ((event_type & ~EVENT_FLAGS) | EVENT_TIME);
 	if (ctx->task) {
 		if (!is_active)
 			cpuctx->task_ctx = ctx;
@@ -3947,11 +3951,11 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * in order to give them the best chance of going on.
 	 */
 	if (is_active & EVENT_PINNED)
-		ctx_groups_sched_in(ctx, &ctx->pinned_groups, cgroup);
+		ctx_groups_sched_in(ctx, &ctx->pinned_groups, event_type);
 
 	/* Then walk through the lower prio flexible groups */
 	if (is_active & EVENT_FLEXIBLE)
-		ctx_groups_sched_in(ctx, &ctx->flexible_groups, cgroup);
+		ctx_groups_sched_in(ctx, &ctx->flexible_groups, event_type);
 }
 
 static void perf_event_context_sched_in(struct task_struct *task)
@@ -3966,11 +3970,11 @@ static void perf_event_context_sched_in(struct task_struct *task)
 
 	if (cpuctx->task_ctx == ctx) {
 		perf_ctx_lock(cpuctx, ctx);
-		perf_ctx_disable(ctx, false);
+		perf_ctx_disable(ctx, 0);
 
 		perf_ctx_sched_task_cb(ctx, true);
 
-		perf_ctx_enable(ctx, false);
+		perf_ctx_enable(ctx, 0);
 		perf_ctx_unlock(cpuctx, ctx);
 		goto rcu_unlock;
 	}
@@ -3983,7 +3987,7 @@ static void perf_event_context_sched_in(struct task_struct *task)
 	if (!ctx->nr_events)
 		goto unlock;
 
-	perf_ctx_disable(ctx, false);
+	perf_ctx_disable(ctx, 0);
 	/*
 	 * We want to keep the following priority order:
 	 * cpu pinned (that don't need to move), task pinned,
@@ -3993,7 +3997,7 @@ static void perf_event_context_sched_in(struct task_struct *task)
 	 * events, no need to flip the cpuctx's events around.
 	 */
 	if (!RB_EMPTY_ROOT(&ctx->pinned_groups.tree)) {
-		perf_ctx_disable(&cpuctx->ctx, false);
+		perf_ctx_disable(&cpuctx->ctx, 0);
 		ctx_sched_out(&cpuctx->ctx, EVENT_FLEXIBLE);
 	}
 
@@ -4002,9 +4006,9 @@ static void perf_event_context_sched_in(struct task_struct *task)
 	perf_ctx_sched_task_cb(cpuctx->task_ctx, true);
 
 	if (!RB_EMPTY_ROOT(&ctx->pinned_groups.tree))
-		perf_ctx_enable(&cpuctx->ctx, false);
+		perf_ctx_enable(&cpuctx->ctx, 0);
 
-	perf_ctx_enable(ctx, false);
+	perf_ctx_enable(ctx, 0);
 
 unlock:
 	perf_ctx_unlock(cpuctx, ctx);
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
maintainer
#11 lore raw

Re: [RFC PATCH v3 07/58] perf: Skip pmu_ctx based on event_type

Quoted 31 lines · Mingwei Zhang
kernel/events/core.c +1 -7
@@ -3299,9 +3309,6 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 	struct perf_event_pmu_context *pmu_ctx;
 	int is_active = ctx->is_active;
-	bool cgroup = event_type & EVENT_CGROUP;
-
-	event_type &= ~EVENT_CGROUP;
 
 	lockdep_assert_held(&ctx->lock);
 
@@ -3336,7 +3343,7 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 		barrier();
 	}
 
-	ctx->is_active &= ~event_type;
+	ctx->is_active &= ~(event_type & ~EVENT_FLAGS);
 	if (!(ctx->is_active & EVENT_ALL))
 		ctx->is_active = 0;
 
@@ -3912,9 +3919,6 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 	int is_active = ctx->is_active;
-	bool cgroup = event_type & EVENT_CGROUP;
-
-	event_type &= ~EVENT_CGROUP;
 
 	lockdep_assert_held(&ctx->lock);
 

kernel/events/core.c ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type) · line 3942

Quoted 8 lines
kernel/events/core.c +1 -1
@@ -3932,7 +3936,7 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 		barrier();
 	}
 
-	ctx->is_active |= (event_type | EVENT_TIME);
+	ctx->is_active |= ((event_type & ~EVENT_FLAGS) | EVENT_TIME);
 	if (ctx->task) {
 		if (!is_active)
 			cpuctx->task_ctx = ctx;

Would it make sense to do something like:

	enum event_type_t active_type = event_type & ~EVENT_FLAGS;

	ctx->is_active &= ~active_type;
and
	ctx->is_active |= (active_type | EVENT_TIME);

Or something along those lines; those expressions become a little unwieldy otherwise.

#12 lore raw

RFC PATCH v3 8/58 perf: Clean up perf ctx time

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

The current perf tracks two timestamps for the normal ctx and cgroup. The same type of variables and similar codes are used to track the timestamps. In the following patch, the third timestamp to track the guest time will be introduced.
To avoid the code duplication, add a new struct perf_time_ctx and factor out a generic function update_perf_time_ctx().

No functional change.

---
 include/linux/perf_event.h | 13 +++++----
 kernel/events/core.c       | 59 +++++++++++++++++++-------------------
 2 files changed, 37 insertions(+), 35 deletions(-)
include/linux/perf_event.h +7 -6
index 45d1ea82aa21..e22cdb6486e6 100644
@@ -906,6 +906,11 @@ struct perf_event_groups {
 	u64		index;
 };
 
+struct perf_time_ctx {
+	u64		time;
+	u64		stamp;
+	u64		offset;
+};
 
 /**
  * struct perf_event_context - event context structure
@@ -945,9 +950,7 @@ struct perf_event_context {
 	/*
 	 * Context clock, runs when context enabled.
 	 */
-	u64				time;
-	u64				timestamp;
-	u64				timeoffset;
+	struct perf_time_ctx		time;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1040,9 +1043,7 @@ struct bpf_perf_event_data_kern {
  * This is a per-cpu dynamically allocated data structure.
  */
 struct perf_cgroup_info {
-	u64				time;
-	u64				timestamp;
-	u64				timeoffset;
+	struct perf_time_ctx		time;
 	int				active;
 };
 
kernel/events/core.c +30 -29

127 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
maintainer
#13 lore raw

Re: [RFC PATCH v3 08/58] perf: Clean up perf ctx time

Quoted 9 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

The current perf tracks two timestamps for the normal ctx and cgroup. The same type of variables and similar codes are used to track the timestamps. In the following patch, the third timestamp to track the guest time will be introduced.
To avoid the code duplication, add a new struct perf_time_ctx and factor out a generic function update_perf_time_ctx().

Quoted 5 lines

No functional change.

kernel/events/core.c +18 -25
@@ -801,11 +801,22 @@ static inline u64 perf_cgroup_event_time
 	return now;
 }
 
-static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv);
-
-static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 now, bool adv)
+static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv)
 {
-	update_perf_time_ctx(&info->time, now, adv);
+	if (adv)
+		time->time += now - time->stamp;
+	time->stamp = now;
+
+	/*
+	 * The above: time' = time + (now - timestamp), can be re-arranged
+	 * into: time` = now + (time - timestamp), which gives a single value
+	 * offset to compute future time without locks on.
+	 *
+	 * See perf_event_time_now(), which can be used from NMI context where
+	 * it's (obviously) not possible to acquire ctx->lock in order to read
+	 * both the above values in a consistent manner.
+	 */
+	WRITE_ONCE(time->offset, time->time - time->stamp);
 }
 
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
@@ -821,7 +832,7 @@ static inline void update_cgrp_time_from
 			cgrp = container_of(css, struct perf_cgroup, css);
 			info = this_cpu_ptr(cgrp->info);
 
-			__update_cgrp_time(info, now, true);
+			update_perf_time_ctx(&info->time, now, true);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -844,7 +855,7 @@ static inline void update_cgrp_time_from
 	 * Do not update time when cgroup is not active
 	 */
 	if (info->active)
-		__update_cgrp_time(info, perf_clock(), true);
+		update_perf_time_ctx(&info->time, perf_clock(), true);
 }
 
 static inline void
@@ -868,7 +879,7 @@ perf_cgroup_set_timestamp(struct perf_cp
 	for (css = &cgrp->css; css; css = css->parent) {
 		cgrp = container_of(css, struct perf_cgroup, css);
 		info = this_cpu_ptr(cgrp->info);
-		__update_cgrp_time(info, ctx->time.stamp, false);
+		update_perf_time_ctx(&info->time, ctx->time.stamp, false);
 		__store_release(&info->active, 1);
 	}
 }
@@ -1478,24 +1489,6 @@ static void perf_unpin_context(struct pe
 	raw_spin_unlock_irqrestore(&ctx->lock, flags);
 }
 
-static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv)
-{
-	if (adv)
-		time->time += now - time->stamp;
-	time->stamp = now;
-
-	/*
-	 * The above: time' = time + (now - timestamp), can be re-arranged
-	 * into: time` = now + (time - timestamp), which gives a single value
-	 * offset to compute future time without locks on.
-	 *
-	 * See perf_event_time_now(), which can be used from NMI context where
-	 * it's (obviously) not possible to acquire ctx->lock in order to read
-	 * both the above values in a consistent manner.
-	 */
-	WRITE_ONCE(time->offset, time->time - time->stamp);
-}
-
 /*
  * Update the record of the current time in a context.
  */
#14 lore raw

RFC PATCH v3 9/58 perf: Add a EVENT_GUEST flag

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the introduced passthrough vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

The ctx time is shared among different PMUs. The time cannot be stopped when a guest is running. It is required to calculate the time for events from other PMUs, e.g., uncore events. Add timeguest to track the guest run time. For an exclude_guest event, the elapsed time equals the ctx time - guest time.
Cgroup has dedicated times. Use the same method to deduct the guest time from the cgroup time as well.

---
 include/linux/perf_event.h |   6 ++
 kernel/events/core.c       | 178 +++++++++++++++++++++++++++++++------
 2 files changed, 155 insertions(+), 29 deletions(-)
include/linux/perf_event.h +6 -0
index e22cdb6486e6..81a5f8399cb8 100644
@@ -952,6 +952,11 @@ struct perf_event_context {
 	 */
 	struct perf_time_ctx		time;
 
+	/*
+	 * Context clock, runs when in the guest mode.
+	 */
+	struct perf_time_ctx		timeguest;
+
 	/*
 	 * These fields let us detect when two contexts have both
 	 * been cloned (inherited) from a common ancestor.
@@ -1044,6 +1049,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +149 -29

341 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#15 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 246 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the introduced passthrough vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

The ctx time is shared among different PMUs. The time cannot be stopped when a guest is running. It is required to calculate the time for events from other PMUs, e.g., uncore events. Add timeguest to track the guest run time. For an exclude_guest event, the elapsed time equals the ctx time - guest time.
Cgroup has dedicated times. Use the same method to deduct the guest time from the cgroup time as well.

---
 include/linux/perf_event.h |   6 ++
 kernel/events/core.c       | 178 +++++++++++++++++++++++++++++++------
 2 files changed, 155 insertions(+), 29 deletions(-)
include/linux/perf_event.h +6 -0
index e22cdb6486e6..81a5f8399cb8 100644
@@ -952,6 +952,11 @@ struct perf_event_context {
 	 */
 	struct perf_time_ctx		time;
 
+	/*
+	 * Context clock, runs when in the guest mode.
+	 */
+	struct perf_time_ctx		timeguest;
+
 	/*
 	 * These fields let us detect when two contexts have both
 	 * been cloned (inherited) from a common ancestor.
@@ -1044,6 +1049,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +69 -14
index c25e2bf27001..57648736e43e 100644
@@ -376,7 +376,8 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
-	EVENT_FLAGS = EVENT_CGROUP,
+	EVENT_GUEST = 0x20,
+	EVENT_FLAGS = EVENT_CGROUP | EVENT_GUEST,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -407,6 +408,7 @@ static atomic_t nr_include_guest_events __read_mostly;
 
 static atomic_t nr_mediated_pmu_vms;
 static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+static DEFINE_PER_CPU(bool, perf_in_guest);
 
 /* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
 static inline bool is_include_guest_event(struct perf_event *event)
@@ -706,6 +708,10 @@ static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
 	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
 		return true;
 
+	if ((event_type & EVENT_GUEST) &&
+	    !(pmu_ctx->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+		return true;
+
 	return false;
 }
 
@@ -770,12 +776,21 @@ static inline int is_cgroup_event(struct perf_event *event)
 	return event->cgrp != NULL;
 }
 
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest);
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now);
+
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time.time;
+	return __perf_event_time_ctx(event, &t->time, &t->timeguest);
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
@@ -784,9 +799,9 @@ static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
 	if (!__load_acquire(&t->active))
-		return t->time.time;
-	now += READ_ONCE(t->time.offset);
-	return now;
+		return __perf_event_time_ctx(event, &t->time, &t->timeguest);
+
+	return __perf_event_time_ctx_now(event, &t->time, &t->timeguest, now);
 }
 
 static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv);
@@ -796,6 +811,18 @@ static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 now, bo
 	update_perf_time_ctx(&info->time, now, adv);
 }
 
+static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info, u64 now, bool adv)
+{
+	update_perf_time_ctx(&info->timeguest, now, adv);
+}
+
+static inline void update_cgrp_time(struct perf_cgroup_info *info, u64 now)
+{
+	__update_cgrp_time(info, now, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_cgrp_guest_time(info, now, true);
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -809,7 +836,7 @@ static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx,
 			cgrp = container_of(css, struct perf_cgroup, css);
 			info = this_cpu_ptr(cgrp->info);
 
-			__update_cgrp_time(info, now, true);
+			update_cgrp_time(info, now);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -832,11 +859,11 @@ static inline void update_cgrp_time_from_event(struct perf_event *event)
 	 * Do not update time when cgroup is not active
 	 */
 	if (info->active)
-		__update_cgrp_time(info, perf_clock(), true);
+		update_cgrp_time(info, perf_clock());
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 	struct perf_event_context *ctx = &cpuctx->ctx;
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -856,8 +883,12 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
 	for (css = &cgrp->css; css; css = css->parent) {
 		cgrp = container_of(css, struct perf_cgroup, css);
 		info = this_cpu_ptr(cgrp->info);
-		__update_cgrp_time(info, ctx->time.stamp, false);
-		__store_release(&info->active, 1);
+		if (guest) {
+			__update_cgrp_guest_time(info, ctx->time.stamp, false);
+		} else {
+			__update_cgrp_time(info, ctx->time.stamp, false);
+			__store_release(&info->active, 1);
+		}
 	}
 }
 
@@ -1061,7 +1092,7 @@ static inline int perf_cgroup_connect(pid_t pid, struct perf_event *event,
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 }
 
@@ -1488,16 +1519,34 @@ static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, boo
  */
 static void __update_context_time(struct perf_event_context *ctx, bool adv)
 {
-	u64 now = perf_clock();
+	lockdep_assert_held(&ctx->lock);
+
+	update_perf_time_ctx(&ctx->time, perf_clock(), adv);
+}
 
+static void __update_context_guest_time(struct perf_event_context *ctx, bool adv)
+{
 	lockdep_assert_held(&ctx->lock);
 
-	update_perf_time_ctx(&ctx->time, now, adv);
+	/* must be called after __update_context_time(); */
+	update_perf_time_ctx(&ctx->timeguest, ctx->time.stamp, adv);
 }
 
 static void update_context_time(struct perf_event_context *ctx)
 {
 	__update_context_time(ctx, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_context_guest_time(ctx, true);
+}
+
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest)
+{
+	if (event->attr.exclude_guest)
+		return time->time - timeguest->time;
+	else
+		return time->time;
 }
 
 static u64 perf_event_time(struct perf_event *event)
@@ -1510,7 +1559,26 @@ static u64 perf_event_time(struct perf_event *event)
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time(event);
 
-	return ctx->time.time;
+	return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
+}
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now)

kernel/events/core.c static u64 perf_event_time(struct perf_event *event) · line 1578

Quoted 10 lines
kernel/events/core.c +10 -0
+{
+	/*
+	 * The exclude_guest event time should be calculated from
+	 * the ctx time -  the guest time.
+	 * The ctx time is now + READ_ONCE(time->offset).
+	 * The guest time is now + READ_ONCE(timeguest->offset).
+	 * So the exclude_guest time is
+	 * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
+	 */
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))

Hi Kan,

we see the following the warning when run perf record command after enabling "CONFIG_DEBUG_PREEMPT" config item.

[  166.779208] BUG: using __this_cpu_read() in preemptible [00000000] code:
perf/9494
[  166.779234] caller is __this_cpu_preempt_check+0x13/0x20
[  166.779241] CPU: 56 UID: 0 PID: 9494 Comm: perf Not tainted 6.11.0-rc4-perf-next-mediated-vpmu-v3+ #80
[  166.779245] Hardware name: Quanta Cloud Technology Inc. QuantaGrid D54Q-2U/S6Q-MB-MPS, BIOS 3A11.uh 12/02/2022
[  166.779248] Call Trace:
[  166.779250]  <TASK>
[  166.779252]  dump_stack_lvl+0x76/0xa0
[  166.779260]  dump_stack+0x10/0x20
[  166.779267]  check_preemption_disabled+0xd7/0xf0
[  166.779273]  __this_cpu_preempt_check+0x13/0x20
[  166.779279]  calc_timer_values+0x193/0x200
[  166.779287]  perf_event_update_userpage+0x4b/0x170
[  166.779294]  ? ring_buffer_attach+0x14c/0x200
[  166.779301]  perf_mmap+0x533/0x5d0
[  166.779309]  mmap_region+0x243/0xaa0
[  166.779322]  do_mmap+0x35b/0x640
[  166.779333]  vm_mmap_pgoff+0xf0/0x1c0
[  166.779345]  ksys_mmap_pgoff+0x17a/0x250
[  166.779354]  __x64_sys_mmap+0x33/0x70
[  166.779362]  x64_sys_call+0x1fa4/0x25f0
[  166.779369]  do_syscall_64+0x70/0x130

The season that kernel complains this is __perf_event_time_ctx_now() calls __this_cpu_read() in preemption enabled context.

To eliminate the warning, we may need to use this_cpu_read() to replace __this_cpu_read().

kernel/events/core.c +0 -0
index ccd61fd06e8d..1eb628f8b3a0 100644
@@ -1581,7 +1581,7 @@ static inline u64 __perf_event_time_ctx_now(struct

perf_event *event,
         * So the exclude_guest time is
         * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset). */
-       if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest)) +       if (event->attr.exclude_guest && this_cpu_read(perf_in_guest)) return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset); else
                return now + READ_ONCE(time->offset);

Quoted 172 lines
kernel/events/core.c +70 -15
+		return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);
+	else
+		return now + READ_ONCE(time->offset);
 }
 
 static u64 perf_event_time_now(struct perf_event *event, u64 now)
@@ -1524,10 +1592,9 @@ static u64 perf_event_time_now(struct perf_event *event, u64 now)
 		return perf_cgroup_event_time_now(event, now);
 
 	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME))
-		return ctx->time.time;
+		return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
 
-	now += READ_ONCE(ctx->time.offset);
-	return now;
+	return __perf_event_time_ctx_now(event, &ctx->time, &ctx->timeguest, now);
 }
 
 static enum event_type_t get_event_type(struct perf_event *event)
@@ -3334,9 +3401,15 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		/* vPMU should not stop time */
+		stop = !(event_type & EVENT_GUEST) &&
+		       ctx == &cpuctx->ctx;
+
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3354,7 +3427,18 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_timestamp(cpuctx, true);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3853,10 +3937,15 @@ static inline void group_update_userpage(struct perf_event *group_event)
 		event_update_userpage(event);
 }
 
+struct merge_sched_data {
+	int can_add_hw;
+	enum event_type_t event_type;
+};
+
 static int merge_sched_in(struct perf_event *event, void *data)
 {
 	struct perf_event_context *ctx = event->ctx;
-	int *can_add_hw = data;
+	struct merge_sched_data *msd = data;
 
 	if (event->state <= PERF_EVENT_STATE_OFF)
 		return 0;
@@ -3864,13 +3953,22 @@ static int merge_sched_in(struct perf_event *event, void *data)
 	if (!event_filter_match(event))
 		return 0;
 
-	if (group_can_go_on(event, *can_add_hw)) {
+	/*
+	 * Don't schedule in any exclude_guest events of PMU with
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+	 */
+	if (__this_cpu_read(perf_in_guest) && event->attr.exclude_guest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+	    !(msd->event_type & EVENT_GUEST))
+		return 0;
+
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 	}
 
 	if (event->state == PERF_EVENT_STATE_INACTIVE) {
-		*can_add_hw = 0;
+		msd->can_add_hw = 0;
 		if (event->attr.pinned) {
 			perf_cgroup_event_disable(event, ctx);
 			perf_event_set_state(event, PERF_EVENT_STATE_ERROR);
@@ -3889,11 +3987,15 @@ static int merge_sched_in(struct perf_event *event, void *data)
 
 static void pmu_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				struct pmu *pmu)
+				struct pmu *pmu,
+				enum event_type_t event_type)
 {
-	int can_add_hw = 1;
+	struct merge_sched_data msd = {
+		.can_add_hw = 1,
+		.event_type = event_type,
+	};
 	visit_groups_merge(ctx, groups, smp_processor_id(), pmu,
-			   merge_sched_in, &can_add_hw);
+			   merge_sched_in, &msd);
 }
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
@@ -3905,14 +4007,14 @@ static void ctx_groups_sched_in(struct perf_event_context *ctx,
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
-		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
+		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu, event_type);
 	}
 }
 
 static void __pmu_ctx_sched_in(struct perf_event_context *ctx,
 			       struct pmu *pmu)
 {
-	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu, 0);
 }
 
 static void
@@ -3927,9 +4029,11 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
-		perf_cgroup_set_timestamp(cpuctx);
+		perf_cgroup_set_timestamp(cpuctx, false);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3945,7 +4049,23 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule in all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+
+		/*
+		 * Update ctx time to set the new start time for
+		 * the exclude_guest events.
+		 */
+		update_context_time(ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	/*
 	 * First go through the list and put on any pinned groups
#16 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 289 lines · Dapeng
Quoted 246 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the introduced passthrough vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

The ctx time is shared among different PMUs. The time cannot be stopped when a guest is running. It is required to calculate the time for events from other PMUs, e.g., uncore events. Add timeguest to track the guest run time. For an exclude_guest event, the elapsed time equals the ctx time - guest time.
Cgroup has dedicated times. Use the same method to deduct the guest time from the cgroup time as well.

---
 include/linux/perf_event.h |   6 ++
 kernel/events/core.c       | 178 +++++++++++++++++++++++++++++++------
 2 files changed, 155 insertions(+), 29 deletions(-)
include/linux/perf_event.h +6 -0
index e22cdb6486e6..81a5f8399cb8 100644
@@ -952,6 +952,11 @@ struct perf_event_context {
 	 */
 	struct perf_time_ctx		time;
 
+	/*
+	 * Context clock, runs when in the guest mode.
+	 */
+	struct perf_time_ctx		timeguest;
+
 	/*
 	 * These fields let us detect when two contexts have both
 	 * been cloned (inherited) from a common ancestor.
@@ -1044,6 +1049,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +69 -14
index c25e2bf27001..57648736e43e 100644
@@ -376,7 +376,8 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
-	EVENT_FLAGS = EVENT_CGROUP,
+	EVENT_GUEST = 0x20,
+	EVENT_FLAGS = EVENT_CGROUP | EVENT_GUEST,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -407,6 +408,7 @@ static atomic_t nr_include_guest_events __read_mostly;
 
 static atomic_t nr_mediated_pmu_vms;
 static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+static DEFINE_PER_CPU(bool, perf_in_guest);
 
 /* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
 static inline bool is_include_guest_event(struct perf_event *event)
@@ -706,6 +708,10 @@ static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
 	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
 		return true;
 
+	if ((event_type & EVENT_GUEST) &&
+	    !(pmu_ctx->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+		return true;
+
 	return false;
 }
 
@@ -770,12 +776,21 @@ static inline int is_cgroup_event(struct perf_event *event)
 	return event->cgrp != NULL;
 }
 
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest);
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now);
+
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time.time;
+	return __perf_event_time_ctx(event, &t->time, &t->timeguest);
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
@@ -784,9 +799,9 @@ static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
 	if (!__load_acquire(&t->active))
-		return t->time.time;
-	now += READ_ONCE(t->time.offset);
-	return now;
+		return __perf_event_time_ctx(event, &t->time, &t->timeguest);
+
+	return __perf_event_time_ctx_now(event, &t->time, &t->timeguest, now);
 }
 
 static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv);
@@ -796,6 +811,18 @@ static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 now, bo
 	update_perf_time_ctx(&info->time, now, adv);
 }
 
+static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info, u64 now, bool adv)
+{
+	update_perf_time_ctx(&info->timeguest, now, adv);
+}
+
+static inline void update_cgrp_time(struct perf_cgroup_info *info, u64 now)
+{
+	__update_cgrp_time(info, now, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_cgrp_guest_time(info, now, true);
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -809,7 +836,7 @@ static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx,
 			cgrp = container_of(css, struct perf_cgroup, css);
 			info = this_cpu_ptr(cgrp->info);
 
-			__update_cgrp_time(info, now, true);
+			update_cgrp_time(info, now);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -832,11 +859,11 @@ static inline void update_cgrp_time_from_event(struct perf_event *event)
 	 * Do not update time when cgroup is not active
 	 */
 	if (info->active)
-		__update_cgrp_time(info, perf_clock(), true);
+		update_cgrp_time(info, perf_clock());
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 	struct perf_event_context *ctx = &cpuctx->ctx;
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -856,8 +883,12 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
 	for (css = &cgrp->css; css; css = css->parent) {
 		cgrp = container_of(css, struct perf_cgroup, css);
 		info = this_cpu_ptr(cgrp->info);
-		__update_cgrp_time(info, ctx->time.stamp, false);
-		__store_release(&info->active, 1);
+		if (guest) {
+			__update_cgrp_guest_time(info, ctx->time.stamp, false);
+		} else {
+			__update_cgrp_time(info, ctx->time.stamp, false);
+			__store_release(&info->active, 1);
+		}
 	}
 }
 
@@ -1061,7 +1092,7 @@ static inline int perf_cgroup_connect(pid_t pid, struct perf_event *event,
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 }
 
@@ -1488,16 +1519,34 @@ static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, boo
  */
 static void __update_context_time(struct perf_event_context *ctx, bool adv)
 {
-	u64 now = perf_clock();
+	lockdep_assert_held(&ctx->lock);
+
+	update_perf_time_ctx(&ctx->time, perf_clock(), adv);
+}
 
+static void __update_context_guest_time(struct perf_event_context *ctx, bool adv)
+{
 	lockdep_assert_held(&ctx->lock);
 
-	update_perf_time_ctx(&ctx->time, now, adv);
+	/* must be called after __update_context_time(); */
+	update_perf_time_ctx(&ctx->timeguest, ctx->time.stamp, adv);
 }
 
 static void update_context_time(struct perf_event_context *ctx)
 {
 	__update_context_time(ctx, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_context_guest_time(ctx, true);
+}
+
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest)
+{
+	if (event->attr.exclude_guest)
+		return time->time - timeguest->time;
+	else
+		return time->time;
 }
 
 static u64 perf_event_time(struct perf_event *event)
@@ -1510,7 +1559,26 @@ static u64 perf_event_time(struct perf_event *event)
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time(event);
 
-	return ctx->time.time;
+	return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
+}
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now)

kernel/events/core.c static u64 perf_event_time(struct perf_event *event) · line 1578

Quoted 10 lines
kernel/events/core.c +10 -0
+{
+	/*
+	 * The exclude_guest event time should be calculated from
+	 * the ctx time -  the guest time.
+	 * The ctx time is now + READ_ONCE(time->offset).
+	 * The guest time is now + READ_ONCE(timeguest->offset).
+	 * So the exclude_guest time is
+	 * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
+	 */
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))

Hi Kan,

we see the following the warning when run perf record command after enabling "CONFIG_DEBUG_PREEMPT" config item.

[  166.779208] BUG: using __this_cpu_read() in preemptible [00000000] code:
perf/9494
[  166.779234] caller is __this_cpu_preempt_check+0x13/0x20
[  166.779241] CPU: 56 UID: 0 PID: 9494 Comm: perf Not tainted 6.11.0-rc4-perf-next-mediated-vpmu-v3+ #80
[  166.779245] Hardware name: Quanta Cloud Technology Inc. QuantaGrid D54Q-2U/S6Q-MB-MPS, BIOS 3A11.uh 12/02/2022
[  166.779248] Call Trace:
[  166.779250]  <TASK>
[  166.779252]  dump_stack_lvl+0x76/0xa0
[  166.779260]  dump_stack+0x10/0x20
[  166.779267]  check_preemption_disabled+0xd7/0xf0
[  166.779273]  __this_cpu_preempt_check+0x13/0x20
[  166.779279]  calc_timer_values+0x193/0x200
[  166.779287]  perf_event_update_userpage+0x4b/0x170
[  166.779294]  ? ring_buffer_attach+0x14c/0x200
[  166.779301]  perf_mmap+0x533/0x5d0
[  166.779309]  mmap_region+0x243/0xaa0
[  166.779322]  do_mmap+0x35b/0x640
[  166.779333]  vm_mmap_pgoff+0xf0/0x1c0
[  166.779345]  ksys_mmap_pgoff+0x17a/0x250
[  166.779354]  __x64_sys_mmap+0x33/0x70
[  166.779362]  x64_sys_call+0x1fa4/0x25f0
[  166.779369]  do_syscall_64+0x70/0x130

Quoted 5 lines

The season that kernel complains this is __perf_event_time_ctx_now() calls __this_cpu_read() in preemption enabled context.

To eliminate the warning, we may need to use this_cpu_read() to replace __this_cpu_read().

Sure.

Besides this, we recently update the time related code in perf.
https://lore.kernel.org/lkml/172311312757.2215.323044538405607858.tip-bot2@tip-bot2/

This patch probably have to be rebased on top of it.

Thanks,
Kan

Quoted 189 lines
kernel/events/core.c +1 -1
index ccd61fd06e8d..1eb628f8b3a0 100644
@@ -1581,7 +1581,7 @@ static inline u64 __perf_event_time_ctx_now(struct
 perf_event *event,
          * So the exclude_guest time is
          * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
          */
-       if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))
+       if (event->attr.exclude_guest && this_cpu_read(perf_in_guest))
                 return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);

        else
                return now + READ_ONCE(time->offset);

Quoted 172 lines
kernel/events/core.c +70 -15
+		return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);
+	else
+		return now + READ_ONCE(time->offset);
 }
 
 static u64 perf_event_time_now(struct perf_event *event, u64 now)
@@ -1524,10 +1592,9 @@ static u64 perf_event_time_now(struct perf_event *event, u64 now)
 		return perf_cgroup_event_time_now(event, now);
 
 	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME))
-		return ctx->time.time;
+		return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
 
-	now += READ_ONCE(ctx->time.offset);
-	return now;
+	return __perf_event_time_ctx_now(event, &ctx->time, &ctx->timeguest, now);
 }
 
 static enum event_type_t get_event_type(struct perf_event *event)
@@ -3334,9 +3401,15 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		/* vPMU should not stop time */
+		stop = !(event_type & EVENT_GUEST) &&
+		       ctx == &cpuctx->ctx;
+
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3354,7 +3427,18 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_timestamp(cpuctx, true);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3853,10 +3937,15 @@ static inline void group_update_userpage(struct perf_event *group_event)
 		event_update_userpage(event);
 }
 
+struct merge_sched_data {
+	int can_add_hw;
+	enum event_type_t event_type;
+};
+
 static int merge_sched_in(struct perf_event *event, void *data)
 {
 	struct perf_event_context *ctx = event->ctx;
-	int *can_add_hw = data;
+	struct merge_sched_data *msd = data;
 
 	if (event->state <= PERF_EVENT_STATE_OFF)
 		return 0;
@@ -3864,13 +3953,22 @@ static int merge_sched_in(struct perf_event *event, void *data)
 	if (!event_filter_match(event))
 		return 0;
 
-	if (group_can_go_on(event, *can_add_hw)) {
+	/*
+	 * Don't schedule in any exclude_guest events of PMU with
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+	 */
+	if (__this_cpu_read(perf_in_guest) && event->attr.exclude_guest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+	    !(msd->event_type & EVENT_GUEST))
+		return 0;
+
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 	}
 
 	if (event->state == PERF_EVENT_STATE_INACTIVE) {
-		*can_add_hw = 0;
+		msd->can_add_hw = 0;
 		if (event->attr.pinned) {
 			perf_cgroup_event_disable(event, ctx);
 			perf_event_set_state(event, PERF_EVENT_STATE_ERROR);
@@ -3889,11 +3987,15 @@ static int merge_sched_in(struct perf_event *event, void *data)
 
 static void pmu_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				struct pmu *pmu)
+				struct pmu *pmu,
+				enum event_type_t event_type)
 {
-	int can_add_hw = 1;
+	struct merge_sched_data msd = {
+		.can_add_hw = 1,
+		.event_type = event_type,
+	};
 	visit_groups_merge(ctx, groups, smp_processor_id(), pmu,
-			   merge_sched_in, &can_add_hw);
+			   merge_sched_in, &msd);
 }
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
@@ -3905,14 +4007,14 @@ static void ctx_groups_sched_in(struct perf_event_context *ctx,
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
-		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
+		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu, event_type);
 	}
 }
 
 static void __pmu_ctx_sched_in(struct perf_event_context *ctx,
 			       struct pmu *pmu)
 {
-	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu, 0);
 }
 
 static void
@@ -3927,9 +4029,11 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
-		perf_cgroup_set_timestamp(cpuctx);
+		perf_cgroup_set_timestamp(cpuctx, false);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3945,7 +4049,23 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule in all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+
+		/*
+		 * Update ctx time to set the new start time for
+		 * the exclude_guest events.
+		 */
+		update_context_time(ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	/*
 	 * First go through the list and put on any pinned groups
maintainer
#17 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

In reply to Mi, Dapeng
Quoted 65 lines · Dapeng
Quoted 4 lines · Mingwei Zhang
(quoted diff) +4 -0
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now)

Review

Quoted 10 lines
(quoted diff) +10 -0
+{
+	/*
+	 * The exclude_guest event time should be calculated from
+	 * the ctx time -  the guest time.
+	 * The ctx time is now + READ_ONCE(time->offset).
+	 * The guest time is now + READ_ONCE(timeguest->offset).
+	 * So the exclude_guest time is
+	 * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
+	 */
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))

Hi Kan,

we see the following the warning when run perf record command after enabling "CONFIG_DEBUG_PREEMPT" config item.

[  166.779208] BUG: using __this_cpu_read() in preemptible [00000000] code:
perf/9494
[  166.779234] caller is __this_cpu_preempt_check+0x13/0x20
[  166.779241] CPU: 56 UID: 0 PID: 9494 Comm: perf Not tainted 6.11.0-rc4-perf-next-mediated-vpmu-v3+ #80
[  166.779245] Hardware name: Quanta Cloud Technology Inc. QuantaGrid D54Q-2U/S6Q-MB-MPS, BIOS 3A11.uh 12/02/2022
[  166.779248] Call Trace:
[  166.779250]  <TASK>
[  166.779252]  dump_stack_lvl+0x76/0xa0
[  166.779260]  dump_stack+0x10/0x20
[  166.779267]  check_preemption_disabled+0xd7/0xf0
[  166.779273]  __this_cpu_preempt_check+0x13/0x20
[  166.779279]  calc_timer_values+0x193/0x200
[  166.779287]  perf_event_update_userpage+0x4b/0x170
[  166.779294]  ? ring_buffer_attach+0x14c/0x200
[  166.779301]  perf_mmap+0x533/0x5d0
[  166.779309]  mmap_region+0x243/0xaa0
[  166.779322]  do_mmap+0x35b/0x640
[  166.779333]  vm_mmap_pgoff+0xf0/0x1c0
[  166.779345]  ksys_mmap_pgoff+0x17a/0x250
[  166.779354]  __x64_sys_mmap+0x33/0x70
[  166.779362]  x64_sys_call+0x1fa4/0x25f0
[  166.779369]  do_syscall_64+0x70/0x130

The season that kernel complains this is __perf_event_time_ctx_now() calls __this_cpu_read() in preemption enabled context.

To eliminate the warning, we may need to use this_cpu_read() to replace __this_cpu_read().

kernel/events/core.c +1 -1
index ccd61fd06e8d..1eb628f8b3a0 100644
@@ -1581,7 +1581,7 @@ static inline u64 __perf_event_time_ctx_now(struct
 perf_event *event,
          * So the exclude_guest time is
          * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
          */
-       if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))
+       if (event->attr.exclude_guest && this_cpu_read(perf_in_guest))
                 return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);
Quoted 2 lines

        else
                return now + READ_ONCE(time->offset);

The saner fix is moving the preempt_disable() in
perf_event_update_userpage() up a few lines, no?

#18 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 63 lines · Peter Zijlstra
Quoted 62 lines · Dapeng
Quoted 4 lines · Mingwei Zhang
(quoted diff) +4 -0
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now)

Review

Quoted 10 lines
(quoted diff) +10 -0
+{
+	/*
+	 * The exclude_guest event time should be calculated from
+	 * the ctx time -  the guest time.
+	 * The ctx time is now + READ_ONCE(time->offset).
+	 * The guest time is now + READ_ONCE(timeguest->offset).
+	 * So the exclude_guest time is
+	 * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
+	 */
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))

Hi Kan,

we see the following the warning when run perf record command after enabling "CONFIG_DEBUG_PREEMPT" config item.

[  166.779208] BUG: using __this_cpu_read() in preemptible [00000000] code:
perf/9494
[  166.779234] caller is __this_cpu_preempt_check+0x13/0x20
[  166.779241] CPU: 56 UID: 0 PID: 9494 Comm: perf Not tainted 6.11.0-rc4-perf-next-mediated-vpmu-v3+ #80
[  166.779245] Hardware name: Quanta Cloud Technology Inc. QuantaGrid D54Q-2U/S6Q-MB-MPS, BIOS 3A11.uh 12/02/2022
[  166.779248] Call Trace:
[  166.779250]  <TASK>
[  166.779252]  dump_stack_lvl+0x76/0xa0
[  166.779260]  dump_stack+0x10/0x20
[  166.779267]  check_preemption_disabled+0xd7/0xf0
[  166.779273]  __this_cpu_preempt_check+0x13/0x20
[  166.779279]  calc_timer_values+0x193/0x200
[  166.779287]  perf_event_update_userpage+0x4b/0x170
[  166.779294]  ? ring_buffer_attach+0x14c/0x200
[  166.779301]  perf_mmap+0x533/0x5d0
[  166.779309]  mmap_region+0x243/0xaa0
[  166.779322]  do_mmap+0x35b/0x640
[  166.779333]  vm_mmap_pgoff+0xf0/0x1c0
[  166.779345]  ksys_mmap_pgoff+0x17a/0x250
[  166.779354]  __x64_sys_mmap+0x33/0x70
[  166.779362]  x64_sys_call+0x1fa4/0x25f0
[  166.779369]  do_syscall_64+0x70/0x130

The season that kernel complains this is __perf_event_time_ctx_now() calls __this_cpu_read() in preemption enabled context.

To eliminate the warning, we may need to use this_cpu_read() to replace __this_cpu_read().

kernel/events/core.c +0 -0
index ccd61fd06e8d..1eb628f8b3a0 100644
@@ -1581,7 +1581,7 @@ static inline u64 __perf_event_time_ctx_now(struct
 perf_event *event,
          * So the exclude_guest time is
          * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
          */
Quoted 8 lines
Quoted 5 lines
kernel/events/core.c +1 -1
-       if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))
+       if (event->attr.exclude_guest && this_cpu_read(perf_in_guest))
                 return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);

        else
                return now + READ_ONCE(time->offset);

The saner fix is moving the preempt_disable() in
perf_event_update_userpage() up a few lines, no?

Yes, the preempt_disable() is to guarantee consistent time stamps. It makes sense to include the time calculation part in the preempt_disable().

Since the time-related code was updated recently, I will fold all the suggestions into the newly re-based code.

Thanks,
Kan

maintainer
#19 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

In reply to Mingwei Zhang

Can you rework this one along these lines?

---

include/linux/perf_event.h +2 -0
@@ -975,6 +975,7 @@ struct perf_event_context {
 	 * Context clock, runs when context enabled.
 	 */
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1066,6 +1067,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +43 -4

73 lines not on this page · raw message

#20 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 2 lines · Peter Zijlstra

Can you rework this one along these lines?

Sure.

I probably also add macros to replace the magic number 0 and 1. For example,

#define T_TOTAL 0
#define T_GUEST 1

Thanks,
Kan

Quoted 99 lines

---

include/linux/perf_event.h +2 -0
@@ -975,6 +975,7 @@ struct perf_event_context {
 	 * Context clock, runs when context enabled.
 	 */
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 
 	/*
 	 * These fields let us detect when two contexts have both
@@ -1066,6 +1067,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +43 -4
@@ -782,12 +782,44 @@ static inline int is_cgroup_event(struct
 	return event->cgrp != NULL;
 }
 
+static_assert(offsetof(struct perf_event, timeguest) -
+	      offsetof(struct perf_event, time) == 
+	      sizeof(struct perf_time_ctx));
+
+static_assert(offsetof(struct perf_cgroup_info, timeguest) -
+	      offsetof(struct perf_cgroup_info, time) ==
+	      sizeof(struct perf_time_ctx));
+
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *times)
+{
+	u64 time = times[0].time;
+	if (event->attr.exclude_guest)
+		time -= times[1].time;
+	return time;
+}
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *times,
+					    u64 now)
+{
+	if (event->attr.exclude_guest) {
+		/*
+		 * (now + times[0].offset) - (now + times[1].offset) :=
+		 * times[0].offset - times[1].offset
+		 */
+		return READ_ONCE(times[0].offset) - READ_ONCE(times[1].offset);
+	}
+
+	return now + READ_ONCE(times[0].offset);
+}
+
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time.time;
+	return __perf_event_time_ctx(event, &t->time);
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
@@ -796,12 +828,12 @@ static inline u64 perf_cgroup_event_time
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
 	if (!__load_acquire(&t->active))
-		return t->time.time;
+		return __perf_event_time_ctx(event, &t->time);
 	now += READ_ONCE(t->time.offset);
-	return now;
+	return __perf_event_time_ctx_now(event, &t->time, now);
 }
 
-static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv)
+static inline void __update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv)
 {
 	if (adv)
 		time->time += now - time->stamp;
@@ -819,6 +851,13 @@ static inline void update_perf_time_ctx(
 	WRITE_ONCE(time->offset, time->time - time->stamp);
 }
 
+static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv)
+{
+	__update_perf_time_ctx(time + 0, now, adv);
+	if (__this_cpu_read(perf_in_guest))
+		__update_perf_time_ctx(time + 1, now, adv)
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
maintainer
#21 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 7 lines · Kan
Quoted 2 lines · Peter Zijlstra

Can you rework this one along these lines?

Sure.

Thanks!

maintainer
#22 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

In reply to Mingwei Zhang
Quoted 18 lines · Mingwei Zhang
kernel/events/core.c +7 -1
@@ -3334,9 +3401,15 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		/* vPMU should not stop time */
+		stop = !(event_type & EVENT_GUEST) &&
+		       ctx == &cpuctx->ctx;
+
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()

kernel/events/core.c ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type) · line 3434

Quoted 9 lines
kernel/events/core.c +5 -1
@@ -3354,7 +3427,18 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */

I thought the premise was that if we have !exclude_guest events, we'll fail the creation of vPMU, and if we have vPMU we'll fail the creation of !exclude_guest events.

As such, they're mutually exclusive, and the above comment doesn't make sense, if we have a vPMU, there are no !exclude_guest events, IOW schedule out the entire context.

Quoted 12 lines
kernel/events/core.c +7 -0
+		is_active = EVENT_ALL;
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_timestamp(cpuctx, true);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))

kernel/events/core.c static int merge_sched_in(struct perf_event *event, void *data) · line 3959

Quoted 8 lines
kernel/events/core.c +4 -1
@@ -3864,13 +3953,22 @@ static int merge_sched_in(struct perf_event *event, void *data)
 	if (!event_filter_match(event))
 		return 0;
 
-	if (group_can_go_on(event, *can_add_hw)) {
+	/*
+	 * Don't schedule in any exclude_guest events of PMU with
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+	 */

More confusion; if we have vPMU there should not be !exclude_guest events, right? So the above then becomes 'Don't schedule in any events'.

Quoted 10 lines
kernel/events/core.c +6 -0
+	if (__this_cpu_read(perf_in_guest) && event->attr.exclude_guest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+	    !(msd->event_type & EVENT_GUEST))
+		return 0;
+
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 

kernel/events/core.c ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type) · line 4056

Quoted 9 lines
kernel/events/core.c +5 -1
@@ -3945,7 +4049,23 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule in all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */

Idem.

Quoted 12 lines
kernel/events/core.c +12 -0
+		is_active = EVENT_ALL;
+
+		/*
+		 * Update ctx time to set the new start time for
+		 * the exclude_guest events.
+		 */
+		update_context_time(ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
#23 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 34 lines · Peter Zijlstra
Quoted 18 lines · Mingwei Zhang
kernel/events/core.c +7 -1
@@ -3334,9 +3401,15 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		/* vPMU should not stop time */
+		stop = !(event_type & EVENT_GUEST) &&
+		       ctx == &cpuctx->ctx;
+
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()

kernel/events/core.c ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type) · line 3434

Quoted 9 lines
kernel/events/core.c +5 -1
@@ -3354,7 +3427,18 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */

I thought the premise was that if we have !exclude_guest events, we'll fail the creation of vPMU, and if we have vPMU we'll fail the creation of !exclude_guest events.

Quoted 3 lines

As such, they're mutually exclusive, and the above comment doesn't make sense, if we have a vPMU, there are no !exclude_guest events, IOW schedule out the entire context.

It's a typo. Right, it should schedule out all exclude_guest events of the passthrough PMUs.

Quoted 16 lines
Quoted 16 lines
kernel/events/core.c +7 -0
+		is_active = EVENT_ALL;
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_timestamp(cpuctx, true);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3864,13 +3953,22 @@ static int merge_sched_in(struct perf_event *event, void *data)
 	if (!event_filter_match(event))
 		return 0;
 
Quoted 8 lines

kernel/events/core.c static int merge_sched_in(struct perf_event *event, void *data) · line 3959

Quoted 5 lines
kernel/events/core.c +4 -1
-	if (group_can_go_on(event, *can_add_hw)) {
+	/*
+	 * Don't schedule in any exclude_guest events of PMU with
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+	 */

More confusion; if we have vPMU there should not be !exclude_guest events, right? So the above then becomes 'Don't schedule in any events'.

Right. I will correct the three comments.

Thanks,
Kan

Quoted 36 lines
Quoted 11 lines
kernel/events/core.c +6 -0
+	if (__this_cpu_read(perf_in_guest) && event->attr.exclude_guest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+	    !(msd->event_type & EVENT_GUEST))
+		return 0;
+
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 

kernel/events/core.c ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type) · line 4056

Quoted 9 lines
kernel/events/core.c +5 -1
@@ -3945,7 +4049,23 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule in all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */

Idem.

Quoted 12 lines
kernel/events/core.c +12 -0
+		is_active = EVENT_ALL;
+
+		/*
+		 * Update ctx time to set the new start time for
+		 * the exclude_guest events.
+		 */
+		update_context_time(ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
#24 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

In reply to Mingwei Zhang
Quoted 333 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the introduced passthrough vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

The ctx time is shared among different PMUs. The time cannot be stopped when a guest is running. It is required to calculate the time for events from other PMUs, e.g., uncore events. Add timeguest to track the guest run time. For an exclude_guest event, the elapsed time equals the ctx time - guest time.
Cgroup has dedicated times. Use the same method to deduct the guest time from the cgroup time as well.

---
 include/linux/perf_event.h |   6 ++
 kernel/events/core.c       | 178 +++++++++++++++++++++++++++++++------
 2 files changed, 155 insertions(+), 29 deletions(-)
include/linux/perf_event.h +6 -0
index e22cdb6486e6..81a5f8399cb8 100644
@@ -952,6 +952,11 @@ struct perf_event_context {
 	 */
 	struct perf_time_ctx		time;
 
+	/*
+	 * Context clock, runs when in the guest mode.
+	 */
+	struct perf_time_ctx		timeguest;
+
 	/*
 	 * These fields let us detect when two contexts have both
 	 * been cloned (inherited) from a common ancestor.
@@ -1044,6 +1049,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +109 -20
index c25e2bf27001..57648736e43e 100644
@@ -376,7 +376,8 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
-	EVENT_FLAGS = EVENT_CGROUP,
+	EVENT_GUEST = 0x20,
+	EVENT_FLAGS = EVENT_CGROUP | EVENT_GUEST,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -407,6 +408,7 @@ static atomic_t nr_include_guest_events __read_mostly;
 
 static atomic_t nr_mediated_pmu_vms;
 static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+static DEFINE_PER_CPU(bool, perf_in_guest);
 
 /* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
 static inline bool is_include_guest_event(struct perf_event *event)
@@ -706,6 +708,10 @@ static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
 	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
 		return true;
 
+	if ((event_type & EVENT_GUEST) &&
+	    !(pmu_ctx->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+		return true;
+
 	return false;
 }
 
@@ -770,12 +776,21 @@ static inline int is_cgroup_event(struct perf_event *event)
 	return event->cgrp != NULL;
 }
 
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest);
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now);
+
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time.time;
+	return __perf_event_time_ctx(event, &t->time, &t->timeguest);
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
@@ -784,9 +799,9 @@ static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
 	if (!__load_acquire(&t->active))
-		return t->time.time;
-	now += READ_ONCE(t->time.offset);
-	return now;
+		return __perf_event_time_ctx(event, &t->time, &t->timeguest);
+
+	return __perf_event_time_ctx_now(event, &t->time, &t->timeguest, now);
 }
 
 static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv);
@@ -796,6 +811,18 @@ static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 now, bo
 	update_perf_time_ctx(&info->time, now, adv);
 }
 
+static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info, u64 now, bool adv)
+{
+	update_perf_time_ctx(&info->timeguest, now, adv);
+}
+
+static inline void update_cgrp_time(struct perf_cgroup_info *info, u64 now)
+{
+	__update_cgrp_time(info, now, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_cgrp_guest_time(info, now, true);
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -809,7 +836,7 @@ static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx,
 			cgrp = container_of(css, struct perf_cgroup, css);
 			info = this_cpu_ptr(cgrp->info);
 
-			__update_cgrp_time(info, now, true);
+			update_cgrp_time(info, now);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -832,11 +859,11 @@ static inline void update_cgrp_time_from_event(struct perf_event *event)
 	 * Do not update time when cgroup is not active
 	 */
 	if (info->active)
-		__update_cgrp_time(info, perf_clock(), true);
+		update_cgrp_time(info, perf_clock());
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 	struct perf_event_context *ctx = &cpuctx->ctx;
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -856,8 +883,12 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
 	for (css = &cgrp->css; css; css = css->parent) {
 		cgrp = container_of(css, struct perf_cgroup, css);
 		info = this_cpu_ptr(cgrp->info);
-		__update_cgrp_time(info, ctx->time.stamp, false);
-		__store_release(&info->active, 1);
+		if (guest) {
+			__update_cgrp_guest_time(info, ctx->time.stamp, false);
+		} else {
+			__update_cgrp_time(info, ctx->time.stamp, false);
+			__store_release(&info->active, 1);
+		}
 	}
 }
 
@@ -1061,7 +1092,7 @@ static inline int perf_cgroup_connect(pid_t pid, struct perf_event *event,
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 }
 
@@ -1488,16 +1519,34 @@ static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, boo
  */
 static void __update_context_time(struct perf_event_context *ctx, bool adv)
 {
-	u64 now = perf_clock();
+	lockdep_assert_held(&ctx->lock);
+
+	update_perf_time_ctx(&ctx->time, perf_clock(), adv);
+}
 
+static void __update_context_guest_time(struct perf_event_context *ctx, bool adv)
+{
 	lockdep_assert_held(&ctx->lock);
 
-	update_perf_time_ctx(&ctx->time, now, adv);
+	/* must be called after __update_context_time(); */
+	update_perf_time_ctx(&ctx->timeguest, ctx->time.stamp, adv);
 }
 
 static void update_context_time(struct perf_event_context *ctx)
 {
 	__update_context_time(ctx, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_context_guest_time(ctx, true);
+}
+
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest)
+{
+	if (event->attr.exclude_guest)
+		return time->time - timeguest->time;
+	else
+		return time->time;
 }
 
 static u64 perf_event_time(struct perf_event *event)
@@ -1510,7 +1559,26 @@ static u64 perf_event_time(struct perf_event *event)
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time(event);
 
-	return ctx->time.time;
+	return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
+}
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now)
+{
+	/*
+	 * The exclude_guest event time should be calculated from
+	 * the ctx time -  the guest time.
+	 * The ctx time is now + READ_ONCE(time->offset).
+	 * The guest time is now + READ_ONCE(timeguest->offset).
+	 * So the exclude_guest time is
+	 * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
+	 */
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))
+		return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);
+	else
+		return now + READ_ONCE(time->offset);
 }
 
 static u64 perf_event_time_now(struct perf_event *event, u64 now)
@@ -1524,10 +1592,9 @@ static u64 perf_event_time_now(struct perf_event *event, u64 now)
 		return perf_cgroup_event_time_now(event, now);
 
 	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME))
-		return ctx->time.time;
+		return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
 
-	now += READ_ONCE(ctx->time.offset);
-	return now;
+	return __perf_event_time_ctx_now(event, &ctx->time, &ctx->timeguest, now);
 }
 
 static enum event_type_t get_event_type(struct perf_event *event)
@@ -3334,9 +3401,15 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		/* vPMU should not stop time */
+		stop = !(event_type & EVENT_GUEST) &&
+		       ctx == &cpuctx->ctx;
+
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3354,7 +3427,18 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_timestamp(cpuctx, true);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3853,10 +3937,15 @@ static inline void group_update_userpage(struct perf_event *group_event)
 		event_update_userpage(event);
 }
 
+struct merge_sched_data {
+	int can_add_hw;
+	enum event_type_t event_type;
+};
+
 static int merge_sched_in(struct perf_event *event, void *data)
 {
 	struct perf_event_context *ctx = event->ctx;
-	int *can_add_hw = data;
+	struct merge_sched_data *msd = data;
 
 	if (event->state <= PERF_EVENT_STATE_OFF)
 		return 0;
@@ -3864,13 +3953,22 @@ static int merge_sched_in(struct perf_event *event, void *data)
 	if (!event_filter_match(event))
 		return 0;
 

kernel/events/core.c static int merge_sched_in(struct perf_event *event, void *data) · line 3964

Quoted 10 lines
kernel/events/core.c +9 -1
-	if (group_can_go_on(event, *can_add_hw)) {
+	/*
+	 * Don't schedule in any exclude_guest events of PMU with
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+	 */
+	if (__this_cpu_read(perf_in_guest) && event->attr.exclude_guest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+	    !(msd->event_type & EVENT_GUEST))
+		return 0;
+

It is possible for event groups to have a mix of software and core PMU events. If the group leader is a software event, event->pmu will point to the software PMU but event->pmu_ctx->pmu will point to the core PMU. When perf_in_guest is true for a CPU and the group leader is passed to merge_sched_in(), the condition above fails as the software PMU does not have PERF_PMU_CAP_PASSTHROUGH_VPMU capability. This can lead to group_sched_in() getting called later where all the sibling events, which includes core PMU events that are not supposed to be scheduled in, to be brought in. So event->pmu_ctx->pmu->capabilities needs to be looked at instead.

Quoted 85 lines
kernel/events/core.c +31 -8
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 	}
 
 	if (event->state == PERF_EVENT_STATE_INACTIVE) {
-		*can_add_hw = 0;
+		msd->can_add_hw = 0;
 		if (event->attr.pinned) {
 			perf_cgroup_event_disable(event, ctx);
 			perf_event_set_state(event, PERF_EVENT_STATE_ERROR);
@@ -3889,11 +3987,15 @@ static int merge_sched_in(struct perf_event *event, void *data)
 
 static void pmu_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				struct pmu *pmu)
+				struct pmu *pmu,
+				enum event_type_t event_type)
 {
-	int can_add_hw = 1;
+	struct merge_sched_data msd = {
+		.can_add_hw = 1,
+		.event_type = event_type,
+	};
 	visit_groups_merge(ctx, groups, smp_processor_id(), pmu,
-			   merge_sched_in, &can_add_hw);
+			   merge_sched_in, &msd);
 }
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
@@ -3905,14 +4007,14 @@ static void ctx_groups_sched_in(struct perf_event_context *ctx,
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
-		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
+		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu, event_type);
 	}
 }
 
 static void __pmu_ctx_sched_in(struct perf_event_context *ctx,
 			       struct pmu *pmu)
 {
-	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu, 0);
 }
 
 static void
@@ -3927,9 +4029,11 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
-		perf_cgroup_set_timestamp(cpuctx);
+		perf_cgroup_set_timestamp(cpuctx, false);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3945,7 +4049,23 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule in all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+
+		/*
+		 * Update ctx time to set the new start time for
+		 * the exclude_guest events.
+		 */
+		update_context_time(ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	/*
 	 * First go through the list and put on any pinned groups
#25 lore raw

Re: [RFC PATCH v3 09/58] perf: Add a EVENT_GUEST flag

Quoted 346 lines · Sandipan Das
Quoted 333 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Current perf doesn't explicitly schedule out all exclude_guest events while the guest is running. There is no problem with the current emulated vPMU. Because perf owns all the PMU counters. It can mask the counter which is assigned to an exclude_guest event when a guest is running (Intel way), or set the corresponding HOSTONLY bit in evsentsel (AMD way). The counter doesn't count when a guest is running.

However, either way doesn't work with the introduced passthrough vPMU. A guest owns all the PMU counters when it's running. The host should not mask any counters. The counter may be used by the guest. The evsentsel may be overwritten.

Perf should explicitly schedule out all exclude_guest events to release the PMU resources when entering a guest, and resume the counting when exiting the guest.

It's possible that an exclude_guest event is created when a guest is running. The new event should not be scheduled in as well.

The ctx time is shared among different PMUs. The time cannot be stopped when a guest is running. It is required to calculate the time for events from other PMUs, e.g., uncore events. Add timeguest to track the guest run time. For an exclude_guest event, the elapsed time equals the ctx time - guest time.
Cgroup has dedicated times. Use the same method to deduct the guest time from the cgroup time as well.

---
 include/linux/perf_event.h |   6 ++
 kernel/events/core.c       | 178 +++++++++++++++++++++++++++++++------
 2 files changed, 155 insertions(+), 29 deletions(-)
include/linux/perf_event.h +6 -0
index e22cdb6486e6..81a5f8399cb8 100644
@@ -952,6 +952,11 @@ struct perf_event_context {
 	 */
 	struct perf_time_ctx		time;
 
+	/*
+	 * Context clock, runs when in the guest mode.
+	 */
+	struct perf_time_ctx		timeguest;
+
 	/*
 	 * These fields let us detect when two contexts have both
 	 * been cloned (inherited) from a common ancestor.
@@ -1044,6 +1049,7 @@ struct bpf_perf_event_data_kern {
  */
 struct perf_cgroup_info {
 	struct perf_time_ctx		time;
+	struct perf_time_ctx		timeguest;
 	int				active;
 };
 
kernel/events/core.c +109 -20
index c25e2bf27001..57648736e43e 100644
@@ -376,7 +376,8 @@ enum event_type_t {
 	/* see ctx_resched() for details */
 	EVENT_CPU = 0x8,
 	EVENT_CGROUP = 0x10,
-	EVENT_FLAGS = EVENT_CGROUP,
+	EVENT_GUEST = 0x20,
+	EVENT_FLAGS = EVENT_CGROUP | EVENT_GUEST,
 	EVENT_ALL = EVENT_FLEXIBLE | EVENT_PINNED,
 };
 
@@ -407,6 +408,7 @@ static atomic_t nr_include_guest_events __read_mostly;
 
 static atomic_t nr_mediated_pmu_vms;
 static DEFINE_MUTEX(perf_mediated_pmu_mutex);
+static DEFINE_PER_CPU(bool, perf_in_guest);
 
 /* !exclude_guest event of PMU with PERF_PMU_CAP_PASSTHROUGH_VPMU */
 static inline bool is_include_guest_event(struct perf_event *event)
@@ -706,6 +708,10 @@ static bool perf_skip_pmu_ctx(struct perf_event_pmu_context *pmu_ctx,
 	if ((event_type & EVENT_CGROUP) && !pmu_ctx->nr_cgroups)
 		return true;
 
+	if ((event_type & EVENT_GUEST) &&
+	    !(pmu_ctx->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU))
+		return true;
+
 	return false;
 }
 
@@ -770,12 +776,21 @@ static inline int is_cgroup_event(struct perf_event *event)
 	return event->cgrp != NULL;
 }
 
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest);
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now);
+
 static inline u64 perf_cgroup_event_time(struct perf_event *event)
 {
 	struct perf_cgroup_info *t;
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
-	return t->time.time;
+	return __perf_event_time_ctx(event, &t->time, &t->timeguest);
 }
 
 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
@@ -784,9 +799,9 @@ static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64 now)
 
 	t = per_cpu_ptr(event->cgrp->info, event->cpu);
 	if (!__load_acquire(&t->active))
-		return t->time.time;
-	now += READ_ONCE(t->time.offset);
-	return now;
+		return __perf_event_time_ctx(event, &t->time, &t->timeguest);
+
+	return __perf_event_time_ctx_now(event, &t->time, &t->timeguest, now);
 }
 
 static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, bool adv);
@@ -796,6 +811,18 @@ static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 now, bo
 	update_perf_time_ctx(&info->time, now, adv);
 }
 
+static inline void __update_cgrp_guest_time(struct perf_cgroup_info *info, u64 now, bool adv)
+{
+	update_perf_time_ctx(&info->timeguest, now, adv);
+}
+
+static inline void update_cgrp_time(struct perf_cgroup_info *info, u64 now)
+{
+	__update_cgrp_time(info, now, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_cgrp_guest_time(info, now, true);
+}
+
 static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx, bool final)
 {
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -809,7 +836,7 @@ static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *cpuctx,
 			cgrp = container_of(css, struct perf_cgroup, css);
 			info = this_cpu_ptr(cgrp->info);
 
-			__update_cgrp_time(info, now, true);
+			update_cgrp_time(info, now);
 			if (final)
 				__store_release(&info->active, 0);
 		}
@@ -832,11 +859,11 @@ static inline void update_cgrp_time_from_event(struct perf_event *event)
 	 * Do not update time when cgroup is not active
 	 */
 	if (info->active)
-		__update_cgrp_time(info, perf_clock(), true);
+		update_cgrp_time(info, perf_clock());
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 	struct perf_event_context *ctx = &cpuctx->ctx;
 	struct perf_cgroup *cgrp = cpuctx->cgrp;
@@ -856,8 +883,12 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
 	for (css = &cgrp->css; css; css = css->parent) {
 		cgrp = container_of(css, struct perf_cgroup, css);
 		info = this_cpu_ptr(cgrp->info);
-		__update_cgrp_time(info, ctx->time.stamp, false);
-		__store_release(&info->active, 1);
+		if (guest) {
+			__update_cgrp_guest_time(info, ctx->time.stamp, false);
+		} else {
+			__update_cgrp_time(info, ctx->time.stamp, false);
+			__store_release(&info->active, 1);
+		}
 	}
 }
 
@@ -1061,7 +1092,7 @@ static inline int perf_cgroup_connect(pid_t pid, struct perf_event *event,
 }
 
 static inline void
-perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx)
+perf_cgroup_set_timestamp(struct perf_cpu_context *cpuctx, bool guest)
 {
 }
 
@@ -1488,16 +1519,34 @@ static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 now, boo
  */
 static void __update_context_time(struct perf_event_context *ctx, bool adv)
 {
-	u64 now = perf_clock();
+	lockdep_assert_held(&ctx->lock);
+
+	update_perf_time_ctx(&ctx->time, perf_clock(), adv);
+}
 
+static void __update_context_guest_time(struct perf_event_context *ctx, bool adv)
+{
 	lockdep_assert_held(&ctx->lock);
 
-	update_perf_time_ctx(&ctx->time, now, adv);
+	/* must be called after __update_context_time(); */
+	update_perf_time_ctx(&ctx->timeguest, ctx->time.stamp, adv);
 }
 
 static void update_context_time(struct perf_event_context *ctx)
 {
 	__update_context_time(ctx, true);
+	if (__this_cpu_read(perf_in_guest))
+		__update_context_guest_time(ctx, true);
+}
+
+static inline u64 __perf_event_time_ctx(struct perf_event *event,
+					struct perf_time_ctx *time,
+					struct perf_time_ctx *timeguest)
+{
+	if (event->attr.exclude_guest)
+		return time->time - timeguest->time;
+	else
+		return time->time;
 }
 
 static u64 perf_event_time(struct perf_event *event)
@@ -1510,7 +1559,26 @@ static u64 perf_event_time(struct perf_event *event)
 	if (is_cgroup_event(event))
 		return perf_cgroup_event_time(event);
 
-	return ctx->time.time;
+	return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
+}
+
+static inline u64 __perf_event_time_ctx_now(struct perf_event *event,
+					    struct perf_time_ctx *time,
+					    struct perf_time_ctx *timeguest,
+					    u64 now)
+{
+	/*
+	 * The exclude_guest event time should be calculated from
+	 * the ctx time -  the guest time.
+	 * The ctx time is now + READ_ONCE(time->offset).
+	 * The guest time is now + READ_ONCE(timeguest->offset).
+	 * So the exclude_guest time is
+	 * READ_ONCE(time->offset) - READ_ONCE(timeguest->offset).
+	 */
+	if (event->attr.exclude_guest && __this_cpu_read(perf_in_guest))
+		return READ_ONCE(time->offset) - READ_ONCE(timeguest->offset);
+	else
+		return now + READ_ONCE(time->offset);
 }
 
 static u64 perf_event_time_now(struct perf_event *event, u64 now)
@@ -1524,10 +1592,9 @@ static u64 perf_event_time_now(struct perf_event *event, u64 now)
 		return perf_cgroup_event_time_now(event, now);
 
 	if (!(__load_acquire(&ctx->is_active) & EVENT_TIME))
-		return ctx->time.time;
+		return __perf_event_time_ctx(event, &ctx->time, &ctx->timeguest);
 
-	now += READ_ONCE(ctx->time.offset);
-	return now;
+	return __perf_event_time_ctx_now(event, &ctx->time, &ctx->timeguest, now);
 }
 
 static enum event_type_t get_event_type(struct perf_event *event)
@@ -3334,9 +3401,15 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 	 * would only update time for the pinned events.
 	 */
 	if (is_active & EVENT_TIME) {
+		bool stop;
+
+		/* vPMU should not stop time */
+		stop = !(event_type & EVENT_GUEST) &&
+		       ctx == &cpuctx->ctx;
+
 		/* update (and stop) ctx time */
 		update_context_time(ctx);
-		update_cgrp_time_from_cpuctx(cpuctx, ctx == &cpuctx->ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, stop);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3354,7 +3427,18 @@ ctx_sched_out(struct perf_event_context *ctx, enum event_type_t event_type)
 			cpuctx->task_ctx = NULL;
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule out all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+		__update_context_guest_time(ctx, false);
+		perf_cgroup_set_timestamp(cpuctx, true);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
@@ -3853,10 +3937,15 @@ static inline void group_update_userpage(struct perf_event *group_event)
 		event_update_userpage(event);
 }
 
+struct merge_sched_data {
+	int can_add_hw;
+	enum event_type_t event_type;
+};
+
 static int merge_sched_in(struct perf_event *event, void *data)
 {
 	struct perf_event_context *ctx = event->ctx;
-	int *can_add_hw = data;
+	struct merge_sched_data *msd = data;
 
 	if (event->state <= PERF_EVENT_STATE_OFF)
 		return 0;
@@ -3864,13 +3953,22 @@ static int merge_sched_in(struct perf_event *event, void *data)
 	if (!event_filter_match(event))
 		return 0;
 

kernel/events/core.c static int merge_sched_in(struct perf_event *event, void *data) · line 3964

Quoted 10 lines
kernel/events/core.c +9 -1
-	if (group_can_go_on(event, *can_add_hw)) {
+	/*
+	 * Don't schedule in any exclude_guest events of PMU with
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU, while a guest is running.
+	 */
+	if (__this_cpu_read(perf_in_guest) && event->attr.exclude_guest &&
+	    event->pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU &&
+	    !(msd->event_type & EVENT_GUEST))
+		return 0;
+

It is possible for event groups to have a mix of software and core PMU events.

Quoted 8 lines

If the group leader is a software event, event->pmu will point to the software PMU but event->pmu_ctx->pmu will point to the core PMU. When perf_in_guest is true for a CPU and the group leader is passed to merge_sched_in(), the condition above fails as the software PMU does not have PERF_PMU_CAP_PASSTHROUGH_VPMU capability. This can lead to group_sched_in() getting called later where all the sibling events, which includes core PMU events that are not supposed to be scheduled in, to be brought in. So event->pmu_ctx->pmu->capabilities needs to be looked at instead.

Right, Thanks.
I will fix it in the next version.

Thanks,
Kan

Quoted 88 lines
Quoted 86 lines
kernel/events/core.c +31 -8
+	if (group_can_go_on(event, msd->can_add_hw)) {
 		if (!group_sched_in(event, ctx))
 			list_add_tail(&event->active_list, get_event_list(event));
 	}
 
 	if (event->state == PERF_EVENT_STATE_INACTIVE) {
-		*can_add_hw = 0;
+		msd->can_add_hw = 0;
 		if (event->attr.pinned) {
 			perf_cgroup_event_disable(event, ctx);
 			perf_event_set_state(event, PERF_EVENT_STATE_ERROR);
@@ -3889,11 +3987,15 @@ static int merge_sched_in(struct perf_event *event, void *data)
 
 static void pmu_groups_sched_in(struct perf_event_context *ctx,
 				struct perf_event_groups *groups,
-				struct pmu *pmu)
+				struct pmu *pmu,
+				enum event_type_t event_type)
 {
-	int can_add_hw = 1;
+	struct merge_sched_data msd = {
+		.can_add_hw = 1,
+		.event_type = event_type,
+	};
 	visit_groups_merge(ctx, groups, smp_processor_id(), pmu,
-			   merge_sched_in, &can_add_hw);
+			   merge_sched_in, &msd);
 }
 
 static void ctx_groups_sched_in(struct perf_event_context *ctx,
@@ -3905,14 +4007,14 @@ static void ctx_groups_sched_in(struct perf_event_context *ctx,
 	list_for_each_entry(pmu_ctx, &ctx->pmu_ctx_list, pmu_ctx_entry) {
 		if (perf_skip_pmu_ctx(pmu_ctx, event_type))
 			continue;
-		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu);
+		pmu_groups_sched_in(ctx, groups, pmu_ctx->pmu, event_type);
 	}
 }
 
 static void __pmu_ctx_sched_in(struct perf_event_context *ctx,
 			       struct pmu *pmu)
 {
-	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu);
+	pmu_groups_sched_in(ctx, &ctx->flexible_groups, pmu, 0);
 }
 
 static void
@@ -3927,9 +4029,11 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 		return;
 
 	if (!(is_active & EVENT_TIME)) {
+		/* EVENT_TIME should be active while the guest runs */
+		WARN_ON_ONCE(event_type & EVENT_GUEST);
 		/* start ctx time */
 		__update_context_time(ctx, false);
-		perf_cgroup_set_timestamp(cpuctx);
+		perf_cgroup_set_timestamp(cpuctx, false);
 		/*
 		 * CPU-release for the below ->is_active store,
 		 * see __load_acquire() in perf_event_time_now()
@@ -3945,7 +4049,23 @@ ctx_sched_in(struct perf_event_context *ctx, enum event_type_t event_type)
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	is_active ^= ctx->is_active; /* changed bits */
+	if (event_type & EVENT_GUEST) {
+		/*
+		 * Schedule in all !exclude_guest events of PMU
+		 * with PERF_PMU_CAP_PASSTHROUGH_VPMU.
+		 */
+		is_active = EVENT_ALL;
+
+		/*
+		 * Update ctx time to set the new start time for
+		 * the exclude_guest events.
+		 */
+		update_context_time(ctx);
+		update_cgrp_time_from_cpuctx(cpuctx, false);
+		barrier();
+	} else {
+		is_active ^= ctx->is_active; /* changed bits */
+	}
 
 	/*
 	 * First go through the list and put on any pinned groups
#26 lore raw

RFC PATCH v3 10/58 perf: Add generic exclude_guest support

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Only KVM knows the exact time when a guest is entering/exiting. Expose two interfaces to KVM to switch the ownership of the PMU resources.

---
 include/linux/perf_event.h |  4 +++
 kernel/events/core.c       | 54 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
include/linux/perf_event.h +4 -0
index 81a5f8399cb8..75773f9890cc 100644
@@ -1738,6 +1738,8 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
+void perf_guest_enter(void);
+void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
 perf_aux_output_begin(struct perf_output_handle *handle,
@@ -1831,6 +1833,8 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)			{ }
+static inline void perf_guest_enter(void)			{ }
+static inline void perf_guest_exit(void)			{ }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL)
kernel/events/core.c +54 -0

60 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
maintainer
#27 lore raw

Re: [RFC PATCH v3 10/58] perf: Add generic exclude_guest support

Quoted 9 lines · Mingwei Zhang
kernel/events/core.c +9 -0
+void perf_guest_exit(void)
+{
+	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+	lockdep_assert_irqs_disabled();
+
+	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))

kernel/events/core.c void perf_put_mediated_pmu(void) · line 5990

Quoted 10 lines
kernel/events/core.c +10 -0
+		goto unlock;
+
+	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
+	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
+	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
+	if (cpuctx->task_ctx) {
+		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
+		ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
+		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
+	}

Does this not violate the scheduling order of events? AFAICT this will do:

cpu pinned
cpu flexible
task pinned
task flexible

as opposed to:

cpu pinned
task pinned
cpu flexible
task flexible

We have the perf_event_sched_in() helper for this.

Quoted 6 lines
kernel/events/core.c +6 -0
+
+	__this_cpu_write(perf_in_guest, false);
+unlock:
+	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}
+EXPORT_SYMBOL_GPL(perf_guest_exit);
#28 lore raw

Re: [RFC PATCH v3 10/58] perf: Add generic exclude_guest support

Quoted 29 lines · Peter Zijlstra
Quoted 9 lines · Mingwei Zhang
kernel/events/core.c +9 -0
+void perf_guest_exit(void)
+{
+	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
+
+	lockdep_assert_irqs_disabled();
+
+	perf_ctx_lock(cpuctx, cpuctx->task_ctx);
+
+	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))

kernel/events/core.c void perf_put_mediated_pmu(void) · line 5990

Quoted 10 lines
kernel/events/core.c +10 -0
+		goto unlock;
+
+	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
+	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
+	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
+	if (cpuctx->task_ctx) {
+		perf_ctx_disable(cpuctx->task_ctx, EVENT_GUEST);
+		ctx_sched_in(cpuctx->task_ctx, EVENT_GUEST);
+		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
+	}

Does this not violate the scheduling order of events? AFAICT this will do:

cpu pinned
cpu flexible
task pinned
task flexible

Quoted 8 lines

as opposed to:

cpu pinned
task pinned
cpu flexible
task flexible

We have the perf_event_sched_in() helper for this.

Yes, we can avoid the sched_in() with EVENT_GUEST flag, then invoke the perf_event_sched_in() helper to do the real schedule. I will do more tests to double check.

Thanks,
Kan

Quoted 8 lines
Quoted 7 lines
kernel/events/core.c +6 -0
+
+	__this_cpu_write(perf_in_guest, false);
+unlock:
+	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
+}
+EXPORT_SYMBOL_GPL(perf_guest_exit);
#29 lore raw

RFC PATCH v3 11/58 x86/irq: Factor out common code for installing kvm irq handler

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

KVM will register irq handler for POSTED_INTR_WAKEUP_VECTOR and KVM_GUEST_PMI_VECTOR, the existing kvm_set_posted_intr_wakeup_handler() is renamed to x86_set_kvm_irq_handler(), and vector input parameter is used to distinguish POSTED_INTR_WARKUP_VECTOR and KVM_GUEST_PMI_VECTOR.

Caller should call x86_set_kvm_irq_handler() once to register a non-dummy handler for each vector. If caller register one handler for a vector, later the caller register the same or different non-dummy handler again, the second call will output warn message.

---
 arch/x86/include/asm/irq.h |  2 +-
 arch/x86/kernel/irq.c      | 18 ++++++++++++------
 arch/x86/kvm/vmx/vmx.c     |  4 ++--
 3 files changed, 15 insertions(+), 9 deletions(-)
arch/x86/include/asm/irq.h +1 -1
index 194dfff84cb1..050a247b69b4 100644
@@ -30,7 +30,7 @@ struct irq_desc;
 extern void fixup_irqs(void);
 
 #if IS_ENABLED(CONFIG_KVM)
-extern void kvm_set_posted_intr_wakeup_handler(void (*handler)(void));
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void));
 #endif
 
 extern void (*x86_platform_ipi_callback)(void);
arch/x86/kernel/irq.c +12 -6
index 385e3a5fc304..18cd418fe106 100644
@@ -312,16 +312,22 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi)
 static void dummy_handler(void) {}
 static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
 
-void kvm_set_posted_intr_wakeup_handler(void (*handler)(void))
+void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
-	if (handler)
+	if (!handler)
+		handler = dummy_handler;
+
+	if (vector == POSTED_INTR_WAKEUP_VECTOR &&
+	    (handler == dummy_handler ||
+	     kvm_posted_intr_wakeup_handler == dummy_handler))
 		kvm_posted_intr_wakeup_handler = handler;
-	else {
-		kvm_posted_intr_wakeup_handler = dummy_handler;
+	else
+		WARN_ON_ONCE(1);
+
+	if (handler == dummy_handler)
 		synchronize_rcu();
-	}
 }
-EXPORT_SYMBOL_GPL(kvm_set_posted_intr_wakeup_handler);
+EXPORT_SYMBOL_GPL(x86_set_kvm_irq_handler);
 
 /*
  * Handler for POSTED_INTERRUPT_VECTOR.
arch/x86/kvm/vmx/vmx.c +2 -2
index b3c83c06f826..ad465881b043 100644
@@ -8292,7 +8292,7 @@ void vmx_migrate_timers(struct kvm_vcpu *vcpu)
 
 void vmx_hardware_unsetup(void)
 {
-	kvm_set_posted_intr_wakeup_handler(NULL);
+	x86_set_kvm_irq_handler(POSTED_INTR_WAKEUP_VECTOR, NULL);
 
 	if (nested)
 		nested_vmx_hardware_unsetup();
@@ -8602,7 +8602,7 @@ __init int vmx_hardware_setup(void)
 	if (r && nested)
 		nested_vmx_hardware_unsetup();
 
-	kvm_set_posted_intr_wakeup_handler(pi_wakeup_handler);
+	x86_set_kvm_irq_handler(POSTED_INTR_WAKEUP_VECTOR, pi_wakeup_handler);
 
 	return r;
 }
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#30 lore raw

RFC PATCH v3 12/58 perf: core/x86: Register a new vector for KVM GUEST PMI

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Create a new vector in the host IDT for kvm guest PMI handling within mediated passthrough vPMU. In addition, guest PMI handler registration is added into x86_set_kvm_irq_handler().

This is the preparation work to support mediated passthrough vPMU to handle kvm guest PMIs without interference from PMI handler of the host PMU.

---
 arch/x86/include/asm/hardirq.h                |  1 +
 arch/x86/include/asm/idtentry.h               |  1 +
 arch/x86/include/asm/irq_vectors.h            |  5 ++++-
 arch/x86/kernel/idt.c                         |  1 +
 arch/x86/kernel/irq.c                         | 21 +++++++++++++++++++
 .../beauty/arch/x86/include/asm/irq_vectors.h |  5 ++++-
 6 files changed, 32 insertions(+), 2 deletions(-)
arch/x86/include/asm/hardirq.h +1 -0
index c67fa6ad098a..42a396763c8d 100644
@@ -19,6 +19,7 @@ typedef struct {
 	unsigned int kvm_posted_intr_ipis;
 	unsigned int kvm_posted_intr_wakeup_ipis;
 	unsigned int kvm_posted_intr_nested_ipis;
+	unsigned int kvm_guest_pmis;
 #endif
 	unsigned int x86_platform_ipis;	/* arch dependent */
 	unsigned int apic_perf_irqs;
arch/x86/include/asm/idtentry.h +1 -0
index d4f24499b256..7b1e3e542b1d 100644
@@ -745,6 +745,7 @@ DECLARE_IDTENTRY_SYSVEC(IRQ_WORK_VECTOR,		sysvec_irq_work);
 DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_VECTOR,		sysvec_kvm_posted_intr_ipi);
 DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_WAKEUP_VECTOR,	sysvec_kvm_posted_intr_wakeup_ipi);
 DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR,	sysvec_kvm_posted_intr_nested_ipi);
+DECLARE_IDTENTRY_SYSVEC(KVM_GUEST_PMI_VECTOR,	        sysvec_kvm_guest_pmi_handler);
 #else
 # define fred_sysvec_kvm_posted_intr_ipi		NULL
 # define fred_sysvec_kvm_posted_intr_wakeup_ipi		NULL
arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..ada270e6f5cb 100644
@@ -77,7 +77,10 @@
  */
 #define IRQ_WORK_VECTOR			0xf6
 
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR		0xf5
+#endif
+
 #define DEFERRED_ERROR_VECTOR		0xf4
 
 /* Vector on which hypervisor callbacks will be delivered */
arch/x86/kernel/idt.c +1 -0
index f445bec516a0..0bec4c7e2308 100644
@@ -157,6 +157,7 @@ static const __initconst struct idt_data apic_idts[] = {
 	INTG(POSTED_INTR_VECTOR,		asm_sysvec_kvm_posted_intr_ipi),
 	INTG(POSTED_INTR_WAKEUP_VECTOR,		asm_sysvec_kvm_posted_intr_wakeup_ipi),
 	INTG(POSTED_INTR_NESTED_VECTOR,		asm_sysvec_kvm_posted_intr_nested_ipi),
+	INTG(KVM_GUEST_PMI_VECTOR,		asm_sysvec_kvm_guest_pmi_handler),
 # endif
 # ifdef CONFIG_IRQ_WORK
 	INTG(IRQ_WORK_VECTOR,			asm_sysvec_irq_work),
arch/x86/kernel/irq.c +21 -0
index 18cd418fe106..b29714e23fc4 100644
@@ -183,6 +183,12 @@ int arch_show_interrupts(struct seq_file *p, int prec)
 		seq_printf(p, "%10u ",
 			   irq_stats(j)->kvm_posted_intr_wakeup_ipis);
 	seq_puts(p, "  Posted-interrupt wakeup event\n");
+
+	seq_printf(p, "%*s: ", prec, "VPMU");
+	for_each_online_cpu(j)
+		seq_printf(p, "%10u ",
+			   irq_stats(j)->kvm_guest_pmis);
+	seq_puts(p, " KVM GUEST PMI\n");
 #endif
 #ifdef CONFIG_X86_POSTED_MSI
 	seq_printf(p, "%*s: ", prec, "PMN");
@@ -311,6 +317,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi)
 #if IS_ENABLED(CONFIG_KVM)
 static void dummy_handler(void) {}
 static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
+static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
 
 void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 {
@@ -321,6 +328,10 @@ void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
 	    (handler == dummy_handler ||
 	     kvm_posted_intr_wakeup_handler == dummy_handler))
 		kvm_posted_intr_wakeup_handler = handler;
+	else if (vector == KVM_GUEST_PMI_VECTOR &&
+		 (handler == dummy_handler ||
+		  kvm_guest_pmi_handler == dummy_handler))
+		kvm_guest_pmi_handler = handler;
 	else
 		WARN_ON_ONCE(1);
 
@@ -356,6 +367,16 @@ DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_kvm_posted_intr_nested_ipi)
 	apic_eoi();
 	inc_irq_stat(kvm_posted_intr_nested_ipis);
 }
+
+/*
+ * Handler for KVM_GUEST_PMI_VECTOR.
+ */
+DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_guest_pmi_handler)
+{
+	apic_eoi();
+	inc_irq_stat(kvm_guest_pmis);
+	kvm_guest_pmi_handler();
+}
 #endif
 
 #ifdef CONFIG_X86_POSTED_MSI
tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..670dcee46631 100644
@@ -77,7 +77,10 @@
  */
 #define IRQ_WORK_VECTOR			0xf6
 
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR           0xf5
+#endif
+
 #define DEFERRED_ERROR_VECTOR		0xf4
 
 /* Vector on which hypervisor callbacks will be delivered */
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#31 lore raw

Re: [RFC PATCH v3 12/58] perf: core/x86: Register a new vector for KVM GUEST PMI

Hello,

Quoted 72 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

Create a new vector in the host IDT for kvm guest PMI handling within mediated passthrough vPMU. In addition, guest PMI handler registration is added into x86_set_kvm_irq_handler().

This is the preparation work to support mediated passthrough vPMU to handle kvm guest PMIs without interference from PMI handler of the host PMU.

---
  arch/x86/include/asm/hardirq.h                |  1 +
  arch/x86/include/asm/idtentry.h               |  1 +
  arch/x86/include/asm/irq_vectors.h            |  5 ++++-
  arch/x86/kernel/idt.c                         |  1 +
  arch/x86/kernel/irq.c                         | 21 +++++++++++++++++++
  .../beauty/arch/x86/include/asm/irq_vectors.h |  5 ++++-
  6 files changed, 32 insertions(+), 2 deletions(-)
arch/x86/include/asm/hardirq.h +1 -0
index c67fa6ad098a..42a396763c8d 100644
@@ -19,6 +19,7 @@ typedef struct {
  	unsigned int kvm_posted_intr_ipis;
  	unsigned int kvm_posted_intr_wakeup_ipis;
  	unsigned int kvm_posted_intr_nested_ipis;
+	unsigned int kvm_guest_pmis;
  #endif
  	unsigned int x86_platform_ipis;	/* arch dependent */
  	unsigned int apic_perf_irqs;
arch/x86/include/asm/idtentry.h +1 -0
index d4f24499b256..7b1e3e542b1d 100644
@@ -745,6 +745,7 @@ DECLARE_IDTENTRY_SYSVEC(IRQ_WORK_VECTOR,		 
 sysvec_irq_work);
  DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_VECTOR,		sysvec_kvm_posted_intr_ipi);
  DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_WAKEUP_VECTOR,	 
 sysvec_kvm_posted_intr_wakeup_ipi);
  DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR,	 
 sysvec_kvm_posted_intr_nested_ipi);
+DECLARE_IDTENTRY_SYSVEC(KVM_GUEST_PMI_VECTOR,	         

sysvec_kvm_guest_pmi_handler);
#else

  # define fred_sysvec_kvm_posted_intr_ipi		NULL
  # define fred_sysvec_kvm_posted_intr_wakeup_ipi		NULL
arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..ada270e6f5cb 100644
@@ -77,7 +77,10 @@
   */
  #define IRQ_WORK_VECTOR			0xf6
 
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR		0xf5
+#endif
+
  #define DEFERRED_ERROR_VECTOR		0xf4
 
  /* Vector on which hypervisor callbacks will be delivered */
arch/x86/kernel/idt.c +0 -0
index f445bec516a0..0bec4c7e2308 100644

arch/x86/kernel/idt.c static const __initconst struct idt_data apic_idts[] · line 161

Quoted 5 lines
arch/x86/kernel/idt.c +1 -0
@@ -157,6 +157,7 @@ static const __initconst struct idt_data apic_idts[]  
 = {
  	INTG(POSTED_INTR_VECTOR,		asm_sysvec_kvm_posted_intr_ipi),
  	INTG(POSTED_INTR_WAKEUP_VECTOR,		asm_sysvec_kvm_posted_intr_wakeup_ipi),
  	INTG(POSTED_INTR_NESTED_VECTOR,		asm_sysvec_kvm_posted_intr_nested_ipi),
+	INTG(KVM_GUEST_PMI_VECTOR,		asm_sysvec_kvm_guest_pmi_handler),

There is a subtle inconsistency in that this code is guarded on CONFIG_HAVE_KVM when the #define KVM_GUEST_PMI_VECTOR is guarded on CONFIG_KVM. Beats me why there are two different flags that are so similar, but it is possible they have different values leading to compilation errors.

I happened to hit this compilation error because make defconfig will define CONFIG_HAVE_KVM but not CONFIG_KVM. CONFIG_KVM is the more strict of the two because it depends on CONFIG_HAVE_KVM.

This code should also be guarded on CONFIG_KVM.

Quoted 75 lines
arch/x86/kernel/idt.c +0 -0
  # endif
  # ifdef CONFIG_IRQ_WORK

INTG(IRQ_WORK_VECTOR, asm_sysvec_irq_work),

arch/x86/kernel/irq.c +6 -0
index 18cd418fe106..b29714e23fc4 100644
@@ -183,6 +183,12 @@ int arch_show_interrupts(struct seq_file *p, int  
 prec)
  		seq_printf(p, "%10u ",
  			   irq_stats(j)->kvm_posted_intr_wakeup_ipis);
  	seq_puts(p, "  Posted-interrupt wakeup event\n");
+
+	seq_printf(p, "%*s: ", prec, "VPMU");
+	for_each_online_cpu(j)
+		seq_printf(p, "%10u ",
+			   irq_stats(j)->kvm_guest_pmis);
+	seq_puts(p, " KVM GUEST PMI\n");
  #endif
  #ifdef CONFIG_X86_POSTED_MSI
  	seq_printf(p, "%*s: ", prec, "PMN");
arch/x86/kernel/irq.c +15 -0
@@ -311,6 +317,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi)
  #if IS_ENABLED(CONFIG_KVM)
  static void dummy_handler(void) {}
  static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
+static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
 
  void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
  {
@@ -321,6 +328,10 @@ void x86_set_kvm_irq_handler(u8 vector, void  
 (*handler)(void))
  	    (handler == dummy_handler ||
  	     kvm_posted_intr_wakeup_handler == dummy_handler))
  		kvm_posted_intr_wakeup_handler = handler;
+	else if (vector == KVM_GUEST_PMI_VECTOR &&
+		 (handler == dummy_handler ||
+		  kvm_guest_pmi_handler == dummy_handler))
+		kvm_guest_pmi_handler = handler;
  	else
  		WARN_ON_ONCE(1);
@@ -356,6 +367,16 @@  
 DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_kvm_posted_intr_nested_ipi)
  	apic_eoi();
  	inc_irq_stat(kvm_posted_intr_nested_ipis);
  }
+
+/*
+ * Handler for KVM_GUEST_PMI_VECTOR.
+ */
+DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_guest_pmi_handler)
+{
+	apic_eoi();
+	inc_irq_stat(kvm_guest_pmis);
+	kvm_guest_pmi_handler();
+}
  #endif
 
  #ifdef CONFIG_X86_POSTED_MSI
tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..670dcee46631 100644
@@ -77,7 +77,10 @@
   */
  #define IRQ_WORK_VECTOR			0xf6
 
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR           0xf5
+#endif
+
  #define DEFERRED_ERROR_VECTOR		0xf4
 
  /* Vector on which hypervisor callbacks will be delivered */
#32 lore raw

Re: [RFC PATCH v3 12/58] perf: core/x86: Register a new vector for KVM GUEST PMI

Quoted 85 lines · Colton Lewis

Hello,

Quoted 69 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>
Create a new vector in the host IDT for kvm guest PMI handling within mediated passthrough vPMU. In addition, guest PMI handler registration is added into x86_set_kvm_irq_handler().
This is the preparation work to support mediated passthrough vPMU to handle kvm guest PMIs without interference from PMI handler of the host PMU.

---
  arch/x86/include/asm/hardirq.h                |  1 +
  arch/x86/include/asm/idtentry.h               |  1 +
  arch/x86/include/asm/irq_vectors.h            |  5 ++++-
  arch/x86/kernel/idt.c                         |  1 +
  arch/x86/kernel/irq.c                         | 21 +++++++++++++++++++
  .../beauty/arch/x86/include/asm/irq_vectors.h |  5 ++++-
  6 files changed, 32 insertions(+), 2 deletions(-)
arch/x86/include/asm/hardirq.h +1 -0
index c67fa6ad098a..42a396763c8d 100644
@@ -19,6 +19,7 @@ typedef struct {
  	unsigned int kvm_posted_intr_ipis;
  	unsigned int kvm_posted_intr_wakeup_ipis;
  	unsigned int kvm_posted_intr_nested_ipis;
+	unsigned int kvm_guest_pmis;
  #endif
  	unsigned int x86_platform_ipis;	/* arch dependent */
  	unsigned int apic_perf_irqs;
arch/x86/include/asm/idtentry.h +1 -0
index d4f24499b256..7b1e3e542b1d 100644
@@ -745,6 +745,7 @@ DECLARE_IDTENTRY_SYSVEC(IRQ_WORK_VECTOR,		 
 sysvec_irq_work);
  DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_VECTOR,		sysvec_kvm_posted_intr_ipi);
  DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_WAKEUP_VECTOR,	 
 sysvec_kvm_posted_intr_wakeup_ipi);
  DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR,	 
 sysvec_kvm_posted_intr_nested_ipi);
+DECLARE_IDTENTRY_SYSVEC(KVM_GUEST_PMI_VECTOR,	         

sysvec_kvm_guest_pmi_handler);
#else

  # define fred_sysvec_kvm_posted_intr_ipi		NULL
  # define fred_sysvec_kvm_posted_intr_wakeup_ipi		NULL
arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..ada270e6f5cb 100644
@@ -77,7 +77,10 @@
   */
  #define IRQ_WORK_VECTOR			0xf6
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR		0xf5
+#endif
+
  #define DEFERRED_ERROR_VECTOR		0xf4
  /* Vector on which hypervisor callbacks will be delivered */
arch/x86/kernel/idt.c +0 -0
index f445bec516a0..0bec4c7e2308 100644

arch/x86/kernel/idt.c static const __initconst struct idt_data apic_idts[] · line 161

Quoted 5 lines
arch/x86/kernel/idt.c +1 -0
@@ -157,6 +157,7 @@ static const __initconst struct idt_data apic_idts[]  
 = {
  	INTG(POSTED_INTR_VECTOR,		asm_sysvec_kvm_posted_intr_ipi),
  	INTG(POSTED_INTR_WAKEUP_VECTOR,		asm_sysvec_kvm_posted_intr_wakeup_ipi),
  	INTG(POSTED_INTR_NESTED_VECTOR,		asm_sysvec_kvm_posted_intr_nested_ipi),
+	INTG(KVM_GUEST_PMI_VECTOR,		asm_sysvec_kvm_guest_pmi_handler),

There is a subtle inconsistency in that this code is guarded on CONFIG_HAVE_KVM when the #define KVM_GUEST_PMI_VECTOR is guarded on CONFIG_KVM. Beats me why there are two different flags that are so similar, but it is possible they have different values leading to compilation errors.

Quoted 5 lines

I happened to hit this compilation error because make defconfig will define CONFIG_HAVE_KVM but not CONFIG_KVM. CONFIG_KVM is the more strict of the two because it depends on CONFIG_HAVE_KVM.

This code should also be guarded on CONFIG_KVM.

Hi Colton,

 what's you kernel base? In latest code base, the CONFIG_HAVE_KVM has been dropped and this part of code has been guarded by CONFIG_KVM.

Please see commit dcf0926e9b89 "x86: replace CONFIG_HAVE_KVM with IS_ENABLED(CONFIG_KVM)". Thanks.

Quoted 72 lines
Quoted 72 lines
arch/x86/kernel/idt.c +0 -0
  # endif
  # ifdef CONFIG_IRQ_WORK

INTG(IRQ_WORK_VECTOR, asm_sysvec_irq_work),

arch/x86/kernel/irq.c +6 -0
index 18cd418fe106..b29714e23fc4 100644
@@ -183,6 +183,12 @@ int arch_show_interrupts(struct seq_file *p, int  
 prec)
  		seq_printf(p, "%10u ",
  			   irq_stats(j)->kvm_posted_intr_wakeup_ipis);
  	seq_puts(p, "  Posted-interrupt wakeup event\n");
+
+	seq_printf(p, "%*s: ", prec, "VPMU");
+	for_each_online_cpu(j)
+		seq_printf(p, "%10u ",
+			   irq_stats(j)->kvm_guest_pmis);
+	seq_puts(p, " KVM GUEST PMI\n");
  #endif
  #ifdef CONFIG_X86_POSTED_MSI
  	seq_printf(p, "%*s: ", prec, "PMN");
arch/x86/kernel/irq.c +15 -0
@@ -311,6 +317,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi)
  #if IS_ENABLED(CONFIG_KVM)
  static void dummy_handler(void) {}
  static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
+static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
  void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
  {
@@ -321,6 +328,10 @@ void x86_set_kvm_irq_handler(u8 vector, void  
 (*handler)(void))
  	    (handler == dummy_handler ||
  	     kvm_posted_intr_wakeup_handler == dummy_handler))
  		kvm_posted_intr_wakeup_handler = handler;
+	else if (vector == KVM_GUEST_PMI_VECTOR &&
+		 (handler == dummy_handler ||
+		  kvm_guest_pmi_handler == dummy_handler))
+		kvm_guest_pmi_handler = handler;
  	else
  		WARN_ON_ONCE(1);
@@ -356,6 +367,16 @@  
 DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_kvm_posted_intr_nested_ipi)
  	apic_eoi();
  	inc_irq_stat(kvm_posted_intr_nested_ipis);
  }
+
+/*
+ * Handler for KVM_GUEST_PMI_VECTOR.
+ */
+DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_guest_pmi_handler)
+{
+	apic_eoi();
+	inc_irq_stat(kvm_guest_pmis);
+	kvm_guest_pmi_handler();
+}
  #endif
  #ifdef CONFIG_X86_POSTED_MSI
tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..670dcee46631 100644
@@ -77,7 +77,10 @@
   */
  #define IRQ_WORK_VECTOR			0xf6
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR           0xf5
+#endif
+
  #define DEFERRED_ERROR_VECTOR		0xf4
  /* Vector on which hypervisor callbacks will be delivered */
#33 lore raw

Re: [RFC PATCH v3 12/58] perf: core/x86: Register a new vector for KVM GUEST PMI

Quoted 95 lines · Dapeng
Quoted 88 lines · Colton Lewis

Hello,

Quoted 70 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>
Create a new vector in the host IDT for kvm guest PMI handling within mediated passthrough vPMU. In addition, guest PMI handler registration is added into x86_set_kvm_irq_handler().
This is the preparation work to support mediated passthrough vPMU to handle kvm guest PMIs without interference from PMI handler of the host PMU.

---
   arch/x86/include/asm/hardirq.h                |  1 +
   arch/x86/include/asm/idtentry.h               |  1 +
   arch/x86/include/asm/irq_vectors.h            |  5 ++++-
   arch/x86/kernel/idt.c                         |  1 +
   arch/x86/kernel/irq.c                         | 21 +++++++++++++++++++
   .../beauty/arch/x86/include/asm/irq_vectors.h |  5 ++++-
   6 files changed, 32 insertions(+), 2 deletions(-)
arch/x86/include/asm/hardirq.h +1 -0
index c67fa6ad098a..42a396763c8d 100644
@@ -19,6 +19,7 @@ typedef struct {
   	unsigned int kvm_posted_intr_ipis;
   	unsigned int kvm_posted_intr_wakeup_ipis;
   	unsigned int kvm_posted_intr_nested_ipis;
+	unsigned int kvm_guest_pmis;
   #endif
   	unsigned int x86_platform_ipis;	/* arch dependent */
   	unsigned int apic_perf_irqs;
arch/x86/include/asm/idtentry.h +0 -0
index d4f24499b256..7b1e3e542b1d 100644
@@ -745,6 +745,7 @@ DECLARE_IDTENTRY_SYSVEC(IRQ_WORK_VECTOR,
 sysvec_irq_work);
   DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_VECTOR,		 
 sysvec_kvm_posted_intr_ipi);
   DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_WAKEUP_VECTOR,
 sysvec_kvm_posted_intr_wakeup_ipi);
   DECLARE_IDTENTRY_SYSVEC(POSTED_INTR_NESTED_VECTOR,

sysvec_kvm_posted_intr_nested_ipi);
+DECLARE_IDTENTRY_SYSVEC(KVM_GUEST_PMI_VECTOR,
sysvec_kvm_guest_pmi_handler);
#else

   # define fred_sysvec_kvm_posted_intr_ipi		NULL
   # define fred_sysvec_kvm_posted_intr_wakeup_ipi		NULL
arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..ada270e6f5cb 100644
@@ -77,7 +77,10 @@
    */
   #define IRQ_WORK_VECTOR			0xf6
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR		0xf5
+#endif
+
   #define DEFERRED_ERROR_VECTOR		0xf4
   /* Vector on which hypervisor callbacks will be delivered */
arch/x86/kernel/idt.c +0 -0
index f445bec516a0..0bec4c7e2308 100644

arch/x86/kernel/idt.c static const __initconst struct idt_data apic_idts[] · line 163

Quoted 7 lines
arch/x86/kernel/idt.c +1 -0
@@ -157,6 +157,7 @@ static const __initconst struct idt_data apic_idts[]
 = {
   	INTG(POSTED_INTR_VECTOR,		asm_sysvec_kvm_posted_intr_ipi),
   	INTG(POSTED_INTR_WAKEUP_VECTOR,		 
 asm_sysvec_kvm_posted_intr_wakeup_ipi),
   	INTG(POSTED_INTR_NESTED_VECTOR,		 
 asm_sysvec_kvm_posted_intr_nested_ipi),
+	INTG(KVM_GUEST_PMI_VECTOR,		asm_sysvec_kvm_guest_pmi_handler),

There is a subtle inconsistency in that this code is guarded on CONFIG_HAVE_KVM when the #define KVM_GUEST_PMI_VECTOR is guarded on CONFIG_KVM. Beats me why there are two different flags that are so similar, but it is possible they have different values leading to compilation errors.

Quoted 5 lines

I happened to hit this compilation error because make defconfig will define CONFIG_HAVE_KVM but not CONFIG_KVM. CONFIG_KVM is the more strict of the two because it depends on CONFIG_HAVE_KVM.

This code should also be guarded on CONFIG_KVM.

Quoted 7 lines

Hi Colton,

 what's you kernel base? In latest code base, the CONFIG_HAVE_KVM has been dropped and this part of code has been guarded by CONFIG_KVM.

Please see commit dcf0926e9b89 "x86: replace CONFIG_HAVE_KVM with IS_ENABLED(CONFIG_KVM)". Thanks.

My kernel base was older than that. Glad to know it is no longer an issue.

Quoted 71 lines
Quoted 71 lines
Quoted 71 lines
   # endif
   # ifdef CONFIG_IRQ_WORK
   	INTG(IRQ_WORK_VECTOR,			asm_sysvec_irq_work),
arch/x86/kernel/irq.c +6 -0
index 18cd418fe106..b29714e23fc4 100644
@@ -183,6 +183,12 @@ int arch_show_interrupts(struct seq_file *p, int
 prec)
   		seq_printf(p, "%10u ",
   			   irq_stats(j)->kvm_posted_intr_wakeup_ipis);
   	seq_puts(p, "  Posted-interrupt wakeup event\n");
+
+	seq_printf(p, "%*s: ", prec, "VPMU");
+	for_each_online_cpu(j)
+		seq_printf(p, "%10u ",
+			   irq_stats(j)->kvm_guest_pmis);
+	seq_puts(p, " KVM GUEST PMI\n");
   #endif
   #ifdef CONFIG_X86_POSTED_MSI
   	seq_printf(p, "%*s: ", prec, "PMN");
arch/x86/kernel/irq.c +15 -0
@@ -311,6 +317,7 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_x86_platform_ipi)
   #if IS_ENABLED(CONFIG_KVM)
   static void dummy_handler(void) {}
   static void (*kvm_posted_intr_wakeup_handler)(void) = dummy_handler;
+static void (*kvm_guest_pmi_handler)(void) = dummy_handler;
   void x86_set_kvm_irq_handler(u8 vector, void (*handler)(void))
   {
@@ -321,6 +328,10 @@ void x86_set_kvm_irq_handler(u8 vector, void
 (*handler)(void))
   	    (handler == dummy_handler ||
   	     kvm_posted_intr_wakeup_handler == dummy_handler))
   		kvm_posted_intr_wakeup_handler = handler;
+	else if (vector == KVM_GUEST_PMI_VECTOR &&
+		 (handler == dummy_handler ||
+		  kvm_guest_pmi_handler == dummy_handler))
+		kvm_guest_pmi_handler = handler;
   	else
   		WARN_ON_ONCE(1);
@@ -356,6 +367,16 @@
 DEFINE_IDTENTRY_SYSVEC_SIMPLE(sysvec_kvm_posted_intr_nested_ipi)
   	apic_eoi();
   	inc_irq_stat(kvm_posted_intr_nested_ipis);
   }
+
+/*
+ * Handler for KVM_GUEST_PMI_VECTOR.
+ */
+DEFINE_IDTENTRY_SYSVEC(sysvec_kvm_guest_pmi_handler)
+{
+	apic_eoi();
+	inc_irq_stat(kvm_guest_pmis);
+	kvm_guest_pmi_handler();
+}
   #endif
   #ifdef CONFIG_X86_POSTED_MSI
tools/perf/trace/beauty/arch/x86/include/asm/irq_vectors.h +4 -1
index 13aea8fc3d45..670dcee46631 100644
@@ -77,7 +77,10 @@
    */
   #define IRQ_WORK_VECTOR			0xf6
-/* 0xf5 - unused, was UV_BAU_MESSAGE */
+#if IS_ENABLED(CONFIG_KVM)
+#define KVM_GUEST_PMI_VECTOR           0xf5
+#endif
+
   #define DEFERRED_ERROR_VECTOR		0xf4
   /* Vector on which hypervisor callbacks will be delivered */
#34 lore raw

RFC PATCH v3 13/58 KVM: x86/pmu: Register KVM_GUEST_PMI_VECTOR handler

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

Add function to register/unregister guest KVM PMI handler at KVM module initialization and destroy. This allows the host PMU with passthough capability enabled can switch PMI handler at PMU context switch.

---
 arch/x86/kvm/x86.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/x86/kvm/x86.c +12 -0
index 8c9e4281d978..f1d589c07068 100644
@@ -13946,6 +13946,16 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size,
 }
 EXPORT_SYMBOL_GPL(kvm_sev_es_string_io);
 
+static void kvm_handle_guest_pmi(void)
+{
+	struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+	if (WARN_ON_ONCE(!vcpu))
+		return;
+
+	kvm_make_request(KVM_REQ_PMI, vcpu);
+}
+
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_entry);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio);
@@ -13980,12 +13990,14 @@ static int __init kvm_x86_init(void)
 {
 	kvm_mmu_x86_module_init();
 	mitigate_smt_rsb &= boot_cpu_has_bug(X86_BUG_SMT_RSB) && cpu_smt_possible();
+	x86_set_kvm_irq_handler(KVM_GUEST_PMI_VECTOR, kvm_handle_guest_pmi);
 	return 0;
 }
 module_init(kvm_x86_init);
 
 static void __exit kvm_x86_exit(void)
 {
+	x86_set_kvm_irq_handler(KVM_GUEST_PMI_VECTOR, NULL);
 	WARN_ON_ONCE(static_branch_unlikely(&kvm_has_noapic_vcpu));
 }
 module_exit(kvm_x86_exit);
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#35 lore raw

RFC PATCH v3 14/58 perf: Add switch_interrupt() interface

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +7 -2
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };
 
 enum perf_addr_filter_action_t {
@@ -1738,7 +1743,7 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
-void perf_guest_enter(void);
+void perf_guest_enter(u32 guest_lvtpc);
 void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
@@ -1833,7 +1838,7 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)			{ }
-static inline void perf_guest_enter(void)			{ }
+static inline void perf_guest_enter(u32 guest_lvtpc)		{ }
 static inline void perf_guest_exit(void)			{ }
 #endif
 
kernel/events/core.c +34 -2
index 57ff737b922b..047ca5748ee2 100644
@@ -422,6 +422,7 @@ static inline bool is_include_guest_event(struct perf_event *event)
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
+static struct pmu *passthru_pmu;
 static struct srcu_struct pmus_srcu;
 static cpumask_var_t perf_online_mask;
 static struct kmem_cache *perf_event_cache;
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}
+
 /* When entering a guest, schedule out all exclude_guest events. */
-void perf_guest_enter(void)
+void perf_guest_enter(u32 guest_lvtpc)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	perf_switch_interrupt(true, guest_lvtpc);
+
 	__this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
 	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
 		goto unlock;
 
+	perf_switch_interrupt(false, 0);
+
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
-	list_add_rcu(&pmu->entry, &pmus);
+	/*
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
+			goto free_dev;
+		}
+	}
+
+	list_add_tail_rcu(&pmu->entry, &pmus);
 	atomic_set(&pmu->exclusive_cnt, 0);
 	ret = 0;
 unlock:
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#36 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 122 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +7 -2
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };
 
 enum perf_addr_filter_action_t {
@@ -1738,7 +1743,7 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
-void perf_guest_enter(void);
+void perf_guest_enter(u32 guest_lvtpc);
 void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
@@ -1833,7 +1838,7 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)			{ }
-static inline void perf_guest_enter(void)			{ }
+static inline void perf_guest_enter(u32 guest_lvtpc)		{ }
 static inline void perf_guest_exit(void)			{ }
 #endif
 
kernel/events/core.c +22 -2
index 57ff737b922b..047ca5748ee2 100644
@@ -422,6 +422,7 @@ static inline bool is_include_guest_event(struct perf_event *event)
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
+static struct pmu *passthru_pmu;
 static struct srcu_struct pmus_srcu;
 static cpumask_var_t perf_online_mask;
 static struct kmem_cache *perf_event_cache;
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}
+
 /* When entering a guest, schedule out all exclude_guest events. */
-void perf_guest_enter(void)
+void perf_guest_enter(u32 guest_lvtpc)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	perf_switch_interrupt(true, guest_lvtpc);
+
 	__this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
 	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
 		goto unlock;
 
+	perf_switch_interrupt(false, 0);
+
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
-	list_add_rcu(&pmu->entry, &pmus);
+	/*
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11875

Quoted 10 lines
kernel/events/core.c +10 -0
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
+			goto free_dev;
+		}
+	}

Our intention is to virtualize IBS PMUs (Op and Fetch) using the same framework. However, if IBS PMUs are also using the PERF_PMU_CAP_PASSTHROUGH_VPMU capability, IBS PMU registration fails at this point because the Core PMU is already registered with PERF_PMU_CAP_PASSTHROUGH_VPMU.

Quoted 5 lines
kernel/events/core.c +2 -0
+
+	list_add_tail_rcu(&pmu->entry, &pmus);
 	atomic_set(&pmu->exclusive_cnt, 0);
 	ret = 0;
 unlock:
#37 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 130 lines · Manali Shukla
Quoted 129 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +7 -2
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };
 
 enum perf_addr_filter_action_t {
@@ -1738,7 +1743,7 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
-void perf_guest_enter(void);
+void perf_guest_enter(u32 guest_lvtpc);
 void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
@@ -1833,7 +1838,7 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)			{ }
-static inline void perf_guest_enter(void)			{ }
+static inline void perf_guest_enter(u32 guest_lvtpc)		{ }
 static inline void perf_guest_exit(void)			{ }
 #endif
 
kernel/events/core.c +29 -2
index 57ff737b922b..047ca5748ee2 100644
@@ -422,6 +422,7 @@ static inline bool is_include_guest_event(struct perf_event *event)
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
+static struct pmu *passthru_pmu;
 static struct srcu_struct pmus_srcu;
 static cpumask_var_t perf_online_mask;
 static struct kmem_cache *perf_event_cache;
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}
+
 /* When entering a guest, schedule out all exclude_guest events. */
-void perf_guest_enter(void)
+void perf_guest_enter(u32 guest_lvtpc)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	perf_switch_interrupt(true, guest_lvtpc);
+
 	__this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
 	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
 		goto unlock;
 
+	perf_switch_interrupt(false, 0);
+
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
-	list_add_rcu(&pmu->entry, &pmus);
+	/*
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
Quoted 9 lines

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11875

Quoted 3 lines
kernel/events/core.c +3 -0
+			goto free_dev;
+		}
+	}

Our intention is to virtualize IBS PMUs (Op and Fetch) using the same framework. However, if IBS PMUs are also using the PERF_PMU_CAP_PASSTHROUGH_VPMU capability, IBS PMU registration fails at this point because the Core PMU is already registered with PERF_PMU_CAP_PASSTHROUGH_VPMU.

The original implementation doesn't limit the number of PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU. But at that time, we could not find a case of more than one PMU with the flag. After several debates, the patch was simplified only to support one PMU with the flag. It should not be hard to change it back.

Thanks,
Kan

Quoted 7 lines
Quoted 5 lines
kernel/events/core.c +2 -0
+
+	list_add_tail_rcu(&pmu->entry, &pmus);
 	atomic_set(&pmu->exclusive_cnt, 0);
 	ret = 0;
 unlock:
#38 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 143 lines · Kan
Quoted 130 lines · Manali Shukla
Quoted 129 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +7 -2
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };
 
 enum perf_addr_filter_action_t {
@@ -1738,7 +1743,7 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
-void perf_guest_enter(void);
+void perf_guest_enter(u32 guest_lvtpc);
 void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
@@ -1833,7 +1838,7 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)			{ }
-static inline void perf_guest_enter(void)			{ }
+static inline void perf_guest_enter(u32 guest_lvtpc)		{ }
 static inline void perf_guest_exit(void)			{ }
 #endif
 
kernel/events/core.c +29 -2
index 57ff737b922b..047ca5748ee2 100644
@@ -422,6 +422,7 @@ static inline bool is_include_guest_event(struct perf_event *event)
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
+static struct pmu *passthru_pmu;
 static struct srcu_struct pmus_srcu;
 static cpumask_var_t perf_online_mask;
 static struct kmem_cache *perf_event_cache;
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}
+
 /* When entering a guest, schedule out all exclude_guest events. */
-void perf_guest_enter(void)
+void perf_guest_enter(u32 guest_lvtpc)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	perf_switch_interrupt(true, guest_lvtpc);
+
 	__this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
 	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
 		goto unlock;
 
+	perf_switch_interrupt(false, 0);
+
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
-	list_add_rcu(&pmu->entry, &pmus);
+	/*
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
Quoted 9 lines

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11875

Quoted 3 lines
kernel/events/core.c +3 -0
+			goto free_dev;
+		}
+	}

Our intention is to virtualize IBS PMUs (Op and Fetch) using the same framework. However, if IBS PMUs are also using the PERF_PMU_CAP_PASSTHROUGH_VPMU capability, IBS PMU registration fails at this point because the Core PMU is already registered with PERF_PMU_CAP_PASSTHROUGH_VPMU.

Quoted 9 lines

The original implementation doesn't limit the number of PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU. But at that time, we could not find a case of more than one PMU with the flag. After several debates, the patch was simplified only to support one PMU with the flag. It should not be hard to change it back.

Thanks,
Kan

Yes, we have a use case to use mediated passthrough vPMU framework for IBS virtualization. So, we will need it.

- Manali

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
kernel/events/core.c +2 -0
+
+	list_add_tail_rcu(&pmu->entry, &pmus);
 	atomic_set(&pmu->exclusive_cnt, 0);
 	ret = 0;
 unlock:
#39 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 145 lines · Manali Shukla
Quoted 143 lines · Kan
Quoted 139 lines · Manali Shukla
Quoted 122 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +7 -2
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
     * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
     */
    int (*check_period)             (struct perf_event *event, u64 value); /* optional */
+
+   /*
+    * Switch the interrupt vectors, e.g., guest enter/exit.
+    */
+   void (*switch_interrupt)        (bool enter, u32 guest_lvtpc); /* optional */
 };
 
 enum perf_addr_filter_action_t {
@@ -1738,7 +1743,7 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
-void perf_guest_enter(void);
+void perf_guest_enter(u32 guest_lvtpc);
 void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
@@ -1833,7 +1838,7 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)                     { }
-static inline void perf_guest_enter(void)                  { }
+static inline void perf_guest_enter(u32 guest_lvtpc)               { }
 static inline void perf_guest_exit(void)                   { }
 #endif
 
kernel/events/core.c +22 -2
index 57ff737b922b..047ca5748ee2 100644
@@ -422,6 +422,7 @@ static inline bool is_include_guest_event(struct perf_event *event)
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
+static struct pmu *passthru_pmu;
 static struct srcu_struct pmus_srcu;
 static cpumask_var_t perf_online_mask;
 static struct kmem_cache *perf_event_cache;
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+   /* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+   if (!passthru_pmu)
+           return;
+
+   if (passthru_pmu->switch_interrupt &&
+       try_module_get(passthru_pmu->module)) {
+           passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+           module_put(passthru_pmu->module);
+   }
+}
+
 /* When entering a guest, schedule out all exclude_guest events. */
-void perf_guest_enter(void)
+void perf_guest_enter(u32 guest_lvtpc)
 {
    struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
            perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
    }
 
+   perf_switch_interrupt(true, guest_lvtpc);
+
    __this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
    if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
            goto unlock;
 
+   perf_switch_interrupt(false, 0);
+
    perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
    ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
    perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
    if (!pmu->event_idx)
            pmu->event_idx = perf_event_idx_default;
 
-   list_add_rcu(&pmu->entry, &pmus);
+   /*
+    * Initialize passthru_pmu with the core pmu that has
+    * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11875

Quoted 10 lines
kernel/events/core.c +10 -0
+    */
+   if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+           if (!passthru_pmu)
+                   passthru_pmu = pmu;
+
+           if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+                   ret = -EINVAL;
+                   goto free_dev;
+           }
+   }

Our intention is to virtualize IBS PMUs (Op and Fetch) using the same framework. However, if IBS PMUs are also using the PERF_PMU_CAP_PASSTHROUGH_VPMU capability, IBS PMU registration fails at this point because the Core PMU is already registered with PERF_PMU_CAP_PASSTHROUGH_VPMU.

Quoted 5 lines
Quoted 5 lines

The original implementation doesn't limit the number of PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU. But at that time, we could not find a case of more than one PMU with the flag. After several debates, the patch was simplified only to support one PMU with the flag. It should not be hard to change it back.

The original implementation is by design having a terrible performance overhead, ie., every PMU context switch at runtime requires a SRCU lock pair and pmu list traversal. To reduce the overhead, we put "passthrough" pmus in the front of the list and quickly exit the pmu traversal when we just pass the last "passthrough" pmu.

I honestly do not like the idea because it is simply a hacky solution with high maintenance cost, but I am not the right person to make the call. So, if perf (kernel) subsystem maintainers are happy, I can withdraw from this one.

My second point is: if you look at the details, the only reason why we traverse the pmu list is to check if a pmu has implemented the "switch_interrupt()" API. If so, invoke it, which will change the PMI vector from NMI to a maskable interrupt for KVM. In AMD case, I bet even if there are multiple passthrough pmus, only one requires switching the interrupt vector. Let me know if this is wrong.

Thanks.
-Mingwei

Quoted 18 lines
Quoted 4 lines

Thanks,
Kan

Yes, we have a use case to use mediated passthrough vPMU framework for IBS virtualization. So, we will need it.

- Manali

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
kernel/events/core.c +2 -0
+
+   list_add_tail_rcu(&pmu->entry, &pmus);
    atomic_set(&pmu->exclusive_cnt, 0);
    ret = 0;
 unlock:
#40 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 170 lines · Mingwei Zhang
Quoted 145 lines · Manali Shukla
Quoted 143 lines · Kan
Quoted 139 lines · Manali Shukla
Quoted 122 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +7 -2
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
     * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
     */
    int (*check_period)             (struct perf_event *event, u64 value); /* optional */
+
+   /*
+    * Switch the interrupt vectors, e.g., guest enter/exit.
+    */
+   void (*switch_interrupt)        (bool enter, u32 guest_lvtpc); /* optional */
 };
 
 enum perf_addr_filter_action_t {
@@ -1738,7 +1743,7 @@ extern int perf_event_period(struct perf_event *event, u64 value);
 extern u64 perf_event_pause(struct perf_event *event, bool reset);
 int perf_get_mediated_pmu(void);
 void perf_put_mediated_pmu(void);
-void perf_guest_enter(void);
+void perf_guest_enter(u32 guest_lvtpc);
 void perf_guest_exit(void);
 #else /* !CONFIG_PERF_EVENTS: */
 static inline void *
@@ -1833,7 +1838,7 @@ static inline int perf_get_mediated_pmu(void)
 }
 
 static inline void perf_put_mediated_pmu(void)                     { }
-static inline void perf_guest_enter(void)                  { }
+static inline void perf_guest_enter(u32 guest_lvtpc)               { }
 static inline void perf_guest_exit(void)                   { }
 #endif
 
kernel/events/core.c +22 -2
index 57ff737b922b..047ca5748ee2 100644
@@ -422,6 +422,7 @@ static inline bool is_include_guest_event(struct perf_event *event)
 
 static LIST_HEAD(pmus);
 static DEFINE_MUTEX(pmus_lock);
+static struct pmu *passthru_pmu;
 static struct srcu_struct pmus_srcu;
 static cpumask_var_t perf_online_mask;
 static struct kmem_cache *perf_event_cache;
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+   /* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+   if (!passthru_pmu)
+           return;
+
+   if (passthru_pmu->switch_interrupt &&
+       try_module_get(passthru_pmu->module)) {
+           passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+           module_put(passthru_pmu->module);
+   }
+}
+
 /* When entering a guest, schedule out all exclude_guest events. */
-void perf_guest_enter(void)
+void perf_guest_enter(u32 guest_lvtpc)
 {
    struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
 
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
            perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
    }
 
+   perf_switch_interrupt(true, guest_lvtpc);
+
    __this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
    if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
            goto unlock;
 
+   perf_switch_interrupt(false, 0);
+
    perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
    ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
    perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
    if (!pmu->event_idx)
            pmu->event_idx = perf_event_idx_default;
 
-   list_add_rcu(&pmu->entry, &pmus);
+   /*
+    * Initialize passthru_pmu with the core pmu that has
+    * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11875

Quoted 10 lines
kernel/events/core.c +10 -0
+    */
+   if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+           if (!passthru_pmu)
+                   passthru_pmu = pmu;
+
+           if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+                   ret = -EINVAL;
+                   goto free_dev;
+           }
+   }

Our intention is to virtualize IBS PMUs (Op and Fetch) using the same framework. However, if IBS PMUs are also using the PERF_PMU_CAP_PASSTHROUGH_VPMU capability, IBS PMU registration fails at this point because the Core PMU is already registered with PERF_PMU_CAP_PASSTHROUGH_VPMU.

Quoted 5 lines
Quoted 5 lines

The original implementation doesn't limit the number of PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU. But at that time, we could not find a case of more than one PMU with the flag. After several debates, the patch was simplified only to support one PMU with the flag. It should not be hard to change it back.

The original implementation is by design having a terrible performance overhead, ie., every PMU context switch at runtime requires a SRCU lock pair and pmu list traversal. To reduce the overhead, we put "passthrough" pmus in the front of the list and quickly exit the pmu traversal when we just pass the last "passthrough" pmu.

I honestly do not like the idea because it is simply a hacky solution with high maintenance cost, but I am not the right person to make the call. So, if perf (kernel) subsystem maintainers are happy, I can withdraw from this one.

My second point is: if you look at the details, the only reason why we traverse the pmu list is to check if a pmu has implemented the "switch_interrupt()" API. If so, invoke it, which will change the PMI vector from NMI to a maskable interrupt for KVM. In AMD case, I bet even if there are multiple passthrough pmus, only one requires switching the interrupt vector. Let me know if this is wrong.

Quoted 3 lines

Thanks.
-Mingwei

Yeah, That is correct. Currently, switching of the interrupt vector is needed only by one passthrough PMU for AMD. It is not required for IBS virtualization and PMC virtualization because VNMI is used to deliver interrupt in both cases.

With the mediated passthrough VPMU enabled, all exclude_guest events for PMUs with PERF_PMU_CAP_PASSTHROUGH_VPMU capability are scheduled out when entering a guest and scheduled back in upon exit. We need this functionality for IBS virtualization and PMC virtualization.

However, the current design allows only one passthrough PMU, which prevents the IBS driver from loading with the PERF_PMU_CAP_PASSTHROUGH_VPMU capability.

As a result, we are unable to utilize functionality of scheduling out and scheduling in exclude_guest events within the mediated passthrough VPMU framework for IBS virtualization.

- Manali

Quoted 18 lines
Quoted 18 lines
Quoted 4 lines

Thanks,
Kan

Yes, we have a use case to use mediated passthrough vPMU framework for IBS virtualization. So, we will need it.

- Manali

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
kernel/events/core.c +2 -0
+
+   list_add_tail_rcu(&pmu->entry, &pmus);
    atomic_set(&pmu->exclusive_cnt, 0);
    ret = 0;
 unlock:
maintainer
#41 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

In reply to Mingwei Zhang
Quoted 5 lines · Mingwei Zhang

The original implementation is by design having a terrible performance overhead, ie., every PMU context switch at runtime requires a SRCU lock pair and pmu list traversal. To reduce the overhead, we put "passthrough" pmus in the front of the list and quickly exit the pmu traversal when we just pass the last "passthrough" pmu.

What was the expensive bit? The SRCU memory barrier or the list iteration? How long is that list really?

#42 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 10 lines · Peter Zijlstra
Quoted 5 lines · Mingwei Zhang

The original implementation is by design having a terrible performance overhead, ie., every PMU context switch at runtime requires a SRCU lock pair and pmu list traversal. To reduce the overhead, we put "passthrough" pmus in the front of the list and quickly exit the pmu traversal when we just pass the last "passthrough" pmu.

What was the expensive bit? The SRCU memory barrier or the list iteration? How long is that list really?

Both. But I don't think there is any performance data.

The length of the list could vary on different platforms. For a modern server, there could be hundreds of PMUs from uncore PMUs, CXL PMUs, IOMMU PMUs, PMUs of accelerator devices and PMUs from all kinds of devices. The number could keep increasing with more and more devices supporting the PMU capability.

Two methods were considered.
- One is to add a global variable to track the "passthrough" pmu. The idea assumes that there is only one "passthrough" pmu that requires the switch, and the situation will not be changed in the near feature. So the SRCU memory barrier and the list iteration can be avoided. It's implemented in the patch

- The other one is always put the "passthrough" pmus in the front of the list. So the unnecessary list iteration can be avoided. It does nothing for the SRCU lock pair.

Thanks,
Kan

maintainer
#43 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 29 lines · Kan
Quoted 10 lines · Peter Zijlstra
Quoted 5 lines · Mingwei Zhang

The original implementation is by design having a terrible performance overhead, ie., every PMU context switch at runtime requires a SRCU lock pair and pmu list traversal. To reduce the overhead, we put "passthrough" pmus in the front of the list and quickly exit the pmu traversal when we just pass the last "passthrough" pmu.

What was the expensive bit? The SRCU memory barrier or the list iteration? How long is that list really?

Both. But I don't think there is any performance data.

The length of the list could vary on different platforms. For a modern server, there could be hundreds of PMUs from uncore PMUs, CXL PMUs, IOMMU PMUs, PMUs of accelerator devices and PMUs from all kinds of devices. The number could keep increasing with more and more devices supporting the PMU capability.

Two methods were considered.
- One is to add a global variable to track the "passthrough" pmu. The idea assumes that there is only one "passthrough" pmu that requires the switch, and the situation will not be changed in the near feature. So the SRCU memory barrier and the list iteration can be avoided. It's implemented in the patch

Quoted 3 lines

- The other one is always put the "passthrough" pmus in the front of the list. So the unnecessary list iteration can be avoided. It does nothing for the SRCU lock pair.

PaulMck has patches that introduce srcu_read_lock_lite(), which would avoid the smp_mb() in most cases.

https://lkml.kernel.org/r/[email protected]

We can also keep a second list, just for the passthrough pmus. A bit like sched_cb_list.

#44 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 34 lines · Peter Zijlstra
Quoted 29 lines · Kan
Quoted 10 lines · Peter Zijlstra
Quoted 5 lines · Mingwei Zhang

The original implementation is by design having a terrible performance overhead, ie., every PMU context switch at runtime requires a SRCU lock pair and pmu list traversal. To reduce the overhead, we put "passthrough" pmus in the front of the list and quickly exit the pmu traversal when we just pass the last "passthrough" pmu.

What was the expensive bit? The SRCU memory barrier or the list iteration? How long is that list really?

Both. But I don't think there is any performance data.

The length of the list could vary on different platforms. For a modern server, there could be hundreds of PMUs from uncore PMUs, CXL PMUs, IOMMU PMUs, PMUs of accelerator devices and PMUs from all kinds of devices. The number could keep increasing with more and more devices supporting the PMU capability.

Two methods were considered.
- One is to add a global variable to track the "passthrough" pmu. The idea assumes that there is only one "passthrough" pmu that requires the switch, and the situation will not be changed in the near feature. So the SRCU memory barrier and the list iteration can be avoided. It's implemented in the patch

Quoted 3 lines

- The other one is always put the "passthrough" pmus in the front of the list. So the unnecessary list iteration can be avoided. It does nothing for the SRCU lock pair.

Quoted 7 lines

PaulMck has patches that introduce srcu_read_lock_lite(), which would avoid the smp_mb() in most cases.

https://lkml.kernel.org/r/[email protected]

We can also keep a second list, just for the passthrough pmus. A bit like sched_cb_list.

Maybe we can do something like pmu_event_list, which has its own lock. So we should not need the srcu and the module reference.

Thanks,
Kan

maintainer
#45 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

In reply to Mingwei Zhang
Quoted 6 lines · Mingwei Zhang
kernel/events/core.c +2 -0
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{

kernel/events/core.c void perf_put_mediated_pmu(void) · line 5956

Quoted 10 lines
kernel/events/core.c +10 -0
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}

Should we move the whole module reference to perf_pmu_(,un}register() ?

Quoted 6 lines
kernel/events/core.c +1 -1
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
-	list_add_rcu(&pmu->entry, &pmus);
+	/*

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11873

Quoted 10 lines
kernel/events/core.c +10 -0
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
+			goto free_dev;

Why impose this limit? Changelog also fails to explain this.

Quoted 10 lines
kernel/events/core.c +4 -0
+		}
+	}
+
+	list_add_tail_rcu(&pmu->entry, &pmus);
 	atomic_set(&pmu->exclusive_cnt, 0);
 	ret = 0;
 unlock:
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#46 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Mingwei Zhang
kernel/events/core.c +0 -0
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
Quoted 14 lines

kernel/events/core.c void perf_put_mediated_pmu(void) · line 5956

Quoted 12 lines
kernel/events/core.c +12 -0
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}

Should we move the whole module reference to perf_pmu_(,un}register() ?

A PMU module can be load/unload anytime. How should we know if the PMU module is available when the reference check is moved to perf_pmu_(,un}register()?

Quoted 5 lines
Quoted 5 lines
kernel/events/core.c +0 -0
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
Quoted 14 lines

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11873

Quoted 12 lines
kernel/events/core.c +11 -1
-	list_add_rcu(&pmu->entry, &pmus);
+	/*
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
+			goto free_dev;

Why impose this limit? Changelog also fails to explain this.

Because the passthru_pmu is global variable. If there are two or more PMUs with the PERF_PMU_CAP_PASSTHROUGH_VPMU, the former one will be implicitly overwritten if without the check.

Thanks
Kan

Quoted 12 lines
Quoted 11 lines
kernel/events/core.c +4 -0
+		}
+	}
+
+	list_add_tail_rcu(&pmu->entry, &pmus);
 	atomic_set(&pmu->exclusive_cnt, 0);
 	ret = 0;
 unlock:
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
maintainer
#47 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 19 lines · Kan
Quoted 4 lines
Quoted 4 lines
kernel/events/core.c +0 -0
@@ -11842,7 +11860,21 @@ int perf_pmu_register(struct pmu *pmu, const char *name, int type)
 	if (!pmu->event_idx)
 		pmu->event_idx = perf_event_idx_default;
 
Quoted 14 lines

kernel/events/core.c int perf_pmu_register(struct pmu *pmu, const char *name, int type) · line 11873

Quoted 12 lines
kernel/events/core.c +11 -1
-	list_add_rcu(&pmu->entry, &pmus);
+	/*
+	 * Initialize passthru_pmu with the core pmu that has
+	 * PERF_PMU_CAP_PASSTHROUGH_VPMU capability.
+	 */
+	if (pmu->capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU) {
+		if (!passthru_pmu)
+			passthru_pmu = pmu;
+
+		if (WARN_ONCE(passthru_pmu != pmu, "Only one passthrough PMU is supported\n")) {
+			ret = -EINVAL;
+			goto free_dev;

Why impose this limit? Changelog also fails to explain this.

Quoted 3 lines

Because the passthru_pmu is global variable. If there are two or more PMUs with the PERF_PMU_CAP_PASSTHROUGH_VPMU, the former one will be implicitly overwritten if without the check.

That is not the question; but is has been answered elsewhere. For some reason you thought the srcu+list thing was expensive.

maintainer
#48 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

In reply to Liang, Kan
Quoted 24 lines · Kan
Quoted 6 lines · Peter Zijlstra
Quoted 4 lines · Mingwei Zhang
kernel/events/core.c +0 -0
@@ -5941,8 +5942,21 @@ void perf_put_mediated_pmu(void)
 }
 EXPORT_SYMBOL_GPL(perf_put_mediated_pmu);
 
Quoted 14 lines

kernel/events/core.c void perf_put_mediated_pmu(void) · line 5956

Quoted 12 lines
kernel/events/core.c +12 -0
+static void perf_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	/* Mediated passthrough PMU should have PASSTHROUGH_VPMU cap. */
+	if (!passthru_pmu)
+		return;
+
+	if (passthru_pmu->switch_interrupt &&
+	    try_module_get(passthru_pmu->module)) {
+		passthru_pmu->switch_interrupt(enter, guest_lvtpc);
+		module_put(passthru_pmu->module);
+	}
+}

Should we move the whole module reference to perf_pmu_(,un}register() ?

Quoted 3 lines

A PMU module can be load/unload anytime. How should we know if the PMU module is available when the reference check is moved to perf_pmu_(,un}register()?

Feh, dunno. I never really use modules. I just think the above is naf -- doubly so because you're saying the SRCU smp_mb() are expensive; but this module reference crap is more expensive than that.

(IOW, the SRCU smp_mb() cannot have been the problem)

maintainer
#49 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

In reply to Mingwei Zhang
Quoted 32 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +0 -0
index 75773f9890cc..aeb08f78f539 100644

include/linux/perf_event.h struct pmu { · line 549

Quoted 9 lines
include/linux/perf_event.h +5 -0
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };

I'm thinking the guets_lvtpc argument shouldn't be part of the interface. That should be PMU implementation data and accessed by the method implementation.

#50 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 38 lines · Peter Zijlstra
Quoted 37 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +2 -0
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
Quoted 8 lines

include/linux/perf_event.h struct pmu { · line 549

Quoted 4 lines
include/linux/perf_event.h +3 -0
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };

I'm thinking the guets_lvtpc argument shouldn't be part of the interface. That should be PMU implementation data and accessed by the method implementation.

I think the name of the perf_switch_interrupt() is too specific. Here should be to switch the guest context. The interrupt should be just part of the context. Maybe a interface as below

void (*switch_guest_ctx) (bool enter, void *data); /* optional */

Thanks,
Kan

maintainer
#51 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 48 lines · Kan
Quoted 38 lines · Peter Zijlstra
Quoted 37 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +2 -0
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
Quoted 8 lines

include/linux/perf_event.h struct pmu { · line 549

Quoted 4 lines
include/linux/perf_event.h +3 -0
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };

I'm thinking the guets_lvtpc argument shouldn't be part of the interface. That should be PMU implementation data and accessed by the method implementation.

Quoted 5 lines

I think the name of the perf_switch_interrupt() is too specific. Here should be to switch the guest context. The interrupt should be just part of the context. Maybe a interface as below

void (*switch_guest_ctx) (bool enter, void *data); /* optional */

I don't think you even need the data thing. For example, the x86/intel implementation can just look at a x86_pmu data field to find the magic value.

#52 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 53 lines · Peter Zijlstra
Quoted 52 lines · Kan
Quoted 38 lines · Peter Zijlstra
Quoted 37 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

There will be a dedicated interrupt vector for guests on some platforms, e.g., Intel. Add an interface to switch the interrupt vector while entering/exiting a guest.

When PMI switch into a new guest vector, guest_lvtpc value need to be reflected onto HW, e,g., guest clear PMI mask bit, the HW PMI mask bit should be cleared also, then PMI can be generated continuously for guest. So guest_lvtpc parameter is added into perf_guest_enter() and switch_interrupt().

At switch_interrupt(), the target pmu with PASSTHROUGH cap should be found. Since only one passthrough pmu is supported, we keep the implementation simply by tracking the pmu as a global variable.

[Simplify the commit with removal of srcu lock/unlock since only one pmu is supported.]

---
 include/linux/perf_event.h |  9 +++++++--
 kernel/events/core.c       | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 41 insertions(+), 4 deletions(-)
include/linux/perf_event.h +2 -0
index 75773f9890cc..aeb08f78f539 100644
@@ -541,6 +541,11 @@ struct pmu {
 	 * Check period value for PERF_EVENT_IOC_PERIOD ioctl.
 	 */
 	int (*check_period)		(struct perf_event *event, u64 value); /* optional */
+
+	/*
Quoted 8 lines

include/linux/perf_event.h struct pmu { · line 549

Quoted 4 lines
include/linux/perf_event.h +3 -0
+	 * Switch the interrupt vectors, e.g., guest enter/exit.
+	 */
+	void (*switch_interrupt)	(bool enter, u32 guest_lvtpc); /* optional */
 };

I'm thinking the guets_lvtpc argument shouldn't be part of the interface. That should be PMU implementation data and accessed by the method implementation.

I think the name of the perf_switch_interrupt() is too specific. Here should be to switch the guest context. The interrupt should be just part of the context. Maybe a interface as below

Quoted 5 lines
Quoted 1 line

void (*switch_guest_ctx) (bool enter, void *data); /* optional */

I don't think you even need the data thing. For example, the x86/intel implementation can just look at a x86_pmu data field to find the magic value.

The new vector is created by KVM, not perf. So it cannot be found in the x86_pmu data field. Perf needs it to update the interrupt vector so the guest PMI can be handled by KVM directly.

Thanks,
Kan

maintainer
#53 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

In reply to Mingwei Zhang
Quoted 10 lines · Mingwei Zhang
kernel/events/core.c +2 -0
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	perf_switch_interrupt(true, guest_lvtpc);
+
 	__this_cpu_write(perf_in_guest, true);
 
 unlock:

kernel/events/core.c void perf_guest_exit(void) · line 6003

Quoted 8 lines
kernel/events/core.c +2 -0
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
 	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
 		goto unlock;
 
+	perf_switch_interrupt(false, 0);
+
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);

This seems to suggest the method is named wrong, it should probably be guest_enter() or somsuch.

#54 lore raw

Re: [RFC PATCH v3 14/58] perf: Add switch_interrupt() interface

Quoted 14 lines · Peter Zijlstra
Quoted 13 lines · Mingwei Zhang
kernel/events/core.c +2 -0
@@ -5962,6 +5976,8 @@ void perf_guest_enter(void)
 		perf_ctx_enable(cpuctx->task_ctx, EVENT_GUEST);
 	}
 
+	perf_switch_interrupt(true, guest_lvtpc);
+
 	__this_cpu_write(perf_in_guest, true);
 
 unlock:
@@ -5980,6 +5996,8 @@ void perf_guest_exit(void)
 	if (WARN_ON_ONCE(!__this_cpu_read(perf_in_guest)))
 		goto unlock;
 
Quoted 9 lines

kernel/events/core.c void perf_guest_exit(void) · line 6003

Quoted 5 lines
kernel/events/core.c +2 -0
+	perf_switch_interrupt(false, 0);
+
 	perf_ctx_disable(&cpuctx->ctx, EVENT_GUEST);
 	ctx_sched_in(&cpuctx->ctx, EVENT_GUEST);
 	perf_ctx_enable(&cpuctx->ctx, EVENT_GUEST);

This seems to suggest the method is named wrong, it should probably be guest_enter() or somsuch.

The ctx_sched_in() is to schedule in the host context after the guest_exit(). The EVENT_GUEST is to indicate the guest ctx switch.

The name may brings some confusion. Maybe I can add a wrap function perf_host_enter() to include the above codes.

Thanks,
Kan

#55 lore raw

RFC PATCH v3 15/58 perf/x86: Support switch_interrupt interface

In reply to Mingwei Zhang

From: Kan Liang <[email protected]>

Implement switch_interrupt interface for x86 PMU, switch PMI to dedicated KVM_GUEST_PMI_VECTOR at perf guest enter, and switch PMI back to NMI at perf guest exit.

---
 arch/x86/events/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/events/core.c +11 -0
index 5bf78cd619bf..b17ef8b6c1a6 100644
@@ -2673,6 +2673,15 @@ static bool x86_pmu_filter(struct pmu *pmu, int cpu)
 	return ret;
 }
 
+static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	if (enter)
+		apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			   (guest_lvtpc & APIC_LVT_MASKED));
+	else
+		apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+
 static struct pmu pmu = {
 	.pmu_enable		= x86_pmu_enable,
 	.pmu_disable		= x86_pmu_disable,
@@ -2702,6 +2711,8 @@ static struct pmu pmu = {
 	.aux_output_match	= x86_pmu_aux_output_match,
 
 	.filter			= x86_pmu_filter,
+
+	.switch_interrupt	= x86_pmu_switch_interrupt,
 };
 
 void arch_perf_update_userpage(struct perf_event *event,
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#56 lore raw

Re: [RFC PATCH v3 15/58] perf/x86: Support switch_interrupt interface

Quoted 22 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Implement switch_interrupt interface for x86 PMU, switch PMI to dedicated KVM_GUEST_PMI_VECTOR at perf guest enter, and switch PMI back to NMI at perf guest exit.

---
  arch/x86/events/core.c | 11 +++++++++++
  1 file changed, 11 insertions(+)
arch/x86/events/core.c +0 -0
index 5bf78cd619bf..b17ef8b6c1a6 100644
@@ -2673,6 +2673,15 @@ static bool x86_pmu_filter(struct pmu *pmu, int  
 cpu)
  	return ret;
  }

arch/x86/events/core.c static bool x86_pmu_filter(struct pmu *pmu, int · line 2685

Quoted 10 lines
arch/x86/events/core.c +9 -0
 
+static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	if (enter)
+		apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+			   (guest_lvtpc & APIC_LVT_MASKED));
+	else
+		apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+

Similar issue I point out in an earlier patch. #define KVM_GUEST_PMI_VECTOR is guarded by CONFIG_KVM but this code is not, which can result in compile errors.

Quoted 12 lines
arch/x86/events/core.c +0 -0
  static struct pmu pmu = {
  	.pmu_enable		= x86_pmu_enable,

.pmu_disable = x86_pmu_disable,

arch/x86/events/core.c +2 -0
@@ -2702,6 +2711,8 @@ static struct pmu pmu = {
  	.aux_output_match	= x86_pmu_aux_output_match,
 
  	.filter			= x86_pmu_filter,
+
+	.switch_interrupt	= x86_pmu_switch_interrupt,
  };
 
  void arch_perf_update_userpage(struct perf_event *event,
#57 lore raw

Re: [RFC PATCH v3 15/58] perf/x86: Support switch_interrupt interface

Quoted 25 lines · Colton Lewis
Quoted 23 lines · Mingwei Zhang

From: Kan Liang <[email protected]>
Implement switch_interrupt interface for x86 PMU, switch PMI to dedicated KVM_GUEST_PMI_VECTOR at perf guest enter, and switch PMI back to NMI at perf guest exit.

---
  arch/x86/events/core.c | 11 +++++++++++
  1 file changed, 11 insertions(+)
arch/x86/events/core.c +4 -0
index 5bf78cd619bf..b17ef8b6c1a6 100644
@@ -2673,6 +2673,15 @@ static bool x86_pmu_filter(struct pmu *pmu, int  
 cpu)
  	return ret;
  }
+static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+	if (enter)
+		apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
Quoted 8 lines

arch/x86/events/core.c static bool x86_pmu_filter(struct pmu *pmu, int cpu) · line 2684

Quoted 5 lines
arch/x86/events/core.c +5 -0
+			   (guest_lvtpc & APIC_LVT_MASKED));
+	else
+		apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+

Similar issue I point out in an earlier patch. #define KVM_GUEST_PMI_VECTOR is guarded by CONFIG_KVM but this code is not, which can result in compile errors.

Yes, thanks for pointing this.

Quoted 11 lines
Quoted 11 lines
arch/x86/events/core.c +2 -0
  static struct pmu pmu = {
  	.pmu_enable		= x86_pmu_enable,
  	.pmu_disable		= x86_pmu_disable,
@@ -2702,6 +2711,8 @@ static struct pmu pmu = {
  	.aux_output_match	= x86_pmu_aux_output_match,
  	.filter			= x86_pmu_filter,
+
+	.switch_interrupt	= x86_pmu_switch_interrupt,
  };
  void arch_perf_update_userpage(struct perf_event *event,
#58 lore raw

Re: [RFC PATCH v3 15/58] perf/x86: Support switch_interrupt interface

In reply to Colton Lewis
Quoted 25 lines · Colton Lewis
Quoted 22 lines · Mingwei Zhang

From: Kan Liang <[email protected]>

Implement switch_interrupt interface for x86 PMU, switch PMI to dedicated KVM_GUEST_PMI_VECTOR at perf guest enter, and switch PMI back to NMI at perf guest exit.

---
  arch/x86/events/core.c | 11 +++++++++++

  1 file changed, 11 insertions(+)
arch/x86/events/core.c +0 -0
index 5bf78cd619bf..b17ef8b6c1a6 100644
@@ -2673,6 +2673,15 @@ static bool x86_pmu_filter(struct pmu *pmu, int
 cpu)
       return ret;
   }
Quoted 13 lines

arch/x86/events/core.c static bool x86_pmu_filter(struct pmu *pmu, int cpu) · line 2684

Quoted 9 lines
arch/x86/events/core.c +9 -0
+static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+    if (enter)
+        apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
+               (guest_lvtpc & APIC_LVT_MASKED));
+    else
+        apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+

Similar issue I point out in an earlier patch. #define KVM_GUEST_PMI_VECTOR is guarded by CONFIG_KVM but this code is not, which can result in compile errors.

Since KVM_GUEST_PMI_VECTOR and the interrupt handler are owned by KVM, how about to simplify it to:

static void x86_pmu_switch_guest_ctx(bool enter, void *data)

{
	if (enter)
		apic_write(APIC_LVTPC, *(u32 *)data);
        ...
}

In KVM side:
perf_guest_enter(whatever_lvtpc_value_it_decides);

Quoted 13 lines
Quoted 12 lines
arch/x86/events/core.c +2 -0
   static struct pmu pmu = {
       .pmu_enable        = x86_pmu_enable,
       .pmu_disable        = x86_pmu_disable,
@@ -2702,6 +2711,8 @@ static struct pmu pmu = {
       .aux_output_match    = x86_pmu_aux_output_match,
 
       .filter            = x86_pmu_filter,
+
+    .switch_interrupt    = x86_pmu_switch_interrupt,
   };
 
   void arch_perf_update_userpage(struct perf_event *event,
#59 lore raw

Re: [RFC PATCH v3 15/58] perf/x86: Support switch_interrupt interface

Quoted 39 lines · Zide
Quoted 25 lines · Colton Lewis
Quoted 23 lines · Mingwei Zhang

From: Kan Liang <[email protected]>
Implement switch_interrupt interface for x86 PMU, switch PMI to dedicated KVM_GUEST_PMI_VECTOR at perf guest enter, and switch PMI back to NMI at perf guest exit.

---
  arch/x86/events/core.c | 11 +++++++++++

  1 file changed, 11 insertions(+)
arch/x86/events/core.c +4 -0
index 5bf78cd619bf..b17ef8b6c1a6 100644
@@ -2673,6 +2673,15 @@ static bool x86_pmu_filter(struct pmu *pmu, int
 cpu)
       return ret;
   }
+static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
+{
+    if (enter)
+        apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
Quoted 8 lines

arch/x86/events/core.c static bool x86_pmu_filter(struct pmu *pmu, int cpu) · line 2684

Quoted 5 lines
arch/x86/events/core.c +5 -0
+               (guest_lvtpc & APIC_LVT_MASKED));
+    else
+        apic_write(APIC_LVTPC, APIC_DM_NMI);
+}
+

Similar issue I point out in an earlier patch. #define KVM_GUEST_PMI_VECTOR is guarded by CONFIG_KVM but this code is not, which can result in compile errors.

Since KVM_GUEST_PMI_VECTOR and the interrupt handler are owned by KVM, how about to simplify it to:

static void x86_pmu_switch_guest_ctx(bool enter, void *data)

Quoted 8 lines
{
	if (enter)
		apic_write(APIC_LVTPC, *(u32 *)data);
        ...
}

In KVM side:
perf_guest_enter(whatever_lvtpc_value_it_decides);

Good point. Would address in v4.

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
arch/x86/events/core.c +2 -0
   static struct pmu pmu = {
       .pmu_enable        = x86_pmu_enable,
       .pmu_disable        = x86_pmu_disable,
@@ -2702,6 +2711,8 @@ static struct pmu pmu = {
       .aux_output_match    = x86_pmu_aux_output_match,
       .filter            = x86_pmu_filter,
+
+    .switch_interrupt    = x86_pmu_switch_interrupt,
   };
   void arch_perf_update_userpage(struct perf_event *event,
#60 lore raw

RFC PATCH v3 16/58 perf/x86: Forbid PMI handler when guest own PMU

In reply to Mingwei Zhang

If a guest PMI is delivered after VM-exit, the KVM maskable interrupt will be held pending until EFLAGS.IF is set. In the meantime, if the logical processor receives an NMI for any reason at all, perf_event_nmi_handler() will be invoked. If there is any active perf event anywhere on the system, x86_pmu_handle_irq() will be invoked, and it will clear IA32_PERF_GLOBAL_STATUS. By the time KVM's PMI handler is invoked, it will be a mystery which counter(s) overflowed.

When LVTPC is using KVM PMI vecotr, PMU is owned by guest, Host NMI let x86_pmu_handle_irq() run, x86_pmu_handle_irq() restore PMU vector to NMI and clear IA32_PERF_GLOBAL_STATUS, this breaks guest vPMU passthrough environment.

So modify perf_event_nmi_handler() to check perf_in_guest per cpu variable, and if so, to simply return without calling x86_pmu_handle_irq().

---
 arch/x86/events/core.c | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)
arch/x86/events/core.c +25 -2
index b17ef8b6c1a6..cb5d8f5fd9ce 100644
@@ -52,6 +52,8 @@ DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events) = {
 	.pmu = &pmu,
 };
 
+DEFINE_PER_CPU(bool, pmi_vector_is_nmi) = true;
+
 DEFINE_STATIC_KEY_FALSE(rdpmc_never_available_key);
 DEFINE_STATIC_KEY_FALSE(rdpmc_always_available_key);
 DEFINE_STATIC_KEY_FALSE(perf_is_hybrid);
@@ -1733,6 +1735,24 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
 	u64 finish_clock;
 	int ret;
 
+	/*
+	 * When guest pmu context is loaded this handler should be forbidden from
+	 * running, the reasons are:
+	 * 1. After perf_guest_enter() is called, and before cpu enter into
+	 *    non-root mode, NMI could happen, but x86_pmu_handle_irq() restore PMU
+	 *    to use NMI vector, which destroy KVM PMI vector setting.
+	 * 2. When VM is running, host NMI other than PMI causes VM exit, KVM will
+	 *    call host NMI handler (vmx_vcpu_enter_exit()) first before KVM save
+	 *    guest PMU context (kvm_pmu_save_pmu_context()), as x86_pmu_handle_irq()
+	 *    clear global_status MSR which has guest status now, then this destroy
+	 *    guest PMU status.
+	 * 3. After VM exit, but before KVM save guest PMU context, host NMI other
+	 *    than PMI could happen, x86_pmu_handle_irq() clear global_status MSR
+	 *    which has guest status now, then this destroy guest PMU status.
+	 */
+	if (!this_cpu_read(pmi_vector_is_nmi))
+		return 0;
+
 	/*
 	 * All PMUs/events that share this PMI handler should make sure to
 	 * increment active_events for their events.
@@ -2675,11 +2695,14 @@ static bool x86_pmu_filter(struct pmu *pmu, int cpu)
 
 static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
 {
-	if (enter)
+	if (enter) {
 		apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
 			   (guest_lvtpc & APIC_LVT_MASKED));
-	else
+		this_cpu_write(pmi_vector_is_nmi, false);
+	} else {
 		apic_write(APIC_LVTPC, APIC_DM_NMI);
+		this_cpu_write(pmi_vector_is_nmi, true);
+	}
 }
 
 static struct pmu pmu = {
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#61 lore raw

Re: [RFC PATCH v3 16/58] perf/x86: Forbid PMI handler when guest own PMU

Quoted 47 lines · Mingwei Zhang

If a guest PMI is delivered after VM-exit, the KVM maskable interrupt will be held pending until EFLAGS.IF is set. In the meantime, if the logical processor receives an NMI for any reason at all, perf_event_nmi_handler() will be invoked. If there is any active perf event anywhere on the system, x86_pmu_handle_irq() will be invoked, and it will clear IA32_PERF_GLOBAL_STATUS. By the time KVM's PMI handler is invoked, it will be a mystery which counter(s) overflowed.

When LVTPC is using KVM PMI vecotr, PMU is owned by guest, Host NMI let x86_pmu_handle_irq() run, x86_pmu_handle_irq() restore PMU vector to NMI and clear IA32_PERF_GLOBAL_STATUS, this breaks guest vPMU passthrough environment.

So modify perf_event_nmi_handler() to check perf_in_guest per cpu variable, and if so, to simply return without calling x86_pmu_handle_irq().

---
 arch/x86/events/core.c | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)
arch/x86/events/core.c +9 -0
index b17ef8b6c1a6..cb5d8f5fd9ce 100644
@@ -52,6 +52,8 @@ DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events) = {
 	.pmu = &pmu,
 };
 
+DEFINE_PER_CPU(bool, pmi_vector_is_nmi) = true;
+
 DEFINE_STATIC_KEY_FALSE(rdpmc_never_available_key);
 DEFINE_STATIC_KEY_FALSE(rdpmc_always_available_key);
 DEFINE_STATIC_KEY_FALSE(perf_is_hybrid);
@@ -1733,6 +1735,24 @@ perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
 	u64 finish_clock;
 	int ret;
 
+	/*
+	 * When guest pmu context is loaded this handler should be forbidden from
+	 * running, the reasons are:
+	 * 1. After perf_guest_enter() is called, and before cpu enter into
+	 *    non-root mode, NMI could happen, but x86_pmu_handle_irq() restore PMU
+	 *    to use NMI vector, which destroy KVM PMI vector setting.
+	 * 2. When VM is running, host NMI other than PMI causes VM exit, KVM will

arch/x86/events/core.c perf_event_nmi_handler(unsigned int cmd, struct pt_regs *regs) · line 1754

Quoted 10 lines
arch/x86/events/core.c +10 -0
+	 *    call host NMI handler (vmx_vcpu_enter_exit()) first before KVM save
+	 *    guest PMU context (kvm_pmu_save_pmu_context()), as x86_pmu_handle_irq()
+	 *    clear global_status MSR which has guest status now, then this destroy
+	 *    guest PMU status.
+	 * 3. After VM exit, but before KVM save guest PMU context, host NMI other
+	 *    than PMI could happen, x86_pmu_handle_irq() clear global_status MSR
+	 *    which has guest status now, then this destroy guest PMU status.
+	 */
+	if (!this_cpu_read(pmi_vector_is_nmi))
+		return 0;

0 -> NMI_DONE

Quoted 21 lines
arch/x86/events/core.c +6 -2
+
 	/*
 	 * All PMUs/events that share this PMI handler should make sure to
 	 * increment active_events for their events.
@@ -2675,11 +2695,14 @@ static bool x86_pmu_filter(struct pmu *pmu, int cpu)
 
 static void x86_pmu_switch_interrupt(bool enter, u32 guest_lvtpc)
 {
-	if (enter)
+	if (enter) {
 		apic_write(APIC_LVTPC, APIC_DM_FIXED | KVM_GUEST_PMI_VECTOR |
 			   (guest_lvtpc & APIC_LVT_MASKED));
-	else
+		this_cpu_write(pmi_vector_is_nmi, false);
+	} else {
 		apic_write(APIC_LVTPC, APIC_DM_NMI);
+		this_cpu_write(pmi_vector_is_nmi, true);
+	}
 }
 
 static struct pmu pmu = {
#62 lore raw

RFC PATCH v3 17/58 perf: core/x86: Plumb passthrough PMU capability from x86_pmu to x86_pmu_cap

In reply to Mingwei Zhang

Plumb passthrough PMU capability to x86_pmu_cap in order to let any kernel entity such as KVM know that host PMU support passthrough PMU mode and has the implementation.

---
 arch/x86/events/core.c            | 1 +
 arch/x86/include/asm/perf_event.h | 1 +
 2 files changed, 2 insertions(+)
arch/x86/events/core.c +1 -0
index cb5d8f5fd9ce..c16ceebf2d70 100644
@@ -3029,6 +3029,7 @@ void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 	cap->events_mask	= (unsigned int)x86_pmu.events_maskl;
 	cap->events_mask_len	= x86_pmu.events_mask_len;
 	cap->pebs_ept		= x86_pmu.pebs_ept;
+	cap->passthrough = !!(pmu.capabilities & PERF_PMU_CAP_PASSTHROUGH_VPMU);
 }
 EXPORT_SYMBOL_GPL(perf_get_x86_pmu_capability);
 
arch/x86/include/asm/perf_event.h +1 -0
index 7f1e17250546..5cf37fe1f30a 100644
@@ -258,6 +258,7 @@ struct x86_pmu_capability {
 	unsigned int	events_mask;
 	int		events_mask_len;
 	unsigned int	pebs_ept	:1;
+	unsigned int	passthrough	:1;
 };
 
 /*
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#63 lore raw

RFC PATCH v3 18/58 KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

In reply to Mingwei Zhang

Introduce enable_passthrough_pmu as a RO KVM kernel module parameter. This variable is true only when the following conditions satisfies:
- set to true when module loaded.
- enable_pmu is true.
- is running on Intel CPU.
- supports PerfMon v4.
- host PMU supports passthrough mode.

The value is always read-only because passthrough PMU currently does not support features like LBR and PEBS, while emualted PMU does. This will end up with two different values for kvm_cap.supported_perf_cap, which is initialized at module load time. Maintaining two different perf capabilities will add complexity. Further, there is not enough motivation to support running two types of PMU implementations at the same time, although it is possible/feasible in reality.

Finally, always propagate enable_passthrough_pmu and perf_capabilities into kvm->arch for each KVM instance.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.h              | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c          |  7 +++++--
 arch/x86/kvm/x86.c              |  8 ++++++++
 arch/x86/kvm/x86.h              |  1 +
 5 files changed, 29 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index f8ca74e7678f..a15c783f20b9 100644
@@ -1406,6 +1406,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
+	bool enable_passthrough_pmu;
 
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +14 -0
index 4d52b0b539ba..cf93be5e7359 100644
@@ -208,6 +208,20 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
+	/* Pass-through vPMU is only supported in Intel CPUs. */
+	if (!is_intel)
+		enable_passthrough_pmu = false;
+
+	/*
+	 * Pass-through vPMU requires at least PerfMon version 4 because the
+	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
+	 * for counter emulation as well as PMU context switch.  In addition, it
+	 * requires host PMU support on passthrough mode. Disable pass-through
+	 * vPMU if any condition fails.
+	 */
+	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)
+		enable_passthrough_pmu = false;
+
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
 		return;
arch/x86/kvm/vmx/vmx.c +5 -2
index ad465881b043..2ad122995f11 100644
@@ -146,6 +146,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
+module_param(enable_passthrough_pmu, bool, 0444);
+
 #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD)
 #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE
 #define KVM_VM_CR0_ALWAYS_ON				\
@@ -7924,7 +7926,8 @@ static __init u64 vmx_get_perf_capabilities(void)
 	if (boot_cpu_has(X86_FEATURE_PDCM))
 		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
 
-	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) {
+	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
+	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&vmx_lbr_caps);
 
 		/*
@@ -7938,7 +7941,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 			perf_cap |= host_perf_cap & PMU_CAP_LBR_FMT;
 	}
 
-	if (vmx_pebs_supported()) {
+	if (vmx_pebs_supported() && !enable_passthrough_pmu) {
 		perf_cap |= host_perf_cap & PERF_CAP_PEBS_MASK;
 
 		/*
arch/x86/kvm/x86.c +8 -0
index f1d589c07068..0c40f551130e 100644
@@ -187,6 +187,10 @@ bool __read_mostly enable_pmu = true;
 EXPORT_SYMBOL_GPL(enable_pmu);
 module_param(enable_pmu, bool, 0444);
 
+/* Enable/disable mediated passthrough PMU virtualization */
+bool __read_mostly enable_passthrough_pmu;
+EXPORT_SYMBOL_GPL(enable_passthrough_pmu);
+
 bool __read_mostly eager_page_split = true;
 module_param(eager_page_split, bool, 0644);
 
@@ -6682,6 +6686,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
+			/* Disable passthrough PMU if enable_pmu is false. */
+			if (!kvm->arch.enable_pmu)
+				kvm->arch.enable_passthrough_pmu = false;
 			r = 0;
 		}
 		mutex_unlock(&kvm->lock);
@@ -12623,6 +12630,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz;
 	kvm->arch.guest_can_read_msr_platform_info = true;
 	kvm->arch.enable_pmu = enable_pmu;
+	kvm->arch.enable_passthrough_pmu = enable_passthrough_pmu;
 
 #if IS_ENABLED(CONFIG_HYPERV)
 	spin_lock_init(&kvm->arch.hv_root_tdp_lock);
arch/x86/kvm/x86.h +1 -0
index d80a4c6b5a38..dc45ba42bec2 100644
@@ -332,6 +332,7 @@ extern u64 host_arch_capabilities;
 extern struct kvm_caps kvm_caps;
 
 extern bool enable_pmu;
+extern bool enable_passthrough_pmu;
 
 /*
  * Get a filtered version of KVM's supported XCR0 that strips out dynamic
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#64 lore raw

Re: [RFC PATCH v3 18/58] KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

As per my feedback in the initial RFC[*]:

2. The module param absolutely must not be exposed to userspace until all patches are in place. The easiest way to do that without creating dependency hell is to simply not create the module param.

[*] https://lore.kernel.org/all/[email protected]

Quoted 37 lines · Mingwei Zhang

Introduce enable_passthrough_pmu as a RO KVM kernel module parameter. This variable is true only when the following conditions satisfies:
- set to true when module loaded.
- enable_pmu is true.
- is running on Intel CPU.
- supports PerfMon v4.
- host PMU supports passthrough mode.

The value is always read-only because passthrough PMU currently does not support features like LBR and PEBS, while emualted PMU does. This will end up with two different values for kvm_cap.supported_perf_cap, which is initialized at module load time. Maintaining two different perf capabilities will add complexity. Further, there is not enough motivation to support running two types of PMU implementations at the same time, although it is possible/feasible in reality.

Finally, always propagate enable_passthrough_pmu and perf_capabilities into kvm->arch for each KVM instance.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.h              | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c          |  7 +++++--
 arch/x86/kvm/x86.c              |  8 ++++++++
 arch/x86/kvm/x86.h              |  1 +
 5 files changed, 29 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index f8ca74e7678f..a15c783f20b9 100644

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1409

Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1406,6 +1406,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
+	bool enable_passthrough_pmu;

Again, as I suggested/requested in the initial RFC[*], drop the per-VM flag as well as kvm_pmu.passthrough. There is zero reason to cache the module param. KVM should always query kvm->arch.enable_pmu prior to checking if the mediated PMU is enabled, so I doubt we even need a helper to check both.

[*] https://lore.kernel.org/all/[email protected]

Quoted 13 lines
arch/x86/include/asm/kvm_host.h +0 -0
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +2 -0
index 4d52b0b539ba..cf93be5e7359 100644
@@ -208,6 +208,20 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
+	/* Pass-through vPMU is only supported in Intel CPUs. */
+	if (!is_intel)

arch/x86/kvm/pmu.h static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops) · line 222

Quoted 10 lines
arch/x86/kvm/pmu.h +10 -0
+		enable_passthrough_pmu = false;
+
+	/*
+	 * Pass-through vPMU requires at least PerfMon version 4 because the
+	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
+	 * for counter emulation as well as PMU context switch.  In addition, it
+	 * requires host PMU support on passthrough mode. Disable pass-through
+	 * vPMU if any condition fails.
+	 */
+	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)

As is quite obvious by the end of the series, the v4 requirement is specific to Intel.

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    (is_intel && kvm_pmu_cap.version < 4) ||
	    (is_amd && kvm_pmu_cap.version < 2))
		enable_passthrough_pmu = false;

Furthermore, there is zero reason to explicitly and manually check the vendor, kvm_init_pmu_capability() takes kvm_pmu_ops. Adding a callback is somewhat undesirable as it would lead to duplicate code, but we can still provide separation of concerns by adding const variables to kvm_pmu_ops, a la MAX_NR_GP_COUNTERS.

E.g.

	if (enable_pmu) {
		perf_get_x86_pmu_capability(&kvm_pmu_cap);

		/*
		 * WARN if perf did NOT disable hardware PMU if the number of
		 * architecturally required GP counters aren't present, i.e. if
		 * there are a non-zero number of counters, but fewer than what
		 * is architecturally required.
		 */
		if (!kvm_pmu_cap.num_counters_gp ||
		    WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs))
			enable_pmu = false;
		else if (pmu_ops->MIN_PMU_VERSION > kvm_pmu_cap.version)
			enable_pmu = false;
	}

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    pmu_ops->MIN_MEDIATED_PMU_VERSION > kvm_pmu_cap.version)
		enable_mediated_pmu = false;
Quoted 10 lines
arch/x86/kvm/pmu.h +2 -0
+		enable_passthrough_pmu = false;
+
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
 		return;
arch/x86/kvm/vmx/vmx.c +0 -0
index ad465881b043..2ad122995f11 100644

arch/x86/kvm/vmx/vmx.c module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO); · line 149

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +1 -0
@@ -146,6 +146,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
+module_param(enable_passthrough_pmu, bool, 0444);

Hmm, we either need to put this param in kvm.ko, or move enable_pmu to vendor modules (or duplicate it there if we need to for backwards compatibility?).

There are advantages to putting params in vendor modules, when it's safe to do so, e.g. it allows toggling the param when (re)loading a vendor module, so I think I'm supportive of having the param live in vendor code. I just don't want to split the two PMU knobs.

Quoted 14 lines
arch/x86/kvm/vmx/vmx.c +2 -1
 #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD)
 #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE
 #define KVM_VM_CR0_ALWAYS_ON				\
@@ -7924,7 +7926,8 @@ static __init u64 vmx_get_perf_capabilities(void)
 	if (boot_cpu_has(X86_FEATURE_PDCM))
 		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
 
-	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) {
+	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
+	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&vmx_lbr_caps);
 
 		/*

arch/x86/kvm/vmx/vmx.c static __init u64 vmx_get_perf_capabilities(void) · line 7944

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7938,7 +7941,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 			perf_cap |= host_perf_cap & PMU_CAP_LBR_FMT;
 	}
 
-	if (vmx_pebs_supported()) {
+	if (vmx_pebs_supported() && !enable_passthrough_pmu) {

Checking enable_mediated_pmu belongs in vmx_pebs_supported(), not in here, otherwise KVM will incorrectly advertise support to userspace:

	if (vmx_pebs_supported()) {
		kvm_cpu_cap_check_and_set(X86_FEATURE_DS);
		kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);
	}
Quoted 18 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 		perf_cap |= host_perf_cap & PERF_CAP_PEBS_MASK;
 		/*
arch/x86/kvm/x86.c +4 -0
index f1d589c07068..0c40f551130e 100644
@@ -187,6 +187,10 @@ bool __read_mostly enable_pmu = true;
 EXPORT_SYMBOL_GPL(enable_pmu);
 module_param(enable_pmu, bool, 0444);
 
+/* Enable/disable mediated passthrough PMU virtualization */
+bool __read_mostly enable_passthrough_pmu;
+EXPORT_SYMBOL_GPL(enable_passthrough_pmu);
+
 bool __read_mostly eager_page_split = true;
 module_param(eager_page_split, bool, 0644);
 

arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6691

Quoted 6 lines
arch/x86/kvm/x86.c +3 -0
@@ -6682,6 +6686,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
+			/* Disable passthrough PMU if enable_pmu is false. */
+			if (!kvm->arch.enable_pmu)
+				kvm->arch.enable_passthrough_pmu = false;

And this code obviously goes away if the per-VM snapshot is removed.

#65 lore raw

Re: [RFC PATCH v3 18/58] KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

Quoted 7 lines · Sean Christopherson

As per my feedback in the initial RFC[*]:

2. The module param absolutely must not be exposed to userspace until all patches are in place. The easiest way to do that without creating dependency hell is to simply not create the module param.

[*] https://lore.kernel.org/all/[email protected]

Sure. It looks we missed this comment. Would address it.

Quoted 48 lines · Mingwei Zhang
Quoted 37 lines

Introduce enable_passthrough_pmu as a RO KVM kernel module parameter. This variable is true only when the following conditions satisfies:
- set to true when module loaded.
- enable_pmu is true.
- is running on Intel CPU.
- supports PerfMon v4.
- host PMU supports passthrough mode.

The value is always read-only because passthrough PMU currently does not support features like LBR and PEBS, while emualted PMU does. This will end up with two different values for kvm_cap.supported_perf_cap, which is initialized at module load time. Maintaining two different perf capabilities will add complexity. Further, there is not enough motivation to support running two types of PMU implementations at the same time, although it is possible/feasible in reality.

Finally, always propagate enable_passthrough_pmu and perf_capabilities into kvm->arch for each KVM instance.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.h              | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c          |  7 +++++--
 arch/x86/kvm/x86.c              |  8 ++++++++
 arch/x86/kvm/x86.h              |  1 +
 5 files changed, 29 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index f8ca74e7678f..a15c783f20b9 100644

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1409

Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1406,6 +1406,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
+	bool enable_passthrough_pmu;

Again, as I suggested/requested in the initial RFC[*], drop the per-VM flag as well as kvm_pmu.passthrough. There is zero reason to cache the module param. KVM should always query kvm->arch.enable_pmu prior to checking if the mediated PMU is enabled, so I doubt we even need a helper to check both.

Quoted 1 line

[*] https://lore.kernel.org/all/[email protected]

Sure.

Quoted 50 lines
Quoted 14 lines
arch/x86/include/asm/kvm_host.h +0 -0
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +2 -0
index 4d52b0b539ba..cf93be5e7359 100644
@@ -208,6 +208,20 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
+	/* Pass-through vPMU is only supported in Intel CPUs. */
+	if (!is_intel)

arch/x86/kvm/pmu.h static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops) · line 222

Quoted 10 lines
arch/x86/kvm/pmu.h +10 -0
+		enable_passthrough_pmu = false;
+
+	/*
+	 * Pass-through vPMU requires at least PerfMon version 4 because the
+	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
+	 * for counter emulation as well as PMU context switch.  In addition, it
+	 * requires host PMU support on passthrough mode. Disable pass-through
+	 * vPMU if any condition fails.
+	 */
+	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)

As is quite obvious by the end of the series, the v4 requirement is specific to Intel.

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    (is_intel && kvm_pmu_cap.version < 4) ||
	    (is_amd && kvm_pmu_cap.version < 2))
		enable_passthrough_pmu = false;

Furthermore, there is zero reason to explicitly and manually check the vendor, kvm_init_pmu_capability() takes kvm_pmu_ops. Adding a callback is somewhat undesirable as it would lead to duplicate code, but we can still provide separation of concerns by adding const variables to kvm_pmu_ops, a la MAX_NR_GP_COUNTERS.

E.g.

	if (enable_pmu) {
		perf_get_x86_pmu_capability(&kvm_pmu_cap);

		/*
		 * WARN if perf did NOT disable hardware PMU if the number of
		 * architecturally required GP counters aren't present, i.e. if
		 * there are a non-zero number of counters, but fewer than what
		 * is architecturally required.
		 */
		if (!kvm_pmu_cap.num_counters_gp ||
		    WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs))
Quoted 8 lines
			enable_pmu = false;
		else if (pmu_ops->MIN_PMU_VERSION > kvm_pmu_cap.version)
			enable_pmu = false;
	}

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    pmu_ops->MIN_MEDIATED_PMU_VERSION > kvm_pmu_cap.version)
		enable_mediated_pmu = false;

Sure.  would do.

Quoted 13 lines
Quoted 13 lines
arch/x86/kvm/pmu.h +2 -0
+		enable_passthrough_pmu = false;
+
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
 		return;
arch/x86/kvm/vmx/vmx.c +0 -0
index ad465881b043..2ad122995f11 100644
@@ -146,6 +146,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO); · line 149

Quoted 1 line
arch/x86/kvm/vmx/vmx.c +1 -0
+module_param(enable_passthrough_pmu, bool, 0444);

Hmm, we either need to put this param in kvm.ko, or move enable_pmu to vendor modules (or duplicate it there if we need to for backwards compatibility?).

There are advantages to putting params in vendor modules, when it's safe to do so, e.g. it allows toggling the param when (re)loading a vendor module, so I think I'm supportive of having the param live in vendor code. I just don't want to split the two PMU knobs.

Since enable_passthrough_pmu has already been in vendor modules,  we'd better duplicate enable_pmu module parameter in vendor modules as the 1st step.

Quoted 23 lines
Quoted 15 lines
arch/x86/kvm/vmx/vmx.c +2 -1
 #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD)
 #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE
 #define KVM_VM_CR0_ALWAYS_ON				\
@@ -7924,7 +7926,8 @@ static __init u64 vmx_get_perf_capabilities(void)
 	if (boot_cpu_has(X86_FEATURE_PDCM))
 		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
 
-	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) {
+	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
+	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&vmx_lbr_caps);
 
 		/*

arch/x86/kvm/vmx/vmx.c static __init u64 vmx_get_perf_capabilities(void) · line 7944

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7938,7 +7941,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 			perf_cap |= host_perf_cap & PMU_CAP_LBR_FMT;
 	}
 
-	if (vmx_pebs_supported()) {
+	if (vmx_pebs_supported() && !enable_passthrough_pmu) {

Checking enable_mediated_pmu belongs in vmx_pebs_supported(), not in here, otherwise KVM will incorrectly advertise support to userspace:

Quoted 4 lines
	if (vmx_pebs_supported()) {
		kvm_cpu_cap_check_and_set(X86_FEATURE_DS);
		kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);
	}

Sure. Thanks for pointing this.

Quoted 26 lines
Quoted 19 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 		perf_cap |= host_perf_cap & PERF_CAP_PEBS_MASK;
 		/*
arch/x86/kvm/x86.c +4 -0
index f1d589c07068..0c40f551130e 100644
@@ -187,6 +187,10 @@ bool __read_mostly enable_pmu = true;
 EXPORT_SYMBOL_GPL(enable_pmu);
 module_param(enable_pmu, bool, 0444);
 
+/* Enable/disable mediated passthrough PMU virtualization */
+bool __read_mostly enable_passthrough_pmu;
+EXPORT_SYMBOL_GPL(enable_passthrough_pmu);
+
 bool __read_mostly eager_page_split = true;
 module_param(eager_page_split, bool, 0644);
 

arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6691

Quoted 6 lines
arch/x86/kvm/x86.c +3 -0
@@ -6682,6 +6686,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
+			/* Disable passthrough PMU if enable_pmu is false. */
+			if (!kvm->arch.enable_pmu)
+				kvm->arch.enable_passthrough_pmu = false;

And this code obviously goes away if the per-VM snapshot is removed.

#66 lore raw

Re: [RFC PATCH v3 18/58] KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

Quoted 13 lines · Dapeng Mi
Quoted 12 lines · Sean Christopherson

arch/x86/kvm/vmx/vmx.c module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO); · line 149

Quoted 9 lines
arch/x86/kvm/vmx/vmx.c +1 -0
index ad465881b043..2ad122995f11 100644
@@ -146,6 +146,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
+module_param(enable_passthrough_pmu, bool, 0444);

Hmm, we either need to put this param in kvm.ko, or move enable_pmu to vendor modules (or duplicate it there if we need to for backwards compatibility?).

Quoted 7 lines
Quoted 4 lines

There are advantages to putting params in vendor modules, when it's safe to do so, e.g. it allows toggling the param when (re)loading a vendor module, so I think I'm supportive of having the param live in vendor code. I just don't want to split the two PMU knobs.

Since enable_passthrough_pmu has already been in vendor modules,  we'd better duplicate enable_pmu module parameter in vendor modules as the 1st step.

Paolo,

Any thoughts on having enable_pmu in vendor modules, vs. putting enable_mediated_pmu in kvm.ko? I don't have a strong opinion, there are pros and cons to both.

#67 lore raw

Re: [RFC PATCH v3 18/58] KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

In reply to Mi, Dapeng
Quoted 4 lines · Dapeng
Quoted 2 lines · Sean Christopherson

As per my feedback in the initial RFC[*]:

Quoted 8 lines
Quoted 5 lines

2. The module param absolutely must not be exposed to userspace until all patches are in place. The easiest way to do that without creating dependency hell is to simply not create the module param.

[*] https://lore.kernel.org/all/[email protected]

Sure. It looks we missed this comment. Would address it.

Dapeng, just synced with Sean offline. His point is that we still need kernel parameter but introduce that in the last part of the series so that any bisect won't trigger the new PMU logic in the middle of the series. But I think you are right to create a global config and make it false.

Quoted 199 lines
Quoted 49 lines · Mingwei Zhang
Quoted 37 lines

Introduce enable_passthrough_pmu as a RO KVM kernel module parameter. This variable is true only when the following conditions satisfies:
- set to true when module loaded.
- enable_pmu is true.
- is running on Intel CPU.
- supports PerfMon v4.
- host PMU supports passthrough mode.

The value is always read-only because passthrough PMU currently does not support features like LBR and PEBS, while emualted PMU does. This will end up with two different values for kvm_cap.supported_perf_cap, which is initialized at module load time. Maintaining two different perf capabilities will add complexity. Further, there is not enough motivation to support running two types of PMU implementations at the same time, although it is possible/feasible in reality.

Finally, always propagate enable_passthrough_pmu and perf_capabilities into kvm->arch for each KVM instance.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.h              | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c          |  7 +++++--
 arch/x86/kvm/x86.c              |  8 ++++++++
 arch/x86/kvm/x86.h              |  1 +
 5 files changed, 29 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index f8ca74e7678f..a15c783f20b9 100644

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1409

Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1406,6 +1406,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
+	bool enable_passthrough_pmu;

Again, as I suggested/requested in the initial RFC[*], drop the per-VM flag as well as kvm_pmu.passthrough. There is zero reason to cache the module param. KVM should always query kvm->arch.enable_pmu prior to checking if the mediated PMU is enabled, so I doubt we even need a helper to check both.

Quoted 1 line

[*] https://lore.kernel.org/all/[email protected]

Sure.

Quoted 50 lines
Quoted 14 lines
arch/x86/include/asm/kvm_host.h +0 -0
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +2 -0
index 4d52b0b539ba..cf93be5e7359 100644
@@ -208,6 +208,20 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
+	/* Pass-through vPMU is only supported in Intel CPUs. */
+	if (!is_intel)

arch/x86/kvm/pmu.h static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops) · line 222

Quoted 10 lines
arch/x86/kvm/pmu.h +10 -0
+		enable_passthrough_pmu = false;
+
+	/*
+	 * Pass-through vPMU requires at least PerfMon version 4 because the
+	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
+	 * for counter emulation as well as PMU context switch.  In addition, it
+	 * requires host PMU support on passthrough mode. Disable pass-through
+	 * vPMU if any condition fails.
+	 */
+	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)

As is quite obvious by the end of the series, the v4 requirement is specific to Intel.

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    (is_intel && kvm_pmu_cap.version < 4) ||
	    (is_amd && kvm_pmu_cap.version < 2))
		enable_passthrough_pmu = false;

Furthermore, there is zero reason to explicitly and manually check the vendor, kvm_init_pmu_capability() takes kvm_pmu_ops. Adding a callback is somewhat undesirable as it would lead to duplicate code, but we can still provide separation of concerns by adding const variables to kvm_pmu_ops, a la MAX_NR_GP_COUNTERS.

E.g.

	if (enable_pmu) {
		perf_get_x86_pmu_capability(&kvm_pmu_cap);

		/*
		 * WARN if perf did NOT disable hardware PMU if the number of
		 * architecturally required GP counters aren't present, i.e. if
		 * there are a non-zero number of counters, but fewer than what
		 * is architecturally required.
		 */
		if (!kvm_pmu_cap.num_counters_gp ||
		    WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs))
Quoted 8 lines
			enable_pmu = false;
		else if (pmu_ops->MIN_PMU_VERSION > kvm_pmu_cap.version)
			enable_pmu = false;
	}

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    pmu_ops->MIN_MEDIATED_PMU_VERSION > kvm_pmu_cap.version)
		enable_mediated_pmu = false;

Sure.  would do.

Quoted 13 lines
Quoted 13 lines
arch/x86/kvm/pmu.h +2 -0
+		enable_passthrough_pmu = false;
+
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
 		return;
arch/x86/kvm/vmx/vmx.c +0 -0
index ad465881b043..2ad122995f11 100644
@@ -146,6 +146,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO); · line 149

Quoted 1 line
arch/x86/kvm/vmx/vmx.c +1 -0
+module_param(enable_passthrough_pmu, bool, 0444);

Hmm, we either need to put this param in kvm.ko, or move enable_pmu to vendor modules (or duplicate it there if we need to for backwards compatibility?).

There are advantages to putting params in vendor modules, when it's safe to do so, e.g. it allows toggling the param when (re)loading a vendor module, so I think I'm supportive of having the param live in vendor code. I just don't want to split the two PMU knobs.

Since enable_passthrough_pmu has already been in vendor modules,  we'd better duplicate enable_pmu module parameter in vendor modules as the 1st step.

Quoted 23 lines
Quoted 15 lines
arch/x86/kvm/vmx/vmx.c +2 -1
 #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD)
 #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE
 #define KVM_VM_CR0_ALWAYS_ON				\
@@ -7924,7 +7926,8 @@ static __init u64 vmx_get_perf_capabilities(void)
 	if (boot_cpu_has(X86_FEATURE_PDCM))
 		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
 
-	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) {
+	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
+	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&vmx_lbr_caps);
 
 		/*

arch/x86/kvm/vmx/vmx.c static __init u64 vmx_get_perf_capabilities(void) · line 7944

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7938,7 +7941,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 			perf_cap |= host_perf_cap & PMU_CAP_LBR_FMT;
 	}
 
-	if (vmx_pebs_supported()) {
+	if (vmx_pebs_supported() && !enable_passthrough_pmu) {

Checking enable_mediated_pmu belongs in vmx_pebs_supported(), not in here, otherwise KVM will incorrectly advertise support to userspace:

Quoted 4 lines
	if (vmx_pebs_supported()) {
		kvm_cpu_cap_check_and_set(X86_FEATURE_DS);
		kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);
	}

Sure. Thanks for pointing this.

Quoted 26 lines
Quoted 19 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 		perf_cap |= host_perf_cap & PERF_CAP_PEBS_MASK;
 		/*
arch/x86/kvm/x86.c +4 -0
index f1d589c07068..0c40f551130e 100644
@@ -187,6 +187,10 @@ bool __read_mostly enable_pmu = true;
 EXPORT_SYMBOL_GPL(enable_pmu);
 module_param(enable_pmu, bool, 0444);
 
+/* Enable/disable mediated passthrough PMU virtualization */
+bool __read_mostly enable_passthrough_pmu;
+EXPORT_SYMBOL_GPL(enable_passthrough_pmu);
+
 bool __read_mostly eager_page_split = true;
 module_param(eager_page_split, bool, 0644);
 

arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6691

Quoted 6 lines
arch/x86/kvm/x86.c +3 -0
@@ -6682,6 +6686,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
+			/* Disable passthrough PMU if enable_pmu is false. */
+			if (!kvm->arch.enable_pmu)
+				kvm->arch.enable_passthrough_pmu = false;

And this code obviously goes away if the per-VM snapshot is removed.

#68 lore raw

Re: [RFC PATCH v3 18/58] KVM: x86/pmu: Introduce enable_passthrough_pmu module parameter

Quoted 8 lines · Mingwei Zhang
Quoted 7 lines · Dapeng
Quoted 6 lines · Sean Christopherson

As per my feedback in the initial RFC[*]:

2. The module param absolutely must not be exposed to userspace until all patches are in place. The easiest way to do that without creating dependency hell is to simply not create the module param.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

[*] https://lore.kernel.org/all/[email protected]

Sure. It looks we missed this comment. Would address it.

Dapeng, just synced with Sean offline. His point is that we still need kernel parameter but introduce that in the last part of the series so that any bisect won't trigger the new PMU logic in the middle of the series. But I think you are right to create a global config and make it false.

Ok, I would add a patch to expose the kernel parameter and put it at the last. Thanks.

Quoted 191 lines
Quoted 191 lines
Quoted 48 lines · Mingwei Zhang
Quoted 37 lines

Introduce enable_passthrough_pmu as a RO KVM kernel module parameter. This variable is true only when the following conditions satisfies:
- set to true when module loaded.
- enable_pmu is true.
- is running on Intel CPU.
- supports PerfMon v4.
- host PMU supports passthrough mode.

The value is always read-only because passthrough PMU currently does not support features like LBR and PEBS, while emualted PMU does. This will end up with two different values for kvm_cap.supported_perf_cap, which is initialized at module load time. Maintaining two different perf capabilities will add complexity. Further, there is not enough motivation to support running two types of PMU implementations at the same time, although it is possible/feasible in reality.

Finally, always propagate enable_passthrough_pmu and perf_capabilities into kvm->arch for each KVM instance.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.h              | 14 ++++++++++++++
 arch/x86/kvm/vmx/vmx.c          |  7 +++++--
 arch/x86/kvm/x86.c              |  8 ++++++++
 arch/x86/kvm/x86.h              |  1 +
 5 files changed, 29 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -0
index f8ca74e7678f..a15c783f20b9 100644

arch/x86/include/asm/kvm_host.h struct kvm_arch { · line 1409

Quoted 4 lines
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1406,6 +1406,7 @@ struct kvm_arch {
 
 	bool bus_lock_detection_enabled;
 	bool enable_pmu;
+	bool enable_passthrough_pmu;

Again, as I suggested/requested in the initial RFC[*], drop the per-VM flag as well as kvm_pmu.passthrough. There is zero reason to cache the module param. KVM should always query kvm->arch.enable_pmu prior to checking if the mediated PMU is enabled, so I doubt we even need a helper to check both.

Quoted 1 line

[*] https://lore.kernel.org/all/[email protected]

Sure.

Quoted 49 lines
Quoted 13 lines
arch/x86/include/asm/kvm_host.h +0 -0
 	u32 notify_window;
 	u32 notify_vmexit_flags;
arch/x86/kvm/pmu.h +2 -0
index 4d52b0b539ba..cf93be5e7359 100644
@@ -208,6 +208,20 @@ static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops)
 			enable_pmu = false;
 	}
 
+	/* Pass-through vPMU is only supported in Intel CPUs. */
+	if (!is_intel)

arch/x86/kvm/pmu.h static inline void kvm_init_pmu_capability(const struct kvm_pmu_ops *pmu_ops) · line 222

Quoted 10 lines
arch/x86/kvm/pmu.h +10 -0
+		enable_passthrough_pmu = false;
+
+	/*
+	 * Pass-through vPMU requires at least PerfMon version 4 because the
+	 * implementation requires the usage of MSR_CORE_PERF_GLOBAL_STATUS_SET
+	 * for counter emulation as well as PMU context switch.  In addition, it
+	 * requires host PMU support on passthrough mode. Disable pass-through
+	 * vPMU if any condition fails.
+	 */
+	if (!enable_pmu || kvm_pmu_cap.version < 4 || !kvm_pmu_cap.passthrough)

As is quite obvious by the end of the series, the v4 requirement is specific to Intel.

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    (is_intel && kvm_pmu_cap.version < 4) ||
	    (is_amd && kvm_pmu_cap.version < 2))
		enable_passthrough_pmu = false;

Furthermore, there is zero reason to explicitly and manually check the vendor, kvm_init_pmu_capability() takes kvm_pmu_ops. Adding a callback is somewhat undesirable as it would lead to duplicate code, but we can still provide separation of concerns by adding const variables to kvm_pmu_ops, a la MAX_NR_GP_COUNTERS.

E.g.

	if (enable_pmu) {
		perf_get_x86_pmu_capability(&kvm_pmu_cap);

		/*
		 * WARN if perf did NOT disable hardware PMU if the number of
		 * architecturally required GP counters aren't present, i.e. if
		 * there are a non-zero number of counters, but fewer than what
		 * is architecturally required.
		 */
		if (!kvm_pmu_cap.num_counters_gp ||
		    WARN_ON_ONCE(kvm_pmu_cap.num_counters_gp < min_nr_gp_ctrs))
Quoted 8 lines
			enable_pmu = false;
		else if (pmu_ops->MIN_PMU_VERSION > kvm_pmu_cap.version)
			enable_pmu = false;
	}

	if (!enable_pmu || !kvm_pmu_cap.passthrough ||
	    pmu_ops->MIN_MEDIATED_PMU_VERSION > kvm_pmu_cap.version)
		enable_mediated_pmu = false;

Sure.  would do.

Quoted 13 lines
Quoted 13 lines
arch/x86/kvm/pmu.h +2 -0
+		enable_passthrough_pmu = false;
+
 	if (!enable_pmu) {
 		memset(&kvm_pmu_cap, 0, sizeof(kvm_pmu_cap));
 		return;
arch/x86/kvm/vmx/vmx.c +0 -0
index ad465881b043..2ad122995f11 100644
@@ -146,6 +146,8 @@ module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO);
 extern bool __read_mostly allow_smaller_maxphyaddr;
 module_param(allow_smaller_maxphyaddr, bool, S_IRUGO);
 
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c module_param_named(preemption_timer, enable_preemption_timer, bool, S_IRUGO); · line 149

Quoted 1 line
arch/x86/kvm/vmx/vmx.c +1 -0
+module_param(enable_passthrough_pmu, bool, 0444);

Hmm, we either need to put this param in kvm.ko, or move enable_pmu to vendor modules (or duplicate it there if we need to for backwards compatibility?).

There are advantages to putting params in vendor modules, when it's safe to do so, e.g. it allows toggling the param when (re)loading a vendor module, so I think I'm supportive of having the param live in vendor code. I just don't want to split the two PMU knobs.

Since enable_passthrough_pmu has already been in vendor modules,  we'd better duplicate enable_pmu module parameter in vendor modules as the 1st step.

Quoted 22 lines
Quoted 14 lines
arch/x86/kvm/vmx/vmx.c +2 -1
 #define KVM_VM_CR0_ALWAYS_OFF (X86_CR0_NW | X86_CR0_CD)
 #define KVM_VM_CR0_ALWAYS_ON_UNRESTRICTED_GUEST X86_CR0_NE
 #define KVM_VM_CR0_ALWAYS_ON				\
@@ -7924,7 +7926,8 @@ static __init u64 vmx_get_perf_capabilities(void)
 	if (boot_cpu_has(X86_FEATURE_PDCM))
 		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
 
-	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR)) {
+	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
+	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&vmx_lbr_caps);
 
 		/*

arch/x86/kvm/vmx/vmx.c static __init u64 vmx_get_perf_capabilities(void) · line 7944

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7938,7 +7941,7 @@ static __init u64 vmx_get_perf_capabilities(void)
 			perf_cap |= host_perf_cap & PMU_CAP_LBR_FMT;
 	}
 
-	if (vmx_pebs_supported()) {
+	if (vmx_pebs_supported() && !enable_passthrough_pmu) {

Checking enable_mediated_pmu belongs in vmx_pebs_supported(), not in here, otherwise KVM will incorrectly advertise support to userspace:

Quoted 4 lines
	if (vmx_pebs_supported()) {
		kvm_cpu_cap_check_and_set(X86_FEATURE_DS);
		kvm_cpu_cap_check_and_set(X86_FEATURE_DTES64);
	}

Sure. Thanks for pointing this.

Quoted 25 lines
Quoted 18 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 		perf_cap |= host_perf_cap & PERF_CAP_PEBS_MASK;
 		/*
arch/x86/kvm/x86.c +4 -0
index f1d589c07068..0c40f551130e 100644
@@ -187,6 +187,10 @@ bool __read_mostly enable_pmu = true;
 EXPORT_SYMBOL_GPL(enable_pmu);
 module_param(enable_pmu, bool, 0444);
 
+/* Enable/disable mediated passthrough PMU virtualization */
+bool __read_mostly enable_passthrough_pmu;
+EXPORT_SYMBOL_GPL(enable_passthrough_pmu);
+
 bool __read_mostly eager_page_split = true;
 module_param(eager_page_split, bool, 0644);
 

arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6691

Quoted 6 lines
arch/x86/kvm/x86.c +3 -0
@@ -6682,6 +6686,9 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,
 		mutex_lock(&kvm->lock);
 		if (!kvm->created_vcpus) {
 			kvm->arch.enable_pmu = !(cap->args[0] & KVM_PMU_CAP_DISABLE);
+			/* Disable passthrough PMU if enable_pmu is false. */
+			if (!kvm->arch.enable_pmu)
+				kvm->arch.enable_passthrough_pmu = false;

And this code obviously goes away if the per-VM snapshot is removed.

#69 lore raw

RFC PATCH v3 19/58 KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs

In reply to Mingwei Zhang

Plumb through pass-through PMU setting from kvm->arch into kvm_pmu on each vcpu created. Note that enabling PMU is decided by VMM when it sets the CPUID bits exposed to guest VM. So plumb through the enabling for each pmu in intel_pmu_refresh().

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/pmu.c              |  1 +
 arch/x86/kvm/vmx/pmu_intel.c    | 12 +++++++++---
 3 files changed, 12 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
index a15c783f20b9..4b3ce6194bdb 100644
@@ -595,6 +595,8 @@ struct kvm_pmu {
 	 * redundant check before cleanup if guest don't use vPMU at all.
 	 */
 	u8 event_count;
+
+	bool passthrough;
 };
 
 struct kvm_pmu_ops;
arch/x86/kvm/pmu.c +1 -0
index a593b03c9aed..5768ea2935e9 100644
@@ -797,6 +797,7 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
 
 	memset(pmu, 0, sizeof(*pmu));
 	static_call(kvm_x86_pmu_init)(vcpu);
+	pmu->passthrough = false;
 	kvm_pmu_refresh(vcpu);
 }
 
arch/x86/kvm/vmx/pmu_intel.c +9 -3
index be40474de6e4..e417fd91e5fe 100644
@@ -470,15 +470,21 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 		return;
 
 	entry = kvm_find_cpuid_entry(vcpu, 0xa);
-	if (!entry)
+	if (!entry || !vcpu->kvm->arch.enable_pmu) {
+		pmu->passthrough = false;
 		return;
-
+	}
 	eax.full = entry->eax;
 	edx.full = entry->edx;
 
 	pmu->version = eax.split.version_id;
-	if (!pmu->version)
+	if (!pmu->version) {
+		pmu->passthrough = false;
 		return;
+	}
+
+	pmu->passthrough = vcpu->kvm->arch.enable_passthrough_pmu &&
+			   lapic_in_kernel(vcpu);
 
 	pmu->nr_arch_gp_counters = min_t(int, eax.split.num_counters,
 					 kvm_pmu_cap.num_counters_gp);
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#70 lore raw

Re: [RFC PATCH v3 19/58] KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs

Quoted 4 lines · Mingwei Zhang

Plumb through pass-through PMU setting from kvm->arch into kvm_pmu on each vcpu created. Note that enabling PMU is decided by VMM when it sets the CPUID bits exposed to guest VM. So plumb through the enabling for each pmu in intel_pmu_refresh().

Why? As with the per-VM snapshot, I see zero reason for this to exist, it's simply:

  kvm->arch.enable_pmu && enable_mediated_pmu && pmu->version;

And in literally every correct usage of pmu->passthrough, kvm->arch.enable_pmu and pmu->version have been checked (though implicitly), i.e. KVM can check enable_mediated_pmu and nothing else.

#71 lore raw

Re: [RFC PATCH v3 19/58] KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs

Quoted 5 lines · Sean Christopherson
Quoted 4 lines · Mingwei Zhang

Plumb through pass-through PMU setting from kvm->arch into kvm_pmu on each vcpu created. Note that enabling PMU is decided by VMM when it sets the CPUID bits exposed to guest VM. So plumb through the enabling for each pmu in intel_pmu_refresh().

Quoted 8 lines

Why? As with the per-VM snapshot, I see zero reason for this to exist, it's simply:

  kvm->arch.enable_pmu && enable_mediated_pmu && pmu->version;

And in literally every correct usage of pmu->passthrough, kvm->arch.enable_pmu and pmu->version have been checked (though implicitly), i.e. KVM can check enable_mediated_pmu and nothing else.

Ok, too many passthrough_pmu flags indeed confuse readers. Besides these dependencies, mediated vPMU also depends on lapic_in_kernel(). We need to set enable_mediated_pmu to false as well if lapic_in_kernel() returns false.

#72 lore raw

Re: [RFC PATCH v3 19/58] KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs

Quoted 12 lines · Dapeng Mi
Quoted 10 lines · Sean Christopherson
Quoted 4 lines · Mingwei Zhang

Plumb through pass-through PMU setting from kvm->arch into kvm_pmu on each vcpu created. Note that enabling PMU is decided by VMM when it sets the CPUID bits exposed to guest VM. So plumb through the enabling for each pmu in intel_pmu_refresh().

Why? As with the per-VM snapshot, I see zero reason for this to exist, it's simply:

  kvm->arch.enable_pmu && enable_mediated_pmu && pmu->version;
Quoted 7 lines
Quoted 3 lines

And in literally every correct usage of pmu->passthrough, kvm->arch.enable_pmu and pmu->version have been checked (though implicitly), i.e. KVM can check enable_mediated_pmu and nothing else.

Ok, too many passthrough_pmu flags indeed confuse readers. Besides these dependencies, mediated vPMU also depends on lapic_in_kernel(). We need to set enable_mediated_pmu to false as well if lapic_in_kernel() returns false.

No, just kill the entire vPMU.

Also, the need for an in-kernel APIC isn't unique to the mediated PMU. KVM simply drops PMIs if there's no APIC.

If we're feeling lucky, we could try a breaking change like so:

arch/x86/kvm/pmu.c +1 -1
index fcd188cc389a..bb08155f6198 100644
@@ -817,7 +817,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
        pmu->pebs_data_cfg_mask = ~0ull;
        bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
 
-       if (!vcpu->kvm->arch.enable_pmu)
+       if (!vcpu->kvm->arch.enable_pmu || !lapic_in_kernel(vcpu))
                return;
 
        static_call(kvm_x86_pmu_refresh)(vcpu);

If we don't want to risk breaking weird setups, we could restrict the behavior to the mediated PMU being enabled:

arch/x86/kvm/pmu.c +2 -1
index fcd188cc389a..bc9673190574 100644
@@ -817,7 +817,8 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
        pmu->pebs_data_cfg_mask = ~0ull;
        bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
 
-       if (!vcpu->kvm->arch.enable_pmu)
+       if (!vcpu->kvm->arch.enable_pmu ||
+           (!lapic_in_kernel(vcpu) && enable_mediated_pmu))
                return;
 
        static_call(kvm_x86_pmu_refresh)(vcpu);
#73 lore raw

Re: [RFC PATCH v3 19/58] KVM: x86/pmu: Plumb through pass-through PMU to vcpu for Intel CPUs

Quoted 48 lines · Sean Christopherson
Quoted 9 lines · Dapeng Mi
Quoted 8 lines · Sean Christopherson
Quoted 4 lines · Mingwei Zhang

Plumb through pass-through PMU setting from kvm->arch into kvm_pmu on each vcpu created. Note that enabling PMU is decided by VMM when it sets the CPUID bits exposed to guest VM. So plumb through the enabling for each pmu in intel_pmu_refresh().

Why? As with the per-VM snapshot, I see zero reason for this to exist, it's simply:

Quoted 8 lines
Quoted 5 lines
  kvm->arch.enable_pmu && enable_mediated_pmu && pmu->version;

And in literally every correct usage of pmu->passthrough, kvm->arch.enable_pmu and pmu->version have been checked (though implicitly), i.e. KVM can check enable_mediated_pmu and nothing else.

Ok, too many passthrough_pmu flags indeed confuse readers. Besides these dependencies, mediated vPMU also depends on lapic_in_kernel(). We need to set enable_mediated_pmu to false as well if lapic_in_kernel() returns false.

No, just kill the entire vPMU.

Also, the need for an in-kernel APIC isn't unique to the mediated PMU. KVM simply drops PMIs if there's no APIC.

If we're feeling lucky, we could try a breaking change like so:

arch/x86/kvm/pmu.c +1 -1
index fcd188cc389a..bb08155f6198 100644
@@ -817,7 +817,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
        pmu->pebs_data_cfg_mask = ~0ull;
        bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
 
-       if (!vcpu->kvm->arch.enable_pmu)
+       if (!vcpu->kvm->arch.enable_pmu || !lapic_in_kernel(vcpu))
                return;
 
        static_call(kvm_x86_pmu_refresh)(vcpu);

If we don't want to risk breaking weird setups, we could restrict the behavior to the mediated PMU being enabled:

arch/x86/kvm/pmu.c +0 -0
index fcd188cc389a..bc9673190574 100644

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 824

Quoted 9 lines
arch/x86/kvm/pmu.c +2 -1
@@ -817,7 +817,8 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
        pmu->pebs_data_cfg_mask = ~0ull;
        bitmap_zero(pmu->all_valid_pmc_idx, X86_PMC_IDX_MAX);
 
-       if (!vcpu->kvm->arch.enable_pmu)
+       if (!vcpu->kvm->arch.enable_pmu ||
+           (!lapic_in_kernel(vcpu) && enable_mediated_pmu))
                return;
 
        static_call(kvm_x86_pmu_refresh)(vcpu);

Sure. would adopt the latter one for safe. :)

#74 lore raw

RFC PATCH v3 20/58 KVM: x86/pmu: Always set global enable bits in passthrough mode

In reply to Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +1 -1
index 5768ea2935e9..e656f72fdace 100644
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);
 }
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#75 lore raw

Re: [RFC PATCH v3 20/58] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 31 lines · Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index 5768ea2935e9..e656f72fdace 100644

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 791

Quoted 6 lines
arch/x86/kvm/pmu.c +1 -1
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);

This is wrong and confusing. From the guest's perspective, and therefore from host userspace's perspective, PERF_GLOBAL_CTRL does not exist. Therefore, the value that is tracked for the guest must be '0'.

I see that intel_passthrough_pmu_msrs() and amd_passthrough_pmu_msrs() intercept accesses to PERF_GLOBAL_CTRL if "pmu->version > 1" (which, by the by, needs to be kvm_pmu_has_perf_global_ctrl()), so there's no weirdness with the guest being able to access MSRs that shouldn't exist.

But KVM shouldn't stuff pmu->global_ctrl, and doing so is a symptom of another flaw. Unless I'm missing something, KVM stuffs pmu->global_ctrl so that the correct value is loaded on VM-Enter, but loading and saving PERF_GLOBAL_CTRL on entry/exit is unnecessary and confusing, as is loading the associated MSRs when restoring (loading) the guest context.

For PERF_GLOBAL_CTRL on Intel, KVM needs to ensure all GP counters are enabled in VMCS.GUEST_IA32_PERF_GLOBAL_CTRL, but that's a "set once and forget" operation, not something that needs to be done on every entry and exit. Of course, loading and saving PERF_GLOBAL_CTRL on every entry/exit is unnecessary for other reasons, but that's largely orthogonal.

On AMD, amd_restore_pmu_context()[*] needs to enable a maximal value for PERF_GLOBAL_CTRL, but I don't think there's any need to load the other MSRs, and the maximal value should come from the above logic, not pmu->global_ctrl.

[*] Side topic, in case I forget later, that API should be "load", not "restore".
There is no assumption or guarantee that KVM is exactly restoring anything, e.g. if PERF_GLOBAL_CTRL doesn't exist in the guest PMU and on the first load.

#76 lore raw

Re: [RFC PATCH v3 20/58] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 59 lines · Sean Christopherson
Quoted 31 lines · Mingwei Zhang

From: Sandipan Das <[email protected]>

Currently, the global control bits for a vcpu are restored to the reset state only if the guest PMU version is less than 2. This works for emulated PMU as the MSRs are intercepted and backing events are created for and managed by the host PMU [1].

If such a guest in run with passthrough PMU, the counters no longer work because the global enable bits are cleared. Hence, set the global enable bits to their reset state if passthrough PMU is used.

A passthrough-capable host may not necessarily support PMU version 2 and it can choose to restore or save the global control state from struct kvm_pmu in the PMU context save and restore helpers depending on the availability of the global control register.

[1] 7b46b733bdb4 ("KVM: x86/pmu: Set enable bits for GP counters in PERF_GLOBAL_CTRL at "RESET"");

---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index 5768ea2935e9..e656f72fdace 100644

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 791

Quoted 6 lines
arch/x86/kvm/pmu.c +1 -1
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);

This is wrong and confusing. From the guest's perspective, and therefore from host userspace's perspective, PERF_GLOBAL_CTRL does not exist. Therefore, the value that is tracked for the guest must be '0'.

I see that intel_passthrough_pmu_msrs() and amd_passthrough_pmu_msrs() intercept accesses to PERF_GLOBAL_CTRL if "pmu->version > 1" (which, by the by, needs to be kvm_pmu_has_perf_global_ctrl()), so there's no weirdness with the guest being able to access MSRs that shouldn't exist.

But KVM shouldn't stuff pmu->global_ctrl, and doing so is a symptom of another flaw. Unless I'm missing something, KVM stuffs pmu->global_ctrl so that the correct value is loaded on VM-Enter, but loading and saving PERF_GLOBAL_CTRL on entry/exit is unnecessary and confusing, as is loading the associated MSRs when restoring (loading) the guest context.

For PERF_GLOBAL_CTRL on Intel, KVM needs to ensure all GP counters are enabled in VMCS.GUEST_IA32_PERF_GLOBAL_CTRL, but that's a "set once and forget" operation, not something that needs to be done on every entry and exit. Of course, loading and saving PERF_GLOBAL_CTRL on every entry/exit is unnecessary for other reasons, but that's largely orthogonal.

Quoted 3 lines

On AMD, amd_restore_pmu_context()[*] needs to enable a maximal value for PERF_GLOBAL_CTRL, but I don't think there's any need to load the other MSRs, and the maximal value should come from the above logic, not pmu->global_ctrl.

Sean, just double confirm, you are suggesting to do one-shot initialization for guest PERF_GLOBAL_CTRL (VMCS.GUEST_IA32_PERF_GLOBAL_CTRL for Intel) after vCPU resets, right?

Quoted 4 lines

[*] Side topic, in case I forget later, that API should be "load", not "restore".
There is no assumption or guarantee that KVM is exactly restoring anything, e.g. if PERF_GLOBAL_CTRL doesn't exist in the guest PMU and on the first load.

Sure.

#77 lore raw

Re: [RFC PATCH v3 20/58] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 37 lines · Dapeng Mi
Quoted 36 lines · Sean Christopherson
Quoted 9 lines
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index 5768ea2935e9..e656f72fdace 100644

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 791

Quoted 6 lines
arch/x86/kvm/pmu.c +1 -1
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);

This is wrong and confusing. From the guest's perspective, and therefore from host userspace's perspective, PERF_GLOBAL_CTRL does not exist. Therefore, the value that is tracked for the guest must be '0'.

I see that intel_passthrough_pmu_msrs() and amd_passthrough_pmu_msrs() intercept accesses to PERF_GLOBAL_CTRL if "pmu->version > 1" (which, by the by, needs to be kvm_pmu_has_perf_global_ctrl()), so there's no weirdness with the guest being able to access MSRs that shouldn't exist.

But KVM shouldn't stuff pmu->global_ctrl, and doing so is a symptom of another flaw. Unless I'm missing something, KVM stuffs pmu->global_ctrl so that the correct value is loaded on VM-Enter, but loading and saving PERF_GLOBAL_CTRL on entry/exit is unnecessary and confusing, as is loading the associated MSRs when restoring (loading) the guest context.

For PERF_GLOBAL_CTRL on Intel, KVM needs to ensure all GP counters are enabled in VMCS.GUEST_IA32_PERF_GLOBAL_CTRL, but that's a "set once and forget" operation, not something that needs to be done on every entry and exit. Of course, loading and saving PERF_GLOBAL_CTRL on every entry/exit is unnecessary for other reasons, but that's largely orthogonal.

Quoted 7 lines
Quoted 3 lines

On AMD, amd_restore_pmu_context()[*] needs to enable a maximal value for PERF_GLOBAL_CTRL, but I don't think there's any need to load the other MSRs, and the maximal value should come from the above logic, not pmu->global_ctrl.

Sean, just double confirm, you are suggesting to do one-shot initialization for guest PERF_GLOBAL_CTRL (VMCS.GUEST_IA32_PERF_GLOBAL_CTRL for Intel) after vCPU resets, right?

No, it would need to be written during refresh(). VMCS.GUEST_IA32_PERF_GLOBAL_CTRL is only static (because it's unreachable) if the guest does NOT have version > 1.

#78 lore raw

Re: [RFC PATCH v3 20/58] KVM: x86/pmu: Always set global enable bits in passthrough mode

Quoted 38 lines · Sean Christopherson
Quoted 37 lines · Dapeng Mi
Quoted 36 lines · Sean Christopherson
Quoted 9 lines
---
 arch/x86/kvm/pmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/pmu.c +0 -0
index 5768ea2935e9..e656f72fdace 100644

arch/x86/kvm/pmu.c void kvm_pmu_refresh(struct kvm_vcpu *vcpu) · line 791

Quoted 6 lines
arch/x86/kvm/pmu.c +1 -1
@@ -787,7 +787,7 @@ void kvm_pmu_refresh(struct kvm_vcpu *vcpu)
 	 * in the global controls).  Emulate that behavior when refreshing the
 	 * PMU so that userspace doesn't need to manually set PERF_GLOBAL_CTRL.
 	 */
-	if (kvm_pmu_has_perf_global_ctrl(pmu) && pmu->nr_arch_gp_counters)
+	if ((pmu->passthrough || kvm_pmu_has_perf_global_ctrl(pmu)) && pmu->nr_arch_gp_counters)
 		pmu->global_ctrl = GENMASK_ULL(pmu->nr_arch_gp_counters - 1, 0);

This is wrong and confusing. From the guest's perspective, and therefore from host userspace's perspective, PERF_GLOBAL_CTRL does not exist. Therefore, the value that is tracked for the guest must be '0'.

I see that intel_passthrough_pmu_msrs() and amd_passthrough_pmu_msrs() intercept accesses to PERF_GLOBAL_CTRL if "pmu->version > 1" (which, by the by, needs to be kvm_pmu_has_perf_global_ctrl()), so there's no weirdness with the guest being able to access MSRs that shouldn't exist.

But KVM shouldn't stuff pmu->global_ctrl, and doing so is a symptom of another flaw. Unless I'm missing something, KVM stuffs pmu->global_ctrl so that the correct value is loaded on VM-Enter, but loading and saving PERF_GLOBAL_CTRL on entry/exit is unnecessary and confusing, as is loading the associated MSRs when restoring (loading) the guest context.

For PERF_GLOBAL_CTRL on Intel, KVM needs to ensure all GP counters are enabled in VMCS.GUEST_IA32_PERF_GLOBAL_CTRL, but that's a "set once and forget" operation, not something that needs to be done on every entry and exit. Of course, loading and saving PERF_GLOBAL_CTRL on every entry/exit is unnecessary for other reasons, but that's largely orthogonal.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

On AMD, amd_restore_pmu_context()[*] needs to enable a maximal value for PERF_GLOBAL_CTRL, but I don't think there's any need to load the other MSRs, and the maximal value should come from the above logic, not pmu->global_ctrl.

Sean, just double confirm, you are suggesting to do one-shot initialization for guest PERF_GLOBAL_CTRL (VMCS.GUEST_IA32_PERF_GLOBAL_CTRL for Intel) after vCPU resets, right?

No, it would need to be written during refresh(). VMCS.GUEST_IA32_PERF_GLOBAL_CTRL is only static (because it's unreachable) if the guest does NOT have version > 1.

oh, yeah, refresh() instead of reset() to be exact.

Quoted 1 line
#79 lore raw

RFC PATCH v3 21/58 KVM: x86/pmu: Add a helper to check if passthrough PMU is enabled

In reply to Mingwei Zhang

Add a helper to check if passthrough PMU is enabled for convenience as it is vendor neutral.

---
 arch/x86/kvm/pmu.h | 5 +++++
 1 file changed, 5 insertions(+)
arch/x86/kvm/pmu.h +5 -0
index cf93be5e7359..56ba0772568c 100644
@@ -48,6 +48,11 @@ struct kvm_pmu_ops {
 
 void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops);
 
+static inline bool is_passthrough_pmu_enabled(struct kvm_vcpu *vcpu)
+{
+	return vcpu_to_pmu(vcpu)->passthrough;
+}
+
 static inline bool kvm_pmu_has_perf_global_ctrl(struct kvm_pmu *pmu)
 {
 	/*
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#80 lore raw

RFC PATCH v3 22/58 KVM: x86/pmu: Add host_perf_cap and initialize it in kvm_x86_vendor_init()

In reply to Mingwei Zhang

Initialize host_perf_cap early in kvm_x86_vendor_init(). This helps KVM recognize the HW PMU capability against its guest VMs. This awareness directly decides the feasibility of passing through RDPMC and indirectly affects the performance in PMU context switch. Having the host PMU feature set cached in host_perf_cap saves a rdmsrl() to IA32_PERF_CAPABILITY MSR on each PMU context switch.

In addition, just opportunistically remove the host_perf_cap initialization in vmx_get_perf_capabilities() so the value is not dependent on module parameter "enable_pmu".

---
 arch/x86/kvm/pmu.h     | 1 +
 arch/x86/kvm/vmx/vmx.c | 4 ----
 arch/x86/kvm/x86.c     | 6 ++++++
 3 files changed, 7 insertions(+), 4 deletions(-)
arch/x86/kvm/pmu.h +1 -0
index 56ba0772568c..e041c8a23e2f 100644
@@ -295,4 +295,5 @@ bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
 extern struct kvm_pmu_ops intel_pmu_ops;
 extern struct kvm_pmu_ops amd_pmu_ops;
+extern u64 __read_mostly host_perf_cap;
 #endif /* __KVM_X86_PMU_H */
arch/x86/kvm/vmx/vmx.c +0 -4
index 2ad122995f11..4d60a8cf2dd1 100644
@@ -7918,14 +7918,10 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 static __init u64 vmx_get_perf_capabilities(void)
 {
 	u64 perf_cap = PMU_CAP_FW_WRITES;
-	u64 host_perf_cap = 0;
 
 	if (!enable_pmu)
 		return 0;
 
-	if (boot_cpu_has(X86_FEATURE_PDCM))
-		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
-
 	if (!cpu_feature_enabled(X86_FEATURE_ARCH_LBR) &&
 	    !enable_passthrough_pmu) {
 		x86_perf_get_lbr(&vmx_lbr_caps);
arch/x86/kvm/x86.c +6 -0
index 0c40f551130e..6db4dc496d2b 100644
@@ -239,6 +239,9 @@ EXPORT_SYMBOL_GPL(host_xss);
 u64 __read_mostly host_arch_capabilities;
 EXPORT_SYMBOL_GPL(host_arch_capabilities);
 
+u64 __read_mostly host_perf_cap;
+EXPORT_SYMBOL_GPL(host_perf_cap);
+
 const struct _kvm_stats_desc kvm_vm_stats_desc[] = {
 	KVM_GENERIC_VM_STATS(),
 	STATS_DESC_COUNTER(VM, mmu_shadow_zapped),
@@ -9793,6 +9796,9 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
 		rdmsrl(MSR_IA32_ARCH_CAPABILITIES, host_arch_capabilities);
 
+	if (boot_cpu_has(X86_FEATURE_PDCM))
+		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
+
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#81 lore raw

Re: [RFC PATCH v3 22/58] KVM: x86/pmu: Add host_perf_cap and initialize it in kvm_x86_vendor_init()

arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(host_xss); · line 243

Quoted 10 lines · Mingwei Zhang
arch/x86/kvm/x86.c +2 -0
index 0c40f551130e..6db4dc496d2b 100644
@@ -239,6 +239,9 @@ EXPORT_SYMBOL_GPL(host_xss);
 u64 __read_mostly host_arch_capabilities;
 EXPORT_SYMBOL_GPL(host_arch_capabilities);
 
+u64 __read_mostly host_perf_cap;
+EXPORT_SYMBOL_GPL(host_perf_cap);

In case you don't get a conflict on rebase, this should go in "struct kvm_host_values" as "perf_capabilities".

Quoted 16 lines
arch/x86/kvm/x86.c +3 -0
 const struct _kvm_stats_desc kvm_vm_stats_desc[] = {
 	KVM_GENERIC_VM_STATS(),
 	STATS_DESC_COUNTER(VM, mmu_shadow_zapped),
@@ -9793,6 +9796,9 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
 		rdmsrl(MSR_IA32_ARCH_CAPABILITIES, host_arch_capabilities);
 
+	if (boot_cpu_has(X86_FEATURE_PDCM))
+		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
+
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#82 lore raw

Re: [RFC PATCH v3 22/58] KVM: x86/pmu: Add host_perf_cap and initialize it in kvm_x86_vendor_init()

Quoted 9 lines · Sean Christopherson
Quoted 8 lines · Mingwei Zhang
arch/x86/kvm/x86.c +0 -0
index 0c40f551130e..6db4dc496d2b 100644
@@ -239,6 +239,9 @@ EXPORT_SYMBOL_GPL(host_xss);
 u64 __read_mostly host_arch_capabilities;
 EXPORT_SYMBOL_GPL(host_arch_capabilities);
 
Quoted 4 lines

arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(host_xss); · line 243

Quoted 2 lines
arch/x86/kvm/x86.c +2 -0
+u64 __read_mostly host_perf_cap;
+EXPORT_SYMBOL_GPL(host_perf_cap);

In case you don't get a conflict on rebase, this should go in "struct kvm_host_values" as "perf_capabilities".

Sure. Thanks.

Quoted 17 lines
Quoted 17 lines
arch/x86/kvm/x86.c +3 -0
 const struct _kvm_stats_desc kvm_vm_stats_desc[] = {
 	KVM_GENERIC_VM_STATS(),
 	STATS_DESC_COUNTER(VM, mmu_shadow_zapped),
@@ -9793,6 +9796,9 @@ int kvm_x86_vendor_init(struct kvm_x86_init_ops *ops)
 	if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
 		rdmsrl(MSR_IA32_ARCH_CAPABILITIES, host_arch_capabilities);
 
+	if (boot_cpu_has(X86_FEATURE_PDCM))
+		rdmsrl(MSR_IA32_PERF_CAPABILITIES, host_perf_cap);
+
 	r = ops->hardware_setup();
 	if (r != 0)
 		goto out_mmu_exit;
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#83 lore raw

RFC PATCH v3 23/58 KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

In reply to Mingwei Zhang

Clear RDPMC_EXITING in vmcs when all counters on the host side are exposed to guest VM. This gives performance to passthrough PMU. However, when guest does not get all counters, intercept RDPMC to prevent access to unexposed counters. Make decision in vmx_vcpu_after_set_cpuid() when guest enables PMU and passthrough PMU is enabled.

---
 arch/x86/kvm/pmu.c     | 16 ++++++++++++++++
 arch/x86/kvm/pmu.h     |  1 +
 arch/x86/kvm/vmx/vmx.c |  5 +++++
 3 files changed, 22 insertions(+)
arch/x86/kvm/pmu.c +16 -0
index e656f72fdace..19104e16a986 100644
@@ -96,6 +96,22 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu) &&
+	    !enable_vmware_backdoor &&
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
+	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
+	    pmu->counter_bitmask[KVM_PMC_FIXED] == (((u64)1 << kvm_pmu_cap.bit_width_fixed)  - 1))
+		return true;
+
+	return false;
+}
+EXPORT_SYMBOL_GPL(kvm_pmu_check_rdpmc_passthrough);
+
 static inline void __kvm_perf_overflow(struct kvm_pmc *pmc, bool in_pmi)
 {
 	struct kvm_pmu *pmu = pmc_to_pmu(pmc);
arch/x86/kvm/pmu.h +1 -0
index e041c8a23e2f..91941a0f6e47 100644
@@ -290,6 +290,7 @@ void kvm_pmu_cleanup(struct kvm_vcpu *vcpu);
 void kvm_pmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
arch/x86/kvm/vmx/vmx.c +5 -0
index 4d60a8cf2dd1..339742350b7a 100644
@@ -7911,6 +7911,11 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);
+
 	/* Refresh #PF interception to account for MAXPHYADDR changes. */
 	vmx_update_exception_bitmap(vcpu);
 }
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#84 lore raw

Re: [RFC PATCH v3 23/58] KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

Quoted 22 lines · Mingwei Zhang

Clear RDPMC_EXITING in vmcs when all counters on the host side are exposed to guest VM. This gives performance to passthrough PMU. However, when guest does not get all counters, intercept RDPMC to prevent access to unexposed counters. Make decision in vmx_vcpu_after_set_cpuid() when guest enables PMU and passthrough PMU is enabled.

---
 arch/x86/kvm/pmu.c     | 16 ++++++++++++++++
 arch/x86/kvm/pmu.h     |  1 +
 arch/x86/kvm/vmx/vmx.c |  5 +++++
 3 files changed, 22 insertions(+)
arch/x86/kvm/pmu.c +0 -0
index e656f72fdace..19104e16a986 100644

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 99

Quoted 4 lines
arch/x86/kvm/pmu.c +1 -0
@@ -96,6 +96,22 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)

As suggested earlier, kvm_rdpmc_in_guest().

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 104

Quoted 5 lines
arch/x86/kvm/pmu.c +5 -0
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu) &&
+	    !enable_vmware_backdoor &&

Please add a comment about the VMware backdoor, I doubt most folks know about VMware's tweaks to RDPMC behavior. It's somewhat obvious from the code and comment in check_rdpmc(), but I think it's worth calling out here too.

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 108

Quoted 4 lines
arch/x86/kvm/pmu.c +4 -0
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
+	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
+	    pmu->counter_bitmask[KVM_PMC_FIXED] == (((u64)1 << kvm_pmu_cap.bit_width_fixed)  - 1))

BIT_ULL? GENMASK_ULL?

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 111

Quoted 3 lines
arch/x86/kvm/pmu.c +3 -0
+		return true;
+
+	return false;

Do this:

	return <true>;

not:

	if (<true>)
		return true;

	return false;

Short-circuiting on certain cases is fine, and I would probably vote for that so it's easier to add comments, but that's obviously not what's done here. E.g. either

	if (!enable_mediated_pmu)
		return false;

	/* comment goes here */
	if (enable_vmware_backdoor)
		return false;

	return <counters checks>;

or

	return <massive combined check>;

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 113

Quoted 2 lines
arch/x86/kvm/pmu.c +2 -0
+}
+EXPORT_SYMBOL_GPL(kvm_pmu_check_rdpmc_passthrough);

Maybe just make this an inline in a header? enable_vmware_backdoor is exported, and presumably enable_mediated_pmu will be too.

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7914

Quoted 9 lines
arch/x86/kvm/vmx/vmx.c +1 -0
index 4d60a8cf2dd1..339742350b7a 100644
@@ -7911,6 +7911,11 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))

No need to follow vmx->vcpu, @vcpu is readily available.

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7917

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);

I wonder if it makes sense to add a helper to change a bit. IIRC, the only reason I didn't add one along with the set/clear helpers was because there weren't many users and I couldn't think of good alternative to "set".

I still don't have a good name, but I think we're reaching the point where it's worth forcing the issue to avoid common goofs, e.g. handling only the "clear" case and no the "set" case.

Maybe changebit? E.g.

static __always_inline void lname##_controls_changebit(struct vcpu_vmx *vmx, u##bits val,	\
						       bool set)				\
{												\
	if (set)										\
		lname##_controls_setbit(vmx, val);						\
	else											\
		lname##_controls_clearbit(vmx, val);						\
}

and then vmx_refresh_apicv_exec_ctrl() can be:

	secondary_exec_controls_changebit(vmx,
					  SECONDARY_EXEC_APIC_REGISTER_VIRT |
					  SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY,
					  kvm_vcpu_apicv_active(vcpu));
	tertiary_exec_controls_changebit(vmx, TERTIARY_EXEC_IPI_VIRT,
					 kvm_vcpu_apicv_active(vcpu) && enable_ipiv);

and this can be:

	exec_controls_changebit(vmx, CPU_BASED_RDPMC_EXITING,
				!kvm_rdpmc_in_guest(vcpu));
#85 lore raw

Re: [RFC PATCH v3 23/58] KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

Quoted 26 lines · Sean Christopherson
Quoted 25 lines · Mingwei Zhang

Clear RDPMC_EXITING in vmcs when all counters on the host side are exposed to guest VM. This gives performance to passthrough PMU. However, when guest does not get all counters, intercept RDPMC to prevent access to unexposed counters. Make decision in vmx_vcpu_after_set_cpuid() when guest enables PMU and passthrough PMU is enabled.

---
 arch/x86/kvm/pmu.c     | 16 ++++++++++++++++
 arch/x86/kvm/pmu.h     |  1 +
 arch/x86/kvm/vmx/vmx.c |  5 +++++
 3 files changed, 22 insertions(+)
arch/x86/kvm/pmu.c +0 -0
index e656f72fdace..19104e16a986 100644
@@ -96,6 +96,22 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
Quoted 2 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 99

Quoted 1 line
arch/x86/kvm/pmu.c +1 -0
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)

As suggested earlier, kvm_rdpmc_in_guest().

Sure.

Quoted 9 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 104

Quoted 6 lines
arch/x86/kvm/pmu.c +5 -0
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu) &&
+	    !enable_vmware_backdoor &&

Please add a comment about the VMware backdoor, I doubt most folks know about VMware's tweaks to RDPMC behavior. It's somewhat obvious from the code and comment in check_rdpmc(), but I think it's worth calling out here too.

Sure.

Quoted 6 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 108

Quoted 5 lines
arch/x86/kvm/pmu.c +4 -0
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
+	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
+	    pmu->counter_bitmask[KVM_PMC_FIXED] == (((u64)1 << kvm_pmu_cap.bit_width_fixed)  - 1))

BIT_ULL? GENMASK_ULL?

Sure.

Quoted 28 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 111

Quoted 4 lines
arch/x86/kvm/pmu.c +3 -0
+		return true;
+
+	return false;

Do this:

	return <true>;

not:

	if (<true>)
		return true;

	return false;

Short-circuiting on certain cases is fine, and I would probably vote for that so it's easier to add comments, but that's obviously not what's done here. E.g. either

	if (!enable_mediated_pmu)
		return false;

	/* comment goes here */
	if (enable_vmware_backdoor)
		return false;

	return <counters checks>;
Quoted 3 lines

or

	return <massive combined check>;

Nice suggestion. Thanks.

Quoted 5 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 113

Quoted 3 lines
arch/x86/kvm/pmu.c +2 -0
+}
+EXPORT_SYMBOL_GPL(kvm_pmu_check_rdpmc_passthrough);

Maybe just make this an inline in a header? enable_vmware_backdoor is exported, and presumably enable_mediated_pmu will be too.

Sure.

Quoted 11 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7914

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +1 -0
index 4d60a8cf2dd1..339742350b7a 100644
@@ -7911,6 +7911,11 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))

No need to follow vmx->vcpu, @vcpu is readily available.

Yes.

Quoted 33 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7917

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);

I wonder if it makes sense to add a helper to change a bit. IIRC, the only reason I didn't add one along with the set/clear helpers was because there weren't many users and I couldn't think of good alternative to "set".

I still don't have a good name, but I think we're reaching the point where it's worth forcing the issue to avoid common goofs, e.g. handling only the "clear" case and no the "set" case.

Maybe changebit? E.g.

static __always_inline void lname##_controls_changebit(struct vcpu_vmx *vmx, u##bits val,	\
						       bool set)				\
{												\
	if (set)										\
		lname##_controls_setbit(vmx, val);						\
	else											\
		lname##_controls_clearbit(vmx, val);						\
}

and then vmx_refresh_apicv_exec_ctrl() can be:

	secondary_exec_controls_changebit(vmx,
					  SECONDARY_EXEC_APIC_REGISTER_VIRT |
					  SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY,
					  kvm_vcpu_apicv_active(vcpu));
	tertiary_exec_controls_changebit(vmx, TERTIARY_EXEC_IPI_VIRT,
					 kvm_vcpu_apicv_active(vcpu) && enable_ipiv);
Quoted 4 lines

and this can be:

	exec_controls_changebit(vmx, CPU_BASED_RDPMC_EXITING,
				!kvm_rdpmc_in_guest(vcpu));

Sure. would add a separate patch to add these helpers.

#86 lore raw

Re: [RFC PATCH v3 23/58] KVM: x86/pmu: Allow RDPMC pass through when all counters exposed to guest

Quoted 136 lines · Dapeng
Quoted 26 lines · Sean Christopherson
Quoted 25 lines · Mingwei Zhang

Clear RDPMC_EXITING in vmcs when all counters on the host side are exposed to guest VM. This gives performance to passthrough PMU. However, when guest does not get all counters, intercept RDPMC to prevent access to unexposed counters. Make decision in vmx_vcpu_after_set_cpuid() when guest enables PMU and passthrough PMU is enabled.

---
 arch/x86/kvm/pmu.c     | 16 ++++++++++++++++
 arch/x86/kvm/pmu.h     |  1 +
 arch/x86/kvm/vmx/vmx.c |  5 +++++
 3 files changed, 22 insertions(+)
arch/x86/kvm/pmu.c +0 -0
index e656f72fdace..19104e16a986 100644
@@ -96,6 +96,22 @@ void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops)
 #undef __KVM_X86_PMU_OP
 }
 
Quoted 2 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 99

Quoted 1 line
arch/x86/kvm/pmu.c +1 -0
+bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)

As suggested earlier, kvm_rdpmc_in_guest().

Sure.

Quoted 8 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 104

Quoted 5 lines
arch/x86/kvm/pmu.c +5 -0
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu) &&
+	    !enable_vmware_backdoor &&

Please add a comment about the VMware backdoor, I doubt most folks know about VMware's tweaks to RDPMC behavior. It's somewhat obvious from the code and comment in check_rdpmc(), but I think it's worth calling out here too.

Sure.

Quoted 5 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 108

Quoted 4 lines
arch/x86/kvm/pmu.c +4 -0
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
+	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
+	    pmu->counter_bitmask[KVM_PMC_FIXED] == (((u64)1 << kvm_pmu_cap.bit_width_fixed)  - 1))

BIT_ULL? GENMASK_ULL?

Sure.

Quoted 27 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 111

Quoted 3 lines
arch/x86/kvm/pmu.c +3 -0
+		return true;
+
+	return false;

Do this:

	return <true>;

not:

	if (<true>)
		return true;

	return false;

Short-circuiting on certain cases is fine, and I would probably vote for that so it's easier to add comments, but that's obviously not what's done here. E.g. either

	if (!enable_mediated_pmu)
		return false;

	/* comment goes here */
	if (enable_vmware_backdoor)
		return false;

	return <counters checks>;
Quoted 3 lines

or

	return <massive combined check>;

Nice suggestion. Thanks.

Quoted 4 lines

arch/x86/kvm/pmu.c void kvm_pmu_ops_update(const struct kvm_pmu_ops *pmu_ops) · line 113

Quoted 2 lines
arch/x86/kvm/pmu.c +2 -0
+}
+EXPORT_SYMBOL_GPL(kvm_pmu_check_rdpmc_passthrough);

Maybe just make this an inline in a header? enable_vmware_backdoor is exported, and presumably enable_mediated_pmu will be too.

Sure.

Quoted 10 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7914

Quoted 9 lines
arch/x86/kvm/vmx/vmx.c +1 -0
index 4d60a8cf2dd1..339742350b7a 100644
@@ -7911,6 +7911,11 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 		vmx->msr_ia32_feature_control_valid_bits &=
 			~FEAT_CTL_SGX_LC_ENABLED;
 
+	if (kvm_pmu_check_rdpmc_passthrough(&vmx->vcpu))

No need to follow vmx->vcpu, @vcpu is readily available.

Yes.

Quoted 32 lines

arch/x86/kvm/vmx/vmx.c void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · line 7917

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+		exec_controls_clearbit(vmx, CPU_BASED_RDPMC_EXITING);
+	else
+		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);

I wonder if it makes sense to add a helper to change a bit. IIRC, the only reason I didn't add one along with the set/clear helpers was because there weren't many users and I couldn't think of good alternative to "set".

I still don't have a good name, but I think we're reaching the point where it's worth forcing the issue to avoid common goofs, e.g. handling only the "clear" case and no the "set" case.

Maybe changebit? E.g.

static __always_inline void lname##_controls_changebit(struct vcpu_vmx *vmx, u##bits val,	\
						       bool set)				\
{												\
	if (set)										\
		lname##_controls_setbit(vmx, val);						\
	else											\
		lname##_controls_clearbit(vmx, val);						\
}

and then vmx_refresh_apicv_exec_ctrl() can be:

	secondary_exec_controls_changebit(vmx,
					  SECONDARY_EXEC_APIC_REGISTER_VIRT |
					  SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY,
					  kvm_vcpu_apicv_active(vcpu));
	tertiary_exec_controls_changebit(vmx, TERTIARY_EXEC_IPI_VIRT,
					 kvm_vcpu_apicv_active(vcpu) && enable_ipiv);
Quoted 5 lines
Quoted 4 lines

and this can be:

	exec_controls_changebit(vmx, CPU_BASED_RDPMC_EXITING,
				!kvm_rdpmc_in_guest(vcpu));

Sure. would add a separate patch to add these helpers.

Hi Sean,

The upcoming v4 patches would include some code which you suggest in the comments, like this one. I would like add a co-developed-by and corresponding SoB tags for you in the patch if the patch includes you suggested code. Is it ok? Or you more prefer the "suggested-by" tag? Thanks.

Quoted 3 lines
#87 lore raw

RFC PATCH v3 24/58 KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS

In reply to Mingwei Zhang

From: Dapeng Mi <[email protected]>

Define macro PMU_CAP_PERF_METRICS to represent bit[15] of MSR_IA32_PERF_CAPABILITIES MSR. This bit is used to represent whether perf metrics feature is enabled.

---
 arch/x86/kvm/vmx/capabilities.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/vmx/capabilities.h +1 -0
index 41a4533f9989..d8317552b634 100644
@@ -22,6 +22,7 @@ extern int __read_mostly pt_mode;
 #define PT_MODE_HOST_GUEST	1
 
 #define PMU_CAP_FW_WRITES	(1ULL << 13)
+#define PMU_CAP_PERF_METRICS	BIT_ULL(15)
 #define PMU_CAP_LBR_FMT		0x3f
 
 struct nested_vmx_msrs {
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#88 lore raw

Re: [RFC PATCH v3 24/58] KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS

Quoted 17 lines · Mingwei Zhang

From: Dapeng Mi <[email protected]>

Define macro PMU_CAP_PERF_METRICS to represent bit[15] of MSR_IA32_PERF_CAPABILITIES MSR. This bit is used to represent whether perf metrics feature is enabled.

---
 arch/x86/kvm/vmx/capabilities.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/vmx/capabilities.h +0 -0
index 41a4533f9989..d8317552b634 100644

arch/x86/kvm/vmx/capabilities.h extern int __read_mostly pt_mode; · line 25

Quoted 4 lines
arch/x86/kvm/vmx/capabilities.h +1 -0
@@ -22,6 +22,7 @@ extern int __read_mostly pt_mode;
 #define PT_MODE_HOST_GUEST	1
 
 #define PMU_CAP_FW_WRITES	(1ULL << 13)
+#define PMU_CAP_PERF_METRICS	BIT_ULL(15)

BIT() should suffice. The 1ULL used for FW_WRITES is unnecessary. Speaking of which, can you update the other #defines while you're at it? The mix of styles annoys me :-)

#define PMU_CAP_FW_WRITES BIT(13)
#define PMU_CAP_PERF_METRICS BIT(15)
#define PMU_CAP_LBR_FMT GENMASK(5, 0)

Quoted 6 lines
arch/x86/kvm/vmx/capabilities.h +0 -0
 #define PMU_CAP_LBR_FMT		0x3f
 
 struct nested_vmx_msrs {
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#89 lore raw

Re: [RFC PATCH v3 24/58] KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS

Quoted 26 lines · Sean Christopherson
Quoted 17 lines · Mingwei Zhang

From: Dapeng Mi <[email protected]>

Define macro PMU_CAP_PERF_METRICS to represent bit[15] of MSR_IA32_PERF_CAPABILITIES MSR. This bit is used to represent whether perf metrics feature is enabled.

---
 arch/x86/kvm/vmx/capabilities.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/vmx/capabilities.h +0 -0
index 41a4533f9989..d8317552b634 100644

arch/x86/kvm/vmx/capabilities.h extern int __read_mostly pt_mode; · line 25

Quoted 4 lines
arch/x86/kvm/vmx/capabilities.h +1 -0
@@ -22,6 +22,7 @@ extern int __read_mostly pt_mode;
 #define PT_MODE_HOST_GUEST	1
 
 #define PMU_CAP_FW_WRITES	(1ULL << 13)
+#define PMU_CAP_PERF_METRICS	BIT_ULL(15)

BIT() should suffice. The 1ULL used for FW_WRITES is unnecessary. Speaking of which, can you update the other #defines while you're at it? The mix of styles annoys me :-)

Quoted 3 lines

#define PMU_CAP_FW_WRITES BIT(13)
#define PMU_CAP_PERF_METRICS BIT(15)
#define PMU_CAP_LBR_FMT GENMASK(5, 0)

Sure.  Could we further move all these  PERF_CAPBILITIES macros into arch/x86/include/asm/msr-index.h? I just found there are already some PERF_CAPBILITIES macros defined in this file and it looks a better place to define these macros.

#define PERF_CAP_PEBS_TRAP             BIT_ULL(6) #define PERF_CAP_ARCH_REG              BIT_ULL(7) #define PERF_CAP_PEBS_FORMAT           0xf00 #define PERF_CAP_PEBS_BASELINE         BIT_ULL(14)

Quoted 6 lines
Quoted 6 lines
arch/x86/kvm/vmx/capabilities.h +0 -0
 #define PMU_CAP_LBR_FMT		0x3f
 
 struct nested_vmx_msrs {
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#90 lore raw

Re: [RFC PATCH v3 24/58] KVM: x86/pmu: Introduce macro PMU_CAP_PERF_METRICS

Quoted 28 lines · Dapeng Mi
Quoted 26 lines · Sean Christopherson
Quoted 17 lines · Mingwei Zhang

From: Dapeng Mi <[email protected]>

Define macro PMU_CAP_PERF_METRICS to represent bit[15] of MSR_IA32_PERF_CAPABILITIES MSR. This bit is used to represent whether perf metrics feature is enabled.

---
 arch/x86/kvm/vmx/capabilities.h | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/vmx/capabilities.h +0 -0
index 41a4533f9989..d8317552b634 100644

arch/x86/kvm/vmx/capabilities.h extern int __read_mostly pt_mode; · line 25

Quoted 4 lines
arch/x86/kvm/vmx/capabilities.h +1 -0
@@ -22,6 +22,7 @@ extern int __read_mostly pt_mode;
 #define PT_MODE_HOST_GUEST	1
 
 #define PMU_CAP_FW_WRITES	(1ULL << 13)
+#define PMU_CAP_PERF_METRICS	BIT_ULL(15)

BIT() should suffice. The 1ULL used for FW_WRITES is unnecessary. Speaking of which, can you update the other #defines while you're at it? The mix of styles annoys me :-)

Quoted 6 lines
Quoted 3 lines

#define PMU_CAP_FW_WRITES BIT(13)
#define PMU_CAP_PERF_METRICS BIT(15)
#define PMU_CAP_LBR_FMT GENMASK(5, 0)

Sure.  Could we further move all these  PERF_CAPBILITIES macros into arch/x86/include/asm/msr-index.h?

Yes, definitely. I didn't even realize this was KVM code.

#91 lore raw

RFC PATCH v3 25/58 KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough allowed

In reply to Mingwei Zhang

Introduce a vendor specific API to check if rdpmc passthrough allowed. RDPMC passthrough requires guest VM have the full ownership of all counters. These include general purpose counters and fixed counters and some vendor specific MSRs such as PERF_METRICS. Since PERF_METRICS MSR is Intel specific, putting the check into vendor specific code.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  1 +
 arch/x86/kvm/pmu.c                     |  1 +
 arch/x86/kvm/pmu.h                     |  1 +
 arch/x86/kvm/svm/pmu.c                 |  6 ++++++
 arch/x86/kvm/vmx/pmu_intel.c           | 16 ++++++++++++++++
 5 files changed, 25 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index f852b13aeefe..fd986d5146e4 100644
@@ -20,6 +20,7 @@ KVM_X86_PMU_OP(get_msr)
 KVM_X86_PMU_OP(set_msr)
 KVM_X86_PMU_OP(refresh)
 KVM_X86_PMU_OP(init)
+KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
arch/x86/kvm/pmu.c +1 -0
index 19104e16a986..3afefe4cf6e2 100644
@@ -102,6 +102,7 @@ bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)
 
 	if (is_passthrough_pmu_enabled(vcpu) &&
 	    !enable_vmware_backdoor &&
+	    static_call(kvm_x86_pmu_is_rdpmc_passthru_allowed)(vcpu) &&
 	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
 	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed &&
 	    pmu->counter_bitmask[KVM_PMC_GP] == (((u64)1 << kvm_pmu_cap.bit_width_gp) - 1) &&
arch/x86/kvm/pmu.h +1 -0
index 91941a0f6e47..e1af6d07b191 100644
@@ -40,6 +40,7 @@ struct kvm_pmu_ops {
 	void (*reset)(struct kvm_vcpu *vcpu);
 	void (*deliver_pmi)(struct kvm_vcpu *vcpu);
 	void (*cleanup)(struct kvm_vcpu *vcpu);
+	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
arch/x86/kvm/svm/pmu.c +6 -0
index dfcc38bd97d3..6b471b1ec9b8 100644
@@ -228,6 +228,11 @@ static void amd_pmu_init(struct kvm_vcpu *vcpu)
 	}
 }
 
+static bool amd_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
+{
+	return true;
+}
+
 struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = amd_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = amd_msr_idx_to_pmc,
@@ -237,6 +242,7 @@ struct kvm_pmu_ops amd_pmu_ops __initdata = {
 	.set_msr = amd_pmu_set_msr,
 	.refresh = amd_pmu_refresh,
 	.init = amd_pmu_init,
+	.is_rdpmc_passthru_allowed = amd_is_rdpmc_passthru_allowed,
 	.EVENTSEL_EVENT = AMD64_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_AMD_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = AMD64_NUM_COUNTERS,
arch/x86/kvm/vmx/pmu_intel.c +16 -0
index e417fd91e5fe..02c9019c6f85 100644
@@ -725,6 +725,21 @@ void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu)
 	}
 }
 
+static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Per Intel SDM vol. 2 for RDPMC, MSR_PERF_METRICS is accessible by
+	 * with type 0x2000 in ECX[31:16], while the index value in ECX[15:0] is
+	 * implementation specific. Therefore, if the host has this MSR, but
+	 * does not expose it to the guest, RDPMC has to be intercepted.
+	 */
+	if ((host_perf_cap & PMU_CAP_PERF_METRICS) &&
+	    !(vcpu_get_perf_capabilities(vcpu) & PMU_CAP_PERF_METRICS))
+		return false;
+
+	return true;
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -736,6 +751,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.reset = intel_pmu_reset,
 	.deliver_pmi = intel_pmu_deliver_pmi,
 	.cleanup = intel_pmu_cleanup,
+	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#92 lore raw

Re: [RFC PATCH v3 25/58] KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough allowed

Quoted 33 lines · Mingwei Zhang

Introduce a vendor specific API to check if rdpmc passthrough allowed. RDPMC passthrough requires guest VM have the full ownership of all counters. These include general purpose counters and fixed counters and some vendor specific MSRs such as PERF_METRICS. Since PERF_METRICS MSR is Intel specific, putting the check into vendor specific code.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  1 +
 arch/x86/kvm/pmu.c                     |  1 +
 arch/x86/kvm/pmu.h                     |  1 +
 arch/x86/kvm/svm/pmu.c                 |  6 ++++++
 arch/x86/kvm/vmx/pmu_intel.c           | 16 ++++++++++++++++
 5 files changed, 25 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index f852b13aeefe..fd986d5146e4 100644
@@ -20,6 +20,7 @@ KVM_X86_PMU_OP(get_msr)
 KVM_X86_PMU_OP(set_msr)
 KVM_X86_PMU_OP(refresh)
 KVM_X86_PMU_OP(init)
+KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
arch/x86/kvm/pmu.c +0 -0
index 19104e16a986..3afefe4cf6e2 100644

arch/x86/kvm/pmu.c bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu) · line 105

Quoted 4 lines
arch/x86/kvm/pmu.c +1 -0
@@ -102,6 +102,7 @@ bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)
 
 	if (is_passthrough_pmu_enabled(vcpu) &&
 	    !enable_vmware_backdoor &&
+	    static_call(kvm_x86_pmu_is_rdpmc_passthru_allowed)(vcpu) &&

If the polarity is inverted, the callback can be OPTIONAL_RET0 on AMD. E.g.

	if (kvm_pmu_call(rdpmc_needs_intercept(vcpu)))
		return false;

arch/x86/kvm/pmu.c bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)

Quoted 4 lines
arch/x86/kvm/pmu.c +4 -0
+static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Per Intel SDM vol. 2 for RDPMC, 

Please don't reference specific sections in the comments. For changelogs it's ok, because changelogs are a snapshot in time. But comments are living things and will become stale in almost every case. And I don't see any reason to reference the SDM, just state the behavior; it's implied that that's the architectural behavior, otherwise KVM is buggy.

Quoted 1 line

MSR_PERF_METRICS is accessible by

This is technically wrong, the SDM states that the RDPMC behavior is implementation specific. That matters to some extent, because if it was _just_ one MSR and was guaranteed to always be that one MSR, it might be worth creating a virtualization hole.

/*
* Intercept RDPMC if the host supports PERF_METRICS, but the guest
* does not, as RDPMC with type 0x2000 accesses implementation specific
* metrics.
*/

All that said, isn't this redundant with the number of fixed counters? I'm having a hell of a time finding anything concrete in the SDM, but IIUC fixed counter 3 is tightly coupled to perf metrics. E.g. rather than add a vendor hook just for this, rely on the fixed counters and refuse to enable the mediated PMU if the underlying CPU model is nonsensical, i.e. perf metrics exists without ctr3.

And I kinda think we have to go that route, because enabling RDPMC interception based on future features is doomed from the start. E.g. if this code had been written prior to PERF_METRICS, older KVMs would have zero clue that RDPMC needs to be intercepted on newer hardware.

#93 lore raw

Re: [RFC PATCH v3 25/58] KVM: x86/pmu: Introduce PMU operator to check if rdpmc passthrough allowed

Quoted 73 lines · Sean Christopherson
Quoted 33 lines · Mingwei Zhang

Introduce a vendor specific API to check if rdpmc passthrough allowed. RDPMC passthrough requires guest VM have the full ownership of all counters. These include general purpose counters and fixed counters and some vendor specific MSRs such as PERF_METRICS. Since PERF_METRICS MSR is Intel specific, putting the check into vendor specific code.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  1 +
 arch/x86/kvm/pmu.c                     |  1 +
 arch/x86/kvm/pmu.h                     |  1 +
 arch/x86/kvm/svm/pmu.c                 |  6 ++++++
 arch/x86/kvm/vmx/pmu_intel.c           | 16 ++++++++++++++++
 5 files changed, 25 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index f852b13aeefe..fd986d5146e4 100644
@@ -20,6 +20,7 @@ KVM_X86_PMU_OP(get_msr)
 KVM_X86_PMU_OP(set_msr)
 KVM_X86_PMU_OP(refresh)
 KVM_X86_PMU_OP(init)
+KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
arch/x86/kvm/pmu.c +0 -0
index 19104e16a986..3afefe4cf6e2 100644

arch/x86/kvm/pmu.c bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu) · line 105

Quoted 4 lines
arch/x86/kvm/pmu.c +1 -0
@@ -102,6 +102,7 @@ bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)
 
 	if (is_passthrough_pmu_enabled(vcpu) &&
 	    !enable_vmware_backdoor &&
+	    static_call(kvm_x86_pmu_is_rdpmc_passthru_allowed)(vcpu) &&

If the polarity is inverted, the callback can be OPTIONAL_RET0 on AMD. E.g.

	if (kvm_pmu_call(rdpmc_needs_intercept(vcpu)))
		return false;

arch/x86/kvm/pmu.c bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu)

Quoted 4 lines
arch/x86/kvm/pmu.c +4 -0
+static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
+{
+	/*
+	 * Per Intel SDM vol. 2 for RDPMC, 

Please don't reference specific sections in the comments. For changelogs it's ok, because changelogs are a snapshot in time. But comments are living things and will become stale in almost every case. And I don't see any reason to reference the SDM, just state the behavior; it's implied that that's the architectural behavior, otherwise KVM is buggy.

Quoted 1 line

MSR_PERF_METRICS is accessible by

This is technically wrong, the SDM states that the RDPMC behavior is implementation specific. That matters to some extent, because if it was _just_ one MSR and was guaranteed to always be that one MSR, it might be worth creating a virtualization hole.

/*
* Intercept RDPMC if the host supports PERF_METRICS, but the guest
* does not, as RDPMC with type 0x2000 accesses implementation specific
* metrics.
*/

All that said, isn't this redundant with the number of fixed counters? I'm having a hell of a time finding anything concrete in the SDM, but IIUC fixed counter 3 is tightly coupled to perf metrics. E.g. rather than add a vendor hook just for this, rely on the fixed counters and refuse to enable the mediated PMU if the underlying CPU model is nonsensical, i.e. perf metrics exists without ctr3.

Quoted 4 lines

And I kinda think we have to go that route, because enabling RDPMC interception based on future features is doomed from the start. E.g. if this code had been written prior to PERF_METRICS, older KVMs would have zero clue that RDPMC needs to be intercepted on newer hardware.

Yeah, this sounds make sense. Fixed counter 3 and PERF_METRICS are always coupled as a whole, and the previous code has already checked the fixed counter bitmap. I think we can drop this patch and just add a comment to explain the reason.

#94 lore raw

RFC PATCH v3 26/58 KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

In PMU passthrough mode, there are three requirements to manage IA32_PERF_GLOBAL_CTRL:
- guest IA32_PERF_GLOBAL_CTRL MSR must be saved at vm exit.
- IA32_PERF_GLOBAL_CTRL MSR must be cleared at vm exit to avoid any counter of running within KVM runloop.
- guest IA32_PERF_GLOBAL_CTRL MSR must be restored at vm entry.

Introduce vmx_set_perf_global_ctrl() function to auto switching IA32_PERF_GLOBAL_CTR and invoke it after the VMM finishes setting up the CPUID bits.

---
 arch/x86/include/asm/vmx.h |   1 +
 arch/x86/kvm/vmx/vmx.c     | 117 +++++++++++++++++++++++++++++++------
 arch/x86/kvm/vmx/vmx.h     |   3 +-
 3 files changed, 103 insertions(+), 18 deletions(-)
arch/x86/include/asm/vmx.h +1 -0
index d77a31039f24..5ed89a099533 100644
@@ -106,6 +106,7 @@
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
+#define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL      0x40000000
 
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
arch/x86/kvm/vmx/vmx.c +100 -17

147 lines not on this page · raw message

arch/x86/kvm/vmx/vmx.h +2 -1
index 7b64e271a931..32e3974c1a2c 100644
@@ -510,7 +510,8 @@ static inline u8 vmx_get_rvi(void)
 	       VM_EXIT_LOAD_IA32_EFER |					\
 	       VM_EXIT_CLEAR_BNDCFGS |					\
 	       VM_EXIT_PT_CONCEAL_PIP |					\
-	       VM_EXIT_CLEAR_IA32_RTIT_CTL)
+	       VM_EXIT_CLEAR_IA32_RTIT_CTL |                            \
+	       VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)
 
 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL			\
 	(PIN_BASED_EXT_INTR_MASK |					\
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#95 lore raw

Re: [RFC PATCH v3 26/58] KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

Quoted 62 lines · Mingwei Zhang

From: Xiong Zhang <[email protected]>

In PMU passthrough mode, there are three requirements to manage IA32_PERF_GLOBAL_CTRL:
- guest IA32_PERF_GLOBAL_CTRL MSR must be saved at vm exit.
- IA32_PERF_GLOBAL_CTRL MSR must be cleared at vm exit to avoid any counter of running within KVM runloop.
- guest IA32_PERF_GLOBAL_CTRL MSR must be restored at vm entry.

Introduce vmx_set_perf_global_ctrl() function to auto switching IA32_PERF_GLOBAL_CTR and invoke it after the VMM finishes setting up the CPUID bits.

---
 arch/x86/include/asm/vmx.h |   1 +
 arch/x86/kvm/vmx/vmx.c     | 117 +++++++++++++++++++++++++++++++------
 arch/x86/kvm/vmx/vmx.h     |   3 +-
 3 files changed, 103 insertions(+), 18 deletions(-)
arch/x86/include/asm/vmx.h +1 -0
index d77a31039f24..5ed89a099533 100644
@@ -106,6 +106,7 @@
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
+#define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL      0x40000000
 
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
arch/x86/kvm/vmx/vmx.c +19 -0
index 339742350b7a..34a420fa98c5 100644
@@ -4394,6 +4394,97 @@ static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx)
 	return pin_based_exec_ctrl;
 }
 
+static void vmx_set_perf_global_ctrl(struct vcpu_vmx *vmx)
+{
+	u32 vmentry_ctrl = vm_entry_controls_get(vmx);
+	u32 vmexit_ctrl = vm_exit_controls_get(vmx);
+	struct vmx_msrs *m;
+	int i;
+
+	if (cpu_has_perf_global_ctrl_bug() ||
+	    !is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
+	}
+
+	if (is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		/*
+		 * Setup auto restore guest PERF_GLOBAL_CTRL MSR at vm entry.
+		 */
+		if (vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4425

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+			vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, 0);
+		} else {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;

This function has much duplicated code to initialize/clear MSR autoload/restore region, we may create two simple helpers to avoid these duplicated code.

static inline void vmx_init_loadstore_msr(struct vmx_msrs *m, int idx, bool load);

static inline void vmx_clear_loadstore_msr(struct vmx_msrs *m, int idx, bool load);

Quoted 133 lines
arch/x86/kvm/vmx/vmx.c +71 -17
+		}
+		/*
+		 * Setup auto clear host PERF_GLOBAL_CTRL msr at vm exit.
+		 */
+		if (vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(HOST_IA32_PERF_GLOBAL_CTRL, 0);
+		} else {
+			m = &vmx->msr_autoload.host;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
+		 * Setup auto save guest PERF_GLOBAL_CTRL msr at vm exit
+		 */
+		if (!(vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autostore.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_EXIT_MSR_STORE_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+		}
+	} else {
+		if (!(vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+		}
+		if (!(vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autoload.host;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, m->nr);
+			}
+		}
+		if (!(vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autostore.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_EXIT_MSR_STORE_COUNT, m->nr);
+			}
+		}
+	}
+
+	vm_entry_controls_set(vmx, vmentry_ctrl);
+	vm_exit_controls_set(vmx, vmexit_ctrl);
+}
+
 static u32 vmx_vmentry_ctrl(void)
 {
 	u32 vmentry_ctrl = vmcs_config.vmentry_ctrl;
@@ -4401,17 +4492,10 @@ static u32 vmx_vmentry_ctrl(void)
 	if (vmx_pt_mode_is_system())
 		vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP |
 				  VM_ENTRY_LOAD_IA32_RTIT_CTL);
-	/*
-	 * IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically.
-	 */
-	vmentry_ctrl &= ~(VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL |
-			  VM_ENTRY_LOAD_IA32_EFER |
-			  VM_ENTRY_IA32E_MODE);
-
-	if (cpu_has_perf_global_ctrl_bug())
-		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
-
-	return vmentry_ctrl;
+	 /*
+	  * IA32e mode, and loading of EFER is toggled dynamically.
+	  */
+	return vmentry_ctrl &= ~(VM_ENTRY_LOAD_IA32_EFER | VM_ENTRY_IA32E_MODE);
 }
 
 static u32 vmx_vmexit_ctrl(void)
@@ -4429,12 +4513,8 @@ static u32 vmx_vmexit_ctrl(void)
 		vmexit_ctrl &= ~(VM_EXIT_PT_CONCEAL_PIP |
 				 VM_EXIT_CLEAR_IA32_RTIT_CTL);
 
-	if (cpu_has_perf_global_ctrl_bug())
-		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
-
-	/* Loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically */
-	return vmexit_ctrl &
-		~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER);
+	/* Loading of EFER is toggled dynamically */
+	return vmexit_ctrl & ~VM_EXIT_LOAD_IA32_EFER;
 }
 
 void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)
@@ -4777,6 +4857,7 @@ static void init_vmcs(struct vcpu_vmx *vmx)
 		vmcs_write64(VM_FUNCTION_CONTROL, 0);
 
 	vmcs_write32(VM_EXIT_MSR_STORE_COUNT, 0);
+	vmcs_write64(VM_EXIT_MSR_STORE_ADDR, __pa(vmx->msr_autostore.guest.val));
 	vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, 0);
 	vmcs_write64(VM_EXIT_MSR_LOAD_ADDR, __pa(vmx->msr_autoload.host.val));
 	vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, 0);
@@ -7916,6 +7997,8 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	else
 		exec_controls_setbit(vmx, CPU_BASED_RDPMC_EXITING);
 
+	vmx_set_perf_global_ctrl(vmx);
+
 	/* Refresh #PF interception to account for MAXPHYADDR changes. */
 	vmx_update_exception_bitmap(vcpu);
 }
arch/x86/kvm/vmx/vmx.h +2 -1
index 7b64e271a931..32e3974c1a2c 100644
@@ -510,7 +510,8 @@ static inline u8 vmx_get_rvi(void)
 	       VM_EXIT_LOAD_IA32_EFER |					\
 	       VM_EXIT_CLEAR_BNDCFGS |					\
 	       VM_EXIT_PT_CONCEAL_PIP |					\
-	       VM_EXIT_CLEAR_IA32_RTIT_CTL)
+	       VM_EXIT_CLEAR_IA32_RTIT_CTL |                            \
+	       VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)
 
 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL			\
 	(PIN_BASED_EXT_INTR_MASK |					\
#96 lore raw

Re: [RFC PATCH v3 26/58] KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

In reply to Mingwei Zhang
Quoted 18 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +10 -0
index 339742350b7a..34a420fa98c5 100644
@@ -4394,6 +4394,97 @@ static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx)
 	return pin_based_exec_ctrl;
 }
 
+static void vmx_set_perf_global_ctrl(struct vcpu_vmx *vmx)
+{
+	u32 vmentry_ctrl = vm_entry_controls_get(vmx);
+	u32 vmexit_ctrl = vm_exit_controls_get(vmx);
+	struct vmx_msrs *m;
+	int i;
+
+	if (cpu_has_perf_global_ctrl_bug() ||
+	    !is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4416

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
+	}
+
+	if (is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		/*
+		 * Setup auto restore guest PERF_GLOBAL_CTRL MSR at vm entry.
+		 */
+		if (vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, 0);

To save and restore Global Ctrl MSR at VMX transitions, I'm wondering if there are particular reasons why we prefer VMCS exec control over VMX-transition MSR areas? If no, I'd suggest to use the MSR area approach only for two reasons:

1. Simpler code. In this patch set, in total it takes ~100 LOC to handle the switch of this MSR.
2. With exec ctr approach, it needs one expensive VMCS read instruction to save guest Global Ctrl on every VM exit and one VMCS write in VM entry. (covered in patch 37)

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +5 -0
+		} else {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4431

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
+		 * Setup auto clear host PERF_GLOBAL_CTRL msr at vm exit.
+		 */
+		if (vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(HOST_IA32_PERF_GLOBAL_CTRL, 0);

ditto.

Quoted 59 lines
arch/x86/kvm/vmx/vmx.c +56 -0
+		} else {
+			m = &vmx->msr_autoload.host;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
+		 * Setup auto save guest PERF_GLOBAL_CTRL msr at vm exit
+		 */
+		if (!(vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autostore.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_EXIT_MSR_STORE_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+		}
+	} else {
+		if (!(vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+		}
+		if (!(vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autoload.host;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, m->nr);
+			}
+		}
+		if (!(vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autostore.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_EXIT_MSR_STORE_COUNT, m->nr);
+			}
+		}
+	}
+
+	vm_entry_controls_set(vmx, vmentry_ctrl);
+	vm_exit_controls_set(vmx, vmexit_ctrl);
+}
+
 static u32 vmx_vmentry_ctrl(void)
 {
 	u32 vmentry_ctrl = vmcs_config.vmentry_ctrl;
#97 lore raw

Re: [RFC PATCH v3 26/58] KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

Quoted 36 lines · Zide
Quoted 18 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +10 -0
index 339742350b7a..34a420fa98c5 100644
@@ -4394,6 +4394,97 @@ static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx)
 	return pin_based_exec_ctrl;
 }
 
+static void vmx_set_perf_global_ctrl(struct vcpu_vmx *vmx)
+{
+	u32 vmentry_ctrl = vm_entry_controls_get(vmx);
+	u32 vmexit_ctrl = vm_exit_controls_get(vmx);
+	struct vmx_msrs *m;
+	int i;
+
+	if (cpu_has_perf_global_ctrl_bug() ||
+	    !is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4416

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
+	}
+
+	if (is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		/*
+		 * Setup auto restore guest PERF_GLOBAL_CTRL MSR at vm entry.
+		 */
+		if (vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, 0);

To save and restore Global Ctrl MSR at VMX transitions, I'm wondering if there are particular reasons why we prefer VMCS exec control over VMX-transition MSR areas? If no, I'd suggest to use the MSR area approach only for two reasons:

Quoted 5 lines

1. Simpler code. In this patch set, in total it takes ~100 LOC to handle the switch of this MSR.
2. With exec ctr approach, it needs one expensive VMCS read instruction to save guest Global Ctrl on every VM exit and one VMCS write in VM entry. (covered in patch 37)

In my opinion, there could be two reasons that we prefer to use VMCS exec control to save/restore GLOBAL_CTRL MSR.

1. VMCS exec_control save/load is prior of MSR area save/load in VM-Exit/VM-Entry process.

    Take VM-Exit as example, the sequence (copy from SDM Chapter 28 "VM EXITS") is

    "

    2. Processor state is saved in the guest-state area (Section 28.3).

    3. MSRs may be saved in the VM-exit MSR-store area (Section 28.4).

    4. The following may be performed in parallel and in any order (Section 28.5):

        — Processor state is loaded based in part on the host-state area and some VM-exit controls.

        — Address-range monitoring is cleared.

    5. MSRs may be loaded from the VM-exit MSR-load area (Section 28.6).

    "

    In our mediated vPMU implementation, we hope the guest counters are disabled (Load 0 host global_ctrl) ASAP. That could help to avoid some race condition issues, such as guest overflow PMI leaks into host in VM-Exit process in theory, although we don't really observe it on Intel platforms.

2. Currently VMX save/restore the MSRs in MSR auto-load/restore area with MSR index. As the recommendation from SDM, GLOBAL_CTRL MSR should be enabled at last among all PMU MSRs. If there are multiple PMU MSRs in the MSR auto-load/restore area, the restoring sequence may not meet this requirement as GLOBAL_CTRL doesn't always have the largest MSR index. Of course, we don't have this issue right now since current implementation only save/restore one MSR global_ctrl with MSR auto-load/store area.

But yes, frequent vmcs_read()/vmcs_write() indeed brings some kind of performance hit. We may need to do a performance evaluation and look at how big the performance impact is.

Quoted 79 lines
Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +5 -0
+		} else {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4431

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
+		 * Setup auto clear host PERF_GLOBAL_CTRL msr at vm exit.
+		 */
+		if (vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(HOST_IA32_PERF_GLOBAL_CTRL, 0);

ditto.

Quoted 59 lines
arch/x86/kvm/vmx/vmx.c +56 -0
+		} else {
+			m = &vmx->msr_autoload.host;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
+		 * Setup auto save guest PERF_GLOBAL_CTRL msr at vm exit
+		 */
+		if (!(vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autostore.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_EXIT_MSR_STORE_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+		}
+	} else {
+		if (!(vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+		}
+		if (!(vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autoload.host;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, m->nr);
+			}
+		}
+		if (!(vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL)) {
+			m = &vmx->msr_autostore.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i >= 0) {
+				m->nr--;
+				m->val[i] = m->val[m->nr];
+				vmcs_write32(VM_EXIT_MSR_STORE_COUNT, m->nr);
+			}
+		}
+	}
+
+	vm_entry_controls_set(vmx, vmentry_ctrl);
+	vm_exit_controls_set(vmx, vmexit_ctrl);
+}
+
 static u32 vmx_vmentry_ctrl(void)
 {
 	u32 vmentry_ctrl = vmcs_config.vmentry_ctrl;
#98 lore raw

Re: [RFC PATCH v3 26/58] KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

In reply to Mingwei Zhang
Quoted 11 lines · Mingwei Zhang
---
 arch/x86/include/asm/vmx.h |   1 +
 arch/x86/kvm/vmx/vmx.c     | 117 +++++++++++++++++++++++++++++++------
 arch/x86/kvm/vmx/vmx.h     |   3 +-
 3 files changed, 103 insertions(+), 18 deletions(-)
arch/x86/include/asm/vmx.h +0 -0
index d77a31039f24..5ed89a099533 100644

arch/x86/include/asm/vmx.h line 109

Quoted 4 lines
arch/x86/include/asm/vmx.h +1 -0
@@ -106,6 +106,7 @@
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
+#define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL      0x40000000

Please add a helper in capabilities.h:

static inline bool cpu_has_save_perf_global_ctrl(void)

{
	return vmcs_config.vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
}

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4397

Quoted 11 lines
arch/x86/include/asm/vmx.h +0 -0
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
arch/x86/kvm/vmx/vmx.c +1 -0
index 339742350b7a..34a420fa98c5 100644
@@ -4394,6 +4394,97 @@ static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx)
 	return pin_based_exec_ctrl;
 }
 
+static void vmx_set_perf_global_ctrl(struct vcpu_vmx *vmx)

This is a misleading and inaccurate name. It does far more than "set" PERF_GLOBAL_CTRL, it arguably doesn't ever "set" the MSR, and it gets the VMWRITE for the guest field wrong too.

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4404

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +7 -0
+{
+	u32 vmentry_ctrl = vm_entry_controls_get(vmx);
+	u32 vmexit_ctrl = vm_exit_controls_get(vmx);
+	struct vmx_msrs *m;
+	int i;
+
+	if (cpu_has_perf_global_ctrl_bug() ||

Note, cpu_has_perf_global_ctrl_bug() broken and needs to be purged:
https://lore.kernel.org/all/[email protected]

Note #2, as mentioned earlier, the mediated PMU should take a hard depenency on the load/save controls.

On to this code, it fails to enable the load/save controls, e.g. if userspace does KVM_SET_CPUID2 without a PMU, then KVM_SET_CPUID2 with a PMU. In that case, KVM will fail to set the control bits, and will fallback to the slow MSR load/save lists.

With all of the above and other ideas combined, something like so:

	bool set = enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl();

	vm_entry_controls_changebit(vmx, VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, set);
	vm_exit_controls_changebit(vmx,
				   VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |
				   VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, set);

And I vote to put this in intel_pmu_refresh(); that avoids needing to figure out a name for the helper, while giving more flexibililty on the local variable name.

Oh! Definitely put it in intel_pmu_refresh(), because the RDPMC and MSR interception logic needs to be there. E.g. toggling CPU_BASED_RDPMC_EXITING based solely on CPUID won't do the right thing if KVM ends up making the behavior depend on PERF_CAPABILITIES.

Ditto for MSRs. Though until my patch/series that drops kvm_pmu_refresh() from kvm_pmu_init() lands[*], trying to update MSR intercepts during refresh() will hit a NULL pointer deref as it's currently called before vmcs01 is allocated :-/

I expect to land that series before mediated PMU, but I don't think it makes sense to take an explicit dependency for this series. To fudge around the issue, maybe do this for the next version?

static void intel_pmu_refresh(struct kvm_vcpu *vcpu)

{
	__intel_pmu_refresh(vcpu);

	/*
	 * FIXME: Drop the MSR bitmap check if/when kvm_pmu_init() no longer
	 *        calls kvm_pmu_refresh(), i.e. when KVM refreshes the PMU only
	 *        after vmcs01 is allocated.
	 */
	if (to_vmx(vcpu)->vmcs01.msr_bitmap)
		intel_update_msr_intercepts(vcpu);

	vm_entry_controls_changebit(vmx, VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL,
				    enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl());

	vm_exit_controls_changebit(vmx,
				   VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |
				   VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL,
				   enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl());
}

or with a local variable for "enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl()". I can't come up with a decent name. :-)

[*] https://lore.kernel.org/all/[email protected]

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4416

Quoted 12 lines
arch/x86/kvm/vmx/vmx.c +12 -0
+	    !is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
+	}
+
+	if (is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		/*
+		 * Setup auto restore guest PERF_GLOBAL_CTRL MSR at vm entry.
+		 */
+		if (vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, 0);

This incorrectly clobbers the guest's value. A simple way to handle this is to always propagate writes to PERF_GLOBAL_CTRL to the VMCS, if the write is allowed and enable_mediated_pmu. I.e. ensure GUEST_IA32_PERF_GLOBAL_CTRL is up-to-date regardless of whether or not it's configured to be loaded. Then there's no need to write it here.

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +5 -0
+		} else {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4431

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
+		 * Setup auto clear host PERF_GLOBAL_CTRL msr at vm exit.
+		 */
+		if (vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(HOST_IA32_PERF_GLOBAL_CTRL, 0);

This should be unnecessary. KVM should clear HOST_IA32_PERF_GLOBAL_CTRL in vmx_set_constant_host_state() if enable_mediated_pmu is true. Arguably, it might make sense to clear it unconditionally, but with a comment explaining that it's only actually constant for the mediated PMU.

And if the mediated PMU requires the VMCS knobs, then all of the load/store list complexity goes away.

Quoted 12 lines
arch/x86/kvm/vmx/vmx.c +0 -5
 static u32 vmx_vmentry_ctrl(void)
 {
 	u32 vmentry_ctrl = vmcs_config.vmentry_ctrl;
@@ -4401,17 +4492,10 @@ static u32 vmx_vmentry_ctrl(void)
 	if (vmx_pt_mode_is_system())
 		vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP |
 				  VM_ENTRY_LOAD_IA32_RTIT_CTL);
-	/*
-	 * IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically.
-	 */
-	vmentry_ctrl &= ~(VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL |
-			  VM_ENTRY_LOAD_IA32_EFER |

arch/x86/kvm/vmx/vmx.c static u32 vmx_vmentry_ctrl(void) · line 4498

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +4 -6
-			  VM_ENTRY_IA32E_MODE);
-
-	if (cpu_has_perf_global_ctrl_bug())
-		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
-
-	return vmentry_ctrl;
+	 /*
+	  * IA32e mode, and loading of EFER is toggled dynamically.
+	  */
+	return vmentry_ctrl &= ~(VM_ENTRY_LOAD_IA32_EFER | VM_ENTRY_IA32E_MODE);

With my above suggestion, these changes are unnecessary. If enable_mediated_pmu is false, or the vCPU doesn't have a PMU, clearing the controls is correct. And when the vCPU is gifted a PMU, KVM will explicitly enabled the controls.

To discourage incorrect usage of these helpers maybe rename them to vmx_get_initial_{vmentry,vmexit}_ctrl()?

arch/x86/kvm/vmx/vmx.c static u32 vmx_vmexit_ctrl(void) · line 4437

Quoted 13 lines
arch/x86/kvm/vmx/vmx.c +0 -6
 }
 
 static u32 vmx_vmexit_ctrl(void)
@@ -4429,12 +4513,8 @@ static u32 vmx_vmexit_ctrl(void)
 		vmexit_ctrl &= ~(VM_EXIT_PT_CONCEAL_PIP |
 				 VM_EXIT_CLEAR_IA32_RTIT_CTL);
 
-	if (cpu_has_perf_global_ctrl_bug())
-		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
-
-	/* Loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically */
-	return vmexit_ctrl &
-		~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER);

But this code needs to *add* clearing of VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL.

#99 lore raw

Re: [RFC PATCH v3 26/58] KVM: x86/pmu: Manage MSR interception for IA32_PERF_GLOBAL_CTRL

Quoted 18 lines · Sean Christopherson
Quoted 11 lines · Mingwei Zhang
---
 arch/x86/include/asm/vmx.h |   1 +
 arch/x86/kvm/vmx/vmx.c     | 117 +++++++++++++++++++++++++++++++------
 arch/x86/kvm/vmx/vmx.h     |   3 +-
 3 files changed, 103 insertions(+), 18 deletions(-)
arch/x86/include/asm/vmx.h +0 -0
index d77a31039f24..5ed89a099533 100644

arch/x86/include/asm/vmx.h line 109

Quoted 4 lines
arch/x86/include/asm/vmx.h +1 -0
@@ -106,6 +106,7 @@
 #define VM_EXIT_CLEAR_BNDCFGS                   0x00800000
 #define VM_EXIT_PT_CONCEAL_PIP			0x01000000
 #define VM_EXIT_CLEAR_IA32_RTIT_CTL		0x02000000
+#define VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL      0x40000000

Please add a helper in capabilities.h:

Quoted 4 lines

static inline bool cpu_has_save_perf_global_ctrl(void)

{
	return vmcs_config.vmexit_ctrl & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
}

Sure.

Quoted 81 lines

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4397

Quoted 12 lines
arch/x86/include/asm/vmx.h +0 -0
 #define VM_EXIT_ALWAYSON_WITHOUT_TRUE_MSR	0x00036dff
 
arch/x86/kvm/vmx/vmx.c +1 -0
index 339742350b7a..34a420fa98c5 100644
@@ -4394,6 +4394,97 @@ static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx)
 	return pin_based_exec_ctrl;
 }
 
+static void vmx_set_perf_global_ctrl(struct vcpu_vmx *vmx)

This is a misleading and inaccurate name. It does far more than "set" PERF_GLOBAL_CTRL, it arguably doesn't ever "set" the MSR, and it gets the VMWRITE for the guest field wrong too.

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4404

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +7 -0
+{
+	u32 vmentry_ctrl = vm_entry_controls_get(vmx);
+	u32 vmexit_ctrl = vm_exit_controls_get(vmx);
+	struct vmx_msrs *m;
+	int i;
+
+	if (cpu_has_perf_global_ctrl_bug() ||

Note, cpu_has_perf_global_ctrl_bug() broken and needs to be purged:
https://lore.kernel.org/all/[email protected]

Note #2, as mentioned earlier, the mediated PMU should take a hard depenency on the load/save controls.

On to this code, it fails to enable the load/save controls, e.g. if userspace does KVM_SET_CPUID2 without a PMU, then KVM_SET_CPUID2 with a PMU. In that case, KVM will fail to set the control bits, and will fallback to the slow MSR load/save lists.

With all of the above and other ideas combined, something like so:

	bool set = enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl();

	vm_entry_controls_changebit(vmx, VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, set);
	vm_exit_controls_changebit(vmx,
				   VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |
				   VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, set);

And I vote to put this in intel_pmu_refresh(); that avoids needing to figure out a name for the helper, while giving more flexibililty on the local variable name.

Oh! Definitely put it in intel_pmu_refresh(), because the RDPMC and MSR interception logic needs to be there. E.g. toggling CPU_BASED_RDPMC_EXITING based solely on CPUID won't do the right thing if KVM ends up making the behavior depend on PERF_CAPABILITIES.

Ditto for MSRs. Though until my patch/series that drops kvm_pmu_refresh() from kvm_pmu_init() lands[*], trying to update MSR intercepts during refresh() will hit a NULL pointer deref as it's currently called before vmcs01 is allocated :-/

I expect to land that series before mediated PMU, but I don't think it makes sense to take an explicit dependency for this series. To fudge around the issue, maybe do this for the next version?

static void intel_pmu_refresh(struct kvm_vcpu *vcpu)

{
	__intel_pmu_refresh(vcpu);

	/*
	 * FIXME: Drop the MSR bitmap check if/when kvm_pmu_init() no longer
	 *        calls kvm_pmu_refresh(), i.e. when KVM refreshes the PMU only
	 *        after vmcs01 is allocated.
	 */
	if (to_vmx(vcpu)->vmcs01.msr_bitmap)
		intel_update_msr_intercepts(vcpu);

	vm_entry_controls_changebit(vmx, VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL,
				    enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl());

	vm_exit_controls_changebit(vmx,
				   VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL |
				   VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL,
				   enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl());
}
Quoted 4 lines

or with a local variable for "enable_mediated_pmu && kvm_pmu_has_perf_global_ctrl()". I can't come up with a decent name. :-)

[*] https://lore.kernel.org/all/[email protected]

Sure. This looks better.

Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +9 -0
+	    !is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
+		vmexit_ctrl &= ~VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL;
+	}
+
+	if (is_passthrough_pmu_enabled(&vmx->vcpu)) {
+		/*
+		 * Setup auto restore guest PERF_GLOBAL_CTRL MSR at vm entry.
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4416

Quoted 3 lines
arch/x86/kvm/vmx/vmx.c +3 -0
+		 */
+		if (vmentry_ctrl & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, 0);

This incorrectly clobbers the guest's value. A simple way to handle this is to always propagate writes to PERF_GLOBAL_CTRL to the VMCS, if the write is allowed and enable_mediated_pmu. I.e. ensure GUEST_IA32_PERF_GLOBAL_CTRL is up-to-date regardless of whether or not it's configured to be loaded. Then there's no need to write it here.

For mediated vPMU, I think we can move this into intel_pmu_refresh() as well, and just after the vm_entry_controls_changebit() helpers. the value should be supported gp counters mask which respects the behavior of PERF_GLOBAL_CTRL reset.

Quoted 12 lines
Quoted 12 lines
arch/x86/kvm/vmx/vmx.c +11 -0
+		} else {
+			m = &vmx->msr_autoload.guest;
+			i = vmx_find_loadstore_msr_slot(m, MSR_CORE_PERF_GLOBAL_CTRL);
+			if (i < 0) {
+				i = m->nr++;
+				vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, m->nr);
+			}
+			m->val[i].index = MSR_CORE_PERF_GLOBAL_CTRL;
+			m->val[i].value = 0;
+		}
+		/*
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c static u32 vmx_pin_based_exec_ctrl(struct vcpu_vmx *vmx) · line 4431

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +4 -0
+		 * Setup auto clear host PERF_GLOBAL_CTRL msr at vm exit.
+		 */
+		if (vmexit_ctrl & VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL) {
+			vmcs_write64(HOST_IA32_PERF_GLOBAL_CTRL, 0);

This should be unnecessary. KVM should clear HOST_IA32_PERF_GLOBAL_CTRL in vmx_set_constant_host_state() if enable_mediated_pmu is true. Arguably, it might make sense to clear it unconditionally, but with a comment explaining that it's only actually constant for the mediated PMU.

Sure. would move this clearing into vmx_set_constant_host_state(). Yeah, I suppose it can be cleared unconditionally since VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL bit never be set for legacy emulated vPMU.

Quoted 21 lines

And if the mediated PMU requires the VMCS knobs, then all of the load/store list complexity goes away.

Quoted 17 lines
arch/x86/kvm/vmx/vmx.c +0 -10
 static u32 vmx_vmentry_ctrl(void)
 {
 	u32 vmentry_ctrl = vmcs_config.vmentry_ctrl;
@@ -4401,17 +4492,10 @@ static u32 vmx_vmentry_ctrl(void)
 	if (vmx_pt_mode_is_system())
 		vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP |
 				  VM_ENTRY_LOAD_IA32_RTIT_CTL);
-	/*
-	 * IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically.
-	 */
-	vmentry_ctrl &= ~(VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL |
-			  VM_ENTRY_LOAD_IA32_EFER |
-			  VM_ENTRY_IA32E_MODE);
-
-	if (cpu_has_perf_global_ctrl_bug())
-		vmentry_ctrl &= ~VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL;
-
Quoted 8 lines

arch/x86/kvm/vmx/vmx.c static u32 vmx_vmentry_ctrl(void) · line 4498

Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +4 -1
-	return vmentry_ctrl;
+	 /*
+	  * IA32e mode, and loading of EFER is toggled dynamically.
+	  */
+	return vmentry_ctrl &= ~(VM_ENTRY_LOAD_IA32_EFER | VM_ENTRY_IA32E_MODE);

With my above suggestion, these changes are unnecessary. If enable_mediated_pmu is false, or the vCPU doesn't have a PMU, clearing the controls is correct. And when the vCPU is gifted a PMU, KVM will explicitly enabled the controls.

Yes.

Quoted 3 lines

To discourage incorrect usage of these helpers maybe rename them to vmx_get_initial_{vmentry,vmexit}_ctrl()?

Sure.

Quoted 8 lines
Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +0 -0
 }
 
 static u32 vmx_vmexit_ctrl(void)
@@ -4429,12 +4513,8 @@ static u32 vmx_vmexit_ctrl(void)
 		vmexit_ctrl &= ~(VM_EXIT_PT_CONCEAL_PIP |
 				 VM_EXIT_CLEAR_IA32_RTIT_CTL);
 
Quoted 7 lines

arch/x86/kvm/vmx/vmx.c static u32 vmx_vmexit_ctrl(void) · line 4437

Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +0 -6
-	if (cpu_has_perf_global_ctrl_bug())
-		vmexit_ctrl &= ~VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL;
-
-	/* Loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically */
-	return vmexit_ctrl &
-		~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER);

But this code needs to *add* clearing of VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL.

Sure.

#100 lore raw

RFC PATCH v3 27/58 KVM: x86/pmu: Create a function prototype to disable MSR interception

In reply to Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/cpuid.c                   | 4 ++++
 arch/x86/kvm/pmu.c                     | 5 +++++
 arch/x86/kvm/pmu.h                     | 2 ++
 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index fd986d5146e4..1b7876dcb3c3 100644
@@ -24,6 +24,7 @@ KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
+KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/cpuid.c +4 -0
index f2f2be5d1141..3deb79b39847 100644
@@ -381,6 +381,10 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_passthrough_pmu_msrs(vcpu);
+
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
arch/x86/kvm/pmu.c +5 -0
index 3afefe4cf6e2..bd94f2d67f5c 100644
@@ -1059,3 +1059,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	kfree(filter);
 	return r;
 }
+
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
+}
arch/x86/kvm/pmu.h +2 -0
index e1af6d07b191..63f876557716 100644
@@ -41,6 +41,7 @@ struct kvm_pmu_ops {
 	void (*deliver_pmi)(struct kvm_vcpu *vcpu);
 	void (*cleanup)(struct kvm_vcpu *vcpu);
 	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
+	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
@@ -292,6 +293,7 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
 bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#101 lore raw

Re: [RFC PATCH v3 27/58] KVM: x86/pmu: Create a function prototype to disable MSR interception

Quoted 58 lines · Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/cpuid.c                   | 4 ++++
 arch/x86/kvm/pmu.c                     | 5 +++++
 arch/x86/kvm/pmu.h                     | 2 ++
 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index fd986d5146e4..1b7876dcb3c3 100644
@@ -24,6 +24,7 @@ KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
+KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/cpuid.c +4 -0
index f2f2be5d1141..3deb79b39847 100644
@@ -381,6 +381,10 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_passthrough_pmu_msrs(vcpu);
+
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
arch/x86/kvm/pmu.c +5 -0
index 3afefe4cf6e2..bd94f2d67f5c 100644
@@ -1059,3 +1059,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	kfree(filter);
 	return r;
 }
+
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
+}
arch/x86/kvm/pmu.h +0 -0
index e1af6d07b191..63f876557716 100644

arch/x86/kvm/pmu.h struct kvm_pmu_ops { · line 44

Quoted 4 lines
arch/x86/kvm/pmu.h +1 -0
@@ -41,6 +41,7 @@ struct kvm_pmu_ops {
 	void (*deliver_pmi)(struct kvm_vcpu *vcpu);
 	void (*cleanup)(struct kvm_vcpu *vcpu);
 	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
+	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);

Seems after_set_cpuid() is a better name. It's more generic to reflect the fact that PMU needs to do something after userspace sets CPUID. Currently PMU needs to update the MSR interception policy, but it may want to do more in the future.

Also, it's more consistent to other APIs called in kvm_vcpu_after_set_cpuid().

Quoted 11 lines
arch/x86/kvm/pmu.h +1 -0
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
@@ -292,6 +293,7 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
 bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
#102 lore raw

Re: [RFC PATCH v3 27/58] KVM: x86/pmu: Create a function prototype to disable MSR interception

Quoted 69 lines · Zide
Quoted 58 lines · Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h | 1 +
 arch/x86/kvm/cpuid.c                   | 4 ++++
 arch/x86/kvm/pmu.c                     | 5 +++++
 arch/x86/kvm/pmu.h                     | 2 ++
 4 files changed, 12 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +1 -0
index fd986d5146e4..1b7876dcb3c3 100644
@@ -24,6 +24,7 @@ KVM_X86_PMU_OP(is_rdpmc_passthru_allowed)
 KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
+KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/cpuid.c +4 -0
index f2f2be5d1141..3deb79b39847 100644
@@ -381,6 +381,10 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 	vcpu->arch.reserved_gpa_bits = kvm_vcpu_reserved_gpa_bits_raw(vcpu);
 
 	kvm_pmu_refresh(vcpu);
+
+	if (is_passthrough_pmu_enabled(vcpu))
+		kvm_pmu_passthrough_pmu_msrs(vcpu);
+
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
 
arch/x86/kvm/pmu.c +5 -0
index 3afefe4cf6e2..bd94f2d67f5c 100644
@@ -1059,3 +1059,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp)
 	kfree(filter);
 	return r;
 }
+
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
+{
+	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
+}
arch/x86/kvm/pmu.h +0 -0
index e1af6d07b191..63f876557716 100644

arch/x86/kvm/pmu.h struct kvm_pmu_ops { · line 44

Quoted 4 lines
arch/x86/kvm/pmu.h +1 -0
@@ -41,6 +41,7 @@ struct kvm_pmu_ops {
 	void (*deliver_pmi)(struct kvm_vcpu *vcpu);
 	void (*cleanup)(struct kvm_vcpu *vcpu);
 	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
+	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);

Seems after_set_cpuid() is a better name. It's more generic to reflect the fact that PMU needs to do something after userspace sets CPUID. Currently PMU needs to update the MSR interception policy, but it may want to do more in the future.

Quoted 2 lines

Also, it's more consistent to other APIs called in kvm_vcpu_after_set_cpuid().

Looks reasonable.

Quoted 12 lines
Quoted 12 lines
arch/x86/kvm/pmu.h +1 -0
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
@@ -292,6 +293,7 @@ void kvm_pmu_destroy(struct kvm_vcpu *vcpu);
 int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
 bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
+void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
#103 lore raw

Re: [RFC PATCH v3 27/58] KVM: x86/pmu: Create a function prototype to disable MSR interception

In reply to Mingwei Zhang
Quoted 2 lines · Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

This hook/patch should be unnecessary, the logic belongs in the vendor refresh() callback.

#104 lore raw

Re: [RFC PATCH v3 27/58] KVM: x86/pmu: Create a function prototype to disable MSR interception

Quoted 5 lines · Sean Christopherson
Quoted 2 lines · Mingwei Zhang

Add one extra pmu function prototype in kvm_pmu_ops to disable PMU MSR interception.

This hook/patch should be unnecessary, the logic belongs in the vendor refresh() callback.

Yes.

#105 lore raw

RFC PATCH v3 28/58 KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU MSRs

In reply to Mingwei Zhang

From: Dapeng Mi <[email protected]>

Event selectors for GP counters and fixed counters control MSR are intercepted for the purpose of security, i.e., preventing guest from using unallowed events to steal information or take advantages of any CPU errata.

Other than event selectors, disable PMU counter MSR interception specified in guest CPUID, counter MSR index outside of exported range will still be intercepted.

Global registers like global_ctrl will passthrough only if pmu version is greater than 1.

---
 arch/x86/kvm/cpuid.c         |  3 +--
 arch/x86/kvm/vmx/pmu_intel.c | 47 ++++++++++++++++++++++++++++++++++++
 2 files changed, 48 insertions(+), 2 deletions(-)
arch/x86/kvm/cpuid.c +1 -2
index 3deb79b39847..f01e2f1ccce1 100644
@@ -382,8 +382,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
 
 	kvm_pmu_refresh(vcpu);
 
-	if (is_passthrough_pmu_enabled(vcpu))
-		kvm_pmu_passthrough_pmu_msrs(vcpu);
+	kvm_pmu_passthrough_pmu_msrs(vcpu);
 
 	vcpu->arch.cr4_guest_rsvd_bits =
 	    __cr4_reserved_bits(guest_cpuid_has, vcpu);
arch/x86/kvm/vmx/pmu_intel.c +47 -0

59 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#106 lore raw

Re: [RFC PATCH v3 28/58] KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU MSRs

Quoted 5 lines · Mingwei Zhang
arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 02c9019c6f85..737de5bf1eee 100644

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 748

Quoted 9 lines
arch/x86/kvm/vmx/pmu_intel.c +6 -0
@@ -740,6 +740,52 @@ static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
 	return true;
 }
 
+/*
+ * Setup PMU MSR interception for both mediated passthrough vPMU and legacy
+ * emulated vPMU. Note that this function is called after each time userspace
+ * set CPUID.
+ */
+static void intel_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)

Function verb is misleading. This doesn't always "passthrough" MSRs, it's also responsible for enabling interception as needed. intel_pmu_update_msr_intercepts()?

Quoted 4 lines
arch/x86/kvm/vmx/pmu_intel.c +4 -0
+{
+	bool msr_intercept = !is_passthrough_pmu_enabled(vcpu);
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	int i;

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 762

Quoted 10 lines
arch/x86/kvm/vmx/pmu_intel.c +10 -0
+
+	/*
+	 * Unexposed PMU MSRs are intercepted by default. However,
+	 * KVM_SET_CPUID{,2} may be invoked multiple times. To ensure MSR
+	 * interception is correct after each call of setting CPUID, explicitly
+	 * touch msr bitmap for each PMU MSR.
+	 */
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+		if (i >= pmu->nr_arch_gp_counters)
+			msr_intercept = true;

Hmm, I like the idea and that y'all remembered to intercept unsupported MSRs, but it's way, way too easy to clobber msr_intercept and fail to re-initialize across for-loops.

Rather than update the variable mid-loop, how about this?

	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, MSR_TYPE_RW, intercept);
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW,
					  intercept || !fw_writes_is_enabled(vcpu));
	}
	for ( ; i < kvm_pmu_cap.num_counters_gp; i++) {
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, MSR_TYPE_RW, true);
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, true);
	}

	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, MSR_TYPE_RW, intercept);
	for ( ; i < kvm_pmu_cap.num_counters_fixed; i++)
		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, MSR_TYPE_RW, true);
Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +5 -0
+		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, MSR_TYPE_RW, msr_intercept);
+		if (fw_writes_is_enabled(vcpu))
+			vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, msr_intercept);
+		else
+			vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, true);

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 777

Quoted 10 lines
arch/x86/kvm/vmx/pmu_intel.c +10 -0
+	}
+
+	msr_intercept = !is_passthrough_pmu_enabled(vcpu);
+	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+		if (i >= pmu->nr_arch_fixed_counters)
+			msr_intercept = true;
+		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, MSR_TYPE_RW, msr_intercept);
+	}
+
+	if (pmu->version > 1 && is_passthrough_pmu_enabled(vcpu) &&

Don't open code kvm_pmu_has_perf_global_ctrl().

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 782

Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +5 -0
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed)
+		msr_intercept = false;
+	else
+		msr_intercept = true;

This reinforces that checking PERF_CAPABILITIES for PERF_METRICS is likely doomed to fail, because doesn't PERF_GLOBAL_CTRL need to be intercepted, strictly speaking, to prevent setting EN_PERF_METRICS?

Quoted 19 lines
arch/x86/kvm/vmx/pmu_intel.c +6 -0
+	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_STATUS, MSR_TYPE_RW, msr_intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, MSR_TYPE_RW, msr_intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, MSR_TYPE_RW, msr_intercept);
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -752,6 +798,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.deliver_pmi = intel_pmu_deliver_pmi,
 	.cleanup = intel_pmu_cleanup,
 	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
+	.passthrough_pmu_msrs = intel_passthrough_pmu_msrs,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#107 lore raw

Re: [RFC PATCH v3 28/58] KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU MSRs

Quoted 9 lines · Sean Christopherson
Quoted 8 lines · Mingwei Zhang
arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 02c9019c6f85..737de5bf1eee 100644
@@ -740,6 +740,52 @@ static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu)
 	return true;
 }
 
Quoted 8 lines

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 748

Quoted 6 lines
arch/x86/kvm/vmx/pmu_intel.c +6 -0
+/*
+ * Setup PMU MSR interception for both mediated passthrough vPMU and legacy
+ * emulated vPMU. Note that this function is called after each time userspace
+ * set CPUID.
+ */
+static void intel_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)

Function verb is misleading. This doesn't always "passthrough" MSRs, it's also responsible for enabling interception as needed. intel_pmu_update_msr_intercepts()?

Yes, it's better. Thanks.

Quoted 21 lines
Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +4 -0
+{
+	bool msr_intercept = !is_passthrough_pmu_enabled(vcpu);
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	int i;

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 762

Quoted 10 lines
arch/x86/kvm/vmx/pmu_intel.c +10 -0
+
+	/*
+	 * Unexposed PMU MSRs are intercepted by default. However,
+	 * KVM_SET_CPUID{,2} may be invoked multiple times. To ensure MSR
+	 * interception is correct after each call of setting CPUID, explicitly
+	 * touch msr bitmap for each PMU MSR.
+	 */
+	for (i = 0; i < kvm_pmu_cap.num_counters_gp; i++) {
+		if (i >= pmu->nr_arch_gp_counters)
+			msr_intercept = true;

Hmm, I like the idea and that y'all remembered to intercept unsupported MSRs, but it's way, way too easy to clobber msr_intercept and fail to re-initialize across for-loops.

Rather than update the variable mid-loop, how about this?

Quoted 14 lines
	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, MSR_TYPE_RW, intercept);
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW,
					  intercept || !fw_writes_is_enabled(vcpu));
	}
	for ( ; i < kvm_pmu_cap.num_counters_gp; i++) {
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, MSR_TYPE_RW, true);
		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, true);
	}

	for (i = 0; i < pmu->nr_arch_fixed_counters; i++)
		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, MSR_TYPE_RW, intercept);
	for ( ; i < kvm_pmu_cap.num_counters_fixed; i++)
		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, MSR_TYPE_RW, true);

Yeah, it's indeed better.

Quoted 18 lines
Quoted 7 lines
arch/x86/kvm/vmx/pmu_intel.c +5 -0
+		vmx_set_intercept_for_msr(vcpu, MSR_IA32_PERFCTR0 + i, MSR_TYPE_RW, msr_intercept);
+		if (fw_writes_is_enabled(vcpu))
+			vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, msr_intercept);
+		else
+			vmx_set_intercept_for_msr(vcpu, MSR_IA32_PMC0 + i, MSR_TYPE_RW, true);

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 777

Quoted 10 lines
arch/x86/kvm/vmx/pmu_intel.c +10 -0
+	}
+
+	msr_intercept = !is_passthrough_pmu_enabled(vcpu);
+	for (i = 0; i < kvm_pmu_cap.num_counters_fixed; i++) {
+		if (i >= pmu->nr_arch_fixed_counters)
+			msr_intercept = true;
+		vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_FIXED_CTR0 + i, MSR_TYPE_RW, msr_intercept);
+	}
+
+	if (pmu->version > 1 && is_passthrough_pmu_enabled(vcpu) &&

Don't open code kvm_pmu_has_perf_global_ctrl().

Oh, yes.

Quoted 9 lines

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 782

Quoted 6 lines
arch/x86/kvm/vmx/pmu_intel.c +5 -0
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed)
+		msr_intercept = false;
+	else
+		msr_intercept = true;

This reinforces that checking PERF_CAPABILITIES for PERF_METRICS is likely doomed to fail, because doesn't PERF_GLOBAL_CTRL need to be intercepted, strictly speaking, to prevent setting EN_PERF_METRICS?

Sean, do you mean we need to check if guest supports PERF_METRICS here? If not, we need to set global MSRs to interception and then avoid guest tries to enable guest PERF_METRICS, right?

Quoted 20 lines
Quoted 20 lines
arch/x86/kvm/vmx/pmu_intel.c +6 -0
+	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_STATUS, MSR_TYPE_RW, msr_intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_CTRL, MSR_TYPE_RW, msr_intercept);
+	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, MSR_TYPE_RW, msr_intercept);
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -752,6 +798,7 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.deliver_pmi = intel_pmu_deliver_pmi,
 	.cleanup = intel_pmu_cleanup,
 	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
+	.passthrough_pmu_msrs = intel_passthrough_pmu_msrs,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#108 lore raw

Re: [RFC PATCH v3 28/58] KVM: x86/pmu: Add intel_passthrough_pmu_msrs() to pass-through PMU MSRs

Quoted 10 lines · Dapeng Mi
Quoted 8 lines · Sean Christopherson

arch/x86/kvm/vmx/pmu_intel.c static bool intel_is_rdpmc_passthru_allowed(struct kvm_vcpu *vcpu) · line 782

Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +5 -0
+	    pmu->nr_arch_gp_counters == kvm_pmu_cap.num_counters_gp &&
+	    pmu->nr_arch_fixed_counters == kvm_pmu_cap.num_counters_fixed)
+		msr_intercept = false;
+	else
+		msr_intercept = true;

This reinforces that checking PERF_CAPABILITIES for PERF_METRICS is likely doomed to fail, because doesn't PERF_GLOBAL_CTRL need to be intercepted, strictly speaking, to prevent setting EN_PERF_METRICS?

Quoted 3 lines

Sean, do you mean we need to check if guest supports PERF_METRICS here? If not, we need to set global MSRs to interception and then avoid guest tries to enable guest PERF_METRICS, right?

Yep, exactly. And that the "nr_arch_fixed_counter == num_counters_fixed" takes care of things here as well.

#109 lore raw

RFC PATCH v3 29/58 KVM: x86/pmu: Avoid legacy vPMU code when accessing global_ctrl in passthrough vPMU

In reply to Mingwei Zhang

Avoid calling into legacy/emulated vPMU logic such as reprogram_counters() when passthrough vPMU is enabled. Note that even when passthrough vPMU is enabled, global_ctrl may still be intercepted if guest VM only sees a subset of the counters.

---
 arch/x86/kvm/pmu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +2 -1
index bd94f2d67f5c..e9047051489e 100644
@@ -713,7 +713,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (pmu->global_ctrl != data) {
 			diff = pmu->global_ctrl ^ data;
 			pmu->global_ctrl = data;
-			reprogram_counters(pmu, diff);
+			if (!is_passthrough_pmu_enabled(vcpu))
+				reprogram_counters(pmu, diff);
 		}
 		break;
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#110 lore raw

RFC PATCH v3 30/58 KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()

In reply to Mingwei Zhang

Reject PMU MSRs interception explicitly in vmx_get_passthrough_msr_slot() since interception of PMU MSRs are specially handled in intel_passthrough_pmu_msrs().

---
 arch/x86/kvm/vmx/vmx.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +8 -1
index 34a420fa98c5..41102658ed21 100644
@@ -166,7 +166,7 @@ module_param(enable_passthrough_pmu, bool, 0444);
 
 /*
  * List of MSRs that can be directly passed to the guest.
- * In addition to these x2apic, PT and LBR MSRs are handled specially.
+ * In addition to these x2apic, PMU, PT and LBR MSRs are handled specially.
  */
 static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {
 	MSR_IA32_SPEC_CTRL,
@@ -695,6 +695,13 @@ static int vmx_get_passthrough_msr_slot(u32 msr)
 	case MSR_LBR_CORE_FROM ... MSR_LBR_CORE_FROM + 8:
 	case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8:
 		/* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */
+	case MSR_IA32_PMC0 ... MSR_IA32_PMC0 + 7:
+	case MSR_IA32_PERFCTR0 ... MSR_IA32_PERFCTR0 + 7:
+	case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + 2:
+	case MSR_CORE_PERF_GLOBAL_STATUS:
+	case MSR_CORE_PERF_GLOBAL_CTRL:
+	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
+		/* PMU MSRs. These are handled in intel_passthrough_pmu_msrs() */
 		return -ENOENT;
 	}
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#111 lore raw

Re: [RFC PATCH v3 30/58] KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot()

Quoted 25 lines · Mingwei Zhang

Reject PMU MSRs interception explicitly in vmx_get_passthrough_msr_slot() since interception of PMU MSRs are specially handled in intel_passthrough_pmu_msrs().

---
 arch/x86/kvm/vmx/vmx.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/vmx.c +1 -1
index 34a420fa98c5..41102658ed21 100644
@@ -166,7 +166,7 @@ module_param(enable_passthrough_pmu, bool, 0444);
 
 /*
  * List of MSRs that can be directly passed to the guest.
- * In addition to these x2apic, PT and LBR MSRs are handled specially.
+ * In addition to these x2apic, PMU, PT and LBR MSRs are handled specially.
  */
 static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {
 	MSR_IA32_SPEC_CTRL,

arch/x86/kvm/vmx/vmx.c static int vmx_get_passthrough_msr_slot(u32 msr) · line 700

Quoted 6 lines
arch/x86/kvm/vmx/vmx.c +3 -0
@@ -695,6 +695,13 @@ static int vmx_get_passthrough_msr_slot(u32 msr)
 	case MSR_LBR_CORE_FROM ... MSR_LBR_CORE_FROM + 8:
 	case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8:
 		/* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */
+	case MSR_IA32_PMC0 ... MSR_IA32_PMC0 + 7:
+	case MSR_IA32_PERFCTR0 ... MSR_IA32_PERFCTR0 + 7:
+	case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + 2:

We'd better to use the macro KVM_MAX_NR_GP_COUNTERS/KVM_MAX_NR_FIXED_COUNTERS to replace these magic number.

arch/x86/kvm/vmx/vmx.c +0 -0
index 6d9ccac839b4..68d9c5e7f91e 100644
@@ -696,9 +696,9 @@ static int vmx_get_passthrough_msr_slot(u32 msr)

        case MSR_LBR_CORE_FROM ... MSR_LBR_CORE_FROM + 8:
        case MSR_LBR_CORE_TO ... MSR_LBR_CORE_TO + 8:
                /* LBR MSRs. These are handled in vmx_update_intercept_for_lbr_msrs() */
-       case MSR_IA32_PMC0 ... MSR_IA32_PMC0 + 7:
-       case MSR_IA32_PERFCTR0 ... MSR_IA32_PERFCTR0 + 7:
-       case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + 2:
+       case MSR_IA32_PMC0 ... MSR_IA32_PMC0 + KVM_MAX_NR_GP_COUNTERS - 1:
+       case MSR_IA32_PERFCTR0 ... MSR_IA32_PERFCTR0 + KVM_MAX_NR_GP_COUNTERS - 1:
+       case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + KVM_MAX_NR_FIXED_COUNTERS -1:
        case MSR_CORE_PERF_GLOBAL_STATUS:
        case MSR_CORE_PERF_GLOBAL_CTRL:
        case MSR_CORE_PERF_GLOBAL_OVF_CTRL:

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +4 -0
+	case MSR_CORE_PERF_GLOBAL_STATUS:
+	case MSR_CORE_PERF_GLOBAL_CTRL:
+	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
+		/* PMU MSRs. These are handled in intel_passthrough_pmu_msrs() */
 		return -ENOENT;
 	}
 
#112 lore raw

RFC PATCH v3 31/58 KVM: x86/pmu: Add counter MSR and selector MSR index into struct kvm_pmc

In reply to Mingwei Zhang

Add the MSR indices for both selector and counter in each kvm_pmc. Giving convenience to mediated passthrough vPMU in scenarios of querying MSR from a given pmc. Note that legacy vPMU does not need this because it never directly accesses PMU MSRs, instead each kvm_pmc is bound to a perf_event.

For actual Zen 4 and later hardware, it will never be the case that the PerfMonV2 CPUID bit is set but the PerfCtrCore bit is not. However, a guest can be booted with PerfMonV2 enabled and PerfCtrCore disabled. KVM does not clear the PerfMonV2 bit from guest CPUID as long as the host has the PerfCtrCore capability.

In this case, passthrough mode will use the K7 legacy MSRs to program events but with the incorrect assumption that there are 6 such counters instead of 4 as advertised by CPUID leaf 0x80000022 EBX. The host kernel will also report unchecked MSR accesses for the absent counters while saving or restoring guest PMU contexts.

Ensure that K7 legacy MSRs are not used as long as the guest CPUID has either PerfCtrCore or PerfMonV2 set.

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/svm/pmu.c          | 13 +++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c    | 13 +++++++++++++
 3 files changed, 28 insertions(+)
arch/x86/include/asm/kvm_host.h +2 -0
index 4b3ce6194bdb..603727312f9c 100644
@@ -522,6 +522,8 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 msr_counter;
+	u64 msr_eventsel;
 	struct perf_event *perf_event;
 	struct kvm_vcpu *vcpu;
 	/*
arch/x86/kvm/svm/pmu.c +13 -0
index 6b471b1ec9b8..64060cbd8210 100644
@@ -177,6 +177,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 {
 	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
 	union cpuid_0x80000022_ebx ebx;
+	int i;
 
 	pmu->version = 1;
 	if (guest_cpuid_has(vcpu, X86_FEATURE_PERFMON_V2)) {
@@ -210,6 +211,18 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
 	pmu->counter_bitmask[KVM_PMC_FIXED] = 0;
 	pmu->nr_arch_fixed_counters = 0;
 	bitmap_set(pmu->all_valid_pmc_idx, 0, pmu->nr_arch_gp_counters);
+
+	if (pmu->version > 1 || guest_cpuid_has(vcpu, X86_FEATURE_PERFCTR_CORE)) {
+		for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+			pmu->gp_counters[i].msr_eventsel = MSR_F15H_PERF_CTL0 + 2 * i;
+			pmu->gp_counters[i].msr_counter = MSR_F15H_PERF_CTR0 + 2 * i;
+		}
+	} else {
+		for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+			pmu->gp_counters[i].msr_eventsel = MSR_K7_EVNTSEL0 + i;
+			pmu->gp_counters[i].msr_counter = MSR_K7_PERFCTR0 + i;
+		}
+	}
 }
 
 static void amd_pmu_init(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/pmu_intel.c +13 -0
index 737de5bf1eee..0de918dc14ea 100644
@@ -562,6 +562,19 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 				~((1ull << pmu->nr_arch_gp_counters) - 1);
 		}
 	}
+
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmu->gp_counters[i].msr_eventsel = MSR_P6_EVNTSEL0 + i;
+		if (fw_writes_is_enabled(vcpu))
+			pmu->gp_counters[i].msr_counter = MSR_IA32_PMC0 + i;
+		else
+			pmu->gp_counters[i].msr_counter = MSR_IA32_PERFCTR0 + i;
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmu->fixed_counters[i].msr_eventsel = MSR_CORE_PERF_FIXED_CTR_CTRL;
+		pmu->fixed_counters[i].msr_counter = MSR_CORE_PERF_FIXED_CTR0 + i;
+	}
 }
 
 static void intel_pmu_init(struct kvm_vcpu *vcpu)
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#113 lore raw

Re: [RFC PATCH v3 31/58] KVM: x86/pmu: Add counter MSR and selector MSR index into struct kvm_pmc

Please squash this with the patch that does the actual save/load. Hmm, maybe it should be put/load, now that I think about it more? That's more consitent with existing KVM terminology.

Anyways, please squash them together, it's very difficult to review them separately.

Quoted 33 lines · Mingwei Zhang

Add the MSR indices for both selector and counter in each kvm_pmc. Giving convenience to mediated passthrough vPMU in scenarios of querying MSR from a given pmc. Note that legacy vPMU does not need this because it never directly accesses PMU MSRs, instead each kvm_pmc is bound to a perf_event.

For actual Zen 4 and later hardware, it will never be the case that the PerfMonV2 CPUID bit is set but the PerfCtrCore bit is not. However, a guest can be booted with PerfMonV2 enabled and PerfCtrCore disabled. KVM does not clear the PerfMonV2 bit from guest CPUID as long as the host has the PerfCtrCore capability.

In this case, passthrough mode will use the K7 legacy MSRs to program events but with the incorrect assumption that there are 6 such counters instead of 4 as advertised by CPUID leaf 0x80000022 EBX. The host kernel will also report unchecked MSR accesses for the absent counters while saving or restoring guest PMU contexts.

Ensure that K7 legacy MSRs are not used as long as the guest CPUID has either PerfCtrCore or PerfMonV2 set.

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/svm/pmu.c          | 13 +++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c    | 13 +++++++++++++
 3 files changed, 28 insertions(+)
arch/x86/include/asm/kvm_host.h +0 -0
index 4b3ce6194bdb..603727312f9c 100644

arch/x86/include/asm/kvm_host.h struct kvm_pmc { · line 526

Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2 -0
@@ -522,6 +522,8 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 msr_counter;
+	u64 msr_eventsel;

There's no need to track these per PMC, the tracking can be per PMU, e.g.

	u64 gp_eventsel_base;
	u64 gp_counter_base;
	u64 gp_shift;
	u64 fixed_base;

Actually, there's no need for a per-PMU fixed base, as that can be shoved into kvm_pmu_ops. LOL, and the upcoming patch hardcodes INTEL_PMC_FIXED_RDPMC_BASE. Naughty, naughty ;-)

It's not pretty, but 16 bytes per PMC isn't trivial.

Hmm, actually, scratch all that. A better alternative would be to provide a helper to put/load counter/selector MSRs, and call that from vendor code. Ooh, I think there's a bug here. On AMD, the guest event selector MSRs need to be loaded _before_ PERF_GLOBAL_CTRL, no? I.e. enable the guest's counters only after all selectors have been switched AMD64_EVENTSEL_GUESTONLY. Otherwise there would be a brief window where KVM could incorrectly enable counters in the host. And the reverse that for put().

But Intel has the opposite ordering, because MSR_CORE_PERF_GLOBAL_CTRL needs to be cleared before changing event selectors.

And so trying to handle this entirely in common code, while noble, is at best fragile and at worst buggy.

The common helper can take the bases and shift, and if we want to have it handle fixed counters, the base for that too.

#114 lore raw

Re: [RFC PATCH v3 31/58] KVM: x86/pmu: Add counter MSR and selector MSR index into struct kvm_pmc

Quoted 3 lines · Sean Christopherson

Please squash this with the patch that does the actual save/load. Hmm, maybe it should be put/load, now that I think about it more? That's more consitent with existing KVM terminology.

Sure. I ever noticed that this in-consistence, but "put" seem not so intuitionistic as "save", so didn't change it.

Quoted 63 lines

Anyways, please squash them together, it's very difficult to review them separately.

Quoted 33 lines · Mingwei Zhang

Add the MSR indices for both selector and counter in each kvm_pmc. Giving convenience to mediated passthrough vPMU in scenarios of querying MSR from a given pmc. Note that legacy vPMU does not need this because it never directly accesses PMU MSRs, instead each kvm_pmc is bound to a perf_event.

For actual Zen 4 and later hardware, it will never be the case that the PerfMonV2 CPUID bit is set but the PerfCtrCore bit is not. However, a guest can be booted with PerfMonV2 enabled and PerfCtrCore disabled. KVM does not clear the PerfMonV2 bit from guest CPUID as long as the host has the PerfCtrCore capability.

In this case, passthrough mode will use the K7 legacy MSRs to program events but with the incorrect assumption that there are 6 such counters instead of 4 as advertised by CPUID leaf 0x80000022 EBX. The host kernel will also report unchecked MSR accesses for the absent counters while saving or restoring guest PMU contexts.

Ensure that K7 legacy MSRs are not used as long as the guest CPUID has either PerfCtrCore or PerfMonV2 set.

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/svm/pmu.c          | 13 +++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c    | 13 +++++++++++++
 3 files changed, 28 insertions(+)
arch/x86/include/asm/kvm_host.h +0 -0
index 4b3ce6194bdb..603727312f9c 100644

arch/x86/include/asm/kvm_host.h struct kvm_pmc { · line 526

Quoted 5 lines
arch/x86/include/asm/kvm_host.h +2 -0
@@ -522,6 +522,8 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 msr_counter;
+	u64 msr_eventsel;

There's no need to track these per PMC, the tracking can be per PMU, e.g.

	u64 gp_eventsel_base;
	u64 gp_counter_base;
	u64 gp_shift;
	u64 fixed_base;

Actually, there's no need for a per-PMU fixed base, as that can be shoved into kvm_pmu_ops. LOL, and the upcoming patch hardcodes INTEL_PMC_FIXED_RDPMC_BASE. Naughty, naughty ;-)

It's not pretty, but 16 bytes per PMC isn't trivial.

Hmm, actually, scratch all that. A better alternative would be to provide a helper to put/load counter/selector MSRs, and call that from vendor code. Ooh, I think there's a bug here. On AMD, the guest event selector MSRs need to be loaded _before_ PERF_GLOBAL_CTRL, no? I.e. enable the guest's counters only after all selectors have been switched AMD64_EVENTSEL_GUESTONLY. Otherwise there would be a brief window where KVM could incorrectly enable counters in the host. And the reverse that for put().

Quoted 2 lines

But Intel has the opposite ordering, because MSR_CORE_PERF_GLOBAL_CTRL needs to be cleared before changing event selectors.

Not quite sure about AMD platforms, but it seems both Intel and AMD platforms follow below sequence to manipulated PMU MSRs.

disable PERF_GLOBAL_CTRL MSR

manipulate counter-level PMU MSR

enable PERF_GLOBAL_CTRL MSR

It seems there is no issues?

Quoted 6 lines

And so trying to handle this entirely in common code, while noble, is at best fragile and at worst buggy.

The common helper can take the bases and shift, and if we want to have it handle fixed counters, the base for that too.

Sure. Would add a callback and vendor specific code would fill the base and shift fields by leveraging the callback.

Quoted 1 line
#115 lore raw

Re: [RFC PATCH v3 31/58] KVM: x86/pmu: Add counter MSR and selector MSR index into struct kvm_pmc

Quoted 8 lines · Dapeng Mi
Quoted 3 lines · Sean Christopherson

Please squash this with the patch that does the actual save/load. Hmm, maybe it should be put/load, now that I think about it more? That's more consitent with existing KVM terminology.

Sure. I ever noticed that this in-consistence, but "put" seem not so intuitionistic as "save", so didn't change it.

Yeah, "put" isn't perfect, but neither is "save", because the save/put path also purges hardware state.

Quoted 34 lines
Quoted 31 lines

arch/x86/include/asm/kvm_host.h struct kvm_pmc { · line 526

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +2 -0
index 4b3ce6194bdb..603727312f9c 100644
@@ -522,6 +522,8 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 msr_counter;
+	u64 msr_eventsel;

There's no need to track these per PMC, the tracking can be per PMU, e.g.

	u64 gp_eventsel_base;
	u64 gp_counter_base;
	u64 gp_shift;
	u64 fixed_base;

Actually, there's no need for a per-PMU fixed base, as that can be shoved into kvm_pmu_ops. LOL, and the upcoming patch hardcodes INTEL_PMC_FIXED_RDPMC_BASE. Naughty, naughty ;-)

It's not pretty, but 16 bytes per PMC isn't trivial.

Hmm, actually, scratch all that. A better alternative would be to provide a helper to put/load counter/selector MSRs, and call that from vendor code. Ooh, I think there's a bug here. On AMD, the guest event selector MSRs need to be loaded _before_ PERF_GLOBAL_CTRL, no? I.e. enable the guest's counters only after all selectors have been switched AMD64_EVENTSEL_GUESTONLY. Otherwise there would be a brief window where KVM could incorrectly enable counters in the host. And the reverse that for put().

Quoted 2 lines

But Intel has the opposite ordering, because MSR_CORE_PERF_GLOBAL_CTRL needs to be cleared before changing event selectors.

Quoted 8 lines

Not quite sure about AMD platforms, but it seems both Intel and AMD platforms follow below sequence to manipulated PMU MSRs.

disable PERF_GLOBAL_CTRL MSR

manipulate counter-level PMU MSR

enable PERF_GLOBAL_CTRL MSR

Nope. kvm_pmu_restore_pmu_context() does:

	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);


	/*
	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
	 * in this case will be intercepted. Accessing to these counters and
	 * selectors will cause #GP in the guest.
	 */
	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
		pmc = &pmu->gp_counters[i];
		wrmsrl(pmc->msr_counter, pmc->counter);
		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel_hw);
	}

	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
		pmc = &pmu->fixed_counters[i];
		wrmsrl(pmc->msr_counter, pmc->counter);
	}

And amd_restore_pmu_context() does:

	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
	rdmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, global_status);

	/* Clear host global_status MSR if non-zero. */
	if (global_status)
		wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, global_status);

	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, pmu->global_status);

	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, pmu->global_ctrl);

So the sequence on AMD is currently:

disable PERF_GLOBAL_CTRL

save host PERF_GLOBAL_STATUS

load guest PERF_GLOBAL_STATUS (clear+set)

load guest PERF_GLOBAL_CTRL

load guest per-counter MSRs

#116 lore raw

Re: [RFC PATCH v3 31/58] KVM: x86/pmu: Add counter MSR and selector MSR index into struct kvm_pmc

Quoted 89 lines · Sean Christopherson
Quoted 6 lines · Dapeng Mi
Quoted 3 lines · Sean Christopherson

Please squash this with the patch that does the actual save/load. Hmm, maybe it should be put/load, now that I think about it more? That's more consitent with existing KVM terminology.

Sure. I ever noticed that this in-consistence, but "put" seem not so intuitionistic as "save", so didn't change it.

Yeah, "put" isn't perfect, but neither is "save", because the save/put path also purges hardware state.

Quoted 33 lines
Quoted 31 lines

arch/x86/include/asm/kvm_host.h struct kvm_pmc { · line 526

Quoted 10 lines
arch/x86/include/asm/kvm_host.h +2 -0
index 4b3ce6194bdb..603727312f9c 100644
@@ -522,6 +522,8 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 msr_counter;
+	u64 msr_eventsel;

There's no need to track these per PMC, the tracking can be per PMU, e.g.

	u64 gp_eventsel_base;
	u64 gp_counter_base;
	u64 gp_shift;
	u64 fixed_base;

Actually, there's no need for a per-PMU fixed base, as that can be shoved into kvm_pmu_ops. LOL, and the upcoming patch hardcodes INTEL_PMC_FIXED_RDPMC_BASE. Naughty, naughty ;-)

It's not pretty, but 16 bytes per PMC isn't trivial.

Hmm, actually, scratch all that. A better alternative would be to provide a helper to put/load counter/selector MSRs, and call that from vendor code. Ooh, I think there's a bug here. On AMD, the guest event selector MSRs need to be loaded _before_ PERF_GLOBAL_CTRL, no? I.e. enable the guest's counters only after all selectors have been switched AMD64_EVENTSEL_GUESTONLY. Otherwise there would be a brief window where KVM could incorrectly enable counters in the host. And the reverse that for put().

Quoted 2 lines

But Intel has the opposite ordering, because MSR_CORE_PERF_GLOBAL_CTRL needs to be cleared before changing event selectors.

Quoted 8 lines

Not quite sure about AMD platforms, but it seems both Intel and AMD platforms follow below sequence to manipulated PMU MSRs.

disable PERF_GLOBAL_CTRL MSR

manipulate counter-level PMU MSR

enable PERF_GLOBAL_CTRL MSR

Nope. kvm_pmu_restore_pmu_context() does:

	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);


	/*
	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
	 * in this case will be intercepted. Accessing to these counters and
	 * selectors will cause #GP in the guest.
	 */
	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
		pmc = &pmu->gp_counters[i];
		wrmsrl(pmc->msr_counter, pmc->counter);
		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel_hw);
	}

	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
		pmc = &pmu->fixed_counters[i];
		wrmsrl(pmc->msr_counter, pmc->counter);
	}

And amd_restore_pmu_context() does:

	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0);
	rdmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, global_status);

	/* Clear host global_status MSR if non-zero. */
	if (global_status)
		wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, global_status);

	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_SET, pmu->global_status);

	wrmsrl(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, pmu->global_ctrl);

So the sequence on AMD is currently:

disable PERF_GLOBAL_CTRL

Quoted 7 lines

save host PERF_GLOBAL_STATUS

load guest PERF_GLOBAL_STATUS (clear+set)

load guest PERF_GLOBAL_CTRL

load guest per-counter MSRs

Checked again, yes, indeed. So the better way to handle this is to define a common helper to manipulate counters MSR in the common code, but call this common helper in the vendor specific callback.

#117 lore raw

RFC PATCH v3 32/58 KVM: x86/pmu: Introduce PMU operation prototypes for save/restore PMU context

In reply to Mingwei Zhang

Plumb through kvm_pmu_ops with these two extra functions to allow pmu context switch.

---
 arch/x86/include/asm/kvm-x86-pmu-ops.h |  2 ++
 arch/x86/kvm/pmu.c                     | 14 ++++++++++++++
 arch/x86/kvm/pmu.h                     |  4 ++++
 3 files changed, 20 insertions(+)
arch/x86/include/asm/kvm-x86-pmu-ops.h +2 -0
index 1b7876dcb3c3..1a848ba6a7a7 100644
@@ -25,6 +25,8 @@ KVM_X86_PMU_OP_OPTIONAL(reset)
 KVM_X86_PMU_OP_OPTIONAL(deliver_pmi)
 KVM_X86_PMU_OP_OPTIONAL(cleanup)
 KVM_X86_PMU_OP_OPTIONAL(passthrough_pmu_msrs)
+KVM_X86_PMU_OP_OPTIONAL(save_pmu_context)
+KVM_X86_PMU_OP_OPTIONAL(restore_pmu_context)
 
 #undef KVM_X86_PMU_OP
 #undef KVM_X86_PMU_OP_OPTIONAL
arch/x86/kvm/pmu.c +14 -0
index e9047051489e..782b564bdf96 100644
@@ -1065,3 +1065,17 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 {
 	static_call_cond(kvm_x86_pmu_passthrough_pmu_msrs)(vcpu);
 }
+
+void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
+{
+	lockdep_assert_irqs_disabled();
+
+	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+}
+
+void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
+{
+	lockdep_assert_irqs_disabled();
+
+	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
+}
arch/x86/kvm/pmu.h +4 -0
index 63f876557716..8bd4b79e363f 100644
@@ -42,6 +42,8 @@ struct kvm_pmu_ops {
 	void (*cleanup)(struct kvm_vcpu *vcpu);
 	bool (*is_rdpmc_passthru_allowed)(struct kvm_vcpu *vcpu);
 	void (*passthrough_pmu_msrs)(struct kvm_vcpu *vcpu);
+	void (*save_pmu_context)(struct kvm_vcpu *vcpu);
+	void (*restore_pmu_context)(struct kvm_vcpu *vcpu);
 
 	const u64 EVENTSEL_EVENT;
 	const int MAX_NR_GP_COUNTERS;
@@ -294,6 +296,8 @@ int kvm_vm_ioctl_set_pmu_event_filter(struct kvm *kvm, void __user *argp);
 void kvm_pmu_trigger_event(struct kvm_vcpu *vcpu, u64 eventsel);
 bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
 void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
+void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu);
+void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#118 lore raw

RFC PATCH v3 33/58 KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU

In reply to Mingwei Zhang

Implement the save/restore of PMU state for pasthrough PMU in Intel. In passthrough mode, KVM owns exclusively the PMU HW when control flow goes to the scope of passthrough PMU. Thus, KVM needs to save the host PMU state and gains the full HW PMU ownership. On the contrary, host regains the ownership of PMU HW from KVM when control flow leaves the scope of passthrough PMU.

Implement PMU context switches for Intel CPUs and opptunistically use rdpmcl() instead of rdmsrl() when reading counters since the former has lower latency in Intel CPUs.

---
 arch/x86/kvm/pmu.c           | 46 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c | 41 +++++++++++++++++++++++++++++++-
 2 files changed, 86 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +46 -0
index 782b564bdf96..9bb733384069 100644
@@ -1068,14 +1068,60 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	u32 i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+
+	/*
+	 * Clear hardware selector MSR content and its counter to avoid
+	 * leakage and also avoid this guest GP counter get accidentally
+	 * enabled during host running when host enable global ctrl.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		rdpmcl(i, pmc->counter);
+		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+		if (pmc->eventsel)
+			wrmsrl(pmc->msr_eventsel, 0);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		rdpmcl(INTEL_PMC_FIXED_RDPMC_BASE | i, pmc->counter);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+	}
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	int i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);
+
+	/*
+	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
+	 * in this case will be intercepted. Accessing to these counters and
+	 * selectors will cause #GP in the guest.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+	}
 }
arch/x86/kvm/vmx/pmu_intel.c +40 -1

59 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#119 lore raw

Re: [RFC PATCH v3 33/58] KVM: x86/pmu: Implement the save/restore of PMU state for Intel CPU

Quoted 68 lines · Mingwei Zhang

Implement the save/restore of PMU state for pasthrough PMU in Intel. In passthrough mode, KVM owns exclusively the PMU HW when control flow goes to the scope of passthrough PMU. Thus, KVM needs to save the host PMU state and gains the full HW PMU ownership. On the contrary, host regains the ownership of PMU HW from KVM when control flow leaves the scope of passthrough PMU.

Implement PMU context switches for Intel CPUs and opptunistically use rdpmcl() instead of rdmsrl() when reading counters since the former has lower latency in Intel CPUs.

---
 arch/x86/kvm/pmu.c           | 46 ++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/pmu_intel.c | 41 +++++++++++++++++++++++++++++++-
 2 files changed, 86 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +30 -0
index 782b564bdf96..9bb733384069 100644
@@ -1068,14 +1068,60 @@ void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	u32 i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_save_pmu_context)(vcpu);
+
+	/*
+	 * Clear hardware selector MSR content and its counter to avoid
+	 * leakage and also avoid this guest GP counter get accidentally
+	 * enabled during host running when host enable global ctrl.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		rdpmcl(i, pmc->counter);
+		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+		if (pmc->eventsel)
+			wrmsrl(pmc->msr_eventsel, 0);
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		rdpmcl(INTEL_PMC_FIXED_RDPMC_BASE | i, pmc->counter);
+		if (pmc->counter)
+			wrmsrl(pmc->msr_counter, 0);
+	}
 }
 
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 {
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	struct kvm_pmc *pmc;
+	int i;
+
 	lockdep_assert_irqs_disabled();
 
 	static_call_cond(kvm_x86_pmu_restore_pmu_context)(vcpu);

arch/x86/kvm/pmu.c void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu) · line 1120

Quoted 10 lines
arch/x86/kvm/pmu.c +10 -0
+
+	/*
+	 * No need to zero out unexposed GP/fixed counters/selectors since RDPMC
+	 * in this case will be intercepted. Accessing to these counters and
+	 * selectors will cause #GP in the guest.
+	 */
+	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
+		pmc = &pmu->gp_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);

pmu->gp_counters[i].eventsel  -> pmc->eventsel

Quoted 12 lines
arch/x86/kvm/pmu.c +6 -0
+	}
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = &pmu->fixed_counters[i];
+		wrmsrl(pmc->msr_counter, pmc->counter);
+	}
 }
arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 0de918dc14ea..89c8f73a48c8 100644

arch/x86/kvm/vmx/pmu_intel.c static void intel_pmu_refresh(struct kvm_vcpu *vcpu) · line 575

Quoted 5 lines
arch/x86/kvm/vmx/pmu_intel.c +1 -1
@@ -572,7 +572,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
 	}
 
 	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
-		pmu->fixed_counters[i].msr_eventsel = MSR_CORE_PERF_FIXED_CTR_CTRL;
+		pmu->fixed_counters[i].msr_eventsel = 0;

Seems unnecessary to clear msr_eventsel for fixed counters. Why?

Quoted 56 lines
arch/x86/kvm/vmx/pmu_intel.c +39 -0
 		pmu->fixed_counters[i].msr_counter = MSR_CORE_PERF_FIXED_CTR0 + i;
 	}
 }
@@ -799,6 +799,43 @@ static void intel_passthrough_pmu_msrs(struct kvm_vcpu *vcpu)
 	vmx_set_intercept_for_msr(vcpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, MSR_TYPE_RW, msr_intercept);
 }
 
+static void intel_save_guest_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+
+	/* Global ctrl register is already saved at VM-exit. */
+	rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, pmu->global_status);
+	/* Clear hardware MSR_CORE_PERF_GLOBAL_STATUS MSR, if non-zero. */
+	if (pmu->global_status)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status);
+
+	rdmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+	/*
+	 * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and
+	 * also avoid these guest fixed counters get accidentially enabled
+	 * during host running when host enable global ctrl.
+	 */
+	if (pmu->fixed_ctr_ctrl)
+		wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, 0);
+}
+
+static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
+	u64 global_status, toggle;
+
+	/* Clear host global_ctrl MSR if non-zero. */
+	wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, 0);
+	rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, global_status);
+	toggle = pmu->global_status ^ global_status;
+	if (global_status & toggle)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, global_status & toggle);
+	if (pmu->global_status & toggle)
+		wrmsrl(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
+
+	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+}
+
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.rdpmc_ecx_to_pmc = intel_rdpmc_ecx_to_pmc,
 	.msr_idx_to_pmc = intel_msr_idx_to_pmc,
@@ -812,6 +849,8 @@ struct kvm_pmu_ops intel_pmu_ops __initdata = {
 	.cleanup = intel_pmu_cleanup,
 	.is_rdpmc_passthru_allowed = intel_is_rdpmc_passthru_allowed,
 	.passthrough_pmu_msrs = intel_passthrough_pmu_msrs,
+	.save_pmu_context = intel_save_guest_pmu_context,
+	.restore_pmu_context = intel_restore_guest_pmu_context,
 	.EVENTSEL_EVENT = ARCH_PERFMON_EVENTSEL_EVENT,
 	.MAX_NR_GP_COUNTERS = KVM_INTEL_PMC_MAX_GENERIC,
 	.MIN_NR_GP_COUNTERS = 1,
#120 lore raw

RFC PATCH v3 34/58 KVM: x86/pmu: Make check_pmu_event_filter() an exported function

In reply to Mingwei Zhang

Make check_pmu_event_filter() exported to usable by vendor modules like kvm_intel. This is because passthrough PMU intercept the guest writes to event selectors and directly do the event filter checking inside the vendor specific set_msr() instead of deferring to the KVM_REQ_PMU handler.

---
 arch/x86/kvm/pmu.c | 3 ++-
 arch/x86/kvm/pmu.h | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)
arch/x86/kvm/pmu.c +2 -1
index 9bb733384069..9aa08472b7df 100644
@@ -443,7 +443,7 @@ static bool is_fixed_event_allowed(struct kvm_x86_pmu_event_filter *filter,
 	return true;
 }
 
-static bool check_pmu_event_filter(struct kvm_pmc *pmc)
+bool check_pmu_event_filter(struct kvm_pmc *pmc)
 {
 	struct kvm_x86_pmu_event_filter *filter;
 	struct kvm *kvm = pmc->vcpu->kvm;
@@ -457,6 +457,7 @@ static bool check_pmu_event_filter(struct kvm_pmc *pmc)
 
 	return is_fixed_event_allowed(filter, pmc->idx);
 }
+EXPORT_SYMBOL_GPL(check_pmu_event_filter);
 
 static bool pmc_event_is_allowed(struct kvm_pmc *pmc)
 {
arch/x86/kvm/pmu.h +1 -0
index 8bd4b79e363f..9cde62f3988e 100644
@@ -298,6 +298,7 @@ bool kvm_pmu_check_rdpmc_passthrough(struct kvm_vcpu *vcpu);
 void kvm_pmu_passthrough_pmu_msrs(struct kvm_vcpu *vcpu);
 void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu);
 void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu);
+bool check_pmu_event_filter(struct kvm_pmc *pmc);
 
 bool is_vmware_backdoor_pmc(u32 pmc_idx);
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#121 lore raw

RFC PATCH v3 35/58 KVM: x86/pmu: Allow writing to event selector for GP counters if event is allowed

In reply to Mingwei Zhang

Only allow writing to event selector if event is allowed in filter. Since passthrough PMU implementation does the PMU context switch at VM Enter/Exit boudary, even if the value of event selector passes the checking, it cannot be written directly to HW since PMU HW is owned by the host PMU at the moment. Because of that, introduce eventsel_hw to cache that value which will be assigned into HW just before VM entry.

Note that regardless of whether an event value is allowed, the value will be cached in pmc->eventsel and guest VM can always read the cached value back. This implementation is consistent with the HW CPU design.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/pmu.c              |  5 ++---
 arch/x86/kvm/vmx/pmu_intel.c    | 13 ++++++++++++-
 3 files changed, 15 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index 603727312f9c..e5c288d4264f 100644
@@ -522,6 +522,7 @@ struct kvm_pmc {
 	 */
 	u64 emulated_counter;
 	u64 eventsel;
+	u64 eventsel_hw;
 	u64 msr_counter;
 	u64 msr_eventsel;
 	struct perf_event *perf_event;
arch/x86/kvm/pmu.c +2 -3
index 9aa08472b7df..545930f743b9 100644
@@ -1085,10 +1085,9 @@ void kvm_pmu_save_pmu_context(struct kvm_vcpu *vcpu)
 	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
 		pmc = &pmu->gp_counters[i];
 		rdpmcl(i, pmc->counter);
-		rdmsrl(pmc->msr_eventsel, pmc->eventsel);
 		if (pmc->counter)
 			wrmsrl(pmc->msr_counter, 0);
-		if (pmc->eventsel)
+		if (pmc->eventsel_hw)
 			wrmsrl(pmc->msr_eventsel, 0);
 	}
 
@@ -1118,7 +1117,7 @@ void kvm_pmu_restore_pmu_context(struct kvm_vcpu *vcpu)
 	for (i = 0; i < pmu->nr_arch_gp_counters; i++) {
 		pmc = &pmu->gp_counters[i];
 		wrmsrl(pmc->msr_counter, pmc->counter);
-		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel);
+		wrmsrl(pmc->msr_eventsel, pmu->gp_counters[i].eventsel_hw);
 	}
 
 	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
arch/x86/kvm/vmx/pmu_intel.c +12 -1
index 89c8f73a48c8..0cd38c5632ee 100644
@@ -399,7 +399,18 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			if (data & reserved_bits)
 				return 1;
 
-			if (data != pmc->eventsel) {
+			if (is_passthrough_pmu_enabled(vcpu)) {
+				pmc->eventsel = data;
+				if (!check_pmu_event_filter(pmc)) {
+					if (pmc->eventsel_hw &
+					    ARCH_PERFMON_EVENTSEL_ENABLE) {
+						pmc->eventsel_hw &= ~ARCH_PERFMON_EVENTSEL_ENABLE;
+						pmc->counter = 0;
+					}
+					return 0;
+				}
+				pmc->eventsel_hw = data;
+			} else if (data != pmc->eventsel) {
 				pmc->eventsel = data;
 				kvm_pmu_request_counter_reprogram(pmc);
 			}
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#122 lore raw

RFC PATCH v3 36/58 KVM: x86/pmu: Allow writing to fixed counter selector if counter is exposed

In reply to Mingwei Zhang

Allow writing to fixed counter selector if counter is exposed. If this fixed counter is filtered out, this counter won't be enabled on HW.

Passthrough PMU implements the context switch at VM Enter/Exit boundary the guest value cannot be directly written to HW since the HW PMU is owned by the host. Introduce a new field fixed_ctr_ctrl_hw in kvm_pmu to cache the guest value. which will be assigne to HW at PMU context restore.

Since passthrough PMU intercept writes to fixed counter selector, there is no need to read the value at pmu context save, but still clear the fix counter ctrl MSR and counters when switching out to host PMU.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/vmx/pmu_intel.c    | 28 ++++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index e5c288d4264f..93c17da8271d 100644
@@ -549,6 +549,7 @@ struct kvm_pmu {
 	unsigned nr_arch_fixed_counters;
 	unsigned available_event_types;
 	u64 fixed_ctr_ctrl;
+	u64 fixed_ctr_ctrl_hw;
 	u64 fixed_ctr_ctrl_mask;
 	u64 global_ctrl;
 	u64 global_status;
arch/x86/kvm/vmx/pmu_intel.c +24 -4
index 0cd38c5632ee..c61936266cbd 100644
@@ -34,6 +34,25 @@
 
 #define MSR_PMC_FULL_WIDTH_BIT      (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)
 
+static void reprogram_fixed_counters_in_passthrough_pmu(struct kvm_pmu *pmu, u64 data)
+{
+	struct kvm_pmc *pmc;
+	u64 new_data = 0;
+	int i;
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = get_fixed_pmc(pmu, MSR_CORE_PERF_FIXED_CTR0 + i);
+		if (check_pmu_event_filter(pmc)) {
+			pmc->current_config = fixed_ctrl_field(data, i);
+			new_data |= (pmc->current_config << (i * 4));
+		} else {
+			pmc->counter = 0;
+		}
+	}
+	pmu->fixed_ctr_ctrl_hw = new_data;
+	pmu->fixed_ctr_ctrl = data;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	struct kvm_pmc *pmc;
@@ -351,7 +370,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (data & pmu->fixed_ctr_ctrl_mask)
 			return 1;
 
-		if (pmu->fixed_ctr_ctrl != data)
+		if (is_passthrough_pmu_enabled(vcpu))
+			reprogram_fixed_counters_in_passthrough_pmu(pmu, data);
+		else if (pmu->fixed_ctr_ctrl != data)
 			reprogram_fixed_counters(pmu, data);
 		break;
 	case MSR_IA32_PEBS_ENABLE:
@@ -820,13 +841,12 @@ static void intel_save_guest_pmu_context(struct kvm_vcpu *vcpu)
 	if (pmu->global_status)
 		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status);
 
-	rdmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
 	/*
 	 * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and
 	 * also avoid these guest fixed counters get accidentially enabled
 	 * during host running when host enable global ctrl.
 	 */
-	if (pmu->fixed_ctr_ctrl)
+	if (pmu->fixed_ctr_ctrl_hw)
 		wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, 0);
 }
 
@@ -844,7 +864,7 @@ static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
 	if (pmu->global_status & toggle)
 		wrmsrl(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
 
-	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
 }
 
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#123 lore raw

Re: [RFC PATCH v3 36/58] KVM: x86/pmu: Allow writing to fixed counter selector if counter is exposed

Quoted 41 lines · Mingwei Zhang

Allow writing to fixed counter selector if counter is exposed. If this fixed counter is filtered out, this counter won't be enabled on HW.

Passthrough PMU implements the context switch at VM Enter/Exit boundary the guest value cannot be directly written to HW since the HW PMU is owned by the host. Introduce a new field fixed_ctr_ctrl_hw in kvm_pmu to cache the guest value. which will be assigne to HW at PMU context restore.

Since passthrough PMU intercept writes to fixed counter selector, there is no need to read the value at pmu context save, but still clear the fix counter ctrl MSR and counters when switching out to host PMU.

---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/vmx/pmu_intel.c    | 28 ++++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
index e5c288d4264f..93c17da8271d 100644
@@ -549,6 +549,7 @@ struct kvm_pmu {
 	unsigned nr_arch_fixed_counters;
 	unsigned available_event_types;
 	u64 fixed_ctr_ctrl;
+	u64 fixed_ctr_ctrl_hw;
 	u64 fixed_ctr_ctrl_mask;
 	u64 global_ctrl;
 	u64 global_status;
arch/x86/kvm/vmx/pmu_intel.c +1 -0
index 0cd38c5632ee..c61936266cbd 100644
@@ -34,6 +34,25 @@
 
 #define MSR_PMC_FULL_WIDTH_BIT      (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0)
 
+static void reprogram_fixed_counters_in_passthrough_pmu(struct kvm_pmu *pmu, u64 data)

arch/x86/kvm/vmx/pmu_intel.c line 47

Quoted 10 lines
arch/x86/kvm/vmx/pmu_intel.c +10 -0
+{
+	struct kvm_pmc *pmc;
+	u64 new_data = 0;
+	int i;
+
+	for (i = 0; i < pmu->nr_arch_fixed_counters; i++) {
+		pmc = get_fixed_pmc(pmu, MSR_CORE_PERF_FIXED_CTR0 + i);
+		if (check_pmu_event_filter(pmc)) {
+			pmc->current_config = fixed_ctrl_field(data, i);
+			new_data |= (pmc->current_config << (i * 4));

Since we already have macro intel_fixed_bits_by_idx() to manipulate fixed_cntr_ctrl, we 'd better to use it.

arch/x86/kvm/vmx/pmu_intel.c +0 -0
index 3dbeb41b85ab..0aa58bffb99d 100644
@@ -44,7 +44,7 @@ static void

reprogram_fixed_counters_in_passthrough_pmu(struct kvm_pmu *pmu, u64 pmc = get_fixed_pmc(pmu, MSR_CORE_PERF_FIXED_CTR0 + i); if (check_pmu_event_filter(pmc)) { pmc->current_config = fixed_ctrl_field(data, i); -                       new_data |= (pmc->current_config << (i * 4)); +                       new_data |= intel_fixed_bits_by_idx(i, pmc->current_config);
                } else {
                        pmc->counter = 0;
                }

Quoted 46 lines
arch/x86/kvm/vmx/pmu_intel.c +13 -4
+		} else {
+			pmc->counter = 0;
+		}
+	}
+	pmu->fixed_ctr_ctrl_hw = new_data;
+	pmu->fixed_ctr_ctrl = data;
+}
+
 static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data)
 {
 	struct kvm_pmc *pmc;
@@ -351,7 +370,9 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 		if (data & pmu->fixed_ctr_ctrl_mask)
 			return 1;
 
-		if (pmu->fixed_ctr_ctrl != data)
+		if (is_passthrough_pmu_enabled(vcpu))
+			reprogram_fixed_counters_in_passthrough_pmu(pmu, data);
+		else if (pmu->fixed_ctr_ctrl != data)
 			reprogram_fixed_counters(pmu, data);
 		break;
 	case MSR_IA32_PEBS_ENABLE:
@@ -820,13 +841,12 @@ static void intel_save_guest_pmu_context(struct kvm_vcpu *vcpu)
 	if (pmu->global_status)
 		wrmsrl(MSR_CORE_PERF_GLOBAL_OVF_CTRL, pmu->global_status);
 
-	rdmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
 	/*
 	 * Clear hardware FIXED_CTR_CTRL MSR to avoid information leakage and
 	 * also avoid these guest fixed counters get accidentially enabled
 	 * during host running when host enable global ctrl.
 	 */
-	if (pmu->fixed_ctr_ctrl)
+	if (pmu->fixed_ctr_ctrl_hw)
 		wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, 0);
 }
 
@@ -844,7 +864,7 @@ static void intel_restore_guest_pmu_context(struct kvm_vcpu *vcpu)
 	if (pmu->global_status & toggle)
 		wrmsrl(MSR_CORE_PERF_GLOBAL_STATUS_SET, pmu->global_status & toggle);
 
-	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl);
+	wrmsrl(MSR_CORE_PERF_FIXED_CTR_CTRL, pmu->fixed_ctr_ctrl_hw);
 }
 
 struct kvm_pmu_ops intel_pmu_ops __initdata = {
#124 lore raw

RFC PATCH v3 37/58 KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary

In reply to Mingwei Zhang

From: Xiong Zhang <[email protected]>

In PMU passthrough mode, use global_ctrl field in struct kvm_pmu as the cached value. This is convenient for KVM to set and get the value from the host side. In addition, load and save the value across VM enter/exit boundary in the following way:

- At VM exit, if processor supports GUEST_VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL, read guest IA32_PERF_GLOBAL_CTRL GUEST_IA32_PERF_GLOBAL_CTRL VMCS field, else read it from VM-exit MSR-stroe array in VMCS. The value is then assigned to global_ctrl.

- At VM Entry, if processor supports GUEST_VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL, read guest IA32_PERF_GLOBAL_CTRL from GUEST_IA32_PERF_GLOBAL_CTRL VMCS field, else read it from VM-entry MSR-load array in VMCS. The value is then assigned to global ctrl.

Implement the above logic into two helper functions and invoke them around VM Enter/exit boundary.

---
 arch/x86/include/asm/kvm_host.h |  2 ++
 arch/x86/kvm/vmx/vmx.c          | 49 ++++++++++++++++++++++++++++++++-
 2 files changed, 50 insertions(+), 1 deletion(-)
arch/x86/include/asm/kvm_host.h +2 -0
index 93c17da8271d..7bf901a53543 100644
@@ -601,6 +601,8 @@ struct kvm_pmu {
 	u8 event_count;
 
 	bool passthrough;
+	int global_ctrl_slot_in_autoload;
+	int global_ctrl_slot_in_autostore;
 };
 
 struct kvm_pmu_ops;
arch/x86/kvm/vmx/vmx.c +48 -1

91 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#125 lore raw

Re: [RFC PATCH v3 37/58] KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7308

Quoted 11 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +7 -0
@@ -7295,6 +7299,46 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 					msrs[i].host, false);
 }
 
+static void save_perf_global_ctrl_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(&vmx->vcpu);
+	int i;
+
+	if (vm_exit_controls_get(vmx) & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL) {
+		pmu->global_ctrl = vmcs_read64(GUEST_IA32_PERF_GLOBAL_CTRL);

As commented in patch 26, compared with MSR auto save/store area approach, the exec control way needs one relatively expensive VMCS read on every VM exit.

Quoted 4 lines
arch/x86/kvm/vmx/vmx.c +4 -0
+	} else {
+		i = pmu->global_ctrl_slot_in_autostore;
+		pmu->global_ctrl = vmx->msr_autostore.guest.val[i].value;
+	}

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7322

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+}
+
+static void load_perf_global_ctrl_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(&vmx->vcpu);
+	u64 global_ctrl = pmu->global_ctrl;
+	int i;
+
+	if (vm_entry_controls_get(vmx) & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+		vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, global_ctrl);

ditto.

We may optimize it by introducing a new flag pmu->global_ctrl_dirty and update GUEST_IA32_PERF_GLOBAL_CTRL only when it's needed. But this makes the code even more complicated.

Quoted 22 lines
arch/x86/kvm/vmx/vmx.c +19 -0
+	} else {
+		i = pmu->global_ctrl_slot_in_autoload;
+		vmx->msr_autoload.guest.val[i].value = global_ctrl;
+	}
+}
+
+static void __atomic_switch_perf_msrs_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	load_perf_global_ctrl_in_passthrough_pmu(vmx);
+}
+
+static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
+{
+	if (is_passthrough_pmu_enabled(&vmx->vcpu))
+		__atomic_switch_perf_msrs_in_passthrough_pmu(vmx);
+	else
+		__atomic_switch_perf_msrs(vmx);
+}
+
 static void vmx_update_hv_timer(struct kvm_vcpu *vcpu, bool force_immediate_exit)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
#126 lore raw

Re: [RFC PATCH v3 37/58] KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary

Quoted 7 lines · Zide
Quoted 4 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +0 -0
@@ -7295,6 +7299,46 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 					msrs[i].host, false);
 }
 
Quoted 10 lines

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7308

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +7 -0
+static void save_perf_global_ctrl_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(&vmx->vcpu);
+	int i;
+
+	if (vm_exit_controls_get(vmx) & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL) {
+		pmu->global_ctrl = vmcs_read64(GUEST_IA32_PERF_GLOBAL_CTRL);

As commented in patch 26, compared with MSR auto save/store area approach, the exec control way needs one relatively expensive VMCS read on every VM exit.

Anyway, let us have a evaluation and data speaks.

Quoted 45 lines
Quoted 5 lines
arch/x86/kvm/vmx/vmx.c +4 -0
+	} else {
+		i = pmu->global_ctrl_slot_in_autostore;
+		pmu->global_ctrl = vmx->msr_autostore.guest.val[i].value;
+	}

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7322

Quoted 10 lines
arch/x86/kvm/vmx/vmx.c +10 -0
+}
+
+static void load_perf_global_ctrl_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(&vmx->vcpu);
+	u64 global_ctrl = pmu->global_ctrl;
+	int i;
+
+	if (vm_entry_controls_get(vmx) & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+		vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, global_ctrl);

ditto.

We may optimize it by introducing a new flag pmu->global_ctrl_dirty and update GUEST_IA32_PERF_GLOBAL_CTRL only when it's needed. But this makes the code even more complicated.

Quoted 22 lines
arch/x86/kvm/vmx/vmx.c +19 -0
+	} else {
+		i = pmu->global_ctrl_slot_in_autoload;
+		vmx->msr_autoload.guest.val[i].value = global_ctrl;
+	}
+}
+
+static void __atomic_switch_perf_msrs_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	load_perf_global_ctrl_in_passthrough_pmu(vmx);
+}
+
+static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
+{
+	if (is_passthrough_pmu_enabled(&vmx->vcpu))
+		__atomic_switch_perf_msrs_in_passthrough_pmu(vmx);
+	else
+		__atomic_switch_perf_msrs(vmx);
+}
+
 static void vmx_update_hv_timer(struct kvm_vcpu *vcpu, bool force_immediate_exit)
 {
 	struct vcpu_vmx *vmx = to_vmx(vcpu);
#127 lore raw

Re: [RFC PATCH v3 37/58] KVM: x86/pmu: Switch IA32_PERF_GLOBAL_CTRL at VM boundary

Quoted 20 lines · Dapeng Mi
Quoted 7 lines · Zide
Quoted 4 lines · Mingwei Zhang
arch/x86/kvm/vmx/vmx.c +0 -0
@@ -7295,6 +7299,46 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx)
 					msrs[i].host, false);
 }
 
Quoted 10 lines

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7308

Quoted 7 lines
arch/x86/kvm/vmx/vmx.c +7 -0
+static void save_perf_global_ctrl_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(&vmx->vcpu);
+	int i;
+
+	if (vm_exit_controls_get(vmx) & VM_EXIT_SAVE_IA32_PERF_GLOBAL_CTRL) {
+		pmu->global_ctrl = vmcs_read64(GUEST_IA32_PERF_GLOBAL_CTRL);

As commented in patch 26, compared with MSR auto save/store area approach, the exec control way needs one relatively expensive VMCS read on every VM exit.

Quoted 1 line

Anyway, let us have a evaluation and data speaks.

No, drop the unconditional VMREAD and VMWRITE, one way or another. No benchmark will notice ~50 extra cycles, but if we write poor code for every feature, those 50 cycles per feature add up.

Furthermore, checking to see if the CPU supports the load/save VMCS controls at runtime beyond ridiculous. The mediated PMU requires ***VERSION 4***; if a CPU supports PMU version 4 and doesn't support the VMCS controls, KVM should yell and disable the passthrough PMU. The amount of complexity added here to support a CPU that should never exist is silly.

Quoted 10 lines
Quoted 10 lines

arch/x86/kvm/vmx/vmx.c static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) · line 7322

Quoted 8 lines
arch/x86/kvm/vmx/vmx.c +8 -0
+static void load_perf_global_ctrl_in_passthrough_pmu(struct vcpu_vmx *vmx)
+{
+	struct kvm_pmu *pmu = vcpu_to_pmu(&vmx->vcpu);
+	u64 global_ctrl = pmu->global_ctrl;
+	int i;
+
+	if (vm_entry_controls_get(vmx) & VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL) {
+		vmcs_write64(GUEST_IA32_PERF_GLOBAL_CTRL, global_ctrl);

ditto.

Quoted 3 lines
Quoted 3 lines

We may optimize it by introducing a new flag pmu->global_ctrl_dirty and update GUEST_IA32_PERF_GLOBAL_CTRL only when it's needed. But this makes the code even more complicated.

I haven't looked at surrounding code too much, but I guarantee there's _zero_ reason to eat a VMWRITE+VMREAD on every transition. If, emphasis on *if*, KVM accesses PERF_GLOBAL_CTRL frequently, e.g. on most exits, then add a VCPU_EXREG_XXX and let KVM's caching infrastructure do the heavy lifting. Don't reinvent the wheel. But first, convince the world that KVM actually accesses the MSR somewhat frequently.

I'll do a more thorough review of this series in the coming weeks (days?). I singled out this one because I happened to stumble across the code when digging into something else.