Skip to contentSkip to conversation

Implement PEBS virtualization for Silvermont

PATCH In review Andi Kleen 8 participants 29 messages May 30, 2014 – Jun 25, 2014

Subsystems (4)
x86 KVM perf events KVM x86
lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

PEBS is very useful (e.g. enabling the more cycles:pp event or memory profiling) Unfortunately it didn't work in virtualization, which is becoming more and more common.

This patch kit implements simple PEBS virtualization for KVM on Silvermont CPUs. Silvermont does not have the leak problems that prevented successfull PEBS virtualization earlier.

It needs some (simple) modifcations on the host perf code, in addition to a PEBS device model in KVM. The guest does not need any modifications.

It also requires running with -cpu host. This may in term cause some other problems with the guest perf (due to writes to various missing MSRs), but these can be addressed separately.

For more details please see the description of the individual patches.

Available in
git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git perf/kvm-pebs-slm

-Andi

author
#1 lore raw

PATCH 1/4 perf: Add PEBS virtualization enable for Silvermont

From: Andi Kleen <[email protected]>

To avoid various problems (like leaking counters) the PEBS virtualization needs white listing per CPU model. Add state to the x86_pmu for this and enable it for Silvermont.

Silvermont is currently the only CPU where it is safe to virtualize PEBS, as it doesn't leak PEBS event through exits (as long as the exit MSR list disables the counter with PEBS_ENABLE)

Also Silvermont is relatively simple to handle,
as it only has one PEBS counter.

Also export the information to (modular) KVM.

Used in followon patches.

Signed-off-by Andi Kleen <[email protected]>
---
 arch/x86/include/asm/perf_event.h         |  6 ++++++
 arch/x86/kernel/cpu/perf_event.h          |  1 +
 arch/x86/kernel/cpu/perf_event_intel.c    |  1 +
 arch/x86/kernel/cpu/perf_event_intel_ds.c | 20 ++++++++++++++++++++
 4 files changed, 28 insertions(+)
arch/x86/include/asm/perf_event.h +6 -0
index 8249df4..c49c7d3 100644
@@ -250,6 +250,9 @@ struct perf_guest_switch_msr {
 extern struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr);
 extern void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap);
 extern void perf_check_microcode(void);
+extern unsigned long long perf_get_ds_area(void);
+extern unsigned long long perf_get_pebs_enable(void);
+extern bool perf_pebs_virtualization(void);
 #else
 static inline struct perf_guest_switch_msr *perf_guest_get_msrs(int *nr)
 {

@@ -264,6 +267,9 @@ static inline void perf_get_x86_pmu_capability(struct x86_pmu_capability *cap)
 
 static inline void perf_events_lapic_init(void)	{ }
 static inline void perf_check_microcode(void) { }
+static inline unsigned long long perf_get_ds_area(void) { return 0; }
+static inline unsigned long long perf_get_pebs_enable(void) { return 0; }
+static inline bool perf_pebs_virtualization(void) { return false; }
 #endif
 
 #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_AMD)
arch/x86/kernel/cpu/perf_event.h +1 -0
index 3b2f9bd..6ab8fdd 100644
@@ -449,6 +449,7 @@ struct x86_pmu {
 	struct event_constraint *pebs_constraints;
 	void		(*pebs_aliases)(struct perf_event *event);
 	int 		max_pebs_events;
+	bool		pebs_virtualization;
 
 	/*
 	 * Intel LBR
arch/x86/kernel/cpu/perf_event_intel.c +1 -0
index aa333d9..86ccb81 100644
@@ -2399,6 +2399,7 @@ __init int intel_pmu_init(void)
 		x86_pmu.pebs_constraints = intel_slm_pebs_event_constraints;
 		x86_pmu.extra_regs = intel_slm_extra_regs;
 		x86_pmu.er_flags |= ERF_HAS_RSP_1;
+		x86_pmu.pebs_virtualization = true;
 		pr_cont("Silvermont events, ");
 		break;
arch/x86/kernel/cpu/perf_event_intel_ds.c +20 -0
index ae96cfa..29622a7 100644
@@ -429,6 +429,26 @@ void reserve_ds_buffers(void)
 	put_online_cpus();
 }
 
+unsigned long long perf_get_ds_area(void)
+{
+	return (u64)__get_cpu_var(cpu_hw_events).ds;
+}
+EXPORT_SYMBOL_GPL(perf_get_ds_area);
+
+unsigned long long perf_get_pebs_enable(void)
+{
+	struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events);
+
+	return cpuc->pebs_enabled;
+}
+EXPORT_SYMBOL_GPL(perf_get_pebs_enable);
+
+bool perf_pebs_virtualization(void)
+{
+	return x86_pmu.pebs_virtualization;
+}
+EXPORT_SYMBOL_GPL(perf_pebs_virtualization);
+
 /*
  * BTS
  */
Signature · 1 line
1.9.0
author
#2 lore raw

PATCH 2/4 perf: Allow guest PEBS for KVM owned counters

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

Currently perf unconditionally disables PEBS for guest.

Now that we have the infrastructure in place to handle it we can allow it for KVM owned guest events. For the perf needs to know that a event is owned by a guest. Add a new state bit in the perf_event for that.

The bit is only set by KVM and cannot be selected by anyone else.

Then change the MSR entry/exit list to allow PEBS for these counters.

Signed-off-by Andi Kleen <[email protected]>
---
 arch/x86/kernel/cpu/perf_event.h       |  1 +
 arch/x86/kernel/cpu/perf_event_intel.c | 14 +++++++++++---
 arch/x86/kvm/pmu.c                     |  1 +
 include/linux/perf_event.h             | 15 ++++++++++++++-
 kernel/events/core.c                   |  7 ++++---
 5 files changed, 31 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/perf_event.h +1 -0
index 6ab8fdd..422bca5 100644
@@ -163,6 +163,7 @@ struct cpu_hw_events {
 	 */
 	u64				intel_ctrl_guest_mask;
 	u64				intel_ctrl_host_mask;
+	u64				intel_ctrl_guest_owned;
 	struct perf_guest_switch_msr	guest_switch_msrs[X86_PMC_IDX_MAX];
 
 	/*
arch/x86/kernel/cpu/perf_event_intel.c +11 -3
index 86ccb81..3bcfda0 100644
@@ -1202,6 +1202,7 @@ static void intel_pmu_disable_event(struct perf_event *event)
 
 	cpuc->intel_ctrl_guest_mask &= ~(1ull << hwc->idx);
 	cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx);
+	cpuc->intel_ctrl_guest_owned &= ~(1ull << hwc->idx);
 	cpuc->intel_cp_status &= ~(1ull << hwc->idx);
 
 	/*

@@ -1274,6 +1275,8 @@ static void intel_pmu_enable_event(struct perf_event *event)
 
 	if (event->attr.exclude_host)
 		cpuc->intel_ctrl_guest_mask |= (1ull << hwc->idx);
+	if (event->guest_owned)
+		cpuc->intel_ctrl_guest_owned |= (1ull << hwc->idx);
 	if (event->attr.exclude_guest)
 		cpuc->intel_ctrl_host_mask |= (1ull << hwc->idx);
 

@@ -1775,18 +1778,23 @@ static struct perf_guest_switch_msr *intel_guest_get_msrs(int *nr)
 {
 	struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events);
 	struct perf_guest_switch_msr *arr = cpuc->guest_switch_msrs;
+	u64 mask;
 
 	arr[0].msr = MSR_CORE_PERF_GLOBAL_CTRL;
 	arr[0].host = x86_pmu.intel_ctrl & ~cpuc->intel_ctrl_guest_mask;
 	arr[0].guest = x86_pmu.intel_ctrl & ~cpuc->intel_ctrl_host_mask;
+
+	arr[1].msr = MSR_IA32_PEBS_ENABLE;
+	arr[1].host = cpuc->pebs_enabled;
 	/*
+	 * For PEBS virtualization only allow guest owned counters.
+	 *
 	 * If PMU counter has PEBS enabled it is not enough to disable counter
 	 * on a guest entry since PEBS memory write can overshoot guest entry
 	 * and corrupt guest memory. Disabling PEBS solves the problem.
 	 */
-	arr[1].msr = MSR_IA32_PEBS_ENABLE;
-	arr[1].host = cpuc->pebs_enabled;
-	arr[1].guest = 0;
+	mask = cpuc->intel_ctrl_guest_owned;
+	arr[1].guest = cpuc->pebs_enabled & (mask | (mask << 32));
 
 	*nr = 2;
 	return arr;
arch/x86/kvm/pmu.c +1 -0
index 5c4f631..4c6f417 100644
@@ -188,6 +188,7 @@ static void reprogram_counter(struct kvm_pmc *pmc, u32 type,
 				PTR_ERR(event));
 		return;
 	}
+	event->guest_owned = true;
 
 	pmc->perf_event = event;
 	clear_bit(pmc->idx, (unsigned long*)&pmc->vcpu->arch.pmu.reprogram_pmi);
include/linux/perf_event.h +14 -1
index 3356abc..ad2b3f6 100644
@@ -437,6 +437,8 @@ struct perf_event {
 	int				cgrp_defer_enabled;
 #endif
 
+	bool				guest_owned;	/* Owned by a guest */
+
 #endif /* CONFIG_PERF_EVENTS */
 };
 

@@ -550,11 +552,22 @@ extern int perf_event_refresh(struct perf_event *event, int refresh);
 extern void perf_event_update_userpage(struct perf_event *event);
 extern int perf_event_release_kernel(struct perf_event *event);
 extern struct perf_event *
+__perf_event_create_kernel_counter(struct perf_event_attr *attr,
+				int cpu,
+				struct task_struct *task,
+				perf_overflow_handler_t callback,
+				void *context, bool guest_owned);
+static inline struct perf_event *
 perf_event_create_kernel_counter(struct perf_event_attr *attr,
 				int cpu,
 				struct task_struct *task,
 				perf_overflow_handler_t callback,
-				void *context);
+				void *context)
+{
+	return __perf_event_create_kernel_counter(attr, cpu, task, callback,
+						  context, false);
+}
+
 extern void perf_pmu_migrate_context(struct pmu *pmu,
 				int src_cpu, int dst_cpu);
 extern u64 perf_event_read_value(struct perf_event *event,
kernel/events/core.c +4 -3
index f83a71a..3450ba7 100644
@@ -7249,10 +7249,10 @@ err_fd:
  * @task: task to profile (NULL for percpu)
  */
 struct perf_event *
-perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu,
+__perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu,
 				 struct task_struct *task,
 				 perf_overflow_handler_t overflow_handler,
-				 void *context)
+				 void *context, bool guest_owned)
 {
 	struct perf_event_context *ctx;
 	struct perf_event *event;

@@ -7268,6 +7268,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr, int cpu,
 		err = PTR_ERR(event);
 		goto err;
 	}
+	event->guest_owned = guest_owned;
 
 	account_event(event);
 

@@ -7290,7 +7291,7 @@ err_free:
 err:
 	return ERR_PTR(err);
 }
-EXPORT_SYMBOL_GPL(perf_event_create_kernel_counter);
+EXPORT_SYMBOL_GPL(__perf_event_create_kernel_counter);
 
 void perf_pmu_migrate_context(struct pmu *pmu, int src_cpu, int dst_cpu)
 {
Signature · 1 line
1.9.0
maintainer
#3 lore raw

Re: [PATCH 2/4] perf: Allow guest PEBS for KVM owned counters

Quoted 9 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Currently perf unconditionally disables PEBS for guest.

Now that we have the infrastructure in place to handle it we can allow it for KVM owned guest events. For the perf needs to know that a event is owned by a guest. Add a new state bit in the perf_event for that.

This doesn't make sense; why does it need to be owned?

#4 lore raw

Re: [PATCH 2/4] perf: Allow guest PEBS for KVM owned counters

Quoted 5 lines · Peter Zijlstra
Quoted 4 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Currently perf unconditionally disables PEBS for guest.

Quoted 7 lines
Quoted 5 lines

Now that we have the infrastructure in place to handle it we can allow it for KVM owned guest events. For the perf needs to know that a event is owned by a guest. Add a new state bit in the perf_event for that.

This doesn't make sense; why does it need to be owned?

Please read the complete patch kit

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
maintainer
#5 lore raw

Re: [PATCH 2/4] perf: Allow guest PEBS for KVM owned counters

Quoted 12 lines · Andi Kleen
Quoted 11 lines · Peter Zijlstra
Quoted 9 lines · Andi Kleen

From: Andi Kleen <[email protected]>

Currently perf unconditionally disables PEBS for guest.

Now that we have the infrastructure in place to handle it we can allow it for KVM owned guest events. For the perf needs to know that a event is owned by a guest. Add a new state bit in the perf_event for that.

Quoted 3 lines
Quoted 1 line

This doesn't make sense; why does it need to be owned?

Please read the complete patch kit

Please write coherent and self sustaining changelogs.

author
#6 lore raw

PATCH 3/4 perf: Handle guest PEBS events with a fake event

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

With PEBS virtualization the PEBS record gets delivered to the guest, but the host sees the PMI. This would normally result in a spurious PEBS PMI that is ignored. But we need to inject the PMI into the guest, so that the guest PMI handler can handle the PEBS record.

Check for this case in the perf PEBS handler. When any guest PEBS counters are active always check the counters explicitely for overflow. If a guest PEBs counter overflowed trigger a fake event. The fake event results in calling the KVM PMI callback, which injects the PMI into the guest. The guest handler then retrieves the correct information from its own PEBS record and the guest state.

Note: in very rare cases with exotic events this may lead to spurious PMIs in the guest.

Signed-off-by Andi Kleen <[email protected]>
---
 arch/x86/kernel/cpu/perf_event_intel_ds.c | 49 +++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_ds.c +49 -0
index 29622a7..0267174 100644
@@ -998,6 +998,53 @@ static void intel_pmu_drain_pebs_core(struct pt_regs *iregs)
 	__intel_pmu_pebs_event(event, iregs, at);
 }
 
+/*
+ * We may be running with virtualized PEBS, so the PEBS record
+ * was logged into the guest's DS and is invisible to us.
+ *
+ * For guest-owned counters we always have to check the counter
+ * and see if they are overflowed, because PEBS thresholds
+ * are not reported in the GLOBAL_STATUS.
+ *
+ * In this case just trigger a fake event for KVM to forward
+ * to the guest as PMI.  The guest will then see the real PEBS
+ * record and read the counter values.
+ *
+ * The contents of the event do not matter.
+ */
+static void intel_pmu_handle_guest_pebs(struct cpu_hw_events *cpuc,
+					struct pt_regs *iregs)
+{
+	int bit;
+	struct perf_event *event;
+
+	if (!cpuc->intel_ctrl_guest_owned)
+		return;
+
+	for_each_set_bit(bit, (unsigned long *)&cpuc->intel_ctrl_guest_owned,
+			 x86_pmu.max_pebs_events) {
+		struct perf_sample_data data;
+		s64 count;
+		int shift;
+
+		event = cpuc->events[bit];
+		if (!event->attr.precise_ip)
+			continue;
+		rdpmcl(event->hw.event_base_rdpmc, count);
+
+		/* sign extend */
+		shift = 64 - x86_pmu.cntval_bits;
+		count = ((s64)((u64)count << shift)) >> shift;
+
+		if (count < 0)
+			continue;
+
+		perf_sample_data_init(&data, 0, event->hw.last_period);
+		if (perf_event_overflow(event, &data, iregs))
+			x86_pmu_stop(event, 0);
+	}
+}
+
 static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs)
 {
 	struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events);

@@ -1010,6 +1057,8 @@ static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs)
 	if (!x86_pmu.pebs_active)
 		return;
 
+	intel_pmu_handle_guest_pebs(cpuc, iregs);
+
 	at  = (struct pebs_record_nhm *)(unsigned long)ds->pebs_buffer_base;
 	top = (struct pebs_record_nhm *)(unsigned long)ds->pebs_index;
Signature · 1 line
1.9.0
maintainer
#7 lore raw

Re: [PATCH 3/4] perf: Handle guest PEBS events with a fake event

Quoted 2 lines · Andi Kleen

Note: in very rare cases with exotic events this may lead to spurious PMIs in the guest.

Qualify that statement so that if someone runs into it we at least know it is known/expected.

#8 lore raw

Re: [PATCH 3/4] perf: Handle guest PEBS events with a fake event

Quoted 7 lines · Peter Zijlstra
Quoted 2 lines · Andi Kleen

Note: in very rare cases with exotic events this may lead to spurious PMIs in the guest.

Qualify that statement so that if someone runs into it we at least know it is known/expected.

You cannot actually observe it, so it's not a real problem. I'll drop the Note.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
author
#9 lore raw

PATCH 4/4 kvm: Implement PEBS virtualization

In reply to Andi Kleen

From: Andi Kleen <[email protected]>

PEBS (Precise Event Bases Sampling) profiling is very powerful, allowing improved sampling precision and much additional information, like address or TSX abort profiling. cycles:p and :pp uses PEBS.

This patch enables PEBS profiling in KVM guests.

PEBS writes profiling records to a virtual address in memory. Since the guest controls the virtual address space the PEBS record is directly delivered to the guest buffer. We set up the PEBS state that is works correctly.The CPU cannot handle any kinds of faults during these guest writes.

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

Typically profilers only set up a single page, so pinning that is not a big problem. The pinning is limited to 17 pages currently (64K+1)

In theory the guest can change its own page tables after the PEBS setup. The host has no way to track that with EPT. But if a guest would do that it could only crash itself. It's not expected that normal profilers do that.

The patch also adds the basic glue to enable the PEBS CPUIDs and other PEBS MSRs, and ask perf to enable PEBS as needed.

Due to various limitations it currently only works on Silvermont based systems.

This patch doesn't implement the extended MSRs some CPUs support. For example latency profiling on SLM will not work at this point.

Timing:

The emulation is somewhat more expensive than a real PMU. This may trigger the expensive PMI detection in the guest. Usually this can be disabled with
echo 0 > /proc/sys/kernel/perf_cpu_time_max_percent

Migration:

In theory it should should be possible (as long as we migrate to a host with the same PEBS event and the same PEBS format), but I'm not sure the basic KVM PMU code supports it correctly: no code to save/restore state, unless I'm missing something. Once the PMU code grows proper migration support it should be straight forward to handle the PEBS state too.

Signed-off-by Andi Kleen <[email protected]>
---
 arch/x86/include/asm/kvm_host.h       |   6 ++
 arch/x86/include/uapi/asm/msr-index.h |   4 +
 arch/x86/kvm/cpuid.c                  |  10 +-
 arch/x86/kvm/pmu.c                    | 184 ++++++++++++++++++++++++++++++++--
 arch/x86/kvm/vmx.c                    |   6 ++
 5 files changed, 196 insertions(+), 14 deletions(-)
arch/x86/include/asm/kvm_host.h +6 -0
index 7de069af..d87cb66 100644
@@ -319,6 +319,8 @@ struct kvm_pmc {
 	struct kvm_vcpu *vcpu;
 };
 
+#define MAX_PINNED_PAGES 17 /* 64k buffer + ds */
+
 struct kvm_pmu {
 	unsigned nr_arch_gp_counters;
 	unsigned nr_arch_fixed_counters;

@@ -335,6 +337,10 @@ struct kvm_pmu {
 	struct kvm_pmc fixed_counters[INTEL_PMC_MAX_FIXED];
 	struct irq_work irq_work;
 	u64 reprogram_pmi;
+	u64 pebs_enable;
+	u64 ds_area;
+	struct page *pinned_pages[MAX_PINNED_PAGES];
+	unsigned num_pinned_pages;
 };
 
 enum {
arch/x86/include/uapi/asm/msr-index.h +4 -0
index fcf2b3a..409a582 100644
@@ -72,6 +72,10 @@
 #define MSR_IA32_PEBS_ENABLE		0x000003f1
 #define MSR_IA32_DS_AREA		0x00000600
 #define MSR_IA32_PERF_CAPABILITIES	0x00000345
+#define PERF_CAP_PEBS_TRAP		(1U << 6)
+#define PERF_CAP_ARCH_REG		(1U << 7)
+#define PERF_CAP_PEBS_FORMAT		(0xf << 8)
+
 #define MSR_PEBS_LD_LAT_THRESHOLD	0x000003f6
 
 #define MSR_MTRRfix64K_00000		0x00000250
arch/x86/kvm/cpuid.c +7 -3
index f47a104..c8cc76b 100644
@@ -260,6 +260,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 	unsigned f_rdtscp = kvm_x86_ops->rdtscp_supported() ? F(RDTSCP) : 0;
 	unsigned f_invpcid = kvm_x86_ops->invpcid_supported() ? F(INVPCID) : 0;
 	unsigned f_mpx = kvm_x86_ops->mpx_supported() ? F(MPX) : 0;
+	bool pebs = perf_pebs_virtualization();
+	unsigned f_ds = pebs ? F(DS) : 0;
+	unsigned f_pdcm = pebs ? F(PDCM) : 0;
+	unsigned f_dtes64 = pebs ? F(DTES64) : 0;
 
 	/* cpuid 1.edx */
 	const u32 kvm_supported_word0_x86_features =

@@ -268,7 +272,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 		F(CX8) | F(APIC) | 0 /* Reserved */ | F(SEP) |
 		F(MTRR) | F(PGE) | F(MCA) | F(CMOV) |
 		F(PAT) | F(PSE36) | 0 /* PSN */ | F(CLFLUSH) |
-		0 /* Reserved, DS, ACPI */ | F(MMX) |
+		f_ds /* Reserved, ACPI */ | F(MMX) |
 		F(FXSR) | F(XMM) | F(XMM2) | F(SELFSNOOP) |
 		0 /* HTT, TM, Reserved, PBE */;
 	/* cpuid 0x80000001.edx */

@@ -283,10 +287,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 		0 /* Reserved */ | f_lm | F(3DNOWEXT) | F(3DNOW);
 	/* cpuid 1.ecx */
 	const u32 kvm_supported_word4_x86_features =
-		F(XMM3) | F(PCLMULQDQ) | 0 /* DTES64, MONITOR */ |
+		F(XMM3) | F(PCLMULQDQ) | f_dtes64 /* MONITOR */ |
 		0 /* DS-CPL, VMX, SMX, EST */ |
 		0 /* TM2 */ | F(SSSE3) | 0 /* CNXT-ID */ | 0 /* Reserved */ |
-		F(FMA) | F(CX16) | 0 /* xTPR Update, PDCM */ |
+		F(FMA) | F(CX16) | f_pdcm /* xTPR Update */ |
 		F(PCID) | 0 /* Reserved, DCA */ | F(XMM4_1) |
 		F(XMM4_2) | F(X2APIC) | F(MOVBE) | F(POPCNT) |
 		0 /* Reserved*/ | F(AES) | F(XSAVE) | 0 /* OSXSAVE */ | F(AVX) |
arch/x86/kvm/pmu.c +173 -11

294 lines not on this page · raw message

arch/x86/kvm/vmx.c +6 -0
index 33e8c02..4f39917 100644
@@ -7288,6 +7288,12 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	atomic_switch_perf_msrs(vmx);
 	debugctlmsr = get_debugctlmsr();
 
+	/* Move this somewhere else? */
+	if (vcpu->arch.pmu.ds_area)
+		add_atomic_switch_msr(vmx, MSR_IA32_DS_AREA,
+				      vcpu->arch.pmu.ds_area,
+				      perf_get_ds_area());
+
 	vmx->__launched = vmx->loaded_vmcs->launched;
 	asm(
 		/* Store host registers */
Signature · 1 line
1.9.0
#10 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 7 lines · Andi Kleen

From: Andi Kleen <[email protected]>

PEBS (Precise Event Bases Sampling) profiling is very powerful, allowing improved sampling precision and much additional information, like address or TSX abort profiling. cycles:p and :pp uses PEBS.

This patch enables PEBS profiling in KVM guests.

That sounds really cool!

Quoted 10 lines

PEBS writes profiling records to a virtual address in memory. Since the guest controls the virtual address space the PEBS record is directly delivered to the guest buffer. We set up the PEBS state that is works correctly.The CPU cannot handle any kinds of faults during these guest writes.

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

It will avoid guest page to be swapped, but shadow paging code may still drop shadow PT pages that build a mapping from DS virtual address to the guest page. With EPT it is less likely to happen (but still possible IIRC depending on memory pressure and how much memory shadow paging code is allowed to use), without EPT it will happen for sure.

Quoted 8 lines

Typically profilers only set up a single page, so pinning that is not a big problem. The pinning is limited to 17 pages currently (64K+1)

In theory the guest can change its own page tables after the PEBS setup. The host has no way to track that with EPT. But if a guest would do that it could only crash itself. It's not expected that normal profilers do that.

Spec says:

The following restrictions should be applied to the DS save area.
• The three DS save area sections should be allocated from a non-paged pool, and marked accessed and dirty. It is the responsibility of the operating system to keep the pages that contain the buffer present and to mark them accessed and dirty. The implication is that the operating system cannot do “lazy” page-table entry propagation for these pages.

There is nothing, as far as I can see, that says what will happen if the condition is not met. I always interpreted it as undefined behaviour so anything can happen including CPU dies completely. You are saying above on one hand that CPU cannot handle any kinds of faults during write to DS area, but on the other hand a guest could only crash itself. Is this architecturally guarantied?

Quoted 445 lines

The patch also adds the basic glue to enable the PEBS CPUIDs and other PEBS MSRs, and ask perf to enable PEBS as needed.

Due to various limitations it currently only works on Silvermont based systems.

This patch doesn't implement the extended MSRs some CPUs support. For example latency profiling on SLM will not work at this point.

Timing:

The emulation is somewhat more expensive than a real PMU. This may trigger the expensive PMI detection in the guest. Usually this can be disabled with
echo 0 > /proc/sys/kernel/perf_cpu_time_max_percent

Migration:

In theory it should should be possible (as long as we migrate to a host with the same PEBS event and the same PEBS format), but I'm not sure the basic KVM PMU code supports it correctly: no code to save/restore state, unless I'm missing something. Once the PMU code grows proper migration support it should be straight forward to handle the PEBS state too.

Signed-off-by Andi Kleen <[email protected]>
---
 arch/x86/include/asm/kvm_host.h       |   6 ++
 arch/x86/include/uapi/asm/msr-index.h |   4 +
 arch/x86/kvm/cpuid.c                  |  10 +-
 arch/x86/kvm/pmu.c                    | 184 ++++++++++++++++++++++++++++++++--
 arch/x86/kvm/vmx.c                    |   6 ++
 5 files changed, 196 insertions(+), 14 deletions(-)
arch/x86/include/asm/kvm_host.h +6 -0
index 7de069af..d87cb66 100644
@@ -319,6 +319,8 @@ struct kvm_pmc {
 	struct kvm_vcpu *vcpu;
 };
 
+#define MAX_PINNED_PAGES 17 /* 64k buffer + ds */
+
 struct kvm_pmu {
 	unsigned nr_arch_gp_counters;
 	unsigned nr_arch_fixed_counters;

@@ -335,6 +337,10 @@ struct kvm_pmu {
 	struct kvm_pmc fixed_counters[INTEL_PMC_MAX_FIXED];
 	struct irq_work irq_work;
 	u64 reprogram_pmi;
+	u64 pebs_enable;
+	u64 ds_area;
+	struct page *pinned_pages[MAX_PINNED_PAGES];
+	unsigned num_pinned_pages;
 };
 
 enum {
arch/x86/include/uapi/asm/msr-index.h +4 -0
index fcf2b3a..409a582 100644
@@ -72,6 +72,10 @@
 #define MSR_IA32_PEBS_ENABLE		0x000003f1
 #define MSR_IA32_DS_AREA		0x00000600
 #define MSR_IA32_PERF_CAPABILITIES	0x00000345
+#define PERF_CAP_PEBS_TRAP		(1U << 6)
+#define PERF_CAP_ARCH_REG		(1U << 7)
+#define PERF_CAP_PEBS_FORMAT		(0xf << 8)
+
 #define MSR_PEBS_LD_LAT_THRESHOLD	0x000003f6
 
 #define MSR_MTRRfix64K_00000		0x00000250
arch/x86/kvm/cpuid.c +7 -3
index f47a104..c8cc76b 100644
@@ -260,6 +260,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 	unsigned f_rdtscp = kvm_x86_ops->rdtscp_supported() ? F(RDTSCP) : 0;
 	unsigned f_invpcid = kvm_x86_ops->invpcid_supported() ? F(INVPCID) : 0;
 	unsigned f_mpx = kvm_x86_ops->mpx_supported() ? F(MPX) : 0;
+	bool pebs = perf_pebs_virtualization();
+	unsigned f_ds = pebs ? F(DS) : 0;
+	unsigned f_pdcm = pebs ? F(PDCM) : 0;
+	unsigned f_dtes64 = pebs ? F(DTES64) : 0;
 
 	/* cpuid 1.edx */
 	const u32 kvm_supported_word0_x86_features =

@@ -268,7 +272,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 		F(CX8) | F(APIC) | 0 /* Reserved */ | F(SEP) |
 		F(MTRR) | F(PGE) | F(MCA) | F(CMOV) |
 		F(PAT) | F(PSE36) | 0 /* PSN */ | F(CLFLUSH) |
-		0 /* Reserved, DS, ACPI */ | F(MMX) |
+		f_ds /* Reserved, ACPI */ | F(MMX) |
 		F(FXSR) | F(XMM) | F(XMM2) | F(SELFSNOOP) |
 		0 /* HTT, TM, Reserved, PBE */;
 	/* cpuid 0x80000001.edx */

@@ -283,10 +287,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
 		0 /* Reserved */ | f_lm | F(3DNOWEXT) | F(3DNOW);
 	/* cpuid 1.ecx */
 	const u32 kvm_supported_word4_x86_features =
-		F(XMM3) | F(PCLMULQDQ) | 0 /* DTES64, MONITOR */ |
+		F(XMM3) | F(PCLMULQDQ) | f_dtes64 /* MONITOR */ |
 		0 /* DS-CPL, VMX, SMX, EST */ |
 		0 /* TM2 */ | F(SSSE3) | 0 /* CNXT-ID */ | 0 /* Reserved */ |
-		F(FMA) | F(CX16) | 0 /* xTPR Update, PDCM */ |
+		F(FMA) | F(CX16) | f_pdcm /* xTPR Update */ |
 		F(PCID) | 0 /* Reserved, DCA */ | F(XMM4_1) |
 		F(XMM4_2) | F(X2APIC) | F(MOVBE) | F(POPCNT) |
 		0 /* Reserved*/ | F(AES) | F(XSAVE) | 0 /* OSXSAVE */ | F(AVX) |
arch/x86/kvm/pmu.c +173 -11
index 4c6f417..6362db7 100644
@@ -15,9 +15,11 @@
 #include <linux/types.h>
 #include <linux/kvm_host.h>
 #include <linux/perf_event.h>
+#include <linux/highmem.h>
 #include "x86.h"
 #include "cpuid.h"
 #include "lapic.h"
+#include "mmu.h"
 
 static struct kvm_arch_event_perf_mapping {
 	u8 eventsel;

@@ -36,9 +38,23 @@ static struct kvm_arch_event_perf_mapping {
 	[7] = { 0x00, 0x30, PERF_COUNT_HW_REF_CPU_CYCLES },
 };
 
+struct debug_store {
+	u64	bts_buffer_base;
+	u64	bts_index;
+	u64	bts_absolute_maximum;
+	u64	bts_interrupt_threshold;
+	u64	pebs_buffer_base;
+	u64	pebs_index;
+	u64	pebs_absolute_maximum;
+	u64	pebs_interrupt_threshold;
+	u64	pebs_event_reset[4];
+};
+
 /* mapping between fixed pmc index and arch_events array */
 int fixed_pmc_events[] = {1, 0, 7};
 
+static u64 host_perf_cap __read_mostly;
+
 static bool pmc_is_gp(struct kvm_pmc *pmc)
 {
 	return pmc->type == KVM_PMC_GP;

@@ -108,7 +124,10 @@ static void kvm_perf_overflow(struct perf_event *perf_event,
 {
 	struct kvm_pmc *pmc = perf_event->overflow_handler_context;
 	struct kvm_pmu *pmu = &pmc->vcpu->arch.pmu;
-	__set_bit(pmc->idx, (unsigned long *)&pmu->global_status);
+	if (perf_event->attr.precise_ip)
+		__set_bit(62, (unsigned long *)&pmu->global_status);
+	else
+		__set_bit(pmc->idx, (unsigned long *)&pmu->global_status);
 }
 
 static void kvm_perf_overflow_intr(struct perf_event *perf_event,

@@ -160,7 +179,7 @@ static void stop_counter(struct kvm_pmc *pmc)
 
 static void reprogram_counter(struct kvm_pmc *pmc, u32 type,
 		unsigned config, bool exclude_user, bool exclude_kernel,
-		bool intr, bool in_tx, bool in_tx_cp)
+		bool intr, bool in_tx, bool in_tx_cp, bool pebs)
 {
 	struct perf_event *event;
 	struct perf_event_attr attr = {

@@ -177,18 +196,20 @@ static void reprogram_counter(struct kvm_pmc *pmc, u32 type,
 		attr.config |= HSW_IN_TX;
 	if (in_tx_cp)
 		attr.config |= HSW_IN_TX_CHECKPOINTED;
+	if (pebs)
+		attr.precise_ip = 1;
 
 	attr.sample_period = (-pmc->counter) & pmc_bitmask(pmc);
 
-	event = perf_event_create_kernel_counter(&attr, -1, current,
-						 intr ? kvm_perf_overflow_intr :
-						 kvm_perf_overflow, pmc);
+	event = __perf_event_create_kernel_counter(&attr, -1, current,
+						 (intr || pebs) ?
+						 kvm_perf_overflow_intr :
+						 kvm_perf_overflow, pmc, true);
 	if (IS_ERR(event)) {
 		printk_once("kvm: pmu event creation failed %ld\n",
 				PTR_ERR(event));
 		return;
 	}
-	event->guest_owned = true;
 
 	pmc->perf_event = event;
 	clear_bit(pmc->idx, (unsigned long*)&pmc->vcpu->arch.pmu.reprogram_pmi);

@@ -211,7 +232,8 @@ static unsigned find_arch_event(struct kvm_pmu *pmu, u8 event_select,
 	return arch_events[i].event_type;
 }
 
-static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)
+static void reprogram_gp_counter(struct kvm_pmu *pmu, struct kvm_pmc *pmc,
+				 u64 eventsel)
 {
 	unsigned config, type = PERF_TYPE_RAW;
 	u8 event_select, unit_mask;

@@ -248,7 +270,8 @@ static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)
 			!(eventsel & ARCH_PERFMON_EVENTSEL_OS),
 			eventsel & ARCH_PERFMON_EVENTSEL_INT,
 			(eventsel & HSW_IN_TX),
-			(eventsel & HSW_IN_TX_CHECKPOINTED));
+			(eventsel & HSW_IN_TX_CHECKPOINTED),
+			test_bit(pmc->idx, (unsigned long *)&pmu->pebs_enable));
 }
 
 static void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 en_pmi, int idx)

@@ -265,7 +288,7 @@ static void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 en_pmi, int idx)
 			arch_events[fixed_pmc_events[idx]].event_type,
 			!(en & 0x2), /* exclude user */
 			!(en & 0x1), /* exclude kernel */
-			pmi, false, false);
+			pmi, false, false, false);
 }
 
 static inline u8 fixed_en_pmi(u64 ctrl, int idx)

@@ -298,7 +321,7 @@ static void reprogram_idx(struct kvm_pmu *pmu, int idx)
 		return;
 
 	if (pmc_is_gp(pmc))
-		reprogram_gp_counter(pmc, pmc->eventsel);
+		reprogram_gp_counter(pmu, pmc, pmc->eventsel);
 	else {
 		int fidx = idx - INTEL_PMC_IDX_FIXED;
 		reprogram_fixed_counter(pmc,

@@ -323,6 +346,12 @@ bool kvm_pmu_msr(struct kvm_vcpu *vcpu, u32 msr)
 	int ret;
 
 	switch (msr) {
+	case MSR_IA32_DS_AREA:
+	case MSR_IA32_PEBS_ENABLE:
+	case MSR_IA32_PERF_CAPABILITIES:
+		ret = perf_pebs_virtualization() ? 1 : 0;
+		break;
+
 	case MSR_CORE_PERF_FIXED_CTR_CTRL:
 	case MSR_CORE_PERF_GLOBAL_STATUS:
 	case MSR_CORE_PERF_GLOBAL_CTRL:

@@ -356,6 +385,18 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data)
 	case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
 		*data = pmu->global_ovf_ctrl;
 		return 0;
+	case MSR_IA32_DS_AREA:
+		*data = pmu->ds_area;
+		return 0;
+	case MSR_IA32_PEBS_ENABLE:
+		*data = pmu->pebs_enable;
+		return 0;
+	case MSR_IA32_PERF_CAPABILITIES:
+		/* Report host PEBS format to guest */
+		*data = host_perf_cap &
+			(PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG |
+			 PERF_CAP_PEBS_FORMAT);
+		return 0;
 	default:
 		if ((pmc = get_gp_pmc(pmu, index, MSR_IA32_PERFCTR0)) ||
 				(pmc = get_fixed_pmc(pmu, index))) {

@@ -369,6 +410,109 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data)
 	return 1;
 }
 
+static void kvm_pmu_release_pin(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = &vcpu->arch.pmu;
+	int i;
+
+	for (i = 0; i < pmu->num_pinned_pages; i++)
+		put_page(pmu->pinned_pages[i]);
+	pmu->num_pinned_pages = 0;
+}
+
+static struct page *get_guest_page(struct kvm_vcpu *vcpu,
+				   unsigned long addr)
+{
+	unsigned long pfn;
+	struct x86_exception exception;
+	gpa_t gpa = vcpu->arch.walk_mmu->gva_to_gpa(vcpu, addr,
+						    PFERR_WRITE_MASK,
+						    &exception);
+
+	if (gpa == UNMAPPED_GVA) {
+		printk_once("Cannot translate guest page %lx\n", addr);
+		return NULL;
+	}
+	pfn = gfn_to_pfn(vcpu->kvm, gpa_to_gfn(gpa));
+	if (is_error_noslot_pfn(pfn)) {
+		printk_once("gfn_to_pfn failed for %llx\n", gpa);
+		return NULL;
+	}
+	return pfn_to_page(pfn);
+}
+
+static int pin_and_copy(struct kvm_vcpu *vcpu,
+			unsigned long addr, void *dst, int len,
+			struct page **p)
+{
+	unsigned long offset = addr & ~PAGE_MASK;
+	void *map;
+
+	*p = get_guest_page(vcpu, addr);
+	if (!*p)
+		return -EIO;
+	map = kmap(*p);
+	memcpy(dst, map + offset, len);
+	kunmap(map);
+	return 0;
+}
+
+/*
+ * Pin the DS area and the PEBS buffer while PEBS is active,
+ * because the CPU cannot tolerate EPT faults for PEBS updates.
+ *
+ * We assume that any guest who changes the DS buffer disables
+ * PEBS first and does not change the page tables during operation.
+ *
+ * When the guest violates these assumptions it may crash itself.
+ * This is expected to not happen with standard profilers.
+ *
+ * No need to clean up anything, as the caller will always eventually
+ * unpin pages.
+ */
+
+static void kvm_pmu_pebs_pin(struct kvm_vcpu *vcpu)
+{
+	struct kvm_pmu *pmu = &vcpu->arch.pmu;
+	struct debug_store ds;
+	int pg;
+	unsigned len;
+	unsigned long offset;
+	unsigned long addr;
+
+	offset = pmu->ds_area & ~PAGE_MASK;
+	len = sizeof(struct debug_store);
+	len = min_t(unsigned, PAGE_SIZE - offset, len);
+	if (pin_and_copy(vcpu, pmu->ds_area, &ds, len,
+			 &pmu->pinned_pages[0]) < 0) {
+		printk_once("Cannot pin ds area %llx\n", pmu->ds_area);
+		return;
+	}
+	pmu->num_pinned_pages++;
+	if (len < sizeof(struct debug_store)) {
+		if (pin_and_copy(vcpu, pmu->ds_area + len, (void *)&ds + len,
+				  sizeof(struct debug_store) - len,
+				  &pmu->pinned_pages[1]) < 0)
+			return;
+		pmu->num_pinned_pages++;
+	}
+
+	pg = pmu->num_pinned_pages;
+	for (addr = ds.pebs_buffer_base;
+	     addr < ds.pebs_absolute_maximum && pg < MAX_PINNED_PAGES;
+	     addr += PAGE_SIZE, pg++) {
+		pmu->pinned_pages[pg] = get_guest_page(vcpu, addr);
+		if (!pmu->pinned_pages[pg]) {
+			printk_once("Cannot pin PEBS buffer %lx (%llx-%llx)\n",
+				 addr,
+				 ds.pebs_buffer_base,
+				 ds.pebs_absolute_maximum);
+			break;
+		}
+	}
+	pmu->num_pinned_pages = pg;
+}
+
 int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
 	struct kvm_pmu *pmu = &vcpu->arch.pmu;

@@ -407,6 +551,20 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 0;
 		}
 		break;
+	case MSR_IA32_DS_AREA:
+		pmu->ds_area = data;
+		return 0;
+	case MSR_IA32_PEBS_ENABLE:
+		if (data & ~0xf0000000fULL)
+			break;
+		if (data && data != pmu->pebs_enable) {
+			kvm_pmu_release_pin(vcpu);
+			kvm_pmu_pebs_pin(vcpu);
+		} else if (data == 0 && pmu->pebs_enable) {
+			kvm_pmu_release_pin(vcpu);
+		}
+		pmu->pebs_enable = data;
+		return 0;
 	default:
 		if ((pmc = get_gp_pmc(pmu, index, MSR_IA32_PERFCTR0)) ||
 				(pmc = get_fixed_pmc(pmu, index))) {

@@ -418,7 +576,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			if (data == pmc->eventsel)
 				return 0;
 			if (!(data & pmu->reserved_bits)) {
-				reprogram_gp_counter(pmc, data);
+				reprogram_gp_counter(pmu, pmc, data);
 				return 0;
 			}
 		}

@@ -514,6 +672,9 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
 	}
 	init_irq_work(&pmu->irq_work, trigger_pmi);
 	kvm_pmu_cpuid_update(vcpu);
+
+	if (boot_cpu_has(X86_FEATURE_PDCM))
+		rdmsrl_safe(MSR_IA32_PERF_CAPABILITIES, &host_perf_cap);
 }
 
 void kvm_pmu_reset(struct kvm_vcpu *vcpu)

@@ -538,6 +699,7 @@ void kvm_pmu_reset(struct kvm_vcpu *vcpu)
 void kvm_pmu_destroy(struct kvm_vcpu *vcpu)
 {
 	kvm_pmu_reset(vcpu);
+	kvm_pmu_release_pin(vcpu);
 }
 
 void kvm_handle_pmu_event(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c +6 -0
index 33e8c02..4f39917 100644
@@ -7288,6 +7288,12 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	atomic_switch_perf_msrs(vmx);
 	debugctlmsr = get_debugctlmsr();
 
+	/* Move this somewhere else? */
+	if (vcpu->arch.pmu.ds_area)
+		add_atomic_switch_msr(vmx, MSR_IA32_DS_AREA,
+				      vcpu->arch.pmu.ds_area,
+				      perf_get_ds_area());
+
 	vmx->__launched = vmx->loaded_vmcs->launched;
 	asm(
 		/* Store host registers */
Signature · 1 line
1.9.0

--
Gleb.

#11 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 5 lines
Quoted 3 lines

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

It will avoid guest page to be swapped, but shadow paging code may still drop shadow PT pages that build a mapping from DS virtual address to the guest page.

You're saying the EPT code could tear down the EPT mappings?

OK that would need to be prevented too. Any suggestions how?

Quoted 3 lines

With EPT it is less likely to happen (but still possible IIRC depending on memory pressure and how much memory shadow paging code is allowed to use), without EPT it will happen for sure.

Don't care about the non EPT case, this is white listed only for EPT supporting CPUs.

Quoted 6 lines

There is nothing, as far as I can see, that says what will happen if the condition is not met. I always interpreted it as undefined behaviour so anything can happen including CPU dies completely. You are saying above on one hand that CPU cannot handle any kinds of faults during write to DS area, but on the other hand a guest could only crash itself. Is this architecturally guarantied?

You essentially would get random page faults, and the PEBS event will be cancelled. No hangs.

It's not architecturally guaranteed, but we white list anyways so we only care about the white listed CPUs at this point. For them I have confirmation that it works.

-Andi

#12 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 7 lines · Andi Kleen
Quoted 5 lines
Quoted 3 lines

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

It will avoid guest page to be swapped, but shadow paging code may still drop shadow PT pages that build a mapping from DS virtual address to the guest page.

You're saying the EPT code could tear down the EPT mappings?

Under memory pressure yes. mmu_shrink_scan() calls prepare_zap_oldest_mmu_page() which destroys oldest mmu pages like its name says. As far as I can tell running nested guest can also result in EPT mapping to be dropped since it will create a lot of shadow pages and this will cause make_mmu_pages_available() to destroy some shadow pages and it may choose EPT pages to destroy.

CCing Marcelo to confirm/correct.

Quoted 2 lines

OK that would need to be prevented too. Any suggestions how?

Only high level. Mark shadow pages involved in translation we want to keep and skip them in prepare_zap_oldest_mmu_page().

Quoted 7 lines
Quoted 4 lines

With EPT it is less likely to happen (but still possible IIRC depending on memory pressure and how much memory shadow paging code is allowed to use), without EPT it will happen for sure.

Don't care about the non EPT case, this is white listed only for EPT supporting CPUs.

User may still disable EPT during module load, so pebs should be dropped from a guest's cpuid in this case.

Quoted 10 lines
Quoted 7 lines

There is nothing, as far as I can see, that says what will happen if the condition is not met. I always interpreted it as undefined behaviour so anything can happen including CPU dies completely. You are saying above on one hand that CPU cannot handle any kinds of faults during write to DS area, but on the other hand a guest could only crash itself. Is this architecturally guarantied?

You essentially would get random page faults, and the PEBS event will be cancelled. No hangs.

Is this a guest who will get those random page faults or a host?

Quoted 6 lines

It's not architecturally guaranteed, but we white list anyways so we only care about the white listed CPUs at this point. For them I have confirmation that it works.

-Andi

--
Gleb.

author
#13 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

BTW I found some more problems in the v1 version.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

With EPT it is less likely to happen (but still possible IIRC depending on memory pressure and how much memory shadow paging code is allowed to use), without EPT it will happen for sure.

Don't care about the non EPT case, this is white listed only for EPT supporting CPUs.

User may still disable EPT during module load, so pebs should be dropped from a guest's cpuid in this case.

Ok.

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

There is nothing, as far as I can see, that says what will happen if the condition is not met. I always interpreted it as undefined behaviour so anything can happen including CPU dies completely. You are saying above on one hand that CPU cannot handle any kinds of faults during write to DS area, but on the other hand a guest could only crash itself. Is this architecturally guarantied?

Quoted 3 lines
Quoted 2 lines

You essentially would get random page faults, and the PEBS event will be cancelled. No hangs.

Is this a guest who will get those random page faults or a host?

The guest (on the white listed CPU models)

-Andi

#14 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

In reply to Gleb Natapov
Quoted 9 lines · Gleb Natapov
Quoted 7 lines · Andi Kleen
Quoted 5 lines
Quoted 3 lines

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

It will avoid guest page to be swapped, but shadow paging code may still drop shadow PT pages that build a mapping from DS virtual address to the guest page.

You're saying the EPT code could tear down the EPT mappings?

Quoted 8 lines

Under memory pressure yes. mmu_shrink_scan() calls prepare_zap_oldest_mmu_page() which destroys oldest mmu pages like its name says. As far as I can tell running nested guest can also result in EPT mapping to be dropped since it will create a lot of shadow pages and this will cause make_mmu_pages_available() to destroy some shadow pages and it may choose EPT pages to destroy.

CCing Marcelo to confirm/correct.

Yes. Given SLAB pressure any shadow pages can be deleted except pinned via root_count=1 ones.

Quoted 3 lines
Quoted 1 line

OK that would need to be prevented too. Any suggestions how?

Only high level. Mark shadow pages involved in translation we want to keep and skip them in prepare_zap_oldest_mmu_page().

Should special case such translations so that they are not zapped (either via page deletion or single entry EPT deletion). Them and any other their parents, bummer.

Maybe its cleaner to check that DS area is EPT mapped before VM-entry.

No way the processor can generate VM-exits ?

Is it not an option to fake a DS-save area in the host (and trap any accesses to the DS_AREA MSR from the guest) ?
Then before notifying the PEBS event, copy from that host area to guests address. Slow probably.

#15 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

In reply to Andi Kleen
Quoted 261 lines · Andi Kleen

From: Andi Kleen <[email protected]>

PEBS (Precise Event Bases Sampling) profiling is very powerful, allowing improved sampling precision and much additional information, like address or TSX abort profiling. cycles:p and :pp uses PEBS.

This patch enables PEBS profiling in KVM guests.

PEBS writes profiling records to a virtual address in memory. Since the guest controls the virtual address space the PEBS record is directly delivered to the guest buffer. We set up the PEBS state that is works correctly.The CPU cannot handle any kinds of faults during these guest writes.

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

Typically profilers only set up a single page, so pinning that is not a big problem. The pinning is limited to 17 pages currently (64K+1)

In theory the guest can change its own page tables after the PEBS setup. The host has no way to track that with EPT. But if a guest would do that it could only crash itself. It's not expected that normal profilers do that.

The patch also adds the basic glue to enable the PEBS CPUIDs and other PEBS MSRs, and ask perf to enable PEBS as needed.

Due to various limitations it currently only works on Silvermont based systems.

This patch doesn't implement the extended MSRs some CPUs support. For example latency profiling on SLM will not work at this point.

Timing:

The emulation is somewhat more expensive than a real PMU. This may trigger the expensive PMI detection in the guest. Usually this can be disabled with
echo 0 > /proc/sys/kernel/perf_cpu_time_max_percent

Migration:

In theory it should should be possible (as long as we migrate to a host with the same PEBS event and the same PEBS format), but I'm not sure the basic KVM PMU code supports it correctly: no code to save/restore state, unless I'm missing something. Once the PMU code grows proper migration support it should be straight forward to handle the PEBS state too.

Signed-off-by Andi Kleen <[email protected]>
---
 arch/x86/include/asm/kvm_host.h       |   6 ++
 arch/x86/include/uapi/asm/msr-index.h |   4 +
 arch/x86/kvm/cpuid.c                  |  10 +-
 arch/x86/kvm/pmu.c                    | 184 ++++++++++++++++++++++++++++++++--
 arch/x86/kvm/vmx.c                    |   6 ++
 5 files changed, 196 insertions(+), 14 deletions(-)
arch/x86/include/asm/kvm_host.h +6 -0
index 7de069af..d87cb66 100644
@@ -319,6 +319,8 @@ struct kvm_pmc {
        struct kvm_vcpu *vcpu;
 };
 
+#define MAX_PINNED_PAGES 17 /* 64k buffer + ds */
+
 struct kvm_pmu {
        unsigned nr_arch_gp_counters;
        unsigned nr_arch_fixed_counters;

@@ -335,6 +337,10 @@ struct kvm_pmu {
        struct kvm_pmc fixed_counters[INTEL_PMC_MAX_FIXED];
        struct irq_work irq_work;
        u64 reprogram_pmi;
+       u64 pebs_enable;
+       u64 ds_area;
+       struct page *pinned_pages[MAX_PINNED_PAGES];
+       unsigned num_pinned_pages;
 };
 
 enum {
arch/x86/include/uapi/asm/msr-index.h +4 -0
index fcf2b3a..409a582 100644
@@ -72,6 +72,10 @@
 #define MSR_IA32_PEBS_ENABLE           0x000003f1
 #define MSR_IA32_DS_AREA               0x00000600
 #define MSR_IA32_PERF_CAPABILITIES     0x00000345
+#define PERF_CAP_PEBS_TRAP             (1U << 6)
+#define PERF_CAP_ARCH_REG              (1U << 7)
+#define PERF_CAP_PEBS_FORMAT           (0xf << 8)
+
 #define MSR_PEBS_LD_LAT_THRESHOLD      0x000003f6
 
 #define MSR_MTRRfix64K_00000           0x00000250
arch/x86/kvm/cpuid.c +7 -3
index f47a104..c8cc76b 100644
@@ -260,6 +260,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
        unsigned f_rdtscp = kvm_x86_ops->rdtscp_supported() ? F(RDTSCP) : 0;
        unsigned f_invpcid = kvm_x86_ops->invpcid_supported() ? F(INVPCID) : 0;
        unsigned f_mpx = kvm_x86_ops->mpx_supported() ? F(MPX) : 0;
+       bool pebs = perf_pebs_virtualization();
+       unsigned f_ds = pebs ? F(DS) : 0;
+       unsigned f_pdcm = pebs ? F(PDCM) : 0;
+       unsigned f_dtes64 = pebs ? F(DTES64) : 0;
 
        /* cpuid 1.edx */
        const u32 kvm_supported_word0_x86_features =

@@ -268,7 +272,7 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
                F(CX8) | F(APIC) | 0 /* Reserved */ | F(SEP) |
                F(MTRR) | F(PGE) | F(MCA) | F(CMOV) |
                F(PAT) | F(PSE36) | 0 /* PSN */ | F(CLFLUSH) |
-               0 /* Reserved, DS, ACPI */ | F(MMX) |
+               f_ds /* Reserved, ACPI */ | F(MMX) |
                F(FXSR) | F(XMM) | F(XMM2) | F(SELFSNOOP) |
                0 /* HTT, TM, Reserved, PBE */;
        /* cpuid 0x80000001.edx */

@@ -283,10 +287,10 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,
                0 /* Reserved */ | f_lm | F(3DNOWEXT) | F(3DNOW);
        /* cpuid 1.ecx */
        const u32 kvm_supported_word4_x86_features =
-               F(XMM3) | F(PCLMULQDQ) | 0 /* DTES64, MONITOR */ |
+               F(XMM3) | F(PCLMULQDQ) | f_dtes64 /* MONITOR */ |
                0 /* DS-CPL, VMX, SMX, EST */ |
                0 /* TM2 */ | F(SSSE3) | 0 /* CNXT-ID */ | 0 /* Reserved */ |
-               F(FMA) | F(CX16) | 0 /* xTPR Update, PDCM */ |
+               F(FMA) | F(CX16) | f_pdcm /* xTPR Update */ |
                F(PCID) | 0 /* Reserved, DCA */ | F(XMM4_1) |
                F(XMM4_2) | F(X2APIC) | F(MOVBE) | F(POPCNT) |
                0 /* Reserved*/ | F(AES) | F(XSAVE) | 0 /* OSXSAVE */ | F(AVX) |
arch/x86/kvm/pmu.c +33 -10
index 4c6f417..6362db7 100644
@@ -15,9 +15,11 @@
 #include <linux/types.h>
 #include <linux/kvm_host.h>
 #include <linux/perf_event.h>
+#include <linux/highmem.h>
 #include "x86.h"
 #include "cpuid.h"
 #include "lapic.h"
+#include "mmu.h"
 
 static struct kvm_arch_event_perf_mapping {
        u8 eventsel;

@@ -36,9 +38,23 @@ static struct kvm_arch_event_perf_mapping {
        [7] = { 0x00, 0x30, PERF_COUNT_HW_REF_CPU_CYCLES },
 };
 
+struct debug_store {
+       u64     bts_buffer_base;
+       u64     bts_index;
+       u64     bts_absolute_maximum;
+       u64     bts_interrupt_threshold;
+       u64     pebs_buffer_base;
+       u64     pebs_index;
+       u64     pebs_absolute_maximum;
+       u64     pebs_interrupt_threshold;
+       u64     pebs_event_reset[4];
+};
+
 /* mapping between fixed pmc index and arch_events array */
 int fixed_pmc_events[] = {1, 0, 7};
 
+static u64 host_perf_cap __read_mostly;
+
 static bool pmc_is_gp(struct kvm_pmc *pmc)
 {
        return pmc->type == KVM_PMC_GP;

@@ -108,7 +124,10 @@ static void kvm_perf_overflow(struct perf_event *perf_event,
 {
        struct kvm_pmc *pmc = perf_event->overflow_handler_context;
        struct kvm_pmu *pmu = &pmc->vcpu->arch.pmu;
-       __set_bit(pmc->idx, (unsigned long *)&pmu->global_status);
+       if (perf_event->attr.precise_ip)
+               __set_bit(62, (unsigned long *)&pmu->global_status);
+       else
+               __set_bit(pmc->idx, (unsigned long *)&pmu->global_status);
 }
 
 static void kvm_perf_overflow_intr(struct perf_event *perf_event,

@@ -160,7 +179,7 @@ static void stop_counter(struct kvm_pmc *pmc)
 
 static void reprogram_counter(struct kvm_pmc *pmc, u32 type,
                unsigned config, bool exclude_user, bool exclude_kernel,
-               bool intr, bool in_tx, bool in_tx_cp)
+               bool intr, bool in_tx, bool in_tx_cp, bool pebs)
 {
        struct perf_event *event;
        struct perf_event_attr attr = {

@@ -177,18 +196,20 @@ static void reprogram_counter(struct kvm_pmc *pmc, u32 type,
                attr.config |= HSW_IN_TX;
        if (in_tx_cp)
                attr.config |= HSW_IN_TX_CHECKPOINTED;
+       if (pebs)
+               attr.precise_ip = 1;
 
        attr.sample_period = (-pmc->counter) & pmc_bitmask(pmc);
 
-       event = perf_event_create_kernel_counter(&attr, -1, current,
-                                                intr ? kvm_perf_overflow_intr :
-                                                kvm_perf_overflow, pmc);
+       event = __perf_event_create_kernel_counter(&attr, -1, current,
+                                                (intr || pebs) ?
+                                                kvm_perf_overflow_intr :
+                                                kvm_perf_overflow, pmc, true);
        if (IS_ERR(event)) {
                printk_once("kvm: pmu event creation failed %ld\n",
                                PTR_ERR(event));
                return;
        }
-       event->guest_owned = true;
 
        pmc->perf_event = event;
        clear_bit(pmc->idx, (unsigned long*)&pmc->vcpu->arch.pmu.reprogram_pmi);

@@ -211,7 +232,8 @@ static unsigned find_arch_event(struct kvm_pmu *pmu, u8 event_select,
        return arch_events[i].event_type;
 }
 
-static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)
+static void reprogram_gp_counter(struct kvm_pmu *pmu, struct kvm_pmc *pmc,
+                                u64 eventsel)
 {
        unsigned config, type = PERF_TYPE_RAW;
        u8 event_select, unit_mask;

@@ -248,7 +270,8 @@ static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)
                        !(eventsel & ARCH_PERFMON_EVENTSEL_OS),
                        eventsel & ARCH_PERFMON_EVENTSEL_INT,
                        (eventsel & HSW_IN_TX),
-                       (eventsel & HSW_IN_TX_CHECKPOINTED));
+                       (eventsel & HSW_IN_TX_CHECKPOINTED),
+                       test_bit(pmc->idx, (unsigned long *)&pmu->pebs_enable));
 }
 
 static void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 en_pmi, int idx)

@@ -265,7 +288,7 @@ static void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 en_pmi, int idx)
                        arch_events[fixed_pmc_events[idx]].event_type,
                        !(en & 0x2), /* exclude user */
                        !(en & 0x1), /* exclude kernel */
-                       pmi, false, false);
+                       pmi, false, false, false);
 }
 
 static inline u8 fixed_en_pmi(u64 ctrl, int idx)

@@ -298,7 +321,7 @@ static void reprogram_idx(struct kvm_pmu *pmu, int idx)
                return;
 
        if (pmc_is_gp(pmc))
-               reprogram_gp_counter(pmc, pmc->eventsel);
+               reprogram_gp_counter(pmu, pmc, pmc->eventsel);
        else {
                int fidx = idx - INTEL_PMC_IDX_FIXED;
                reprogram_fixed_counter(pmc,

arch/x86/kvm/pmu.c bool kvm_pmu_msr(struct kvm_vcpu *vcpu, u32 msr) · line 352

Quoted 7 lines
arch/x86/kvm/pmu.c +4 -0
@@ -323,6 +346,12 @@ bool kvm_pmu_msr(struct kvm_vcpu *vcpu, u32 msr)
        int ret;
 
        switch (msr) {
+       case MSR_IA32_DS_AREA:
+       case MSR_IA32_PEBS_ENABLE:
+       case MSR_IA32_PERF_CAPABILITIES:
+               ret = perf_pebs_virtualization() ? 1 : 0;

Should this also check the CPUID exposed to the guest? The KVM PMU module is careful to not expose PMU MSRs to the guest unless the right CPUID bits have been exposed.

It seems to me that with this patch, there is no way to expose a PMU-without-PEBS to the guest if the host has PEBS.

It would be a bigger concern if we expected virtual PMU migration to work, but I think it would be nice to update kvm_pmu_cpuid_update() to notice the presence/absence of the new CPUID bits, and then store that into per-VM kvm_pmu->pebs_allowed rather than relying only on the per-host perf_pebs_virtualization().

Quoted 206 lines
arch/x86/kvm/pmu.c +136 -1
+               break;
+
        case MSR_CORE_PERF_FIXED_CTR_CTRL:
        case MSR_CORE_PERF_GLOBAL_STATUS:
        case MSR_CORE_PERF_GLOBAL_CTRL:

@@ -356,6 +385,18 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data)
        case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
                *data = pmu->global_ovf_ctrl;
                return 0;
+       case MSR_IA32_DS_AREA:
+               *data = pmu->ds_area;
+               return 0;
+       case MSR_IA32_PEBS_ENABLE:
+               *data = pmu->pebs_enable;
+               return 0;
+       case MSR_IA32_PERF_CAPABILITIES:
+               /* Report host PEBS format to guest */
+               *data = host_perf_cap &
+                       (PERF_CAP_PEBS_TRAP | PERF_CAP_ARCH_REG |
+                        PERF_CAP_PEBS_FORMAT);
+               return 0;
        default:
                if ((pmc = get_gp_pmc(pmu, index, MSR_IA32_PERFCTR0)) ||
                                (pmc = get_fixed_pmc(pmu, index))) {

@@ -369,6 +410,109 @@ int kvm_pmu_get_msr(struct kvm_vcpu *vcpu, u32 index, u64 *data)
        return 1;
 }
 
+static void kvm_pmu_release_pin(struct kvm_vcpu *vcpu)
+{
+       struct kvm_pmu *pmu = &vcpu->arch.pmu;
+       int i;
+
+       for (i = 0; i < pmu->num_pinned_pages; i++)
+               put_page(pmu->pinned_pages[i]);
+       pmu->num_pinned_pages = 0;
+}
+
+static struct page *get_guest_page(struct kvm_vcpu *vcpu,
+                                  unsigned long addr)
+{
+       unsigned long pfn;
+       struct x86_exception exception;
+       gpa_t gpa = vcpu->arch.walk_mmu->gva_to_gpa(vcpu, addr,
+                                                   PFERR_WRITE_MASK,
+                                                   &exception);
+
+       if (gpa == UNMAPPED_GVA) {
+               printk_once("Cannot translate guest page %lx\n", addr);
+               return NULL;
+       }
+       pfn = gfn_to_pfn(vcpu->kvm, gpa_to_gfn(gpa));
+       if (is_error_noslot_pfn(pfn)) {
+               printk_once("gfn_to_pfn failed for %llx\n", gpa);
+               return NULL;
+       }
+       return pfn_to_page(pfn);
+}
+
+static int pin_and_copy(struct kvm_vcpu *vcpu,
+                       unsigned long addr, void *dst, int len,
+                       struct page **p)
+{
+       unsigned long offset = addr & ~PAGE_MASK;
+       void *map;
+
+       *p = get_guest_page(vcpu, addr);
+       if (!*p)
+               return -EIO;
+       map = kmap(*p);
+       memcpy(dst, map + offset, len);
+       kunmap(map);
+       return 0;
+}
+
+/*
+ * Pin the DS area and the PEBS buffer while PEBS is active,
+ * because the CPU cannot tolerate EPT faults for PEBS updates.
+ *
+ * We assume that any guest who changes the DS buffer disables
+ * PEBS first and does not change the page tables during operation.
+ *
+ * When the guest violates these assumptions it may crash itself.
+ * This is expected to not happen with standard profilers.
+ *
+ * No need to clean up anything, as the caller will always eventually
+ * unpin pages.
+ */
+
+static void kvm_pmu_pebs_pin(struct kvm_vcpu *vcpu)
+{
+       struct kvm_pmu *pmu = &vcpu->arch.pmu;
+       struct debug_store ds;
+       int pg;
+       unsigned len;
+       unsigned long offset;
+       unsigned long addr;
+
+       offset = pmu->ds_area & ~PAGE_MASK;
+       len = sizeof(struct debug_store);
+       len = min_t(unsigned, PAGE_SIZE - offset, len);
+       if (pin_and_copy(vcpu, pmu->ds_area, &ds, len,
+                        &pmu->pinned_pages[0]) < 0) {
+               printk_once("Cannot pin ds area %llx\n", pmu->ds_area);
+               return;
+       }
+       pmu->num_pinned_pages++;
+       if (len < sizeof(struct debug_store)) {
+               if (pin_and_copy(vcpu, pmu->ds_area + len, (void *)&ds + len,
+                                 sizeof(struct debug_store) - len,
+                                 &pmu->pinned_pages[1]) < 0)
+                       return;
+               pmu->num_pinned_pages++;
+       }
+
+       pg = pmu->num_pinned_pages;
+       for (addr = ds.pebs_buffer_base;
+            addr < ds.pebs_absolute_maximum && pg < MAX_PINNED_PAGES;
+            addr += PAGE_SIZE, pg++) {
+               pmu->pinned_pages[pg] = get_guest_page(vcpu, addr);
+               if (!pmu->pinned_pages[pg]) {
+                       printk_once("Cannot pin PEBS buffer %lx (%llx-%llx)\n",
+                                addr,
+                                ds.pebs_buffer_base,
+                                ds.pebs_absolute_maximum);
+                       break;
+               }
+       }
+       pmu->num_pinned_pages = pg;
+}
+
 int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 {
        struct kvm_pmu *pmu = &vcpu->arch.pmu;

@@ -407,6 +551,20 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
                        return 0;
                }
                break;
+       case MSR_IA32_DS_AREA:
+               pmu->ds_area = data;
+               return 0;
+       case MSR_IA32_PEBS_ENABLE:
+               if (data & ~0xf0000000fULL)
+                       break;
+               if (data && data != pmu->pebs_enable) {
+                       kvm_pmu_release_pin(vcpu);
+                       kvm_pmu_pebs_pin(vcpu);
+               } else if (data == 0 && pmu->pebs_enable) {
+                       kvm_pmu_release_pin(vcpu);
+               }
+               pmu->pebs_enable = data;
+               return 0;
        default:
                if ((pmc = get_gp_pmc(pmu, index, MSR_IA32_PERFCTR0)) ||
                                (pmc = get_fixed_pmc(pmu, index))) {

@@ -418,7 +576,7 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
                        if (data == pmc->eventsel)
                                return 0;
                        if (!(data & pmu->reserved_bits)) {
-                               reprogram_gp_counter(pmc, data);
+                               reprogram_gp_counter(pmu, pmc, data);
                                return 0;
                        }
                }

@@ -514,6 +672,9 @@ void kvm_pmu_init(struct kvm_vcpu *vcpu)
        }
        init_irq_work(&pmu->irq_work, trigger_pmi);
        kvm_pmu_cpuid_update(vcpu);
+
+       if (boot_cpu_has(X86_FEATURE_PDCM))
+               rdmsrl_safe(MSR_IA32_PERF_CAPABILITIES, &host_perf_cap);
 }
 
 void kvm_pmu_reset(struct kvm_vcpu *vcpu)

@@ -538,6 +699,7 @@ void kvm_pmu_reset(struct kvm_vcpu *vcpu)
 void kvm_pmu_destroy(struct kvm_vcpu *vcpu)
 {
        kvm_pmu_reset(vcpu);
+       kvm_pmu_release_pin(vcpu);
 }
 
 void kvm_handle_pmu_event(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c +6 -1
index 33e8c02..4f39917 100644
@@ -7288,6 +7288,12 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
        atomic_switch_perf_msrs(vmx);
        debugctlmsr = get_debugctlmsr();
 
+       /* Move this somewhere else? */
+       if (vcpu->arch.pmu.ds_area)
+               add_atomic_switch_msr(vmx, MSR_IA32_DS_AREA,
+                                     vcpu->arch.pmu.ds_area,
+                                     perf_get_ds_area());
+
        vmx->__launched = vmx->loaded_vmcs->launched;
        asm(
                /* Store host registers */
--

1.9.0

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#16 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 2 lines

It seems to me that with this patch, there is no way to expose a PMU-without-PEBS to the guest if the host has PEBS.

If you clear the CPUIDs then noone would ilikely access it.

But fair enough, I'll add extra checks for CPUID.

Quoted 5 lines

It would be a bigger concern if we expected virtual PMU migration to work, but I think it would be nice to update kvm_pmu_cpuid_update() to notice the presence/absence of the new CPUID bits, and then store that into per-VM kvm_pmu->pebs_allowed rather than relying only on the per-host perf_pebs_virtualization().

I hope at some point it can work. There shouldn't be any problems with migrating to the same CPU model, in many cases (same event and same PEBS format) it'll likely even work between models or gracefully degrade.

BTW in practice it'll likely work anyways because many profilers regularly re-set the PMU.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
maintainer
#17 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Il 02/06/2014 21:57, Andi Kleen ha scritto:

Quoted 10 lines
Quoted 5 lines

It would be a bigger concern if we expected virtual PMU migration to work, but I think it would be nice to update kvm_pmu_cpuid_update() to notice the presence/absence of the new CPUID bits, and then store that into per-VM kvm_pmu->pebs_allowed rather than relying only on the per-host perf_pebs_virtualization().

I hope at some point it can work. There shouldn't be any problems with migrating to the same CPU model, in many cases (same event and same PEBS format) it'll likely even work between models or gracefully degrade.

The code is there in both kernel and QEMU, it's just very little tested.

Paolo

#18 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

In reply to Andi Kleen

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 559

Quoted 12 lines · Andi Kleen

{

arch/x86/kvm/pmu.c +6 -0
 	struct kvm_pmu *pmu = &vcpu->arch.pmu;

@@ -407,6 +551,20 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 0;
 		}
 		break;
+	case MSR_IA32_DS_AREA:
+		pmu->ds_area = data;
+		return 0;
+	case MSR_IA32_PEBS_ENABLE:
+		if (data & ~0xf0000000fULL)
+			break;

Bit 63 == PS_ENABLE ?

arch/x86/kvm/vmx.c static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 7291

Quoted 10 lines
arch/x86/kvm/pmu.c +0 -0
 void kvm_handle_pmu_event(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c +1 -0
index 33e8c02..4f39917 100644
@@ -7288,6 +7288,12 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	atomic_switch_perf_msrs(vmx);
 	debugctlmsr = get_debugctlmsr();
 
+	/* Move this somewhere else? */

Unless you hook into vcpu->arch.pmu.ds_area and perf_get_ds_area() writers, it has to be at every vcpu entry.

Could compare values in MSR save area to avoid switch.

arch/x86/kvm/vmx.c static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 7295

Quoted 4 lines
arch/x86/kvm/vmx.c +4 -0
+	if (vcpu->arch.pmu.ds_area)
+		add_atomic_switch_msr(vmx, MSR_IA32_DS_AREA,
+				      vcpu->arch.pmu.ds_area,
+				      perf_get_ds_area());

Should clear_atomic_switch_msr before
add_atomic_switch_msr.

#19 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 15 lines · Marcelo Tosatti

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 559

Quoted 12 lines · Andi Kleen

{

arch/x86/kvm/pmu.c +6 -0
 	struct kvm_pmu *pmu = &vcpu->arch.pmu;

@@ -407,6 +551,20 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 0;
 		}
 		break;
+	case MSR_IA32_DS_AREA:
+		pmu->ds_area = data;
+		return 0;
+	case MSR_IA32_PEBS_ENABLE:
+		if (data & ~0xf0000000fULL)
+			break;

Bit 63 == PS_ENABLE ?

PEBS_EN is [3:0] for each counter, but only one bit on Silvermont. LL_EN is [36:32], but currently unused.

Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/pmu.c +0 -0
 void kvm_handle_pmu_event(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c +0 -0
index 33e8c02..4f39917 100644
@@ -7288,6 +7288,12 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	atomic_switch_perf_msrs(vmx);
 	debugctlmsr = get_debugctlmsr();
Quoted 6 lines

arch/x86/kvm/vmx.c static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 7291

Quoted 1 line
arch/x86/kvm/vmx.c +1 -0
+	/* Move this somewhere else? */

Unless you hook into vcpu->arch.pmu.ds_area and perf_get_ds_area() writers, it has to be at every vcpu entry.

Could compare values in MSR save area to avoid switch.

Ok.

Quoted 8 lines

arch/x86/kvm/vmx.c static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 7295

Quoted 5 lines
arch/x86/kvm/vmx.c +4 -0
+	if (vcpu->arch.pmu.ds_area)
+		add_atomic_switch_msr(vmx, MSR_IA32_DS_AREA,
+				      vcpu->arch.pmu.ds_area,
+				      perf_get_ds_area());

Should clear_atomic_switch_msr before
add_atomic_switch_msr.

Ok.

BTW how about general PMU migration? As far as I can tell there is no code to save/restore the state for that currently, right?

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#20 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 48 lines · Andi Kleen
Quoted 15 lines · Marcelo Tosatti

arch/x86/kvm/pmu.c int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · line 559

Quoted 12 lines · Andi Kleen

{

arch/x86/kvm/pmu.c +6 -0
 	struct kvm_pmu *pmu = &vcpu->arch.pmu;

@@ -407,6 +551,20 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
 			return 0;
 		}
 		break;
+	case MSR_IA32_DS_AREA:
+		pmu->ds_area = data;
+		return 0;
+	case MSR_IA32_PEBS_ENABLE:
+		if (data & ~0xf0000000fULL)
+			break;

Bit 63 == PS_ENABLE ?

PEBS_EN is [3:0] for each counter, but only one bit on Silvermont. LL_EN is [36:32], but currently unused.

Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/pmu.c +0 -0
 void kvm_handle_pmu_event(struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c +0 -0
index 33e8c02..4f39917 100644
@@ -7288,6 +7288,12 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu)
 	atomic_switch_perf_msrs(vmx);
 	debugctlmsr = get_debugctlmsr();
Quoted 6 lines

arch/x86/kvm/vmx.c static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 7291

Quoted 1 line
arch/x86/kvm/vmx.c +1 -0
+	/* Move this somewhere else? */

Unless you hook into vcpu->arch.pmu.ds_area and perf_get_ds_area() writers, it has to be at every vcpu entry.

Could compare values in MSR save area to avoid switch.

Ok.

Quoted 8 lines

arch/x86/kvm/vmx.c static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 7295

Quoted 5 lines
arch/x86/kvm/vmx.c +4 -0
+	if (vcpu->arch.pmu.ds_area)
+		add_atomic_switch_msr(vmx, MSR_IA32_DS_AREA,
+				      vcpu->arch.pmu.ds_area,
+				      perf_get_ds_area());

Should clear_atomic_switch_msr before
add_atomic_switch_msr.

Quoted 6 lines

Ok.

BTW how about general PMU migration? As far as I can tell there is no code to save/restore the state for that currently, right?

-Andi

Paolo wrote support for it, recently. Paolo?

maintainer
#21 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Il 10/06/2014 23:06, Marcelo Tosatti ha scritto:

Quoted 4 lines
Quoted 2 lines

BTW how about general PMU migration? As far as I can tell there is no code to save/restore the state for that currently, right?

Paolo wrote support for it, recently. Paolo?

Yes, on the KVM side all that is needed is to special case MSR reads and writes that have side effects, for example:

        case MSR_CORE_PERF_GLOBAL_STATUS:
                if (msr_info->host_initiated) {
                        pmu->global_status = data;
                        return 0;
                }
                break; /* RO MSR */
        case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
                if (!(data & (pmu->global_ctrl_mask & ~(3ull<<62)))) {
                        if (!msr_info->host_initiated)
                                pmu->global_status &= ~data;
                        pmu->global_ovf_ctrl = data;
                        return 0;
                }
                break;

Right now this is only needed for writes.

Userspace then can read/write these MSRs, and add them to the migration stream. QEMU has code for that.

Paolo

#22 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 2 lines

Userspace then can read/write these MSRs, and add them to the migration stream. QEMU has code for that.

Thanks. The PEBS setup always redoes its state, can be arbitarily often redone.

So the only change needed would be to add the MSRs to some list in qemu?

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
maintainer
#23 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 7 lines
Quoted 2 lines

Userspace then can read/write these MSRs, and add them to the migration stream. QEMU has code for that.

Thanks. The PEBS setup always redoes its state, can be arbitarily often redone.

So the only change needed would be to add the MSRs to some list in qemu?

Yes, and also adding them to the migration stream if the MSRs do not have the default (all-zero? need to look at the SDM) values.

Paolo

#24 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

In reply to Andi Kleen
Quoted 18 lines · Andi Kleen

From: Andi Kleen <[email protected]>

PEBS (Precise Event Bases Sampling) profiling is very powerful, allowing improved sampling precision and much additional information, like address or TSX abort profiling. cycles:p and :pp uses PEBS.

This patch enables PEBS profiling in KVM guests.

PEBS writes profiling records to a virtual address in memory. Since the guest controls the virtual address space the PEBS record is directly delivered to the guest buffer. We set up the PEBS state that is works correctly.The CPU cannot handle any kinds of faults during these guest writes.

To avoid any problems with guest pages being swapped by the host we pin the pages when the PEBS buffer is setup, by intercepting that MSR.

Quoted 9 lines

Typically profilers only set up a single page, so pinning that is not a big problem. The pinning is limited to 17 pages currently (64K+1)

In theory the guest can change its own page tables after the PEBS setup. The host has no way to track that with EPT. But if a guest would do that it could only crash itself. It's not expected that normal profilers do that.

Talking a bit with Gleb about this, I think this is impossible.

First, it's not sufficient to pin the debug store area, you also have to pin the guest page tables that are used to map the debug store. But even if you do that, as soon as the guest fork()s, it will create a new pgd which the host will be free to swap out. The processor can then attempt a PEBS store to an unmapped address which will fail, even though the guest is configured correctly.

author
#25 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 6 lines

First, it's not sufficient to pin the debug store area, you also have to pin the guest page tables that are used to map the debug store. But even if you do that, as soon as the guest fork()s, it will create a new pgd which the host will be free to swap out. The processor can then attempt a PEBS store to an unmapped address which will fail, even though the guest is configured correctly.

That's a good point. You're right of course.

The only way I can think around it would be to intercept CR3 writes while PEBS is active and always pin all the table pages leading to the PEBS buffer. That's slow, but should be only needed while PEBS is running.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#26 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 7 lines · Andi Kleen
Quoted 6 lines

First, it's not sufficient to pin the debug store area, you also have to pin the guest page tables that are used to map the debug store. But even if you do that, as soon as the guest fork()s, it will create a new pgd which the host will be free to swap out. The processor can then attempt a PEBS store to an unmapped address which will fail, even though the guest is configured correctly.

Quoted 8 lines

That's a good point. You're right of course.

The only way I can think around it would be to intercept CR3 writes while PEBS is active and always pin all the table pages leading to the PEBS buffer. That's slow, but should be only needed while PEBS is running.

-Andi

Suppose that can be done separately from the pinned spte patchset. And it requires accounting into mlock limits as well, as noted.

One set of pagetables per pinned virtual address leading down to the last translations is sufficient per-vcpu.

#27 lore raw

Re: [PATCH 4/4] kvm: Implement PEBS virtualization

Quoted 14 lines · Marcelo Tosatti
Quoted 13 lines · Andi Kleen
Quoted 6 lines

First, it's not sufficient to pin the debug store area, you also have to pin the guest page tables that are used to map the debug store. But even if you do that, as soon as the guest fork()s, it will create a new pgd which the host will be free to swap out. The processor can then attempt a PEBS store to an unmapped address which will fail, even though the guest is configured correctly.

That's a good point. You're right of course.

The only way I can think around it would be to intercept CR3 writes while PEBS is active and always pin all the table pages leading to the PEBS buffer. That's slow, but should be only needed while PEBS is running.

Quoted 6 lines
Quoted 1 line

-Andi

Suppose that can be done separately from the pinned spte patchset. And it requires accounting into mlock limits as well, as noted.

One set of pagetables per pinned virtual address leading down to the last translations is sufficient per-vcpu.

Or 4, and use the CR3 exit filter to prevent vmexits among the last 4 LRU CR3 values.

maintainer
#28 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen

PEBS is very useful (e.g. enabling the more cycles:pp event or memory profiling) Unfortunately it didn't work in virtualization, which is becoming more and more common.

This patch kit implements simple PEBS virtualization for KVM on Silvermont CPUs. Silvermont does not have the leak problems that prevented successfull PEBS virtualization earlier.

Silvermont is such an underpowered thing, who in his right mind would run anything virt on it to further reduce performance?