VMWARE faults on accessing disabled counters

RFC PATCH In review Jiri Olsa 2 participants 4 messages Aug 31, 2016

Subsystems (2)

x86 perf events

lore

Share & tools

lore

Jiri Olsa

author

2016-08-31 12:04 UTC #0 lore raw

RFC PATCH 1/1 VMWARE faults on accessing disabled counters

hi,
when booting under VMWARE we've got following dmesg lines:

[    0.051567] perf_event_intel: CPUID marked event: 'cpu cycles' unavailable
[    0.051567] perf_event_intel: CPUID marked event: 'instructions' unavailable
[    0.051568] perf_event_intel: CPUID marked event: 'bus cycles' unavailable
[    0.051568] perf_event_intel: CPUID marked event: 'cache references' unavailable
[    0.051569] perf_event_intel: CPUID marked event: 'cache misses' unavailable
[    0.051570] perf_event_intel: CPUID marked event: 'branch instructions' unavailable
[    0.051570] perf_event_intel: CPUID marked event: 'branch misses' unavailable

that means all the architectural events are disabled by CPUID(0xa)

The kernel code sets intel_perfmon_event_map to prevent those event to be configured by PERF_TYPE_HARDWARE pmu type. However they can still be configured by via PERF_TYPE_RAW type.

We're getting GP fault on VMWARE when reading cycles PMC configured throgh the PERF_TYPE_RAW interface:

#4 [ffff88007c603e10] do_general_protection at ffffffff8163da9e #5 [ffff88007c603e40] general_protection at ffffffff8163d3a8

    [exception RIP: native_read_pmc+6]
    RIP: ffffffff81058d66  RSP: ffff88007c603ef0  RFLAGS: 00010083
    RAX: ffffffff81957ee0  RBX: 0000000000000000  RCX: 0000000040000002
    RDX: 000000000ff8f719  RSI: ffff88007c617fa8  RDI: 0000000040000002
    RBP: ffff88007c603ef0   R8: 00007ffde5053150   R9: 0000000000000000
    R10: 00007ffde5052530  R11: 00007fbb22aedc70  R12: ffffffff80000001
    R13: ffff880079b74400  R14: ffff880079b74578  R15: 0000000000000010
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0000

#6 [ffff88007c603ef8] x86_perf_event_update at ffffffff81029e03 #7 [ffff88007c603f30] x86_pmu_read at ffffffff8102a079 #8 [ffff88007c603f40] __perf_event_read at ffffffff811590de

I couldn't find what real HW rdpmc does on this situation, so I'm not sure if we actually want to prevent this.. patch below tries to catch this case.

thanks,
jirka

---
 arch/x86/events/core.c       |  8 ++++-
 arch/x86/events/intel/core.c | 72 ++++++++++++++++++++++++++++++++------------
 arch/x86/events/perf_event.h |  6 ++++
 3 files changed, 65 insertions(+), 21 deletions(-)

arch/x86/events/core.c +7 -1

index 473519100b11..d836c5922b12 100644
@@ -534,8 +534,14 @@ int x86_pmu_hw_config(struct perf_event *event)
 	if (!event->attr.exclude_kernel)
 		event->hw.config |= ARCH_PERFMON_EVENTSEL_OS;
 
-	if (event->attr.type == PERF_TYPE_RAW)
+	if (event->attr.type == PERF_TYPE_RAW) {
+		u64 arch_config = event->attr.config & INTEL_ARCH_EVENT_MASK;
+
+		if (x86_pmu_event_disabled(arch_config))
+			return -ENOENT;
+
 		event->hw.config |= event->attr.config & X86_RAW_EVENT_MASK;
+	}
 
 	if (event->attr.sample_period && x86_pmu.limit_period) {
 		if (x86_pmu.limit_period(event, event->attr.sample_period) >

arch/x86/events/intel/core.c +52 ~~-20~~

138 lines not on this page · raw message

arch/x86/events/perf_event.h +6 -0

index 01ddfeadaee6..69cca7dc8de4 100644
@@ -514,6 +514,7 @@ struct x86_pmu {
 	int		(*addr_offset)(int index, bool eventsel);
 	int		(*rdpmc_index)(int index);
 	u64		(*event_map)(int);
+	bool		(*event_disabled)(int);
 	int		max_events;
 	int		num_counters;
 	int		num_counters_fixed;

@@ -715,6 +716,11 @@ static inline int x86_pmu_rdpmc_index(int index)
 	return x86_pmu.rdpmc_index ? x86_pmu.rdpmc_index(index) : index;
 }
 
+static inline bool x86_pmu_event_disabled(u64 config)
+{
+	return x86_pmu.event_disabled ? x86_pmu.event_disabled(config) : false;
+}
+
 int x86_add_exclusive(unsigned int what);
 
 void x86_del_exclusive(unsigned int what);

Signature · 1 line

2.7.4

Peter Zijlstra

maintainer

2016-08-31 13:11 UTC #1 lore raw

Quoted 31 lines · Jiri Olsa

hi,
when booting under VMWARE we've got following dmesg lines:
[    0.051567] perf_event_intel: CPUID marked event: 'cpu cycles' unavailable
[    0.051567] perf_event_intel: CPUID marked event: 'instructions' unavailable
[    0.051568] perf_event_intel: CPUID marked event: 'bus cycles' unavailable
[    0.051568] perf_event_intel: CPUID marked event: 'cache references' unavailable
[    0.051569] perf_event_intel: CPUID marked event: 'cache misses' unavailable
[    0.051570] perf_event_intel: CPUID marked event: 'branch instructions' unavailable
[    0.051570] perf_event_intel: CPUID marked event: 'branch misses' unavailable
that means all the architectural events are disabled by CPUID(0xa)
The kernel code sets intel_perfmon_event_map to prevent those event to be configured by PERF_TYPE_HARDWARE pmu type. However they can still be configured by via PERF_TYPE_RAW type.
We're getting GP fault on VMWARE when reading cycles PMC configured throgh the PERF_TYPE_RAW interface:
#4 [ffff88007c603e10] do_general_protection at ffffffff8163da9e #5 [ffff88007c603e40] general_protection at ffffffff8163d3a8
    [exception RIP: native_read_pmc+6]
    RIP: ffffffff81058d66  RSP: ffff88007c603ef0  RFLAGS: 00010083
    RAX: ffffffff81957ee0  RBX: 0000000000000000  RCX: 0000000040000002
    RDX: 000000000ff8f719  RSI: ffff88007c617fa8  RDI: 0000000040000002
    RBP: ffff88007c603ef0   R8: 00007ffde5053150   R9: 0000000000000000
    R10: 00007ffde5052530  R11: 00007fbb22aedc70  R12: ffffffff80000001
    R13: ffff880079b74400  R14: ffff880079b74578  R15: 0000000000000010
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0000

Quoted 7 lines

#6 [ffff88007c603ef8] x86_perf_event_update at ffffffff81029e03 #7 [ffff88007c603f30] x86_pmu_read at ffffffff8102a079 #8 [ffff88007c603f40] __perf_event_read at ffffffff811590de
I couldn't find what real HW rdpmc does on this situation, so I'm not sure if we actually want to prevent this.. patch below tries to catch this case.

Typically real hardware allows you to program any old crap. The results, as in what the counter does, is undefined. Some actually count, some do not.

I'm not exactly thrilled by this patch, it adds a lot of code for a weird case. What happens when you stuff another non existing even in? GP again?

/me mutters vile things about virt

Jiri Olsa

author

2016-08-31 13:19 UTC #2 lore raw

Quoted 40 lines · Peter Zijlstra

Quoted 31 lines · Jiri Olsa
hi,
when booting under VMWARE we've got following dmesg lines:
[    0.051567] perf_event_intel: CPUID marked event: 'cpu cycles' unavailable
[    0.051567] perf_event_intel: CPUID marked event: 'instructions' unavailable
[    0.051568] perf_event_intel: CPUID marked event: 'bus cycles' unavailable
[    0.051568] perf_event_intel: CPUID marked event: 'cache references' unavailable
[    0.051569] perf_event_intel: CPUID marked event: 'cache misses' unavailable
[    0.051570] perf_event_intel: CPUID marked event: 'branch instructions' unavailable
[    0.051570] perf_event_intel: CPUID marked event: 'branch misses' unavailable
that means all the architectural events are disabled by CPUID(0xa)
The kernel code sets intel_perfmon_event_map to prevent those event to be configured by PERF_TYPE_HARDWARE pmu type. However they can still be configured by via PERF_TYPE_RAW type.
We're getting GP fault on VMWARE when reading cycles PMC configured throgh the PERF_TYPE_RAW interface:
#4 [ffff88007c603e10] do_general_protection at ffffffff8163da9e #5 [ffff88007c603e40] general_protection at ffffffff8163d3a8
    [exception RIP: native_read_pmc+6]
    RIP: ffffffff81058d66  RSP: ffff88007c603ef0  RFLAGS: 00010083
    RAX: ffffffff81957ee0  RBX: 0000000000000000  RCX: 0000000040000002
    RDX: 000000000ff8f719  RSI: ffff88007c617fa8  RDI: 0000000040000002
    RBP: ffff88007c603ef0   R8: 00007ffde5053150   R9: 0000000000000000
    R10: 00007ffde5052530  R11: 00007fbb22aedc70  R12: ffffffff80000001
    R13: ffff880079b74400  R14: ffff880079b74578  R15: 0000000000000010
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0000
Quoted 7 lines
#6 [ffff88007c603ef8] x86_perf_event_update at ffffffff81029e03 #7 [ffff88007c603f30] x86_pmu_read at ffffffff8102a079 #8 [ffff88007c603f40] __perf_event_read at ffffffff811590de
I couldn't find what real HW rdpmc does on this situation, so I'm not sure if we actually want to prevent this.. patch below tries to catch this case.

Quoted 7 lines

Typically real hardware allows you to program any old crap. The results, as in what the counter does, is undefined. Some actually count, some do not.
I'm not exactly thrilled by this patch, it adds a lot of code for a weird case. What happens when you stuff another non existing even in? GP again?

I guess if real HW does not fault on this we dont need to bother, and treat it as the VMWARE issue.. but I couldn't find this info

jirka

Peter Zijlstra

maintainer

2016-08-31 13:41 UTC #3 lore raw

Quoted 4 lines · Jiri Olsa

I guess if real HW does not fault on this we dont need to bother, and treat it as the VMWARE issue.. but I couldn't find this info

So I've once had a machine that backlisted one entry, but I've forgotten what machine that was. But IIRC, you could program the thing just fine, the results were crap though, which is why CPUID blacklisted the counter.

/me pokes around some code...

Ah, see the AAJ80 code. That was some Nehalem part and the comment very much implies programming the value 'works', you just get crap results.