perf: honoring cpuid for number of fixed counters
Subsystems (2)
RFC PATCH 1/1 perf: honoring cpuid for number of fixed counters
From: "Palik, Imre" <[email protected]>
perf doesn't seem to honor the number of fixed counters specified by cpuid leaf 0xa. It always assume that intel CPUs have at least 3 fixed counters.
So if some of the fixed counters are masked out by the hypervisor, it still tries to check/set them. This is good for testing the masking code in the hypervisor, but not so nice otherwise.
This patch makes perf pehave somewhat nicer when the number of fixed counters is less than three.
--- arch/x86/kernel/cpu/perf_event.c | 2 ++ arch/x86/kernel/cpu/perf_event_intel.c | 7 ------- 2 files changed, 2 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/perf_event.c +2 -0
@@ -647,6 +647,8 @@ static void perf_sched_init(struct perf_sched *sched, struct perf_event **events sched->state.event = idx;/* start with min weight */ sched->state.weight = wmin; sched->state.unassigned = num; + sched->state.used[0 ] = + ~0UL << (INTEL_PMC_IDX_FIXED + x86_pmu.num_counters_fixed); }static void perf_sched_save_state (struct perf_sched *sched)
arch/x86/kernel/cpu/perf_event_intel.c +0 -7
@@ -3042,13 +3042,6 @@ __init int intel_pmu_init(void) x86_pmu.max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , x86_pmu.num_counters); -/* -* Quirk: v2 perfmon does not report fixed-purpose events, so -* assume at least 3 events: -*/ -if (version >1 ) - x86_pmu.num_counters_fixed =max ((int )edx.split.num_counters_fixed,3 ); -if (boot_cpu_has (X86_FEATURE_PDCM )) {u64 capabilities;
Signature · 1 line
1.7.9.5
Peter Zijlstra
maintainerQuoted 15 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
perf doesn't seem to honor the number of fixed counters specified by cpuid leaf
0xa. It always assume that intel CPUs have at least 3 fixed counters.So if some of the fixed counters are masked out by the hypervisor, it still tries to check/set them. This is good for testing the masking code in the hypervisor, but not so nice otherwise.
This patch makes perf pehave somewhat nicer when the number of fixed counters is less than three.
arch/x86/kernel/cpu/perf_event_intel.c +0
-0@@ -3042,13 +3042,6 @@ __init int intel_pmu_init(void) x86_pmu.max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , x86_pmu.num_counters);
arch/x86/kernel/cpu/perf_event_intel.c __init int intel_pmu_init(void) · line 3046
Quoted 10 lines
arch/x86/kernel/cpu/perf_event_intel.c +0
-7-/* -* Quirk: v2 perfmon does not report fixed-purpose events, so -* assume at least 3 events: -*/ -if (version >1 ) - x86_pmu.num_counters_fixed =max ((int )edx.split.num_counters_fixed,3 ); -if (boot_cpu_has (X86_FEATURE_PDCM )) {u64 capabilities;
So the problem is that there is real hardware out there that gets the CPUID stuff wrong, and this patch penalizes that by then not using the fixed counters.
Further, the Intel Arch PerfMon v2 spec actually specifies there to be 3 fixed function counters.
So anything that says it is v2+ and does not have the 3, is non compliant.
I would suggest you go fix your hypervisor.
Lacking that option; you could probe the MSRs to see if they're really there using wrmsr_safe() or something like that -- see check_hw_exists().
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Imre Palik
From: "Palik, Imre" <[email protected]>
perf doesn't seem to honor the number of fixed counters specified by cpuid leaf
0xa. It always assume that intel CPUs have at least 3 fixed counters.So if some of the fixed counters are masked out by the hypervisor, it still tries to check/set them. This is good for testing the masking code in the hypervisor, but not so nice otherwise.
This patch makes perf pehave somewhat nicer when the number of fixed counters is less than three.
arch/x86/kernel/cpu/perf_event_intel.c +0
-0@@ -3042,13 +3042,6 @@ __init int intel_pmu_init(void) x86_pmu.max_pebs_events =min_t (unsigned,MAX_PEBS_EVENTS , x86_pmu.num_counters);
Quoted 13 lines
arch/x86/kernel/cpu/perf_event_intel.c __init int intel_pmu_init(void) · line 3046
Quoted 9 lines
arch/x86/kernel/cpu/perf_event_intel.c +0
-7-/* -* Quirk: v2 perfmon does not report fixed-purpose events, so -* assume at least 3 events: -*/ -if (version >1 ) - x86_pmu.num_counters_fixed =max ((int )edx.split.num_counters_fixed,3 ); -if (boot_cpu_has (X86_FEATURE_PDCM )) {u64 capabilities;So the problem is that there is real hardware out there that gets the CPUID stuff wrong, and this patch penalizes that by then not using the fixed counters.
I haven't thought about this. Thanks.
Quoted 7 lines
Further, the Intel Arch PerfMon v2 spec actually specifies there to be 3 fixed function counters.
So anything that says it is v2+ and does not have the 3, is non compliant.
I would suggest you go fix your hypervisor.
If I set up the hypervisor to advertise Arch PerfMon v1 (0 fixed counters), then without my patch, perf still tries to use fixed counters. So something is clearly broken here.
Quoted 3 lines
Lacking that option; you could probe the MSRs to see if they're really there using
wrmsr_safe()or something like that -- seecheck_hw_exists().
I'll send something along these lines soon.
Peter Zijlstra
maintainerQuoted 4 lines · Imre Palik
Quoted 3 lines · Peter Zijlstra
Further, the Intel Arch PerfMon v2 spec actually specifies there to be 3 fixed function counters.
Quoted 8 lines
Quoted 4 lines
So anything that says it is v2+ and does not have the 3, is non compliant.
I would suggest you go fix your hypervisor.
If I set up the hypervisor to advertise Arch PerfMon v1 (0 fixed counters), then without my patch, perf still tries to use fixed counters. So something is clearly broken here.
So the code you deleted does if (version > 1), and last I checked that should return false if version == 1.
So please check what's happening there first.
Quoted 10 lines · Peter Zijlstra
Quoted 9 lines · Imre Palik
Quoted 7 lines · Peter Zijlstra
Further, the Intel Arch PerfMon v2 spec actually specifies there to be 3 fixed function counters.
So anything that says it is v2+ and does not have the 3, is non compliant.
I would suggest you go fix your hypervisor.
Quoted 8 lines
Quoted 3 lines
If I set up the hypervisor to advertise Arch PerfMon v1 (0 fixed counters), then without my patch, perf still tries to use fixed counters. So something is clearly broken here.
So the code you deleted does if (version > 1), and last I checked that should return false if version == 1.
So please check what's happening there first.
Checked, before sent my last mail.
The trouble is that the number of fixed counters is not taken into account when scheduling the events, and the cpu model based event constraints will favour fixed counters. So perf tries to use them.
Peter Zijlstra
maintainerQuoted 3 lines · Imre Palik
The trouble is that the number of fixed counters is not taken into account when scheduling the events, and the cpu model based event constraints will favour fixed counters. So perf tries to use them.
Ah! so that is what your hunk below does. Tricky, and without comment that.
---
arch/x86/kernel/cpu/perf_event.c +2 -0
@@ -647,6 +647,8 @@ static void perf_sched_init(struct perf_sched *sched, struct perf_event **events sched->state.event = idx;/* start with min weight */ sched->state.weight = wmin; sched->state.unassigned = num; + sched->state.used[0 ] = + ~0UL << (INTEL_PMC_IDX_FIXED + x86_pmu.num_counters_fixed); }static void perf_sched_save_state (struct perf_sched *sched)
---
Please change the FIXED_EVENT constraints init instead; that way validate_event() will actually work too, otherwise it thinks it can schedule the fixed function only events.
That is, change the below loop from intel_pmu_init():
if (x86_pmu.event_constraints) {
/*
* event on fixed counter2 (REF_CYCLES) only works on this
* counter, so do not extend mask to generic counters
*/
for_each_event_constraint(c, x86_pmu.event_constraints) {
if (c->cmask != FIXED_EVENT_FLAGS
|| c->idxmsk64 == INTEL_PMC_MSK_FIXED_REF_CYCLES) {
continue;
}
c->idxmsk64 |= (1ULL << x86_pmu.num_counters) - 1;
c->weight += x86_pmu.num_counters;
}
}To clear all counters that are not in fact present, that way we keep the event constraints correct instead of working around invalid constraints.
Quoted 48 lines · Peter Zijlstra
Quoted 3 lines · Imre Palik
The trouble is that the number of fixed counters is not taken into account when scheduling the events, and the cpu model based event constraints will favour fixed counters. So perf tries to use them.
Ah! so that is what your hunk below does. Tricky, and without comment that.
---
arch/x86/kernel/cpu/perf_event.c +2
-1@@ -647,6 +647,8 @@ static void perf_sched_init(struct perf_sched *sched, struct perf_event **events sched->state.event = idx;/* start with min weight */ sched->state.weight = wmin; sched->state.unassigned = num; + sched->state.used[0 ] = + ~0UL << (INTEL_PMC_IDX_FIXED + x86_pmu.num_counters_fixed); }static void perf_sched_save_state (struct perf_sched *sched) ---Please change the
FIXED_EVENTconstraints init instead; that wayvalidate_event()will actually work too, otherwise it thinks it can schedule the fixed function only events.That is, change the below loop from
intel_pmu_init():if (x86_pmu.event_constraints) { /* * event on fixed counter2 (REF_CYCLES) only works on this * counter, so do not extend mask to generic counters */ for_each_event_constraint(c, x86_pmu.event_constraints) { if (c->cmask != FIXED_EVENT_FLAGS || c->idxmsk64 == INTEL_PMC_MSK_FIXED_REF_CYCLES) { continue; } c->idxmsk64 |= (1ULL << x86_pmu.num_counters) - 1; c->weight += x86_pmu.num_counters; } }
Quoted 2 lines
To clear all counters that are not in fact present, that way we keep the event constraints correct instead of working around invalid constraints.
Thanks.
I knew there should be a better way ...
Will post a new patch soon.