Basic perf PMU support for Haswell v5
Subsystems (2)
This is based on v7 of the full Haswell PMU support, but ported to the latest perf/core and stripped down to the bare bones
Only for very extremly basic usage.
Most interesting new features are not in this patchkit
(full version isgit://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git hsw/pmu5)
Contains support for:
- Basic Haswell PMU and PEBS support
- Late unmasking of the PMI
- Support for wide counters
v2: Addressed Stephane's feedback. See individual patches for details.
v3: now even more bite-sized. Qualifier constraints merged earlier.
v4: Rename some variables, add some comments and other minor changes.
v5: Address some minor review feedback. Port to latest perf/core Add some Reviewed/Tested-bys.
-Andi
PATCH 1/5 perf, x86: Add PEBSv2 record support v2
From: Andi Kleen <[email protected]>
Add support for the v2 PEBS format. It has a superset of the v1 PEBS fields, but has a longer record so we need to adjust the code paths.
The main advantage is the new "EventingRip" support which directly gives the instruction, not off-by-one instruction. So with precise == 2 we use that directly and don't try to use LBRs and walking basic blocks. This lowers the overhead significantly.
Some other features are added in later patches.
v2: Rename various identifiers. Add more comments. Get rid of a cast.
--- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event_intel_ds.c | 113 +++++++++++++++++++++++------ 2 files changed, 91 insertions(+), 24 deletions(-)
arch/x86/kernel/cpu/perf_event.c +1 -1
@@ -397,7 +397,7 @@ int x86_pmu_hw_config(struct perf_event *event)* check that PEBS LBR correction does not conflict with * whatever the user is asking with attr->branch_sample_type */ -if (event->attr.precise_ip >1 ) { +if (event->attr.precise_ip >1 && x86_pmu.intel_cap.pebs_format <2 ) {u64 *br_type = &event->attr.branch_sample_type;if (has_branch_stack (event)) {
arch/x86/kernel/cpu/perf_event_intel_ds.c +90 -23
161 lines not on this page · raw message
Signature · 1 line
1.7.7.6
Re: [PATCH 1/5] perf, x86: Add PEBSv2 record support v2
Quoted 4 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add support for the v2 PEBS format. It has a superset of the v1 PEBS fields, but has a longer record so we need to adjust the code paths.
The changelog talks about a 'v2' PEBS format - the code talks about 'fmt2':
arch/x86/kernel/cpu/perf_event_intel_ds.c struct pebs_record_nhm { · line 47
Quoted 4 lines
arch/x86/kernel/cpu/perf_event_intel_ds.c +4
-0+/* +* Same as pebs_record_nhm, with two additional fields. +*/ +struct pebs_record_fmt2 {
In light of the existing pebs_record_core and pebs_record_nhm naming a new perf_record_hsw would be more natural.
Or if numbered versions are used, all PEBS record hardware versions should be numbered.
Right now your extension makes it all look pretty inconsistent.
Thanks,
Ingo
PATCH 2/5 perf, x86: Basic Haswell PMU support v4
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add basic Haswell PMU support.
Similar to SandyBridge, but has a few new events and two new counter bits.
There are some new counter flags that need to be prevented from being set on fixed counters, and allowed to be set for generic counters.
Also we add support for the counter 2 constraint to handle all raw events.
Contains fixes from Stephane Eranian
v2: Folded TSX bits into standard FIXED_EVENT_CONSTRAINTS
v3: Use SNB LBR init code. Comment fix (Stephane Eranian)
v4: Add the counter2 constraints. Fix comment in the right place.
--- arch/x86/include/asm/perf_event.h | 5 ++ arch/x86/kernel/cpu/perf_event.h | 5 ++- arch/x86/kernel/cpu/perf_event_intel.c | 71 ++++++++++++++++++++++++++++++++ 3 files changed, 80 insertions(+), 1 deletions(-)
arch/x86/include/asm/perf_event.h +5 -0
@@ -29,8 +29,13 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL +#define HSW_INTX (1ULL << 32) +#define HSW_INTX_CHECKPOINTED (1ULL << 33) +#define AMD64_EVENTSEL_GUESTONLY (1ULL << 40)#define AMD64_EVENTSEL_HOSTONLY (1ULL << 41) +#define AMD_PERFMON_EVENTSEL_GUESTONLY (1ULL << 40) +#define AMD_PERFMON_EVENTSEL_HOSTONLY (1ULL << 41)#define AMD64_EVENTSEL_EVENT \ (ARCH_PERFMON_EVENTSEL_EVENT | (0x0FULL << 32))
arch/x86/kernel/cpu/perf_event.h +4 -1
@@ -219,11 +219,14 @@ struct cpu_hw_events {* - inv * - edge * - cnt-mask +* - intx +* - intx_cp * The other filters are supported by fixed counters. * The any-thread option is supported starting with v3. */ +#define FIXED_EVENT_FLAGS (X86_RAW_EVENT_MASK|HSW_INTX|HSW_INTX_CHECKPOINTED)#define FIXED_EVENT_CONSTRAINT (c, n) \ - EVENT_CONSTRAINT(c, (1ULL << (32+n)), X86_RAW_EVENT_MASK) + EVENT_CONSTRAINT(c, (1ULL << (32+n)), FIXED_EVENT_FLAGS)/* * Constraint on the Event code + UMask
arch/x86/kernel/cpu/perf_event_intel.c +71 -0
95 lines not on this page · raw message
Signature · 1 line
1.7.7.6
PATCH 3/5 perf, x86: Basic Haswell PEBS support v4
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add basic PEBS support for Haswell.
The constraints are similar to SandyBridge with a few new events.
v2: Readd missing pebs_aliases
v3: Readd missing hunk. Fix some constraints.
v4: Fix typo in PEBS event table (Stephane Eranian)
--- arch/x86/kernel/cpu/perf_event.h | 2 ++ arch/x86/kernel/cpu/perf_event_intel.c | 6 ++++-- arch/x86/kernel/cpu/perf_event_intel_ds.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 35 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -591,6 +591,8 @@ extern struct event_constraint intel_snb_pebs_event_constraints[];extern struct event_constraint intel_ivb_pebs_event_constraints[]; +extern struct event_constraint intel_hsw_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_enable (struct perf_event *event);
arch/x86/kernel/cpu/perf_event_intel.c +4 -2
@@ -827,7 +827,8 @@ static inline bool intel_pmu_needs_lbr_smpl(struct perf_event *event)return true;/* implicit branch sampling to correct PEBS skid */ -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) +if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 && + x86_pmu.intel_cap.pebs_format <2 )return true;return false;
@@ -2167,8 +2168,9 @@ __init int intel_pmu_init(void)intel_pmu_lbr_init_snb (); x86_pmu.event_constraints = intel_hsw_event_constraints; - + x86_pmu.pebs_constraints = intel_hsw_pebs_event_constraints; x86_pmu.extra_regs = intel_snb_extra_regs; + x86_pmu.pebs_aliases = intel_pebs_aliases_snb;/* all extra regs are per-cpu when HT is on */ x86_pmu.er_flags |=ERF_HAS_RSP_1 ; x86_pmu.er_flags |=ERF_NO_HT_SHARING ;
arch/x86/kernel/cpu/perf_event_intel_ds.c +29 -0
@@ -437,6 +437,35 @@ struct event_constraint intel_ivb_pebs_event_constraints[] = {EVENT_CONSTRAINT_END }; +struct event_constraint intel_hsw_pebs_event_constraints[] = { +INTEL_UEVENT_CONSTRAINT (0x01c0 ,0x2 ),/* INST_RETIRED.PRECDIST */ +INTEL_UEVENT_CONSTRAINT (0x01c2 ,0xf ),/* UOPS_RETIRED.ALL */ +INTEL_UEVENT_CONSTRAINT (0x02c2 ,0xf ),/* UOPS_RETIRED.RETIRE_SLOTS */ +INTEL_EVENT_CONSTRAINT (0xc4 ,0xf ),/* BR_INST_RETIRED.* */ +INTEL_UEVENT_CONSTRAINT (0x01c5 ,0xf ),/* BR_MISP_RETIRED.CONDITIONAL */ +INTEL_UEVENT_CONSTRAINT (0x04c5 ,0xf ),/* BR_MISP_RETIRED.ALL_BRANCHES */ +INTEL_UEVENT_CONSTRAINT (0x20c5 ,0xf ),/* BR_MISP_RETIRED.NEAR_TAKEN */ +INTEL_EVENT_CONSTRAINT (0xcd ,0x8 ),/* MEM_TRANS_RETIRED.* */ +INTEL_UEVENT_CONSTRAINT (0x11d0 ,0xf ),/* MEM_UOPS_RETIRED.STLB_MISS_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x12d0 ,0xf ),/* MEM_UOPS_RETIRED.STLB_MISS_STORES */ +INTEL_UEVENT_CONSTRAINT (0x21d0 ,0xf ),/* MEM_UOPS_RETIRED.LOCK_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x41d0 ,0xf ),/* MEM_UOPS_RETIRED.SPLIT_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x42d0 ,0xf ),/* MEM_UOPS_RETIRED.SPLIT_STORES */ +INTEL_UEVENT_CONSTRAINT (0x81d0 ,0xf ),/* MEM_UOPS_RETIRED.ALL_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x82d0 ,0xf ),/* MEM_UOPS_RETIRED.ALL_STORES */ +INTEL_UEVENT_CONSTRAINT (0x01d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L1_HIT */ +INTEL_UEVENT_CONSTRAINT (0x02d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L2_HIT */ +INTEL_UEVENT_CONSTRAINT (0x04d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L3_HIT */ +INTEL_UEVENT_CONSTRAINT (0x40d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.HIT_LFB */ +INTEL_UEVENT_CONSTRAINT (0x01d2 ,0xf ),/* MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS */ +INTEL_UEVENT_CONSTRAINT (0x02d2 ,0xf ),/* MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT */ +INTEL_UEVENT_CONSTRAINT (0x01d3 ,0xf ),/* MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM */ +INTEL_UEVENT_CONSTRAINT (0x04c8 ,0xf ),/* HLE_RETIRED.Abort */ +INTEL_UEVENT_CONSTRAINT (0x04c9 ,0xf ),/* RTM_RETIRED.Abort */ + +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
Signature · 1 line
1.7.7.6
PATCH 4/5 perf, x86: Support full width counting v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Recent Intel CPUs like Haswell and IvyBridge have a new alternative MSR range for perfctrs that allows writing the full counter width. Enable this range if the hardware reports it using a new capability bit.
This lowers the overhead of perf stat slightly because it has to do less interrupts to accumulate the counter value. On Haswell it also avoids some problems with TSX aborting when the end of the counter range is reached.
v2: Print the feature at boot
--- arch/x86/include/uapi/asm/msr-index.h | 3 +++ arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel.c | 7 +++++++ 3 files changed, 11 insertions(+), 0 deletions(-)
arch/x86/include/uapi/asm/msr-index.h +3 -0
@@ -163,6 +163,9 @@#define MSR_KNC_EVNTSEL0 0x00000028#define MSR_KNC_EVNTSEL1 0x00000029 +/* Alternative perfctr range with full access. */ +#define MSR_IA32_PMC0 0x000004c1 +/* AMD64 MSRs. Not complete. See the architecture manual for a more complete list. */
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -278,6 +278,7 @@ union perf_capabilities { u64 pebs_arch_reg:1; u64 pebs_format:4; u64 smm_freeze:1; + u64 fw_write:1; };u64 capabilities; };
arch/x86/kernel/cpu/perf_event_intel.c +7 -0
@@ -2228,5 +2228,12 @@ __init int intel_pmu_init(void) } } +/* Support full width counters using alternative MSR range */ +if (x86_pmu.intel_cap.fw_write) { + x86_pmu.max_period = x86_pmu.cntval_mask; + x86_pmu.perfctr =MSR_IA32_PMC0 ; +pr_cont ("full-width counters, " ); + } +return 0 ; }
Signature · 1 line
1.7.7.6
Re: [PATCH 4/5] perf, x86: Support full width counting v2
Quoted 38 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Recent Intel CPUs like Haswell and IvyBridge have a new alternative MSR range for perfctrs that allows writing the full counter width. Enable this range if the hardware reports it using a new capability bit.
This lowers the overhead of perf stat slightly because it has to do less interrupts to accumulate the counter value. On Haswell it also avoids some problems with TSX aborting when the end of the counter range is reached.
v2: Print the feature at boot
Reviewed-by Stephane Eranian <[email protected]>Signed-off-by Andi Kleen <[email protected]>--- arch/x86/include/uapi/asm/msr-index.h | 3 +++ arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel.c | 7 +++++++ 3 files changed, 11 insertions(+), 0 deletions(-)arch/x86/include/uapi/asm/msr-index.h +3
-0@@ -163,6 +163,9 @@#define MSR_KNC_EVNTSEL0 0x00000028#define MSR_KNC_EVNTSEL1 0x00000029 +/* Alternative perfctr range with full access. */ +#define MSR_IA32_PMC0 0x000004c1 +/* AMD64 MSRs. Not complete. See the architecture manual for a more complete list. */ arch/x86/kernel/cpu/perf_event.h +0
-0
arch/x86/kernel/cpu/perf_event.h union perf_capabilities { · line 281
Quoted 4 lines
arch/x86/kernel/cpu/perf_event.h +1
-0@@ -278,6 +278,7 @@ union perf_capabilities { u64 pebs_arch_reg:1; u64 pebs_format:4; u64 smm_freeze:1; + u64 fw_write:1;
No brownies for obfuscation: that should be named full_width_write, not a meaningless abbreviation that anyone crossing the code could mistake as 'firewall write' or 'forward write' or anything.
Also a comment line at the field should explain its meaning.
Thanks,
Ingo
PATCH 5/5 perf, x86: Move NMI clearing to end of PMI handler after the counter registers are reset
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
This avoids some problems with spurious PMIs on Haswell. Haswell seems to behave more like P4 in this regard. Do the same thing as the P4 perf handler by unmasking the NMI only at the end. Shouldn't make any difference for earlier non P4 cores.
--- arch/x86/kernel/cpu/perf_event_intel.c | 16 ++++++---------- 1 files changed, 6 insertions(+), 10 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +6 -10
@@ -1122,16 +1122,6 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) cpuc = &__get_cpu_var (cpu_hw_events); -/* -* Some chipsets need to unmask the LVTPC in a particular spot -* inside the nmi handler. As a result, the unmasking was pushed -* into all the nmi handlers. -* -* This handler doesn't seem to have any issues with the unmasking -* so it was left at the top. -*/ -apic_write (APIC_LVTPC ,APIC_DM_NMI ); -intel_pmu_disable_all (); handled =intel_pmu_drain_bts_buffer (); status =intel_pmu_get_status ();
@@ -1191,6 +1181,12 @@ again: done:intel_pmu_enable_all (0 ); +/* +* Only unmask the NMI after the overflow counters +* have been reset. This avoids spurious NMIs on +* Haswell CPUs. +*/ +apic_write (APIC_LVTPC ,APIC_DM_NMI );return handled; }
Signature · 1 line
1.7.7.6
Re: [PATCH 5/5] perf, x86: Move NMI clearing to end of PMI handler after the counter registers are reset
Quoted 7 lines · Andi Kleen
From: Andi Kleen <[email protected]>
This avoids some problems with spurious PMIs on Haswell. Haswell seems to behave more like P4 in this regard. Do the same thing as the P4 perf handler by unmasking the NMI only at the end. Shouldn't make any difference for earlier non P4 cores.
Was this stress-tested on all affected main CPU types, or only on Haswell?
Thanks,
Ingo
Re: [PATCH 5/5] perf, x86: Move NMI clearing to end of PMI handler after the counter registers are reset
Quoted 2 lines · Ingo Molnar
Was this stress-tested on all affected main CPU types, or only on Haswell?
I tested it on Haswell and Ivy Bridge. I can also try Westmere and a Saltwell(Atom), but for the majority of other family 6 systems I'll need to rely on the community.
White listing is somewhat difficult because it affects the architectural mode too.
I don't really expect problems from this change, we should probably have always done it like this.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Re: [PATCH 5/5] perf, x86: Move NMI clearing to end of PMI handler after the counter registers are reset
Quoted 7 lines · Andi Kleen
Quoted 2 lines · Ingo Molnar
Was this stress-tested on all affected main CPU types, or only on Haswell?
I tested it on Haswell and Ivy Bridge. I can also try Westmere and a Saltwell(Atom), but for the majority of other family 6 systems I'll need to rely on the community.
The systems you tested should be OK.
Quoted 2 lines
White listing is somewhat difficult because it affects the architectural mode too.
Yeah, I'd rather avoid that.
Quoted 2 lines
I don't really expect problems from this change, we should probably have always done it like this.
I expect potential problems: the ordering of the operations in the NMI handler was always very fragile, resulting in hard to debug hangs - which sometimes needed hours long very intense PMU stress-testing to trigger.
That is why I asked how heavily you've tested this. Once the series passes review I'll keep this patch last to make it easy to revert/zap if it causes problems.
Thanks,
Ingo