Basic perf PMU support for Haswell v9
Subsystems (2)
This is based on v7 of the full Haswell PMU support, rebased, reviewer-optimized and stripped down to the bare bones
Most interesting new features are not in this patchkit
(full version is git://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.git hsw/pmu5)
Contains support for:
- Basic Haswell PMU and PEBS support
- Late unmasking of the PMI
- Basic LBRv4 support
v2: Addressed Stephane's feedback. See individual patches for details.
v3: now even more bite-sized. Qualifier constraints merged earlier.
v4: Rename some variables, add some comments and other minor changes. Add some Reviewed/Tested-bys.
v5: Address some minor review feedback. Port to latest perf/core
v6: Add just some variable names, add comments, edit descriptions, some more testing, rebased to latest perf/core
v7: Expand comment
v8: Rename structure field.
v9: No wide counters, but add basic LBRs. Add some more constraints. Rebase to 3.9rc1
-Andi
PATCH 1/5 perf, x86: Add Haswell PEBS record support v4
From: Andi Kleen <[email protected]>
Add support for the Haswell extended (fmt2) PEBS format.
It has a superset of the nhm (fmt1) PEBS fields, but has a longer record so we need to adjust the code paths.
The main advantage is the new "EventingRip" support which directly gives the instruction, not off-by-one instruction. So with precise == 2 we use that directly and don't try to use LBRs and walking basic blocks. This lowers the overhead of using precise significantly.
Some other features are added in later patches.
v2: Rename various identifiers. Add more comments. Get rid of a cast.
v3: fmt2->hsw rename
v4: ip_of_the_event->real_ip rename
--- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event_intel_ds.c | 113 +++++++++++++++++++++++------ 2 files changed, 91 insertions(+), 24 deletions(-)
arch/x86/kernel/cpu/perf_event.c +1 -1
@@ -397,7 +397,7 @@ int x86_pmu_hw_config(struct perf_event *event)* check that PEBS LBR correction does not conflict with * whatever the user is asking with attr->branch_sample_type */ -if (event->attr.precise_ip >1 ) { +if (event->attr.precise_ip >1 && x86_pmu.intel_cap.pebs_format <2 ) {u64 *br_type = &event->attr.branch_sample_type;if (has_branch_stack (event)) {
arch/x86/kernel/cpu/perf_event_intel_ds.c +90 -23
161 lines not on this page · raw message
Signature · 1 line
1.7.7.6
PATCH 2/5 perf, x86: Basic Haswell PMU support v6
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add basic Haswell PMU support.
Similar to SandyBridge, but has a few new events and two new counter bits.
There are some new counter flags that need to be prevented from being set on fixed counters, and allowed to be set for generic counters.
Also we add support for the counter 2 constraint to handle all raw events.
Contains fixes from Stephane Eranian
v2: Folded TSX bits into standard FIXED_EVENT_CONSTRAINTS
v3: Use SNB LBR init code. Comment fix (Stephane Eranian)
v4: Add the counter2 constraints. Fix comment in the right place.
v5: Expand comment
v6: Add CYCLE_ACTIVITY.* to counter constraints
--- arch/x86/include/asm/perf_event.h | 3 + arch/x86/kernel/cpu/perf_event.h | 5 ++- arch/x86/kernel/cpu/perf_event_intel.c | 79 ++++++++++++++++++++++++++++++++ 3 files changed, 86 insertions(+), 1 deletions(-)
arch/x86/include/asm/perf_event.h +3 -0
@@ -29,6 +29,9 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL +#define HSW_INTX (1ULL << 32) +#define HSW_INTX_CHECKPOINTED (1ULL << 33) +#define AMD64_EVENTSEL_INT_CORE_ENABLE (1ULL << 36)#define AMD64_EVENTSEL_GUESTONLY (1ULL << 40)#define AMD64_EVENTSEL_HOSTONLY (1ULL << 41)
arch/x86/kernel/cpu/perf_event.h +4 -1
@@ -219,11 +219,14 @@ struct cpu_hw_events {* - inv * - edge * - cnt-mask +* - intx +* - intx_checkpointed * The other filters are supported by fixed counters. * The any-thread option is supported starting with v3. */ +#define FIXED_EVENT_FLAGS (X86_RAW_EVENT_MASK|HSW_INTX|HSW_INTX_CHECKPOINTED)#define FIXED_EVENT_CONSTRAINT (c, n) \ - EVENT_CONSTRAINT(c, (1ULL << (32+n)), X86_RAW_EVENT_MASK) + EVENT_CONSTRAINT(c, (1ULL << (32+n)), FIXED_EVENT_FLAGS)/* * Constraint on the Event code + UMask
arch/x86/kernel/cpu/perf_event_intel.c +79 -0
103 lines not on this page · raw message
Signature · 1 line
1.7.7.6
PATCH 3/5 perf, x86: Basic Haswell PEBS support v4
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add basic PEBS support for Haswell.
The constraints are similar to SandyBridge with a few new events.
v2: Readd missing pebs_aliases
v3: Readd missing hunk. Fix some constraints.
v4: Fix typo in PEBS event table (Stephane Eranian)
--- arch/x86/kernel/cpu/perf_event.h | 2 ++ arch/x86/kernel/cpu/perf_event_intel.c | 6 ++++-- arch/x86/kernel/cpu/perf_event_intel_ds.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 35 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -591,6 +591,8 @@ extern struct event_constraint intel_snb_pebs_event_constraints[];extern struct event_constraint intel_ivb_pebs_event_constraints[]; +extern struct event_constraint intel_hsw_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_enable (struct perf_event *event);
arch/x86/kernel/cpu/perf_event_intel.c +4 -2
@@ -854,7 +854,8 @@ static inline bool intel_pmu_needs_lbr_smpl(struct perf_event *event)return true;/* implicit branch sampling to correct PEBS skid */ -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) +if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 && + x86_pmu.intel_cap.pebs_format <2 )return true;return false;
@@ -2200,8 +2201,9 @@ __init int intel_pmu_init(void)intel_pmu_lbr_init_snb (); x86_pmu.event_constraints = intel_hsw_event_constraints; - + x86_pmu.pebs_constraints = intel_hsw_pebs_event_constraints; x86_pmu.extra_regs = intel_snb_extra_regs; + x86_pmu.pebs_aliases = intel_pebs_aliases_snb;/* all extra regs are per-cpu when HT is on */ x86_pmu.er_flags |=ERF_HAS_RSP_1 ; x86_pmu.er_flags |=ERF_NO_HT_SHARING ;
arch/x86/kernel/cpu/perf_event_intel_ds.c +29 -0
@@ -437,6 +437,35 @@ struct event_constraint intel_ivb_pebs_event_constraints[] = {EVENT_CONSTRAINT_END }; +struct event_constraint intel_hsw_pebs_event_constraints[] = { +INTEL_UEVENT_CONSTRAINT (0x01c0 ,0x2 ),/* INST_RETIRED.PRECDIST */ +INTEL_UEVENT_CONSTRAINT (0x01c2 ,0xf ),/* UOPS_RETIRED.ALL */ +INTEL_UEVENT_CONSTRAINT (0x02c2 ,0xf ),/* UOPS_RETIRED.RETIRE_SLOTS */ +INTEL_EVENT_CONSTRAINT (0xc4 ,0xf ),/* BR_INST_RETIRED.* */ +INTEL_UEVENT_CONSTRAINT (0x01c5 ,0xf ),/* BR_MISP_RETIRED.CONDITIONAL */ +INTEL_UEVENT_CONSTRAINT (0x04c5 ,0xf ),/* BR_MISP_RETIRED.ALL_BRANCHES */ +INTEL_UEVENT_CONSTRAINT (0x20c5 ,0xf ),/* BR_MISP_RETIRED.NEAR_TAKEN */ +INTEL_EVENT_CONSTRAINT (0xcd ,0x8 ),/* MEM_TRANS_RETIRED.* */ +INTEL_UEVENT_CONSTRAINT (0x11d0 ,0xf ),/* MEM_UOPS_RETIRED.STLB_MISS_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x12d0 ,0xf ),/* MEM_UOPS_RETIRED.STLB_MISS_STORES */ +INTEL_UEVENT_CONSTRAINT (0x21d0 ,0xf ),/* MEM_UOPS_RETIRED.LOCK_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x41d0 ,0xf ),/* MEM_UOPS_RETIRED.SPLIT_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x42d0 ,0xf ),/* MEM_UOPS_RETIRED.SPLIT_STORES */ +INTEL_UEVENT_CONSTRAINT (0x81d0 ,0xf ),/* MEM_UOPS_RETIRED.ALL_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x82d0 ,0xf ),/* MEM_UOPS_RETIRED.ALL_STORES */ +INTEL_UEVENT_CONSTRAINT (0x01d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L1_HIT */ +INTEL_UEVENT_CONSTRAINT (0x02d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L2_HIT */ +INTEL_UEVENT_CONSTRAINT (0x04d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L3_HIT */ +INTEL_UEVENT_CONSTRAINT (0x40d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.HIT_LFB */ +INTEL_UEVENT_CONSTRAINT (0x01d2 ,0xf ),/* MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS */ +INTEL_UEVENT_CONSTRAINT (0x02d2 ,0xf ),/* MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT */ +INTEL_UEVENT_CONSTRAINT (0x01d3 ,0xf ),/* MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM */ +INTEL_UEVENT_CONSTRAINT (0x04c8 ,0xf ),/* HLE_RETIRED.Abort */ +INTEL_UEVENT_CONSTRAINT (0x04c9 ,0xf ),/* RTM_RETIRED.Abort */ + +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
Signature · 1 line
1.7.7.6
PATCH 4/5 perf, x86: Move NMI clearing to end of PMI handler after the counter registers are reset
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
This avoids some problems with spurious PMIs on Haswell. Haswell seems to behave more like P4 in this regard. Do the same thing as the P4 perf handler by unmasking the NMI only at the end. Shouldn't make any difference for earlier family 6 cores.
Tested on Haswell, IvyBridge, Westmere, Saltwell (Atom)
--- arch/x86/kernel/cpu/perf_event_intel.c | 16 ++++++---------- 1 files changed, 6 insertions(+), 10 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +6 -10
@@ -1149,16 +1149,6 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) cpuc = &__get_cpu_var (cpu_hw_events); -/* -* Some chipsets need to unmask the LVTPC in a particular spot -* inside the nmi handler. As a result, the unmasking was pushed -* into all the nmi handlers. -* -* This handler doesn't seem to have any issues with the unmasking -* so it was left at the top. -*/ -apic_write (APIC_LVTPC ,APIC_DM_NMI ); -intel_pmu_disable_all (); handled =intel_pmu_drain_bts_buffer (); status =intel_pmu_get_status ();
@@ -1218,6 +1208,12 @@ again: done:intel_pmu_enable_all (0 ); +/* +* Only unmask the NMI after the overflow counters +* have been reset. This avoids spurious NMIs on +* Haswell CPUs. +*/ +apic_write (APIC_LVTPC ,APIC_DM_NMI );return handled; }
Signature · 1 line
1.7.7.6
PATCH 5/5 perf, x86: Support Haswell v4 LBR format
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Haswell has two additional LBR from flags for TSX: intx and abort, implemented as a new v4 version of the LBR format.
Handle those in and adjust the sign extension code to still correctly extend. The flags are exported similarly in the LBR record to the existing misprediction flag
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 18 +++++++++++++++--- include/linux/perf_event.h | 7 ++++++- 2 files changed, 21 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +15 -3
@@ -12,6 +12,7 @@ enum {LBR_FORMAT_LIP =0x01 ,LBR_FORMAT_EIP =0x02 ,LBR_FORMAT_EIP_FLAGS =0x03 , + LBR_FORMAT_EIP_FLAGS2 = 0x04, };/*
@@ -56,6 +57,8 @@ enum { LBR_FAR)#define LBR_FROM_FLAG_MISPRED (1ULL << 63) +#define LBR_FROM_FLAG_INTX (1ULL << 62) +#define LBR_FROM_FLAG_ABORT (1ULL << 61)#define for_each_branch_sample_type (x) \ for ((x) = PERF_SAMPLE_BRANCH_USER; \
@@ -270,21 +273,30 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)for (i =0 ; i < x86_pmu.lbr_nr; i++) {unsigned long lbr_idx = (tos - i) & mask; -u64 from, to, mis =0 , pred =0 ; +u64 from, to, mis =0 , pred =0 , intx =0 , abort =0 ;rdmsrl (x86_pmu.lbr_from + lbr_idx, from);rdmsrl (x86_pmu.lbr_to + lbr_idx, to); -if (lbr_format ==LBR_FORMAT_EIP_FLAGS ) { +if (lbr_format ==LBR_FORMAT_EIP_FLAGS || + lbr_format ==LBR_FORMAT_EIP_FLAGS2 ) { mis = !!(from &LBR_FROM_FLAG_MISPRED ); pred = !mis; - from = (u64)((((s64 )from) <<1 ) >>1 ); +if (lbr_format ==LBR_FORMAT_EIP_FLAGS ) + from = (u64)((((s64 )from) <<1 ) >>1 ); +else if (lbr_format ==LBR_FORMAT_EIP_FLAGS2 ) { + intx = !!(from &LBR_FROM_FLAG_INTX ); + abort = !!(from &LBR_FROM_FLAG_ABORT ); + from = (u64)((((s64 )from) <<3 ) >>3 ); + } } cpuc->lbr_entries[i].from = from; cpuc->lbr_entries[i].to = to; cpuc->lbr_entries[i].mispred = mis; cpuc->lbr_entries[i].predicted = pred; + cpuc->lbr_entries[i].intx = intx; + cpuc->lbr_entries[i].abort = abort; cpuc->lbr_entries[i].reserved =0 ; } cpuc->lbr_stack.nr = i;
include/linux/perf_event.h +6 -1
@@ -74,13 +74,18 @@ struct perf_raw_record {* * support for mispred, predicted is optional. In case it * is not supported mispred = predicted = 0. +* +* intx: running in a hardware transaction +* abort: aborting a hardware transaction */ struct perf_branch_entry {__u64 from;__u64 to;__u64 mispred:1 ,/* target mispredicted */ predicted:1 ,/* target predicted */ - reserved:62 ; + intx:1 ,/* in transaction */ + abort:1 ,/* transaction abort */ + reserved:60 ; };/*
Signature · 1 line
1.7.7.6
Quoted 11 lines · Andi Kleen
This is based on v7 of the full Haswell PMU support, rebased, reviewer-optimized and stripped down to the bare bones
Most interesting new features are not in this patchkit
(full version isgit://git.kernel.org/pub/scm/linux/kernel/git/ak/linux-misc.githsw/pmu5)Contains support for:
- Basic Haswell PMU and PEBS support
- Late unmasking of the PMI
- Basic LBRv4 support
Quoted 11 lines
v2: Addressed Stephane's feedback. See individual patches for details.
v3: now even more bite-sized. Qualifier constraints merged earlier.
v4: Rename some variables, add some comments and other minor changes. Add some Reviewed/Tested-bys.
v5: Address some minor review feedback. Port to latest perf/core
v6: Add just some variable names, add comments, edit descriptions, some more testing, rebased to latest perf/core
v7: Expand comment
v8: Rename structure field.
v9: No wide counters, but add basic LBRs. Add some more constraints. Rebase to 3.9rc1
I had another look at your latest patches and they are still quite sloppy:
you never ran them through checkpatch.pl, right? Tons of warnings, quite a few of which are legitimate.
These are all basic kernel hacking 101 mistakes you commited and we are already into version 9 of your patch-set ...
Having to point out trivial errors in your patches again and again is not a very efficient use of my time, frankly.
Thanks,
Ingo