perf PMU support for Haswell v6
Subsystems (3)
[Updated version for the latest master tree and various fixes, addressing review feedback. See end for details.
This should be ready for merging now, just waiting for Peter&Ingo.]
This adds perf PMU support for the upcoming Haswell core. The patchkit is fairly large, mainly due to various enhancement for TSX. TSX tuning relies heavily on the PMU, so I tried hard to make all facilities easily available. In addition it also has some other enhancements.
This includes changes to the core perf code, to the x86 specific part, to the perf user land tools and to KVM
Available atgit://git.kernel.org/pub/scm/linux/kernel/ak/linux-misc.git hsw/pmu3
High level overview:
- Basic Haswell PMU support
- Easy high level TSX measurement in perf stat -T
- Transaction events and attributes implemented with sysfs enumeration
- Export arch perfmon events in sysfs
- Generic weightend profiling for memory latency and transaction abort costs.
- Support for address profiling
- Support for filtering events inside/outside transactions
- KVM support to do this from guests
- Support for filtering/sorting/bucketing transaction abort types based on PEBS information
- LBR support for transactions
For more details on the Haswell PMU please see the SDM. For more details on TSX please see http://halobates.de/adding-lock-elision-to-linux.pdf
Some of the added features could be added to older CPUs too. I plan to do this, but in separate patches.
Review appreciated.
v2: Removed generic transaction events and qualifiers and use sysfs enumeration. Also export arch perfmon, so that the qualifiers work. Fixed various issues this exposed. Don't use a special macro for the TSX constraints anymore. Address other review feedback. Added pdir event in sysfs.
v3: Fix various bugs and address review comments. tx-aborts instead of cpu/tx-aborts/ works now (with some limitations) cpu/instructions,intx=1/ works now
v4:
Addressed all review feedback (I hope). See changelog in individual patches. KVM support now works again with more changes.
Forbid some more flag combinations that don't work well.
v5:
Rebased on latest perf/core. New method for sysfs events. Obsolete patches dropped. Added one patch from Stephane. Fixed generic aliases inside cpu//
Improved transaction flags decoding
Addressed all review feedback (except for two minor items in perf tools from Namhyung)
v6:
Fix WERROR=1 build with latest fixes.
Address KVM feedback.
Improve transaction flags display.
-Andi
PATCH 1/32 perf, x86: Add PEBSv2 record support
From: Andi Kleen <[email protected]>
Add support for the v2 PEBS format. It has a superset of the v1 PEBS fields, but has a longer record so we need to adjust the code paths.
The main advantage is the new "EventingRip" support which directly gives the instruction, not off-by-one instruction. So with precise == 2 we use that directly and don't try to use LBRs and walking basic blocks. This lowers the overhead significantly.
Some other features are added in later patches.
--- arch/x86/kernel/cpu/perf_event.c | 2 +- arch/x86/kernel/cpu/perf_event_intel_ds.c | 101 ++++++++++++++++++++++------- 2 files changed, 79 insertions(+), 24 deletions(-)
arch/x86/kernel/cpu/perf_event.c +1 -1
@@ -403,7 +403,7 @@ int x86_pmu_hw_config(struct perf_event *event)* check that PEBS LBR correction does not conflict with * whatever the user is asking with attr->branch_sample_type */ -if (event->attr.precise_ip >1 ) { +if (event->attr.precise_ip >1 && x86_pmu.intel_cap.pebs_format <2 ) {u64 *br_type = &event->attr.branch_sample_type;if (has_branch_stack (event)) {
arch/x86/kernel/cpu/perf_event_intel_ds.c +78 -23
146 lines not on this page · raw message
Signature · 1 line
1.7.7.6
PATCH 2/32 perf, x86: Basic Haswell PMU support v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add basic Haswell PMU support.
Similar to SandyBridge, but has a few new events. Further differences are handled in followon patches.
There are some new counter flags that need to be prevented from being set on fixed counters.
Contains fixes from Stephane Eranian
v2: Folded TSX bits into standard FIXED_EVENT_CONSTRAINTS
--- arch/x86/include/asm/perf_event.h | 3 +++ arch/x86/kernel/cpu/perf_event.h | 5 ++++- arch/x86/kernel/cpu/perf_event_intel.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 36 insertions(+), 1 deletions(-)
arch/x86/include/asm/perf_event.h +3 -0
@@ -29,6 +29,9 @@#define ARCH_PERFMON_EVENTSEL_INV (1ULL << 23)#define ARCH_PERFMON_EVENTSEL_CMASK 0xFF000000ULL +#define HSW_INTX (1ULL << 32) +#define HSW_INTX_CHECKPOINTED (1ULL << 33) +#define AMD_PERFMON_EVENTSEL_GUESTONLY (1ULL << 40)#define AMD_PERFMON_EVENTSEL_HOSTONLY (1ULL << 41)
arch/x86/kernel/cpu/perf_event.h +4 -1
@@ -219,11 +219,14 @@ struct cpu_hw_events {* - inv * - edge * - cnt-mask +* - intx +* - intx_cp * The other filters are supported by fixed counters. * The any-thread option is supported starting with v3. */ +#define FIXED_EVENT_FLAGS (X86_RAW_EVENT_MASK|HSW_INTX|HSW_INTX_CHECKPOINTED)#define FIXED_EVENT_CONSTRAINT (c, n) \ - EVENT_CONSTRAINT(c, (1ULL << (32+n)), X86_RAW_EVENT_MASK) + EVENT_CONSTRAINT(c, (1ULL << (32+n)), FIXED_EVENT_FLAGS)/* * Constraint on the Event code + UMask
arch/x86/kernel/cpu/perf_event_intel.c +29 -0
@@ -133,6 +133,17 @@ static struct extra_reg intel_snb_extra_regs[] __read_mostly = { EVENT_EXTRA_END }; +static struct event_constraint intel_hsw_event_constraints[] = +{ +FIXED_EVENT_CONSTRAINT (0x00c0 ,0 ),/* INST_RETIRED.ANY */ +FIXED_EVENT_CONSTRAINT (0x003c ,1 ),/* CPU_CLK_UNHALTED.CORE */ +FIXED_EVENT_CONSTRAINT (0x0300 ,2 ),/* CPU_CLK_UNHALTED.REF */ +INTEL_EVENT_CONSTRAINT (0x48 ,0x4 ),/* L1D_PEND_MISS.PENDING */ +INTEL_UEVENT_CONSTRAINT (0x01c0 ,0x2 ),/* INST_RETIRED.PREC_DIST */ +INTEL_EVENT_CONSTRAINT (0xcd ,0x8 ),/* MEM_TRANS_RETIRED.LOAD_LATENCY */ +EVENT_CONSTRAINT_END +}; +static u64 intel_pmu_event_map (int hw_event) {return intel_perfmon_event_map[hw_event];
@@ -2107,6 +2118,24 @@ __init int intel_pmu_init(void)break ; +case 60 :/* Haswell Client */ +case 70 : +case 71 : +memcpy (hw_cache_event_ids, snb_hw_cache_event_ids, +sizeof (hw_cache_event_ids)); + +intel_pmu_lbr_init_nhm (); + + x86_pmu.event_constraints = intel_hsw_event_constraints; + + x86_pmu.extra_regs = intel_snb_extra_regs; +/* all extra regs are per-cpu when HT is on */ + x86_pmu.er_flags |=ERF_HAS_RSP_1 ; + x86_pmu.er_flags |=ERF_NO_HT_SHARING ; + +pr_cont ("Haswell events, " ); +break ; +default :switch (x86_pmu.version) {case 1 :
Signature · 1 line
1.7.7.6
PATCH 3/32 perf, x86: Basic Haswell PEBS support v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add basic PEBS support for Haswell.
The constraints are similar to SandyBridge with a few new events.
v2: Readd missing pebs_aliases
v3: Readd missing hunk. Fix some constraints.
--- arch/x86/kernel/cpu/perf_event.h | 2 ++ arch/x86/kernel/cpu/perf_event_intel.c | 6 ++++-- arch/x86/kernel/cpu/perf_event_intel_ds.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 35 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -596,6 +596,8 @@ extern struct event_constraint intel_snb_pebs_event_constraints[];extern struct event_constraint intel_ivb_pebs_event_constraints[]; +extern struct event_constraint intel_hsw_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_enable (struct perf_event *event);
arch/x86/kernel/cpu/perf_event_intel.c +4 -2
@@ -826,7 +826,8 @@ static inline bool intel_pmu_needs_lbr_smpl(struct perf_event *event)return true;/* implicit branch sampling to correct PEBS skid */ -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) +if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 && + x86_pmu.intel_cap.pebs_format <2 )return true;return false;
@@ -2127,8 +2128,9 @@ __init int intel_pmu_init(void)intel_pmu_lbr_init_nhm (); x86_pmu.event_constraints = intel_hsw_event_constraints; - + x86_pmu.pebs_constraints = intel_hsw_pebs_event_constraints; x86_pmu.extra_regs = intel_snb_extra_regs; + x86_pmu.pebs_aliases = intel_pebs_aliases_snb;/* all extra regs are per-cpu when HT is on */ x86_pmu.er_flags |=ERF_HAS_RSP_1 ; x86_pmu.er_flags |=ERF_NO_HT_SHARING ;
arch/x86/kernel/cpu/perf_event_intel_ds.c +29 -0
@@ -427,6 +427,35 @@ struct event_constraint intel_ivb_pebs_event_constraints[] = {EVENT_CONSTRAINT_END }; +struct event_constraint intel_hsw_pebs_event_constraints[] = { +INTEL_UEVENT_CONSTRAINT (0x01c0 ,0x2 ),/* INST_RETIRED.PRECDIST */ +INTEL_UEVENT_CONSTRAINT (0x01c2 ,0xf ),/* UOPS_RETIRED.ALL */ +INTEL_UEVENT_CONSTRAINT (0x02c2 ,0xf ),/* UOPS_RETIRED.RETIRE_SLOTS */ +INTEL_EVENT_CONSTRAINT (0xc4 ,0xf ),/* BR_INST_RETIRED.* */ +INTEL_UEVENT_CONSTRAINT (0x01c5 ,0xf ),/* BR_MISP_RETIRED.CONDITIONAL */ +INTEL_UEVENT_CONSTRAINT (0x04c5 ,0xf ),/* BR_MISP_RETIRED.ALL_BRANCHES */ +INTEL_UEVENT_CONSTRAINT (0x20c5 ,0xf ),/* BR_MISP_RETIRED.NEAR_TAKEN */ +INTEL_EVENT_CONSTRAINT (0xcd ,0x8 ),/* MEM_TRANS_RETIRED.* */ +INTEL_UEVENT_CONSTRAINT (0x11d0 ,0xf ),/* MEM_UOPS_RETIRED.STLB_MISS_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x12d0 ,0xf ),/* MEM_UOPS_RETIRED.STLB_MISS_STORES */ +INTEL_UEVENT_CONSTRAINT (0x21d0 ,0xf ),/* MEM_UOPS_RETIRED.LOCK_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x41d0 ,0xf ),/* MEM_UOPS_RETIRED.SPLIT_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x42d0 ,0xf ),/* MEM_UOPS_RETIRED.SPLIT_STORES */ +INTEL_UEVENT_CONSTRAINT (0x81d0 ,0xf ),/* MEM_UOPS_RETIRED.ALL_LOADS */ +INTEL_UEVENT_CONSTRAINT (0x82d0 ,0xf ),/* MEM_UOPS_RETIRED.ALL_STORES */ +INTEL_UEVENT_CONSTRAINT (0x01d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L1_HIT */ +INTEL_UEVENT_CONSTRAINT (0x02d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L2_HIT */ +INTEL_UEVENT_CONSTRAINT (0x04d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.L3_HIT */ +INTEL_UEVENT_CONSTRAINT (0x40d1 ,0xf ),/* MEM_LOAD_UOPS_RETIRED.HIT_LFB */ +INTEL_UEVENT_CONSTRAINT (0x01d2 ,0xf ),/* MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_MISS */ +INTEL_UEVENT_CONSTRAINT (0x02d2 ,0xf ),/* MEM_LOAD_UOPS_LLC_HIT_RETIRED.XSNP_HIT */ +INTEL_UEVENT_CONSTRAINT (0x02d3 ,0xf ),/* MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM */ +INTEL_UEVENT_CONSTRAINT (0x04c8 ,0xf ),/* HLE_RETIRED.Abort */ +INTEL_UEVENT_CONSTRAINT (0x04c9 ,0xf ),/* RTM_RETIRED.Abort */ + +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
Signature · 1 line
1.7.7.6
PATCH 4/32 perf, x86: Support the TSX intx/intx_cp qualifiers v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Implement the TSX transaction and checkpointed transaction qualifiers for Haswell. This allows e.g. to profile the number of cycles in transactions.
The checkpointed qualifier requires forcing the event to counter 2, implement this with a custom constraint for Haswell.
Also add sysfs format attributes for intx/intx_cp
[Updated from earlier version that used generic attributes, now does raw + sysfs formats]
v2: Moved bad hunk. Forbid some bad combinations.
--- arch/x86/kernel/cpu/perf_event_intel.c | 61 ++++++++++++++++++++++++++++++++ 1 files changed, 61 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +61 -0
@@ -13,6 +13,7 @@#include <linux/slab.h> #include <linux/export.h> +#include <asm/cpufeature.h> #include <asm/hardirq.h> #include <asm/apic.h>
@@ -1597,6 +1598,44 @@ static void core_pmu_enable_all(int added) } } +static int hsw_hw_config (struct perf_event *event) +{ +int ret =intel_pmu_hw_config (event); + +if (ret) +return ret; +if (!boot_cpu_has (X86_FEATURE_RTM ) && !boot_cpu_has (X86_FEATURE_HLE )) +return 0 ; + event->hw.config |= event->attr.config & (HSW_INTX |HSW_INTX_CHECKPOINTED ); + +/* +* INTX/INTX-CP do not play well with PEBS or ANY thread mode. +*/ +if ((event->hw.config & (HSW_INTX |HSW_INTX_CHECKPOINTED )) && + ((event->hw.config &ARCH_PERFMON_EVENTSEL_ANY ) || + event->attr.precise_ip >0 )) +return -EIO ; +return 0 ; +} + +static struct event_constraint counter2_constraint = +EVENT_CONSTRAINT (0 ,0x4 ,0 ); + +static struct event_constraint * +hsw_get_event_constraints (struct cpu_hw_events *cpuc,struct perf_event *event) +{ +struct event_constraint *c =intel_get_event_constraints (cpuc, event); + +/* Handle special quirk on intx_checkpointed only in counter 2 */ +if (event->hw.config &HSW_INTX_CHECKPOINTED ) { +if (c->idxmsk64 & (1U <<2 )) +return &counter2_constraint; +return &emptyconstraint; + } + +return c; +} +PMU_FORMAT_ATTR (event,"config:0-7" );PMU_FORMAT_ATTR (umask,"config:8-15" );PMU_FORMAT_ATTR (edge,"config:18" );
@@ -1604,6 +1643,8 @@ PMU_FORMAT_ATTR(pc, "config:19" );PMU_FORMAT_ATTR (any,"config:21" );/* v3 + */ PMU_FORMAT_ATTR (inv,"config:23" );PMU_FORMAT_ATTR (cmask,"config:24-31" ); +PMU_FORMAT_ATTR (intx,"config:32" ); +PMU_FORMAT_ATTR (intx_cp,"config:33" );static struct attribute *intel_arch_formats_attr[] = { &format_attr_event.attr,
@@ -1761,6 +1802,23 @@ static struct attribute *intel_arch3_formats_attr[] = { NULL, }; +/* Arch3 + TSX support */ +static struct attribute *intel_hsw_formats_attr[]__read_mostly = { + &format_attr_event.attr, + &format_attr_umask.attr, + &format_attr_edge.attr, + &format_attr_pc.attr, + &format_attr_any.attr, + &format_attr_inv.attr, + &format_attr_cmask.attr, + &format_attr_intx.attr, + &format_attr_intx_cp.attr, + + &format_attr_offcore_rsp.attr,/* XXX do NHM/WSM + SNB breakout */ +NULL , +}; + +static __initconst const struct x86_pmu intel_pmu = { .name ="Intel" , .handle_irq = intel_pmu_handle_irq,
@@ -2135,6 +2193,9 @@ __init int intel_pmu_init(void) x86_pmu.er_flags |=ERF_HAS_RSP_1 ; x86_pmu.er_flags |=ERF_NO_HT_SHARING ; + x86_pmu.hw_config = hsw_hw_config; + x86_pmu.get_event_constraints = hsw_get_event_constraints; + x86_pmu.format_attrs = intel_hsw_formats_attr;pr_cont ("Haswell events, " );break ;
Signature · 1 line
1.7.7.6
PATCH 5/32 perf, kvm: Support the intx/intx_cp modifiers in KVM arch perfmon emulation v4
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
This is not arch perfmon, but older CPUs will just ignore it. This makes it possible to do at least some TSX measurements from a KVM guest
Cc: [email protected]
Cc: [email protected]
v2: Various fixes to address review feedback
v3: Ignore the bits when no CPUID. No #GP. Force raw events with TSX bits.
v4: Use reserved bits for #GP
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/pmu.c | 32 ++++++++++++++++++++++++-------- 2 files changed, 25 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -318,6 +318,7 @@ struct kvm_pmu {u64 global_ovf_ctrl;u64 counter_bitmask[2 ];u64 global_ctrl_mask; +u64 reserved_bits;u8 version;struct kvm_pmc gp_counters[INTEL_PMC_MAX_GENERIC ];struct kvm_pmc fixed_counters[INTEL_PMC_MAX_FIXED ];
arch/x86/kvm/pmu.c +24 -8
@@ -160,7 +160,7 @@ static void stop_counter(struct kvm_pmc *pmc)static void reprogram_counter (struct kvm_pmc *pmc,u32 type,unsigned config,bool exclude_user,bool exclude_kernel, -bool intr) +bool intr,bool intx,bool intx_cp) {struct perf_event *event;struct perf_event_attr attr = {
@@ -173,6 +173,10 @@ static void reprogram_counter(struct kvm_pmc *pmc, u32 type, .exclude_kernel = exclude_kernel, .config = config, }; +if (intx) + attr.config |=HSW_INTX ; +if (intx_cp) + attr.config |=HSW_INTX_CHECKPOINTED ; attr.sample_period = (-pmc->counter) &pmc_bitmask (pmc);
@@ -206,7 +210,8 @@ static unsigned find_arch_event(struct kvm_pmu *pmu, u8 event_select,return arch_events[i].event_type; } -static void reprogram_gp_counter (struct kvm_pmc *pmc,u64 eventsel) +static void reprogram_gp_counter (struct kvm_pmu *pmu,struct kvm_pmc *pmc, +u64 eventsel) {unsigned config, type =PERF_TYPE_RAW ;u8 event_select, unit_mask;
@@ -226,7 +231,9 @@ static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)if (!(eventsel & (ARCH_PERFMON_EVENTSEL_EDGE |ARCH_PERFMON_EVENTSEL_INV | -ARCH_PERFMON_EVENTSEL_CMASK ))) { +ARCH_PERFMON_EVENTSEL_CMASK | +HSW_INTX | +HSW_INTX_CHECKPOINTED ))) { config =find_arch_event (&pmc->vcpu->arch.pmu, event_select, unit_mask);if (config !=PERF_COUNT_HW_MAX )
@@ -239,7 +246,9 @@ static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)reprogram_counter (pmc, type, config, !(eventsel &ARCH_PERFMON_EVENTSEL_USR ), !(eventsel &ARCH_PERFMON_EVENTSEL_OS ), - eventsel &ARCH_PERFMON_EVENTSEL_INT ); + eventsel &ARCH_PERFMON_EVENTSEL_INT , + (eventsel &HSW_INTX ), + (eventsel &HSW_INTX_CHECKPOINTED )); }static void reprogram_fixed_counter (struct kvm_pmc *pmc,u8 en_pmi,int idx)
@@ -256,7 +265,7 @@ static void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 en_pmi, int idx) arch_events[fixed_pmc_events[idx]].event_type, !(en &0x2 ),/* exclude user */ !(en &0x1 ),/* exclude kernel */ - pmi); + pmi, false, false); }static inline u8 fixed_en_pmi (u64 ctrl,int idx)
@@ -289,7 +298,7 @@ static void reprogram_idx(struct kvm_pmu *pmu, int idx)return ;if (pmc_is_gp (pmc)) -reprogram_gp_counter (pmc, pmc->eventsel); +reprogram_gp_counter (pmu, pmc, pmc->eventsel);else {int fidx = idx -INTEL_PMC_IDX_FIXED ; reprogram_fixed_counter(pmc,
@@ -400,8 +409,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data) }else if ((pmc =get_gp_pmc (pmu, index,MSR_P6_EVNTSEL0 ))) {if (data == pmc->eventsel)return 0 ; -if (!(data &0xffffffff00200000ull )) { -reprogram_gp_counter (pmc, data); +if (!(data & pmu->reserved_bits)) { +reprogram_gp_counter (pmu, pmc, data);return 0 ; } }
@@ -442,6 +451,7 @@ void kvm_pmu_cpuid_update(struct kvm_vcpu *vcpu) pmu->counter_bitmask[KVM_PMC_GP ] =0 ; pmu->counter_bitmask[KVM_PMC_FIXED ] =0 ; pmu->version =0 ; + pmu->reserved_bits =0xffffffff00200000ull ; entry =kvm_find_cpuid_entry (vcpu,0xa ,0 );if (!entry)
@@ -470,6 +480,12 @@ void kvm_pmu_cpuid_update(struct kvm_vcpu *vcpu) pmu->global_ctrl = ((1 << pmu->nr_arch_gp_counters) -1 ) | (((1ull << pmu->nr_arch_fixed_counters) -1 ) <<INTEL_PMC_IDX_FIXED ); pmu->global_ctrl_mask = ~pmu->global_ctrl; + + entry =kvm_find_cpuid_entry (vcpu,7 ,0 ); +if (entry && + (boot_cpu_has (X86_FEATURE_HLE ) ||boot_cpu_has (X86_FEATURE_RTM )) && + (entry->ebx & (X86_FEATURE_HLE |X86_FEATURE_RTM ))) + pmu->reserved_bits ^=HSW_INTX |HSW_INTX_CHECKPOINTED ; }void kvm_pmu_init (struct kvm_vcpu *vcpu)
Signature · 1 line
1.7.7.6
Re: [PATCH 05/32] perf, kvm: Support the intx/intx_cp modifiers in KVM arch perfmon emulation v4
Quoted 54 lines · Andi Kleen
From: Andi Kleen <[email protected]>
This is not arch perfmon, but older CPUs will just ignore it. This makes it possible to do at least some TSX measurements from a KVM guest
Cc: [email protected]
Cc: [email protected]
v2: Various fixes to address review feedback
v3: Ignore the bits when no CPUID. No #GP. Force raw events with TSX bits.
v4: Use reserved bits for #GPSigned-off-by Andi Kleen <[email protected]>--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/pmu.c | 32 ++++++++++++++++++++++++-------- 2 files changed, 25 insertions(+), 8 deletions(-)arch/x86/include/asm/kvm_host.h +1
-0@@ -318,6 +318,7 @@ struct kvm_pmu {u64 global_ovf_ctrl;u64 counter_bitmask[2 ];u64 global_ctrl_mask; +u64 reserved_bits;u8 version;struct kvm_pmc gp_counters[INTEL_PMC_MAX_GENERIC ];struct kvm_pmc fixed_counters[INTEL_PMC_MAX_FIXED ];arch/x86/kvm/pmu.c +5
-1@@ -160,7 +160,7 @@ static void stop_counter(struct kvm_pmc *pmc)static void reprogram_counter (struct kvm_pmc *pmc,u32 type,unsigned config,bool exclude_user,bool exclude_kernel, -bool intr) +bool intr,bool intx,bool intx_cp) {struct perf_event *event;struct perf_event_attr attr = {@@ -173,6 +173,10 @@ static void reprogram_counter(struct kvm_pmc *pmc, u32 type, .exclude_kernel = exclude_kernel, .config = config, }; +if (intx) + attr.config |=HSW_INTX ; +if (intx_cp) + attr.config |=HSW_INTX_CHECKPOINTED ; attr.sample_period = (-pmc->counter) &pmc_bitmask (pmc);
arch/x86/kvm/pmu.c static unsigned find_arch_event(struct kvm_pmu *pmu, u8 event_select, · line 215
Quoted 7 lines
arch/x86/kvm/pmu.c +2
-1@@ -206,7 +210,8 @@ static unsigned find_arch_event(struct kvm_pmu *pmu, u8 event_select,return arch_events[i].event_type; } -static void reprogram_gp_counter (struct kvm_pmc *pmc,u64 eventsel) +static void reprogram_gp_counter (struct kvm_pmu *pmu,struct kvm_pmc *pmc, +u64 eventsel) {
pmu parameter is no longer used in this patch version. Otherwise looks good.
Quoted 76 lines
arch/x86/kvm/pmu.c +17
-6unsigned config, type =PERF_TYPE_RAW ;u8 event_select, unit_mask;@@ -226,7 +231,9 @@ static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)if (!(eventsel & (ARCH_PERFMON_EVENTSEL_EDGE |ARCH_PERFMON_EVENTSEL_INV | -ARCH_PERFMON_EVENTSEL_CMASK ))) { +ARCH_PERFMON_EVENTSEL_CMASK | +HSW_INTX | +HSW_INTX_CHECKPOINTED ))) { config =find_arch_event (&pmc->vcpu->arch.pmu, event_select, unit_mask);if (config !=PERF_COUNT_HW_MAX )@@ -239,7 +246,9 @@ static void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel)reprogram_counter (pmc, type, config, !(eventsel &ARCH_PERFMON_EVENTSEL_USR ), !(eventsel &ARCH_PERFMON_EVENTSEL_OS ), - eventsel &ARCH_PERFMON_EVENTSEL_INT ); + eventsel &ARCH_PERFMON_EVENTSEL_INT , + (eventsel &HSW_INTX ), + (eventsel &HSW_INTX_CHECKPOINTED )); }static void reprogram_fixed_counter (struct kvm_pmc *pmc,u8 en_pmi,int idx)@@ -256,7 +265,7 @@ static void reprogram_fixed_counter(struct kvm_pmc *pmc, u8 en_pmi, int idx) arch_events[fixed_pmc_events[idx]].event_type, !(en &0x2 ),/* exclude user */ !(en &0x1 ),/* exclude kernel */ - pmi); + pmi, false, false); }static inline u8 fixed_en_pmi (u64 ctrl,int idx)@@ -289,7 +298,7 @@ static void reprogram_idx(struct kvm_pmu *pmu, int idx)return ;if (pmc_is_gp (pmc)) -reprogram_gp_counter (pmc, pmc->eventsel); +reprogram_gp_counter (pmu, pmc, pmc->eventsel);else {int fidx = idx -INTEL_PMC_IDX_FIXED ; reprogram_fixed_counter(pmc,@@ -400,8 +409,8 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, u32 index, u64 data) }else if ((pmc =get_gp_pmc (pmu, index,MSR_P6_EVNTSEL0 ))) {if (data == pmc->eventsel)return 0 ; -if (!(data &0xffffffff00200000ull )) { -reprogram_gp_counter (pmc, data); +if (!(data & pmu->reserved_bits)) { +reprogram_gp_counter (pmu, pmc, data);return 0 ; } }@@ -442,6 +451,7 @@ void kvm_pmu_cpuid_update(struct kvm_vcpu *vcpu) pmu->counter_bitmask[KVM_PMC_GP ] =0 ; pmu->counter_bitmask[KVM_PMC_FIXED ] =0 ; pmu->version =0 ; + pmu->reserved_bits =0xffffffff00200000ull ; entry =kvm_find_cpuid_entry (vcpu,0xa ,0 );if (!entry)@@ -470,6 +480,12 @@ void kvm_pmu_cpuid_update(struct kvm_vcpu *vcpu) pmu->global_ctrl = ((1 << pmu->nr_arch_gp_counters) -1 ) | (((1ull << pmu->nr_arch_fixed_counters) -1 ) <<INTEL_PMC_IDX_FIXED ); pmu->global_ctrl_mask = ~pmu->global_ctrl; + + entry =kvm_find_cpuid_entry (vcpu,7 ,0 ); +if (entry && + (boot_cpu_has (X86_FEATURE_HLE ) ||boot_cpu_has (X86_FEATURE_RTM )) && + (entry->ebx & (X86_FEATURE_HLE |X86_FEATURE_RTM ))) + pmu->reserved_bits ^=HSW_INTX |HSW_INTX_CHECKPOINTED ; }void kvm_pmu_init (struct kvm_vcpu *vcpu)Signature · 1 line
1.7.7.6
--
Gleb.
PATCH 6/32 perf, x86: Support PERF_SAMPLE_ADDR on Haswell
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Haswell supplies the address for every PEBS memory event, so always fill it in when the user requested it. It will be 0 when not useful (no memory access)
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +4 -0
@@ -630,6 +630,10 @@ static void __intel_pmu_pebs_event(struct perf_event *event,else regs.flags &= ~PERF_EFLAGS_EXACT ; +if ((event->attr.sample_type &PERF_SAMPLE_ADDR ) && + x86_pmu.intel_cap.pebs_format >=2 ) + data.addr = ((struct pebs_record_v2 *)pebs)->nhm.dla; +if (has_branch_stack (event)) data.br_stack = &cpuc->lbr_stack;
Signature · 1 line
1.7.7.6
PATCH 7/32 perf, x86: Support Haswell v4 LBR format
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Haswell has two additional LBR from flags for TSX: intx and abort, implemented as a new v4 version of the LBR format.
Handle those in and adjust the sign extension code to still correctly extend. The flags are exported similarly in the LBR record to the existing misprediction flag
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 18 +++++++++++++++--- include/linux/perf_event.h | 7 ++++++- 2 files changed, 21 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +15 -3
@@ -12,6 +12,7 @@ enum {LBR_FORMAT_LIP =0x01 ,LBR_FORMAT_EIP =0x02 ,LBR_FORMAT_EIP_FLAGS =0x03 , + LBR_FORMAT_EIP_FLAGS2 = 0x04, };/*
@@ -56,6 +57,8 @@ enum { LBR_FAR)#define LBR_FROM_FLAG_MISPRED (1ULL << 63) +#define LBR_FROM_FLAG_INTX (1ULL << 62) +#define LBR_FROM_FLAG_ABORT (1ULL << 61)#define for_each_branch_sample_type (x) \ for ((x) = PERF_SAMPLE_BRANCH_USER; \
@@ -270,21 +273,30 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)for (i =0 ; i < x86_pmu.lbr_nr; i++) {unsigned long lbr_idx = (tos - i) & mask; -u64 from, to, mis =0 , pred =0 ; +u64 from, to, mis =0 , pred =0 , intx =0 , abort =0 ;rdmsrl (x86_pmu.lbr_from + lbr_idx, from);rdmsrl (x86_pmu.lbr_to + lbr_idx, to); -if (lbr_format ==LBR_FORMAT_EIP_FLAGS ) { +if (lbr_format ==LBR_FORMAT_EIP_FLAGS || + lbr_format ==LBR_FORMAT_EIP_FLAGS2 ) { mis = !!(from &LBR_FROM_FLAG_MISPRED ); pred = !mis; - from = (u64)((((s64 )from) <<1 ) >>1 ); +if (lbr_format ==LBR_FORMAT_EIP_FLAGS ) + from = (u64)((((s64 )from) <<1 ) >>1 ); +else if (lbr_format ==LBR_FORMAT_EIP_FLAGS2 ) { + intx = !!(from &LBR_FROM_FLAG_INTX ); + abort = !!(from &LBR_FROM_FLAG_ABORT ); + from = (u64)((((s64 )from) <<3 ) >>3 ); + } } cpuc->lbr_entries[i].from = from; cpuc->lbr_entries[i].to = to; cpuc->lbr_entries[i].mispred = mis; cpuc->lbr_entries[i].predicted = pred; + cpuc->lbr_entries[i].intx = intx; + cpuc->lbr_entries[i].abort = abort; cpuc->lbr_entries[i].reserved =0 ; } cpuc->lbr_stack.nr = i;
include/linux/perf_event.h +6 -1
@@ -74,13 +74,18 @@ struct perf_raw_record {* * support for mispred, predicted is optional. In case it * is not supported mispred = predicted = 0. +* +* intx: running in a hardware transaction +* abort: aborting a hardware transaction */ struct perf_branch_entry {__u64 from;__u64 to;__u64 mispred:1 ,/* target mispredicted */ predicted:1 ,/* target predicted */ - reserved:62 ; + intx:1 ,/* in transaction */ + abort:1 ,/* transaction abort */ + reserved:60 ; };/*
Signature · 1 line
1.7.7.6
PATCH 8/32 perf, x86: Disable LBR recording for unknown LBR_FMT
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
When the LBR format is unknown disable LBR recording. This prevents crashes when the LBR address is misdecoded and mis-sign extended.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +2 -1
@@ -13,6 +13,7 @@ enum {LBR_FORMAT_EIP =0x02 ,LBR_FORMAT_EIP_FLAGS =0x03 ,LBR_FORMAT_EIP_FLAGS2 =0x04 , + LBR_FORMAT_MAX_KNOWN = LBR_FORMAT_EIP_FLAGS2, };/*
@@ -392,7 +393,7 @@ int intel_pmu_setup_lbr_filter(struct perf_event *event)/* * no LBR on this PMU */ -if (!x86_pmu.lbr_nr) +if (!x86_pmu.lbr_nr || x86_pmu.intel_cap.lbr_format >LBR_FORMAT_MAX_KNOWN )return -EOPNOTSUPP ;/*
Signature · 1 line
1.7.7.6
PATCH 9/32 perf, x86: Support LBR filtering by INTX/NOTX/ABORT v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add LBR filtering for branch in transaction, branch not in transaction or transaction abort. This is exposed as new sample types.
v2: Rename ABORT to ABORTTX
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 31 +++++++++++++++++++++++++-- include/uapi/linux/perf_event.h | 5 +++- 2 files changed, 32 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +28 -3
@@ -85,9 +85,13 @@ enum {X86_BR_JMP =1 <<9 ,/* jump */ X86_BR_IRQ =1 <<10 ,/* hw interrupt or trap or fault */ X86_BR_IND_CALL =1 <<11 ,/* indirect calls */ +X86_BR_ABORT =1 <<12 ,/* transaction abort */ +X86_BR_INTX =1 <<13 ,/* in transaction */ + X86_BR_NOTX = 1 << 14,/* not in transaction */ };#define X86_BR_PLM (X86_BR_USER | X86_BR_KERNEL) +#define X86_BR_ANYTX (X86_BR_NOTX | X86_BR_INTX)#define X86_BR_ANY \ (X86_BR_CALL |\
@@ -99,6 +103,7 @@ enum {X86_BR_JCC |\X86_BR_JMP |\X86_BR_IRQ |\ +X86_BR_ABORT |\ X86_BR_IND_CALL) #define X86_BR_ALL (X86_BR_PLM | X86_BR_ANY)
@@ -347,6 +352,16 @@ static void intel_pmu_setup_sw_lbr_filter(struct perf_event *event)if (br_type &PERF_SAMPLE_BRANCH_IND_CALL ) mask |=X86_BR_IND_CALL ; + +if (br_type &PERF_SAMPLE_BRANCH_ABORTTX ) + mask |=X86_BR_ABORT ; + +if (br_type &PERF_SAMPLE_BRANCH_INTX ) + mask |=X86_BR_INTX ; + +if (br_type &PERF_SAMPLE_BRANCH_NOTX ) + mask |=X86_BR_NOTX ; +/* * stash actual user request into reg, it may * be used by fixup code for some CPU
@@ -393,7 +408,8 @@ int intel_pmu_setup_lbr_filter(struct perf_event *event)/* * no LBR on this PMU */ -if (!x86_pmu.lbr_nr || x86_pmu.intel_cap.lbr_format >LBR_FORMAT_MAX_KNOWN ) +if (!x86_pmu.lbr_nr || + x86_pmu.intel_cap.lbr_format >LBR_FORMAT_MAX_KNOWN )return -EOPNOTSUPP ;/*
@@ -421,7 +437,7 @@ int intel_pmu_setup_lbr_filter(struct perf_event *event)* decoded (e.g., text page not present), then X86_BR_NONE is * returned. */ -static int branch_type (unsigned long from,unsigned long to) +static int branch_type (unsigned long from,unsigned long to,int abort) {struct insn insn;void *addr;
@@ -441,6 +457,9 @@ static int branch_type(unsigned long from, unsigned long to)if (from ==0 || to ==0 )return X86_BR_NONE ; +if (abort) +return X86_BR_ABORT | to_plm; +if (from_plm ==X86_BR_USER ) {/* * can happen if measuring at the user level only
@@ -577,7 +596,13 @@ intel_pmu_lbr_filter(struct cpu_hw_events *cpuc) from = cpuc->lbr_entries[i].from; to = cpuc->lbr_entries[i].to; - type =branch_type (from, to); + type =branch_type (from, to, cpuc->lbr_entries[i].abort); +if (type !=X86_BR_NONE && (br_sel &X86_BR_ANYTX )) { +if (cpuc->lbr_entries[i].intx) + type |=X86_BR_INTX ; +else + type |=X86_BR_NOTX ; + }/* if type does not correspond, then discard */ if (type ==X86_BR_NONE || (br_sel & type) != type) {
include/uapi/linux/perf_event.h +4 -1
@@ -155,8 +155,11 @@ enum perf_branch_sample_type {PERF_SAMPLE_BRANCH_ANY_CALL =1U <<4 ,/* any call branch */ PERF_SAMPLE_BRANCH_ANY_RETURN =1U <<5 ,/* any return branch */ PERF_SAMPLE_BRANCH_IND_CALL =1U <<6 ,/* indirect calls */ +PERF_SAMPLE_BRANCH_ABORTTX =1U <<7 ,/* transaction aborts */ +PERF_SAMPLE_BRANCH_INTX =1U <<8 ,/* in transaction (flag) */ +PERF_SAMPLE_BRANCH_NOTX =1U <<9 ,/* not in transaction (flag) */ - PERF_SAMPLE_BRANCH_MAX = 1U << 7, /* non-ABI */ + PERF_SAMPLE_BRANCH_MAX = 1U << 10, /* non-ABI */ };#define PERF_SAMPLE_BRANCH_PLM_ALL \
Signature · 1 line
1.7.7.6
PATCH 10/32 perf, tools: Add abort_tx,no_tx,in_tx branch filter options to perf record -j v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Make perf record -j aware of the new in_tx,no_tx,abort_tx branch qualifiers.
v2: ABORT -> ABORTTX
v3: Add more _
--- tools/perf/Documentation/perf-record.txt | 3 +++ tools/perf/builtin-record.c | 3 +++ 2 files changed, 6 insertions(+), 0 deletions(-)
tools/perf/Documentation/perf-record.txt +3 -0
@@ -172,6 +172,9 @@ following filters are defined: - u: only when the branch target is at the user level - k: only when the branch target is in the kernel - hv: only when the target is at the hypervisor level + - in_tx: only when the target is in a hardware transaction + - no_tx: only when the target is not in a hardware transaction + - abort_tx: only when the target is a hardware transaction abort + The option requires at least one branch type among any, any_call, any_ret, ind_call.
tools/perf/builtin-record.c +3 -0
@@ -763,6 +763,9 @@ static const struct branch_mode branch_modes[] = {BRANCH_OPT ("any_call" ,PERF_SAMPLE_BRANCH_ANY_CALL ),BRANCH_OPT ("any_ret" ,PERF_SAMPLE_BRANCH_ANY_RETURN ),BRANCH_OPT ("ind_call" ,PERF_SAMPLE_BRANCH_IND_CALL ), +BRANCH_OPT ("abort_tx" ,PERF_SAMPLE_BRANCH_ABORTTX ), +BRANCH_OPT ("in_tx" ,PERF_SAMPLE_BRANCH_INTX ), +BRANCH_OPT ("no_tx" ,PERF_SAMPLE_BRANCH_NOTX ), BRANCH_END };
Signature · 1 line
1.7.7.6
PATCH 11/32 perf, tools: Support sorting by intx, abort branch flags
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Extend the perf branch sorting code to support sorting by intx or abort qualifiers. Also print out those qualifiers.
--- tools/perf/builtin-report.c | 3 +- tools/perf/builtin-top.c | 4 ++- tools/perf/perf.h | 4 ++- tools/perf/util/hist.h | 2 + tools/perf/util/sort.c | 55 +++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/sort.h | 2 + 6 files changed, 67 insertions(+), 3 deletions(-)
tools/perf/builtin-report.c +2 -1
@@ -595,7 +595,8 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused) "Use the stdio interface"),OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" ,"sort by key(s): pid, comm, dso, symbol, parent, dso_to," -" dso_from, symbol_to, symbol_from, mispredict" ), +" dso_from, symbol_to, symbol_from, mispredict, srcline," +" abort, intx" ),OPT_BOOLEAN (0 ,"showcpuutilization" , &symbol_conf.show_cpu_utilization,"Show sample percentage for different cpu modes" ), OPT_STRING('p', "parent", &parent_pattern, "regex",
tools/perf/builtin-top.c +3 -1
@@ -1226,7 +1226,9 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show counter open errors, etc)" ),OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" , -"sort by key(s): pid, comm, dso, symbol, parent" ), +"sort by key(s): pid, comm, dso, symbol, parent, dso_to," +" dso_from, symbol_to, symbol_from, mispredict, srcline," +" abort, intx" ),OPT_BOOLEAN ('n' ,"show-nr-samples" , &symbol_conf.show_nr_samples,"Show a column with the number of samples" ), OPT_CALLBACK_DEFAULT('G', "call-graph", &top, "output_type,min_percent, call_order",
tools/perf/perf.h +3 -1
@@ -194,7 +194,9 @@ struct ip_callchain {struct branch_flags {u64 mispred:1 ;u64 predicted:1 ; -u64 reserved:62 ; +u64 intx:1 ; +u64 abort:1 ; +u64 reserved:60 ; };struct branch_entry {
tools/perf/util/hist.h +2 -0
@@ -43,6 +43,8 @@ enum hist_column {HISTC_PARENT ,HISTC_CPU ,HISTC_MISPREDICT , +HISTC_INTX , +HISTC_ABORT ,HISTC_SYMBOL_FROM ,HISTC_SYMBOL_TO , HISTC_DSO_FROM,
tools/perf/util/sort.c +55 -0
73 lines not on this page · raw message
tools/perf/util/sort.h +2 -0
@@ -118,6 +118,8 @@ enum sort_type {SORT_SYM_TO ,SORT_MISPREDICT ,SORT_SRCLINE , +SORT_ABORT , + SORT_INTX, };/*
Signature · 1 line
1.7.7.6
PATCH 12/32 perf, x86: Support full width counting
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Recent Intel CPUs have a new alternative MSR range for perfctrs that allows writing the full counter width. Enable this range if the hardware reports it using a new capability bit. This lowers overhead of perf stat slightly because it has to do less interrupts to accumulate the counter value. On Haswell it also avoids some problems with TSX aborting when the end of the counter range is reached.
--- arch/x86/include/asm/msr-index.h | 3 +++ arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel.c | 6 ++++++ 3 files changed, 10 insertions(+), 0 deletions(-)
arch/x86/include/asm/msr-index.h +3 -0
@@ -126,6 +126,9 @@#define MSR_KNC_EVNTSEL0 0x00000028#define MSR_KNC_EVNTSEL1 0x00000029 +/* Alternative perfctr range with full access. */ +#define MSR_IA32_PMC0 0x000004c1 +/* AMD64 MSRs. Not complete. See the architecture manual for a more complete list. */
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -278,6 +278,7 @@ union perf_capabilities { u64 pebs_arch_reg:1; u64 pebs_format:4; u64 smm_freeze:1; + u64 fw_write:1; };u64 capabilities; };
arch/x86/kernel/cpu/perf_event_intel.c +6 -0
@@ -2247,5 +2247,11 @@ __init int intel_pmu_init(void) } } +/* Support full width counters using alternative MSR range */ +if (x86_pmu.intel_cap.fw_write) { + x86_pmu.max_period = x86_pmu.cntval_mask; + x86_pmu.perfctr =MSR_IA32_PMC0 ; + } +return 0 ; }
Signature · 1 line
1.7.7.6
PATCH 13/32 perf, x86: Avoid checkpointed counters causing excessive TSX aborts v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
With checkpointed counters there can be a situation where the counter is overflowing, aborts the transaction, is set back to a non overflowing checkpoint, causes interupt. The interrupt doesn't see the overflow because it has been checkpointed. This is then a spurious PMI, typically with a ugly NMI message. It can also lead to excessive aborts.
Avoid this problem by:
- Using the full counter width for counting counters (previous patch)
- Forbid sampling for checkpointed counters. It's not too useful anyways, checkpointing is mainly for counting.
- On a PMI always set back checkpointed counters to zero.
v2: Add unlikely. Add comment
v3: Allow large sampling periods with CP for KVM
--- arch/x86/kernel/cpu/perf_event_intel.c | 34 ++++++++++++++++++++++++++++++++ 1 files changed, 34 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +34 -0
@@ -1079,6 +1079,17 @@ static void intel_pmu_enable_event(struct perf_event *event)int intel_pmu_save_and_restart (struct perf_event *event) {x86_perf_event_update (event); +/* +* For a checkpointed counter always reset back to 0. This +* avoids a situation where the counter overflows, aborts the +* transaction and is then set back to shortly before the +* overflow, and overflows and aborts again. +*/ +if (unlikely (event->hw.config &HSW_INTX_CHECKPOINTED )) { +/* No race with NMIs because the counter should not be armed */ +wrmsrl (event->hw.event_base,0 ); +local64_set (&event->hw.prev_count,0 ); + }return x86_perf_event_set_period (event); }
@@ -1162,6 +1173,15 @@ again: x86_pmu.drain_pebs (regs); } +/* +* To avoid spurious interrupts with perf stat always reset checkpointed +* counters. +* +* XXX move somewhere else. +*/ +if (cpuc->events[2 ] && (cpuc->events[2 ]->hw.config &HSW_INTX_CHECKPOINTED )) + status |= (1ULL <<2 ); +for_each_set_bit (bit, (unsigned long *)&status,X86_PMC_IDX_MAX ) {struct perf_event *event = cpuc->events[bit];
@@ -1615,6 +1635,20 @@ static int hsw_hw_config(struct perf_event *event) ((event->hw.config &ARCH_PERFMON_EVENTSEL_ANY ) || event->attr.precise_ip > 0)) return -EIO; +if (event->hw.config &HSW_INTX_CHECKPOINTED ) { +/* +* Sampling of checkpointed events can cause situations where +* the CPU constantly aborts because of a overflow, which is +* then checkpointed back and ignored. Forbid checkpointing +* for sampling. +* +* But still allow a long sampling period, so that perf stat +* from KVM works. +*/ +if (event->attr.sample_period >0 && + event->attr.sample_period <0x7fffffff ) +return -EIO ; + }return 0 ; }
Signature · 1 line
1.7.7.6
PATCH 14/32 perf, core: Add a concept of a weightened sample
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
For some events it's useful to weight sample with a hardware provided number. This expresses how expensive the action the sample represent was. This allows the profiler to scale the samples to be more informative to the programmer.
There is already the period which is used similarly, but it means something different, so I chose to not overload it. Instead a new sample type for WEIGHT is added.
Can be used for multiple things. Initially it is used for TSX abort costs and profiling by memory latencies (so to make expensive load appear higher up in the histograms) The concept is quite generic and can be extended to many other kinds of events or architectures, as long as the hardware provides suitable auxillary values. In principle it could be also used for software tracpoints.
This adds the generic glue. A new optional sample format for a 64bit weight value.
--- include/linux/perf_event.h | 2 ++ include/uapi/linux/perf_event.h | 8 ++++++-- kernel/events/core.c | 6 ++++++ 3 files changed, 14 insertions(+), 2 deletions(-)
include/linux/perf_event.h +2 -0
@@ -588,6 +588,7 @@ struct perf_sample_data {struct perf_branch_stack *br_stack;struct perf_regs_user regs_user;u64 stack_user_size; +u64 weight; }; static inline void perf_sample_data_init(struct perf_sample_data *data,
@@ -601,6 +602,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, data->regs_user.abi =PERF_SAMPLE_REGS_ABI_NONE ; data->regs_user.regs =NULL ; data->stack_user_size =0 ; + data->weight =0 ; } extern void perf_output_sample(struct perf_output_handle *handle,
include/uapi/linux/perf_event.h +6 -2
@@ -132,8 +132,10 @@ enum perf_event_sample_format {PERF_SAMPLE_BRANCH_STACK =1U <<11 ,PERF_SAMPLE_REGS_USER =1U <<12 ,PERF_SAMPLE_STACK_USER =1U <<13 , +PERF_SAMPLE_WEIGHT =1U <<14 , + + PERF_SAMPLE_MAX = 1U << 15, /* non-ABI */ - PERF_SAMPLE_MAX = 1U << 14, /* non-ABI */ };/*
@@ -201,8 +203,9 @@ enum perf_event_read_format {PERF_FORMAT_TOTAL_TIME_RUNNING =1U <<1 ,PERF_FORMAT_ID =1U <<2 ,PERF_FORMAT_GROUP =1U <<3 , +PERF_FORMAT_WEIGHT =1U <<4 , - PERF_FORMAT_MAX = 1U << 4, /* non-ABI */ + PERF_FORMAT_MAX = 1U << 5, /* non-ABI */ };#define PERF_ATTR_SIZE_VER0 64/* sizeof first published struct */
@@ -562,6 +565,7 @@ enum perf_event_type {* { u64 stream_id;} && PERF_SAMPLE_STREAM_ID * { u32 cpu, res; } && PERF_SAMPLE_CPU * { u64 period; } && PERF_SAMPLE_PERIOD +* { u64 weight; } && PERF_SAMPLE_WEIGHT * * { struct read_format values; } && PERF_SAMPLE_READ *
kernel/events/core.c +6 -0
@@ -952,6 +952,9 @@ static void perf_event__header_size(struct perf_event *event)if (sample_type &PERF_SAMPLE_PERIOD ) size +=sizeof (data->period); +if (sample_type &PERF_SAMPLE_WEIGHT ) + size +=sizeof (data->weight); +if (sample_type &PERF_SAMPLE_READ ) size += event->read_size;
@@ -4080,6 +4083,9 @@ void perf_output_sample(struct perf_output_handle *handle,if (sample_type &PERF_SAMPLE_PERIOD )perf_output_put (handle, data->period); +if (sample_type &PERF_SAMPLE_WEIGHT ) +perf_output_put (handle, data->weight); +if (sample_type &PERF_SAMPLE_READ )perf_output_read (handle, event);
Signature · 1 line
1.7.7.6
PATCH 15/32 perf, x86: Support weight samples for PEBS
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
When a weighted sample is requested, first try to report the TSX abort cost on Haswell. If that is not available report the memory latency. This allows profiling both by abort cost and by memory latencies.
Memory latencies requires enabling a different PEBS mode (LL). When both address and weight is requested address wins.
The LL mode only works for memory related PEBS events, so add a separate event constraint table for those.
I only did this for Haswell for now, but it could be added for several other Intel CPUs too by just adding the right table for them.
--- arch/x86/kernel/cpu/perf_event.h | 4 ++ arch/x86/kernel/cpu/perf_event_intel.c | 4 ++ arch/x86/kernel/cpu/perf_event_intel_ds.c | 47 +++++++++++++++++++++++++++- 3 files changed, 53 insertions(+), 2 deletions(-)
arch/x86/kernel/cpu/perf_event.h +4 -0
@@ -168,6 +168,7 @@ struct cpu_hw_events {u64 perf_ctr_virt_mask;void *kfree_on_online; +u8 *memory_latency_events; };#define __EVENT_CONSTRAINT (c, n, m, w, o) {\
@@ -390,6 +391,7 @@ struct x86_pmu {struct event_constraint *pebs_constraints;void (*pebs_aliases)(struct perf_event *event);int max_pebs_events; +struct event_constraint *memory_lat_events;/* * Intel LBR
@@ -599,6 +601,8 @@ extern struct event_constraint intel_ivb_pebs_event_constraints[];extern struct event_constraint intel_hsw_pebs_event_constraints[]; +extern struct event_constraint intel_hsw_memory_latency_events[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_enable (struct perf_event *event);
arch/x86/kernel/cpu/perf_event_intel.c +4 -0
@@ -1624,6 +1624,9 @@ static int hsw_hw_config(struct perf_event *event)if (ret)return ret; +/* PEBS cannot capture both */ +if (event->attr.sample_type &PERF_SAMPLE_ADDR ) + event->attr.sample_type &= ~PERF_SAMPLE_WEIGHT ;if (!boot_cpu_has (X86_FEATURE_RTM ) && !boot_cpu_has (X86_FEATURE_HLE ))return 0 ; event->hw.config |= event->attr.config & (HSW_INTX |HSW_INTX_CHECKPOINTED );
@@ -2230,6 +2233,7 @@ __init int intel_pmu_init(void) x86_pmu.hw_config = hsw_hw_config; x86_pmu.get_event_constraints = hsw_get_event_constraints; x86_pmu.format_attrs = intel_hsw_formats_attr; + x86_pmu.memory_lat_events = intel_hsw_memory_latency_events;pr_cont ("Haswell events, " );break ;
arch/x86/kernel/cpu/perf_event_intel_ds.c +45 -2
77 lines not on this page · raw message
Signature · 1 line
1.7.7.6
PATCH 16/32 perf, tools: Add support for weight v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
perf record has a new option -W that enables weightened sampling.
Add sorting support in top/report for the average weight per sample and the total weight sum. This allows to both compare relative cost per event and the total cost over the measurement period.
Add the necessary glue to perf report, record and the library.
v2: Merge with new hist refactoring.
v3: Fix manpage. Remove value check.
Rename global_weight to weight and weight to local_weight.
--- tools/perf/Documentation/perf-record.txt | 6 +++ tools/perf/builtin-annotate.c | 2 +- tools/perf/builtin-diff.c | 7 ++-- tools/perf/builtin-record.c | 2 + tools/perf/builtin-report.c | 7 ++-- tools/perf/builtin-top.c | 5 ++- tools/perf/perf.h | 1 + tools/perf/util/event.h | 1 + tools/perf/util/evsel.c | 10 ++++++ tools/perf/util/hist.c | 22 +++++++++---- tools/perf/util/hist.h | 8 +++- tools/perf/util/session.c | 3 ++ tools/perf/util/sort.c | 51 +++++++++++++++++++++++++++++- tools/perf/util/sort.h | 3 ++ 14 files changed, 109 insertions(+), 19 deletions(-)
tools/perf/Documentation/perf-record.txt +6 -0
@@ -185,6 +185,12 @@ is enabled for all the sampling events. The sampled branch type is the same for The various filters must be specified as a comma separated list: --branch-filter any_ret,u,k Note that this feature may not be available on all processors. +-W:: +--weight:: +Enable weightened sampling. An additional weight is recorded per sample and can be +displayed with the weight and local_weight sort keys. This currently works for TSX +abort events and some memory events in precise mode on modern Intel CPUs. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-annotate.c +1 -1
@@ -62,7 +62,7 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,return 0 ; } - he =__hists__add_entry (&evsel->hists, al,NULL ,1 ); + he =__hists__add_entry (&evsel->hists, al,NULL ,1 ,1 );if (he ==NULL )return -ENOMEM ;
tools/perf/builtin-diff.c +4 -3
@@ -248,9 +248,10 @@ int perf_diff__formula(char *buf, size_t size, struct hist_entry *he) }static int hists__add_entry (struct hists *self, -struct addr_location *al,u64 period) +struct addr_location *al,u64 period, +u64 weight) { -if (__hists__add_entry (self, al,NULL , period) !=NULL ) +if (__hists__add_entry (self, al,NULL , period, weight) !=NULL )return 0 ;return -ENOMEM ; }
@@ -272,7 +273,7 @@ static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,if (al.filtered)return 0 ; -if (hists__add_entry (&evsel->hists, &al, sample->period)) { +if (hists__add_entry (&evsel->hists, &al, sample->period, sample->weight)) {pr_warning ("problem incrementing symbol period, skipping event\n" );return -1 ; }
tools/perf/builtin-record.c +2 -0
@@ -1044,6 +1044,8 @@ const struct option record_options[] = {OPT_CALLBACK ('j' ,"branch-filter" , &record.opts.branch_stack,"branch filter mask" ,"branch stack filter modes" , parse_branch_stack), +OPT_BOOLEAN ('W' ,"weight" , &record.opts.sample_weight, +"sample by weight (on special events only)" ), OPT_END() };
tools/perf/builtin-report.c +4 -3
@@ -88,7 +88,7 @@ static int perf_report__add_branch_hist_entry(struct perf_tool *tool,* and not events sampled. Thus we use a pseudo period of 1. */ he =__hists__add_branch_entry (&evsel->hists, al, parent, - &bi[i],1 ); + &bi[i],1 ,1 );if (he) {struct annotation *notes; err = -ENOMEM ;
@@ -146,7 +146,8 @@ static int perf_evsel__add_hist_entry(struct perf_evsel *evsel,return err; } - he =__hists__add_entry (&evsel->hists, al, parent, sample->period); + he =__hists__add_entry (&evsel->hists, al, parent, sample->period, + sample->weight);if (he ==NULL )return -ENOMEM ;
@@ -596,7 +597,7 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" ,"sort by key(s): pid, comm, dso, symbol, parent, dso_to," " dso_from, symbol_to, symbol_from, mispredict, srcline," -" abort, intx" ), +" abort, intx, weight, local_weight" ),OPT_BOOLEAN (0 ,"showcpuutilization" , &symbol_conf.show_cpu_utilization,"Show sample percentage for different cpu modes" ), OPT_STRING('p', "parent", &parent_pattern, "regex",
tools/perf/builtin-top.c +3 -2
@@ -271,7 +271,8 @@ static struct hist_entry *perf_evsel__add_hist_entry(struct perf_evsel *evsel, {struct hist_entry *he; - he =__hists__add_entry (&evsel->hists, al,NULL , sample->period); + he =__hists__add_entry (&evsel->hists, al,NULL , sample->period, + sample->weight);if (he ==NULL )return NULL ;
@@ -1228,7 +1229,7 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" ,"sort by key(s): pid, comm, dso, symbol, parent, dso_to," " dso_from, symbol_to, symbol_from, mispredict, srcline," -" abort, intx" ), +" abort, intx, weight, local_weight" ),OPT_BOOLEAN ('n' ,"show-nr-samples" , &symbol_conf.show_nr_samples,"Show a column with the number of samples" ), OPT_CALLBACK_DEFAULT('G', "call-graph", &top, "output_type,min_percent, call_order",
tools/perf/perf.h +1 -0
@@ -235,6 +235,7 @@ struct perf_record_opts {bool pipe_output;bool raw_samples;bool sample_address; +bool sample_weight;bool sample_time;bool sample_id_all_missing;bool exclude_guest_missing;
tools/perf/util/event.h +1 -0
@@ -88,6 +88,7 @@ struct perf_sample {u64 id;u64 stream_id;u64 period; +u64 weight;u32 cpu;u32 raw_size;void *raw_data;
tools/perf/util/evsel.c +10 -0
@@ -445,6 +445,9 @@ void perf_evsel__config(struct perf_evsel *evsel, struct perf_record_opts *opts, attr->mmap_data = track; } +if (opts->sample_weight) + attr->sample_type |=PERF_SAMPLE_WEIGHT ; +if (opts->call_graph) { attr->sample_type |=PERF_SAMPLE_CALLCHAIN ;
@@ -870,6 +873,7 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event, data->cpu = data->pid = data->tid =-1 ; data->stream_id = data->id = data->time =-1ULL ; data->period =1 ; + data->weight =0 ;if (event->header.type !=PERF_RECORD_SAMPLE ) {if (!evsel->attr.sample_id_all)
@@ -941,6 +945,12 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event, array++; } + data->weight =0 ; +if (type &PERF_SAMPLE_WEIGHT ) { + data->weight = *array; + array++; + } +if (type &PERF_SAMPLE_READ ) {fprintf (stderr,"PERF_SAMPLE_READ is unsupported for now\n" );return -1 ;
tools/perf/util/hist.c +15 -7
@@ -151,9 +151,11 @@ static void hist_entry__add_cpumode_period(struct hist_entry *he, } } -static void he_stat__add_period (struct he_stat *he_stat,u64 period) +static void he_stat__add_period (struct he_stat *he_stat,u64 period, +u64 weight) { he_stat->period += period; + he_stat->weight += weight; he_stat->nr_events +=1 ; }
@@ -165,12 +167,14 @@ static void he_stat__add_stat(struct he_stat *dest, struct he_stat *src) dest->period_guest_sys += src->period_guest_sys; dest->period_guest_us += src->period_guest_us; dest->nr_events += src->nr_events; + dest->weight += src->weight; }static void hist_entry__decay (struct hist_entry *he) { he->stat.period = (he->stat.period *7 ) /8 ; he->stat.nr_events = (he->stat.nr_events *7 ) /8 ; +/* XXX need decay for weight too? */ }static bool hists__decay_entry (struct hists *hists,struct hist_entry *he)
@@ -268,7 +272,8 @@ static u8 symbol__parent_filter(const struct symbol *parent)static struct hist_entry *add_hist_entry (struct hists *hists,struct hist_entry *entry,struct addr_location *al, -u64 period) +u64 period, +u64 weight) {struct rb_node **p;struct rb_node *parent =NULL ;
@@ -286,7 +291,7 @@ static struct hist_entry *add_hist_entry(struct hists *hists, cmp =hist_entry__cmp (entry, he);if (!cmp) { -he_stat__add_period (&he->stat, period); +he_stat__add_period (&he->stat, period, weight);/* If the map of an existing hist_entry has * become out-of-date due to an exec() or
@@ -314,6 +319,7 @@ static struct hist_entry *add_hist_entry(struct hists *hists,rb_link_node (&he->rb_node_in, parent, p);rb_insert_color (&he->rb_node_in, hists->entries_in); + he->stat.weight += weight; out:hist_entry__add_cpumode_period (he, al->cpumode, period); out_unlock:
@@ -325,7 +331,8 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self, struct addr_location *al, struct symbol *sym_parent, struct branch_info *bi, - u64 period) + u64 period, + u64 weight) { struct hist_entry entry = { .thread = al->thread,
@@ -346,12 +353,13 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self, .hists = self, }; -return add_hist_entry (self, &entry, al, period); +return add_hist_entry (self, &entry, al, period, weight); }struct hist_entry *__hists__add_entry (struct hists *self,struct addr_location *al, -struct symbol *sym_parent,u64 period) +struct symbol *sym_parent,u64 period, +u64 weight) {struct hist_entry entry = { .thread = al->thread,
@@ -371,7 +379,7 @@ struct hist_entry *__hists__add_entry(struct hists *self, .hists = self, }; -return add_hist_entry (self, &entry, al, period); +return add_hist_entry (self, &entry, al, period, weight); }int64_t
tools/perf/util/hist.h +6 -2
@@ -50,6 +50,8 @@ enum hist_column {HISTC_DSO_FROM ,HISTC_DSO_TO ,HISTC_SRCLINE , +HISTC_WEIGHT , +HISTC_GLOBAL_WEIGHT , HISTC_NR_COLS, /* Last entry */ };
@@ -74,7 +76,8 @@ struct hists {struct hist_entry *__hists__add_entry (struct hists *self,struct addr_location *al, -struct symbol *parent,u64 period); +struct symbol *parent,u64 period, +u64 weight);int64_t hist_entry__cmp (struct hist_entry *left,struct hist_entry *right);int64_t hist_entry__collapse (struct hist_entry *left,struct hist_entry *right); int hist_entry__sort_snprintf(struct hist_entry *self, char *bf, size_t size,
@@ -85,7 +88,8 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self,struct addr_location *al, struct symbol *sym_parent, struct branch_info *bi, - u64 period); + u64 period, + u64 weight);void hists__output_resort (struct hists *self);void hists__output_resort_threaded (struct hists *hists);
tools/perf/util/session.c +3 -0
@@ -1006,6 +1006,9 @@ static void dump_sample(struct perf_evsel *evsel, union perf_event *event,if (sample_type &PERF_SAMPLE_STACK_USER )stack_user__printf (&sample->user_stack); + +if (sample_type &PERF_SAMPLE_WEIGHT ) +printf ("... weight: %" PRIu64"\n" , sample->weight); } static struct machine *
tools/perf/util/sort.c +50 -1
69 lines not on this page · raw message
tools/perf/util/sort.h +3 -0
@@ -49,6 +49,7 @@ struct he_stat {u64 period_us;u64 period_guest_sys;u64 period_guest_us; +u64 weight;u32 nr_events; };
@@ -120,6 +121,8 @@ enum sort_type {SORT_SRCLINE ,SORT_ABORT ,SORT_INTX , +SORT_LOCAL_WEIGHT , + SORT_GLOBAL_WEIGHT, };/*
Signature · 1 line
1.7.7.6
PATCH 17/32 perf, tools: Handle XBEGIN like a jump
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
So that the browser still shows the abort label
--- tools/perf/util/annotate.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
tools/perf/util/annotate.c +2 -0
@@ -401,6 +401,8 @@ static struct ins instructions[] = { { .name = "testb", .ops = &mov_ops, }, { .name = "testl", .ops = &mov_ops, }, { .name = "xadd", .ops = &mov_ops, }, + { .name = "xbeginl", .ops = &jump_ops, }, + { .name = "xbeginq", .ops = &jump_ops, }, }; static int ins__cmp(const void *name, const void *insp)
Signature · 1 line
1.7.7.6
PATCH 18/32 perf, x86: Support for printing PMU state on spurious PMIs v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
I had some problems with spurious PMIs, so print the PMU state on a spurious one. This will not interact well with other NMI users. Disabled by default, has to be explicitely enabled through sysfs.
Optional, but useful for debugging.
v2: Move to /sys/devices/cpu
v3: Print in more cases
--- arch/x86/kernel/cpu/perf_event.c | 3 +++ arch/x86/kernel/cpu/perf_event.h | 2 ++ arch/x86/kernel/cpu/perf_event_intel.c | 11 ++++++----- 3 files changed, 11 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/perf_event.c +3 -0
@@ -38,6 +38,7 @@#include "perf_event.h" struct x86_pmu x86_pmu__read_mostly ; +int print_spurious_pmi__read_mostly ; DEFINE_PER_CPU(struct cpu_hw_events, cpu_hw_events) = { .enabled =1 ,
@@ -1758,9 +1759,11 @@ static ssize_t set_attr_rdpmc(struct device *cdev, } static DEVICE_ATTR(rdpmc, S_IRUSR | S_IWUSR, get_attr_rdpmc, set_attr_rdpmc); +static DEVICE_INT_ATTR(print_spurious_pmi, 0644, print_spurious_pmi);static struct attribute *x86_pmu_attrs[] = { &dev_attr_rdpmc.attr, + &dev_attr_print_spurious_pmi.attr.attr,NULL , };
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -664,3 +664,5 @@ static inline struct intel_shared_regs *allocate_shared_regs(int cpu) }#endif /* CONFIG_CPU_SUP_INTEL */ + +extern int print_spurious_pmi;
arch/x86/kernel/cpu/perf_event_intel.c +6 -5
@@ -1146,11 +1146,8 @@ static int intel_pmu_handle_irq(struct pt_regs *regs)intel_pmu_disable_all (); handled =intel_pmu_drain_bts_buffer (); status =intel_pmu_get_status (); -if (!status) { -intel_pmu_enable_all (0 ); -return handled; - } - +if (!status) + goto done; loops =0 ; again:intel_pmu_ack_status (status);
@@ -1210,6 +1207,10 @@ again: goto again; done: +if (!handled && print_spurious_pmi) { +pr_debug ("Spurious PMI\n" ); +perf_event_print_debug (); + }intel_pmu_enable_all (0 );return handled; }
Signature · 1 line
1.7.7.6
PATCH 19/32 perf, core: Add generic transaction flags v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a generic qualifier for transaction events, as a new sample type that returns a flag word. This is particularly useful for qualifying aborts: to distinguish aborts which happen due to asynchronous events (like conflicts caused by another CPU) versus instructions that lead to an abort.
The tuning strategies are very different for those cases, so it's important to distinguish them easily and early.
Since it's inconvenient and inflexible to filter for this in the kernel we report all the events out and allow some post processing in user space.
The flags are based on the Intel TSX events, but should be fairly generic and mostly applicable to other architectures too. In addition to various flag words there's also reserved space to report an program supplied abort code. For TSX this is used to distinguish specific classes of aborts, like a lock busy abort when doing lock elision.
This adds the perf core glue needed for reporting the new flag word out.
v2: Add MEM/MISC
--- include/linux/perf_event.h | 2 ++ include/uapi/linux/perf_event.h | 26 ++++++++++++++++++++++++-- kernel/events/core.c | 6 ++++++ 3 files changed, 32 insertions(+), 2 deletions(-)
include/linux/perf_event.h +2 -0
@@ -589,6 +589,7 @@ struct perf_sample_data {struct perf_regs_user regs_user;u64 stack_user_size;u64 weight; +u64 transaction; }; static inline void perf_sample_data_init(struct perf_sample_data *data,
@@ -603,6 +604,7 @@ static inline void perf_sample_data_init(struct perf_sample_data *data, data->regs_user.regs =NULL ; data->stack_user_size =0 ; data->weight =0 ; + data->transaction =0 ; } extern void perf_output_sample(struct perf_output_handle *handle,
include/uapi/linux/perf_event.h +24 -2
@@ -133,9 +133,9 @@ enum perf_event_sample_format {PERF_SAMPLE_REGS_USER =1U <<12 ,PERF_SAMPLE_STACK_USER =1U <<13 ,PERF_SAMPLE_WEIGHT =1U <<14 , +PERF_SAMPLE_TRANSACTION =1U <<15 , - PERF_SAMPLE_MAX = 1U << 15, /* non-ABI */ - + PERF_SAMPLE_MAX = 1U << 16, /* non-ABI */ };/*
@@ -179,6 +179,28 @@ enum perf_sample_regs_abi { };/* +* Values for the transaction event qualifier, mostly for abort events. +*/ +enum { +PERF_SAMPLE_TXN_ELISION = (1 <<0 ),/* From elision */ +PERF_SAMPLE_TXN_TRANSACTION = (1 <<1 ),/* From transaction */ +PERF_SAMPLE_TXN_SYNC = (1 <<2 ),/* Instruction is related */ +PERF_SAMPLE_TXN_ASYNC = (1 <<3 ),/* Instruction not related */ +PERF_SAMPLE_TXN_RETRY = (1 <<4 ),/* Retry possible */ +PERF_SAMPLE_TXN_CONFLICT = (1 <<5 ),/* Conflict abort */ +PERF_SAMPLE_TXN_CAPACITY = (1 <<6 ),/* Capacity abort */ +PERF_SAMPLE_TXN_MEMORY = (1 <<7 ),/* Memory related abort */ +PERF_SAMPLE_TXN_MISC = (1 <<8 ),/* Misc aborts */ + +PERF_SAMPLE_TXN_MAX = (1 <<9 ),/* non-ABI */ + +/* bits 24..31 are reserved for the abort code */ + +PERF_SAMPLE_TXN_ABORT_MASK =0xff000000 , +PERF_SAMPLE_TXN_ABORT_SHIFT =24 , +}; + +/* * The format of the data returned by read() on a perf event fd, * as specified by attr.read_format: *
kernel/events/core.c +6 -0
@@ -955,6 +955,9 @@ static void perf_event__header_size(struct perf_event *event)if (sample_type &PERF_SAMPLE_WEIGHT ) size +=sizeof (data->weight); +if (sample_type &PERF_SAMPLE_TRANSACTION ) + size +=sizeof (data->transaction); +if (sample_type &PERF_SAMPLE_READ ) size += event->read_size;
@@ -4086,6 +4089,9 @@ void perf_output_sample(struct perf_output_handle *handle,if (sample_type &PERF_SAMPLE_WEIGHT )perf_output_put (handle, data->weight); +if (sample_type &PERF_SAMPLE_TRANSACTION ) +perf_output_put (handle, data->transaction); +if (sample_type &PERF_SAMPLE_READ )perf_output_read (handle, event);
Signature · 1 line
1.7.7.6
PATCH 20/32 perf, x86: Add Haswell specific transaction flag reporting
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
In the PEBS handler report the transaction flags using the new generic transaction flags facility. Most of them come from the "tsx_tuning" field in PEBSv2, but the abort code is derived from the RAX register reported in the PEBS record.
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +9 -0
@@ -677,6 +677,15 @@ static void __intel_pmu_pebs_event(struct perf_event *event, data.weight = ((struct pebs_record_v2 *)pebs)->nhm.lat; } +if ((event->attr.sample_type &PERF_SAMPLE_TRANSACTION ) && + x86_pmu.intel_cap.pebs_format >=2 ) { + data.transaction = + ((struct pebs_record_v2 *)pebs)->tsx_tuning >>32 ; +if ((data.transaction &PERF_SAMPLE_TXN_TRANSACTION ) && + (pebs->ax &1 )) + data.transaction |= pebs->ax &0xff000000 ; + } +if (has_branch_stack (event)) data.br_stack = &cpuc->lbr_stack;
Signature · 1 line
1.7.7.6
PATCH 21/32 perf, tools: Add support for record transaction flags v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add the glue in the user tools to record transaction flags with --transaction (-T was already taken) and dump them.
Followon patches will use them.
v2: Fix manpage
--- tools/perf/Documentation/perf-record.txt | 4 +++- tools/perf/builtin-record.c | 2 ++ tools/perf/perf.h | 1 + tools/perf/util/event.h | 1 + tools/perf/util/evsel.c | 9 +++++++++ tools/perf/util/session.c | 3 +++ 6 files changed, 19 insertions(+), 1 deletions(-)
tools/perf/Documentation/perf-record.txt +3 -1
@@ -185,12 +185,14 @@ is enabled for all the sampling events. The sampled branch type is the same for The various filters must be specified as a comma separated list: --branch-filter any_ret,u,k Note that this feature may not be available on all processors. --W:: --weight:: Enable weightened sampling. An additional weight is recorded per sample and can be displayed with the weight and local_weight sort keys. This currently works for TSX abort events and some memory events in precise mode on modern Intel CPUs. +--transaction:: +Record transaction flags for transaction related events. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +2 -0
@@ -1046,6 +1046,8 @@ const struct option record_options[] = { parse_branch_stack),OPT_BOOLEAN ('W' ,"weight" , &record.opts.sample_weight,"sample by weight (on special events only)" ), +OPT_BOOLEAN (0 ,"transaction" , &record.opts.sample_transaction, +"sample transaction flags (special events only)" ), OPT_END() };
tools/perf/perf.h +1 -0
@@ -247,6 +247,7 @@ struct perf_record_opts {u64 default_interval;u64 user_interval;u16 stack_dump_size; + bool sample_transaction; };#endif
tools/perf/util/event.h +1 -0
@@ -89,6 +89,7 @@ struct perf_sample {u64 stream_id;u64 period;u64 weight; +u64 transaction;u32 cpu;u32 raw_size;void *raw_data;
tools/perf/util/evsel.c +9 -0
@@ -448,6 +448,9 @@ void perf_evsel__config(struct perf_evsel *evsel, struct perf_record_opts *opts,if (opts->sample_weight) attr->sample_type |=PERF_SAMPLE_WEIGHT ; +if (opts->sample_transaction) + attr->sample_type |=PERF_SAMPLE_TRANSACTION ; +if (opts->call_graph) { attr->sample_type |=PERF_SAMPLE_CALLCHAIN ;
@@ -951,6 +954,12 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event, array++; } + data->transaction =0 ; +if (type &PERF_SAMPLE_TRANSACTION ) { + data->transaction = *array; + array++; + } +if (type &PERF_SAMPLE_READ ) {fprintf (stderr,"PERF_SAMPLE_READ is unsupported for now\n" );return -1 ;
tools/perf/util/session.c +3 -0
@@ -1009,6 +1009,9 @@ static void dump_sample(struct perf_evsel *evsel, union perf_event *event,if (sample_type &PERF_SAMPLE_WEIGHT )printf ("... weight: %" PRIu64"\n" , sample->weight); + +if (sample_type &PERF_SAMPLE_TRANSACTION ) +printf ("... transaction: %" PRIx64"\n" , sample->transaction); } static struct machine *
Signature · 1 line
1.7.7.6
PATCH 22/32 perf, tools: Point --sort documentation to --help
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
The --sort documentation for top and report was hopelessly out-of-date Instead of having two more places that would need to be updated, just point to --help.
--- tools/perf/Documentation/perf-report.txt | 2 +- tools/perf/Documentation/perf-top.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/perf/Documentation/perf-report.txt +1 -1
@@ -57,7 +57,7 @@ OPTIONS -s:: --sort=:: - Sort by key(s): pid, comm, dso, symbol, parent, srcline. + Sort by key(s): See --help for a full list. -p:: --parent=<regex>::
tools/perf/Documentation/perf-top.txt +1 -1
@@ -112,7 +112,7 @@ Default is to monitor all CPUS. -s:: --sort:: - Sort by key(s): pid, comm, dso, symbol, parent, srcline. + Sort by key(s): see --help for a full list. -n:: --show-nr-samples::
Signature · 1 line
1.7.7.6
PATCH 23/32 perf, tools: Add browser support for transaction flags v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add histogram support for the transaction flags. Each flags instance becomes a separate histogram. Support sorting and displaying the flags in report and top.
The patch is fairly large, but it's really mostly just plumbing to pass the flags around.
v2: Increase column. Fix flags decoding. Use longer strings for flags to be more user friendly.
v3: Fix WERROR=1 build. Tidy display
--- tools/perf/builtin-annotate.c | 2 +- tools/perf/builtin-diff.c | 8 +++-- tools/perf/builtin-report.c | 4 +- tools/perf/builtin-top.c | 4 +- tools/perf/util/hist.c | 3 +- tools/perf/util/hist.h | 3 +- tools/perf/util/sort.c | 61 +++++++++++++++++++++++++++++++++++++++++ tools/perf/util/sort.h | 2 + 8 files changed, 77 insertions(+), 10 deletions(-)
tools/perf/builtin-annotate.c +1 -1
@@ -62,7 +62,7 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,return 0 ; } - he =__hists__add_entry (&evsel->hists, al,NULL ,1 ,1 ); + he =__hists__add_entry (&evsel->hists, al,NULL ,1 ,1 ,0 );if (he ==NULL )return -ENOMEM ;
tools/perf/builtin-diff.c +5 -3
@@ -249,9 +249,10 @@ int perf_diff__formula(char *buf, size_t size, struct hist_entry *he)static int hists__add_entry (struct hists *self,struct addr_location *al,u64 period, -u64 weight) +u64 weight,u64 transaction) { -if (__hists__add_entry (self, al,NULL , period, weight) !=NULL ) +if (__hists__add_entry (self, al,NULL , period, weight, transaction) + !=NULL )return 0 ;return -ENOMEM ; }
@@ -273,7 +274,8 @@ static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,if (al.filtered)return 0 ; -if (hists__add_entry (&evsel->hists, &al, sample->period, sample->weight)) { +if (hists__add_entry (&evsel->hists, &al, sample->period, sample->weight, + sample->transaction)) {pr_warning ("problem incrementing symbol period, skipping event\n" );return -1 ; }
tools/perf/builtin-report.c +2 -2
@@ -147,7 +147,7 @@ static int perf_evsel__add_hist_entry(struct perf_evsel *evsel, } he =__hists__add_entry (&evsel->hists, al, parent, sample->period, - sample->weight); + sample->weight, sample->transaction);if (he ==NULL )return -ENOMEM ;
@@ -597,7 +597,7 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" ,"sort by key(s): pid, comm, dso, symbol, parent, dso_to," " dso_from, symbol_to, symbol_from, mispredict, srcline," -" abort, intx, weight, local_weight" ), +" abort, intx, weight, local_weight, transaction" ),OPT_BOOLEAN (0 ,"showcpuutilization" , &symbol_conf.show_cpu_utilization,"Show sample percentage for different cpu modes" ), OPT_STRING('p', "parent", &parent_pattern, "regex",
tools/perf/builtin-top.c +2 -2
@@ -272,7 +272,7 @@ static struct hist_entry *perf_evsel__add_hist_entry(struct perf_evsel *evsel,struct hist_entry *he; he =__hists__add_entry (&evsel->hists, al,NULL , sample->period, - sample->weight); + sample->weight, sample->transaction);if (he ==NULL )return NULL ;
@@ -1229,7 +1229,7 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)OPT_STRING ('s' ,"sort" , &sort_order,"key[,key2...]" ,"sort by key(s): pid, comm, dso, symbol, parent, dso_to," " dso_from, symbol_to, symbol_from, mispredict, srcline," -" abort, intx, weight, local_weight" ), +" abort, intx, weight, local_weight, transaction" ),OPT_BOOLEAN ('n' ,"show-nr-samples" , &symbol_conf.show_nr_samples,"Show a column with the number of samples" ), OPT_CALLBACK_DEFAULT('G', "call-graph", &top, "output_type,min_percent, call_order",
tools/perf/util/hist.c +2 -1
@@ -359,7 +359,7 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self,struct hist_entry *__hists__add_entry (struct hists *self,struct addr_location *al,struct symbol *sym_parent,u64 period, -u64 weight) +u64 weight,u64 transaction) {struct hist_entry entry = { .thread = al->thread,
@@ -377,6 +377,7 @@ struct hist_entry *__hists__add_entry(struct hists *self, .parent = sym_parent, .filtered = symbol__parent_filter(sym_parent), .hists = self, + .transaction = transaction, };return add_hist_entry (self, &entry, al, period, weight);
tools/perf/util/hist.h +2 -1
@@ -52,6 +52,7 @@ enum hist_column {HISTC_SRCLINE ,HISTC_WEIGHT ,HISTC_GLOBAL_WEIGHT , +HISTC_TRANSACTION , HISTC_NR_COLS, /* Last entry */ };
@@ -77,7 +78,7 @@ struct hists {struct hist_entry *__hists__add_entry (struct hists *self,struct addr_location *al,struct symbol *parent,u64 period, -u64 weight); +u64 weight,u64 transaction);int64_t hist_entry__cmp (struct hist_entry *left,struct hist_entry *right);int64_t hist_entry__collapse (struct hist_entry *left,struct hist_entry *right); int hist_entry__sort_snprintf(struct hist_entry *self, char *bf, size_t size,
tools/perf/util/sort.c +61 -0
79 lines not on this page · raw message
tools/perf/util/sort.h +2 -0
@@ -82,6 +82,7 @@ struct hist_entry {struct map_symbol ms;struct thread *thread;u64 ip; +u64 transaction;s32 cpu;struct hist_entry_diff diff;
@@ -123,6 +124,7 @@ enum sort_type {SORT_INTX ,SORT_LOCAL_WEIGHT ,SORT_GLOBAL_WEIGHT , + SORT_TRANSACTION, };/*
Signature · 1 line
1.7.7.6
PATCH 24/32 perf, tools: Add arbitary aliases and support names with -
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
- Add missing scanner symbol for arbitrary aliases inside the config region.
- looks nicer than _, so allow - in the event names. Used for various of the arch perfmon and Haswell events.
--- tools/perf/util/parse-events.l | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
tools/perf/util/parse-events.l +2 -0
@@ -81,6 +81,7 @@ num_dec [0-9]+ num_hex 0x[a-fA-F0-9]+ num_raw_hex [a-fA-F0-9]+ name [a-zA-Z_*?][a-zA-Z0-9_*?]* +name_minus [a-zA-Z_*?][a-zA-Z0-9\-_*?]* modifier_event [ukhpGH]{1,8} modifier_bp [rwx]{1,3}
@@ -168,6 +169,7 @@ period { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_SAMPLE_PERIOD); } branch_type { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE); } , { return ','; } "/" { BEGIN(INITIAL); return '/'; } +{name_minus} { return str(yyscanner, PE_NAME); } } mem: { BEGIN(mem); return PE_PREFIX_MEM; }
Signature · 1 line
1.7.7.6
PATCH 25/32 tools, perf: Add a precise event qualifier
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a precise qualifier, like cpu/event=0x3c,precise=1/
This is needed so that the kernel can request enabling PEBS for TSX events. The parser bails out on any sysfs parse errors, so this is needed in any case to handle any event on the TSX perf kernel.
--- tools/perf/util/parse-events.c | 6 ++++++ tools/perf/util/parse-events.h | 1 + tools/perf/util/parse-events.l | 1 + 3 files changed, 8 insertions(+), 0 deletions(-)
tools/perf/util/parse-events.c +6 -0
@@ -526,6 +526,12 @@ do { \case PARSE_EVENTS__TERM_TYPE_NAME :CHECK_TYPE_VAL (STR );break ; +case PARSE_EVENTS__TERM_TYPE_PRECISE : +CHECK_TYPE_VAL (NUM ); +if ((unsigned )term->val.num >2 ) +return -EINVAL ; + attr->precise_ip = term->val.num; +break ;default :return -EINVAL ; }
tools/perf/util/parse-events.h +1 -0
@@ -49,6 +49,7 @@ enum {PARSE_EVENTS__TERM_TYPE_NAME ,PARSE_EVENTS__TERM_TYPE_SAMPLE_PERIOD ,PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE , + PARSE_EVENTS__TERM_TYPE_PRECISE, }; struct parse_events__term {
tools/perf/util/parse-events.l +1 -0
@@ -169,6 +169,7 @@ period { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_SAMPLE_PERIOD); } branch_type { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_BRANCH_SAMPLE_TYPE); } , { return ','; } "/" { BEGIN(INITIAL); return '/'; } +precise { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_PRECISE); } {name_minus} { return str(yyscanner, PE_NAME); } }
Signature · 1 line
1.7.7.6
Re: [PATCH 25/32] tools, perf: Add a precise event qualifier
Quoted 21 lines · Andi Kleen
From: Andi Kleen <[email protected]>
Add a precise qualifier, like cpu/event=0x3c,precise=1/
This is needed so that the kernel can request enabling PEBS for TSX events. The parser bails out on any sysfs parse errors, so this is needed in any case to handle any event on the TSX perf kernel.
Signed-off-by Andi Kleen <[email protected]>--- tools/perf/util/parse-events.c | 6 ++++++ tools/perf/util/parse-events.h | 1 + tools/perf/util/parse-events.l | 1 + 3 files changed, 8 insertions(+), 0 deletions(-)tools/perf/util/parse-events.c +0
-0
tools/perf/util/parse-events.c do { \ · line 532
Quoted 7 lines
tools/perf/util/parse-events.c +4
-0@@ -526,6 +526,12 @@ do { \case PARSE_EVENTS__TERM_TYPE_NAME :CHECK_TYPE_VAL (STR );break ; +case PARSE_EVENTS__TERM_TYPE_PRECISE : +CHECK_TYPE_VAL (NUM ); +if ((unsigned )term->val.num >2 ) +return -EINVAL ;
Should we test for >3?
* precise_ip:
*
* 0 - SAMPLE_IP can have arbitrary skid
* 1 - SAMPLE_IP must have constant skid
* 2 - SAMPLE_IP requested to have 0 skid
* 3 - SAMPLE_IP must have 0 skid
Maybe it's not implemented in hw yet, but in general it's allowed.
thanks,
jirka
Re: [PATCH 25/32] tools, perf: Add a precise event qualifier
Quoted 10 lines
Should we test for >3?
*
precise_ip:
*
* 0 -SAMPLE_IPcan have arbitrary skid
* 1 -SAMPLE_IPmust have constant skid
* 2 -SAMPLE_IPrequested to have 0 skid
* 3 -SAMPLE_IPmust have 0 skidMaybe it's not implemented in hw yet, but in general it's allowed.
Fixed.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
PATCH 26/32 perf, x86: improve sysfs event mapping with event string
In reply to Andi KleenFrom: Stephane Eranian <[email protected]>
This patch extends Jiri's changes to make generic events mapping visible via sysfs. The patch extends the mechanism to non-generic events by allowing the mappings to be hardcoded in strings.
This mechanism will be used by the PEBS-LL patch
later on.
--- arch/x86/kernel/cpu/perf_event.c | 28 +++++++++++++--------------- arch/x86/kernel/cpu/perf_event.h | 26 ++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/perf_event.c +13 -15
@@ -1317,20 +1317,22 @@ static struct attribute_group x86_pmu_format_group = { .attrs = NULL, }; -struct perf_pmu_events_attr { -struct device_attribute attr; -u64 id; -}; -/* * Remove all undefined events (x86_pmu.event_map(id) == 0) * out of events_attr attributes. */ static void __init filter_events (struct attribute **attrs) { +struct device_attribute *d; +struct perf_pmu_events_attr *pmu_attr;int i, j;for (i =0 ; attrs[i]; i++) { + d = (struct device_attribute *)attrs[i]; + pmu_attr = container_of(d, struct perf_pmu_events_attr, attr); +/* str trumps id */ +if (pmu_attr->event_str) +continue ;if (x86_pmu.event_map (i))continue ;
@@ -1342,24 +1344,20 @@ static void __init filter_events(struct attribute **attrs) } } -static ssize_t events_sysfs_show (struct device *dev,struct device_attribute *attr, +ssize_t events_sysfs_show (struct device *dev,struct device_attribute *attr,char *page) {struct perf_pmu_events_attr *pmu_attr = \ container_of(attr, struct perf_pmu_events_attr, attr);u64 config = x86_pmu.event_map (pmu_attr->id); -return x86_pmu.events_sysfs_show (page, config); -} -#define EVENT_VAR (_id) event_attr_##_id -#define EVENT_PTR (_id) &event_attr_##_id.attr.attr +/* string trumps id */ +if (pmu_attr->event_str) +return sprintf (page,"%s" , pmu_attr->event_str); -#define EVENT_ATTR (_name, _id) \ -static struct perf_pmu_events_attr EVENT_VAR(_id) = { \ - .attr = __ATTR(_name, 0444, events_sysfs_show, NULL), \ - .id = PERF_COUNT_HW_##_id, \ -}; +return x86_pmu.events_sysfs_show (page, config); +}EVENT_ATTR (cpu-cycles,CPU_CYCLES );EVENT_ATTR (instructions,INSTRUCTIONS );
arch/x86/kernel/cpu/perf_event.h +26 -0
@@ -425,6 +425,29 @@ do { \#define ERF_NO_HT_SHARING 1#define ERF_HAS_RSP_1 2 +#define EVENT_VAR (_id) event_attr_##_id +#define EVENT_PTR (_id) &event_attr_##_id.attr.attr + +#define EVENT_ATTR (_name, _id) \ +static struct perf_pmu_events_attr EVENT_VAR(_id) = { \ + .attr = __ATTR(_name, 0444, events_sysfs_show, NULL), \ + .id = PERF_COUNT_HW_##_id, \ + .event_str = NULL, \ +}; + +#define EVENT_ATTR_STR (_name, v, str) \ +static struct perf_pmu_events_attr event_attr_##v = { \ + .attr = __ATTR(_name, 0444, events_sysfs_show, NULL),\ + .id = 0, \ + .event_str = str, \ +}; + +struct perf_pmu_events_attr { +struct device_attribute attr; +u64 id; +const char *event_str; +}; +extern struct x86_pmu x86_pmu__read_mostly ; DECLARE_PER_CPU(struct cpu_hw_events, cpu_hw_events);
@@ -643,6 +666,9 @@ int p6_pmu_init(void);int knc_pmu_init (void ); +ssize_t events_sysfs_show (struct device *dev,struct device_attribute *attr, +char *page); +#else /* CONFIG_CPU_SUP_INTEL */ static inline void reserve_ds_buffers (void )
Signature · 1 line
1.7.7.6
PATCH 27/32 perf, x86: Support CPU specific sysfs events
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a way for the CPU initialization code to register additional events, and merge them into the events attribute directory. Used in the next patch.
--- arch/x86/kernel/cpu/perf_event.c | 29 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event.h | 1 + 2 files changed, 30 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event.c +29 -0
@@ -1344,6 +1344,30 @@ static void __init filter_events(struct attribute **attrs) } } +/* Merge two pointer arrays */ +static __init struct attribute **merge_attr (struct attribute **a, +struct attribute **b) +{ +struct attribute **new; +int j, i; + +for (j =0 ; a[j]; j++) + ; +for (i =0 ; b[i]; i++) + j++; + j++; + new =kmalloc (sizeof (struct attribute *) * j,GFP_KERNEL ); +if (!new) +return a; + j =0 ; +for (i =0 ; a[i]; i++) + new[j++] = a[i]; +for (i =0 ; b[i]; i++) + new[j++] = b[i]; + new[j] =NULL ; +return new; +} +ssize_t events_sysfs_show (struct device *dev,struct device_attribute *attr,char *page) {
@@ -1481,6 +1505,11 @@ static int __init init_hw_perf_events(void)else filter_events (x86_pmu_events_group.attrs); +if (x86_pmu.cpu_events) + x86_pmu_events_group.attrs = +merge_attr (x86_pmu_events_group.attrs, + x86_pmu.cpu_events); +pr_info ("... version: %d\n" , x86_pmu.version);pr_info ("... bit width: %d\n" , x86_pmu.cntval_bits);pr_info ("... generic registers: %d\n" , x86_pmu.num_counters);
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -360,6 +360,7 @@ struct x86_pmu {struct attribute **format_attrs;ssize_t (*events_sysfs_show)(char *page,u64 config); +struct attribute **cpu_events;/* * CPU Hotplug hooks
Signature · 1 line
1.7.7.6
PATCH 28/32 perf, x86: Add Haswell TSX event aliases v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add infrastructure to generate event aliases in /sys/devices/cpu/events/
And use this to set up user friendly aliases for the common TSX events. TSX tuning relies heavily on the PMU, so it's important to be user friendly.
This replaces the generic transaction events in an earlier version of this patchkit.
tx-start/commit/abort to count RTM transactions
el-start/commit/abort to count HLE ("elision") transactions
tx-conflict/overflow to count conflict/overflow for both combined.The general abort events exist in precise and non precise variants Since the common case is sampling plain "tx-aborts" in precise.
This is very important because abort sampling only really works with PEBS enabled, otherwise it would report the IP after the abort, not the abort point. But counting with PEBS has more overhead, so also have tx/el-abort-count aliases that do not enable PEBS for perf stat.
It would be nice to switch automatically between those two, like in the previous version, but that would need more new infrastructure for sysfs first.
There is an tx-abort<->tx-aborts alias too, because I found myself using both variants.
Also added friendly aliases for cpu/cycles,intx=1/ and cpu/cycles,intx=1,intx_cp=1/ and the same for instructions. These will be used by perf stat -T, and are also useful for users directly.
So for example to get transactional cycles can use "perf stat -e cycles-t"
v2: Move to new sysfs infrastructure
--- arch/x86/kernel/cpu/perf_event_intel.c | 47 ++++++++++++++++++++++++++++++++ 1 files changed, 47 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +47 -0
@@ -2010,6 +2010,52 @@ static __init void intel_nehalem_quirk(void) } } +/* Haswell special events */ +EVENT_ATTR_STR (tx-start, tx_start,"event=0xc9,umask=0x1" ); +EVENT_ATTR_STR (tx-commit, tx_commit,"event=0xc9,umask=0x2" ); +EVENT_ATTR_STR (tx-abort, tx_abort,"event=0xc9,umask=0x4,precise=2" ); +EVENT_ATTR_STR (tx-abort-count, tx_abort_count,"event=0xc9,umask=0x4" ); +/* alias */ +EVENT_ATTR_STR (tx-aborts, tx_aborts,"event=0xc9,umask=0x4,precise=2" ); +EVENT_ATTR_STR (tx-capacity, tx_capacity,"event=0x54,umask=0x2" ); +EVENT_ATTR_STR (tx-conflict, tx_conflict,"event=0x54,umask=0x1" ); +EVENT_ATTR_STR (el-start, el_start,"event=0xc8,umask=0x1" ); +EVENT_ATTR_STR (el-commit, el_commit,"event=0xc8,umask=0x2" ); +EVENT_ATTR_STR (el-abort, el_abort,"event=0xc8,umask=0x4,precise=2" ); +EVENT_ATTR_STR (el-abort-count, el_abort_count,"event=0xc8,umask=0x4" ); +/* alias */ +EVENT_ATTR_STR (el-aborts, el_aborts,"event=0xc8,umask=0x4,precise=2" ); +/* shared with tx-* */ +EVENT_ATTR_STR (el-capacity, el_capacity,"event=0x54,umask=0x2" ); +/* shared with tx-* */ +EVENT_ATTR_STR (el-conflict, el_conflict,"event=0x54,umask=0x1" ); +EVENT_ATTR_STR (cycles-t, cycles_t,"event=0x3c,intx=1" ); +EVENT_ATTR_STR (cycles-ct, cycles_ct,"event=0x3c,intx=1,intx_cp=1" ); +EVENT_ATTR_STR (instructions-t, instructions_t,"event=0xc0,intx=1" ); +EVENT_ATTR_STR (instructions-ct,instructions_ct,"event=0xc0,intx=1,intx_cp=1" ); + +static struct attribute *hsw_events_attrs[] = { +EVENT_PTR (tx_start), +EVENT_PTR (tx_commit), +EVENT_PTR (tx_abort), +EVENT_PTR (tx_aborts), +EVENT_PTR (tx_abort_count), +EVENT_PTR (tx_capacity), +EVENT_PTR (tx_conflict), +EVENT_PTR (el_start), +EVENT_PTR (el_commit), +EVENT_PTR (el_abort), +EVENT_PTR (el_aborts), +EVENT_PTR (el_abort_count), +EVENT_PTR (el_capacity), +EVENT_PTR (el_conflict), +EVENT_PTR (cycles_t), +EVENT_PTR (cycles_ct), +EVENT_PTR (instructions_t), +EVENT_PTR (instructions_ct), +NULL +}; +__init int intel_pmu_init (void ) {union cpuid10_edx edx;
@@ -2235,6 +2281,7 @@ __init int intel_pmu_init(void) x86_pmu.get_event_constraints = hsw_get_event_constraints; x86_pmu.format_attrs = intel_hsw_formats_attr; x86_pmu.memory_lat_events = intel_hsw_memory_latency_events; + x86_pmu.cpu_events = hsw_events_attrs;pr_cont ("Haswell events, " );break ;
Signature · 1 line
1.7.7.6
PATCH 29/32 perf, tools: Add perf stat --transaction v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add support to perf stat to print the basic transactional execution statistics:
Total cycles, Cycles in Transaction, Cycles in aborted transsactions using the intx and intx_checkpoint qualifiers.
Transaction Starts and Elision Starts, to compute the average transaction length.
This is a reasonable overview over the success of the transactions.
Enable with a new --transaction / -T option.
This requires measuring these events in a group, since they depend on each other.
This is implemented by using TM sysfs events exported by the kernel
v2: Only print the extended statistics when the option is enabled. This avoids negative output when the user specifies the -T events in separate groups.
--- tools/perf/Documentation/perf-stat.txt | 4 + tools/perf/builtin-stat.c | 101 +++++++++++++++++++++++++++++++- tools/perf/util/evsel.h | 6 ++ 3 files changed, 108 insertions(+), 3 deletions(-)
tools/perf/Documentation/perf-stat.txt +4 -0
@@ -114,6 +114,10 @@ with it. --append may be used here. Examples: perf stat --repeat 10 --null --sync --pre 'make -s O=defconfig-build/clean' -- make -s -j64 O=defconfig-build/ bzImage +-T:: +--transaction:: + +Print statistics of transactional execution if supported. EXAMPLES --------
tools/perf/builtin-stat.c +98 -3
156 lines not on this page · raw message
tools/perf/util/evsel.h +6 -0
@@ -158,6 +158,12 @@ static inline bool perf_evsel__match2(struct perf_evsel *e1, (e1->attr.config == e2->attr.config); } +#define perf_evsel__cmp (a, b) \ + ((a) && \ + (b) && \ + (a)->attr.type == (b)->attr.type && \ + (a)->attr.config == (b)->attr.config) +int __perf_evsel__read_on_cpu (struct perf_evsel *evsel,int cpu,int thread,bool scale);
Signature · 1 line
1.7.7.6
PATCH 30/32 perf, x86: Add a Haswell precise instructions event v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add a instructions-p event alias that uses the PDIR randomized instruction retirement event. This is useful to avoid some systematic sampling shadow problems. Normally PEBS sampling has a systematic shadow. With PDIR enabled the hardware adds some randomization that statistically avoids this problem. In this sense, it's more precise over a whole sampling interval, but an individual sample can be less precise. But since we sample overall it's a more precise event.
This could be used before using the explicit event code syntax, but it's easier and more user friendly to use with an "instructions-p" alias. I expect this will eventually become a common use case.
Right now for Haswell, will add to Ivy Bridge later too.
v2: Use new sysfs infrastructure
--- arch/x86/kernel/cpu/perf_event_intel.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +2 -0
@@ -2033,6 +2033,7 @@ EVENT_ATTR_STR(cycles-t, cycles_t, "event=0x3c,intx=1");EVENT_ATTR_STR (cycles-ct, cycles_ct,"event=0x3c,intx=1,intx_cp=1" );EVENT_ATTR_STR (instructions-t, instructions_t,"event=0xc0,intx=1" );EVENT_ATTR_STR (instructions-ct,instructions_ct,"event=0xc0,intx=1,intx_cp=1" ); +EVENT_ATTR_STR (instructions-p, instructions_p,"event=0xc0,umask=0x01,precise=2" );static struct attribute *hsw_events_attrs[] = {EVENT_PTR (tx_start),
@@ -2053,6 +2054,7 @@ static struct attribute *hsw_events_attrs[] = {EVENT_PTR (cycles_ct),EVENT_PTR (instructions_t),EVENT_PTR (instructions_ct), +EVENT_PTR (instructions_p), NULL };
Signature · 1 line
1.7.7.6
PATCH 31/32 perf, tools: Default to cpu// for events v3
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
When an event fails to parse and it's not in a new style format, try to parse it again as a cpu event.
This allows to use sysfs exported events directly without //, so I can use
perf record -e tx-aborts ...
instead of
perf record -e cpu/tx-aborts/
v2: Handle multiple events
v3: Move to separate function
--- tools/perf/util/parse-events.c | 45 +++++++++++++++++++++++++++++++++++++++- 1 files changed, 44 insertions(+), 1 deletions(-)
tools/perf/util/parse-events.c +44 -1
@@ -768,6 +768,47 @@ int parse_events_name(struct list_head *list, char *name)return 0 ; } +static void str_append (char **s,int *len,const char *a) +{ +int olen = *s ?strlen (*s) :0 ; +int nlen = olen +strlen (a) +1 ; +if (*len < nlen) { + *len = *len *2 ; +if (*len < nlen) + *len = nlen; + *s =realloc (*s, *len); +if (!*s) +exit (ENOMEM ); +if (olen ==0 ) + **s =0 ; + } +strcat (*s, a); +} + +static int parse_events__scanner (const char *str,void *data,int start_token); + +static int parse_events_fixup (int ret,const char *str,void *data, +int start_token) +{ +char *o =strdup (str); +char *s =NULL ; +char *t = o; +char *p; +int len =0 ; + +if (!o) +return ret; +while ((p =strsep (&t,"," )) !=NULL ) { +if (s) +str_append (&s, &len,"," ); +str_append (&s, &len,"cpu/" ); +str_append (&s, &len, p); +str_append (&s, &len,"/" ); + } +free (o); +return parse_events__scanner (s, data, start_token); +} +static int parse_events__scanner (const char *str,void *data,int start_token) {YY_BUFFER_STATE buffer;
@@ -788,7 +829,9 @@ static int parse_events__scanner(const char *str, void *data, int start_token)parse_events__flush_buffer (buffer, scanner);parse_events__delete_buffer (buffer, scanner);parse_events_lex_destroy (scanner); -return ret; +if (ret && !strchr (str,'/' )) + ret =parse_events_fixup (ret, str, data, start_token); +return ret; }/*
Signature · 1 line
1.7.7.6
Re: [PATCH 31/32] perf, tools: Default to cpu// for events v3
Quoted 29 lines · Andi Kleen
From: Andi Kleen <[email protected]>
When an event fails to parse and it's not in a new style format, try to parse it again as a cpu event.
This allows to use sysfs exported events directly without //, so I can use
perf record -e tx-aborts ...
instead of
perf record -e cpu/tx-aborts/
v2: Handle multiple events
v3: Move to separate functionSigned-off-by Andi Kleen <[email protected]>--- tools/perf/util/parse-events.c | 45 +++++++++++++++++++++++++++++++++++++++- 1 files changed, 44 insertions(+), 1 deletions(-)tools/perf/util/parse-events.c +1
-0@@ -768,6 +768,47 @@ int parse_events_name(struct list_head *list, char *name)return 0 ; } +static void str_append (char **s,int *len,const char *a)
tools/perf/util/parse-events.c int parse_events_name(struct list_head *list, char *name) · line 781
Quoted 10 lines
tools/perf/util/parse-events.c +10
-0+{ +int olen = *s ?strlen (*s) :0 ; +int nlen = olen +strlen (a) +1 ; +if (*len < nlen) { + *len = *len *2 ; +if (*len < nlen) + *len = nlen; + *s =realloc (*s, *len); +if (!*s) +exit (ENOMEM );
Rather use BUG_ON or event better return -ENOMEM.
tools/perf/util/parse-events.c int parse_events_name(struct list_head *list, char *name) · line 786
Quoted 5 lines
tools/perf/util/parse-events.c +5
-0+if (olen ==0 ) + **s =0 ; + } +strcat (*s, a); +}
Could this one be moved to util/string.c in some generic form?
thanks,
jirka
Re: [PATCH 31/32] perf, tools: Default to cpu// for events v3
Quoted 5 lines
Quoted 5 lines
tools/perf/util/parse-events.c +5
-0+static void str_append (char **s,int *len,const char *a) +{ +int olen = *s ?strlen (*s) :0 ; +int nlen = olen +strlen (a) +1 ; +if (*len < nlen) {
Quoted 8 lines
tools/perf/util/parse-events.c int parse_events_name(struct list_head *list, char *name) · line 781
Quoted 6 lines
tools/perf/util/parse-events.c +6
-0+ *len = *len *2 ; +if (*len < nlen) + *len = nlen; + *s =realloc (*s, *len); +if (!*s) +exit (ENOMEM );Rather use
BUG_ONor event better return-ENOMEM.
Neither makes sense: the callers won't handle it and it's not a bug. If you're out of memory in user space the only thing you can do is to exit.
Quoted 8 lines
tools/perf/util/parse-events.c int parse_events_name(struct list_head *list, char *name) · line 786
Quoted 6 lines
tools/perf/util/parse-events.c +5
-0+if (olen ==0 ) + **s =0 ; + } +strcat (*s, a); +}Could this one be moved to
util/string.cin some generic form?
Ok.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Re: [PATCH 31/32] perf, tools: Default to cpu// for events v3
Em Wed, Nov 28, 2012 at 08:04:36PM +0100, Andi Kleen escreveu:
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
tools/perf/util/parse-events.c +5
-0+static void str_append (char **s,int *len,const char *a) +{ +int olen = *s ?strlen (*s) :0 ; +int nlen = olen +strlen (a) +1 ; +if (*len < nlen) {Quoted 8 lines
tools/perf/util/parse-events.c int parse_events_name(struct list_head *list, char *name) · line 781
Quoted 6 lines
tools/perf/util/parse-events.c +6
-0+ *len = *len *2 ; +if (*len < nlen) + *len = nlen; + *s =realloc (*s, *len); +if (!*s) +exit (ENOMEM );Rather use
BUG_ONor event better return-ENOMEM.
Quoted 3 lines
Neither makes sense: the callers won't handle it and it's not a bug. If you're out of memory in user space the only thing you can do is to exit.
Nope, just propagate it back and handle at the callers, the memory exhaustion may be temporary, just warn the user.
- Arnaldo
Quoted 15 lines
Quoted 8 lines
tools/perf/util/parse-events.c int parse_events_name(struct list_head *list, char *name) · line 786
Quoted 6 lines
tools/perf/util/parse-events.c +5
-0+if (olen ==0 ) + **s =0 ; + } +strcat (*s, a); +}Could this one be moved to
util/string.cin some generic form?Ok.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
PATCH 32/32 perf, tools: List kernel supplied event aliases in perf list v2
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
List the kernel supplied pmu event aliases in perf list
It's better when the users can actually see them.
v2: Fix pattern matching
--- tools/perf/Documentation/perf-list.txt | 4 +- tools/perf/builtin-list.c | 3 + tools/perf/util/parse-events.c | 5 ++- tools/perf/util/pmu.c | 72 ++++++++++++++++++++++++++++++++ tools/perf/util/pmu.h | 3 + 5 files changed, 85 insertions(+), 2 deletions(-)
tools/perf/Documentation/perf-list.txt +3 -1
@@ -8,7 +8,7 @@ perf-list - List all symbolic event types SYNOPSIS -------- [verse] -'perf list' [hw|sw|cache|tracepoint|event_glob] +'perf list' [hw|sw|cache|tracepoint|pmu|event_glob] DESCRIPTION -----------
@@ -104,6 +104,8 @@ To limit the list use: 'subsys_glob:event_glob' to filter by tracepoint subsystems such as sched, block, etc. +. 'pmu' to print the kernel supplied PMU events. + . If none of the above is matched, it will apply the supplied glob to all events, printing the ones that match.
tools/perf/builtin-list.c +3 -0
@@ -13,6 +13,7 @@#include "util/parse-events.h" #include "util/cache.h" +#include "util/pmu.h" int cmd_list (int argc,const char **argv,const char *prefix__maybe_unused ) {
@@ -37,6 +38,8 @@ int cmd_list(int argc, const char **argv, const char *prefix __maybe_unused)else if (strcmp (argv[i],"cache" ) ==0 ||strcmp (argv[i],"hwcache" ) ==0 )print_hwcache_events (NULL , false); +else if (strcmp (argv[i],"pmu" ) ==0 ) +print_pmu_events (NULL , false);else if (strcmp (argv[i],"--raw-dump" ) ==0 )print_events (NULL , true);else {
tools/perf/util/parse-events.c +4 -1
@@ -1069,6 +1069,8 @@ int print_hwcache_events(const char *event_glob, bool name_only) } } +if (printed) +printf ("\n" );return printed; }
@@ -1123,11 +1125,12 @@ void print_events(const char *event_glob, bool name_only)print_hwcache_events (event_glob, name_only); +print_pmu_events (event_glob, name_only); +if (event_glob !=NULL )return ;if (!name_only) { -printf ("\n" );printf (" %-50s [%s]\n" ,"rNNN" , event_type_descriptors[PERF_TYPE_RAW ]);
tools/perf/util/pmu.c +72 -0
78 lines not on this page · raw message
tools/perf/util/pmu.h +3 -0
@@ -3,6 +3,7 @@#include <linux/bitops.h> #include "../../../include/uapi/linux/perf_event.h" +#include <stdbool.h> enum {PERF_PMU_FORMAT_VALUE_CONFIG ,
@@ -49,5 +50,7 @@ void perf_pmu__set_format(unsigned long *bits, long from, long to);struct perf_pmu *perf_pmu__scan (struct perf_pmu *pmu); +void print_pmu_events (const char *event_glob,bool name_only); +int perf_pmu__test (void );#endif /* __PMU_H */
Signature · 1 line
1.7.7.6