perf: Add basic Skylake PMU support v2
Subsystems (2)
This patchkit adds support for the Intel Skylake core PMU to perf, documented in the recently released SDM 054[1] Vol3, 17.9 and 18.12.
The main user visible feature is timed branch records, which allows to get cycle counts for individual basic blocks, and a time stamp for PEBS records which improves multi-record PEBS. The LBRs (branch records) also have been extended to 32, which allows more accurate branch sampling and deeper call stacks.
v2:
Fix time stamp handling with non default clock. Fix LBR freezing.
Some minor cleanups.
Moved user tools support for cycles into separate patchkit.
-Andi
[1] http://www.cps.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html
PATCH 1/9 x86: Add a native_perf_sched_clock_from_tsc
From: Andi Kleen <[email protected]>
PEBSv3 has a raw TSC time stamp in its memory buffer that later needs to to be converted to perf_clock. Add a native_sched_clock_from_tsc() that works the same as native_sched_clock, but starts with an already given TSC value.
Right now I ignore paravirt for this. It will just get the native clock. But there isn't para virtualized PEBS anyways.
--- arch/x86/include/asm/tsc.h | 1 + arch/x86/kernel/tsc.c | 8 ++++++++ 2 files changed, 9 insertions(+)
arch/x86/include/asm/tsc.h +1 -0
@@ -51,6 +51,7 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern int check_tsc_disabled (void );extern unsigned long native_calibrate_tsc (void ); +extern unsigned long long native_sched_clock_from_tsc (u64 tsc);extern int tsc_clocksource_reliable;
arch/x86/kernel/tsc.c +8 -0
@@ -296,6 +296,14 @@ u64 native_sched_clock(void)return cycles_2_ns (tsc_now); } +/* +* Generate a sched_clock if you already have a TSC value. +*/ +u64 native_sched_clock_from_tsc (u64 tsc) +{ +return cycles_2_ns (tsc); +} +/* We need to define a real function for sched_clock, to override the weak default version */ #ifdef CONFIG_PARAVIRT
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86: Add a native_perf_sched_clock_from_tsc( )
perf/x86: Add a native_perf_sched_clock_from_tsc()
Commit-ID: a94cab2376cb35f236be14e2833cef63a8762a31 Gitweb: http://git.kernel.org/tip/a94cab2376cb35f236be14e2833cef63a8762a31 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:39 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:55 +0200
perf/x86: Add a native_perf_sched_clock_from_tsc()
PEBSv3 has a raw TSC time stamp in its memory buffer that later needs to to be converted to perf_clock.
Add a native_sched_clock_from_tsc() that works the same as native_sched_clock(), but starts with an already given TSC value.
Paravirt is ignored, it will just get the native clock. But there isn't a para virtualized PEBS anyway.
--- arch/x86/include/asm/tsc.h | 1 + arch/x86/kernel/tsc.c | 8 ++++++++ 2 files changed, 9 insertions(+)
arch/x86/include/asm/tsc.h +1 -0
@@ -51,6 +51,7 @@ extern int unsynchronized_tsc(void);extern int check_tsc_unstable (void );extern int check_tsc_disabled (void );extern unsigned long native_calibrate_tsc (void ); +extern unsigned long long native_sched_clock_from_tsc (u64 tsc);extern int tsc_clocksource_reliable;
arch/x86/kernel/tsc.c +8 -0
@@ -296,6 +296,14 @@ u64 native_sched_clock(void)return cycles_2_ns (tsc_now); } +/* +* Generate a sched_clock if you already have a TSC value. +*/ +u64 native_sched_clock_from_tsc (u64 tsc) +{ +return cycles_2_ns (tsc); +} +/* We need to define a real function for sched_clock, to override the weak default version */ #ifdef CONFIG_PARAVIRT
PATCH 2/9 x86, perf: Add support for PEBSv3 profiling
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
PEBSv3 is the same as the existing PEBSv2 used on Haswell, but it adds a new TSC field. Add support to the generic PEBS handler to handle the new format, and overwrite the perf time stamp using the new native_sched_clock_from_tsc() Right now the time stamp is just slightly more accurate, as it is nearer the actual event trigger point. With the PEBS threshold > 1 patchkit it will be much more accurate, avoid the problems with MMAP mismatches earlier.
The accurate time stamping is only implemented for the default trace clock for now.
v2: Use _skl prefix. Check for default clock_id.
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 36 ++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +33 -3
@@ -224,6 +224,19 @@ union hsw_tsx_tuning {#define PEBS_HSW_TSX_FLAGS 0xff00000000ULL +/* Same as HSW, plus TSC */ + +struct pebs_record_skl { +u64 flags, ip; +u64 ax, bx, cx, dx; +u64 si, di, bp, sp; +u64 r8, r9, r10, r11; +u64 r12, r13, r14, r15; +u64 status, dla, dse, lat; +u64 real_ip, tsx_tuning; +u64 tsc; +}; +void init_debug_store_on_cpu (int cpu) {struct debug_store *ds =per_cpu (cpu_hw_events, cpu).ds;
@@ -827,7 +840,7 @@ static int intel_pmu_pebs_fixup_ip(struct pt_regs *regs)return 0 ; } -static inline u64 intel_hsw_weight (struct pebs_record_hsw *pebs) +static inline u64 intel_hsw_weight (struct pebs_record_skl *pebs) {if (pebs->tsx_tuning) {union hsw_tsx_tuning tsx = { .value = pebs->tsx_tuning };
@@ -836,7 +849,7 @@ static inline u64 intel_hsw_weight(struct pebs_record_hsw *pebs)return 0 ; } -static inline u64 intel_hsw_transaction (struct pebs_record_hsw *pebs) +static inline u64 intel_hsw_transaction (struct pebs_record_skl *pebs) {u64 txn = (pebs->tsx_tuning &PEBS_HSW_TSX_FLAGS ) >>32 ;
@@ -858,7 +871,7 @@ static void __intel_pmu_pebs_event(struct perf_event *event,* unconditionally access the 'extra' entries. */ struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -struct pebs_record_hsw *pebs = __pebs; +struct pebs_record_skl *pebs = __pebs;struct perf_sample_data data;struct pt_regs regs;u64 sample_type;
@@ -958,6 +971,16 @@ static void __intel_pmu_pebs_event(struct perf_event *event, data.txn =intel_hsw_transaction (pebs); } +/* +* v3 supplies an accurate time stamp, so we use that +* for the time stamp. +* +* We can only do this for the default trace clock. +*/ +if (x86_pmu.intel_cap.pebs_format >=3 && + event->attr.use_clockid ==0 ) + data.time =native_sched_clock_from_tsc (pebs->tsc); +if (has_branch_stack (event)) data.br_stack = &cpuc->lbr_stack;
@@ -1098,6 +1121,13 @@ void __init intel_ds_init(void) x86_pmu.drain_pebs = intel_pmu_drain_pebs_nhm;break ; +case 3 : +pr_cont ("PEBS fmt3%c, " , pebs_type); + x86_pmu.pebs_record_size = +sizeof (struct pebs_record_skl ); + x86_pmu.drain_pebs = intel_pmu_drain_pebs_nhm; +break ; +default :printk (KERN_CONT "no PEBS fmt%d%c, " , format, pebs_type); x86_pmu.pebs =0 ;
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel: Add support for PEBSv3 profiling
perf/x86/intel: Add support for PEBSv3 profiling
Commit-ID: 2f7ebf2ec2a7b311318aae10b8373b0bd93001a7 Gitweb: http://git.kernel.org/tip/2f7ebf2ec2a7b311318aae10b8373b0bd93001a7 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:40 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:56 +0200
perf/x86/intel: Add support for PEBSv3 profiling
PEBSv3 is the same as the existing PEBSv2 used on Haswell, but it adds a new TSC field. Add support to the generic PEBS handler to handle the new format, and overwrite the perf time stamp using the new native_sched_clock_from_tsc().
Right now the time stamp is just slightly more accurate, as it is nearer the actual event trigger point. With the PEBS threshold > 1 patchkit it will be much more accurate, avoid the problems with MMAP mismatches earlier.
The accurate time stamping is only implemented for the default trace clock for now.
v2: Use _skl prefix. Check for default clock_id.
--- arch/x86/kernel/cpu/perf_event_intel_ds.c | 36 ++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_ds.c +33 -3
@@ -224,6 +224,19 @@ union hsw_tsx_tuning {#define PEBS_HSW_TSX_FLAGS 0xff00000000ULL +/* Same as HSW, plus TSC */ + +struct pebs_record_skl { +u64 flags, ip; +u64 ax, bx, cx, dx; +u64 si, di, bp, sp; +u64 r8, r9, r10, r11; +u64 r12, r13, r14, r15; +u64 status, dla, dse, lat; +u64 real_ip, tsx_tuning; +u64 tsc; +}; +void init_debug_store_on_cpu (int cpu) {struct debug_store *ds =per_cpu (cpu_hw_events, cpu).ds;
@@ -885,7 +898,7 @@ static int intel_pmu_pebs_fixup_ip(struct pt_regs *regs)return 0 ; } -static inline u64 intel_hsw_weight (struct pebs_record_hsw *pebs) +static inline u64 intel_hsw_weight (struct pebs_record_skl *pebs) {if (pebs->tsx_tuning) {union hsw_tsx_tuning tsx = { .value = pebs->tsx_tuning };
@@ -894,7 +907,7 @@ static inline u64 intel_hsw_weight(struct pebs_record_hsw *pebs)return 0 ; } -static inline u64 intel_hsw_transaction (struct pebs_record_hsw *pebs) +static inline u64 intel_hsw_transaction (struct pebs_record_skl *pebs) {u64 txn = (pebs->tsx_tuning &PEBS_HSW_TSX_FLAGS ) >>32 ;
@@ -918,7 +931,7 @@ static void setup_pebs_sample_data(struct perf_event *event,* unconditionally access the 'extra' entries. */ struct cpu_hw_events *cpuc =this_cpu_ptr (&cpu_hw_events); -struct pebs_record_hsw *pebs = __pebs; +struct pebs_record_skl *pebs = __pebs;u64 sample_type;int fll, fst, dsrc;int fl = event->hw.flags;
@@ -1016,6 +1029,16 @@ static void setup_pebs_sample_data(struct perf_event *event, data->txn =intel_hsw_transaction (pebs); } +/* +* v3 supplies an accurate time stamp, so we use that +* for the time stamp. +* +* We can only do this for the default trace clock. +*/ +if (x86_pmu.intel_cap.pebs_format >=3 && + event->attr.use_clockid ==0 ) + data->time =native_sched_clock_from_tsc (pebs->tsc); +if (has_branch_stack (event)) data->br_stack = &cpuc->lbr_stack; }
@@ -1245,6 +1268,13 @@ void __init intel_ds_init(void) x86_pmu.drain_pebs = intel_pmu_drain_pebs_nhm;break ; +case 3 : +pr_cont ("PEBS fmt3%c, " , pebs_type); + x86_pmu.pebs_record_size = +sizeof (struct pebs_record_skl ); + x86_pmu.drain_pebs = intel_pmu_drain_pebs_nhm; +break ; +default :printk (KERN_CONT "no PEBS fmt%d%c, " , format, pebs_type); x86_pmu.pebs =0 ;
PATCH 3/9 x86: Add new MSRs and MSR bits used for Skylake perfmon
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add new MSRs (LBR_INFO) and some new MSR bits used by the Skylake PMU driver.
--- arch/x86/include/asm/perf_event.h | 7 +++++++ arch/x86/include/uapi/asm/msr-index.h | 6 ++++++ 2 files changed, 13 insertions(+)
arch/x86/include/asm/perf_event.h +7 -0
@@ -159,6 +159,13 @@ struct x86_pmu_capability {*/ #define INTEL_PMC_IDX_FIXED_BTS (INTEL_PMC_IDX_FIXED + 16) +#define GLOBAL_STATUS_COND_CHG BIT_ULL(63) +#define GLOBAL_STATUS_BUFFER_OVF BIT_ULL(62) +#define GLOBAL_STATUS_UNC_OVF BIT_ULL(61) +#define GLOBAL_STATUS_ASIF BIT_ULL(60) +#define GLOBAL_STATUS_COUNTERS_FROZEN BIT_ULL(59) +#define GLOBAL_STATUS_LBRS_FROZEN BIT_ULL(58) +/* * IBS cpuid feature detection */
arch/x86/include/uapi/asm/msr-index.h +6 -0
@@ -72,6 +72,12 @@#define MSR_LBR_CORE_FROM 0x00000040#define MSR_LBR_CORE_TO 0x00000060 +#define MSR_LBR_INFO_0 0x00000dc0/* ... 0xddf for _31 */ +#define LBR_INFO_MISPRED (1UL << 63) +#define LBR_INFO_IN_TX (1UL << 62) +#define LBR_INFO_ABORT (1UL << 61) +#define LBR_INFO_CYCLES 0xffff +#define MSR_IA32_PEBS_ENABLE 0x000003f1#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] x86: Add new MSRs and MSR bits used for Intel Skylake PMU support
x86: Add new MSRs and MSR bits used for Intel Skylake PMU support
Commit-ID: b83ff1c8617aac03a1cf807aafa848fe0f0908f2 Gitweb: http://git.kernel.org/tip/b83ff1c8617aac03a1cf807aafa848fe0f0908f2 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:41 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:56 +0200
x86: Add new MSRs and MSR bits used for Intel Skylake PMU support
Add new MSRs (LBR_INFO) and some new MSR bits used by the Intel Skylake PMU driver.
--- arch/x86/include/asm/msr-index.h | 6 ++++++ arch/x86/include/asm/perf_event.h | 7 +++++++ 2 files changed, 13 insertions(+)
arch/x86/include/asm/msr-index.h +6 -0
@@ -73,6 +73,12 @@#define MSR_LBR_CORE_FROM 0x00000040#define MSR_LBR_CORE_TO 0x00000060 +#define MSR_LBR_INFO_0 0x00000dc0/* ... 0xddf for _31 */ +#define LBR_INFO_MISPRED BIT_ULL(63) +#define LBR_INFO_IN_TX BIT_ULL(62) +#define LBR_INFO_ABORT BIT_ULL(61) +#define LBR_INFO_CYCLES 0xffff +#define MSR_IA32_PEBS_ENABLE 0x000003f1#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345
arch/x86/include/asm/perf_event.h +7 -0
@@ -159,6 +159,13 @@ struct x86_pmu_capability {*/ #define INTEL_PMC_IDX_FIXED_BTS (INTEL_PMC_IDX_FIXED + 16) +#define GLOBAL_STATUS_COND_CHG BIT_ULL(63) +#define GLOBAL_STATUS_BUFFER_OVF BIT_ULL(62) +#define GLOBAL_STATUS_UNC_OVF BIT_ULL(61) +#define GLOBAL_STATUS_ASIF BIT_ULL(60) +#define GLOBAL_STATUS_COUNTERS_FROZEN BIT_ULL(59) +#define GLOBAL_STATUS_LBRS_FROZEN BIT_ULL(58) +/* * IBS cpuid feature detection */
PATCH 4/9 perf: Add cycles to branch_info
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Skylake supports reporting the time in cycles a branch in the LBR took, to give a rough indication of the basic block performance.
Export the cycle information in the branch_info structure. This can be done by just reusing some currently zero padding.
This is just the generic header change. The architecture still needs to fill it in.
There's no attempt to convert to real time, as we really want cycles here.
--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/uapi/linux/perf_event.h +3 -1
@@ -904,6 +904,7 @@ union perf_mem_data_src {* * in_tx: running in a hardware transaction * abort: aborting a hardware transaction +* cycles: cycles from last branch (or 0 if not supported) */ struct perf_branch_entry {__u64 from;
@@ -912,7 +913,8 @@ struct perf_branch_entry { predicted:1,/* target predicted */ in_tx:1, /* in transaction */ abort:1, /* transaction abort */ - reserved:60; + cycles:16, /* cycle count to last branch */ + reserved:44; };#endif /* _UAPI_LINUX_PERF_EVENT_H */
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf: Add cycles to branch_info
perf: Add cycles to branch_info
Commit-ID: 71ef3c6b9d4665ee7afbbe4c208a98917dcfc32f Gitweb: http://git.kernel.org/tip/71ef3c6b9d4665ee7afbbe4c208a98917dcfc32f Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:42 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:57 +0200
perf: Add cycles to branch_info
Intel Skylake supports reporting the time in cycles a branch in the LBR took, to give a rough indication of the basic block performance.
Export the cycle information in the branch_info structure. This can be done by just reusing some currently zero padding.
This is just the generic header change. The architecture still needs to fill it in.
There's no attempt to convert to real time, as we really want cycles here.
--- include/uapi/linux/perf_event.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/uapi/linux/perf_event.h +3 -1
@@ -951,6 +951,7 @@ union perf_mem_data_src {* * in_tx: running in a hardware transaction * abort: aborting a hardware transaction +* cycles: cycles from last branch (or 0 if not supported) */ struct perf_branch_entry {__u64 from;
@@ -959,7 +960,8 @@ struct perf_branch_entry { predicted:1,/* target predicted */ in_tx:1, /* in transaction */ abort:1, /* transaction abort */ - reserved:60; + cycles:16, /* cycle count to last branch */ + reserved:44; };#endif /* _UAPI_LINUX_PERF_EVENT_H */
PATCH 5/9 x86, perf: Add support for LBRv5
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add support for the new LBRv5 format used on Skylake.
The flags for mispredict, abort, in_tx etc. moved to range of separate LBR_INFO_* MSRs. Teach the LBR code to read those. The original LBR registers stay the same, except they have full sign extension now.
LBR_INFO also reports a cycle count to the last branch. Report the cycle information using the new "cycles" branch_info output field.
In addition we have to context switch and clear the new INFO MSRs to avoid any information leaks.
--- arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel_lbr.c | 21 ++++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -595,6 +595,7 @@ struct x86_pmu {struct x86_perf_task_context {u64 lbr_from[MAX_LBR_ENTRIES ];u64 lbr_to[MAX_LBR_ENTRIES ]; +u64 lbr_info[MAX_LBR_ENTRIES ];int lbr_callstack_users;int lbr_stack_state; };
arch/x86/kernel/cpu/perf_event_intel_lbr.c +20 -1
@@ -13,7 +13,8 @@ enum {LBR_FORMAT_EIP =0x02 ,LBR_FORMAT_EIP_FLAGS =0x03 ,LBR_FORMAT_EIP_FLAGS2 =0x04 , - LBR_FORMAT_MAX_KNOWN = LBR_FORMAT_EIP_FLAGS2, +LBR_FORMAT_INFO =0x05 , + LBR_FORMAT_MAX_KNOWN = LBR_FORMAT_INFO, }; static enum {
@@ -184,6 +185,8 @@ static void intel_pmu_lbr_reset_64(void)for (i =0 ; i < x86_pmu.lbr_nr; i++) {wrmsrl (x86_pmu.lbr_from + i,0 );wrmsrl (x86_pmu.lbr_to + i,0 ); +if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_INFO ) +wrmsrl (MSR_LBR_INFO_0 + i,0 ); } }
@@ -232,6 +235,8 @@ static void __intel_pmu_lbr_restore(struct x86_perf_task_context *task_ctx) lbr_idx = (tos - i) & mask;wrmsrl (x86_pmu.lbr_from + lbr_idx, task_ctx->lbr_from[i]);wrmsrl (x86_pmu.lbr_to + lbr_idx, task_ctx->lbr_to[i]); +if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_INFO ) +wrmsrl (MSR_LBR_INFO_0 + i, task_ctx->lbr_info[i]); } task_ctx->lbr_stack_state =LBR_NONE ; }
@@ -253,6 +258,8 @@ static void __intel_pmu_lbr_save(struct x86_perf_task_context *task_ctx) lbr_idx = (tos - i) & mask;rdmsrl (x86_pmu.lbr_from + lbr_idx, task_ctx->lbr_from[i]);rdmsrl (x86_pmu.lbr_to + lbr_idx, task_ctx->lbr_to[i]); +if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_INFO ) +rdmsrl (MSR_LBR_INFO_0 + i, task_ctx->lbr_info[i]); } task_ctx->lbr_stack_state =LBR_VALID ; }
@@ -417,11 +424,22 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)unsigned long lbr_idx = (tos - i) & mask;u64 from, to, mis =0 , pred =0 , in_tx =0 , abort =0 ;int skip =0 ; +u16 cycles =0 ;int lbr_flags = lbr_desc[lbr_format];rdmsrl (x86_pmu.lbr_from + lbr_idx, from);rdmsrl (x86_pmu.lbr_to + lbr_idx, to); +if (lbr_format ==LBR_FORMAT_INFO ) { +u64 info; + +rdmsrl (MSR_LBR_INFO_0 + lbr_idx, info); + mis = !!(info &LBR_INFO_MISPRED ); + pred = !mis; + in_tx = !!(info &LBR_INFO_IN_TX ); + abort = !!(info &LBR_INFO_ABORT ); + cycles = (info &LBR_INFO_CYCLES ); + }if (lbr_flags &LBR_EIP_FLAGS ) { mis = !!(from &LBR_FROM_FLAG_MISPRED ); pred = !mis;
@@ -451,6 +469,7 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc) cpuc->lbr_entries[out].predicted = pred; cpuc->lbr_entries[out].in_tx = in_tx; cpuc->lbr_entries[out].abort = abort; + cpuc->lbr_entries[out].cycles = cycles; cpuc->lbr_entries[out].reserved =0 ; out++; }
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel/lbr: Add support for LBRv5
perf/x86/intel/lbr: Add support for LBRv5
Commit-ID: 50eab8f6ecd77ae4f9742f8e21ea50705ce0f830 Gitweb: http://git.kernel.org/tip/50eab8f6ecd77ae4f9742f8e21ea50705ce0f830 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:43 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:57 +0200
perf/x86/intel/lbr: Add support for LBRv5
Add support for the new LBRv5 format used on Intel Skylake CPUs.
The flags for mispredict, abort, in_tx etc. moved to range of separate LBR_INFO_* MSRs. Teach the LBR code to read those. The original LBR registers stay the same, except they have full sign extension now.
LBR_INFO also reports a cycle count to the last branch. Report the cycle information using the new "cycles" branch_info output field.
In addition we have to context switch and clear the new INFO MSRs to avoid any information leaks.
--- arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel_lbr.c | 21 ++++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -625,6 +625,7 @@ struct x86_pmu {struct x86_perf_task_context {u64 lbr_from[MAX_LBR_ENTRIES ];u64 lbr_to[MAX_LBR_ENTRIES ]; +u64 lbr_info[MAX_LBR_ENTRIES ];int lbr_callstack_users;int lbr_stack_state; };
arch/x86/kernel/cpu/perf_event_intel_lbr.c +20 -1
@@ -13,7 +13,8 @@ enum {LBR_FORMAT_EIP =0x02 ,LBR_FORMAT_EIP_FLAGS =0x03 ,LBR_FORMAT_EIP_FLAGS2 =0x04 , - LBR_FORMAT_MAX_KNOWN = LBR_FORMAT_EIP_FLAGS2, +LBR_FORMAT_INFO =0x05 , + LBR_FORMAT_MAX_KNOWN = LBR_FORMAT_INFO, }; static enum {
@@ -186,6 +187,8 @@ static void intel_pmu_lbr_reset_64(void)for (i =0 ; i < x86_pmu.lbr_nr; i++) {wrmsrl (x86_pmu.lbr_from + i,0 );wrmsrl (x86_pmu.lbr_to + i,0 ); +if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_INFO ) +wrmsrl (MSR_LBR_INFO_0 + i,0 ); } }
@@ -234,6 +237,8 @@ static void __intel_pmu_lbr_restore(struct x86_perf_task_context *task_ctx) lbr_idx = (tos - i) & mask;wrmsrl (x86_pmu.lbr_from + lbr_idx, task_ctx->lbr_from[i]);wrmsrl (x86_pmu.lbr_to + lbr_idx, task_ctx->lbr_to[i]); +if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_INFO ) +wrmsrl (MSR_LBR_INFO_0 + i, task_ctx->lbr_info[i]); } task_ctx->lbr_stack_state =LBR_NONE ; }
@@ -255,6 +260,8 @@ static void __intel_pmu_lbr_save(struct x86_perf_task_context *task_ctx) lbr_idx = (tos - i) & mask;rdmsrl (x86_pmu.lbr_from + lbr_idx, task_ctx->lbr_from[i]);rdmsrl (x86_pmu.lbr_to + lbr_idx, task_ctx->lbr_to[i]); +if (x86_pmu.intel_cap.lbr_format ==LBR_FORMAT_INFO ) +rdmsrl (MSR_LBR_INFO_0 + i, task_ctx->lbr_info[i]); } task_ctx->lbr_stack_state =LBR_VALID ; }
@@ -416,11 +423,22 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)unsigned long lbr_idx = (tos - i) & mask;u64 from, to, mis =0 , pred =0 , in_tx =0 , abort =0 ;int skip =0 ; +u16 cycles =0 ;int lbr_flags = lbr_desc[lbr_format];rdmsrl (x86_pmu.lbr_from + lbr_idx, from);rdmsrl (x86_pmu.lbr_to + lbr_idx, to); +if (lbr_format ==LBR_FORMAT_INFO ) { +u64 info; + +rdmsrl (MSR_LBR_INFO_0 + lbr_idx, info); + mis = !!(info &LBR_INFO_MISPRED ); + pred = !mis; + in_tx = !!(info &LBR_INFO_IN_TX ); + abort = !!(info &LBR_INFO_ABORT ); + cycles = (info &LBR_INFO_CYCLES ); + }if (lbr_flags &LBR_EIP_FLAGS ) { mis = !!(from &LBR_FROM_FLAG_MISPRED ); pred = !mis;
@@ -450,6 +468,7 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc) cpuc->lbr_entries[out].predicted = pred; cpuc->lbr_entries[out].in_tx = in_tx; cpuc->lbr_entries[out].abort = abort; + cpuc->lbr_entries[out].cycles = cycles; cpuc->lbr_entries[out].reserved =0 ; out++; }
PATCH 6/9 x86, perf: Add Skylake support
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
Add perf core PMU support for future Intel Skylake CPU cores.
The code is based on Haswell/Broadwell.
There is a new cache event list, based on the updated Haswell event list.
Skylake has removed most counter constraints on basic events, so the basic constraints table now only has a single entry (plus the fixed counters)
TSX support and various other setups are all shared with Haswell.
Skylake has 32 LBR entries. Add a new LBR init function to set this up. The filters are all the same as Haswell.
It also has a new LBR format with a separate LBR_INFO_* MSR, but that has been already added earlier.
--- arch/x86/kernel/cpu/perf_event.h | 6 +- arch/x86/kernel/cpu/perf_event_intel.c | 232 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event_intel_ds.c | 22 +++ arch/x86/kernel/cpu/perf_event_intel_lbr.c | 20 +++ 4 files changed, 279 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event.h +5 -1
@@ -138,7 +138,7 @@ struct intel_excl_cntrs {unsigned core_id;/* per-core: core id */ }; -#define MAX_LBR_ENTRIES 16 +#define MAX_LBR_ENTRIES 32enum {X86_PERF_KFREE_SHARED =0 ,
@@ -841,6 +841,8 @@ extern struct event_constraint intel_ivb_pebs_event_constraints[];extern struct event_constraint intel_hsw_pebs_event_constraints[]; +extern struct event_constraint intel_skl_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_enable (struct perf_event *event);
@@ -877,6 +879,8 @@ void intel_pmu_lbr_init_snb(void);void intel_pmu_lbr_init_hsw (void ); +void intel_pmu_lbr_init_skl (void ); +int intel_pmu_setup_lbr_filter (struct perf_event *event);void intel_pt_interrupt (void );
arch/x86/kernel/cpu/perf_event_intel.c +232 -0
256 lines not on this page · raw message
arch/x86/kernel/cpu/perf_event_intel_ds.c +22 -0
@@ -678,6 +678,28 @@ struct event_constraint intel_hsw_pebs_event_constraints[] = {EVENT_CONSTRAINT_END }; +struct event_constraint intel_skl_pebs_event_constraints[] = { +INTEL_FLAGS_UEVENT_CONSTRAINT (0x1c0 ,0x2 ),/* INST_RETIRED.PREC_DIST */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_NA (0x01c2 ,0xf ),/* UOPS_RETIRED.ALL */ +/* UOPS_RETIRED.ALL, inv=1, cmask=16 (cycles:p). */ +INTEL_FLAGS_EVENT_CONSTRAINT (0x108001c2 ,0xf ), +INTEL_PLD_CONSTRAINT (0x1cd ,0xf ),/* MEM_TRANS_RETIRED.* */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x12d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x21d0 ,0xf ),/* MEM_INST_RETIRED.LOCK_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x22d0 ,0xf ),/* MEM_INST_RETIRED.LOCK_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x41d0 ,0xf ),/* MEM_INST_RETIRED.SPLIT_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x42d0 ,0xf ),/* MEM_INST_RETIRED.SPLIT_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x81d0 ,0xf ),/* MEM_INST_RETIRED.ALL_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x82d0 ,0xf ),/* MEM_INST_RETIRED.ALL_STORES */ +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD (0xd1 ,0xf ),/* MEM_LOAD_RETIRED.* */ +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD (0xd2 ,0xf ),/* MEM_LOAD_L3_HIT_RETIRED.* */ +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD (0xd3 ,0xf ),/* MEM_LOAD_L3_MISS_RETIRED.* */ +/* Allow all events as PEBS with no flags */ +INTEL_ALL_EVENT_CONSTRAINT (0 ,0xf ), +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
arch/x86/kernel/cpu/perf_event_intel_lbr.c +20 -0
@@ -961,6 +961,26 @@ void intel_pmu_lbr_init_hsw(void)pr_cont ("16-deep LBR, " ); } +/* skylake */ +__init void intel_pmu_lbr_init_skl (void ) +{ + x86_pmu.lbr_nr =32 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_NHM_FROM ; + x86_pmu.lbr_to =MSR_LBR_NHM_TO ; + + x86_pmu.lbr_sel_mask =LBR_SEL_MASK ; + x86_pmu.lbr_sel_map = hsw_lbr_sel_map; + +/* +* SW branch filter usage: +* - support syscall, sysret capture. +* That requires LBR_FAR but that means far +* jmp need to be filtered out +*/ +pr_cont ("32-deep LBR, " ); +} +/* atom */ void __init intel_pmu_lbr_init_atom (void ) {
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel: Add Intel Skylake PMU support
perf/x86/intel: Add Intel Skylake PMU support
Commit-ID: 9a92e16fd7b4ccd9aabcbc4d42a3fb5f9a3cf4a1 Gitweb: http://git.kernel.org/tip/9a92e16fd7b4ccd9aabcbc4d42a3fb5f9a3cf4a1 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:44 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:58 +0200
perf/x86/intel: Add Intel Skylake PMU support
Add perf core PMU support for future Intel Skylake CPU cores.
The code is based on Haswell/Broadwell.
There is a new cache event list, based on the updated Haswell event list.
Skylake has removed most counter constraints on basic events, so the basic constraints table now only has a single entry (plus the fixed counters).
TSX support and various other setups are all shared with Haswell.
Skylake has 32 LBR entries. Add a new LBR init function to set this up. The filters are all the same as Haswell.
It also has a new LBR format with a separate LBR_INFO_* MSR, but that has been already added earlier.
--- arch/x86/kernel/cpu/perf_event.h | 6 +- arch/x86/kernel/cpu/perf_event_intel.c | 232 +++++++++++++++++++++++++++++ arch/x86/kernel/cpu/perf_event_intel_ds.c | 22 +++ arch/x86/kernel/cpu/perf_event_intel_lbr.c | 20 +++ 4 files changed, 279 insertions(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event.h +5 -1
@@ -165,7 +165,7 @@ struct intel_excl_cntrs {unsigned core_id;/* per-core: core id */ }; -#define MAX_LBR_ENTRIES 16 +#define MAX_LBR_ENTRIES 32enum {X86_PERF_KFREE_SHARED =0 ,
@@ -861,6 +861,8 @@ extern struct event_constraint intel_ivb_pebs_event_constraints[];extern struct event_constraint intel_hsw_pebs_event_constraints[]; +extern struct event_constraint intel_skl_pebs_event_constraints[]; +struct event_constraint *intel_pebs_constraints (struct perf_event *event);void intel_pmu_pebs_enable (struct perf_event *event);
@@ -899,6 +901,8 @@ void intel_pmu_lbr_init_snb(void);void intel_pmu_lbr_init_hsw (void ); +void intel_pmu_lbr_init_skl (void ); +int intel_pmu_setup_lbr_filter (struct perf_event *event);void intel_pt_interrupt (void );
arch/x86/kernel/cpu/perf_event_intel.c +232 -0
256 lines not on this page · raw message
arch/x86/kernel/cpu/perf_event_intel_ds.c +22 -0
@@ -688,6 +688,28 @@ struct event_constraint intel_hsw_pebs_event_constraints[] = {EVENT_CONSTRAINT_END }; +struct event_constraint intel_skl_pebs_event_constraints[] = { +INTEL_FLAGS_UEVENT_CONSTRAINT (0x1c0 ,0x2 ),/* INST_RETIRED.PREC_DIST */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_NA (0x01c2 ,0xf ),/* UOPS_RETIRED.ALL */ +/* UOPS_RETIRED.ALL, inv=1, cmask=16 (cycles:p). */ +INTEL_FLAGS_EVENT_CONSTRAINT (0x108001c2 ,0xf ), +INTEL_PLD_CONSTRAINT (0x1cd ,0xf ),/* MEM_TRANS_RETIRED.* */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x11d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x12d0 ,0xf ),/* MEM_INST_RETIRED.STLB_MISS_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x21d0 ,0xf ),/* MEM_INST_RETIRED.LOCK_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x22d0 ,0xf ),/* MEM_INST_RETIRED.LOCK_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x41d0 ,0xf ),/* MEM_INST_RETIRED.SPLIT_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x42d0 ,0xf ),/* MEM_INST_RETIRED.SPLIT_STORES */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_LD (0x81d0 ,0xf ),/* MEM_INST_RETIRED.ALL_LOADS */ +INTEL_FLAGS_UEVENT_CONSTRAINT_DATALA_ST (0x82d0 ,0xf ),/* MEM_INST_RETIRED.ALL_STORES */ +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD (0xd1 ,0xf ),/* MEM_LOAD_RETIRED.* */ +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD (0xd2 ,0xf ),/* MEM_LOAD_L3_HIT_RETIRED.* */ +INTEL_FLAGS_EVENT_CONSTRAINT_DATALA_LD (0xd3 ,0xf ),/* MEM_LOAD_L3_MISS_RETIRED.* */ +/* Allow all events as PEBS with no flags */ +INTEL_ALL_EVENT_CONSTRAINT (0 ,0xf ), +EVENT_CONSTRAINT_END +}; +struct event_constraint *intel_pebs_constraints (struct perf_event *event) {struct event_constraint *c;
arch/x86/kernel/cpu/perf_event_intel_lbr.c +20 -0
@@ -973,6 +973,26 @@ void intel_pmu_lbr_init_hsw(void)pr_cont ("16-deep LBR, " ); } +/* skylake */ +__init void intel_pmu_lbr_init_skl (void ) +{ + x86_pmu.lbr_nr =32 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_NHM_FROM ; + x86_pmu.lbr_to =MSR_LBR_NHM_TO ; + + x86_pmu.lbr_sel_mask =LBR_SEL_MASK ; + x86_pmu.lbr_sel_map = hsw_lbr_sel_map; + +/* +* SW branch filter usage: +* - support syscall, sysret capture. +* That requires LBR_FAR but that means far +* jmp need to be filtered out +*/ +pr_cont ("32-deep LBR, " ); +} +/* atom */ void __init intel_pmu_lbr_init_atom (void ) {
PATCH 7/9 x86, perf: Handle new arch perfmon v4 status bits
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
ArchPerfmon v4 has some new status bits in GLOBAL_STATUS. These need to be ignored when deciding whether a NMI was a NMI to avoid eating all NMIs when they stay set (see b292d7a104)
This patch ignores the new ASIF bit, which indicates that SGX interfered with the PMU, and also the new LBR freezing bits, which are set when the LBRs get frozen, plus the existing CondChange (set by JTAG debuggers and some buggy BIOSes)
--- arch/x86/kernel/cpu/perf_event_intel.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +7 -6
@@ -1821,13 +1821,14 @@ again:intel_pmu_lbr_read ();/* -* CondChgd bit 63 doesn't mean any overflow status. Ignore -* and clear the bit. +* Ignore a range of extra bits in status that do not indicate +* overflow by themselves. */ -if (__test_and_clear_bit (63 , (unsigned long *)&status)) { -if (!status) - goto done; - } + status &= ~(GLOBAL_STATUS_COND_CHG | +GLOBAL_STATUS_ASIF | +GLOBAL_STATUS_LBRS_FROZEN ); +if (!status) + goto done;/* * PEBS overflow sets bit 62 in the global status register
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel: Handle new arch perfmon v4 status bits
perf/x86/intel: Handle new arch perfmon v4 status bits
Commit-ID: d8020bee1d0caa90e7b9d6f39ac1fdfaaee7f67f Gitweb: http://git.kernel.org/tip/d8020bee1d0caa90e7b9d6f39ac1fdfaaee7f67f Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:45 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:57 +0200
perf/x86/intel: Handle new arch perfmon v4 status bits
ArchPerfmon v4 has some new status bits in GLOBAL_STATUS.
These need to be ignored when deciding whether a NMI was an NMI, to avoid eating all NMIs when they stay set, see:
b292d7a10487 ("perf/x86/intel: ignore CondChgd bit to avoid false NMI handling")
This patch ignores the new ASIF bit, which indicates that SGX interfered with the PMU, and also the new LBR freezing bits, which are set when the LBRs get frozen, plus the existing CondChange (set by JTAG debuggers and some buggy BIOSes)
--- arch/x86/kernel/cpu/perf_event_intel.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/perf_event_intel.c +7 -6
@@ -1611,13 +1611,14 @@ again:intel_pmu_lbr_read ();/* -* CondChgd bit 63 doesn't mean any overflow status. Ignore -* and clear the bit. +* Ignore a range of extra bits in status that do not indicate +* overflow by themselves. */ -if (__test_and_clear_bit (63 , (unsigned long *)&status)) { -if (!status) - goto done; - } + status &= ~(GLOBAL_STATUS_COND_CHG | +GLOBAL_STATUS_ASIF | +GLOBAL_STATUS_LBRS_FROZEN ); +if (!status) + goto done;/* * PEBS overflow sets bit 62 in the global status register
PATCH 8/9 x86, perf: Optimize v4 LBR unfreezing
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
In Arch perfmon v4 the GLOBAL_STATUS reset automatically unfreezes LBRs. So no need to do it manually in the LBR code. Add a check to skip it.
v2: Move test up to beginning of function.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +7 -0
@@ -139,6 +139,13 @@ static void __intel_pmu_lbr_enable(bool pmi)u64 debugctl, lbr_select =0 , orig_debugctl;/* +* No need to unfreeze manually, as v4 can do that as part +* of the GLOBAL_STATUS ack. +*/ +if (pmi && x86_pmu.version >=4 ) +return ; + +/* * No need to reprogram LBR_SELECT in a PMI, as it * did not change. */
Signature · 1 line
1.9.3
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel/lbr: Optimize v4 LBR unfreezing
perf/x86/intel/lbr: Optimize v4 LBR unfreezing
Commit-ID: 425507fa5f321bb5ce1b5eb57a9586e0cf0b9802 Gitweb: http://git.kernel.org/tip/425507fa5f321bb5ce1b5eb57a9586e0cf0b9802 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:46 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:58 +0200
perf/x86/intel/lbr: Optimize v4 LBR unfreezing
In Arch perfmon v4 the GLOBAL_STATUS reset automatically unfreezes LBRs. So no need to do it manually in the LBR code. Add a check to skip it.
v2: Move test up to beginning of function.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +7 -0
@@ -141,6 +141,13 @@ static void __intel_pmu_lbr_enable(bool pmi)u64 debugctl, lbr_select =0 , orig_debugctl;/* +* No need to unfreeze manually, as v4 can do that as part +* of the GLOBAL_STATUS ack. +*/ +if (pmi && x86_pmu.version >=4 ) +return ; + +/* * No need to reprogram LBR_SELECT in a PMI, as it * did not change. */
PATCH 9/9 x86, perf: Move PMU ACK after LBR read
In reply to Andi KleenFrom: Andi Kleen <[email protected]>
With Arch Perfmon v4 the PMU ack unfreezes the LBRs. So we need to do the PMU ack after the LBR reading, otherwise the LBRs would be polluted by the PMI handler.
This is a minimal change. In principle the ACK could be moved much later.
--- arch/x86/kernel/cpu/perf_event_intel.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event_intel.c +1 -1
@@ -1804,6 +1804,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) loops =0 ; again: +intel_pmu_lbr_read ();intel_pmu_ack_status (status);if (++loops >100 ) {static bool warned = false;
@@ -1818,7 +1819,6 @@ again:inc_irq_stat (apic_perf_irqs); -intel_pmu_lbr_read ();/* * Ignore a range of extra bits in status that do not indicate
Signature · 1 line
1.9.3
Peter Zijlstra
maintainerRe: [PATCH 9/9] x86, perf: Move PMU ACK after LBR read
Quoted 5 lines · Andi Kleen
From: Andi Kleen <[email protected]>
With Arch Perfmon v4 the PMU ack unfreezes the LBRs. So we need to do the PMU ack after the LBR reading, otherwise the LBRs would be polluted by the PMI handler.
Hmm, we should move these last three patches before the SKL enablement patch no, otherwise things will misbehave -- say a bisection lands in between.
/me shuffles patches.
Quoted 1 line
This is a minimal change. In principle the ACK could be moved much later.
Right, so the more complete change would be to use the new and improved FREEZE_ON_PMI and reenable both the LBRs and the CTRs with the STATUS_RESET MSR, right?
Does it make sense to have a new handle_irq() routine for that?
Re: [PATCH 9/9] x86, perf: Move PMU ACK after LBR read
Quoted 13 lines · Peter Zijlstra
Quoted 5 lines · Andi Kleen
From: Andi Kleen <[email protected]>
With Arch Perfmon v4 the PMU ack unfreezes the LBRs. So we need to do the PMU ack after the LBR reading, otherwise the LBRs would be polluted by the PMI handler.
Hmm, we should move these last three patches before the SKL enablement patch no, otherwise things will misbehave -- say a bisection lands in between.
/me shuffles patches.
Quoted 7 lines
Quoted 1 line
This is a minimal change. In principle the ACK could be moved much later.
Right, so the more complete change would be to use the new and improved
FREEZE_ON_PMIand reenable both the LBRs and the CTRs with theSTATUS_RESETMSR, right?Does it make sense to have a new
handle_irq()routine for that?
Were we not already using FREEZE_ON_PMI with LBR (except for one erratum on HSW)?
It would make sense to me to have an "optimized" and clean handle_irq for the newer PMU.
We the caveat that any change to the core of it would now have to be done twice.
Peter Zijlstra
maintainerRe: [PATCH 9/9] x86, perf: Move PMU ACK after LBR read
Quoted 10 lines · Stephane Eranian
Quoted 7 lines
Quoted 1 line
This is a minimal change. In principle the ACK could be moved much later.
Right, so the more complete change would be to use the new and improved
FREEZE_ON_PMIand reenable both the LBRs and the CTRs with theSTATUS_RESETMSR, right?Does it make sense to have a new
handle_irq()routine for that?Were we not already using
FREEZE_ON_PMIwith LBR (except for one erratum on HSW)?
That's FREEZE_LBRS_ON_PMI, I was referring to FREEZE_PERFMON_ON_PMI, which we've not used so far.
I think Andi tried using it before, but there's some issues with it on v3, but v4 should have fixed all that.
Andi can you perhaps explain what the problem with FREEZE_PERFMON_ON_PMI on v3 was again?
Quoted 3 lines
It would make sense to me to have an "optimized" and clean
handle_irqfor the newer PMU.
We the caveat that any change to the core of it would now have to be done twice.
We could pull that out in a shared function of course, if possible.
Re: [PATCH 9/9] x86, perf: Move PMU ACK after LBR read
Quoted 7 lines · Peter Zijlstra
Quoted 6 lines · Stephane Eranian
Quoted 6 lines
Quoted 1 line
This is a minimal change. In principle the ACK could be moved much later.
Right, so the more complete change would be to use the new and improved
FREEZE_ON_PMIand reenable both the LBRs and the CTRs with theSTATUS_RESETMSR, right?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Does it make sense to have a new
handle_irq()routine for that?Were we not already using
FREEZE_ON_PMIwith LBR (except for one erratum on HSW)?That's
FREEZE_LBRS_ON_PMI, I was referring toFREEZE_PERFMON_ON_PMI, which we've not used so far.
Ah, yes that one was not used so far. I don't quite remember why.
I think with PEBS, you don't need it or it should be off or something like this.
Quoted 3 lines
I think Andi tried using it before, but there's some issues with it on v3, but v4 should have fixed all that.
I was referring to a LBR issue on v3 (HSW) and call stack mode.
Quoted 3 lines
Andi can you perhaps explain what the problem with
FREEZE_PERFMON_ON_PMIon v3 was again?
Andi, Is that what I am alluding to above?
Quoted 5 lines
Quoted 3 lines
It would make sense to me to have an "optimized" and clean
handle_irqfor the newer PMU.
We the caveat that any change to the core of it would now have to be done twice.We could pull that out in a shared function of course, if possible.
Good.
Re: [PATCH 9/9] x86, perf: Move PMU ACK after LBR read
Quoted 10 lines · Stephane Eranian
Quoted 9 lines · Peter Zijlstra
Quoted 8 lines · Stephane Eranian
Quoted 7 lines
Quoted 1 line
This is a minimal change. In principle the ACK could be moved much later.
Right, so the more complete change would be to use the new and improved
FREEZE_ON_PMIand reenable both the LBRs and the CTRs with theSTATUS_RESETMSR, right?Does it make sense to have a new
handle_irq()routine for that?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Were we not already using
FREEZE_ON_PMIwith LBR (except for one erratum on HSW)?That's
FREEZE_LBRS_ON_PMI, I was referring toFREEZE_PERFMON_ON_PMI, which we've not used so far.Ah, yes that one was not used so far. I don't quite remember why.
I think with PEBS, you don't need it or it should be off or something like this.
The LBR freeze should work with call stack mode if you use PEBS events. So in theory we could allow call-stack lbr for kernel with such a restriction. But we have a working kernel backtrace anyways, so it's not really critical.
Quoted 8 lines
Quoted 4 lines
I think Andi tried using it before, but there's some issues with it on v3, but v4 should have fixed all that.
I was referring to a LBR issue on v3 (HSW) and call stack mode.
Quoted 2 lines
Andi can you perhaps explain what the problem with
FREEZE_PERFMON_ON_PMIon v3 was again?
I'm not sure why we never used it.
The update of the DEBUGCTL can race with other operations, but it's rather obscure.
But I would not do it right now.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only
tip-bot for Andi Kleen
merge bot[tip:perf/core] perf/x86/intel: Move PMU ACK to after LBR read
In reply to Andi Kleenperf/x86/intel: Move PMU ACK to after LBR read
Commit-ID: 0f29e573dd32bb8598e74271454e97c962da5e05 Gitweb: http://git.kernel.org/tip/0f29e573dd32bb8598e74271454e97c962da5e05 Author: Andi Kleen <[email protected]> AuthorDate: Sun, 10 May 2015 12:22:47 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 4 Aug 2015 10:16:58 +0200
perf/x86/intel: Move PMU ACK to after LBR read
With Arch Perfmon v4 the PMU ack unfreezes the LBRs. So we need to do the PMU ack after the LBR reading, otherwise the LBRs would be polluted by the PMI handler.
This is a minimal change. In principle the ACK could be moved much later.
--- arch/x86/kernel/cpu/perf_event_intel.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/cpu/perf_event_intel.c +1 -1
@@ -1594,6 +1594,7 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) loops =0 ; again: +intel_pmu_lbr_read ();intel_pmu_ack_status (status);if (++loops >100 ) {static bool warned = false;
@@ -1608,7 +1609,6 @@ again:inc_irq_stat (apic_perf_irqs); -intel_pmu_lbr_read ();/* * Ignore a range of extra bits in status that do not indicate