perf: add support for sampling taken branches
Subsystems (9)
v6 cover perf: add support for sampling taken branches
This patchset adds an important and useful new feature toperf_events: branch stack sampling. In other words, the ability to capture taken branches into each sample.
Statistical sampling of taken branch should not be confused for branch tracing. Not all branches are necessarily captured
Sampling taken branches is important for basic block profiling, statistical call graph, function call counts. Many of those measurements can help drive a compiler optimizer.
The branch stack is a software abstraction which sits on top of the PMU hardware. As such, it is not available on all processors. For now, the patch provides the generic interface and the Intel X86 implementation where it leverages the Last Branch Record (LBR) feature (from Core2 to SandyBridge).
Branch stack sampling is supported for both per-thread and system-wide modes.
It is possible to filter the type and privilege level of branches to sample. The target of the branch is used to determine the privilege level.
For each branch, the source and destination are captured. On some hardware platforms, it may be possible to also extract the target prediction and, in that case, it is also exposed to end users.
The branch stack can record a variable number of taken branches per sample. Those branches are always consecutive in time. The number of branches captured depends on the filtering and the underlying hardware. On Intel Nehalem and later, up to 16 consecutive branches can be captured per sample.
Branch sampling is always coupled with an event. It can be any PMU event but it can't be a SW or tracepoint event.
Branch sampling is requested by setting a new sample_type flag called: PERF_SAMPLE_BRANCH_STACK.
To support branch filtering, we introduce a new field to the perf_event_attr struct: branch_sample_type. We chose NOT to overload the config1, config2 field because those are related to the event encoding. Branch stack is a separate feature which is combined with the event.
The branch_sample_type is a bitmask of possible filters. The following filters are defined (more can be added):
- PERF_SAMPLE_BRANCH_ANY : any control flow change - PERF_SAMPLE_BRANCH_USER : branches when target is at user level - PERF_SAMPLE_BRANCH_KERNEL : branches when target is at kernel level - PERF_SAMPLE_BRANCH_HV : branches when target is at hypervisor level
- PERF_SAMPLE_BRANCH_ANY_CALL: call branches (incl. syscalls)
- PERF_SAMPLE_BRANCH_ANY_RET : return branches (incl. syscall returns)
- PERF_SAMPLE_BRANCH_IND_CALL: indirect calls
It is possible to combine filters, e.g., IND_CALL|USER|KERNEL.
When the privilege level is not specified, the branch stack inherits that of the associated event.
Some processors may not offer hardware branch filtering, e.g., Intel Atom. Some may have HW filtering bugs (e.g., Nehalem). The Intel X86 implementation in this patchset also provides a SW branch filter which works on a best effort basis. It can compensate for the lack of LBR filtering. But first and foremost, it helps work around LBR filtering errata. The goal is to only capture the type of branches requested by the user.
It is possible to combine branch stack sampling with PEBS on Intel X86 processors. Depending on the precise_sampling mode, there are certain filterting restrictions. When precise_sampling=1, then there are no filtering restrictions. When precise_sampling > 1, then only ANY|USER|KERNEL filter can be used. This comes from the fact that the kernel uses LBR to compensate for the PEBS off-by-1 skid on the instruction pointer.
To demonstrate how the perf_event branch stack sampling interface works, the patchset also modifies perf record to capture taken branches. Similarly perf report is enhanced to display a histogram of taken branches.
I would like to thank Roberto Vitillo @ LBL for his work on the perf tool for this.
Enough talking, let's take a simple example. Our trivial test program goes like this:
void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}int main(void)
{
unsigned long i;
for (i=0; i < N; i++)
f1(i);
return 0;
}
$ perf record -b any branchy $ perf report -b # Events: 23K cycles # # Overhead Source Symbol Target Symbol # ........ ................ ................
18.13% [.] f1 [.] main
18.10% [.] main [.] main
18.01% [.] main [.] f1
15.69% [.] f1 [.] f1
9.11% [.] f3 [.] f1
6.78% [.] f1 [.] f3
6.74% [.] f1 [.] f2
6.71% [.] f2 [.] f1 Of the total number of branches captured, 18.13% were from f1() -> main().
Let's make this clearer by filtering the user call branches only:
$ perf record -b any_call -e cycles:u branchy
$ perf report -b
# Events: 19K cycles
#
# Overhead Source Symbol Target Symbol
# ........ ......................... .........................
#
52.50% [.] main [.] f1
23.99% [.] f1 [.] f3
23.48% [.] f1 [.] f2
0.03% [.] _IO_default_xsputn [.] _IO_new_file_overflow
0.01% [k] _start [k] __libc_start_main Now it is more obvious. %52 of all the captured branches where calls from main() -> f1(). The rest is split 50/50 between f1() -> f2() and f1() -> f3() which is expected given that f1() dispatches based on odd vs. even values of n which is constantly increasing.
Here is a kernel example, where we want to sample indirect calls:
$ perf record -a -C 1 -b ind_call -e r1c4:k sleep 10
$ perf report -b
#
# Overhead Source Symbol Target Symbol
# ........ .......................... ..........................
#
36.36% [k] __delay [k] delay_tsc
9.09% [k] ktime_get [k] read_tsc
9.09% [k] getnstimeofday [k] read_tsc
9.09% [k] notifier_call_chain [k] tick_notify
4.55% [k] cpuidle_idle_call [k] intel_idle
4.55% [k] cpuidle_idle_call [k] menu_reflect
2.27% [k] handle_irq [k] handle_edge_irq
2.27% [k] ack_apic_edge [k] native_apic_mem_write
2.27% [k] hpet_interrupt_handler [k] hrtimer_interrupt
2.27% [k] __run_hrtimer [k] watchdog_timer_fn
2.27% [k] enqueue_task [k] enqueue_task_rt
2.27% [k] try_to_wake_up [k] select_task_rq_rt
2.27% [k] do_timer [k] read_tsc Due to HW limitations, branch filtering may be approximate on Core, Atom processors. It is more accurate on Nehalem, Westmere and best on Sandy Bridge.
In version 2, we've updated the patch to tip/master (commit 5734857) and we've incoporated the feedback from v1 concerning anynous bitfield struct for branch_stack_entry and the hanlding of i386 ABI binaries on 64-bit host in the instr decoder for the LBR SW filter.
In version 3, we've updated to 3.2.0-tip. The Atom revision check has been put into its own patch. We fixed a browser issue with report report. We fixed all the style issues as well.
In version 4, we've modified the branch stack API to add a missing priv level : hypervisor. There is a new PERF_SAMPLE_BRANCH_HV. It is not used on Intel X86. Thanks to [email protected] for pointing this out. We also fix compilation error on ARM.
In version 4, we also extend the patch to include the changes necessary to the perf tool to support reading perf.data files which were produced from older perf_event ABI revisions. This patch set extends the ABI with a new field in struct perf_event_attr. That struct is saved as is in the perf.data file. Therefore, older perf.data files contain smaller perf_event_attr struct, yet perf must process them transparently. That's not the case today. It dies with 'incompatible file format'.
The patch solves this problem and, at the same time, decouples endianness detection from the size of perf_event_attr. Endianness is now detected via the signature (the first 8 bytes of the file). We introduce a new signature (PERFILE2). It is not laid out the same way in the file based on the endianness of the host where the file is written. Therefore, we can dynamically detect the endianness by simply reading the first 8 bytes. The size of the perf_event_attr struct can then be processed according to the endianness. The ambiguity between the size being at the same time, the endianness marker and the actual size is gone. We can now distinguish an older ABI by the size and not confuse it with an endianness mismatch.
In version 5, we fix the PEBS+LBR vs. BRANCH_STACK check in x86_pmu_hw_config. We also changed the handling of PERF_SAMPLE_BRANCH_HV on X86. It is now ignored instead of triggering an error. That enables: perf record -b any -e cycles, without having to force a priv level on the branch type. We also fix an uninitialized variable bug in the perf tool reported by reviewers. Thanks to Anshuman Khandual for his comments.
In version 6, we have fixed several issues in the perf tool code and especially in patch 11. We have fully implemented the --sort option on the branch source and target. We have fixed several column alignment issues. We have integrated feedback from David Ahern, concerning patch 16 and the ability to read perf.data files written from a different ABI. Perf will now reject any perf.data file that has a larger perf_event_attr size. Perf provides compatibility backward and not forward.
Roberto Agostino Vitillo (3):
perf: add code to support PERF_SAMPLE_BRANCH_STACK
perf: add support for sampling taken branch to perf record
perf: add support for taken branch sampling to perf report
Stephane Eranian (15):
perf: add generic taken branch sampling support
perf: add Intel LBR MSR definitions
perf: add Intel X86 LBR sharing logic
perf: sync branch stack sampling with X86 precise_sampling
perf: add Intel X86 LBR mappings for PERF_SAMPLE_BRANCH filters
perf: disable LBR support for older Intel Atom processors
perf: implement PERF_SAMPLE_BRANCH for Intel X86
perf: add LBR software filter support for Intel X86
perf: disable PERF_SAMPLE_BRANCH_* when not supported
perf: add hook to flush branch_stack on context switch
perf: fix endianness detection in perf.data
perf: add ABI reference sizes
perf: enable reading of perf.data files from different ABI rev
perf: fix bug print_event_desc()
perf: make perf able to read file from older ABIs
arch/alpha/kernel/perf_event.c | 4 + arch/arm/kernel/perf_event.c | 4 + arch/mips/kernel/perf_event_mipsxx.c | 4 + arch/powerpc/kernel/perf_event.c | 4 + arch/sh/kernel/perf_event.c | 4 + arch/sparc/kernel/perf_event.c | 4 + arch/x86/include/asm/msr-index.h | 7 + arch/x86/kernel/cpu/perf_event.c | 85 ++++- arch/x86/kernel/cpu/perf_event.h | 19 + arch/x86/kernel/cpu/perf_event_amd.c | 3 + arch/x86/kernel/cpu/perf_event_intel.c | 120 +++++-- arch/x86/kernel/cpu/perf_event_intel_ds.c | 22 +- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 526 ++++++++++++++++++++++++++-- include/linux/perf_event.h | 82 ++++- kernel/events/core.c | 177 ++++++++++ kernel/events/hw_breakpoint.c | 6 + tools/perf/Documentation/perf-record.txt | 25 ++ tools/perf/Documentation/perf-report.txt | 7 + tools/perf/builtin-record.c | 74 ++++ tools/perf/builtin-report.c | 98 +++++- tools/perf/perf.h | 18 + tools/perf/util/annotate.c | 2 +- tools/perf/util/event.h | 1 + tools/perf/util/evsel.c | 14 + tools/perf/util/header.c | 230 +++++++++++-- tools/perf/util/hist.c | 93 ++++- tools/perf/util/hist.h | 7 + tools/perf/util/session.c | 72 ++++ tools/perf/util/session.h | 4 + tools/perf/util/sort.c | 362 ++++++++++++++----- tools/perf/util/sort.h | 5 + tools/perf/util/symbol.h | 13 + 32 files changed, 1866 insertions(+), 230 deletions(-)
Signature · 1 line
1.7.4.1
PATCH v6 1/18 perf: add generic taken branch sampling support
This patch adds the ability to sample taken branches to the perf_event interface.
The ability to capture taken branches is very useful for all sorts of analysis. For instance, basic block profiling, call counts, statistical call graph.
This new capability requires hardware assist and as such may not be available on all HW platforms. On Intel X86, it is implemented on top of the Last Branch Record (LBR) facility.
To enable taken branches sampling, the PERF_SAMPLE_BRANCH_STACK bit must be set in attr->sample_type.
Sampled taken branches may be filtered by type and/or priv levels.
The patch adds a new field, called branch_sample_type, to the perf_event_attr structure. It contains a bitmask of filters to apply to the sampled taken branches.
Filters may be implemented in HW. If the HW filter does not exist or is not good enough, some arch may also implement a SW filter.
The following generic filters are currently defined:
- PERF_SAMPLE_USER
only branches whose targets are at the user level
- PERF_SAMPLE_KERNEL
only branches whose targets are at the kernel level
- PERF_SAMPLE_HV
only branches whose targets are at the hypervisor level
- PERF_SAMPLE_ANY
any type of branches (subject to priv levels filters)
- PERF_SAMPLE_ANY_CALL
any call branches (may incl. syscall on some arch)
- PERF_SAMPLE_ANY_RET
any return branches (may incl. syscall returns on some arch)
- PERF_SAMPLE_IND_CALL
indirect call branches
Obviously filter may be combined. The priv level bits are optional. If not provided, the priv level of the associated event are used. It is possible to collect branches at a priv level different from the associated event. Use of kernel, hv priv levels is subject to permissions and availability (hv).
The number of taken branch records present in each sample may vary based on HW, the type of sampled branches, the executed code. Therefore each sample contains the number of taken branches it contains.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 21 +++++--- include/linux/perf_event.h | 71 ++++++++++++++++++++++++++-- kernel/events/core.c | 68 ++++++++++++++++++++++++++ 3 files changed, 148 insertions(+), 12 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +13 -8
@@ -142,9 +142,11 @@ static void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)rdmsrl (x86_pmu.lbr_from + lbr_idx, msr_lastbranch.lbr); - cpuc->lbr_entries[i].from = msr_lastbranch.from; - cpuc->lbr_entries[i].to = msr_lastbranch.to; - cpuc->lbr_entries[i].flags =0 ; + cpuc->lbr_entries[i].from = msr_lastbranch.from; + cpuc->lbr_entries[i].to = msr_lastbranch.to; + cpuc->lbr_entries[i].mispred =0 ; + cpuc->lbr_entries[i].predicted =0 ; + cpuc->lbr_entries[i].reserved =0 ; } cpuc->lbr_stack.nr = i; }
@@ -165,19 +167,22 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)for (i =0 ; i < x86_pmu.lbr_nr; i++) {unsigned long lbr_idx = (tos - i) & mask; -u64 from, to, flags =0 ; +u64 from, to, mis =0 , pred =0 ;rdmsrl (x86_pmu.lbr_from + lbr_idx, from);rdmsrl (x86_pmu.lbr_to + lbr_idx, to);if (lbr_format ==LBR_FORMAT_EIP_FLAGS ) { - flags = !!(from &LBR_FROM_FLAG_MISPRED ); + mis = !!(from &LBR_FROM_FLAG_MISPRED ); + pred = !mis; from = (u64)((((s64 )from) <<1 ) >>1 ); } - cpuc->lbr_entries[i].from = from; - cpuc->lbr_entries[i].to = to; - cpuc->lbr_entries[i].flags = flags; + cpuc->lbr_entries[i].from = from; + cpuc->lbr_entries[i].to = to; + cpuc->lbr_entries[i].mispred = mis; + cpuc->lbr_entries[i].predicted = pred; + cpuc->lbr_entries[i].reserved =0 ; } cpuc->lbr_stack.nr = i; }
include/linux/perf_event.h +67 -4
127 lines not on this page · raw message
kernel/events/core.c +68 -0
92 lines not on this page · raw message
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf: Add generic taken branch sampling support
Commit-ID: bce38cd53e5ddba9cb6d708c4ef3d04a4016ec7e Gitweb: http://git.kernel.org/tip/bce38cd53e5ddba9cb6d708c4ef3d04a4016ec7e Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:51 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:39 +0100
perf: Add generic taken branch sampling support
This patch adds the ability to sample taken branches to the perf_event interface.
The ability to capture taken branches is very useful for all sorts of analysis. For instance, basic block profiling, call counts, statistical call graph.
This new capability requires hardware assist and as such may not be available on all HW platforms. On Intel x86 it is implemented on top of the Last Branch Record (LBR) facility.
To enable taken branches sampling, the PERF_SAMPLE_BRANCH_STACK bit must be set in attr->sample_type.
Sampled taken branches may be filtered by type and/or priv levels.
The patch adds a new field, called branch_sample_type, to the perf_event_attr structure. It contains a bitmask of filters to apply to the sampled taken branches.
Filters may be implemented in HW. If the HW filter does not exist or is not good enough, some arch may also implement a SW filter.
The following generic filters are currently defined:
- PERF_SAMPLE_USER
only branches whose targets are at the user level
- PERF_SAMPLE_KERNEL
only branches whose targets are at the kernel level
- PERF_SAMPLE_HV
only branches whose targets are at the hypervisor level
- PERF_SAMPLE_ANY
any type of branches (subject to priv levels filters)
- PERF_SAMPLE_ANY_CALL
any call branches (may incl. syscall on some arch)
- PERF_SAMPLE_ANY_RET
any return branches (may incl. syscall returns on some arch)
- PERF_SAMPLE_IND_CALL
indirect call branches
Obviously filter may be combined. The priv level bits are optional. If not provided, the priv level of the associated event are used. It is possible to collect branches at a priv level different from the associated event. Use of kernel, hv priv levels is subject to permissions and availability (hv).
The number of taken branch records present in each sample may vary based on HW, the type of sampled branches, the executed code. Therefore each sample contains the number of taken branches it contains.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 21 +++++--- include/linux/perf_event.h | 71 ++++++++++++++++++++++++++-- kernel/events/core.c | 68 ++++++++++++++++++++++++++ 3 files changed, 148 insertions(+), 12 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +13 -8
@@ -142,9 +142,11 @@ static void intel_pmu_lbr_read_32(struct cpu_hw_events *cpuc)rdmsrl (x86_pmu.lbr_from + lbr_idx, msr_lastbranch.lbr); - cpuc->lbr_entries[i].from = msr_lastbranch.from; - cpuc->lbr_entries[i].to = msr_lastbranch.to; - cpuc->lbr_entries[i].flags =0 ; + cpuc->lbr_entries[i].from = msr_lastbranch.from; + cpuc->lbr_entries[i].to = msr_lastbranch.to; + cpuc->lbr_entries[i].mispred =0 ; + cpuc->lbr_entries[i].predicted =0 ; + cpuc->lbr_entries[i].reserved =0 ; } cpuc->lbr_stack.nr = i; }
@@ -165,19 +167,22 @@ static void intel_pmu_lbr_read_64(struct cpu_hw_events *cpuc)for (i =0 ; i < x86_pmu.lbr_nr; i++) {unsigned long lbr_idx = (tos - i) & mask; -u64 from, to, flags =0 ; +u64 from, to, mis =0 , pred =0 ;rdmsrl (x86_pmu.lbr_from + lbr_idx, from);rdmsrl (x86_pmu.lbr_to + lbr_idx, to);if (lbr_format ==LBR_FORMAT_EIP_FLAGS ) { - flags = !!(from &LBR_FROM_FLAG_MISPRED ); + mis = !!(from &LBR_FROM_FLAG_MISPRED ); + pred = !mis; from = (u64)((((s64 )from) <<1 ) >>1 ); } - cpuc->lbr_entries[i].from = from; - cpuc->lbr_entries[i].to = to; - cpuc->lbr_entries[i].flags = flags; + cpuc->lbr_entries[i].from = from; + cpuc->lbr_entries[i].to = to; + cpuc->lbr_entries[i].mispred = mis; + cpuc->lbr_entries[i].predicted = pred; + cpuc->lbr_entries[i].reserved =0 ; } cpuc->lbr_stack.nr = i; }
include/linux/perf_event.h +67 -4
127 lines not on this page · raw message
kernel/events/core.c +68 -0
92 lines not on this page · raw message
PATCH v6 2/18 perf: add Intel LBR MSR definitions
In reply to Stephane EranianThis patch adds the LBR definitions for NHM/WSM/SNB and Core. It also adds the definitions for the architected LBR MSR:LBR_SELECT, LBRT_TOS.
--- arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kernel/cpu/perf_event_intel_lbr.c | 18 +++++++++--------- 2 files changed, 16 insertions(+), 9 deletions(-)
arch/x86/include/asm/msr-index.h +7 -0
@@ -56,6 +56,13 @@#define MSR_OFFCORE_RSP_0 0x000001a6#define MSR_OFFCORE_RSP_1 0x000001a7 +#define MSR_LBR_SELECT 0x000001c8 +#define MSR_LBR_TOS 0x000001c9 +#define MSR_LBR_NHM_FROM 0x00000680 +#define MSR_LBR_NHM_TO 0x000006c0 +#define MSR_LBR_CORE_FROM 0x00000040 +#define MSR_LBR_CORE_TO 0x00000060 +#define MSR_IA32_PEBS_ENABLE 0x000003f1#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345
arch/x86/kernel/cpu/perf_event_intel_lbr.c +9 -9
@@ -203,23 +203,23 @@ void intel_pmu_lbr_read(void)void intel_pmu_lbr_init_core (void ) { x86_pmu.lbr_nr =4 ; - x86_pmu.lbr_tos =0x01c9 ; - x86_pmu.lbr_from =0x40 ; - x86_pmu.lbr_to =0x60 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_CORE_FROM ; + x86_pmu.lbr_to =MSR_LBR_CORE_TO ; }void intel_pmu_lbr_init_nhm (void ) { x86_pmu.lbr_nr =16 ; - x86_pmu.lbr_tos =0x01c9 ; - x86_pmu.lbr_from =0x680 ; - x86_pmu.lbr_to =0x6c0 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_NHM_FROM ; + x86_pmu.lbr_to =MSR_LBR_NHM_TO ; }void intel_pmu_lbr_init_atom (void ) { x86_pmu.lbr_nr =8 ; - x86_pmu.lbr_tos =0x01c9 ; - x86_pmu.lbr_from =0x40 ; - x86_pmu.lbr_to =0x60 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_CORE_FROM ; + x86_pmu.lbr_to =MSR_LBR_CORE_TO ; }
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Add Intel LBR MSR definitions
Commit-ID: 225ce53910edc3c2322b1e4f2ed049a9196cd0b3 Gitweb: http://git.kernel.org/tip/225ce53910edc3c2322b1e4f2ed049a9196cd0b3 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:52 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:39 +0100
perf/x86: Add Intel LBR MSR definitions
This patch adds the LBR definitions for NHM/WSM/SNB and Core. It also adds the definitions for the architected LBR MSR:LBR_SELECT, LBRT_TOS.
--- arch/x86/include/asm/msr-index.h | 7 +++++++ arch/x86/kernel/cpu/perf_event_intel_lbr.c | 18 +++++++++--------- 2 files changed, 16 insertions(+), 9 deletions(-)
arch/x86/include/asm/msr-index.h +7 -0
@@ -56,6 +56,13 @@#define MSR_OFFCORE_RSP_0 0x000001a6#define MSR_OFFCORE_RSP_1 0x000001a7 +#define MSR_LBR_SELECT 0x000001c8 +#define MSR_LBR_TOS 0x000001c9 +#define MSR_LBR_NHM_FROM 0x00000680 +#define MSR_LBR_NHM_TO 0x000006c0 +#define MSR_LBR_CORE_FROM 0x00000040 +#define MSR_LBR_CORE_TO 0x00000060 +#define MSR_IA32_PEBS_ENABLE 0x000003f1#define MSR_IA32_DS_AREA 0x00000600#define MSR_IA32_PERF_CAPABILITIES 0x00000345
arch/x86/kernel/cpu/perf_event_intel_lbr.c +9 -9
@@ -203,23 +203,23 @@ void intel_pmu_lbr_read(void)void intel_pmu_lbr_init_core (void ) { x86_pmu.lbr_nr =4 ; - x86_pmu.lbr_tos =0x01c9 ; - x86_pmu.lbr_from =0x40 ; - x86_pmu.lbr_to =0x60 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_CORE_FROM ; + x86_pmu.lbr_to =MSR_LBR_CORE_TO ; }void intel_pmu_lbr_init_nhm (void ) { x86_pmu.lbr_nr =16 ; - x86_pmu.lbr_tos =0x01c9 ; - x86_pmu.lbr_from =0x680 ; - x86_pmu.lbr_to =0x6c0 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_NHM_FROM ; + x86_pmu.lbr_to =MSR_LBR_NHM_TO ; }void intel_pmu_lbr_init_atom (void ) { x86_pmu.lbr_nr =8 ; - x86_pmu.lbr_tos =0x01c9 ; - x86_pmu.lbr_from =0x40 ; - x86_pmu.lbr_to =0x60 ; + x86_pmu.lbr_tos =MSR_LBR_TOS ; + x86_pmu.lbr_from =MSR_LBR_CORE_FROM ; + x86_pmu.lbr_to =MSR_LBR_CORE_TO ; }
PATCH v6 3/18 perf: add Intel X86 LBR sharing logic
In reply to Stephane EranianThe Intel LBR on some recent processor is capable of filtering branches by type. The filter is configurable via the LBR_SELECT MSR register.
There are limitation on how this register can be used.
On Nehalem/Westmere, the LBR_SELECT is shared by the two HT threads when HT is on. It is private to each core when HT is off.
On SandyBridge, the LBR_SELECT register is private to each thread when HT is on. It is private to each core when HT is off.
The kernel must manage the sharing of LBR_SELECT. It allows multiple users on the same logical CPU to use LBR_SELECT as long as they program it with the same value. Across sibling CPUs (HT threads), the same restriction applies on NHM/WSM.
This patch implements this sharing logic by leveraging the mechanism put in place for managing the offcore_response shared MSR.
We modify __intel_shared_reg_get_constraints() to cause x86_get_event_constraint() to be called because LBR may be associated with events that may be counter constrained.
--- arch/x86/kernel/cpu/perf_event.c | 4 ++ arch/x86/kernel/cpu/perf_event.h | 4 ++ arch/x86/kernel/cpu/perf_event_intel.c | 70 ++++++++++++++++++++------------ 3 files changed, 52 insertions(+), 26 deletions(-)
arch/x86/kernel/cpu/perf_event.c +4 -0
@@ -426,6 +426,10 @@ static int __x86_pmu_event_init(struct perf_event *event)/* mark unused */ event->hw.extra_reg.idx =EXTRA_REG_NONE ; +/* mark not used */ + event->hw.extra_reg.idx =EXTRA_REG_NONE ; + event->hw.branch_reg.idx =EXTRA_REG_NONE ; +return x86_pmu.hw_config (event); }
arch/x86/kernel/cpu/perf_event.h +4 -0
@@ -33,6 +33,7 @@ enum extra_reg_type {EXTRA_REG_RSP_0 =0 ,/* offcore_response_0 */ EXTRA_REG_RSP_1 =1 ,/* offcore_response_1 */ +EXTRA_REG_LBR =2 ,/* lbr_select */ EXTRA_REG_MAX /* number of entries needed */ };
@@ -130,6 +131,7 @@ struct cpu_hw_events {void *lbr_context;struct perf_branch_stack lbr_stack;struct perf_branch_entry lbr_entries[MAX_LBR_ENTRIES ]; +struct er_account *lbr_sel;/* * Intel host/guest exclude bits
@@ -340,6 +342,8 @@ struct x86_pmu {*/ unsigned long lbr_tos, lbr_from, lbr_to;/* MSR base regs */ int lbr_nr;/* hardware stack size */ +u64 lbr_sel_mask;/* LBR_SELECT valid bits */ +const int *lbr_sel_map;/* lbr_select mappings */ /* * Extra registers for events
arch/x86/kernel/cpu/perf_event_intel.c +44 -26
121 lines not on this page · raw message
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Add Intel LBR sharing logic
Commit-ID: b36817e8863090f1f24e538106ca50fa1d9e4003 Gitweb: http://git.kernel.org/tip/b36817e8863090f1f24e538106ca50fa1d9e4003 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:53 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:40 +0100
perf/x86: Add Intel LBR sharing logic
The Intel LBR on some recent processor is capable of filtering branches by type. The filter is configurable via the LBR_SELECT MSR register.
There are limitation on how this register can be used.
On Nehalem/Westmere, the LBR_SELECT is shared by the two HT threads when HT is on. It is private to each core when HT is off.
On SandyBridge, the LBR_SELECT register is private to each thread when HT is on. It is private to each core when HT is off.
The kernel must manage the sharing of LBR_SELECT. It allows multiple users on the same logical CPU to use LBR_SELECT as long as they program it with the same value. Across sibling CPUs (HT threads), the same restriction applies on NHM/WSM.
This patch implements this sharing logic by leveraging the mechanism put in place for managing the offcore_response shared MSR.
We modify __intel_shared_reg_get_constraints() to cause x86_get_event_constraint() to be called because LBR may be associated with events that may be counter constrained.
--- arch/x86/kernel/cpu/perf_event.c | 4 ++ arch/x86/kernel/cpu/perf_event.h | 4 ++ arch/x86/kernel/cpu/perf_event_intel.c | 70 ++++++++++++++++++++------------ 3 files changed, 52 insertions(+), 26 deletions(-)
arch/x86/kernel/cpu/perf_event.c +4 -0
@@ -426,6 +426,10 @@ static int __x86_pmu_event_init(struct perf_event *event)/* mark unused */ event->hw.extra_reg.idx =EXTRA_REG_NONE ; +/* mark not used */ + event->hw.extra_reg.idx =EXTRA_REG_NONE ; + event->hw.branch_reg.idx =EXTRA_REG_NONE ; +return x86_pmu.hw_config (event); }
arch/x86/kernel/cpu/perf_event.h +4 -0
@@ -33,6 +33,7 @@ enum extra_reg_type {EXTRA_REG_RSP_0 =0 ,/* offcore_response_0 */ EXTRA_REG_RSP_1 =1 ,/* offcore_response_1 */ +EXTRA_REG_LBR =2 ,/* lbr_select */ EXTRA_REG_MAX /* number of entries needed */ };
@@ -130,6 +131,7 @@ struct cpu_hw_events {void *lbr_context;struct perf_branch_stack lbr_stack;struct perf_branch_entry lbr_entries[MAX_LBR_ENTRIES ]; +struct er_account *lbr_sel;/* * Intel host/guest exclude bits
@@ -342,6 +344,8 @@ struct x86_pmu {*/ unsigned long lbr_tos, lbr_from, lbr_to;/* MSR base regs */ int lbr_nr;/* hardware stack size */ +u64 lbr_sel_mask;/* LBR_SELECT valid bits */ +const int *lbr_sel_map;/* lbr_select mappings */ /* * Extra registers for events
arch/x86/kernel/cpu/perf_event_intel.c +44 -26
121 lines not on this page · raw message
PATCH v6 4/18 perf: sync branch stack sampling with X86 precise_sampling
In reply to Stephane EranianIf precise sampling is enabled on Intel X86, then perf_event uses PEBS. To correct for the off-by-one error of PEBS, perf_event uses LBR when precise_sample > 1.
On Intel X86, PERF_SAMPLE_BRANCH_STACK is implemented using LBR, therefore both features must be coordinated as they may not configure LBR the same way.
For PEBS, LBR needs to capture all branches at the priv level of the associated event.
This patch checks that the branch type and priv level of BRANCH_STACK is compatible with that of the PEBS LBR requirement, thereby allowing:
$ perf record -b any,u -e instructions:upp ....
But
$ perf record -b any_call,u -e instructions:upp
Is not possible.
--- arch/x86/kernel/cpu/perf_event.c | 60 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 60 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event.c +60 -0
@@ -353,6 +353,36 @@ int x86_setup_perfctr(struct perf_event *event)return 0 ; } +/* +* check that branch_sample_type is compatible with +* settings needed for precise_ip > 1 which implies +* using the LBR to capture ALL taken branches at the +* priv levels of the measurement +*/ +static inline int precise_br_compat (struct perf_event *event) +{ +u64 m = event->attr.branch_sample_type; +u64 b =0 ; + +/* must capture all branches */ +if (!(m &PERF_SAMPLE_BRANCH_ANY )) +return 0 ; + + m &=PERF_SAMPLE_BRANCH_KERNEL |PERF_SAMPLE_BRANCH_USER ; + +if (!event->attr.exclude_user) + b |=PERF_SAMPLE_BRANCH_USER ; + +if (!event->attr.exclude_kernel) + b |=PERF_SAMPLE_BRANCH_KERNEL ; + +/* +* ignore PERF_SAMPLE_BRANCH_HV, not supported on X86 +*/ + +return m == b; +} +int x86_pmu_hw_config (struct perf_event *event) {if (event->attr.precise_ip) {
@@ -369,6 +399,36 @@ int x86_pmu_hw_config(struct perf_event *event)if (event->attr.precise_ip > precise)return -EOPNOTSUPP ; +/* +* check that PEBS LBR correction does not conflict with +* whatever the user is asking with attr->branch_sample_type +*/ +if (event->attr.precise_ip >1 ) { +u64 *br_type = &event->attr.branch_sample_type; + +if (has_branch_stack (event)) { +if (!precise_br_compat (event)) +return -EOPNOTSUPP ; + +/* branch_sample_type is compatible */ + + }else { +/* +* user did not specify branch_sample_type +* +* For PEBS fixups, we capture all +* the branches at the priv level of the +* event. +*/ + *br_type =PERF_SAMPLE_BRANCH_ANY ; + +if (!event->attr.exclude_user) + *br_type |=PERF_SAMPLE_BRANCH_USER ; + +if (!event->attr.exclude_kernel) + *br_type |=PERF_SAMPLE_BRANCH_KERNEL ; + } + } }/*
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Sync branch stack sampling with precise_sampling
Commit-ID: ff3fb511ba377e8a0a7f553cc352237f70d08121 Gitweb: http://git.kernel.org/tip/ff3fb511ba377e8a0a7f553cc352237f70d08121 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:54 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:40 +0100
perf/x86: Sync branch stack sampling with precise_sampling
If precise sampling is enabled on Intel x86 then perf_event uses PEBS. To correct for the off-by-one error of PEBS, perf_event uses LBR when precise_sample > 1.
On Intel x86 PERF_SAMPLE_BRANCH_STACK is implemented using LBR, therefore both features must be coordinated as they may not configure LBR the same way.
For PEBS, LBR needs to capture all branches at the priv level of the associated event.
This patch checks that the branch type and priv level of BRANCH_STACK is compatible with that of the PEBS LBR requirement, thereby allowing:
$ perf record -b any,u -e instructions:upp ....
But:
$ perf record -b any_call,u -e instructions:upp
Is not possible.
--- arch/x86/kernel/cpu/perf_event.c | 60 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 60 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event.c +60 -0
@@ -353,6 +353,36 @@ int x86_setup_perfctr(struct perf_event *event)return 0 ; } +/* +* check that branch_sample_type is compatible with +* settings needed for precise_ip > 1 which implies +* using the LBR to capture ALL taken branches at the +* priv levels of the measurement +*/ +static inline int precise_br_compat (struct perf_event *event) +{ +u64 m = event->attr.branch_sample_type; +u64 b =0 ; + +/* must capture all branches */ +if (!(m &PERF_SAMPLE_BRANCH_ANY )) +return 0 ; + + m &=PERF_SAMPLE_BRANCH_KERNEL |PERF_SAMPLE_BRANCH_USER ; + +if (!event->attr.exclude_user) + b |=PERF_SAMPLE_BRANCH_USER ; + +if (!event->attr.exclude_kernel) + b |=PERF_SAMPLE_BRANCH_KERNEL ; + +/* +* ignore PERF_SAMPLE_BRANCH_HV, not supported on x86 +*/ + +return m == b; +} +int x86_pmu_hw_config (struct perf_event *event) {if (event->attr.precise_ip) {
@@ -369,6 +399,36 @@ int x86_pmu_hw_config(struct perf_event *event)if (event->attr.precise_ip > precise)return -EOPNOTSUPP ; +/* +* check that PEBS LBR correction does not conflict with +* whatever the user is asking with attr->branch_sample_type +*/ +if (event->attr.precise_ip >1 ) { +u64 *br_type = &event->attr.branch_sample_type; + +if (has_branch_stack (event)) { +if (!precise_br_compat (event)) +return -EOPNOTSUPP ; + +/* branch_sample_type is compatible */ + + }else { +/* +* user did not specify branch_sample_type +* +* For PEBS fixups, we capture all +* the branches at the priv level of the +* event. +*/ + *br_type =PERF_SAMPLE_BRANCH_ANY ; + +if (!event->attr.exclude_user) + *br_type |=PERF_SAMPLE_BRANCH_USER ; + +if (!event->attr.exclude_kernel) + *br_type |=PERF_SAMPLE_BRANCH_KERNEL ; + } + } }/*
PATCH v6 5/18 perf: add Intel X86 LBR mappings for PERF_SAMPLE_BRANCH filters
In reply to Stephane EranianThis patch adds the mappings from the generic PERF_SAMPLE_BRANCH_* filters to the actual Intel X86 LBR filters, whenever they exist.
--- arch/x86/kernel/cpu/perf_event.h | 2 + arch/x86/kernel/cpu/perf_event_intel.c | 2 +- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 103 +++++++++++++++++++++++++++- 3 files changed, 104 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -535,6 +535,8 @@ void intel_pmu_lbr_init_nhm(void);void intel_pmu_lbr_init_atom (void ); +void intel_pmu_lbr_init_snb (void ); +int p4_pmu_init (void );int p6_pmu_init (void );
arch/x86/kernel/cpu/perf_event_intel.c +1 -1
@@ -1757,7 +1757,7 @@ __init int intel_pmu_init(void)memcpy (hw_cache_event_ids, snb_hw_cache_event_ids,sizeof (hw_cache_event_ids)); -intel_pmu_lbr_init_nhm (); +intel_pmu_lbr_init_snb (); x86_pmu.event_constraints = intel_snb_event_constraints; x86_pmu.pebs_constraints = intel_snb_pebs_event_constraints;
arch/x86/kernel/cpu/perf_event_intel_lbr.c +101 -2
141 lines not on this page · raw message
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Add Intel LBR mappings for PERF_SAMPLE_BRANCH filters
Commit-ID: c5cc2cd906ea9fe73e3c93f9ad824996faa278cc Gitweb: http://git.kernel.org/tip/c5cc2cd906ea9fe73e3c93f9ad824996faa278cc Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:55 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:41 +0100
perf/x86: Add Intel LBR mappings for PERF_SAMPLE_BRANCH filters
This patch adds the mappings from the generic PERF_SAMPLE_BRANCH_* filters to the actual Intel x86LBR filters, whenever they exist.
--- arch/x86/kernel/cpu/perf_event.h | 2 + arch/x86/kernel/cpu/perf_event_intel.c | 2 +- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 103 +++++++++++++++++++++++++++- 3 files changed, 104 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -539,6 +539,8 @@ void intel_pmu_lbr_init_nhm(void);void intel_pmu_lbr_init_atom (void ); +void intel_pmu_lbr_init_snb (void ); +int p4_pmu_init (void );int p6_pmu_init (void );
arch/x86/kernel/cpu/perf_event_intel.c +1 -1
@@ -1757,7 +1757,7 @@ __init int intel_pmu_init(void)memcpy (hw_cache_event_ids, snb_hw_cache_event_ids,sizeof (hw_cache_event_ids)); -intel_pmu_lbr_init_nhm (); +intel_pmu_lbr_init_snb (); x86_pmu.event_constraints = intel_snb_event_constraints; x86_pmu.pebs_constraints = intel_snb_pebs_event_constraints;
arch/x86/kernel/cpu/perf_event_intel_lbr.c +101 -2
141 lines not on this page · raw message
PATCH v6 6/18 perf: disable LBR support for older Intel Atom processors
In reply to Stephane EranianThe patch adds a restriction for Intel Atom LBR support. Only steppings 10 (PineView) and more recent are supported. Older models, do not have a functional LBR. Their LBR does not freeze on PMU interrupt which makes LBR unusable in the context of perf_events.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +10 -0
@@ -315,6 +315,16 @@ void intel_pmu_lbr_init_snb(void)/* atom */ void intel_pmu_lbr_init_atom (void ) { +/* +* only models starting at stepping 10 seems +* to have an operational LBR which can freeze +* on PMU interrupt +*/ +if (boot_cpu_data.x86_mask <10 ) { +pr_cont ("LBR disabled due to erratum" ); +return ; + } + x86_pmu.lbr_nr =8 ; x86_pmu.lbr_tos =MSR_LBR_TOS ; x86_pmu.lbr_from =MSR_LBR_CORE_FROM ;
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Disable LBR support for older Intel Atom processors
Commit-ID: 88c9a65e13f393fd60d8b9e9c659a34f9e39967d Gitweb: http://git.kernel.org/tip/88c9a65e13f393fd60d8b9e9c659a34f9e39967d Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:56 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:41 +0100
perf/x86: Disable LBR support for older Intel Atom processors
The patch adds a restriction for Intel Atom LBR support. Only steppings 10 (PineView) and more recent are supported. Older models do not have a functional LBR. Their LBR does not freeze on PMU interrupt which makes LBR unusable in the context of perf_events.
--- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +10 -0
@@ -315,6 +315,16 @@ void intel_pmu_lbr_init_snb(void)/* atom */ void intel_pmu_lbr_init_atom (void ) { +/* +* only models starting at stepping 10 seems +* to have an operational LBR which can freeze +* on PMU interrupt +*/ +if (boot_cpu_data.x86_mask <10 ) { +pr_cont ("LBR disabled due to erratum" ); +return ; + } + x86_pmu.lbr_nr =8 ; x86_pmu.lbr_tos =MSR_LBR_TOS ; x86_pmu.lbr_from =MSR_LBR_CORE_FROM ;
PATCH v6 7/18 perf: implement PERF_SAMPLE_BRANCH for Intel X86
In reply to Stephane EranianThis patch implements PERF_SAMPLE_BRANCH support for Intel X86 processors. It connects PERF_SAMPLE_BRANCH to the actual LBR.
The patch adds the hooks in the PMU irq handler to save the LBR on counter overflow for both regular and PEBS modes.
--- arch/x86/kernel/cpu/perf_event.h | 2 + arch/x86/kernel/cpu/perf_event_intel.c | 35 +++++++++++ arch/x86/kernel/cpu/perf_event_intel_ds.c | 10 +-- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 86 +++++++++++++++++++++++++++- 4 files changed, 125 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -537,6 +537,8 @@ void intel_pmu_lbr_init_atom(void);void intel_pmu_lbr_init_snb (void ); +int intel_pmu_setup_lbr_filter (struct perf_event *event); +int p4_pmu_init (void );int p6_pmu_init (void );
arch/x86/kernel/cpu/perf_event_intel.c +35 -0
@@ -727,6 +727,19 @@ static __initconst const u64 atom_hw_cache_event_ids }, }; +static inline bool intel_pmu_needs_lbr_smpl (struct perf_event *event) +{ +/* user explicitly requested branch sampling */ +if (has_branch_stack (event)) +return true; + +/* implicit branch sampling to correct PEBS skid */ +if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) +return true; + +return false; +} +static void intel_pmu_disable_all (void ) {struct cpu_hw_events *cpuc = &__get_cpu_var (cpu_hw_events);
@@ -881,6 +894,13 @@ static void intel_pmu_disable_event(struct perf_event *event) cpuc->intel_ctrl_guest_mask &= ~(1ull << hwc->idx); cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); +/* +* must disable before any actual event +* because any event may be combined with LBR +*/ +if (intel_pmu_needs_lbr_smpl (event)) +intel_pmu_lbr_disable (event); +if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) {intel_pmu_disable_fixed (hwc);return ;
@@ -935,6 +955,12 @@ static void intel_pmu_enable_event(struct perf_event *event)intel_pmu_enable_bts (hwc->config);return ; } +/* +* must enabled before any actual event +* because any event may be combined with LBR +*/ +if (intel_pmu_needs_lbr_smpl (event)) +intel_pmu_lbr_enable (event);if (event->attr.exclude_host) cpuc->intel_ctrl_guest_mask |= (1ull << hwc->idx);
@@ -1057,6 +1083,9 @@ static int intel_pmu_handle_irq(struct pt_regs *regs) data.period = event->hw.last_period; +if (has_branch_stack (event)) + data.br_stack = &cpuc->lbr_stack; +if (perf_event_overflow (event, &data, regs))x86_pmu_stop (event,0 ); }
@@ -1305,6 +1334,12 @@ static int intel_pmu_hw_config(struct perf_event *event) event->hw.config = alt_config; } +if (intel_pmu_needs_lbr_smpl (event)) { + ret =intel_pmu_setup_lbr_filter (event); +if (ret) +return ret; + } +if (event->attr.type !=PERF_TYPE_RAW )return 0 ;
arch/x86/kernel/cpu/perf_event_intel_ds.c +4 -6
@@ -439,9 +439,6 @@ void intel_pmu_pebs_enable(struct perf_event *event) hwc->config &= ~ARCH_PERFMON_EVENTSEL_INT ; cpuc->pebs_enabled |=1ULL << hwc->idx; - -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) -intel_pmu_lbr_enable (event); }void intel_pmu_pebs_disable (struct perf_event *event)
@@ -454,9 +451,6 @@ void intel_pmu_pebs_disable(struct perf_event *event)wrmsrl (MSR_IA32_PEBS_ENABLE , cpuc->pebs_enabled); hwc->config |=ARCH_PERFMON_EVENTSEL_INT ; - -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) -intel_pmu_lbr_disable (event); }void intel_pmu_pebs_enable_all (void )
@@ -572,6 +566,7 @@ static void __intel_pmu_pebs_event(struct perf_event *event,* both formats and we don't use the other fields in this * routine. */ +struct cpu_hw_events *cpuc = &__get_cpu_var (cpu_hw_events);struct pebs_record_core *pebs = __pebs;struct perf_sample_data data;struct pt_regs regs;
@@ -602,6 +597,9 @@ static void __intel_pmu_pebs_event(struct perf_event *event,else regs.flags &= ~PERF_EFLAGS_EXACT ; +if (has_branch_stack (event)) + data.br_stack = &cpuc->lbr_stack; +if (perf_event_overflow (event, &data, ®s))x86_pmu_stop (event,0 ); }
arch/x86/kernel/cpu/perf_event_intel_lbr.c +84 -2
123 lines not on this page · raw message
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Implement PERF_SAMPLE_BRANCH for Intel CPUs
Commit-ID: 60ce0fbd072695866cb27b729690ab59dce705a5 Gitweb: http://git.kernel.org/tip/60ce0fbd072695866cb27b729690ab59dce705a5 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:57 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:41 +0100
perf/x86: Implement PERF_SAMPLE_BRANCH for Intel CPUs
This patch implements PERF_SAMPLE_BRANCH support for Intel x86processors. It connects PERF_SAMPLE_BRANCH to the actual LBR.
The patch adds the hooks in the PMU irq handler to save the LBR on counter overflow for both regular and PEBS modes.
--- arch/x86/kernel/cpu/perf_event.h | 2 + arch/x86/kernel/cpu/perf_event_intel.c | 35 +++++++++++ arch/x86/kernel/cpu/perf_event_intel_ds.c | 10 +-- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 86 +++++++++++++++++++++++++++- 4 files changed, 125 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/perf_event.h +2 -0
@@ -541,6 +541,8 @@ void intel_pmu_lbr_init_atom(void);void intel_pmu_lbr_init_snb (void ); +int intel_pmu_setup_lbr_filter (struct perf_event *event); +int p4_pmu_init (void );int p6_pmu_init (void );
arch/x86/kernel/cpu/perf_event_intel.c +35 -0
@@ -727,6 +727,19 @@ static __initconst const u64 atom_hw_cache_event_ids }, }; +static inline bool intel_pmu_needs_lbr_smpl (struct perf_event *event) +{ +/* user explicitly requested branch sampling */ +if (has_branch_stack (event)) +return true; + +/* implicit branch sampling to correct PEBS skid */ +if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) +return true; + +return false; +} +static void intel_pmu_disable_all (void ) {struct cpu_hw_events *cpuc = &__get_cpu_var (cpu_hw_events);
@@ -881,6 +894,13 @@ static void intel_pmu_disable_event(struct perf_event *event) cpuc->intel_ctrl_guest_mask &= ~(1ull << hwc->idx); cpuc->intel_ctrl_host_mask &= ~(1ull << hwc->idx); +/* +* must disable before any actual event +* because any event may be combined with LBR +*/ +if (intel_pmu_needs_lbr_smpl (event)) +intel_pmu_lbr_disable (event); +if (unlikely (hwc->config_base ==MSR_ARCH_PERFMON_FIXED_CTR_CTRL )) {intel_pmu_disable_fixed (hwc);return ;
@@ -935,6 +955,12 @@ static void intel_pmu_enable_event(struct perf_event *event)intel_pmu_enable_bts (hwc->config);return ; } +/* +* must enabled before any actual event +* because any event may be combined with LBR +*/ +if (intel_pmu_needs_lbr_smpl (event)) +intel_pmu_lbr_enable (event);if (event->attr.exclude_host) cpuc->intel_ctrl_guest_mask |= (1ull << hwc->idx);
@@ -1057,6 +1083,9 @@ again: data.period = event->hw.last_period; +if (has_branch_stack (event)) + data.br_stack = &cpuc->lbr_stack; +if (perf_event_overflow (event, &data, regs))x86_pmu_stop (event,0 ); }
@@ -1305,6 +1334,12 @@ static int intel_pmu_hw_config(struct perf_event *event) event->hw.config = alt_config; } +if (intel_pmu_needs_lbr_smpl (event)) { + ret =intel_pmu_setup_lbr_filter (event); +if (ret) +return ret; + } +if (event->attr.type !=PERF_TYPE_RAW )return 0 ;
arch/x86/kernel/cpu/perf_event_intel_ds.c +4 -6
@@ -439,9 +439,6 @@ void intel_pmu_pebs_enable(struct perf_event *event) hwc->config &= ~ARCH_PERFMON_EVENTSEL_INT ; cpuc->pebs_enabled |=1ULL << hwc->idx; - -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) -intel_pmu_lbr_enable (event); }void intel_pmu_pebs_disable (struct perf_event *event)
@@ -454,9 +451,6 @@ void intel_pmu_pebs_disable(struct perf_event *event)wrmsrl (MSR_IA32_PEBS_ENABLE , cpuc->pebs_enabled); hwc->config |=ARCH_PERFMON_EVENTSEL_INT ; - -if (x86_pmu.intel_cap.pebs_trap && event->attr.precise_ip >1 ) -intel_pmu_lbr_disable (event); }void intel_pmu_pebs_enable_all (void )
@@ -572,6 +566,7 @@ static void __intel_pmu_pebs_event(struct perf_event *event,* both formats and we don't use the other fields in this * routine. */ +struct cpu_hw_events *cpuc = &__get_cpu_var (cpu_hw_events);struct pebs_record_core *pebs = __pebs;struct perf_sample_data data;struct pt_regs regs;
@@ -602,6 +597,9 @@ static void __intel_pmu_pebs_event(struct perf_event *event,else regs.flags &= ~PERF_EFLAGS_EXACT ; +if (has_branch_stack (event)) + data.br_stack = &cpuc->lbr_stack; +if (perf_event_overflow (event, &data, ®s))x86_pmu_stop (event,0 ); }
arch/x86/kernel/cpu/perf_event_intel_lbr.c +84 -2
123 lines not on this page · raw message
PATCH v6 8/18 perf: add LBR software filter support for Intel X86
In reply to Stephane EranianThis patch adds an internal sofware filter to complement the (optional) LBR hardware filter.
The software filter is necessary:
- as a substitute when there is no HW LBR filter (e.g., Atom, Core)
- to complement HW LBR filter in case of errata (e.g., Nehalem/Westmere)
- to provide finer grain filtering (e.g., all processors)
Sometimes, the LBR HW filter cannot distinguish between two types of branches. For instance, to capture syscall as CALLS, it is necessary to enable the LBR_FAR filter which will also capture JMP instructions. Thus, a second pass is necessary to filter those out, this is what the SW filter can do.
The SW filter is built on top of the internal x86 disassembler. It is a best effort filter especially for user level code. It is subject to the availability of the text page of the program.
The SW filter is enabled on all Intel X86 processors. It is bypassed when the user is capturing all branches at all priv levels.
--- arch/x86/kernel/cpu/perf_event.h | 10 + arch/x86/kernel/cpu/perf_event_intel_ds.c | 12 +- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 332 ++++++++++++++++++++++++++-- 3 files changed, 321 insertions(+), 33 deletions(-)
arch/x86/kernel/cpu/perf_event.h +10 -0
@@ -132,6 +132,7 @@ struct cpu_hw_events {struct perf_branch_stack lbr_stack;struct perf_branch_entry lbr_entries[MAX_LBR_ENTRIES ];struct er_account *lbr_sel; +u64 br_sel;/* * Intel host/guest exclude bits
@@ -455,6 +456,15 @@ extern struct event_constraint emptyconstraint;extern struct event_constraint unconstrained; +static inline bool kernel_ip (unsigned long ip) +{ +#ifdef CONFIG_X86_32 +return ip >PAGE_OFFSET ; +#else +return (long )ip <0 ; +#endif +} +#ifdef CONFIG_CPU_SUP_AMD int amd_pmu_init (void );
arch/x86/kernel/cpu/perf_event_intel_ds.c +1 -11
@@ -3,6 +3,7 @@#include <linux/slab.h> #include <asm/perf_event.h> +#include <asm/insn.h> #include "perf_event.h"
@@ -469,17 +470,6 @@ void intel_pmu_pebs_disable_all(void)wrmsrl (MSR_IA32_PEBS_ENABLE ,0 ); } -#include <asm/insn.h> - -static inline bool kernel_ip (unsigned long ip) -{ -#ifdef CONFIG_X86_32 -return ip >PAGE_OFFSET ; -#else -return (long )ip <0 ; -#endif -} -static int intel_pmu_pebs_fixup_ip (struct pt_regs *regs) {struct cpu_hw_events *cpuc = &__get_cpu_var (cpu_hw_events);
arch/x86/kernel/cpu/perf_event_intel_lbr.c +310 -22
405 lines not on this page · raw message
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf/x86: Add LBR software filter support for Intel CPUs
Commit-ID: 3e702ff6d1ea12dcf1c798ecb61e7f3a1579df42 Gitweb: http://git.kernel.org/tip/3e702ff6d1ea12dcf1c798ecb61e7f3a1579df42 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:58 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:42 +0100
perf/x86: Add LBR software filter support for Intel CPUs
This patch adds an internal sofware filter to complement the (optional) LBR hardware filter.
The software filter is necessary:
- as a substitute when there is no HW LBR filter (e.g., Atom, Core)
- to complement HW LBR filter in case of errata (e.g., Nehalem/Westmere)
- to provide finer grain filtering (e.g., all processors)
Sometimes the LBR HW filter cannot distinguish between two types of branches. For instance, to capture syscall as CALLS, it is necessary to enable the LBR_FAR filter which will also capture JMP instructions. Thus, a second pass is necessary to filter those out, this is what the SW filter can do.
The SW filter is built on top of the internal x86 disassembler. It is a best effort filter especially for user level code. It is subject to the availability of the text page of the program.
The SW filter is enabled on all Intel processors. It is bypassed when the user is capturing all branches at all priv levels.
--- arch/x86/kernel/cpu/perf_event.h | 10 + arch/x86/kernel/cpu/perf_event_intel_ds.c | 12 +- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 332 ++++++++++++++++++++++++++-- 3 files changed, 321 insertions(+), 33 deletions(-)
arch/x86/kernel/cpu/perf_event.h +10 -0
@@ -132,6 +132,7 @@ struct cpu_hw_events {struct perf_branch_stack lbr_stack;struct perf_branch_entry lbr_entries[MAX_LBR_ENTRIES ];struct er_account *lbr_sel; +u64 br_sel;/* * Intel host/guest exclude bits
@@ -459,6 +460,15 @@ extern struct event_constraint emptyconstraint;extern struct event_constraint unconstrained; +static inline bool kernel_ip (unsigned long ip) +{ +#ifdef CONFIG_X86_32 +return ip >PAGE_OFFSET ; +#else +return (long )ip <0 ; +#endif +} +#ifdef CONFIG_CPU_SUP_AMD int amd_pmu_init (void );
arch/x86/kernel/cpu/perf_event_intel_ds.c +1 -11
@@ -3,6 +3,7 @@#include <linux/slab.h> #include <asm/perf_event.h> +#include <asm/insn.h> #include "perf_event.h"
@@ -469,17 +470,6 @@ void intel_pmu_pebs_disable_all(void)wrmsrl (MSR_IA32_PEBS_ENABLE ,0 ); } -#include <asm/insn.h> - -static inline bool kernel_ip (unsigned long ip) -{ -#ifdef CONFIG_X86_32 -return ip >PAGE_OFFSET ; -#else -return (long )ip <0 ; -#endif -} -static int intel_pmu_pebs_fixup_ip (struct pt_regs *regs) {struct cpu_hw_events *cpuc = &__get_cpu_var (cpu_hw_events);
arch/x86/kernel/cpu/perf_event_intel_lbr.c +310 -22
405 lines not on this page · raw message
PATCH v6 9/18 perf: disable PERF_SAMPLE_BRANCH_* when not supported
In reply to Stephane EranianPERF_SAMPLE_BRANCH_* is disabled for:
- SW events (sw counters, tracepoints)
- HW breakpoints
- ALL but Intel X86 architecture
- AMD64 processors
--- arch/alpha/kernel/perf_event.c | 4 ++++ arch/arm/kernel/perf_event.c | 4 ++++ arch/mips/kernel/perf_event_mipsxx.c | 4 ++++ arch/powerpc/kernel/perf_event.c | 4 ++++ arch/sh/kernel/perf_event.c | 4 ++++ arch/sparc/kernel/perf_event.c | 4 ++++ arch/x86/kernel/cpu/perf_event_amd.c | 3 +++ kernel/events/core.c | 24 ++++++++++++++++++++++++ kernel/events/hw_breakpoint.c | 6 ++++++ 9 files changed, 57 insertions(+), 0 deletions(-)
arch/alpha/kernel/perf_event.c +4 -0
@@ -685,6 +685,10 @@ static int alpha_pmu_event_init(struct perf_event *event) {int err; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_RAW :case PERF_TYPE_HARDWARE :
arch/arm/kernel/perf_event.c +4 -0
@@ -539,6 +539,10 @@ static int armpmu_event_init(struct perf_event *event)int err =0 ;atomic_t *active_events = &armpmu->active_events; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +if (armpmu->map_event (event) == -ENOENT )return -ENOENT ;
arch/mips/kernel/perf_event_mipsxx.c +4 -0
@@ -606,6 +606,10 @@ static int mipspmu_event_init(struct perf_event *event) {int err =0 ; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_RAW :case PERF_TYPE_HARDWARE :
arch/powerpc/kernel/perf_event.c +4 -0
@@ -1078,6 +1078,10 @@ static int power_pmu_event_init(struct perf_event *event)if (!ppmu)return -ENOENT ; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_HARDWARE : ev = event->attr.config;
arch/sh/kernel/perf_event.c +4 -0
@@ -310,6 +310,10 @@ static int sh_pmu_event_init(struct perf_event *event) {int err; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_RAW :case PERF_TYPE_HW_CACHE :
arch/sparc/kernel/perf_event.c +4 -0
@@ -1105,6 +1105,10 @@ static int sparc_pmu_event_init(struct perf_event *event)if (atomic_read (&nmi_active) <0 )return -ENODEV ; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (attr->type) {case PERF_TYPE_HARDWARE :if (attr->config >= sparc_pmu->max_events)
arch/x86/kernel/cpu/perf_event_amd.c +3 -0
@@ -138,6 +138,9 @@ static int amd_pmu_hw_config(struct perf_event *event)if (ret)return ret; +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +if (event->attr.exclude_host && event->attr.exclude_guest)/* * When HO == GO == 1 the hardware treats that as GO == HO == 0
kernel/events/core.c +24 -0
@@ -5044,6 +5044,12 @@ static int perf_swevent_init(struct perf_event *event)if (event->attr.type !=PERF_TYPE_SOFTWARE )return -ENOENT ; +/* +* no branch sampling for software events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event_id) {case PERF_COUNT_SW_CPU_CLOCK :case PERF_COUNT_SW_TASK_CLOCK :
@@ -5154,6 +5160,12 @@ static int perf_tp_event_init(struct perf_event *event)if (event->attr.type !=PERF_TYPE_TRACEPOINT )return -ENOENT ; +/* +* no branch sampling for tracepoint events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; + err =perf_trace_init (event);if (err)return err;
@@ -5379,6 +5391,12 @@ static int cpu_clock_event_init(struct perf_event *event)if (event->attr.config !=PERF_COUNT_SW_CPU_CLOCK )return -ENOENT ; +/* +* no branch sampling for software events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +perf_swevent_init_hrtimer (event);return 0 ;
@@ -5453,6 +5471,12 @@ static int task_clock_event_init(struct perf_event *event)if (event->attr.config !=PERF_COUNT_SW_TASK_CLOCK )return -ENOENT ; +/* +* no branch sampling for software events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +perf_swevent_init_hrtimer (event);return 0 ;
kernel/events/hw_breakpoint.c +6 -0
@@ -581,6 +581,12 @@ static int hw_breakpoint_event_init(struct perf_event *bp)if (bp->attr.type !=PERF_TYPE_BREAKPOINT )return -ENOENT ; +/* +* no branch sampling for breakpoint events +*/ +if (has_branch_stack (bp)) +return -EOPNOTSUPP ; + err =register_perf_hw_breakpoint (bp);if (err)return err;
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf: Disable PERF_SAMPLE_BRANCH_* when not supported
Commit-ID: 2481c5fa6db0237e4f0168f88913178b2b495b7c Gitweb: http://git.kernel.org/tip/2481c5fa6db0237e4f0168f88913178b2b495b7c Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:20:59 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:42 +0100
perf: Disable PERF_SAMPLE_BRANCH_* when not supported
PERF_SAMPLE_BRANCH_* is disabled for:
- SW events (sw counters, tracepoints)
- HW breakpoints
- ALL but Intel x86 architecture
- AMD64 processors
--- arch/alpha/kernel/perf_event.c | 4 ++++ arch/arm/kernel/perf_event.c | 4 ++++ arch/mips/kernel/perf_event_mipsxx.c | 4 ++++ arch/powerpc/kernel/perf_event.c | 4 ++++ arch/sh/kernel/perf_event.c | 4 ++++ arch/sparc/kernel/perf_event.c | 4 ++++ arch/x86/kernel/cpu/perf_event_amd.c | 3 +++ kernel/events/core.c | 24 ++++++++++++++++++++++++ kernel/events/hw_breakpoint.c | 6 ++++++ 9 files changed, 57 insertions(+), 0 deletions(-)
arch/alpha/kernel/perf_event.c +4 -0
@@ -685,6 +685,10 @@ static int alpha_pmu_event_init(struct perf_event *event) {int err; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_RAW :case PERF_TYPE_HARDWARE :
arch/arm/kernel/perf_event.c +4 -0
@@ -539,6 +539,10 @@ static int armpmu_event_init(struct perf_event *event)int err =0 ;atomic_t *active_events = &armpmu->active_events; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +if (armpmu->map_event (event) == -ENOENT )return -ENOENT ;
arch/mips/kernel/perf_event_mipsxx.c +4 -0
@@ -606,6 +606,10 @@ static int mipspmu_event_init(struct perf_event *event) {int err =0 ; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_RAW :case PERF_TYPE_HARDWARE :
arch/powerpc/kernel/perf_event.c +4 -0
@@ -1084,6 +1084,10 @@ static int power_pmu_event_init(struct perf_event *event)if (!ppmu)return -ENOENT ; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_HARDWARE : ev = event->attr.config;
arch/sh/kernel/perf_event.c +4 -0
@@ -310,6 +310,10 @@ static int sh_pmu_event_init(struct perf_event *event) {int err; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event->attr.type) {case PERF_TYPE_RAW :case PERF_TYPE_HW_CACHE :
arch/sparc/kernel/perf_event.c +4 -0
@@ -1105,6 +1105,10 @@ static int sparc_pmu_event_init(struct perf_event *event)if (atomic_read (&nmi_active) <0 )return -ENODEV ; +/* does not support taken branch sampling */ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (attr->type) {case PERF_TYPE_HARDWARE :if (attr->config >= sparc_pmu->max_events)
arch/x86/kernel/cpu/perf_event_amd.c +3 -0
@@ -139,6 +139,9 @@ static int amd_pmu_hw_config(struct perf_event *event)if (ret)return ret; +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +if (event->attr.exclude_host && event->attr.exclude_guest)/* * When HO == GO == 1 the hardware treats that as GO == HO == 0
kernel/events/core.c +24 -0
@@ -5044,6 +5044,12 @@ static int perf_swevent_init(struct perf_event *event)if (event->attr.type !=PERF_TYPE_SOFTWARE )return -ENOENT ; +/* +* no branch sampling for software events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +switch (event_id) {case PERF_COUNT_SW_CPU_CLOCK :case PERF_COUNT_SW_TASK_CLOCK :
@@ -5154,6 +5160,12 @@ static int perf_tp_event_init(struct perf_event *event)if (event->attr.type !=PERF_TYPE_TRACEPOINT )return -ENOENT ; +/* +* no branch sampling for tracepoint events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; + err =perf_trace_init (event);if (err)return err;
@@ -5379,6 +5391,12 @@ static int cpu_clock_event_init(struct perf_event *event)if (event->attr.config !=PERF_COUNT_SW_CPU_CLOCK )return -ENOENT ; +/* +* no branch sampling for software events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +perf_swevent_init_hrtimer (event);return 0 ;
@@ -5453,6 +5471,12 @@ static int task_clock_event_init(struct perf_event *event)if (event->attr.config !=PERF_COUNT_SW_TASK_CLOCK )return -ENOENT ; +/* +* no branch sampling for software events +*/ +if (has_branch_stack (event)) +return -EOPNOTSUPP ; +perf_swevent_init_hrtimer (event);return 0 ;
kernel/events/hw_breakpoint.c +6 -0
@@ -581,6 +581,12 @@ static int hw_breakpoint_event_init(struct perf_event *bp)if (bp->attr.type !=PERF_TYPE_BREAKPOINT )return -ENOENT ; +/* +* no branch sampling for breakpoint events +*/ +if (has_branch_stack (bp)) +return -EOPNOTSUPP ; + err =register_perf_hw_breakpoint (bp);if (err)return err;
PATCH v6 10/18 perf: add hook to flush branch_stack on context switch
In reply to Stephane EranianWith branch stack sampling, it is possible to filter by priv levels. In system-wide mode, that means it is possible to capture only user level branches. The builtin SW LBR filter needs to disassemble code based on LBR captured addresses. For that, it needs to know the task the addresses are associated with. Because of context switches, the content of the branch stack buffer may contain addresses from different tasks.
We need a hook on context switch to either flush the branch stack or save it. This patch adds a new hook in struct pmu which is called during context switches. The hook is called only when necessary. That is when a system-wide context has, at least, one event which uses PERF_SAMPLE_BRANCH_STACK. The hook is never called for per-thread context.
In this version, the Intel X86 code simply flushes (reset) the LBR on context switches (fill with zeroes). Those zeroed branches are then filtered out by the SW filter.
--- arch/x86/kernel/cpu/perf_event.c | 21 +++++--- arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel.c | 13 +++++ include/linux/perf_event.h | 9 +++- kernel/events/core.c | 85 ++++++++++++++++++++++++++++++++ 5 files changed, 121 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/perf_event.c +14 -7
@@ -1671,25 +1671,32 @@ static const struct attribute_group *x86_pmu_attr_groups[] = { NULL, }; +static void x86_pmu_flush_branch_stack (void ) +{ +if (x86_pmu.flush_branch_stack) + x86_pmu.flush_branch_stack (); +} +static struct pmu pmu = { - .pmu_enable = x86_pmu_enable, - .pmu_disable = x86_pmu_disable, + .pmu_enable = x86_pmu_enable, + .pmu_disable = x86_pmu_disable, .attr_groups = x86_pmu_attr_groups, .event_init = x86_pmu_event_init, - .add = x86_pmu_add, - .del = x86_pmu_del, - .start = x86_pmu_start, - .stop = x86_pmu_stop, - .read = x86_pmu_read, + .add = x86_pmu_add, + .del = x86_pmu_del, + .start = x86_pmu_start, + .stop = x86_pmu_stop, + .read = x86_pmu_read, .start_txn = x86_pmu_start_txn, .cancel_txn = x86_pmu_cancel_txn, .commit_txn = x86_pmu_commit_txn, .event_idx = x86_pmu_event_idx, + .flush_branch_stack = x86_pmu_flush_branch_stack, };void perf_update_user_clock (struct perf_event_mmap_page *userpg,u64 now)
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -322,6 +322,7 @@ struct x86_pmu {void (*cpu_starting)(int cpu);void (*cpu_dying)(int cpu);void (*cpu_dead)(int cpu); +void (*flush_branch_stack)(void );/* * Intel Arch Perfmon v2+
arch/x86/kernel/cpu/perf_event_intel.c +13 -0
@@ -1539,6 +1539,18 @@ static void intel_pmu_cpu_dying(int cpu)fini_debug_store_on_cpu (cpu); } +static void intel_pmu_flush_branch_stack (void ) +{ +/* +* Intel LBR does not tag entries with the +* PID of the current task, then we need to +* flush it on ctxsw +* For now, we simply reset it +*/ +if (x86_pmu.lbr_nr) +intel_pmu_lbr_reset (); +} +static __initconst const struct x86_pmu intel_pmu = { .name ="Intel" , .handle_irq = intel_pmu_handle_irq,
@@ -1566,6 +1578,7 @@ static __initconst const struct x86_pmu intel_pmu = { .cpu_starting = intel_pmu_cpu_starting, .cpu_dying = intel_pmu_cpu_dying, .guest_get_msrs = intel_guest_get_msrs, + .flush_branch_stack = intel_pmu_flush_branch_stack, };static __init void intel_clovertown_quirk (void )
include/linux/perf_event.h +8 -1
@@ -746,6 +746,11 @@ struct pmu {* if no implementation is provided it will default to: event->hw.idx + 1. */ int (*event_idx) (struct perf_event *event);/*optional */ + +/* +* flush branch stack on context-switches (needed in cpu-wide mode) +*/ +void (*flush_branch_stack) (void ); };/**
@@ -976,7 +981,8 @@ struct perf_event_context {u64 parent_gen;u64 generation;int pin_count; -int nr_cgroups;/* cgroup events present */ +int nr_cgroups;/* cgroup evts */ +int nr_branch_stack;/* branch_stack evt */ struct rcu_head rcu_head; };
@@ -1041,6 +1047,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr,extern u64 perf_event_read_value (struct perf_event *event,u64 *enabled,u64 *running); +struct perf_sample_data {u64 type;
kernel/events/core.c +85 -0
127 lines not on this page · raw message
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf: Add callback to flush branch_stack on context switch
Commit-ID: d010b3326cf06b3406cdd88af16dcf4e4b6fec2e Gitweb: http://git.kernel.org/tip/d010b3326cf06b3406cdd88af16dcf4e4b6fec2e Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:00 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 5 Mar 2012 14:55:42 +0100
perf: Add callback to flush branch_stack on context switch
With branch stack sampling, it is possible to filter by priv levels.
In system-wide mode, that means it is possible to capture only user level branches. The builtin SW LBR filter needs to disassemble code based on LBR captured addresses. For that, it needs to know the task the addresses are associated with. Because of context switches, the content of the branch stack buffer may contain addresses from different tasks.
We need a callback on context switch to either flush the branch stack or save it. This patch adds a new callback in struct pmu which is called during context switches. The callback is called only when necessary. That is when a system-wide context has, at least, one event which uses PERF_SAMPLE_BRANCH_STACK. The callback is never called for per-thread context.
In this version, the Intel x86 code simply flushes (resets) the LBR on context switches (fills it with zeroes). Those zeroed branches are then filtered out by the SW filter.
--- arch/x86/kernel/cpu/perf_event.c | 21 +++++--- arch/x86/kernel/cpu/perf_event.h | 1 + arch/x86/kernel/cpu/perf_event_intel.c | 13 +++++ include/linux/perf_event.h | 9 +++- kernel/events/core.c | 85 ++++++++++++++++++++++++++++++++ 5 files changed, 121 insertions(+), 8 deletions(-)
arch/x86/kernel/cpu/perf_event.c +14 -7
@@ -1671,25 +1671,32 @@ static const struct attribute_group *x86_pmu_attr_groups[] = { NULL, }; +static void x86_pmu_flush_branch_stack (void ) +{ +if (x86_pmu.flush_branch_stack) + x86_pmu.flush_branch_stack (); +} +static struct pmu pmu = { - .pmu_enable = x86_pmu_enable, - .pmu_disable = x86_pmu_disable, + .pmu_enable = x86_pmu_enable, + .pmu_disable = x86_pmu_disable, .attr_groups = x86_pmu_attr_groups, .event_init = x86_pmu_event_init, - .add = x86_pmu_add, - .del = x86_pmu_del, - .start = x86_pmu_start, - .stop = x86_pmu_stop, - .read = x86_pmu_read, + .add = x86_pmu_add, + .del = x86_pmu_del, + .start = x86_pmu_start, + .stop = x86_pmu_stop, + .read = x86_pmu_read, .start_txn = x86_pmu_start_txn, .cancel_txn = x86_pmu_cancel_txn, .commit_txn = x86_pmu_commit_txn, .event_idx = x86_pmu_event_idx, + .flush_branch_stack = x86_pmu_flush_branch_stack, };void perf_update_user_clock (struct perf_event_mmap_page *userpg,u64 now)
arch/x86/kernel/cpu/perf_event.h +1 -0
@@ -324,6 +324,7 @@ struct x86_pmu {void (*cpu_starting)(int cpu);void (*cpu_dying)(int cpu);void (*cpu_dead)(int cpu); +void (*flush_branch_stack)(void );/* * Intel Arch Perfmon v2+
arch/x86/kernel/cpu/perf_event_intel.c +13 -0
@@ -1539,6 +1539,18 @@ static void intel_pmu_cpu_dying(int cpu)fini_debug_store_on_cpu (cpu); } +static void intel_pmu_flush_branch_stack (void ) +{ +/* +* Intel LBR does not tag entries with the +* PID of the current task, then we need to +* flush it on ctxsw +* For now, we simply reset it +*/ +if (x86_pmu.lbr_nr) +intel_pmu_lbr_reset (); +} +static __initconst const struct x86_pmu intel_pmu = { .name ="Intel" , .handle_irq = intel_pmu_handle_irq,
@@ -1566,6 +1578,7 @@ static __initconst const struct x86_pmu intel_pmu = { .cpu_starting = intel_pmu_cpu_starting, .cpu_dying = intel_pmu_cpu_dying, .guest_get_msrs = intel_guest_get_msrs, + .flush_branch_stack = intel_pmu_flush_branch_stack, };static __init void intel_clovertown_quirk (void )
include/linux/perf_event.h +8 -1
@@ -746,6 +746,11 @@ struct pmu {* if no implementation is provided it will default to: event->hw.idx + 1. */ int (*event_idx) (struct perf_event *event);/*optional */ + +/* +* flush branch stack on context-switches (needed in cpu-wide mode) +*/ +void (*flush_branch_stack) (void ); };/**
@@ -979,7 +984,8 @@ struct perf_event_context {u64 parent_gen;u64 generation;int pin_count; -int nr_cgroups;/* cgroup events present */ +int nr_cgroups;/* cgroup evts */ +int nr_branch_stack;/* branch_stack evt */ struct rcu_head rcu_head; };
@@ -1044,6 +1050,7 @@ perf_event_create_kernel_counter(struct perf_event_attr *attr,extern u64 perf_event_read_value (struct perf_event *event,u64 *enabled,u64 *running); +struct perf_sample_data {u64 type;
kernel/events/core.c +85 -0
127 lines not on this page · raw message
PATCH v6 11/18 perf: add code to support PERF_SAMPLE_BRANCH_STACK
In reply to Stephane EranianFrom: Roberto Agostino Vitillo <[email protected]>
This patch adds:
- ability to parse samples with PERF_SAMPLE_BRANCH_STACK
- sort on branches (dso_from, symbol_from, dso_to, symbol_to, mispredict)
- build histograms on branches
--- tools/perf/perf.h | 17 +++ tools/perf/util/event.h | 1 + tools/perf/util/evsel.c | 10 ++ tools/perf/util/hist.c | 122 +++++++++++++++----- tools/perf/util/hist.h | 11 ++ tools/perf/util/session.c | 72 +++++++++++ tools/perf/util/session.h | 4 + tools/perf/util/sort.c | 287 +++++++++++++++++++++++++++++++++++++-------- tools/perf/util/sort.h | 9 ++ tools/perf/util/symbol.h | 13 ++ 10 files changed, 468 insertions(+), 78 deletions(-)
tools/perf/perf.h +17 -0
@@ -180,6 +180,23 @@ struct ip_callchain {u64 ips[0 ]; }; +struct branch_flags { +u64 mispred:1 ; +u64 predicted:1 ; +u64 reserved:62 ; +}; + +struct branch_entry { +u64 from; +u64 to; +struct branch_flags flags; +}; + +struct branch_stack { +u64 nr; +struct branch_entry entries[0 ]; +}; +extern bool perf_host, perf_guest;extern const char perf_version_string[];
tools/perf/util/event.h +1 -0
@@ -81,6 +81,7 @@ struct perf_sample {u32 raw_size;void *raw_data;struct ip_callchain *callchain; +struct branch_stack *branch_stack; };#define BUILD_ID_SIZE 20
tools/perf/util/evsel.c +10 -0
@@ -576,6 +576,16 @@ int perf_event__parse_sample(const union perf_event *event, u64 type, data->raw_data = (void *) pdata; } +if (type &PERF_SAMPLE_BRANCH_STACK ) { +u64 sz; + + data->branch_stack = (struct branch_stack *)array; + array++;/* nr */ + + sz = data->branch_stack->nr *sizeof (struct branch_entry ); + sz /=sizeof (u64); + array += sz; + }return 0 ; }
tools/perf/util/hist.c +95 -27
168 lines not on this page · raw message
tools/perf/util/hist.h +11 -0
@@ -41,6 +41,11 @@ enum hist_column {HISTC_COMM ,HISTC_PARENT ,HISTC_CPU , +HISTC_MISPREDICT , +HISTC_SYMBOL_FROM , +HISTC_SYMBOL_TO , +HISTC_DSO_FROM , +HISTC_DSO_TO , HISTC_NR_COLS, /* Last entry */ };
@@ -73,6 +78,12 @@ int hist_entry__snprintf(struct hist_entry *self, char *bf, size_t size, struct hists *hists);void hist_entry__free (struct hist_entry *); +struct hist_entry *__hists__add_branch_entry (struct hists *self, +struct addr_location *al, +struct symbol *sym_parent, +struct branch_info *bi, +u64 period); +void hists__output_resort (struct hists *self);void hists__output_resort_threaded (struct hists *hists);void hists__collapse_resort (struct hists *self);
tools/perf/util/session.c +72 -0
90 lines not on this page · raw message
tools/perf/util/session.h +4 -0
@@ -73,6 +73,10 @@ int perf_session__resolve_callchain(struct perf_session *self, struct perf_evselstruct ip_callchain *chain, struct symbol **parent); +struct branch_info *machine__resolve_bstack (struct machine *self, +struct thread *thread, +struct branch_stack *bs); +bool perf_session__has_traces (struct perf_session *self,const char *msg);void mem_bswap_64 (void *src,int byte_size);
tools/perf/util/sort.c +236 -51
362 lines not on this page · raw message
tools/perf/util/sort.h +9 -0
@@ -31,11 +31,14 @@ extern const char *parent_pattern;extern const char default_sort_order[];extern int sort__need_collapse;extern int sort__has_parent; +extern bool sort__branch_mode;extern char *field_sep;extern struct sort_entry sort_comm;extern struct sort_entry sort_dso;extern struct sort_entry sort_sym;extern struct sort_entry sort_parent; +extern struct sort_entry sort_lbr_dso; +extern struct sort_entry sort_lbr_sym;extern enum sort_type sort__first_dimension;/**
@@ -72,6 +75,7 @@ struct hist_entry {struct hist_entry *pair;struct rb_root sorted_chain; }; +struct branch_info *branch_info;struct callchain_root callchain[0 ]; };
@@ -82,6 +86,11 @@ enum sort_type {SORT_SYM ,SORT_PARENT ,SORT_CPU , +SORT_DSO_FROM , +SORT_DSO_TO , +SORT_SYM_FROM , +SORT_SYM_TO , + SORT_MISPREDICT, };/*
tools/perf/util/symbol.h +13 -0
@@ -5,6 +5,7 @@#include <stdbool.h> #include <stdint.h> #include "map.h" +#include "../perf.h" #include <linux/list.h> #include <linux/rbtree.h> #include <stdio.h>
@@ -120,6 +121,18 @@ struct map_symbol {bool has_children; }; +struct addr_map_symbol { +struct map *map; +struct symbol *sym; +u64 addr; +}; + +struct branch_info { +struct addr_map_symbol from; +struct addr_map_symbol to; +struct branch_flags flags; +}; +struct addr_location {struct thread *thread;struct map *map;
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf tools: Add code to support PERF_SAMPLE_BRANCH_STACK
Commit-ID: b5387528f31d98acedf06e930554b563d87e2383 Gitweb: http://git.kernel.org/tip/b5387528f31d98acedf06e930554b563d87e2383 Author: Roberto Agostino Vitillo <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:01 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:04 +0100
perf tools: Add code to support PERF_SAMPLE_BRANCH_STACK
This patch adds:
- ability to parse samples with PERF_SAMPLE_BRANCH_STACK
- sort on branches (dso_from, symbol_from, dso_to, symbol_to, mispredict)
- build histograms on branches
--- tools/perf/perf.h | 17 +++ tools/perf/util/event.h | 1 + tools/perf/util/evsel.c | 10 ++ tools/perf/util/hist.c | 122 +++++++++++++++----- tools/perf/util/hist.h | 11 ++ tools/perf/util/session.c | 72 +++++++++++ tools/perf/util/session.h | 4 + tools/perf/util/sort.c | 287 +++++++++++++++++++++++++++++++++++++-------- tools/perf/util/sort.h | 9 ++ tools/perf/util/symbol.h | 13 ++ 10 files changed, 468 insertions(+), 78 deletions(-)
tools/perf/perf.h +17 -0
@@ -179,6 +179,23 @@ struct ip_callchain {u64 ips[0 ]; }; +struct branch_flags { +u64 mispred:1 ; +u64 predicted:1 ; +u64 reserved:62 ; +}; + +struct branch_entry { +u64 from; +u64 to; +struct branch_flags flags; +}; + +struct branch_stack { +u64 nr; +struct branch_entry entries[0 ]; +}; +extern bool perf_host, perf_guest;extern const char perf_version_string[];
tools/perf/util/event.h +1 -0
@@ -81,6 +81,7 @@ struct perf_sample {u32 raw_size;void *raw_data;struct ip_callchain *callchain; +struct branch_stack *branch_stack; };#define BUILD_ID_SIZE 20
tools/perf/util/evsel.c +10 -0
@@ -576,6 +576,16 @@ int perf_event__parse_sample(const union perf_event *event, u64 type, data->raw_data = (void *) pdata; } +if (type &PERF_SAMPLE_BRANCH_STACK ) { +u64 sz; + + data->branch_stack = (struct branch_stack *)array; + array++;/* nr */ + + sz = data->branch_stack->nr *sizeof (struct branch_entry ); + sz /=sizeof (u64); + array += sz; + }return 0 ; }
tools/perf/util/hist.c +95 -27
168 lines not on this page · raw message
tools/perf/util/hist.h +11 -0
@@ -42,6 +42,11 @@ enum hist_column {HISTC_COMM ,HISTC_PARENT ,HISTC_CPU , +HISTC_MISPREDICT , +HISTC_SYMBOL_FROM , +HISTC_SYMBOL_TO , +HISTC_DSO_FROM , +HISTC_DSO_TO , HISTC_NR_COLS, /* Last entry */ };
@@ -74,6 +79,12 @@ int hist_entry__snprintf(struct hist_entry *self, char *bf, size_t size, struct hists *hists);void hist_entry__free (struct hist_entry *); +struct hist_entry *__hists__add_branch_entry (struct hists *self, +struct addr_location *al, +struct symbol *sym_parent, +struct branch_info *bi, +u64 period); +void hists__output_resort (struct hists *self);void hists__output_resort_threaded (struct hists *hists);void hists__collapse_resort (struct hists *self);
tools/perf/util/session.c +72 -0
90 lines not on this page · raw message
tools/perf/util/session.h +4 -0
@@ -73,6 +73,10 @@ int perf_session__resolve_callchain(struct perf_session *self, struct perf_evselstruct ip_callchain *chain, struct symbol **parent); +struct branch_info *machine__resolve_bstack (struct machine *self, +struct thread *thread, +struct branch_stack *bs); +bool perf_session__has_traces (struct perf_session *self,const char *msg);void mem_bswap_64 (void *src,int byte_size);
tools/perf/util/sort.c +236 -51
362 lines not on this page · raw message
tools/perf/util/sort.h +9 -0
@@ -31,11 +31,14 @@ extern const char *parent_pattern;extern const char default_sort_order[];extern int sort__need_collapse;extern int sort__has_parent; +extern bool sort__branch_mode;extern char *field_sep;extern struct sort_entry sort_comm;extern struct sort_entry sort_dso;extern struct sort_entry sort_sym;extern struct sort_entry sort_parent; +extern struct sort_entry sort_lbr_dso; +extern struct sort_entry sort_lbr_sym;extern enum sort_type sort__first_dimension;/**
@@ -72,6 +75,7 @@ struct hist_entry {struct hist_entry *pair;struct rb_root sorted_chain; }; +struct branch_info *branch_info;struct callchain_root callchain[0 ]; };
@@ -82,6 +86,11 @@ enum sort_type {SORT_SYM ,SORT_PARENT ,SORT_CPU , +SORT_DSO_FROM , +SORT_DSO_TO , +SORT_SYM_FROM , +SORT_SYM_TO , + SORT_MISPREDICT, };/*
tools/perf/util/symbol.h +13 -0
@@ -5,6 +5,7 @@#include <stdbool.h> #include <stdint.h> #include "map.h" +#include "../perf.h" #include <linux/list.h> #include <linux/rbtree.h> #include <stdio.h>
@@ -120,6 +121,18 @@ struct map_symbol {bool has_children; }; +struct addr_map_symbol { +struct map *map; +struct symbol *sym; +u64 addr; +}; + +struct branch_info { +struct addr_map_symbol from; +struct addr_map_symbol to; +struct branch_flags flags; +}; +struct addr_location {struct thread *thread;struct map *map;
PATCH v6 12/18 perf: add support for sampling taken branch to perf record
In reply to Stephane EranianFrom: Roberto Agostino Vitillo <[email protected]>
This patch adds a new option to enable taken branch stack sampling, i.e., leverage the PERF_SAMPLE_BRANCH_STACK feature of perf_events.
There is a new option to active this mode: -b.
It is possible to pass a set of filters to select the type of branches to sample.
The following filters are available:
- any : any type of branches
- any_call : any function call or system call
- any_ret : any function return or system call return
- any_ind : any indirect branch
- u: only when the branch target is at the user level
- k: only when the branch target is in the kernel
- hv: only when the branch target is in the hypervisor
Filters can be combined by passing a comma separated list to the option:
$ perf record -b any_call,u -e cycles:u branchy
--- tools/perf/Documentation/perf-record.txt | 25 ++++++++++ tools/perf/builtin-record.c | 74 ++++++++++++++++++++++++++++++ tools/perf/perf.h | 1 + tools/perf/util/evsel.c | 4 ++ 4 files changed, 104 insertions(+), 0 deletions(-)
tools/perf/Documentation/perf-record.txt +25 -0
@@ -152,6 +152,31 @@ an empty cgroup (monitor all the time) using, e.g., -G foo,,bar. Cgroups must ha corresponding events, i.e., they always refer to events defined earlier on the command line. +-b:: +--branch-stack:: +Enable taken branch stack sampling. Each sample captures a series of consecutive +taken branches. The number of branches captured with each sample depends on the +underlying hardware, the type of branches of interest, and the executed code. +It is possible to select the types of branches captured by enabling filters. The +following filters are defined: + + - any : any type of branches + - any_call: any function call or system call + - any_ret: any function return or system call return + - any_ind: any indirect branch + - u: only when the branch target is at the user level + - k: only when the branch target is in the kernel + - hv: only when the target is at the hypervisor level + ++ +At least one of any, any_call, any_ret, any_ind must be provided. The privilege levels may +be ommitted, in which case, the privilege levels of the associated event are applied to the +branch filter. Both kernel (k) and hypervisor (hv) privilege levels are subject to +permissions. When sampling on multiple events, branch stack sampling is enabled for all +the sampling events. The sampled branch type is the same for all events. +Note that taken branch sampling may not be available on all processors. +The various filters must be specified as a comma separated list: -b any_ret,u,k + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +74 -0
86 lines not on this page · raw message
tools/perf/perf.h +1 -0
@@ -222,6 +222,7 @@ struct perf_record_opts {unsigned int freq;unsigned int mmap_pages;unsigned int user_freq; +int branch_stack;u64 default_interval;u64 user_interval;const char *cpu_list;
tools/perf/util/evsel.c +4 -0
@@ -126,6 +126,10 @@ void perf_evsel__config(struct perf_evsel *evsel, struct perf_record_opts *opts) attr->watermark =0 ; attr->wakeup_events =1 ; } +if (opts->branch_stack) { + attr->sample_type |=PERF_SAMPLE_BRANCH_STACK ; + attr->branch_sample_type = opts->branch_stack; + } attr->mmap = track; attr->comm = track;
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf record: Add support for sampling taken branch
Commit-ID: bdfebd848f2a14e639031a0b0e61d7c7ee5e5fd2 Gitweb: http://git.kernel.org/tip/bdfebd848f2a14e639031a0b0e61d7c7ee5e5fd2 Author: Roberto Agostino Vitillo <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:02 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:05 +0100
perf record: Add support for sampling taken branch
This patch adds a new option to enable taken branch stack sampling, i.e., leverage the PERF_SAMPLE_BRANCH_STACK feature of perf_events.
There is a new option to active this mode: -b.
It is possible to pass a set of filters to select the type of branches to sample.
The following filters are available:
- any : any type of branches
- any_call : any function call or system call
- any_ret : any function return or system call return
- any_ind : any indirect branch
- u: only when the branch target is at the user level
- k: only when the branch target is in the kernel
- hv: only when the branch target is in the hypervisor
Filters can be combined by passing a comma separated list to the option:
$ perf record -b any_call,u -e cycles:u branchy
--- tools/perf/Documentation/perf-record.txt | 25 ++++++++++ tools/perf/builtin-record.c | 74 ++++++++++++++++++++++++++++++ tools/perf/perf.h | 1 + tools/perf/util/evsel.c | 4 ++ 4 files changed, 104 insertions(+), 0 deletions(-)
tools/perf/Documentation/perf-record.txt +25 -0
@@ -152,6 +152,31 @@ an empty cgroup (monitor all the time) using, e.g., -G foo,,bar. Cgroups must ha corresponding events, i.e., they always refer to events defined earlier on the command line. +-b:: +--branch-stack:: +Enable taken branch stack sampling. Each sample captures a series of consecutive +taken branches. The number of branches captured with each sample depends on the +underlying hardware, the type of branches of interest, and the executed code. +It is possible to select the types of branches captured by enabling filters. The +following filters are defined: + + - any : any type of branches + - any_call: any function call or system call + - any_ret: any function return or system call return + - any_ind: any indirect branch + - u: only when the branch target is at the user level + - k: only when the branch target is in the kernel + - hv: only when the target is at the hypervisor level + ++ +At least one of any, any_call, any_ret, any_ind must be provided. The privilege levels may +be ommitted, in which case, the privilege levels of the associated event are applied to the +branch filter. Both kernel (k) and hypervisor (hv) privilege levels are subject to +permissions. When sampling on multiple events, branch stack sampling is enabled for all +the sampling events. The sampled branch type is the same for all events. +Note that taken branch sampling may not be available on all processors. +The various filters must be specified as a comma separated list: -b any_ret,u,k + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-list[1]
tools/perf/builtin-record.c +74 -0
86 lines not on this page · raw message
tools/perf/perf.h +1 -0
@@ -222,6 +222,7 @@ struct perf_record_opts {unsigned int freq;unsigned int mmap_pages;unsigned int user_freq; +int branch_stack;u64 default_interval;u64 user_interval;const char *cpu_list;
tools/perf/util/evsel.c +4 -0
@@ -126,6 +126,10 @@ void perf_evsel__config(struct perf_evsel *evsel, struct perf_record_opts *opts) attr->watermark =0 ; attr->wakeup_events =1 ; } +if (opts->branch_stack) { + attr->sample_type |=PERF_SAMPLE_BRANCH_STACK ; + attr->branch_sample_type = opts->branch_stack; + } attr->mmap = track; attr->comm = track;
PATCH v6 13/18 perf: add support for taken branch sampling to perf report
In reply to Stephane EranianFrom: Roberto Agostino Vitillo <[email protected]>
This patch adds support for taken branch sampling, i.e, the PERF_SAMPLE_BRANCH_STACK feature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.
The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:
void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}int main(void)
{
unsigned long i;
for (i=0; i < N; i++)
f1(i);
return 0;
}
Here is the output captured on Nehalem, if we are
only interested in user level function calls.
$ perf record -b any_call,u -e cycles:u branchy
$ perf report -b --sort=symbol
52.34% [.] main [.] f1
24.04% [.] f1 [.] f3
23.60% [.] f1 [.] f2
0.01% [k] _IO_new_file_xsputn [k] _IO_file_overflow
0.01% [k] _IO_vfprintf_internal [k] _IO_new_file_xsputn
0.01% [k] _IO_vfprintf_internal [k] strchrnul
0.01% [k] __printf [k] _IO_vfprintf_internal
0.01% [k] main [k] __printfAbout half (52%) of the call branches captured are from main() -> f1(). The second half (24%+23%) is split in two equal shares between f1() -> f2(), f1() ->f3(). The output is as expected given the code.
It should be noted, that using -b in perf record does not eliminate information in the perf.data file. Consequently, a typical profile can also be obtained by perf report by simply not using its -b option.
It is possible to sort on branch related columns:
- dso_from, symbol_from
- dso_to, symbol_to
- mispredict
--- tools/perf/Documentation/perf-report.txt | 7 ++ tools/perf/builtin-report.c | 107 +++++++++++++++++++++++++++--- 2 files changed, 105 insertions(+), 9 deletions(-)
tools/perf/Documentation/perf-report.txt +7 -0
@@ -153,6 +153,13 @@ OPTIONS information which may be very large and thus may clutter the display. It currently includes: cpu and numa topology of the host system. +-b:: +--branch-stack:: + Use the addresses of sampled taken branches instead of the instruction + address to build the histograms. To generate meaningful output, the + perf.data file must have been obtained using perf record -b xxx where + xxx is a branch filter option. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-annotate[1]
tools/perf/builtin-report.c +98 -9
147 lines not on this page · raw message
Signature · 1 line
1.7.4.1
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
Em Thu, Feb 09, 2012 at 11:21:03PM +0100, Stephane Eranian escreveu:
Quoted 12 lines
From: Roberto Agostino Vitillo <[email protected]>
This patch adds support for taken branch sampling, i.e, the
PERF_SAMPLE_BRANCH_STACKfeature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
Quoted 7 lines
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:
Thanks for addressing the comments, looks good now,
- Arnaldo
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
In reply to Stephane EranianQuoted 45 lines · Stephane Eranian
From: Roberto Agostino Vitillo <[email protected]>
This patch adds support for taken branch sampling, i.e, the
PERF_SAMPLE_BRANCH_STACKfeature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)f2(); else f3(); }int main(void)
{
unsigned long i;for (i=0; i < N; i++)
f1(i);
return 0;
}Here is the output captured on Nehalem, if we are only interested in user level function calls.
$ perf record -b any_call,u -e cycles:u branchy
Quoted 9 lines
$ perf report -b --sort=symbol
52.34% [.] main [.] f1
24.04% [.] f1 [.] f3
23.60% [.] f1 [.] f2
0.01% [k] _IO_new_file_xsputn [k] _IO_file_overflow
0.01% [k] _IO_vfprintf_internal [k] _IO_new_file_xsputn
0.01% [k] _IO_vfprintf_internal [k] strchrnul
0.01% [k] __printf [k] _IO_vfprintf_internal
0.01% [k] main [k] __printfOk, nice feature.
One detail needs to be fixed though, if someone does:
perf record -b ...
then 'perf report' should *default* to the above branch stack output style, without having to specify -b again.
Having --branch/--no-branch present in perf report is fine if someone wants to force either direction, but the default absolutely must be picked up from the perf.data and should be the obvious behavior.
Other than that it looks good to me, so if this detail is fixed (can be a delta patch on top of the existing series) and there's no problems with it I can pick it up for v3.4.
Thanks,
Ingo
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
Quoted 58 lines · Ingo Molnar
Quoted 45 lines · Stephane Eranian
From: Roberto Agostino Vitillo <[email protected]>
This patch adds support for taken branch sampling, i.e, the
PERF_SAMPLE_BRANCH_STACKfeature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}
int main(void)
{
unsigned long i;for (i=0; i < N; i++)
f1(i);
return 0;
}Here is the output captured on Nehalem, if we are only interested in user level function calls.
$ perf record -b any_call,u -e cycles:u branchyQuoted 9 lines
$ perf report -b --sort=symbol 52.34% [.] main [.] f1 24.04% [.] f1 [.] f3 23.60% [.] f1 [.] f2 0.01% [k] _IO_new_file_xsputn [k] _IO_file_overflow 0.01% [k] _IO_vfprintf_internal [k] _IO_new_file_xsputn 0.01% [k] _IO_vfprintf_internal [k] strchrnul 0.01% [k] __printf [k] _IO_vfprintf_internal 0.01% [k] main [k] __printf
Quoted 9 lines
Ok, nice feature.
One detail needs to be fixed though, if someone does:
perf record -b ...
then 'perf report' should *default* to the above branch stack output style, without having to specify -b again.
Fair enough.
I'll check how we could do that. It's not so obvious as the code stands. I think we may need to add a new feature bit for that. It would avoid having to sniff either the cmdline, the event desc or worst the samples themselves.
Quoted 9 lines
Having --branch/--no-branch present in perf report is fine if someone wants to force either direction, but the default absolutely must be picked up from the perf.data and should be the obvious behavior.
Other than that it looks good to me, so if this detail is fixed (can be a delta patch on top of the existing series) and there's no problems with it I can pick it up for v3.4.
It'll be a delta patch.
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
Quoted 68 lines · Stephane Eranian
Quoted 58 lines · Ingo Molnar
Quoted 45 lines · Stephane Eranian
From: Roberto Agostino Vitillo <[email protected]>
This patch adds support for taken branch sampling, i.e, the
PERF_SAMPLE_BRANCH_STACKfeature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}
int main(void)
{
unsigned long i;for (i=0; i < N; i++)
f1(i);
return 0;
}Here is the output captured on Nehalem, if we are only interested in user level function calls.
$ perf record -b any_call,u -e cycles:u branchyQuoted 9 lines
$ perf report -b --sort=symbol 52.34% [.] main [.] f1 24.04% [.] f1 [.] f3 23.60% [.] f1 [.] f2 0.01% [k] _IO_new_file_xsputn [k] _IO_file_overflow 0.01% [k] _IO_vfprintf_internal [k] _IO_new_file_xsputn 0.01% [k] _IO_vfprintf_internal [k] strchrnul 0.01% [k] __printf [k] _IO_vfprintf_internal 0.01% [k] main [k] __printfQuoted 9 lines
Ok, nice feature.
One detail needs to be fixed though, if someone does:
perf record -b ...
then 'perf report' should *default* to the above branch stack output style, without having to specify -b again.
Quoted 6 lines
Fair enough.
I'll check how we could do that. It's not so obvious as the code stands. I think we may need to add a new feature bit for that. It would avoid having to sniff either the cmdline, the event desc or worst the samples themselves.
Yeah, a feature bit for that looks like the ideal solution anyway.
Btw., the exact perf record command line ought to be reproducible from the metadata stored in the perf.data.
It should be possible to type:
perf record --replay
or so, which takes a look at the perf.data and repeats that exact measurement. Something like this:
perf record -R -F 10000
could be used to repeat the last measurement, with higher frequency sampling.
Thanks,
Ingo
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
Quoted 73 lines · Ingo Molnar
Quoted 70 lines · Stephane Eranian
Quoted 58 lines · Ingo Molnar
Quoted 45 lines · Stephane Eranian
From: Roberto Agostino Vitillo <[email protected]>
This patch adds support for taken branch sampling, i.e, the
PERF_SAMPLE_BRANCH_STACKfeature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}
int main(void)
{
unsigned long i;for (i=0; i < N; i++)
f1(i);
return 0;
}Here is the output captured on Nehalem, if we are only interested in user level function calls.
$ perf record -b any_call,u -e cycles:u branchyQuoted 9 lines
$ perf report -b --sort=symbol 52.34% [.] main [.] f1 24.04% [.] f1 [.] f3 23.60% [.] f1 [.] f2 0.01% [k] _IO_new_file_xsputn [k] _IO_file_overflow 0.01% [k] _IO_vfprintf_internal [k] _IO_new_file_xsputn 0.01% [k] _IO_vfprintf_internal [k] strchrnul 0.01% [k] __printf [k] _IO_vfprintf_internal 0.01% [k] main [k] __printfQuoted 9 lines
Ok, nice feature.
One detail needs to be fixed though, if someone does:
perf record -b ...
then 'perf report' should *default* to the above branch stack output style, without having to specify -b again.
Fair enough.
Quoted 8 lines
Quoted 4 lines
I'll check how we could do that. It's not so obvious as the code stands. I think we may need to add a new feature bit for that. It would avoid having to sniff either the cmdline, the event desc or worst the samples themselves.
Yeah, a feature bit for that looks like the ideal solution anyway.
Ok, so I looked at that today. Adding the feature bit is trivial. But what's not easy is to get to the feature bit in perf report by the time we need it. Very quickly after parsing the options, we setup a bunch of things such as browser mode, sorting order based on the -b option. But to get to the feature bit, we would need to wait until after the session is created in __cmd_report() which is way later.
So we either hoist perf_session__new() very early, i.e., as soon as we have the filename or we write yet another parse_header() function just to get to the feature bits. I would rather choose the first option. But none is really pretty...
Arnaldo, any better idea?
Quoted 18 lines
Btw., the exact perf record command line ought to be reproducible from the metadata stored in the perf.data.
It should be possible to type:
perf record --replay
or so, which takes a look at the perf.data and repeats that exact measurement. Something like this:
perf record -R -F 10000
could be used to repeat the last measurement, with higher frequency sampling.
Thanks,
Ingo
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
Em Thu, Feb 23, 2012 at 05:53:53PM +0100, Stephane Eranian escreveu:
Quoted 8 lines
Ok, so I looked at that today. Adding the feature bit is trivial. But what's not easy is to get to the feature bit in perf report by the time we need it. Very quickly after parsing the options, we setup a bunch of things such as browser mode, sorting order based on the -b option. But to get to the feature bit, we would need to wait until after the session is created in
__cmd_report()which is way later.
Quoted 6 lines
So we either hoist
perf_session__new()very early, i.e., as soon as we have the filename or we write yet anotherparse_header()function just to get to the feature bits. I would rather choose the first option. But none is really pretty...Arnaldo, any better idea?
Humm, lemme see...
By the time we need it is when we do setup_sorting, right? Perhaps we can delay that till after we do perf_session__new and thus have read the headers...
/me looks at the code.
- Arnaldo
Re: [PATCH v6 13/18] perf: add support for taken branch sampling to perf report
Quoted 14 lines · Arnaldo Carvalho de Melo
Em Thu, Feb 23, 2012 at 05:53:53PM +0100, Stephane Eranian escreveu:
Quoted 13 lines
Ok, so I looked at that today. Adding the feature bit is trivial. But what's not easy is to get to the feature bit in perf report by the time we need it. Very quickly after parsing the options, we setup a bunch of things such as browser mode, sorting order based on the -b option. But to get to the feature bit, we would need to wait until after the session is created in
__cmd_report()which is way later.So we either hoist
perf_session__new()very early, i.e., as soon as we have the filename or we write yet anotherparse_header()function just to get to the feature bits. I would rather choose the first option. But none is really pretty...
Quoted 9 lines
Quoted 1 line
Arnaldo, any better idea?
Humm, lemme see...
By the time we need it is when we do
setup_sorting, right? Perhaps we can delay that till after we doperf_session__newand thus have read the headers...
Well, there is also the UI setup (use_stdio setup_browser and the likes). Then, yes, it's sorting.
Quoted 3 lines
/me looks at the code.
- Arnaldo
[tip:perf/hw-branch-sampling] perf report: Add support for taken branch sampling
In reply to Stephane EranianCommit-ID: b50311dc2ac1c04ad19163c2359910b25e16caf6 Gitweb: http://git.kernel.org/tip/b50311dc2ac1c04ad19163c2359910b25e16caf6 Author: Roberto Agostino Vitillo <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:03 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:05 +0100
perf report: Add support for taken branch sampling
This patch adds support for taken branch sampling, i.e, the PERF_SAMPLE_BRANCH_STACK feature to perf report. In other words, to display histograms based on taken branches rather than executed instructions addresses.
The new option is called -b and it takes no argument. To generate meaningful output, the perf.data must have been obtained using perf record -b xxx ... where xxx is a branch filter option.
The output shows symbols, modules, sorted by 'who branches where' the most often. The percentages reported in the first column refer to the total number of branches captured and not the usual number of samples.
Here is a quick example.
Here branchy is simple test program which looks as follows:
void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}int main(void)
{
unsigned long i;
for (i=0; i < N; i++)
f1(i);
return 0;
}
Here is the output captured on Nehalem, if we are only interested in user level function calls.
$ perf record -b any_call,u -e cycles:u branchy
$ perf report -b --sort=symbol
52.34% [.] main [.] f1
24.04% [.] f1 [.] f3
23.60% [.] f1 [.] f2
0.01% [k] _IO_new_file_xsputn [k] _IO_file_overflow
0.01% [k] _IO_vfprintf_internal [k] _IO_new_file_xsputn
0.01% [k] _IO_vfprintf_internal [k] strchrnul
0.01% [k] __printf [k] _IO_vfprintf_internal
0.01% [k] main [k] __printfAbout half (52%) of the call branches captured are from main() -> f1(). The second half (24%+23%) is split in two equal shares between f1() -> f2(), f1() ->f3(). The output is as expected given the code.
It should be noted, that using -b in perf record does not eliminate information in the perf.data file. Consequently, a typical profile can also be obtained by perf report by simply not using its -b option.
It is possible to sort on branch related columns:
- dso_from, symbol_from
- dso_to, symbol_to
- mispredict
--- tools/perf/Documentation/perf-report.txt | 7 ++ tools/perf/builtin-report.c | 107 +++++++++++++++++++++++++++--- 2 files changed, 105 insertions(+), 9 deletions(-)
tools/perf/Documentation/perf-report.txt +7 -0
@@ -153,6 +153,13 @@ OPTIONS information which may be very large and thus may clutter the display. It currently includes: cpu and numa topology of the host system. +-b:: +--branch-stack:: + Use the addresses of sampled taken branches instead of the instruction + address to build the histograms. To generate meaningful output, the + perf.data file must have been obtained using perf record -b xxx where + xxx is a branch filter option. + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-annotate[1]
tools/perf/builtin-report.c +98 -9
147 lines not on this page · raw message
PATCH v6 14/18 perf: fix endianness detection in perf.data
In reply to Stephane EranianThe current version of perf detects whether or not the perf.data file is written in a different endianness using the attr_size field in the header of the file. This field represents sizeof(struct perf_event_attr) as known to perf record. If the sizes do not match, then perf tries the byte-swapped version. If they match, then the tool assumes a different endianness.
The issue with the approach is that it assumes the size of perf_event_attr always has to match between perf record and perf report. However, the kernel perf_event ABI is extensible. New fields can be added to struct perf_event_attr. Consequently, it is not possible to use attr_size to detect endianness.
This patch takes another approach by using the magic number written at the beginning of the perf.data file to detect endianness. The magic number is an eight-byte signature. It's primary purpose is to identify (signature) a perf.data file. But it could also be used to encode the endianness.
The patch introduces a new value for this signature. The key difference is that the signature is written differently in the file depending on the endianness. Thus, by comparing the signature from the file with the tool's own signature it is possible to detect endianness. The new signature is "PERFILE2".
Backward compatiblity with existing perf.data file is ensured.
--- tools/perf/util/header.c | 77 ++++++++++++++++++++++++++++++++++++++-------- 1 files changed, 64 insertions(+), 13 deletions(-)
tools/perf/util/header.c +64 -13
@@ -63,9 +63,20 @@ char *perf_header__find_event(u64 id)return NULL ; } -static const char *__perf_magic ="PERFFILE" ; +/* +* magic2 = "PERFILE2" +* must be a numerical value to let the endianness +* determine the memory layout. That way we are able +* to detect endianness when reading the perf.data file +* back. +* +* we check for legacy (PERFFILE) format. +*/ +static const char *__perf_magic1 ="PERFFILE" ; +static const u64 __perf_magic2 =0x32454c4946524550ULL ; +static const u64 __perf_magic2_sw =0x50455246494c4532ULL ; -#define PERF_MAGIC (*(u64 *)__perf_magic) +#define PERF_MAGIC __perf_magic2struct perf_file_attr {struct perf_event_attr attr;
@@ -1620,24 +1631,59 @@ int perf_header__process_sections(struct perf_header *header, int fd,return err; } +static int check_magic_endian (u64 *magic,struct perf_file_header *header, +struct perf_header *ph) +{ +int ret; + +/* check for legacy format */ + ret =memcmp (magic, __perf_magic1,sizeof (*magic)); +if (ret ==0 ) { +pr_debug ("legacy perf.data format\n" ); +if (!header) +return -1 ; + +if (header->attr_size !=sizeof (struct perf_file_attr )) { +u64 attr_size =bswap_64 (header->attr_size); + +if (attr_size !=sizeof (struct perf_file_attr )) +return -1 ; + + ph->needs_swap = true; + } +return 0 ; + } + +/* check magic number with same endianness */ +if (*magic == __perf_magic2) +return 0 ; + +/* check magic number but opposite endianness */ +if (*magic != __perf_magic2_sw) +return -1 ; + + ph->needs_swap = true; + +return 0 ; +} +int perf_file_header__read (struct perf_file_header *header,struct perf_header *ph,int fd) { +int ret; +lseek (fd,0 ,SEEK_SET ); -if (readn (fd, header,sizeof (*header)) <=0 || -memcmp (&header->magic, __perf_magic,sizeof (header->magic))) + ret =readn (fd, header,sizeof (*header)); +if (ret <=0 )return -1 ; -if (header->attr_size !=sizeof (struct perf_file_attr )) { -u64 attr_size =bswap_64 (header->attr_size); - -if (attr_size !=sizeof (struct perf_file_attr )) -return -1 ; +if (check_magic_endian (&header->magic, header, ph) <0 ) +return -1 ; +if (ph->needs_swap) {mem_bswap_64 (header, offsetof(struct perf_file_header , - adds_features)); - ph->needs_swap = true; + adds_features)); }if (header->size !=sizeof (*header)) {
@@ -1873,8 +1919,13 @@ static int perf_file_header__read_pipe(struct perf_pipe_file_header *header, struct perf_header *ph, int fd, bool repipe) { - if (readn(fd, header, sizeof(*header)) <= 0 || - memcmp(&header->magic, __perf_magic, sizeof(header->magic))) + int ret; + + ret =readn (fd, header,sizeof (*header)); +if (ret <=0 ) +return -1 ; + +if (check_magic_endian (&header->magic,NULL , ph) <0 )return -1 ;if (repipe &&do_write (STDOUT_FILENO , header,sizeof (*header)) <0 )
Signature · 1 line
1.7.4.1
Re: [PATCH v6 14/18] perf: fix endianness detection in perf.data
Quoted 26 lines · Stephane Eranian
The current version of perf detects whether or not the perf.data file is written in a different endianness using the
attr_sizefield in the header of the file. This field representssizeof(struct perf_event_attr)as known to perf record. If the sizes do not match, then perf tries the byte-swapped version. If they match, then the tool assumes a different endianness.The issue with the approach is that it assumes the size of
perf_event_attralways has to match between perf record and perf report. However, the kernelperf_eventABI is extensible. New fields can be added to structperf_event_attr. Consequently, it is not possible to useattr_sizeto detect endianness.This patch takes another approach by using the magic number written at the beginning of the perf.data file to detect endianness. The magic number is an eight-byte signature. It's primary purpose is to identify (signature) a perf.data file. But it could also be used to encode the endianness.
The patch introduces a new value for this signature. The key difference is that the signature is written differently in the file depending on the endianness. Thus, by comparing the signature from the file with the tool's own signature it is possible to detect endianness. The new signature is "PERFILE2".
Quoted 4 lines
Backward compatiblity with existing perf.data file is ensured.
Signed-off-by Stephane Eranian <[email protected]>
PATCH v6 15/18 perf: add ABI reference sizes
In reply to Stephane EranianThis patch adds reference sizes for revision 1 and 2 of the perf_event ABI, i.e., the size of the perf_event_attr struct.
With Rev1: config2 was added = +8 bytes With Rev2: branch_sample_type was added = +8 bytes
Adds the definition for Rev1, Rev2.
This is useful for tools trying to decode the revision numbers based on the size of the struct.
--- include/linux/perf_event.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
include/linux/perf_event.h +2 -0
@@ -192,6 +192,8 @@ enum perf_event_read_format { };#define PERF_ATTR_SIZE_VER0 64/* sizeof first published struct */ +#define PERF_ATTR_SIZE_VER1 72/* add: config2 */ +#define PERF_ATTR_SIZE_VER2 80/* add: branch_sample_type */ /* * Hardware event_id to monitor via a performance monitoring event:
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf: Add ABI reference sizes
Commit-ID: cb5d76999029ae7a517cb07dfa732c1b5a934fc2 Gitweb: http://git.kernel.org/tip/cb5d76999029ae7a517cb07dfa732c1b5a934fc2 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:05 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:05 +0100
perf: Add ABI reference sizes
This patch adds reference sizes for revision 1 and 2 of the perf_event ABI, i.e., the size of the perf_event_attr struct.
With Rev1: config2 was added = +8 bytes With Rev2: branch_sample_type was added = +8 bytes
Adds the definition for Rev1, Rev2.
This is useful for tools trying to decode the revision numbers based on the size of the struct.
--- include/linux/perf_event.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
include/linux/perf_event.h +2 -0
@@ -192,6 +192,8 @@ enum perf_event_read_format { };#define PERF_ATTR_SIZE_VER0 64/* sizeof first published struct */ +#define PERF_ATTR_SIZE_VER1 72/* add: config2 */ +#define PERF_ATTR_SIZE_VER2 80/* add: branch_sample_type */ /* * Hardware event_id to monitor via a performance monitoring event:
PATCH v6 16/18 perf: enable reading of perf.data files from different ABI rev
In reply to Stephane EranianThis patch allows perf to process perf.data files generated using an ABI that has a different perf_event_attr struct size, i.e., a different ABI version.
The perf_event_attr can be extended, yet perf needs to cope with older perf.data files. Similarly, perf must be able to cope with a perf.data file which is using a newer version of the ABI than what it knows about.
This patch adds read_attr(), a routine that reads a perf_event_attr struct from a file incrementally based on its advertised size. If the on-file struct is smaller than what perf knows, then the extra fields are zeroed. If the on-file struct is bigger, then perf only uses what it knows about, the rest is skipped.
--- tools/perf/util/header.c | 53 ++++++++++++++++++++++++++++++++++++++++---- tools/perf/util/session.c | 4 +- 2 files changed, 50 insertions(+), 7 deletions(-)
tools/perf/util/header.c +48 -5
@@ -1959,6 +1959,51 @@ static int perf_header__read_pipe(struct perf_session *session, int fd)return 0 ; } +static int read_attr (int fd,struct perf_header *ph, +struct perf_file_attr *f_attr) +{ +struct perf_event_attr *attr = &f_attr->attr; +size_t sz, left; +size_t our_sz =sizeof (f_attr->attr); +int ret; + +memset (f_attr,0 ,sizeof (*f_attr)); + +/* read minimal guaranteed structure */ + ret =readn (fd, attr,PERF_ATTR_SIZE_VER0 ); +if (ret <=0 ) { +pr_debug ("cannot read %d bytes of header attr\n" , +PERF_ATTR_SIZE_VER0 ); +return -1 ; + } + +/* on file perf_event_attr size */ + sz = attr->size; +if (ph->needs_swap) + sz =bswap_32 (sz); + +if (sz ==0 ) { +/* assume ABI0 */ + sz =PERF_ATTR_SIZE_VER0 ; + }else if (sz > our_sz) { +pr_debug ("file uses a more recent and unsupported ABI" +" (%zu bytes extra)\n" , sz - our_sz); +return -1 ; + } +/* what we have not yet read and that we know about */ + left = sz -PERF_ATTR_SIZE_VER0 ; +if (left) { +void *ptr = attr; + ptr +=PERF_ATTR_SIZE_VER0 ; + + ret =readn (fd, ptr, left); + } +/* read perf_file_section, ids are read in caller */ + ret =readn (fd, &f_attr->ids,sizeof (f_attr->ids)); + +return ret <=0 ?-1 :0 ; +} +int perf_session__read_header (struct perf_session *session,int fd) {struct perf_header *header = &session->header;
@@ -1974,19 +2019,17 @@ int perf_session__read_header(struct perf_session *session, int fd)if (session->fd_pipe)return perf_header__read_pipe (session, fd); -if (perf_file_header__read (&f_header, header, fd) <0 ) { -pr_debug ("incompatible file format\n" ); +if (perf_file_header__read (&f_header, header, fd) <0 )return -EINVAL ; - } - nr_attrs = f_header.attrs.size /sizeof (f_attr); + nr_attrs = f_header.attrs.size / f_header.attr_size;lseek (fd, f_header.attrs.offset,SEEK_SET );for (i =0 ; i < nr_attrs; i++) {struct perf_evsel *evsel;off_t tmp; -if (readn (fd, &f_attr,sizeof (f_attr)) <=0 ) +if (read_attr (fd, header, &f_attr) <0 ) goto out_errno;if (header->needs_swap)
tools/perf/util/session.c +2 -2
@@ -24,7 +24,7 @@ static int perf_session__open(struct perf_session *self, bool force) self->fd =STDIN_FILENO ;if (perf_session__read_header (self, self->fd) <0 ) -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" );return 0 ; }
@@ -56,7 +56,7 @@ static int perf_session__open(struct perf_session *self, bool force) }if (perf_session__read_header (self, self->fd) <0 ) { -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" ); goto out_close; }
Signature · 1 line
1.7.4.1
Re: [PATCH v6 16/18] perf: enable reading of perf.data files from different ABI rev
Quoted 60 lines · Stephane Eranian
This patch allows perf to process perf.data files generated using an ABI that has a different
perf_event_attrstruct size, i.e., a different ABI version.The
perf_event_attrcan be extended, yet perf needs to cope with older perf.data files. Similarly, perf must be able to cope with a perf.data file which is using a newer version of the ABI than what it knows about.This patch adds
read_attr(), a routine that reads aperf_event_attrstruct from a file incrementally based on its advertised size. If the on-file struct is smaller than what perf knows, then the extra fields are zeroed. If the on-file struct is bigger, then perf only uses what it knows about, the rest is skipped.Signed-off-by Stephane Eranian <[email protected]>--- tools/perf/util/header.c | 53 ++++++++++++++++++++++++++++++++++++++++---- tools/perf/util/session.c | 4 +- 2 files changed, 50 insertions(+), 7 deletions(-)tools/perf/util/header.c +31
-0@@ -1959,6 +1959,51 @@ static int perf_header__read_pipe(struct perf_session *session, int fd)return 0 ; } +static int read_attr (int fd,struct perf_header *ph, +struct perf_file_attr *f_attr) +{ +struct perf_event_attr *attr = &f_attr->attr; +size_t sz, left; +size_t our_sz =sizeof (f_attr->attr); +int ret; + +memset (f_attr,0 ,sizeof (*f_attr)); + +/* read minimal guaranteed structure */ + ret =readn (fd, attr,PERF_ATTR_SIZE_VER0 ); +if (ret <=0 ) { +pr_debug ("cannot read %d bytes of header attr\n" , +PERF_ATTR_SIZE_VER0 ); +return -1 ; + } + +/* on file perf_event_attr size */ + sz = attr->size; +if (ph->needs_swap) + sz =bswap_32 (sz); + +if (sz ==0 ) { +/* assume ABI0 */ + sz =PERF_ATTR_SIZE_VER0 ; + }else if (sz > our_sz) { +pr_debug ("file uses a more recent and unsupported ABI" +" (%zu bytes extra)\n" , sz - our_sz); +return -1 ; + }
tools/perf/util/header.c static int perf_header__read_pipe(struct perf_session *session, int fd) · line 2002
Quoted 10 lines
tools/perf/util/header.c +10
-0+/* what we have not yet read and that we know about */ + left = sz -PERF_ATTR_SIZE_VER0 ; +if (left) { +void *ptr = attr; + ptr +=PERF_ATTR_SIZE_VER0 ; + + ret =readn (fd, ptr, left); + } +/* read perf_file_section, ids are read in caller */ + ret =readn (fd, &f_attr->ids,sizeof (f_attr->ids));
I thought the above 2 lines were to be removed.
David
Quoted 52 lines
tools/perf/util/header.c +7
-5+ +return ret <=0 ?-1 :0 ; +} +int perf_session__read_header (struct perf_session *session,int fd) {struct perf_header *header = &session->header;@@ -1974,19 +2019,17 @@ int perf_session__read_header(struct perf_session *session, int fd)if (session->fd_pipe)return perf_header__read_pipe (session, fd); -if (perf_file_header__read (&f_header, header, fd) <0 ) { -pr_debug ("incompatible file format\n" ); +if (perf_file_header__read (&f_header, header, fd) <0 )return -EINVAL ; - } - nr_attrs = f_header.attrs.size /sizeof (f_attr); + nr_attrs = f_header.attrs.size / f_header.attr_size;lseek (fd, f_header.attrs.offset,SEEK_SET );for (i =0 ; i < nr_attrs; i++) {struct perf_evsel *evsel;off_t tmp; -if (readn (fd, &f_attr,sizeof (f_attr)) <=0 ) +if (read_attr (fd, header, &f_attr) <0 ) goto out_errno;if (header->needs_swap)tools/perf/util/session.c +2
-2@@ -24,7 +24,7 @@ static int perf_session__open(struct perf_session *self, bool force) self->fd =STDIN_FILENO ;if (perf_session__read_header (self, self->fd) <0 ) -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" );return 0 ; }@@ -56,7 +56,7 @@ static int perf_session__open(struct perf_session *self, bool force) }if (perf_session__read_header (self, self->fd) <0 ) { -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" ); goto out_close; }
Re: [PATCH v6 16/18] perf: enable reading of perf.data files from different ABI rev
Quoted 65 lines · David Ahern
Quoted 64 lines · Stephane Eranian
This patch allows perf to process perf.data files generated using an ABI that has a different
perf_event_attrstruct size, i.e., a different ABI version.The
perf_event_attrcan be extended, yet perf needs to cope with older perf.data files. Similarly, perf must be able to cope with a perf.data file which is using a newer version of the ABI than what it knows about.This patch adds
read_attr(), a routine that reads aperf_event_attrstruct from a file incrementally based on its advertised size. If the on-file struct is smaller than what perf knows, then the extra fields are zeroed. If the on-file struct is bigger, then perf only uses what it knows about, the rest is skipped.Signed-off-by Stephane Eranian <[email protected]>---
tools/perf/util/header.c | 53 ++++++++++++++++++++++++++++++++++++++++----tools/perf/util/session.c| 4 +-2 files changed, 50 insertions(+), 7 deletions(-)tools/perf/util/header.c +35
-0@@ -1959,6 +1959,51 @@ static int perf_header__read_pipe(struct perf_session *session, int fd) return 0; } +static int read_attr (int fd,struct perf_header *ph, + struct perf_file_attr *f_attr) +{ + struct perf_event_attr *attr = &f_attr->attr; + size_t sz, left; + size_t our_sz = sizeof(f_attr->attr); + int ret; + + memset(f_attr, 0, sizeof(*f_attr)); + + /* read minimal guaranteed structure */ + ret = readn(fd, attr, PERF_ATTR_SIZE_VER0); + if (ret <= 0) { + pr_debug("cannot read %d bytes of header attr\n", + PERF_ATTR_SIZE_VER0); + return -1; + } + + /* on file perf_event_attr size */ + sz = attr->size; + if (ph->needs_swap) + sz = bswap_32(sz); + + if (sz == 0) { + /* assume ABI0 */ + sz = PERF_ATTR_SIZE_VER0; + } else if (sz > our_sz) { + pr_debug("file uses a more recent and unsupported ABI" + " (%zu bytes extra)\n", sz - our_sz); + return -1; + } + /* what we have not yet read and that we know about */ + left = sz - PERF_ATTR_SIZE_VER0; + if (left) { + void *ptr = attr;
Quoted 9 lines
tools/perf/util/header.c static int perf_header__read_pipe(struct perf_session *session, int fd) · line 2002
Quoted 6 lines
tools/perf/util/header.c +6
-0+ ptr += PERF_ATTR_SIZE_VER0; + + ret = readn(fd, ptr, left); + } + /* read perf_file_section, ids are read in caller */ + ret = readn(fd, &f_attr->ids, sizeof(f_attr->ids));I thought the above 2 lines were to be removed.
No, They are needed to get the offsets of the ids[] table. The field name are confusing. This is not reading the ids[], but just the offset of where the table is. Without that you get bogus offsets. I ran into that bug during testing...
Quoted 55 lines
David
Quoted 52 lines
tools/perf/util/header.c +7
-5+ + return ret <= 0 ? -1 : 0; +} + int perf_session__read_header(struct perf_session *session, int fd) { struct perf_header *header = &session->header;@@ -1974,19 +2019,17 @@ int perf_session__read_header(struct perf_session *session, int fd) if (session->fd_pipe) return perf_header__read_pipe(session, fd); - if (perf_file_header__read(&f_header, header, fd) < 0) { - pr_debug("incompatible file format\n"); + if (perf_file_header__read(&f_header, header, fd) < 0) return -EINVAL; - } - nr_attrs = f_header.attrs.size / sizeof(f_attr); + nr_attrs = f_header.attrs.size / f_header.attr_size; lseek(fd, f_header.attrs.offset, SEEK_SET); for (i = 0; i < nr_attrs; i++) { struct perf_evsel *evsel; off_t tmp; - if (readn(fd, &f_attr, sizeof(f_attr)) <= 0) + if (read_attr(fd, header, &f_attr) < 0) goto out_errno; if (header->needs_swap)tools/perf/util/session.c +2
-2@@ -24,7 +24,7 @@ static int perf_session__open(struct perf_session *self, bool force) self->fd = STDIN_FILENO; if (perf_session__read_header(self, self->fd) < 0) - pr_err("incompatible file format"); + pr_err("incompatible file format (rerun with -v to learn more)"); return 0; }@@ -56,7 +56,7 @@ static int perf_session__open(struct perf_session *self, bool force) } if (perf_session__read_header(self, self->fd) < 0) { - pr_err("incompatible file format"); + pr_err("incompatible file format (rerun with -v to learn more)"); goto out_close; }
Re: [PATCH v6 16/18] perf: enable reading of perf.data files from different ABI rev
Quoted 10 lines · Stephane Eranian
Quoted 5 lines
tools/perf/util/header.c static int perf_header__read_pipe(struct perf_session *session, int fd) · line 2002
Quoted 2 lines
tools/perf/util/header.c +2
-0+/* read perf_file_section, ids are read in caller */ + ret =readn (fd, &f_attr->ids,sizeof (f_attr->ids));I thought the above 2 lines were to be removed.
No, They are needed to get the offsets of the ids[] table. The field name are confusing. This is not reading the ids[], but just the offset of where the table is. Without that you get bogus offsets. I ran into that bug during testing...
Hmm.... ok, so the perf_file_attr is now read in back to back reads -- attr then ids -- versus today where it gets both in one readn.
Quoted 57 lines
Quoted 57 lines
David
Quoted 52 lines
tools/perf/util/header.c +7
-5+ +return ret <=0 ?-1 :0 ; +} +int perf_session__read_header (struct perf_session *session,int fd) {struct perf_header *header = &session->header;@@ -1974,19 +2019,17 @@ int perf_session__read_header(struct perf_session *session, int fd)if (session->fd_pipe)return perf_header__read_pipe (session, fd); -if (perf_file_header__read (&f_header, header, fd) <0 ) { -pr_debug ("incompatible file format\n" ); +if (perf_file_header__read (&f_header, header, fd) <0 )return -EINVAL ; - } - nr_attrs = f_header.attrs.size /sizeof (f_attr); + nr_attrs = f_header.attrs.size / f_header.attr_size;lseek (fd, f_header.attrs.offset,SEEK_SET );for (i =0 ; i < nr_attrs; i++) {struct perf_evsel *evsel;off_t tmp; -if (readn (fd, &f_attr,sizeof (f_attr)) <=0 ) +if (read_attr (fd, header, &f_attr) <0 ) goto out_errno;if (header->needs_swap)tools/perf/util/session.c +2
-2@@ -24,7 +24,7 @@ static int perf_session__open(struct perf_session *self, bool force) self->fd =STDIN_FILENO ;if (perf_session__read_header (self, self->fd) <0 ) -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" );return 0 ; }@@ -56,7 +56,7 @@ static int perf_session__open(struct perf_session *self, bool force) }if (perf_session__read_header (self, self->fd) <0 ) { -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" ); goto out_close; }
Re: [PATCH v6 16/18] perf: enable reading of perf.data files from different ABI rev
Quoted 9 lines · David Ahern
Quoted 6 lines · Stephane Eranian
Quoted 5 lines
Quoted 2 lines
+ /* read
perf_file_section, ids are read in caller */
+ ret =readn(fd, &f_attr->ids, sizeof(f_attr->ids));I thought the above 2 lines were to be removed.
Quoted 9 lines
Quoted 4 lines
No, They are needed to get the offsets of the ids[] table. The field name are confusing. This is not reading the ids[], but just the offset of where the table is. Without that you get bogus offsets. I ran into that bug during testing...
Hmm.... ok, so the
perf_file_attris now read in back to back reads -- attr then ids -- versus today where it gets both in one readn.
Yes, you have to do that in case on file sizeof(perf_event_attr) < tool sizeof(perf_event_attr). perf_event_attr is encapsulated into perf_file_attr.
Quoted 57 lines
Quoted 57 lines
Quoted 57 lines
David
Quoted 52 lines
tools/perf/util/header.c +4
-0+ + return ret <= 0 ? -1 : 0; +} +int
perf_session__read_header(struct perf_session *session, int fd){
structperf_header*header =&session->header;tools/perf/util/header.c +3
-5@@ -1974,19 +2019,17 @@ int perf_session__read_header(struct perf_session *session, int fd) if (session->fd_pipe) return perf_header__read_pipe(session, fd); - if (perf_file_header__read(&f_header, header, fd) < 0) { - pr_debug("incompatible file format\n"); + if (perf_file_header__read(&f_header, header, fd) < 0) return -EINVAL; - } - nr_attrs = f_header.attrs.size / sizeof(f_attr); + nr_attrs = f_header.attrs.size / f_header.attr_size; lseek(fd, f_header.attrs.offset, SEEK_SET); for (i = 0; i < nr_attrs; i++) { struct perf_evsel *evsel; off_t tmp; - if (readn(fd, &f_attr, sizeof(f_attr)) <= 0) + if (read_attr(fd, header, &f_attr) < 0) goto out_errno; if (header->needs_swap)tools/perf/util/session.c +2
-2@@ -24,7 +24,7 @@ static int perf_session__open(struct perf_session *self, bool force) self->fd = STDIN_FILENO; if (perf_session__read_header(self, self->fd) < 0) - pr_err("incompatible file format"); + pr_err("incompatible file format (rerun with -v to learn more)"); return 0; }@@ -56,7 +56,7 @@ static int perf_session__open(struct perf_session *self, bool force) } if (perf_session__read_header(self, self->fd) < 0) { - pr_err("incompatible file format"); + pr_err("incompatible file format (rerun with -v to learn more)"); goto out_close; }
Re: [PATCH v6 16/18] perf: enable reading of perf.data files from different ABI rev
In reply to Stephane EranianQuoted 9 lines · Stephane Eranian
This patch allows perf to process perf.data files generated using an ABI that has a different
perf_event_attrstruct size, i.e., a different ABI version.The
perf_event_attrcan be extended, yet perf needs to cope with older perf.data files. Similarly, perf must be able to cope with a perf.data file which is using a newer version of the ABI than what it knows about.
Quoted 7 lines
This patch adds
read_attr(), a routine that reads aperf_event_attrstruct from a file incrementally based on its advertised size. If the on-file struct is smaller than what perf knows, then the extra fields are zeroed. If the on-file struct is bigger, then perf only uses what it knows about, the rest is skipped.Signed-off-by Stephane Eranian <[email protected]>
[tip:perf/hw-branch-sampling] perf tools: Enable reading of perf. data files from different ABI rev
In reply to Stephane EranianCommit-ID: 69996df486fc3921bbaaa17fca0d68f537f9eabf Gitweb: http://git.kernel.org/tip/69996df486fc3921bbaaa17fca0d68f537f9eabf Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:06 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:06 +0100
perf tools: Enable reading of perf.data files from different ABI rev
This patch allows perf to process perf.data files generated using an ABI that has a different perf_event_attr struct size, i.e., a different ABI version.
The perf_event_attr can be extended, yet perf needs to cope with older perf.data files. Similarly, perf must be able to cope with a perf.data file which is using a newer version of the ABI than what it knows about.
This patch adds read_attr(), a routine that reads a perf_event_attr struct from a file incrementally based on its advertised size. If the on-file struct is smaller than what perf knows, then the extra fields are zeroed. If the on-file struct is bigger, then perf only uses what it knows about, the rest is skipped.
--- tools/perf/util/header.c | 53 ++++++++++++++++++++++++++++++++++++++++---- tools/perf/util/session.c | 4 +- 2 files changed, 50 insertions(+), 7 deletions(-)
tools/perf/util/header.c +48 -5
@@ -1973,6 +1973,51 @@ static int perf_header__read_pipe(struct perf_session *session, int fd)return 0 ; } +static int read_attr (int fd,struct perf_header *ph, +struct perf_file_attr *f_attr) +{ +struct perf_event_attr *attr = &f_attr->attr; +size_t sz, left; +size_t our_sz =sizeof (f_attr->attr); +int ret; + +memset (f_attr,0 ,sizeof (*f_attr)); + +/* read minimal guaranteed structure */ + ret =readn (fd, attr,PERF_ATTR_SIZE_VER0 ); +if (ret <=0 ) { +pr_debug ("cannot read %d bytes of header attr\n" , +PERF_ATTR_SIZE_VER0 ); +return -1 ; + } + +/* on file perf_event_attr size */ + sz = attr->size; +if (ph->needs_swap) + sz =bswap_32 (sz); + +if (sz ==0 ) { +/* assume ABI0 */ + sz =PERF_ATTR_SIZE_VER0 ; + }else if (sz > our_sz) { +pr_debug ("file uses a more recent and unsupported ABI" +" (%zu bytes extra)\n" , sz - our_sz); +return -1 ; + } +/* what we have not yet read and that we know about */ + left = sz -PERF_ATTR_SIZE_VER0 ; +if (left) { +void *ptr = attr; + ptr +=PERF_ATTR_SIZE_VER0 ; + + ret =readn (fd, ptr, left); + } +/* read perf_file_section, ids are read in caller */ + ret =readn (fd, &f_attr->ids,sizeof (f_attr->ids)); + +return ret <=0 ?-1 :0 ; +} +int perf_session__read_header (struct perf_session *session,int fd) {struct perf_header *header = &session->header;
@@ -1988,19 +2033,17 @@ int perf_session__read_header(struct perf_session *session, int fd)if (session->fd_pipe)return perf_header__read_pipe (session, fd); -if (perf_file_header__read (&f_header, header, fd) <0 ) { -pr_debug ("incompatible file format\n" ); +if (perf_file_header__read (&f_header, header, fd) <0 )return -EINVAL ; - } - nr_attrs = f_header.attrs.size /sizeof (f_attr); + nr_attrs = f_header.attrs.size / f_header.attr_size;lseek (fd, f_header.attrs.offset,SEEK_SET );for (i =0 ; i < nr_attrs; i++) {struct perf_evsel *evsel;off_t tmp; -if (readn (fd, &f_attr,sizeof (f_attr)) <=0 ) +if (read_attr (fd, header, &f_attr) <0 ) goto out_errno;if (header->needs_swap)
tools/perf/util/session.c +2 -2
@@ -24,7 +24,7 @@ static int perf_session__open(struct perf_session *self, bool force) self->fd =STDIN_FILENO ;if (perf_session__read_header (self, self->fd) <0 ) -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" );return 0 ; }
@@ -56,7 +56,7 @@ static int perf_session__open(struct perf_session *self, bool force) }if (perf_session__read_header (self, self->fd) <0 ) { -pr_err ("incompatible file format" ); +pr_err ("incompatible file format (rerun with -v to learn more)" ); goto out_close; }
PATCH v6 17/18 perf: fix bug print_event_desc()
In reply to Stephane EranianThis patches cleans up local variable types for msz and ret. They need to be size_t and ssize_t respectively.
It also fixes a bug whereby perf would not read attr struct with a different size than what it knows about.
--- tools/perf/util/header.c | 19 +++++++++---------- 1 files changed, 9 insertions(+), 10 deletions(-)
tools/perf/util/header.c +9 -10
@@ -1144,8 +1144,9 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)uint64_t id;void *buf =NULL ;char *str; -u32 nre, sz, nr, i, j, msz; -int ret; +u32 nre, sz, nr, i, j; +ssize_t ret; +size_t msz;/* number of events */ ret =read (fd, &nre,sizeof (nre));
@@ -1162,25 +1163,23 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)if (ph->needs_swap) sz =bswap_32 (sz); -/* -* ensure it is at least to our ABI rev -*/ -if (sz < (u32 )sizeof (attr)) - goto error; -memset (&attr,0 ,sizeof (attr)); -/* read entire region to sync up to next field */ +/* buffer to hold on file attr struct */ buf =malloc (sz);if (!buf) goto error; msz =sizeof (attr); -if (sz < msz) +if (sz < (ssize_t )msz) msz = sz;for (i =0 ; i < nre; i++) { +/* +* must read entire on-file attr struct to +* sync up with layout. +*/ ret =read (fd, buf, sz);if (ret != (ssize_t )sz) goto error;
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf tools: Fix ABI compatibility bug in print_event_desc()
Commit-ID: 62db90681c0db8c76e5db006b929a2edd5d12ae6 Gitweb: http://git.kernel.org/tip/62db90681c0db8c76e5db006b929a2edd5d12ae6 Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:07 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:06 +0100
perf tools: Fix ABI compatibility bug in print_event_desc()
This patches cleans up local variable types for msz and ret. They need to be size_t and ssize_t respectively.
It also fixes a bug whereby perf would not read attr struct with a different size than what it knows about.
--- tools/perf/util/header.c | 19 +++++++++---------- 1 files changed, 9 insertions(+), 10 deletions(-)
tools/perf/util/header.c +9 -10
@@ -1144,8 +1144,9 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)uint64_t id;void *buf =NULL ;char *str; -u32 nre, sz, nr, i, j, msz; -int ret; +u32 nre, sz, nr, i, j; +ssize_t ret; +size_t msz;/* number of events */ ret =read (fd, &nre,sizeof (nre));
@@ -1162,25 +1163,23 @@ static void print_event_desc(struct perf_header *ph, int fd, FILE *fp)if (ph->needs_swap) sz =bswap_32 (sz); -/* -* ensure it is at least to our ABI rev -*/ -if (sz < (u32 )sizeof (attr)) - goto error; -memset (&attr,0 ,sizeof (attr)); -/* read entire region to sync up to next field */ +/* buffer to hold on file attr struct */ buf =malloc (sz);if (!buf) goto error; msz =sizeof (attr); -if (sz < msz) +if (sz < (ssize_t )msz) msz = sz;for (i =0 ; i < nre; i++) { +/* +* must read entire on-file attr struct to +* sync up with layout. +*/ ret =read (fd, buf, sz);if (ret != (ssize_t )sz) goto error;
PATCH v6 18/18 perf: make perf able to read file from older ABIs
In reply to Stephane EranianThis patches provides a way to handle legacy perf.data files. Legacy files are those using the older PERFFILE signature.
For those, it is still necessary to detect endianness but without comparing their header->attr_size with the tool's own version as it may be different. Instead, we use a reference table for all known sizes from the legacy era.
We try all the combinations for sizes and endianness. If we find a match, we proceed, otherwise we return: "incompatible file format".
This is also done for the pipe-mode file format.
--- tools/perf/util/header.c | 122 +++++++++++++++++++++++++++++++++++----------- 1 files changed, 94 insertions(+), 28 deletions(-)
tools/perf/util/header.c +94 -28
@@ -1630,35 +1630,101 @@ int perf_header__process_sections(struct perf_header *header, int fd,return err; } -static int check_magic_endian (u64 *magic,struct perf_file_header *header, -struct perf_header *ph) +static const int attr_file_abi_sizes[] = { + [0 ] =PERF_ATTR_SIZE_VER0 , + [1 ] =PERF_ATTR_SIZE_VER1 , +0 , +}; + +/* +* In the legacy file format, the magic number is not used to encode endianness. +* hdr_sz was used to encode endianness. But given that hdr_sz can vary based +* on ABI revisions, we need to try all combinations for all endianness to +* detect the endianness. +*/ +static int try_all_file_abis (uint64_t hdr_sz,struct perf_header *ph) { -int ret; +uint64_t ref_size, attr_size; +int i; -/* check for legacy format */ - ret =memcmp (magic, __perf_magic1,sizeof (*magic)); -if (ret ==0 ) { -pr_debug ("legacy perf.data format\n" ); -if (!header) -return -1 ; +for (i =0 ; attr_file_abi_sizes[i]; i++) { + ref_size = attr_file_abi_sizes[i] + +sizeof (struct perf_file_section ); +if (hdr_sz != ref_size) { + attr_size =bswap_64 (hdr_sz); +if (attr_size != ref_size) +continue ; -if (header->attr_size !=sizeof (struct perf_file_attr )) { -u64 attr_size =bswap_64 (header->attr_size); + ph->needs_swap = true; + } +pr_debug ("ABI%d perf.data file detected, need_swap=%d\n" , + i, + ph->needs_swap); +return 0 ; + } +/* could not determine endianness */ +return -1 ; +} -if (attr_size !=sizeof (struct perf_file_attr )) -return -1 ; +#define PERF_PIPE_HDR_VER0 16 + +static const size_t attr_pipe_abi_sizes[] = { + [0 ] =PERF_PIPE_HDR_VER0 , +0 , +}; + +/* +* In the legacy pipe format, there is an implicit assumption that endiannesss +* between host recording the samples, and host parsing the samples is the +* same. This is not always the case given that the pipe output may always be +* redirected into a file and analyzed on a different machine with possibly a +* different endianness and perf_event ABI revsions in the perf tool itself. +*/ +static int try_all_pipe_abis (uint64_t hdr_sz,struct perf_header *ph) +{ +u64 attr_size; +int i; + +for (i =0 ; attr_pipe_abi_sizes[i]; i++) { +if (hdr_sz != attr_pipe_abi_sizes[i]) { + attr_size =bswap_64 (hdr_sz); +if (attr_size != hdr_sz) +continue ; ph->needs_swap = true; } +pr_debug ("Pipe ABI%d perf.data file detected\n" , i);return 0 ; } +return -1 ; +} + +static int check_magic_endian (u64 magic,uint64_t hdr_sz, +bool is_pipe,struct perf_header *ph) +{ +int ret; + +/* check for legacy format */ + ret =memcmp (&magic, __perf_magic1,sizeof (magic)); +if (ret ==0 ) { +pr_debug ("legacy perf.data format\n" ); +if (is_pipe) +return try_all_pipe_abis (hdr_sz, ph); + +return try_all_file_abis (hdr_sz, ph); + } +/* +* the new magic number serves two purposes: +* - unique number to identify actual perf.data files +* - encode endianness of file +*/ -/* check magic number with same endianness */ -if (*magic == __perf_magic2) +/* check magic number with one endianness */ +if (magic == __perf_magic2)return 0 ; -/* check magic number but opposite endianness */ -if (*magic != __perf_magic2_sw) +/* check magic number with opposite endianness */ +if (magic != __perf_magic2_sw)return -1 ; ph->needs_swap = true;
@@ -1677,8 +1743,11 @@ int perf_file_header__read(struct perf_file_header *header,if (ret <=0 )return -1 ; -if (check_magic_endian (&header->magic, header, ph) <0 ) +if (check_magic_endian (header->magic, + header->attr_size, false, ph) <0 ) { +pr_debug ("magic/endian check failed\n" );return -1 ; + }if (ph->needs_swap) { mem_bswap_64(header, offsetof(struct perf_file_header,
@@ -1924,21 +1993,17 @@ static int perf_file_header__read_pipe(struct perf_pipe_file_header *header,if (ret <=0 )return -1 ; -if (check_magic_endian (&header->magic,NULL , ph) <0 ) +if (check_magic_endian (header->magic, header->size, true, ph) <0 ) { +pr_debug ("endian/magic failed\n" );return -1 ; + } + +if (ph->needs_swap) + header->size =bswap_64 (header->size);if (repipe &&do_write (STDOUT_FILENO , header,sizeof (*header)) <0 )return -1 ; -if (header->size !=sizeof (*header)) { -u64 size =bswap_64 (header->size); - -if (size !=sizeof (*header)) -return -1 ; - - ph->needs_swap = true; - } -return 0 ; }
@@ -1978,6 +2043,7 @@ static int read_attr(int fd, struct perf_header *ph,/* on file perf_event_attr size */ sz = attr->size; +if (ph->needs_swap) sz =bswap_32 (sz);
Signature · 1 line
1.7.4.1
[tip:perf/hw-branch-sampling] perf tools: Make perf able to read files from older ABIs
Commit-ID: 114382a0aea97974803c942f106d462cbca5c64d Gitweb: http://git.kernel.org/tip/114382a0aea97974803c942f106d462cbca5c64d Author: Stephane Eranian <[email protected]> AuthorDate: Thu, 9 Feb 2012 23:21:08 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Fri, 9 Mar 2012 08:26:07 +0100
perf tools: Make perf able to read files from older ABIs
This patches provides a way to handle legacy perf.data files. Legacy files are those using the older PERFFILE signature.
For those, it is still necessary to detect endianness but without comparing their header->attr_size with the tool's own version as it may be different. Instead, we use a reference table for all known sizes from the legacy era.
We try all the combinations for sizes and endianness. If we find a match, we proceed, otherwise we return: "incompatible file format".
This is also done for the pipe-mode file format.
--- tools/perf/util/header.c | 122 +++++++++++++++++++++++++++++++++++----------- 1 files changed, 94 insertions(+), 28 deletions(-)
tools/perf/util/header.c +94 -28
@@ -1803,35 +1803,101 @@ out_free:return err; } -static int check_magic_endian (u64 *magic,struct perf_file_header *header, -struct perf_header *ph) +static const int attr_file_abi_sizes[] = { + [0 ] =PERF_ATTR_SIZE_VER0 , + [1 ] =PERF_ATTR_SIZE_VER1 , +0 , +}; + +/* +* In the legacy file format, the magic number is not used to encode endianness. +* hdr_sz was used to encode endianness. But given that hdr_sz can vary based +* on ABI revisions, we need to try all combinations for all endianness to +* detect the endianness. +*/ +static int try_all_file_abis (uint64_t hdr_sz,struct perf_header *ph) { -int ret; +uint64_t ref_size, attr_size; +int i; -/* check for legacy format */ - ret =memcmp (magic, __perf_magic1,sizeof (*magic)); -if (ret ==0 ) { -pr_debug ("legacy perf.data format\n" ); -if (!header) -return -1 ; +for (i =0 ; attr_file_abi_sizes[i]; i++) { + ref_size = attr_file_abi_sizes[i] + +sizeof (struct perf_file_section ); +if (hdr_sz != ref_size) { + attr_size =bswap_64 (hdr_sz); +if (attr_size != ref_size) +continue ; -if (header->attr_size !=sizeof (struct perf_file_attr )) { -u64 attr_size =bswap_64 (header->attr_size); + ph->needs_swap = true; + } +pr_debug ("ABI%d perf.data file detected, need_swap=%d\n" , + i, + ph->needs_swap); +return 0 ; + } +/* could not determine endianness */ +return -1 ; +} -if (attr_size !=sizeof (struct perf_file_attr )) -return -1 ; +#define PERF_PIPE_HDR_VER0 16 + +static const size_t attr_pipe_abi_sizes[] = { + [0 ] =PERF_PIPE_HDR_VER0 , +0 , +}; + +/* +* In the legacy pipe format, there is an implicit assumption that endiannesss +* between host recording the samples, and host parsing the samples is the +* same. This is not always the case given that the pipe output may always be +* redirected into a file and analyzed on a different machine with possibly a +* different endianness and perf_event ABI revsions in the perf tool itself. +*/ +static int try_all_pipe_abis (uint64_t hdr_sz,struct perf_header *ph) +{ +u64 attr_size; +int i; + +for (i =0 ; attr_pipe_abi_sizes[i]; i++) { +if (hdr_sz != attr_pipe_abi_sizes[i]) { + attr_size =bswap_64 (hdr_sz); +if (attr_size != hdr_sz) +continue ; ph->needs_swap = true; } +pr_debug ("Pipe ABI%d perf.data file detected\n" , i);return 0 ; } +return -1 ; +} + +static int check_magic_endian (u64 magic,uint64_t hdr_sz, +bool is_pipe,struct perf_header *ph) +{ +int ret; + +/* check for legacy format */ + ret =memcmp (&magic, __perf_magic1,sizeof (magic)); +if (ret ==0 ) { +pr_debug ("legacy perf.data format\n" ); +if (is_pipe) +return try_all_pipe_abis (hdr_sz, ph); + +return try_all_file_abis (hdr_sz, ph); + } +/* +* the new magic number serves two purposes: +* - unique number to identify actual perf.data files +* - encode endianness of file +*/ -/* check magic number with same endianness */ -if (*magic == __perf_magic2) +/* check magic number with one endianness */ +if (magic == __perf_magic2)return 0 ; -/* check magic number but opposite endianness */ -if (*magic != __perf_magic2_sw) +/* check magic number with opposite endianness */ +if (magic != __perf_magic2_sw)return -1 ; ph->needs_swap = true;
@@ -1850,8 +1916,11 @@ int perf_file_header__read(struct perf_file_header *header,if (ret <=0 )return -1 ; -if (check_magic_endian (&header->magic, header, ph) <0 ) +if (check_magic_endian (header->magic, + header->attr_size, false, ph) <0 ) { +pr_debug ("magic/endian check failed\n" );return -1 ; + }if (ph->needs_swap) { mem_bswap_64(header, offsetof(struct perf_file_header,
@@ -1938,21 +2007,17 @@ static int perf_file_header__read_pipe(struct perf_pipe_file_header *header,if (ret <=0 )return -1 ; -if (check_magic_endian (&header->magic,NULL , ph) <0 ) +if (check_magic_endian (header->magic, header->size, true, ph) <0 ) { +pr_debug ("endian/magic failed\n" );return -1 ; + } + +if (ph->needs_swap) + header->size =bswap_64 (header->size);if (repipe &&do_write (STDOUT_FILENO , header,sizeof (*header)) <0 )return -1 ; -if (header->size !=sizeof (*header)) { -u64 size =bswap_64 (header->size); - -if (size !=sizeof (*header)) -return -1 ; - - ph->needs_swap = true; - } -return 0 ; }
@@ -1992,6 +2057,7 @@ static int read_attr(int fd, struct perf_header *ph,/* on file perf_event_attr size */ sz = attr->size; +if (ph->needs_swap) sz =bswap_32 (sz);
Quoted 213 lines · Stephane Eranian
This patchset adds an important and useful new feature to
perf_events: branch stack sampling. In other words, the ability to capture taken branches into each sample.Statistical sampling of taken branch should not be confused for branch tracing. Not all branches are necessarily captured
Sampling taken branches is important for basic block profiling, statistical call graph, function call counts. Many of those measurements can help drive a compiler optimizer.
The branch stack is a software abstraction which sits on top of the PMU hardware. As such, it is not available on all processors. For now, the patch provides the generic interface and the Intel X86 implementation where it leverages the Last Branch Record (LBR) feature (from Core2 to SandyBridge).
Branch stack sampling is supported for both per-thread and system-wide modes.
It is possible to filter the type and privilege level of branches to sample. The target of the branch is used to determine the privilege level.
For each branch, the source and destination are captured. On some hardware platforms, it may be possible to also extract the target prediction and, in that case, it is also exposed to end users.
The branch stack can record a variable number of taken branches per sample. Those branches are always consecutive in time. The number of branches captured depends on the filtering and the underlying hardware. On Intel Nehalem and later, up to 16 consecutive branches can be captured per sample.
Branch sampling is always coupled with an event. It can be any PMU event but it can't be a SW or tracepoint event.
Branch sampling is requested by setting a new
sample_typeflag called:PERF_SAMPLE_BRANCH_STACK.To support branch filtering, we introduce a new field to the
perf_event_attrstruct:branch_sample_type. We chose NOT to overload the config1, config2 field because those are related to the event encoding. Branch stack is a separate feature which is combined with the event.The
branch_sample_typeis a bitmask of possible filters. The following filters are defined (more can be added):- PERF_SAMPLE_BRANCH_ANY : any control flow change - PERF_SAMPLE_BRANCH_USER : branches when target is at user level - PERF_SAMPLE_BRANCH_KERNEL : branches when target is at kernel level - PERF_SAMPLE_BRANCH_HV : branches when target is at hypervisor level-
PERF_SAMPLE_BRANCH_ANY_CALL: call branches (incl. syscalls)
-PERF_SAMPLE_BRANCH_ANY_RET: return branches (incl. syscall returns)
-PERF_SAMPLE_BRANCH_IND_CALL: indirect callsIt is possible to combine filters, e.g., IND_CALL|USER|KERNEL.
When the privilege level is not specified, the branch stack inherits that of the associated event.
Some processors may not offer hardware branch filtering, e.g., Intel Atom. Some may have HW filtering bugs (e.g., Nehalem). The Intel X86 implementation in this patchset also provides a SW branch filter which works on a best effort basis. It can compensate for the lack of LBR filtering. But first and foremost, it helps work around LBR filtering errata. The goal is to only capture the type of branches requested by the user.
It is possible to combine branch stack sampling with PEBS on Intel X86 processors. Depending on the
precise_samplingmode, there are certain filterting restrictions. When precise_sampling=1, then there are no filtering restrictions. Whenprecise_sampling> 1, then only ANY|USER|KERNEL filter can be used. This comes from the fact that the kernel uses LBR to compensate for the PEBS off-by-1 skid on the instruction pointer.To demonstrate how the
perf_eventbranch stack sampling interface works, the patchset also modifies perf record to capture taken branches. Similarly perf report is enhanced to display a histogram of taken branches.I would like to thank Roberto Vitillo @ LBL for his work on the perf tool for this.
Enough talking, let's take a simple example. Our trivial test program goes like this:
void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)f2(); else f3(); }int main(void)
{
unsigned long i;for (i=0; i < N; i++)
f1(i);
return 0;
}$ perf record -b any branchy $ perf report -b # Events: 23K cycles # # Overhead Source Symbol Target Symbol # ........ ................ ................18.13% [.] f1 [.] main 18.10% [.] main [.] main 18.01% [.] main [.] f1 15.69% [.] f1 [.] f1 9.11% [.] f3 [.] f1 6.78% [.] f1 [.] f3 6.74% [.] f1 [.] f2 6.71% [.] f2 [.] f1Of the total number of branches captured, 18.13% were from
f1()->main().Let's make this clearer by filtering the user call branches only:
$ perf record -b any_call -e cycles:u branchy $ perf report -b # Events: 19K cycles # # Overhead Source Symbol Target Symbol # ........ ......................... ......................... # 52.50% [.] main [.] f1 23.99% [.] f1 [.] f3 23.48% [.] f1 [.] f2 0.03% [.] _IO_default_xsputn [.] _IO_new_file_overflow 0.01% [k] _start [k] __libc_start_mainNow it is more obvious. %52 of all the captured branches where calls from
main()->f1(). The rest is split 50/50 betweenf1()->f2()andf1()->f3()which is expected given thatf1()dispatches based on odd vs. even values of n which is constantly increasing.Here is a kernel example, where we want to sample indirect calls:
$ perf record -a -C 1 -b ind_call -e r1c4:k sleep 10 $ perf report -b # # Overhead Source Symbol Target Symbol # ........ .......................... .......................... # 36.36% [k] __delay [k] delay_tsc 9.09% [k] ktime_get [k] read_tsc 9.09% [k] getnstimeofday [k] read_tsc 9.09% [k] notifier_call_chain [k] tick_notify 4.55% [k] cpuidle_idle_call [k] intel_idle 4.55% [k] cpuidle_idle_call [k] menu_reflect 2.27% [k] handle_irq [k] handle_edge_irq 2.27% [k] ack_apic_edge [k] native_apic_mem_write 2.27% [k] hpet_interrupt_handler [k] hrtimer_interrupt 2.27% [k] __run_hrtimer [k] watchdog_timer_fn 2.27% [k] enqueue_task [k] enqueue_task_rt 2.27% [k] try_to_wake_up [k] select_task_rq_rt 2.27% [k] do_timer [k] read_tscDue to HW limitations, branch filtering may be approximate on Core, Atom processors. It is more accurate on Nehalem, Westmere and best on Sandy Bridge.
In version 2, we've updated the patch to tip/master (commit 5734857) and we've incoporated the feedback from v1 concerning anynous bitfield struct for
branch_stack_entryand the hanlding of i386 ABI binaries on 64-bit host in the instr decoder for the LBR SW filter.In version 3, we've updated to 3.2.0-tip. The Atom revision check has been put into its own patch. We fixed a browser issue with report report. We fixed all the style issues as well.
In version 4, we've modified the branch stack API to add a missing priv level : hypervisor. There is a new
PERF_SAMPLE_BRANCH_HV. It is not used on Intel X86. Thanks to [email protected] for pointing this out. We also fix compilation error on ARM.In version 4, we also extend the patch to include the changes necessary to the perf tool to support reading perf.data files which were produced from older
perf_eventABI revisions. This patch set extends the ABI with a new field in structperf_event_attr. That struct is saved as is in the perf.data file. Therefore, older perf.data files contain smallerperf_event_attrstruct, yet perf must process them transparently. That's not the case today. It dies with 'incompatible file format'.The patch solves this problem and, at the same time, decouples endianness detection from the size of
perf_event_attr. Endianness is now detected via the signature (the first 8 bytes of the file). We introduce a new signature (PERFILE2). It is not laid out the same way in the file based on the endianness of the host where the file is written. Therefore, we can dynamically detect the endianness by simply reading the first 8 bytes. The size of theperf_event_attrstruct can then be processed according to the endianness. The ambiguity between the size being at the same time, the endianness marker and the actual size is gone. We can now distinguish an older ABI by the size and not confuse it with an endianness mismatch.In version 5, we fix the PEBS+LBR vs.
BRANCH_STACKcheck inx86_pmu_hw_config. We also changed the handling ofPERF_SAMPLE_BRANCH_HVon X86. It is now ignored instead of triggering an error. That enables: perf record -b any -e cycles, without having to force a priv level on the branch type. We also fix an uninitialized variable bug in the perf tool reported by reviewers. Thanks to Anshuman Khandual for his comments.
Quoted 7 lines
In version 6, we have fixed several issues in the perf tool code and especially in patch 11. We have fully implemented the --sort option on the branch source and target. We have fixed several column alignment issues. We have integrated feedback from David Ahern, concerning patch 16 and the ability to read perf.data files written from a different ABI. Perf will now reject any perf.data file that has a larger
perf_event_attrsize. Perf provides compatibility backward and not forward.
Hey Stephane,
Could you please specify which tip tree I can apply and try out the V6 patchset ? Thank you.
Quoted 60 lines
Signed-off-by Stephane Eranian <[email protected]>Roberto Agostino Vitillo (3):
perf: add code to supportPERF_SAMPLE_BRANCH_STACK
perf: add support for sampling taken branch to perf record
perf: add support for taken branch sampling to perf reportStephane Eranian (15):
perf: add generic taken branch sampling support
perf: add Intel LBR MSR definitions
perf: add Intel X86 LBR sharing logic
perf: sync branch stack sampling with X86precise_sampling
perf: add Intel X86 LBR mappings forPERF_SAMPLE_BRANCHfilters
perf: disable LBR support for older Intel Atom processors
perf: implementPERF_SAMPLE_BRANCHfor Intel X86
perf: add LBR software filter support for Intel X86
perf: disable PERF_SAMPLE_BRANCH_* when not supported
perf: add hook to flushbranch_stackon context switch
perf: fix endianness detection in perf.data
perf: add ABI reference sizes
perf: enable reading of perf.data files from different ABI rev
perf: fix bugprint_event_desc()
perf: make perf able to read file from older ABIsarch/alpha/kernel/perf_event.c | 4 + arch/arm/kernel/perf_event.c | 4 + arch/mips/kernel/perf_event_mipsxx.c | 4 + arch/powerpc/kernel/perf_event.c | 4 + arch/sh/kernel/perf_event.c | 4 + arch/sparc/kernel/perf_event.c | 4 + arch/x86/include/asm/msr-index.h | 7 + arch/x86/kernel/cpu/perf_event.c | 85 ++++- arch/x86/kernel/cpu/perf_event.h | 19 + arch/x86/kernel/cpu/perf_event_amd.c | 3 + arch/x86/kernel/cpu/perf_event_intel.c | 120 +++++-- arch/x86/kernel/cpu/perf_event_intel_ds.c | 22 +- arch/x86/kernel/cpu/perf_event_intel_lbr.c | 526 ++++++++++++++++++++++++++-- include/linux/perf_event.h | 82 ++++- kernel/events/core.c | 177 ++++++++++ kernel/events/hw_breakpoint.c | 6 + tools/perf/Documentation/perf-record.txt | 25 ++ tools/perf/Documentation/perf-report.txt | 7 + tools/perf/builtin-record.c | 74 ++++ tools/perf/builtin-report.c | 98 +++++- tools/perf/perf.h | 18 + tools/perf/util/annotate.c | 2 +- tools/perf/util/event.h | 1 + tools/perf/util/evsel.c | 14 + tools/perf/util/header.c | 230 +++++++++++-- tools/perf/util/hist.c | 93 ++++- tools/perf/util/hist.h | 7 + tools/perf/util/session.c | 72 ++++ tools/perf/util/session.h | 4 + tools/perf/util/sort.c | 362 ++++++++++++++----- tools/perf/util/sort.h | 5 + tools/perf/util/symbol.h | 13 + 32 files changed, 1866 insertions(+), 230 deletions(-)
Signature · 3 lines
Anshuman Khandual Linux Technology Centre IBM Systems and Technology Group
Quoted 223 lines · Anshuman Khandual
Quoted 213 lines · Stephane Eranian
This patchset adds an important and useful new feature to
perf_events: branch stack sampling. In other words, the ability to capture taken branches into each sample.Statistical sampling of taken branch should not be confused for branch tracing. Not all branches are necessarily captured
Sampling taken branches is important for basic block profiling, statistical call graph, function call counts. Many of those measurements can help drive a compiler optimizer.
The branch stack is a software abstraction which sits on top of the PMU hardware. As such, it is not available on all processors. For now, the patch provides the generic interface and the Intel X86 implementation where it leverages the Last Branch Record (LBR) feature (from Core2 to SandyBridge).
Branch stack sampling is supported for both per-thread and system-wide modes.
It is possible to filter the type and privilege level of branches to sample. The target of the branch is used to determine the privilege level.
For each branch, the source and destination are captured. On some hardware platforms, it may be possible to also extract the target prediction and, in that case, it is also exposed to end users.
The branch stack can record a variable number of taken branches per sample. Those branches are always consecutive in time. The number of branches captured depends on the filtering and the underlying hardware. On Intel Nehalem and later, up to 16 consecutive branches can be captured per sample.
Branch sampling is always coupled with an event. It can be any PMU event but it can't be a SW or tracepoint event.
Branch sampling is requested by setting a new
sample_typeflag called:PERF_SAMPLE_BRANCH_STACK.To support branch filtering, we introduce a new field to the
perf_event_attrstruct:branch_sample_type. We chose NOT to overload the config1, config2 field because those are related to the event encoding. Branch stack is a separate feature which is combined with the event.The
branch_sample_typeis a bitmask of possible filters.
The following filters are defined (more can be added):
-PERF_SAMPLE_BRANCH_ANY: any control flow change
-PERF_SAMPLE_BRANCH_USER: branches when target is at user level
-PERF_SAMPLE_BRANCH_KERNEL: branches when target is at kernel level
-PERF_SAMPLE_BRANCH_HV: branches when target is at hypervisor level
-PERF_SAMPLE_BRANCH_ANY_CALL: call branches (incl. syscalls)
-PERF_SAMPLE_BRANCH_ANY_RET: return branches (incl. syscall returns)
-PERF_SAMPLE_BRANCH_IND_CALL: indirect callsIt is possible to combine filters, e.g., IND_CALL|USER|KERNEL.
When the privilege level is not specified, the branch stack inherits that of the associated event.
Some processors may not offer hardware branch filtering, e.g., Intel Atom. Some may have HW filtering bugs (e.g., Nehalem). The Intel X86 implementation in this patchset also provides a SW branch filter which works on a best effort basis. It can compensate for the lack of LBR filtering. But first and foremost, it helps work around LBR filtering errata. The goal is to only capture the type of branches requested by the user.
It is possible to combine branch stack sampling with PEBS on Intel X86 processors. Depending on the
precise_samplingmode, there are certain filterting restrictions. When precise_sampling=1, then there are no filtering restrictions. Whenprecise_sampling> 1, then only ANY|USER|KERNEL filter can be used. This comes from the fact that the kernel uses LBR to compensate for the PEBS off-by-1 skid on the instruction pointer.To demonstrate how the
perf_eventbranch stack sampling interface works, the patchset also modifies perf record to capture taken branches. Similarly perf report is enhanced to display a histogram of taken branches.I would like to thank Roberto Vitillo @ LBL for his work on the perf tool for this.
Enough talking, let's take a simple example. Our trivial test program goes like this:
void f2(void)
{}
void f3(void)
{}
void f1(unsigned long n)
{
if (n & 1UL)
f2();
else
f3();
}
int main(void)
{
unsigned long i;for (i=0; i < N; i++)
f1(i);
return 0;
}$ perf record -b any branchy $ perf report -b # Events: 23K cycles # # Overhead Source Symbol Target Symbol # ........ ................ ................18.13% [.] f1 [.] main
18.10% [.] main [.] main
18.01% [.] main [.] f1
15.69% [.] f1 [.] f1
9.11% [.] f3 [.] f1
6.78% [.] f1 [.] f3
6.74% [.] f1 [.] f2
6.71% [.] f2 [.] f1Of the total number of branches captured, 18.13% were from
f1()->main().Let's make this clearer by filtering the user call branches only:
$ perf record -b any_call -e cycles:u branchy $ perf report -b # Events: 19K cycles # # Overhead Source Symbol Target Symbol # ........ ......................... ......................... # 52.50% [.] main [.] f1 23.99% [.] f1 [.] f3 23.48% [.] f1 [.] f2 0.03% [.] _IO_default_xsputn [.] _IO_new_file_overflow 0.01% [k] _start [k] __libc_start_mainNow it is more obvious. %52 of all the captured branches where calls from
main()->f1(). The rest is split 50/50 betweenf1()->f2()andf1()->f3()which is expected given thatf1()dispatches based on odd vs. even values of n which is constantly increasing.Here is a kernel example, where we want to sample indirect calls:
$ perf record -a -C 1 -b ind_call -e r1c4:k sleep 10 $ perf report -b # # Overhead Source Symbol Target Symbol # ........ .......................... .......................... # 36.36% [k] __delay [k] delay_tsc 9.09% [k] ktime_get [k] read_tsc 9.09% [k] getnstimeofday [k] read_tsc 9.09% [k] notifier_call_chain [k] tick_notify 4.55% [k] cpuidle_idle_call [k] intel_idle 4.55% [k] cpuidle_idle_call [k] menu_reflect 2.27% [k] handle_irq [k] handle_edge_irq 2.27% [k] ack_apic_edge [k] native_apic_mem_write 2.27% [k] hpet_interrupt_handler [k] hrtimer_interrupt 2.27% [k] __run_hrtimer [k] watchdog_timer_fn 2.27% [k] enqueue_task [k] enqueue_task_rt 2.27% [k] try_to_wake_up [k] select_task_rq_rt 2.27% [k] do_timer [k] read_tscDue to HW limitations, branch filtering may be approximate on Core, Atom processors. It is more accurate on Nehalem, Westmere and best on Sandy Bridge.
In version 2, we've updated the patch to tip/master (commit 5734857) and we've incoporated the feedback from v1 concerning anynous bitfield struct for
branch_stack_entryand the hanlding of i386 ABI binaries on 64-bit host in the instr decoder for the LBR SW filter.In version 3, we've updated to 3.2.0-tip. The Atom revision check has been put into its own patch. We fixed a browser issue with report report. We fixed all the style issues as well.
In version 4, we've modified the branch stack API to add a missing priv level : hypervisor. There is a new
PERF_SAMPLE_BRANCH_HV. It is not used on Intel X86. Thanks to [email protected] for pointing this out. We also fix compilation error on ARM.In version 4, we also extend the patch to include the changes necessary to the perf tool to support reading perf.data files which were produced from older
perf_eventABI revisions. This patch set extends the ABI with a new field in structperf_event_attr. That struct is saved as is in the perf.data file. Therefore, older perf.data files contain smallerperf_event_attrstruct, yet perf must process them transparently. That's not the case today. It dies with 'incompatible file format'.The patch solves this problem and, at the same time, decouples endianness detection from the size of
perf_event_attr. Endianness is now detected via the signature (the first 8 bytes of the file). We introduce a new signature (PERFILE2). It is not laid out the same way in the file based on the endianness of the host where the file is written. Therefore, we can dynamically detect the endianness by simply reading the first 8 bytes. The size of theperf_event_attrstruct can then be processed according to the endianness. The ambiguity between the size being at the same time, the endianness marker and the actual size is gone. We can now distinguish an older ABI by the size and not confuse it with an endianness mismatch.In version 5, we fix the PEBS+LBR vs.
BRANCH_STACKcheck inx86_pmu_hw_config. We also changed the handling ofPERF_SAMPLE_BRANCH_HVon X86. It is now ignored instead of triggering an error. That enables: perf record -b any -e cycles, without having to force a priv level on the branch type. We also fix an uninitialized variable bug in the perf tool reported by reviewers. Thanks to Anshuman Khandual for his comments.Quoted 7 lines
In version 6, we have fixed several issues in the perf tool code and especially in patch 11. We have fully implemented the --sort option on the branch source and target. We have fixed several column alignment issues. We have integrated feedback from David Ahern, concerning patch 16 and the ability to read perf.data files written from a different ABI. Perf will now reject any perf.data file that has a larger
perf_event_attrsize. Perf provides compatibility backward and not forward.
Quoted 5 lines
Hey Stephane,
Could you please specify which tip tree I can apply and try out the V6 patchset ? Thank you.
V6 was posted prior to the jump_label changes. I suspect any tip before:
77a73e5 static keys: Introduce 'struct static_key', very_[un]likely(), static_key_slow_[inc|dec]()
Should work, though I have not tried.
Quoted 68 lines
Quoted 59 lines
Signed-off-by Stephane Eranian <[email protected]>Roberto Agostino Vitillo (3):
perf: add code to supportPERF_SAMPLE_BRANCH_STACK
perf: add support for sampling taken branch to perf record
perf: add support for taken branch sampling to perf reportStephane Eranian (15):
perf: add generic taken branch sampling support
perf: add Intel LBR MSR definitions
perf: add Intel X86 LBR sharing logic
perf: sync branch stack sampling with X86precise_sampling
perf: add Intel X86 LBR mappings forPERF_SAMPLE_BRANCHfilters
perf: disable LBR support for older Intel Atom processors
perf: implementPERF_SAMPLE_BRANCHfor Intel X86
perf: add LBR software filter support for Intel X86
perf: disable PERF_SAMPLE_BRANCH_* when not supported
perf: add hook to flushbranch_stackon context switch
perf: fix endianness detection in perf.data
perf: add ABI reference sizes
perf: enable reading of perf.data files from different ABI rev
perf: fix bugprint_event_desc()
perf: make perf able to read file from older ABIsarch/alpha/kernel/perf_event.c | 4 +
arch/arm/kernel/perf_event.c| 4 +arch/mips/kernel/perf_event_mipsxx.c| 4 +arch/powerpc/kernel/perf_event.c| 4 +arch/sh/kernel/perf_event.c| 4 +arch/sparc/kernel/perf_event.c| 4 +arch/x86/include/asm/msr-index.h | 7 + arch/x86/kernel/cpu/perf_event.c | 85 ++++- arch/x86/kernel/cpu/perf_event.h | 19 +arch/x86/kernel/cpu/perf_event_amd.c | 3 +
arch/x86/kernel/cpu/perf_event_intel.c| 120 +++++--arch/x86/kernel/cpu/perf_event_intel_ds.c| 22 +-arch/x86/kernel/cpu/perf_event_intel_lbr.c| 526 ++++++++++++++++++++++++++--include/linux/perf_event.h| 82 ++++-kernel/events/core.c| 177 ++++++++++kernel/events/hw_breakpoint.c| 6 +tools/perf/Documentation/perf-record.txt| 25 ++tools/perf/Documentation/perf-report.txt| 7 +tools/perf/builtin-record.c| 74 ++++tools/perf/builtin-report.c| 98 +++++-tools/perf/perf.h| 18 +tools/perf/util/annotate.c| 2 +-tools/perf/util/event.h| 1 +tools/perf/util/evsel.c| 14 +tools/perf/util/header.c| 230 +++++++++++--tools/perf/util/hist.c| 93 ++++-tools/perf/util/hist.h| 7 +tools/perf/util/session.c| 72 ++++tools/perf/util/session.h| 4 +tools/perf/util/sort.c| 362 ++++++++++++++-----tools/perf/util/sort.h| 5 +tools/perf/util/symbol.h| 13 +Quoted 2 lines
32 files changed, 1866 insertions(+), 230 deletions(-)--
Anshuman Khandual
Linux Technology Centre
IBM Systems and Technology Group