Skip to contentSkip to conversation

perf: add ability to sample direct call branches

PATCH In review Stephane Eranian 4 participants 12 messages Oct 13, 2015 – Oct 20, 2015

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover perf: add ability to sample direct call branches

This short patch series improves the perf_events interface by providing a new branch_sample_type bit to sample only direct call branches. Up until now, you could specify PERF_SAMPLE_BRANCH_ANY_CALL (any calls) or PERF_SAMPLE_BRANCH_IND_CALL (indirect calls). But there was no way to sample only direct calls. This series adds PERF_SAMPLE_BRANCH_CALL.

This covers direct function calls (incl. zero length calls) but not syscalls. It can be used for those who want to analyze direct calls only.

The series includes the kernel generic code changes. The x86 support based on the LBR filter (or sofware filter) and the PPC check.

The series also includes the changes to perf record to support the new filter:

    $ perf record -j call -e cycles ......

Patch is relative to tip.git @ commit e6f195f Merge branch 'ras/core'

Stephane Eranian (4):
perf: add PERF_SAMPLE_BRANCH_CALL
perf/x86: add support for PERF_SAMPLE_BRANCH_CALL perf/powerpc: add support for PERF_SAMPLE_BRANCH_CALL perf record: add ability to sample call branches

 arch/powerpc/perf/power8-pmu.c             | 3 +++
 arch/x86/kernel/cpu/perf_event_intel_lbr.c | 4 ++++
 include/uapi/linux/perf_event.h            | 2 ++
 tools/perf/Documentation/perf-record.txt   | 1 +
 tools/perf/util/parse-branch-options.c     | 1 +
 5 files changed, 11 insertions(+)
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/4 perf: add PERF_SAMPLE_BRANCH_CALL

Add a new branch sample type to cover only call branches (function calls). The current ANY_CALL included direct, indirect calls and far jumps.

We want to be able to differentiate indirect from direct calls. Therefore we introduce PERF_SAMPLE_BRANCH_CALL. The implementation is up to each architecture.

---
 include/uapi/linux/perf_event.h | 2 ++
 1 file changed, 2 insertions(+)
include/uapi/linux/perf_event.h +2 -0
index 2881145..e6c1b47 100644
@@ -168,6 +168,7 @@ enum perf_branch_sample_type_shift {
 
 	PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT	= 11, /* call/ret stack */
 	PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT	= 12, /* indirect jumps */
+	PERF_SAMPLE_BRANCH_CALL_SHIFT		= 13, /* direct call */
 
 	PERF_SAMPLE_BRANCH_MAX_SHIFT		/* non-ABI */
 };
@@ -188,6 +189,7 @@ enum perf_branch_sample_type {
 
 	PERF_SAMPLE_BRANCH_CALL_STACK	= 1U << PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT,
 	PERF_SAMPLE_BRANCH_IND_JUMP	= 1U << PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT,
+	PERF_SAMPLE_BRANCH_CALL		= 1U << PERF_SAMPLE_BRANCH_CALL_SHIFT,
 
 	PERF_SAMPLE_BRANCH_MAX		= 1U << PERF_SAMPLE_BRANCH_MAX_SHIFT,
 };
Signature · 1 line
1.9.1
#2 lore raw

[tip:perf/core] perf: Add PERF_SAMPLE_BRANCH_CALL

perf: Add PERF_SAMPLE_BRANCH_CALL
Commit-ID:  c229bf9dc179d2023e185c0f705bdf68484c1e73
Gitweb:     http://git.kernel.org/tip/c229bf9dc179d2023e185c0f705bdf68484c1e73
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 13 Oct 2015 09:09:08 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 20 Oct 2015 10:30:53 +0200

perf: Add PERF_SAMPLE_BRANCH_CALL

Add a new branch sample type to cover only call branches (function calls). The current ANY_CALL included direct, indirect calls and far jumps.

We want to be able to differentiate indirect from direct calls. Therefore we introduce PERF_SAMPLE_BRANCH_CALL. The implementation is up to each architecture.

---
 include/uapi/linux/perf_event.h | 2 ++
 1 file changed, 2 insertions(+)
include/uapi/linux/perf_event.h +2 -0
index 6c72e72..6512213 100644
@@ -168,6 +168,7 @@ enum perf_branch_sample_type_shift {
 
 	PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT	= 11, /* call/ret stack */
 	PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT	= 12, /* indirect jumps */
+	PERF_SAMPLE_BRANCH_CALL_SHIFT		= 13, /* direct call */
 
 	PERF_SAMPLE_BRANCH_MAX_SHIFT		/* non-ABI */
 };
@@ -188,6 +189,7 @@ enum perf_branch_sample_type {
 
 	PERF_SAMPLE_BRANCH_CALL_STACK	= 1U << PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT,
 	PERF_SAMPLE_BRANCH_IND_JUMP	= 1U << PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT,
+	PERF_SAMPLE_BRANCH_CALL		= 1U << PERF_SAMPLE_BRANCH_CALL_SHIFT,
 
 	PERF_SAMPLE_BRANCH_MAX		= 1U << PERF_SAMPLE_BRANCH_MAX_SHIFT,
 };
#3 lore raw

PATCH 2/4 perf/x86: add support for PERF_SAMPLE_BRANCH_CALL

In reply to Stephane Eranian

This patch enables the suport for the PERF_SAMPLE_BRANCH_CALL for Intel x86 processors. When the processor support LBR filtering this the selection is done in hardware. Otherwise, the filter is applied by software. Note that we chose to include zero length calls because they also represent calls.

---
 arch/x86/kernel/cpu/perf_event_intel_lbr.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +4 -0
index ad0b8b0..bfd0b71 100644
@@ -555,6 +555,8 @@ static int intel_pmu_setup_sw_lbr_filter(struct perf_event *event)
 	if (br_type & PERF_SAMPLE_BRANCH_IND_JUMP)
 		mask |= X86_BR_IND_JMP;
 
+	if (br_type & PERF_SAMPLE_BRANCH_CALL)
+		mask |= X86_BR_CALL | X86_BR_ZERO_CALL;
 	/*
 	 * stash actual user request into reg, it may
 	 * be used by fixup code for some CPU
@@ -890,6 +892,7 @@ static const int snb_lbr_sel_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = {
 	[PERF_SAMPLE_BRANCH_IND_CALL_SHIFT]	= LBR_IND_CALL,
 	[PERF_SAMPLE_BRANCH_COND_SHIFT]		= LBR_JCC,
 	[PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT]	= LBR_IND_JMP,
+	[PERF_SAMPLE_BRANCH_CALL_SHIFT]		= LBR_REL_CALL,
 };
 
 static const int hsw_lbr_sel_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = {
@@ -905,6 +908,7 @@ static const int hsw_lbr_sel_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = {
 	[PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT]	= LBR_REL_CALL | LBR_IND_CALL
 						| LBR_RETURN | LBR_CALL_STACK,
 	[PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT]	= LBR_IND_JMP,
+	[PERF_SAMPLE_BRANCH_CALL_SHIFT]		= LBR_REL_CALL,
 };
 
 /* core */
Signature · 1 line
1.9.1
#4 lore raw

Re: [PATCH 2/4] perf/x86: add support for PERF_SAMPLE_BRANCH_CALL

Quoted 16 lines · Stephane Eranian

This patch enables the suport for the PERF_SAMPLE_BRANCH_CALL for Intel x86 processors. When the processor support LBR filtering this the selection is done in hardware. Otherwise, the filter is applied by software. Note that we chose to include zero length calls because they also represent calls.

---
 arch/x86/kernel/cpu/perf_event_intel_lbr.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +0 -0
index ad0b8b0..bfd0b71 100644

arch/x86/kernel/cpu/perf_event_intel_lbr.c static int intel_pmu_setup_sw_lbr_filter(struct perf_event *event) · line 559

Quoted 5 lines
arch/x86/kernel/cpu/perf_event_intel_lbr.c +2 -0
@@ -555,6 +555,8 @@ static int intel_pmu_setup_sw_lbr_filter(struct perf_event *event)
 	if (br_type & PERF_SAMPLE_BRANCH_IND_JUMP)
 		mask |= X86_BR_IND_JMP;
 
+	if (br_type & PERF_SAMPLE_BRANCH_CALL)
+		mask |= X86_BR_CALL | X86_BR_ZERO_CALL;

I'm wondering how frequent zero-length calls are. If they still occur in typical user-space, would it make sense to also have a separate branch sampling type for zero length calls?

Intel documents zero length calls as ones that (ab-)use the call instruction to push the current IP on the stack:

call next_addr
next_addr:
pop %reg

which can take over 10 cycles on certain microarchitectures (and it unbalances whatever call stack tracking/caching the CPU does as well).

So it might make sense to analyze them separately. I guess that's the reason why Intel added a separate flag for them in the PMU.

Thanks,

Ingo

#5 lore raw

Re: [PATCH 2/4] perf/x86: add support for PERF_SAMPLE_BRANCH_CALL

Quoted 3 lines

I'm wondering how frequent zero-length calls are. If they still occur in typical user-space, would it make sense to also have a separate branch sampling type for zero length calls?

Apparently not too old icc compiled 32bit PIC binaries still contain it. For gcc it was fixed for much longer.

But I'm not sure it's that interesting to sample by itself.

Quoted 11 lines

push the current IP on the stack:

call next_addr
next_addr:
pop %reg

which can take over 10 cycles on certain microarchitectures (and it unbalances whatever call stack tracking/caching the CPU does as well).

So it might make sense to analyze them separately. I guess that's the reason why Intel added a separate flag for them in the PMU.

X86_BR_ZERO_CALL is only a software filter. There's no direct support for it in the Intel hardware. It was added to make the LBR call stack more reliable, which otherwise gets messed up by the zero length calls.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#6 lore raw

Re: [PATCH 2/4] perf/x86: add support for PERF_SAMPLE_BRANCH_CALL

In reply to Ingo Molnar
Quoted 23 lines · Ingo Molnar
Quoted 19 lines · Stephane Eranian

This patch enables the suport for the PERF_SAMPLE_BRANCH_CALL for Intel x86 processors. When the processor support LBR filtering this the selection is done in hardware. Otherwise, the filter is applied by software. Note that we chose to include zero length calls because they also represent calls.

---
 arch/x86/kernel/cpu/perf_event_intel_lbr.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +0 -0
index ad0b8b0..bfd0b71 100644
@@ -555,6 +555,8 @@ static int intel_pmu_setup_sw_lbr_filter(struct perf_event *event)
      if (br_type & PERF_SAMPLE_BRANCH_IND_JUMP)
              mask |= X86_BR_IND_JMP;
 
Quoted 7 lines

arch/x86/kernel/cpu/perf_event_intel_lbr.c static int intel_pmu_setup_sw_lbr_filter(struct perf_event *event) · line 559

Quoted 2 lines
arch/x86/kernel/cpu/perf_event_intel_lbr.c +2 -0
+     if (br_type & PERF_SAMPLE_BRANCH_CALL)
+             mask |= X86_BR_CALL | X86_BR_ZERO_CALL;

I'm wondering how frequent zero-length calls are. If they still occur in typical user-space, would it make sense to also have a separate branch sampling type for zero length calls?

We could add that. It would rely on the sw filter to catch only the zero calls as Andi
mentioned. But I am wondering about the data quality because we would catch zero calls without being able to determine how many we sampled vs. how many have occurred. There is no PMU event counting zero call branches.

Quoted 16 lines

Intel documents zero length calls as ones that (ab-)use the call instruction to push the current IP on the stack:

call next_addr
next_addr:
pop %reg

which can take over 10 cycles on certain microarchitectures (and it unbalances whatever call stack tracking/caching the CPU does as well).

So it might make sense to analyze them separately. I guess that's the reason why Intel added a separate flag for them in the PMU.

Thanks,

Ingo

#7 lore raw

[tip:perf/core] perf/x86: Add support for PERF_SAMPLE_BRANCH_CALL

In reply to Stephane Eranian
perf/x86: Add support for PERF_SAMPLE_BRANCH_CALL
Commit-ID:  d892819faa6860d469aae71d70c336b391c25505
Gitweb:     http://git.kernel.org/tip/d892819faa6860d469aae71d70c336b391c25505
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 13 Oct 2015 09:09:09 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 20 Oct 2015 10:30:53 +0200

perf/x86: Add support for PERF_SAMPLE_BRANCH_CALL

This patch enables the suport for the PERF_SAMPLE_BRANCH_CALL for Intel x86 processors. When the processor support LBR filtering this the selection is done in hardware. Otherwise, the filter is applied by software. Note that we chose to include zero length calls because they also represent calls.

---
 arch/x86/kernel/cpu/perf_event_intel_lbr.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/cpu/perf_event_intel_lbr.c +4 -0
index ad0b8b0..bfd0b71 100644
@@ -555,6 +555,8 @@ static int intel_pmu_setup_sw_lbr_filter(struct perf_event *event)
 	if (br_type & PERF_SAMPLE_BRANCH_IND_JUMP)
 		mask |= X86_BR_IND_JMP;
 
+	if (br_type & PERF_SAMPLE_BRANCH_CALL)
+		mask |= X86_BR_CALL | X86_BR_ZERO_CALL;
 	/*
 	 * stash actual user request into reg, it may
 	 * be used by fixup code for some CPU
@@ -890,6 +892,7 @@ static const int snb_lbr_sel_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = {
 	[PERF_SAMPLE_BRANCH_IND_CALL_SHIFT]	= LBR_IND_CALL,
 	[PERF_SAMPLE_BRANCH_COND_SHIFT]		= LBR_JCC,
 	[PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT]	= LBR_IND_JMP,
+	[PERF_SAMPLE_BRANCH_CALL_SHIFT]		= LBR_REL_CALL,
 };
 
 static const int hsw_lbr_sel_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = {
@@ -905,6 +908,7 @@ static const int hsw_lbr_sel_map[PERF_SAMPLE_BRANCH_MAX_SHIFT] = {
 	[PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT]	= LBR_REL_CALL | LBR_IND_CALL
 						| LBR_RETURN | LBR_CALL_STACK,
 	[PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT]	= LBR_IND_JMP,
+	[PERF_SAMPLE_BRANCH_CALL_SHIFT]		= LBR_REL_CALL,
 };
 
 /* core */
#8 lore raw

PATCH 3/4 perf/powerpc: add support for PERF_SAMPLE_BRANCH_CALL

In reply to Stephane Eranian

The patch catches PERF_SAMPLE_BRANCH_CALL because it is not clear whether this is actually supported by the hardware.

---
 arch/powerpc/perf/power8-pmu.c | 3 +++
 1 file changed, 3 insertions(+)
arch/powerpc/perf/power8-pmu.c +3 -0
index 396351d..7d5e295 100644
@@ -676,6 +676,9 @@ static u64 power8_bhrb_filter_map(u64 branch_sample_type)
 	if (branch_sample_type & PERF_SAMPLE_BRANCH_IND_CALL)
 		return -1;
 
+	if (branch_sample_type & PERF_SAMPLE_BRANCH_CALL)
+		return -1;
+
 	if (branch_sample_type & PERF_SAMPLE_BRANCH_ANY_CALL) {
 		pmu_bhrb_filter |= POWER8_MMCRA_IFM1;
 		return pmu_bhrb_filter;
Signature · 1 line
1.9.1
#9 lore raw

[tip:perf/core] perf/powerpc: Add support for PERF_SAMPLE_BRANCH_CALL

perf/powerpc: Add support for PERF_SAMPLE_BRANCH_CALL
Commit-ID:  24f1a79a5fc10858e05ee0bf651ec99abfc0319b
Gitweb:     http://git.kernel.org/tip/24f1a79a5fc10858e05ee0bf651ec99abfc0319b
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 13 Oct 2015 09:09:10 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 20 Oct 2015 10:30:54 +0200

perf/powerpc: Add support for PERF_SAMPLE_BRANCH_CALL

The patch catches PERF_SAMPLE_BRANCH_CALL because it is not clear whether this is actually supported by the hardware.

---
 arch/powerpc/perf/power8-pmu.c | 3 +++
 1 file changed, 3 insertions(+)
arch/powerpc/perf/power8-pmu.c +3 -0
index 396351d..7d5e295 100644
@@ -676,6 +676,9 @@ static u64 power8_bhrb_filter_map(u64 branch_sample_type)
 	if (branch_sample_type & PERF_SAMPLE_BRANCH_IND_CALL)
 		return -1;
 
+	if (branch_sample_type & PERF_SAMPLE_BRANCH_CALL)
+		return -1;
+
 	if (branch_sample_type & PERF_SAMPLE_BRANCH_ANY_CALL) {
 		pmu_bhrb_filter |= POWER8_MMCRA_IFM1;
 		return pmu_bhrb_filter;
#10 lore raw

PATCH 4/4 perf record: add ability to sample call branches

In reply to Stephane Eranian

This patch add a new branch type sampling filter to perf record. It is named 'call' and maps to PERF_SAMPLE_BRANCH_CALL. It samples direct call branches only, unlike 'any_call' which includes indirect calls as well.

 $ perf record -j call -e cycles .....

The man page is updated accordingly.

---
 tools/perf/Documentation/perf-record.txt | 1 +
 tools/perf/util/parse-branch-options.c   | 1 +
 2 files changed, 2 insertions(+)
tools/perf/Documentation/perf-record.txt +1 -0
index 2e9ce77..b027d28 100644
@@ -236,6 +236,7 @@ It is possible to select the types of branches captured by enabling filters. The
         - any_call: any function call or system call
         - any_ret: any function return or system call return
         - ind_call: any indirect branch
+        - call: direct calls, including far (to/from kernel) calls
         - u:  only when the branch target is at the user level
         - k: only when the branch target is in the kernel
         - hv: only when the target is at the hypervisor level
tools/perf/util/parse-branch-options.c +1 -0
index a3b1e13..355eecf 100644
@@ -27,6 +27,7 @@ static const struct branch_mode branch_modes[] = {
 	BRANCH_OPT("no_tx", PERF_SAMPLE_BRANCH_NO_TX),
 	BRANCH_OPT("cond", PERF_SAMPLE_BRANCH_COND),
 	BRANCH_OPT("ind_jmp", PERF_SAMPLE_BRANCH_IND_JUMP),
+	BRANCH_OPT("call", PERF_SAMPLE_BRANCH_CALL),
 	BRANCH_END
 };
 
Signature · 1 line
1.9.1
#11 lore raw

[tip:perf/core] perf record: Add ability to sample call branches

perf record: Add ability to sample call branches
Commit-ID:  43e41adc9e8c36545888d78fed2ef8d102a938dc
Gitweb:     http://git.kernel.org/tip/43e41adc9e8c36545888d78fed2ef8d102a938dc
Author:     Stephane Eranian <[email protected]>
AuthorDate: Tue, 13 Oct 2015 09:09:11 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 20 Oct 2015 10:30:55 +0200

perf record: Add ability to sample call branches

This patch add a new branch type sampling filter to perf record. It is named 'call' and maps to PERF_SAMPLE_BRANCH_CALL. It samples direct call branches only, unlike 'any_call' which includes indirect calls as well.

 $ perf record -j call -e cycles .....

The man page is updated accordingly.

---
 tools/perf/Documentation/perf-record.txt | 1 +
 tools/perf/util/parse-branch-options.c   | 1 +
 2 files changed, 2 insertions(+)
tools/perf/Documentation/perf-record.txt +1 -0
index 2e9ce77..b027d28 100644
@@ -236,6 +236,7 @@ following filters are defined:
         - any_call: any function call or system call
         - any_ret: any function return or system call return
         - ind_call: any indirect branch
+        - call: direct calls, including far (to/from kernel) calls
         - u:  only when the branch target is at the user level
         - k: only when the branch target is in the kernel
         - hv: only when the target is at the hypervisor level
tools/perf/util/parse-branch-options.c +1 -0
index a3b1e13..355eecf 100644
@@ -27,6 +27,7 @@ static const struct branch_mode branch_modes[] = {
 	BRANCH_OPT("no_tx", PERF_SAMPLE_BRANCH_NO_TX),
 	BRANCH_OPT("cond", PERF_SAMPLE_BRANCH_COND),
 	BRANCH_OPT("ind_jmp", PERF_SAMPLE_BRANCH_IND_JUMP),
+	BRANCH_OPT("call", PERF_SAMPLE_BRANCH_CALL),
 	BRANCH_END
 };