perf: expose thread context switch out event type to user space
cover perf: expose thread context switch out event type to user space
Here is a series of small patches that implement exposing type of context-switch-out event as a part of PERF_RECORD_SWITCH[_CPU_WIDE] record.
Introduced types of context-switch-out events assumed to be:
a) preempt: task->state == TASK_RUNNING
misc &= PERF_RECORD_MISC_SWITCH_OUT
b) yield: !preempt - using new bit PERF_RECORD_MISC_SWITCH_OUT_YIELD:
misc &= PERF_RECORD_MISC_SWITCH_OUT|PERF_RECORD_MISC_SWITCH_OUT_YIELD
Perf tool report and script commands output has been extended to decode new yield bit and the updated output looks like in the examples below.
The documentation has been updated to mention yield switch out events and its decoding symbols in perf script output.
The changes have been manually tested on Fedora 27 with the patched kernel:git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf/core
perf report -D -i system-wide.perf:
0x1b9c50 [0x30]: event: 15
.
. ... raw event: size 48 bytes
. 0000: 0f 00 00 00 00 20 30 00 01 1e 00 00 01 1e 00 00 ..... 0......... . 0010: 00 00 00 00 00 00 00 00 85 ae d4 e3 3e 0e 00 00 ............>... . 0020: 54 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 T...............
5 15663273127557 0x1b9c50 [0x30]: PERF_RECORD_SWITCH_CPU_WIDE OUT next pid/tid: 7681/7681
0x2646c0 [0x30]: event: 15
.
. ... raw event: size 48 bytes
. 0000: 0f 00 00 00 00 60 30 00 00 00 00 00 00 00 00 00 .....`0......... . 0010: 00 1e 00 00 00 1e 00 00 29 1e d5 e3 3e 0e 00 00 ........)...>... . 0020: 56 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 V...............
7 15663273156137 0x2646c0 [0x30]: PERF_RECORD_SWITCH_CPU_WIDE OUT yield next pid/tid: 0/0
perf script --show-switch-events -F +misc -I -i system-wide.perf:
amplxe-perf 7681 [005] S 15663.273151: PERF_RECORD_SWITCH_CPU_WIDE OUT next pid/tid: 39/39 migration/5 39 [005] 15663.273152: PERF_RECORD_SWITCH_CPU_WIDE IN prev pid/tid: 7681/7681 amplxe-perf 7680 [007] K 15663.273153: 1 context-switch:
aaa488 schedule ([kernel.kallsyms]) 1a9f50 __poll_nocancel (inlined)
amplxe-perf 7680 [007] Sy 15663.273156: PERF_RECORD_SWITCH_CPU_WIDE OUT yield next pid/tid: 0/0
migration/5 39 [005] K 15663.273157:
---
Alexey Budankov (3):
perf/core: store context switch out type into Perf trace perf report: extend raw dump (-D) out with switch out event type perf script: extend misc field decoding with switch out event type
include/uapi/linux/perf_event.h | 5 +++++ kernel/events/core.c | 4 +++- tools/include/uapi/linux/perf_event.h | 5 +++++ tools/perf/Documentation/perf-script.txt | 17 +++++++++-------- tools/perf/builtin-script.c | 5 ++++- tools/perf/util/event.c | 4 +++- 6 files changed, 29 insertions(+), 11 deletions(-)
PATCH 1/3 perf/core: store context switch out type into Perf trace
Store thread context-switch-out event type into Perf trace as a part of PERF_RECORD_SWITCH[_CPU_WIDE] records.
Introduced types of switch-out events assumed to be
a) preempt: task->state == TASK_RUNNING and b) yield: !preempt;
New yield event type is encoded using special PERF_RECORD_MISC_SWITCH_OUT_YIELD bit extending PERF_RECORD_MISC_SWITCH_OUT meaning traditional preemption switch out event:
misc &= PERF_RECORD_MISC_SWITCH_OUT | PERF_RECORD_MISC_SWITCH_OUT_YIELD
--- include/uapi/linux/perf_event.h | 5 +++++ kernel/events/core.c | 4 +++- 2 files changed, 8 insertions(+), 1 deletion(-)
include/uapi/linux/perf_event.h +5 -0
@@ -654,6 +654,11 @@ struct perf_event_mmap_page {* perf_event_attr::precise_ip. */ #define PERF_RECORD_MISC_EXACT_IP (1 << 14) +/* +* Indicates that thread explicitly yielded cpu due to +* a call of some synchronization API e.g. futex system call +*/ +#define PERF_RECORD_MISC_SWITCH_OUT_YIELD (1 << 14)/* * Reserve the last bit to indicate some extended misc field */
kernel/events/core.c +3 -1
@@ -7216,6 +7216,8 @@ static void perf_event_switch(struct task_struct *task, struct task_struct *next_prev, bool sched_in) {struct perf_switch_event switch_event; +__u16 switch_type = sched_in ?0 :PERF_RECORD_MISC_SWITCH_OUT | + (task->state ==TASK_RUNNING ?0 :PERF_RECORD_MISC_SWITCH_OUT_YIELD );/* N.B. caller checks nr_switch_events != 0 */
@@ -7225,7 +7227,7 @@ static void perf_event_switch(struct task_struct *task, .event_id = { .header = {/* .type */ - .misc = sched_in ? 0 : PERF_RECORD_MISC_SWITCH_OUT, + .misc = switch_type, /* .size */ },/* .next_prev_pid */
PATCH 2/3 perf report: extend raw dump (-D) out with switch out event type
In reply to Alexey BudankovPrint additional 'yield' tag for PERF_RECORD_SWITCH[_CPU_WIDE] OUT records when event header misc field contains PERF_RECORD_MISC_SWITCH_OUT_YIELD bit set designating synchronization context switch out event:
perf report -D -i system-wide.perf:
...
0x1b9c50 [0x30]: event: 15
.
. ... raw event: size 48 bytes
. 0000: 0f 00 00 00 00 20 30 00 01 1e 00 00 01 1e 00 00 ..... 0......... . 0010: 00 00 00 00 00 00 00 00 85 ae d4 e3 3e 0e 00 00 ............>... . 0020: 54 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 T...............
5 15663273127557 0x1b9c50 [0x30]: PERF_RECORD_SWITCH_CPU_WIDE OUT next pid/tid: 7681/7681
0x2646c0 [0x30]: event: 15
.
. ... raw event: size 48 bytes
. 0000: 0f 00 00 00 00 60 30 00 00 00 00 00 00 00 00 00 .....`0......... . 0010: 00 1e 00 00 00 1e 00 00 29 1e d5 e3 3e 0e 00 00 ........)...>... . 0020: 56 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 V...............
7 15663273156137 0x2646c0 [0x30]: PERF_RECORD_SWITCH_CPU_WIDE OUT yield next pid/tid: 0/0
...
--- tools/include/uapi/linux/perf_event.h | 5 +++++ tools/perf/util/event.c | 4 +++- 2 files changed, 8 insertions(+), 1 deletion(-)
tools/include/uapi/linux/perf_event.h +5 -0
@@ -654,6 +654,11 @@ struct perf_event_mmap_page {* perf_event_attr::precise_ip. */ #define PERF_RECORD_MISC_EXACT_IP (1 << 14) +/* +* Indicates that thread explicitly yielded cpu due to +* a call of some synchronization API e.g. futex system call +*/ +#define PERF_RECORD_MISC_SWITCH_OUT_YIELD (1 << 14)/* * Reserve the last bit to indicate some extended misc field */
tools/perf/util/event.c +3 -1
@@ -1421,7 +1421,9 @@ size_t perf_event__fprintf_itrace_start(union perf_event *event, FILE *fp)size_t perf_event__fprintf_switch (union perf_event *event,FILE *fp) {bool out = event->header.misc &PERF_RECORD_MISC_SWITCH_OUT ; -const char *in_out = out ?"OUT" :"IN " ; +const char *in_out = !out ?"IN " : + !(event->header.misc &PERF_RECORD_MISC_SWITCH_OUT_YIELD ) ? +"OUT" :"OUT yield" ;if (event->header.type ==PERF_RECORD_SWITCH )return fprintf (fp," %s\n" , in_out);
PATCH 3/3 perf script: extend misc field deconding with switch out event type
In reply to Alexey BudankovAppend 'y' sign to 'S' tag designating the type of context switch out event so 'S' means preemption context switch and 'Sy' means synchronization context switch. Documentation is extended to cover new presentation changes.
perf script --show-switch-events -F +misc -I -i system-wide.perf:
amplxe-perf 7681 [005] S 15663.273151: PERF_RECORD_SWITCH_CPU_WIDE OUT next pid/tid: 39/39 migration/5 39 [005] 15663.273152: PERF_RECORD_SWITCH_CPU_WIDE IN prev pid/tid: 7681/7681 amplxe-perf 7680 [007] K 15663.273153: 1 context-switch:
aaa488 schedule ([kernel.kallsyms]) 1a9f50 __poll_nocancel (inlined)
amplxe-perf 7680 [007] Sy 15663.273156: PERF_RECORD_SWITCH_CPU_WIDE OUT yield next pid/tid: 0/0
migration/5 39 [005] K 15663.273157:
--- tools/perf/Documentation/perf-script.txt | 17 +++++++++-------- tools/perf/builtin-script.c | 5 ++++- 2 files changed, 13 insertions(+), 9 deletions(-)
tools/perf/Documentation/perf-script.txt +9 -8
@@ -228,14 +228,15 @@ OPTIONS For sample events it's possible to display misc field with -F +misc option, following letters are displayed for each bit: - PERF_RECORD_MISC_KERNEL K - PERF_RECORD_MISC_USER U - PERF_RECORD_MISC_HYPERVISOR H - PERF_RECORD_MISC_GUEST_KERNEL G - PERF_RECORD_MISC_GUEST_USER g - PERF_RECORD_MISC_MMAP_DATA* M - PERF_RECORD_MISC_COMM_EXEC E - PERF_RECORD_MISC_SWITCH_OUT S + PERF_RECORD_MISC_KERNEL K + PERF_RECORD_MISC_USER U + PERF_RECORD_MISC_HYPERVISOR H + PERF_RECORD_MISC_GUEST_KERNEL G + PERF_RECORD_MISC_GUEST_USER g + PERF_RECORD_MISC_MMAP_DATA* M + PERF_RECORD_MISC_COMM_EXEC E + PERF_RECORD_MISC_SWITCH_OUT S + PERF_RECORD_MISC_SWITCH_OUT_YIELD Sy $ perf script -F +misc ... sched-messaging 1414 K 28690.636582: 4590 cycles ...
tools/perf/builtin-script.c +4 -1
@@ -657,8 +657,11 @@ static int perf_sample__fprintf_start(struct perf_sample *sample,break ;case PERF_RECORD_SWITCH :case PERF_RECORD_SWITCH_CPU_WIDE : -if (has (SWITCH_OUT )) +if (has (SWITCH_OUT )) { ret +=fprintf (fp,"S" ); +if (sample->misc &PERF_RECORD_MISC_SWITCH_OUT_YIELD ) + ret +=fprintf (fp,"y" ); + }default :break ; }
Arnaldo Carvalho de Melo
maintainerEm Mon, Mar 05, 2018 at 02:35:02PM +0300, Alexey Budankov escreveu:
Quoted 9 lines
Here is a series of small patches that implement exposing type of context-switch-out event as a part of PERF_RECORD_SWITCH[_CPU_WIDE] record.
Introduced types of context-switch-out events assumed to be:
a) preempt:task->state==TASK_RUNNING
misc &=PERF_RECORD_MISC_SWITCH_OUTb) yield: !preempt - using new bit
PERF_RECORD_MISC_SWITCH_OUT_YIELD:
misc &= PERF_RECORD_MISC_SWITCH_OUT|PERF_RECORD_MISC_SWITCH_OUT_YIELD
Quoted 3 lines
Perf tool report and script commands output has been extended to decode new yield bit and the updated output looks like in the examples below.
I'm just waiting for the current reviewers to be satisfied with this, but I think this is a great addition and 'perf trace' is another tool that should jump into this, showing forced context switches together with syscalls.
- Arnaldo
Quoted 50 lines
The documentation has been updated to mention yield switch out events and its decoding symbols in perf script output.
The changes have been manually tested on Fedora 27 with the patched kernel:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitperf/coreperf report -D -i system-wide.perf:
0x1b9c50[0x30]: event: 15
.
. ... raw event: size 48 bytes. 0000: 0f 00 00 00 00 20 30 00 01 1e 00 00 01 1e 00 00 ..... 0......... . 0010: 00 00 00 00 00 00 00 00 85 ae d4 e3 3e 0e 00 00 ............>... . 0020: 54 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 T...............5 15663273127557
0x1b9c50[0x30]:PERF_RECORD_SWITCH_CPU_WIDEOUT next pid/tid: 7681/7681
0x2646c0[0x30]: event: 15
.
. ... raw event: size 48 bytes. 0000: 0f 00 00 00 00 60 30 00 00 00 00 00 00 00 00 00 .....`0......... . 0010: 00 1e 00 00 00 1e 00 00 29 1e d5 e3 3e 0e 00 00 ........)...>... . 0020: 56 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 V...............7 15663273156137
0x2646c0[0x30]:PERF_RECORD_SWITCH_CPU_WIDEOUT yield next pid/tid: 0/0perf script --show-switch-events -F +misc -I -i system-wide.perf:
amplxe-perf 7681 [005] S 15663.273151: PERF_RECORD_SWITCH_CPU_WIDE OUT next pid/tid: 39/39 migration/5 39 [005] 15663.273152: PERF_RECORD_SWITCH_CPU_WIDE IN prev pid/tid: 7681/7681 amplxe-perf 7680 [007] K 15663.273153: 1 context-switch:aaa488 schedule ([kernel.kallsyms]) 1a9f50
__poll_nocancel(inlined)amplxe-perf 7680 [007] Sy 15663.273156:
PERF_RECORD_SWITCH_CPU_WIDEOUT yield next pid/tid: 0/0
migration/5 39 [005] K 15663.273157:---
Alexey Budankov (3):
perf/core: store context switch out type into Perf trace perf report: extend raw dump (-D) out with switch out event type perf script: extend misc field decoding with switch out event typeinclude/uapi/linux/perf_event.h | 5 +++++ kernel/events/core.c | 4 +++- tools/include/uapi/linux/perf_event.h | 5 +++++ tools/perf/Documentation/perf-script.txt | 17 +++++++++-------- tools/perf/builtin-script.c | 5 ++++- tools/perf/util/event.c | 4 +++- 6 files changed, 29 insertions(+), 11 deletions(-)
Hi Arnaldo,
Quoted 12 lines · Arnaldo Carvalho de Melo
Em Mon, Mar 05, 2018 at 02:35:02PM +0300, Alexey Budankov escreveu:
Quoted 9 lines
Here is a series of small patches that implement exposing type of context-switch-out event as a part of PERF_RECORD_SWITCH[_CPU_WIDE] record.
Introduced types of context-switch-out events assumed to be:
a) preempt:task->state==TASK_RUNNING
misc &=PERF_RECORD_MISC_SWITCH_OUTb) yield: !preempt - using new bit
PERF_RECORD_MISC_SWITCH_OUT_YIELD:misc &= PERF_RECORD_MISC_SWITCH_OUT|PERF_RECORD_MISC_SWITCH_OUT_YIELD
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
Perf tool report and script commands output has been extended to decode new yield bit and the updated output looks like in the examples below.
I'm just waiting for the current reviewers to be satisfied with this, but I think this is a great addition and 'perf trace' is another tool that should jump into this, showing forced context switches together with syscalls.
It's great to know there is a value in that change for other Perf tools.
Extending perf trace (strace inspired) tool in that respect might makes sense. I anticipate possible tracing overhead needs to be expected and probably handled somehow.
But, anyway, yep, per-thread syscall traces enriched with typed context switch boundaries may be the great extension, in comparison to the original strace tool.
IMHO, some simple summary metrics like amount of preempt or yield context switches (per-thread or per-process) could bring even more value into perf trace tool functionality.
BR,
Alexey
Quoted 54 lines
- Arnaldo
Quoted 50 lines
The documentation has been updated to mention yield switch out events and its decoding symbols in perf script output.
The changes have been manually tested on Fedora 27 with the patched kernel:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitperf/coreperf report -D -i system-wide.perf:
0x1b9c50[0x30]: event: 15
.
. ... raw event: size 48 bytes. 0000: 0f 00 00 00 00 20 30 00 01 1e 00 00 01 1e 00 00 ..... 0......... . 0010: 00 00 00 00 00 00 00 00 85 ae d4 e3 3e 0e 00 00 ............>... . 0020: 54 00 00 00 00 00 00 00 05 00 00 00 00 00 00 00 T...............5 15663273127557
0x1b9c50[0x30]:PERF_RECORD_SWITCH_CPU_WIDEOUT next pid/tid: 7681/7681
0x2646c0[0x30]: event: 15
.
. ... raw event: size 48 bytes. 0000: 0f 00 00 00 00 60 30 00 00 00 00 00 00 00 00 00 .....`0......... . 0010: 00 1e 00 00 00 1e 00 00 29 1e d5 e3 3e 0e 00 00 ........)...>... . 0020: 56 00 00 00 00 00 00 00 07 00 00 00 00 00 00 00 V...............7 15663273156137
0x2646c0[0x30]:PERF_RECORD_SWITCH_CPU_WIDEOUT yield next pid/tid: 0/0perf script --show-switch-events -F +misc -I -i system-wide.perf:
amplxe-perf 7681 [005] S 15663.273151: PERF_RECORD_SWITCH_CPU_WIDE OUT next pid/tid: 39/39 migration/5 39 [005] 15663.273152: PERF_RECORD_SWITCH_CPU_WIDE IN prev pid/tid: 7681/7681 amplxe-perf 7680 [007] K 15663.273153: 1 context-switch:aaa488 schedule ([kernel.kallsyms]) 1a9f50
__poll_nocancel(inlined)amplxe-perf 7680 [007] Sy 15663.273156:
PERF_RECORD_SWITCH_CPU_WIDEOUT yield next pid/tid: 0/0
migration/5 39 [005] K 15663.273157:---
Alexey Budankov (3):
perf/core: store context switch out type into Perf trace perf report: extend raw dump (-D) out with switch out event type perf script: extend misc field decoding with switch out event typeinclude/uapi/linux/perf_event.h | 5 +++++ kernel/events/core.c | 4 +++- tools/include/uapi/linux/perf_event.h | 5 +++++ tools/perf/Documentation/perf-script.txt | 17 +++++++++-------- tools/perf/builtin-script.c | 5 ++++- tools/perf/util/event.c | 4 +++- 6 files changed, 29 insertions(+), 11 deletions(-)
Patches look good to me.