Skip to content

perf: support enable and disable commands in stat and record modes

Alexey Budankov lkml 13 versions – Applied

Status from this message: Thanks, applied.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v12 Latest15 patches 9 repliesApplied
  2. v1115 patches 7 repliesSuperseded
  3. v1015 patches 5 repliesSuperseded
Earlier versions · 10
VersionPostedPatchesRepliesStatus
v915 patches17 repliesSuperseded
v813 patches35 repliesSuperseded
v713 patches30 repliesSuperseded
v613 patches9 repliesSuperseded
v513 patches16 repliesSuperseded
v410 patches11 repliesSuperseded
v3 posting 2/29 patches15 repliesSuperseded
v3 posting 1/29 patches0 repliesSuperseded
v211 patches13 repliesSuperseded
v18 patches8 repliesSuperseded

Changes in v12 from the cover letter

- fixed fdarray sanity test (36) failure - due to usage of uninitialized memory after realloc() of priv data; added explicit memset(0) for newly allocated part
- captured and mentioned acquired Acked-by tags

Patches in v12 Applied

#Subject and filesSizeRepliesApplied
1/15tools/libperf: avoid internal moving of fdarray fdstools/lib/api/fd/array.c tools/perf/tests/fdarray.c+7 -241 reply
2/15tools/libperf: add flags to fdarray fds objectstools/lib/api/fd/array.c tools/lib/api/fd/array.h tools/lib/perf/evlist.c +3 more+25 -121 reply
3/15tools/libperf: avoid counting of nonfilterable fdarray fdstools/lib/api/fd/array.c+2 -11 reply
4/15perf evlist: introduce control file descriptorstools/perf/util/evlist.c tools/perf/util/evlist.h+8 -00 replies
5/15perf evlist: implement control command handling functionstools/perf/util/evlist.c tools/perf/util/evlist.h+158 -04 replies
6/15perf stat: factor out body of event handling loop for system widetools/perf/builtin-stat.c+11 -60 replies
7/15perf stat: move target check to loop control statementtools/perf/builtin-stat.c+1 -30 replies
8/15perf stat: factor out body of event handling loop for fork casetools/perf/builtin-stat.c+1 -40 replies
9/15perf stat: factor out event handling loop into dispatch_events()tools/perf/builtin-stat.c+24 -120 replies
10/15perf stat: extend -D,--delay option with -1 valuetools/perf/Documentation/perf-stat.txt tools/perf/builtin-stat.c tools/perf/util/evlist.h +1 more+21 -70 replies
11/15perf stat: implement control commands handlingtools/perf/builtin-stat.c+65 -182 replies
12/15perf stat: introduce --control fd:ctl-fd[,ack-fd] optionstools/perf/Documentation/perf-stat.txt tools/perf/builtin-stat.c tools/perf/util/stat.h+80 -10 replies
13/15perf record: extend -D,--delay option with -1 valuetools/perf/Documentation/perf-record.txt tools/perf/builtin-record.c tools/perf/builtin-trace.c +1 more+13 -80 replies
14/15perf record: implement control commands handlingtools/perf/builtin-record.c+16 -00 replies
15/15perf record: introduce --control fd:ctl-fd[,ack-fd] optionstools/perf/Documentation/perf-record.txt tools/perf/builtin-record.c tools/perf/util/record.h+78 -00 replies

Reviews trailers per patch

All 15 patches have a Reviewed-by or Acked-by.

PatchJiri OlsaNamhyung Kim
1/15A-b A-b
2/15A-b A-b
3/15A-b A-b
4/15A-b A-b
5/15A-b A-b
6/15A-b A-b
7/15A-b A-b
8/15A-b A-b
9/15A-b A-b
10/15A-b A-b
11/15A-b A-b
12/15A-b A-b
13/15A-b A-b
14/15A-b A-b
15/15A-b A-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v12 thread

Cover letter [PATCH v12 00/15] perf: support enable and disable commands in stat and record modes

Changes in v12:
- fixed fdarray sanity test (36) failure - due to usage of uninitialized memory after realloc() of priv data; added explicit memset(0) for newly allocated part
- captured and mentioned acquired Acked-by tags

v11: https://lore.kernel.org/lkml/[email protected]/

Changes in v11:
- added comments to struct ctl_fd
- explicitly coded data_size decrement

v10: https://lore.kernel.org/lkml/[email protected]/

Changes in v10:
- reused struct priv, renamed props into flags and used unsigned int type as a storage
- implemented and adopted usage of evlist__ctlfd_initialized() function
- updated while condition with checking !(done || stop)
- implemented compute_tts() to calculate the remainder of timeout to wait in poll()

v9: https://lore.kernel.org/lkml/[email protected]/

Changes in v9:
- avoided screwing of fds by fdarray__filter() to fix staleness of pos returned by fdarray__add()
- implemented notion of properties for fds kept by fdarray object and introduced nonfilterable property for control fd
- avoided counting of nonfilterable fds by fdarray__filter()
- replaced strlen() with sizeof(str)-1 for CMD_TAGS
- avoided mixing timeout and interval processing in one function
- renamed child to child_exited variable
- processed time_to_sleep < time_diff condition
- placed evlist__ctlfd_process() call after evlist__poll() and evlist__filter_pollfd() calls in record mode

v8: https://lore.kernel.org/lkml/[email protected]/

Changes in v8:
- avoided moving of fds at fdarray__filter() call
- skipped counting of fds with zeroed revents at fdarray__filter() call
- converted explicit --ctl-fd[-ack] into --control fd:ctl-fd[,ack-fd option
- updated docs to accommodate --control fd:ctl-fd[,ack-fd] option

v7: https://lore.kernel.org/lkml/[email protected]/

Changes in v7:
- added missing perf-record.txt changes
- adjusted docs wording for --ctl-fd,ctl-fd-ack options to additionally mention --delay=-1 effect

v6: https://lore.kernel.org/lkml/[email protected]/

Changes in v6:
- split re-factoring of events handling loops for stat mode into smaller incremental parts
- added parts missing at v5
- corrected v5 runtime issues

v5: https://lore.kernel.org/lkml/[email protected]/

Changes in v5:
- split re-factoring of events handling loops for stat mode into smaller incremental parts

v4: https://lore.kernel.org/lkml/[email protected]/

Changes in v4:
- made checking of ctlfd state unconditional in record trace streaming loop
- introduced static poll fds to keep evlist__filter_pollfd() unaffected
- handled ret code of evlist__initialize_ctlfd() where need
- renamed and structured handle_events() function
- applied anonymous structs where needed

v3: https://lore.kernel.org/lkml/[email protected]/

Changes in v3:
- renamed functions and types from perf_evlist_ to evlist_ to avoid clash with libperf code;
- extended commands to be strings of variable length consisting of command name and also possibly including command specific data;
- merged docs update with the code changes;
- updated docs for -D,--delay=-1 option for stat and record modes;

v2: https://lore.kernel.org/lkml/[email protected]/

Changes in v2:
- renamed resume and pause commands to enable and disable ones, renamed CTL_CMD_RESUME and CTL_CMD_PAUSE to CTL_CMD_ENABLE and CTL_CMD_DISABLE to fit to the appropriate ioctls and avoid mixing up with PAUSE_OUTPUT ioctl;
- factored out event handling loop into a handle_events() for stat mode;
- separated -D,--delay=-1 into separate patches for stat and record modes;

v1: https://lore.kernel.org/lkml/[email protected]/

repo: tip of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/core

The patch set implements handling of 'start disabled', 'enable' and 'disable' external control commands which can be provided for stat and record modes of the tool from an external controlling process. 'start disabled' command can be used to postpone enabling of events in the beginning of a monitoring session. 'enable' and 'disable' commands can be used to enable and disable events correspondingly any time after the start of the session.

The 'start disabled', 'enable' and 'disable' external control commands can be used to focus measurement on specially selected time intervals of workload execution. Focused measurement reduces tool intrusion and influence on workload behavior, reduces distortion and amount of collected and stored data, mitigates data accuracy loss because measurement and data capturing happen only during intervals of interest.

A controlling process can be a bash shell script [1], native executable or any other language program that can directly work with file descriptors, e.g. pipes [2], and spawn a process, specially the tool one.

-D,--delay <val> option is extended with -1 value to skip events enabling in the beginning of a monitoring session ('start disabled' command). --control fd:ctl-fd[,ack-fd] command line option is introduced to provide the tool with a pair of file descriptors to listen to control commands and reply to the controlling process on the completion of received commands.

The tool reads control command message from ctl-fd descriptor, handles the command and optionally replies acknowledgement message to ack-fd descriptor, if it is specified on the command line. 'enable' command is recognized as 'enable' string message and 'disable' command is recognized as 'disable' string message both received from ctl-fd descriptor. Completion message is 'ack\n' and sent to ack-fd descriptor.

Example bash script demonstrating simple use case follows:

#!/bin/bash

ctl_dir=/tmp/

ctl_fifo=${ctl_dir}perf_ctl.fifo
test -p ${ctl_fifo} && unlink ${ctl_fifo}
mkfifo ${ctl_fifo} && exec {ctl_fd}<>${ctl_fifo}

ctl_ack_fifo=${ctl_dir}perf_ctl_ack.fifo
test -p ${ctl_ack_fifo} && unlink ${ctl_ack_fifo} mkfifo ${ctl_ack_fifo} && exec {ctl_fd_ack}<>${ctl_ack_fifo}

perf stat -D -1 -e cpu-cycles -a -I 1000       \
          --control fd:${ctl_fd},${ctl_fd_ack} \
          -- sleep 40 &

perf_pid=$!

sleep 5 && echo 'enable' >&${ctl_fd} && read -u ${ctl_fd_ack} e1 && echo "enabled(${e1})" sleep 10 && echo 'disable' >&${ctl_fd} && read -u ${ctl_fd_ack} d1 && echo "disabled(${d1})" sleep 5 && echo 'enable' >&${ctl_fd} && read -u ${ctl_fd_ack} e2 && echo "enabled(${e2})" sleep 10 && echo 'disable' >&${ctl_fd} && read -u ${ctl_fd_ack} d2 && echo "disabled(${d2})"

exec {ctl_fd_ack}>&- && unlink ${ctl_ack_fifo}
exec {ctl_fd}>&- && unlink ${ctl_fifo}

wait -n ${perf_pid}
exit $?

Script output:

[root@host dir] example
Events disabled
# time counts unit events

     1.001101062      <not counted>      cpu-cycles                                                  
     2.002994944      <not counted>      cpu-cycles                                                  
     3.004864340      <not counted>      cpu-cycles                                                  
     4.006727177      <not counted>      cpu-cycles                                                  
Events enabled
enabled(ack)
     4.993808464          3,124,246      cpu-cycles                                                  
     5.008597004          3,325,624      cpu-cycles                                                  
     6.010387483         83,472,992      cpu-cycles                                                  
     7.012266598         55,877,621      cpu-cycles                                                  
     8.014175695         97,892,729      cpu-cycles                                                  
     9.016056093         68,461,242      cpu-cycles                                                  
    10.017937507         55,449,643      cpu-cycles                                                  
    11.019830154         68,938,167      cpu-cycles                                                  
    12.021719952         55,164,101      cpu-cycles                                                  
    13.023627550         70,535,720      cpu-cycles                                                  
    14.025580995         53,240,125      cpu-cycles                                                  
disabled(ack)
    14.997518260         53,558,068      cpu-cycles                                                  
Events disabled
    15.027216416      <not counted>      cpu-cycles                                                  
    16.029052729      <not counted>      cpu-cycles                                                  
    17.030904762      <not counted>      cpu-cycles                                                  
    18.032073424      <not counted>      cpu-cycles                                                  
    19.033805074      <not counted>      cpu-cycles                                                  
Events enabled
enabled(ack)
    20.001279097          3,021,022      cpu-cycles                                                  
    20.035044381          6,434,367      cpu-cycles                                                  
    21.036923813         89,358,251      cpu-cycles                                                  
    22.038825169         72,516,351      cpu-cycles                                                  
#           time             counts unit events
    23.040715596         55,046,157      cpu-cycles                                                  
    24.042643757         78,128,649      cpu-cycles                                                  
    25.044558535         61,052,428      cpu-cycles                                                  
    26.046452785         62,142,806      cpu-cycles                                                  
    27.048353021         74,477,971      cpu-cycles                                                  
    28.050241286         61,001,623      cpu-cycles                                                  
    29.052149961         61,653,502      cpu-cycles                                                  
disabled(ack)
    30.004980264         82,729,640      cpu-cycles                                                  
Events disabled
    30.053516176      <not counted>      cpu-cycles                                                  
    31.055348366      <not counted>      cpu-cycles                                                  
    32.057202097      <not counted>      cpu-cycles                                                  
    33.059040702      <not counted>      cpu-cycles                                                  
    34.060843288      <not counted>      cpu-cycles                                                  
    35.000888624      <not counted>      cpu-cycles                                                  

[root@host dir]#

[1] http://man7.org/linux/man-pages/man1/bash.1.html
[2] http://man7.org/linux/man-pages/man2/pipe.2.html

---
Alexey Budankov (15):
tools/libperf: avoid internal moving of fdarray fds tools/libperf: add flags to fdarray fds objects tools/libperf: avoid counting of nonfilterable fdarray fds perf evlist: introduce control file descriptors
perf evlist: implement control command handling functions perf stat: factor out body of event handling loop for system wide perf stat: move target check to loop control statement perf stat: factor out body of event handling loop for fork case perf stat: factor out event handling loop into dispatch_events() perf stat: extend -D,--delay option with -1 value
perf stat: implement control commands handling
perf stat: introduce --control fd:ctl-fd[,ack-fd] options perf record: extend -D,--delay option with -1 value perf record: implement control commands handling
perf record: introduce --control fd:ctl-fd[,ack-fd] options

 tools/lib/api/fd/array.c                 |  23 +--
 tools/lib/api/fd/array.h                 |  16 +-
 tools/lib/perf/evlist.c                  |   6 +-
 tools/lib/perf/include/internal/evlist.h |   2 +-
 tools/perf/Documentation/perf-record.txt |  44 +++++-
 tools/perf/Documentation/perf-stat.txt   |  44 +++++-
 tools/perf/builtin-record.c              |  65 +++++++-
 tools/perf/builtin-stat.c                | 183 ++++++++++++++++++-----
 tools/perf/builtin-trace.c               |   2 +-
 tools/perf/tests/fdarray.c               |  22 +--
 tools/perf/util/evlist.c                 | 145 +++++++++++++++++-
 tools/perf/util/evlist.h                 |  26 ++++
 tools/perf/util/record.h                 |   4 +-
 tools/perf/util/stat.h                   |   4 +-
 14 files changed, 500 insertions(+), 86 deletions(-)
Signature · 1 line
2.24.1