perf stat: add interval counter printing
v4 cover perf stat: add interval counter printing
This small patchset extends perf stat with the -I option.
The -I option enables interval printing. It expects an interval duration in milliseconds. Minimum is 100ms. Once, activated perf stat prints events deltas since last printout. All modes are supported.
$ perf stat -I 1000 -e cycles noploop 10 noploop for 10 seconds 1.000086918 2385155642 cycles # 0.000 GHz 2.000267937 2392279774 cycles # 0.000 GHz 3.000385400 2390971450 cycles # 0.000 GHz 4.000504408 2390996752 cycles # 0.000 GHz 5.000626878 2390853097 cycles # 0.000 GHz
The output format makes it easy to feed into a plotting program such as gnuplot when the -I option is used in combination with the -x option:
$ perf stat -x, -I 1000 -e cycles noploop 10 noploop for 10 seconds 1.000084113,2378775498,cycles 2.000245798,2391056897,cycles 3.000354445,2392089414,cycles 4.000459115,2390936603,cycles 5.000565341,2392108173,cycles
The first patch adds the code to keep previous counts. The second is the actual extension of perf stat.
In v2, we changed the option from -T to -I based on feedback from Andi Kleen (who is using -T in an upcoming patch).
In v3, added compute_deltas() to evsel.c to factorize deltas computation based on Jiri's comments.
In v4, we made sure that counts are printed in case the run is interrupted by CTRL-C based on Andi Kleen's feedback.
---
Stephane Eranian (2):
perf tools: add evsel prev_raw_count field
perf stat: add interval printing
tools/perf/Documentation/perf-stat.txt | 4 + tools/perf/builtin-stat.c | 154 ++++++++++++++++++++++++++++---- tools/perf/util/evsel.c | 26 ++++++ tools/perf/util/evsel.h | 1 + 4 files changed, 170 insertions(+), 15 deletions(-)
Signature · 1 line
1.7.9.5
PATCH v4 1/2 perf tools: add evsel prev_raw_count field
This field will be used by commands which print
counter deltas on regular timer intervals,
such as perf stat -I.
--- tools/perf/util/evsel.c | 26 ++++++++++++++++++++++++++ tools/perf/util/evsel.h | 1 + 2 files changed, 27 insertions(+)
tools/perf/util/evsel.c +26 -0
@@ -631,6 +631,28 @@ void perf_evsel__delete(struct perf_evsel *evsel)free (evsel); } +static inline void compute_deltas (struct perf_evsel *evsel, +int cpu, +struct perf_counts_values *count) +{ +struct perf_counts_values tmp; + +if (!evsel->prev_raw_counts) +return ; + +if (cpu ==-1 ) { + tmp = evsel->prev_raw_counts->aggr; + evsel->prev_raw_counts->aggr = *count; + }else { + tmp = evsel->prev_raw_counts->cpu[cpu]; + evsel->prev_raw_counts->cpu[cpu] = *count; + } + + count->val = count->val - tmp.val; + count->ena = count->ena - tmp.ena; + count->run = count->run - tmp.run; +} +int __perf_evsel__read_on_cpu (struct perf_evsel *evsel,int cpu,int thread,bool scale) {
@@ -646,6 +668,8 @@ int __perf_evsel__read_on_cpu(struct perf_evsel *evsel,if (readn (FD (evsel, cpu, thread), &count, nv *sizeof (u64)) <0 )return -errno; +compute_deltas (evsel, cpu, &count); +if (scale) {if (count.run ==0 ) count.val =0 ;
@@ -684,6 +708,8 @@ int __perf_evsel__read(struct perf_evsel *evsel, } } +compute_deltas (evsel,-1 , aggr); + evsel->counts->scaled =0 ;if (scale) {if (aggr->run ==0 ) {
tools/perf/util/evsel.h +1 -0
@@ -53,6 +53,7 @@ struct perf_evsel {struct xyarray *sample_id;u64 *id;struct perf_counts *counts; +struct perf_counts *prev_raw_counts;int idx;u32 ids;struct hists hists;
Signature · 1 line
1.7.9.5
PATCH v4 2/2 perf stat: add interval printing
In reply to Stephane EranianThis patch adds a new printing mode for perf stat. It allows internval printing. That means perf stat can now print event deltas at regular time interval. This is useful to detect phases in programs.
The -I option enables interval printing. It expects an interval duration in milliseconds. Minimum is 100ms. Once, activated perf stat prints events deltas since last printout. All modes are supported.
$ perf stat -I 1000 -e cycles noploop 10 noploop for 10 seconds 1.000086918 2385155642 cycles # 0.000 GHz 2.000267937 2392279774 cycles # 0.000 GHz 3.000385400 2390971450 cycles # 0.000 GHz 4.000504408 2390996752 cycles # 0.000 GHz 5.000626878 2390853097 cycles # 0.000 GHz
The output format makes it easy to feed into a plotting program such as gnuplot when the -I option is used in combination with the -x option:
$ perf stat -x, -I 1000 -e cycles noploop 10 noploop for 10 seconds 1.000084113,2378775498,cycles 2.000245798,2391056897,cycles 3.000354445,2392089414,cycles 4.000459115,2390936603,cycles 5.000565341,2392108173,cycles
--- tools/perf/Documentation/perf-stat.txt | 4 + tools/perf/builtin-stat.c | 154 ++++++++++++++++++++++++++++---- 2 files changed, 143 insertions(+), 15 deletions(-)
tools/perf/Documentation/perf-stat.txt +4 -0
@@ -114,6 +114,10 @@ Log output to fd, instead of stderr. Complementary to --output, and mutually ex perf stat --repeat 10 --null --sync --pre 'make -s O=defconfig-build/clean' -- make -s -j64 O=defconfig-build/ bzImage +-I msecs:: +--interval-print msecs:: + print count deltas every N milliseconds (minimum: 100ms) + example: perf stat -I 1000 -e cycles -a sleep 5 EXAMPLES --------
tools/perf/builtin-stat.c +139 -15
304 lines not on this page · raw message
Signature · 1 line
1.7.9.5
Namhyung Kim
maintainerRe: [PATCH v4 2/2] perf stat: add interval printing
Hi Stephane,
Quoted 5 lines · Stephane Eranian
This patch adds a new printing mode for perf stat. It allows internval printing. That means perf stat can now print event deltas at regular time interval. This is useful to detect phases in programs.
Quoted 7 lines
The -I option enables interval printing. It expects an interval duration in milliseconds. Minimum is 100ms. Once, activated perf stat prints events deltas since last printout. All modes are supported.
$ perf stat -I 1000 -e cycles noploop 10 noploop for 10 seconds
Is this line an output from perf stat?
In addition, how about adding a head line like:
# time count event #
Quoted 20 lines
1.000086918 2385155642 cycles # 0.000 GHz 2.000267937 2392279774 cycles # 0.000 GHz 3.000385400 2390971450 cycles # 0.000 GHz 4.000504408 2390996752 cycles # 0.000 GHz 5.000626878 2390853097 cycles # 0.000 GHzThe output format makes it easy to feed into a plotting program such as gnuplot when the -I option is used in combination with the -x option:
$ perf stat -x, -I 1000 -e cycles noploop 10 noploop for 10 seconds 1.000084113,2378775498,cycles 2.000245798,2391056897,cycles 3.000354445,2392089414,cycles 4.000459115,2390936603,cycles 5.000565341,2392108173,cyclesSigned-off-by Stephane Eranian <[email protected]>---
[snip]
Quoted 10 lines
tools/perf/builtin-stat.c +2
-0@@ -877,6 +977,8 @@ static void print_counter(struct perf_evsel *counter)static void print_stat (int argc,const char **argv) {struct perf_evsel *counter; +struct timespec ts, rs; +char prefix[64 ] = {0 , };int i;fflush (stdout);
tools/perf/builtin-stat.c static void print_stat(int argc, const char **argv) · line 1008
Quoted 8 lines
tools/perf/builtin-stat.c +5
-0@@ -899,12 +1001,18 @@ static void print_stat(int argc, const char **argv)fprintf (output,":\n\n" ); } +if (interval) { +clock_gettime (CLOCK_MONOTONIC , &ts); +diff_timespec (&rs, &ts, &ref_time); +sprintf (prefix,"%lu.%09lu%s" , rs.tv_sec, rs.tv_nsec, csv_sep); + }
AFAICS the only caller of print_stat() is cmd_stat() and it'll call this only if interval is 0. So why not just setting prefix to NULL then?
Quoted 13 lines
tools/perf/builtin-stat.c +3
-2+if (no_aggr) {list_for_each_entry (counter, &evsel_list->entries, node) -print_counter (counter); +print_counter (counter, prefix); }else {list_for_each_entry (counter, &evsel_list->entries, node) -print_counter_aggr (counter); +print_counter_aggr (counter, prefix); }if (!csv_output) {
tools/perf/builtin-stat.c static volatile int signr = -1; · line 1036
Quoted 5 lines
tools/perf/builtin-stat.c +1
-1@@ -925,7 +1033,7 @@ static volatile int signr = -1;static void skip_signal (int signo) { -if (child_pid ==-1 ) +if ((child_pid ==-1 ) || interval)
Looks like it needs a whitespace :)
Quoted 13 lines
tools/perf/builtin-stat.c +2
-0done =1 ; signr = signo;@@ -1145,6 +1253,8 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) "command to run prior to the measured command"),OPT_STRING (0 ,"post" , &post_cmd,"command" ,"command to run after to the measured command" ), +OPT_INTEGER ('I' ,"interval-print" , &interval, +"print counts at regular interval in ms (>= 100)" ),OPT_END () }; const char * const stat_usage[] = {
tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) · line 1362
Quoted 8 lines
tools/perf/builtin-stat.c +5
-0@@ -1245,12 +1355,23 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)usage_with_options (stat_usage, options);return -1 ; } +if (interval <0 || (interval >0 && interval <100 )) { +pr_err ("print interval must be >= 100ms\n" ); +usage_with_options (stat_usage, options); +return -1 ; + }
How about making 'interval' unsigned and simplify the condition a bit:
if (interval && interval < 100) {
...
}Thanks,
Namhyung
tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) · line 1374
Quoted 12 lines
tools/perf/builtin-stat.c +6
-0list_for_each_entry (pos, &evsel_list->entries, node) {if (perf_evsel__alloc_stat_priv (pos) <0 ||perf_evsel__alloc_counts (pos,perf_evsel__nr_cpus (pos)) <0 ) goto out_free_fd; } +if (interval) { +list_for_each_entry (pos, &evsel_list->entries, node) { +if (perf_evsel__alloc_prev_raw_counts (pos) <0 ) + goto out_free_fd; + } + }
It's not about your patch, but I can't find where it frees evsel->counts
- a counter part of perf_evsel__alloc_counts(). Seems we leak that?
Quoted 27 lines
tools/perf/builtin-stat.c +5
-2/* * We dont want to block the signals - that would cause @@ -1260,6 +1381,7 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)*/ atexit (sig_atexit);signal (SIGINT , skip_signal); +signal (SIGCHLD , skip_signal);signal (SIGALRM , skip_signal);signal (SIGABRT , skip_signal);@@ -1272,11 +1394,13 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) status =run_perf_stat (argc, argv); } -if (status !=-1 ) +if (status !=-1 && !interval)print_stat (argc, argv); out_free_fd: -list_for_each_entry (pos, &evsel_list->entries, node) +list_for_each_entry (pos, &evsel_list->entries, node) {perf_evsel__free_stat_priv (pos); +perf_evsel__free_prev_raw_counts (pos); + }perf_evlist__delete_maps (evsel_list); out:perf_evlist__delete (evsel_list);
Re: [PATCH v4 2/2] perf stat: add interval printing
Quoted 16 lines · Namhyung Kim
Hi Stephane,
Quoted 5 lines · Stephane Eranian
This patch adds a new printing mode for perf stat. It allows internval printing. That means perf stat can now print event deltas at regular time interval. This is useful to detect phases in programs.
Quoted 7 lines
The -I option enables interval printing. It expects an interval duration in milliseconds. Minimum is 100ms. Once, activated perf stat prints events deltas since last printout. All modes are supported.
$ perf stat -I 1000 -e cycles noploop 10 noploop for 10 seconds
Quoted 6 lines
Is this line an output from perf stat?
In addition, how about adding a head line like:
# time count event #
Will add that in non -x mode.
Quoted 34 lines
Quoted 20 lines
1.000086918 2385155642 cycles # 0.000 GHz 2.000267937 2392279774 cycles # 0.000 GHz 3.000385400 2390971450 cycles # 0.000 GHz 4.000504408 2390996752 cycles # 0.000 GHz 5.000626878 2390853097 cycles # 0.000 GHzThe output format makes it easy to feed into a plotting program such as gnuplot when the -I option is used in combination with the -x option:
$ perf stat -x, -I 1000 -e cycles noploop 10 noploop for 10 seconds 1.000084113,2378775498,cycles 2.000245798,2391056897,cycles 3.000354445,2392089414,cycles 4.000459115,2390936603,cycles 5.000565341,2392108173,cyclesSigned-off-by Stephane Eranian <[email protected]>---
[snip]
Quoted 13 lines
tools/perf/builtin-stat.c +2
-0@@ -877,6 +977,8 @@ static void print_counter(struct perf_evsel *counter)static void print_stat (int argc,const char **argv) {struct perf_evsel *counter; +struct timespec ts, rs; +char prefix[64 ] = {0 , };int i;fflush (stdout);@@ -899,12 +1001,18 @@ static void print_stat(int argc, const char **argv)fprintf (output,":\n\n" ); }
Quoted 9 lines
tools/perf/builtin-stat.c static void print_stat(int argc, const char **argv) · line 1008
Quoted 5 lines
tools/perf/builtin-stat.c +5
-0+if (interval) { +clock_gettime (CLOCK_MONOTONIC , &ts); +diff_timespec (&rs, &ts, &ref_time); +sprintf (prefix,"%lu.%09lu%s" , rs.tv_sec, rs.tv_nsec, csv_sep); + }AFAICS the only caller of
print_stat()iscmd_stat()and it'll call this only if interval is 0. So why not just setting prefix to NULL then?
I don't understand your point here. Prefix is set ONLY when interval is non zero. Prefix is setup before print_counter() so that each counter for each interval is timestamped with the same value.
Quoted 15 lines
Quoted 15 lines
tools/perf/builtin-stat.c +3
-2+if (no_aggr) {list_for_each_entry (counter, &evsel_list->entries, node) -print_counter (counter); +print_counter (counter, prefix); }else {list_for_each_entry (counter, &evsel_list->entries, node) -print_counter_aggr (counter); +print_counter_aggr (counter, prefix); }if (!csv_output) {@@ -925,7 +1033,7 @@ static volatile int signr = -1;
Quoted 7 lines
tools/perf/builtin-stat.c static volatile int signr = -1; · line 1036
Quoted 4 lines
tools/perf/builtin-stat.c +1
-1static void skip_signal (int signo) { -if (child_pid ==-1 ) +if ((child_pid ==-1 ) || interval)Looks like it needs a whitespace :)
Fixed.
Quoted 16 lines
Quoted 16 lines
tools/perf/builtin-stat.c +2
-0done =1 ; signr = signo;@@ -1145,6 +1253,8 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) "command to run prior to the measured command"),OPT_STRING (0 ,"post" , &post_cmd,"command" ,"command to run after to the measured command" ), +OPT_INTEGER ('I' ,"interval-print" , &interval, +"print counts at regular interval in ms (>= 100)" ),OPT_END () }; const char * const stat_usage[] = {@@ -1245,12 +1355,23 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)usage_with_options (stat_usage, options);return -1 ;
Quoted 9 lines
tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) · line 1362
Quoted 6 lines
tools/perf/builtin-stat.c +5
-0} +if (interval <0 || (interval >0 && interval <100 )) { +pr_err ("print interval must be >= 100ms\n" ); +usage_with_options (stat_usage, options); +return -1 ; + }How about making 'interval' unsigned and simplify the condition a bit:
Done.
Quoted 20 lines
if (interval && interval < 100) { ... }Thanks,
Namhyungtools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) · line 1374
Quoted 12 lines
tools/perf/builtin-stat.c +6
-0list_for_each_entry (pos, &evsel_list->entries, node) {if (perf_evsel__alloc_stat_priv (pos) <0 ||perf_evsel__alloc_counts (pos,perf_evsel__nr_cpus (pos)) <0 ) goto out_free_fd; } +if (interval) { +list_for_each_entry (pos, &evsel_list->entries, node) { +if (perf_evsel__alloc_prev_raw_counts (pos) <0 ) + goto out_free_fd; + } + }
Quoted 3 lines
It's not about your patch, but I can't find where it frees
evsel->counts
- a counter part ofperf_evsel__alloc_counts(). Seems we leak that?
You are right. But it's beyond the scope of my patch. Truth is, there is not even a perf_evsel__free_counts().
Quoted 28 lines
Quoted 28 lines
tools/perf/builtin-stat.c +5
-2/* * We dont want to block the signals - that would cause @@ -1260,6 +1381,7 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)*/ atexit (sig_atexit);signal (SIGINT , skip_signal); +signal (SIGCHLD , skip_signal);signal (SIGALRM , skip_signal);signal (SIGABRT , skip_signal);@@ -1272,11 +1394,13 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) status =run_perf_stat (argc, argv); } -if (status !=-1 ) +if (status !=-1 && !interval)print_stat (argc, argv); out_free_fd: -list_for_each_entry (pos, &evsel_list->entries, node) +list_for_each_entry (pos, &evsel_list->entries, node) {perf_evsel__free_stat_priv (pos); +perf_evsel__free_prev_raw_counts (pos); + }perf_evlist__delete_maps (evsel_list); out:perf_evlist__delete (evsel_list);
Namhyung Kim
maintainerRe: [PATCH v4 2/2] perf stat: add interval printing
Hi Stephane,
Quoted 7 lines · Stephane Eranian
Quoted 3 lines · Namhyung Kim
AFAICS the only caller of
print_stat()iscmd_stat()and it'll call this only if interval is 0. So why not just setting prefix to NULL then?I don't understand your point here. Prefix is set ONLY when interval is non zero. Prefix is setup before
print_counter()so that each counter for each interval is timestamped with the same value.
Please see below.
Quoted 3 lines
Quoted 3 lines
tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) · line 1398
Quoted 3 lines
tools/perf/builtin-stat.c +1
-1-if (status !=-1 ) +if (status !=-1 && !interval)print_stat (argc, argv);
Here, print_stat() is called only if interval is 0. So no need to check the interval inside the print_stat(), right?
Thanks,
Namhyung
Re: [PATCH v4 2/2] perf stat: add interval printing
Quoted 14 lines · Namhyung Kim
Hi Stephane,
Quoted 7 lines · Stephane Eranian
Quoted 3 lines · Namhyung Kim
AFAICS the only caller of
print_stat()iscmd_stat()and it'll call this only if interval is 0. So why not just setting prefix to NULL then?I don't understand your point here. Prefix is set ONLY when interval is non zero. Prefix is setup before
print_counter()so that each counter for each interval is timestamped with the same value.Please see below.
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
tools/perf/builtin-stat.c int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused) · line 1398
Quoted 3 lines
tools/perf/builtin-stat.c +1
-1-if (status !=-1 ) +if (status !=-1 && !interval)print_stat (argc, argv);Here,
print_stat()is called only if interval is 0. So no need to check the interval inside theprint_stat(), right?
Ok, got it. Due to some leftover code. Will fix in V6. Thanks.