Add fine grained sampled metrics for perf script

---
 tools/perf/Documentation/perf-script.txt | 10 +++-
 tools/perf/builtin-script.c              | 97 +++++++++++++++++++++++++++++++-
 tools/perf/util/metricgroup.c            |  4 ++
 3 files changed, 108 insertions(+), 3 deletions(-)

tools/perf/Documentation/perf-script.txt +1 -1

index 2811fcf684cb..974ceb12c7f3 100644
@@ -117,7 +117,7 @@ OPTIONS
         Comma separated list of fields to print. Options are:
         comm, tid, pid, time, cpu, event, trace, ip, sym, dso, addr, symoff,
         srcline, period, iregs, uregs, brstack, brstacksym, flags, bpf-output, brstackinsn,
-        brstackoff, callindent, insn, insnlen, synth, phys_addr.
+	brstackoff, callindent, insn, insnlen, synth, phys_addr, metric.
         Field list can be prepended with the type, trace, sw or hw,
         to indicate to which event type the field list applies.
         e.g., -F sw:comm,tid,time,ip,sym  and -F trace:time,cpu,trace

On Fri, Nov 17, 2017 at 01:43:00PM -0800, Andi Kleen wrote:

SNIP

Quoted 23 lines

---
 tools/perf/Documentation/perf-script.txt | 10 +++-
 tools/perf/builtin-script.c              | 97 +++++++++++++++++++++++++++++++-
 tools/perf/util/metricgroup.c            |  4 ++
 3 files changed, 108 insertions(+), 3 deletions(-)

tools/perf/Documentation/perf-script.txt +1 -1

index 2811fcf684cb..974ceb12c7f3 100644
@@ -117,7 +117,7 @@ OPTIONS
         Comma separated list of fields to print. Options are:
         comm, tid, pid, time, cpu, event, trace, ip, sym, dso, addr, symoff,
         srcline, period, iregs, uregs, brstack, brstacksym, flags, bpf-output, brstackinsn,
-        brstackoff, callindent, insn, insnlen, synth, phys_addr.
+	brstackoff, callindent, insn, insnlen, synth, phys_addr, metric.
         Field list can be prepended with the type, trace, sw or hw,
         to indicate to which event type the field list applies.
         e.g., -F sw:comm,tid,time,ip,sym  and -F trace:time,cpu,trace

@@ -217,6 +217,14 @@ OPTIONS
 
 	The brstackoff field will print an offset into a specific dso/binary.
 

tools/perf/Documentation/perf-script.txt OPTIONS · line 226

Quoted 7 lines

tools/perf/Documentation/perf-script.txt +7 -0

+	With the metric option perf script can compute metrics for
+	sampling periods, similar to perf stat. This requires
+	specifying a group with multiple metrics with the :S option
+	for perf record. perf will sample on the first event, and
+	compute metrics for all the events in the group. Please note
+	that the metric computed is averaged over the whole sampling
+	period, not just for the sample point.

hum, is it? I see you call perf_stat__reset_shadow_stats for every group start.. which I'd think means you see metric for current leader sample point

Quoted 27 lines · Jiri Olsa

On Fri, Nov 17, 2017 at 01:43:00PM -0800, Andi Kleen wrote:

SNIP

Quoted 23 lines

---
 tools/perf/Documentation/perf-script.txt | 10 +++-
 tools/perf/builtin-script.c              | 97 +++++++++++++++++++++++++++++++-
 tools/perf/util/metricgroup.c            |  4 ++
 3 files changed, 108 insertions(+), 3 deletions(-)

tools/perf/Documentation/perf-script.txt +1 -1

index 2811fcf684cb..974ceb12c7f3 100644
@@ -117,7 +117,7 @@ OPTIONS
         Comma separated list of fields to print. Options are:
         comm, tid, pid, time, cpu, event, trace, ip, sym, dso, addr, symoff,
         srcline, period, iregs, uregs, brstack, brstacksym, flags, bpf-output, brstackinsn,
-        brstackoff, callindent, insn, insnlen, synth, phys_addr.
+	brstackoff, callindent, insn, insnlen, synth, phys_addr, metric.
         Field list can be prepended with the type, trace, sw or hw,
         to indicate to which event type the field list applies.
         e.g., -F sw:comm,tid,time,ip,sym  and -F trace:time,cpu,trace

@@ -217,6 +217,14 @@ OPTIONS
 
 	The brstackoff field will print an offset into a specific dso/binary.
 

Quoted 11 lines

tools/perf/Documentation/perf-script.txt OPTIONS · line 226

Quoted 7 lines

tools/perf/Documentation/perf-script.txt +7 -0

+	With the metric option perf script can compute metrics for
+	sampling periods, similar to perf stat. This requires
+	specifying a group with multiple metrics with the :S option
+	for perf record. perf will sample on the first event, and
+	compute metrics for all the events in the group. Please note
+	that the metric computed is averaged over the whole sampling
+	period, not just for the sample point.

hum, is it? I see you call perf_stat__reset_shadow_stats for every group start.. which I'd think means you see metric for current leader sample point

Yes it is.

It's for the complete sampling period because it is computed over the delta from the last sample to the previous sample.

There isn't really a metric at a point, it is always over a interval.

Quoted 6 lines

Yes it is.

It's for the complete sampling period because it is computed over the delta from the last sample to the previous sample.

There isn't really a metric at a point, it is always over a interval.

agreed, it's the count we meassured from the last sample.. but the 'averaged' word above implies to me we compute some average over the 'sampling' period, which we dont do

Quoted 7 lines

Quoted 6 lines

Yes it is.

It's for the complete sampling period because it is computed over the delta from the last sample to the previous sample.

There isn't really a metric at a point, it is always over a interval.

Quoted 3 lines

agreed, it's the count we meassured from the last sample.. but the 'averaged' word above implies to me we compute some average over the 'sampling' period, which we dont do

Do you have a better word in mind?

AFAIK average is the right word for this because it's the summary for that time period.

Quoted 7 lines · Andi Kleen

Quoted 7 lines

Quoted 6 lines

Yes it is.

It's for the complete sampling period because it is computed over the delta from the last sample to the previous sample.

There isn't really a metric at a point, it is always over a interval.

Quoted 8 lines

Quoted 3 lines

agreed, it's the count we meassured from the last sample.. but the 'averaged' word above implies to me we compute some average over the 'sampling' period, which we dont do

Do you have a better word in mind?

AFAIK average is the right word for this because it's the summary for that time period.

the way I understand it is that we take the values from the current sample and count the metric value.. so the phrase:

.. the metric computed is averaged over the whole sampling period, not just for the sample point ...

does not make sense to me.. because we take the value of that single 'sample point'.. I dont see any average sum in there

Quoted 17 lines · Jiri Olsa

Quoted 7 lines · Andi Kleen

Quoted 7 lines

Quoted 6 lines

Yes it is.

It's for the complete sampling period because it is computed over the delta from the last sample to the previous sample.

There isn't really a metric at a point, it is always over a interval.

Quoted 8 lines

Quoted 3 lines

agreed, it's the count we meassured from the last sample.. but the 'averaged' word above implies to me we compute some average over the 'sampling' period, which we dont do

Do you have a better word in mind?

AFAIK average is the right word for this because it's the summary for that time period.

Quoted 8 lines

the way I understand it is that we take the values from the current sample and count the metric value.. so the phrase:

.. the metric computed is averaged over the whole sampling period, not just for the sample point ...

does not make sense to me.. because we take the value of that single 'sample point'.. I dont see any average sum in there

The current samples contains the sum of event counts for a sampling period.

EventA-1 EventA-2 EventA-3 EventA-4
EventB-1 EventB-2 EventC-3

gap with no events overflow

|-----------------------------------------------------------------|
period-start                                             period-end
^                                                                 ^
|                                                                 |
previous sample                                      current sample

So EventA = 4 and EventB = 3 at the sample point

I generate a metric, let's say EventA / EventB. It applies to the whole period.

But the metric is over a longer time which does not have the same behavior. For example the gap above doesn't have any events, while they are clustered at the beginning and end of the sample period.

But we're summing everything together. The metric doesn't know that the gap is different than the busy period.

That's what I'm trying to express with averaging.

TopDown:

Note TopDown requires disabling SMT if you have it enabled (e.g. by offlining the extra CPUs), because SMT would require sampling per core, which is not supported.

$ perf record -e '{ref-cycles,topdown-fetch-bubbles,topdown-recovery-bubbles,\
topdown-slots-retired,topdown-total-slots,topdown-slots-issued}:S' -a sleep 1
$ perf script --header -I -F cpu,ip,sym,event,metric,period
...
[000]     121108               ref-cycles:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     190350    topdown-fetch-bubbles:  ffffffff8165222e copy_user_enhanced_fast_string
[000]       2055 topdown-recovery-bubbles:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     148729    topdown-slots-retired:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     144324      topdown-total-slots:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     160852     topdown-slots-issued:  ffffffff8165222e copy_user_enhanced_fast_string
[000]   metric:     33.0% frontend bound
[000]   metric:      3.5% bad speculation
[000]   metric:     25.8% retiring
[000]   metric:     37.7% backend bound
[000]     112112               ref-cycles:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     357222    topdown-fetch-bubbles:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]       3325 topdown-recovery-bubbles:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     323553    topdown-slots-retired:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     270507      topdown-total-slots:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     341226     topdown-slots-issued:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]   metric:     33.0% frontend bound
[000]   metric:      2.9% bad speculation
[000]   metric:     29.9% retiring
[000]   metric:     34.2% backend bound

Git tree:
git://git.kernel.org/pub/scm/limux/kernel/git/ak/linux-misc.git perf/script-metric-3

v1: Initial post
v2:
Remove already merged patches.
Use evsel->priv for new fields
Port to new base line, support fp output.
Handle stats in ->stats, not ->priv
Minor cleanups
v3:
Enable EVENT_UPDATE in perf record, and record unit/scale/cpu map/thread map Drop the previous zero cpu map hack.

SNIP

Quoted 35 lines

TopDown:

Note TopDown requires disabling SMT if you have it enabled (e.g. by offlining the extra CPUs), because SMT would require sampling per core, which is not supported.

$ perf record -e '{ref-cycles,topdown-fetch-bubbles,topdown-recovery-bubbles,\
topdown-slots-retired,topdown-total-slots,topdown-slots-issued}:S' -a sleep 1
$ perf script --header -I -F cpu,ip,sym,event,metric,period
...
[000]     121108               ref-cycles:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     190350    topdown-fetch-bubbles:  ffffffff8165222e copy_user_enhanced_fast_string
[000]       2055 topdown-recovery-bubbles:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     148729    topdown-slots-retired:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     144324      topdown-total-slots:  ffffffff8165222e copy_user_enhanced_fast_string
[000]     160852     topdown-slots-issued:  ffffffff8165222e copy_user_enhanced_fast_string
[000]   metric:     33.0% frontend bound
[000]   metric:      3.5% bad speculation
[000]   metric:     25.8% retiring
[000]   metric:     37.7% backend bound
[000]     112112               ref-cycles:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     357222    topdown-fetch-bubbles:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]       3325 topdown-recovery-bubbles:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     323553    topdown-slots-retired:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     270507      topdown-total-slots:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]     341226     topdown-slots-issued:  ffffffff8165aec8 _raw_spin_lock_irqsave
[000]   metric:     33.0% frontend bound
[000]   metric:      2.9% bad speculation
[000]   metric:     29.9% retiring
[000]   metric:     34.2% backend bound

Git tree:
git://git.kernel.org/pub/scm/limux/kernel/git/ak/linux-misc.git perf/script-metric-3

Quoted 10 lines

v1: Initial post
v2:
Remove already merged patches.
Use evsel->priv for new fields
Port to new base line, support fp output.
Handle stats in ->stats, not ->priv
Minor cleanups
v3:
Enable EVENT_UPDATE in perf record, and record unit/scale/cpu map/thread map Drop the previous zero cpu map hack.

Acked-by Jiri Olsa <[email protected]>