Skip to content

Ian Rogers linux-perf-users lkml 4 versions – Applied

Status from this message: Applied to perf-tools-next, thanks!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest18 patches 14 repliesApplied
  2. v318 patches 11 repliesSuperseded
  3. v218 patches 2 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v122 patches13 repliesSuperseded

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/18perf metricgroup: Add care to picking the evsel for displaying a metrictools/perf/util/metricgroup.c+47 -10 replies
2/18perf expr: Add #target_cpu literaltools/perf/util/expr.c+3 -00 replies
3/18perf jevents: Add set of common metrics based on default onestools/perf/pmu-events/arch/common/common/metrics.json tools/perf/pmu-events/empty-pmu-events.c tools/perf/pmu-events/jevents.py +2 more+212 -428 replies
4/18perf jevents: Add metric DefaultShowEventstools/perf/builtin-stat.c tools/perf/pmu-events/arch/common/common/metrics.json tools/perf/pmu-events/empty-pmu-events.c +6 more+102 -1070 replies
5/18perf stat: Add detail -d,-dd,-ddd metricstools/perf/builtin-stat.c tools/perf/pmu-events/arch/common/common/metrics.json tools/perf/pmu-events/empty-pmu-events.c+113 -1130 replies
6/18perf script: Change metric format to use json metricstools/perf/builtin-script.c+230 -220 replies
7/18perf stat: Remove hard coded shadow metricstools/perf/builtin-script.c tools/perf/util/stat-display.c tools/perf/util/stat-shadow.c +1 more+6 -3930 replies
8/18perf stat: Fix default metricgroup display on hybridtools/perf/util/stat-shadow.c+3 -50 replies
9/18perf stat: Sort default events/metricstools/perf/builtin-stat.c+32 -00 replies
10/18perf stat: Remove "unit" workarounds for metric-onlytools/perf/util/stat-display.c+8 -390 replies
11/18perf test stat+json: Improve metric-only testingtools/perf/tests/shell/lib/perf_json_output_lint.py tools/perf/tests/shell/stat+json_output.sh+3 -30 replies
12/18perf test stat: Ignore failures in Default[234] metricgroupstools/perf/tests/shell/stat_all_metricgroups.sh+3 -00 replies
13/18perf test stat: Update std_output testing metric expectationstools/perf/tests/shell/stat+std_output.sh+2 -20 replies
14/18perf test metrics: Update all metrics for possibly failing default metricstools/perf/tests/shell/stat_all_metrics.sh+6 -10 replies
15/18perf test stat: Update shadow test to use metricstools/perf/tests/shell/stat+shadow_stat.sh+2 -20 replies
16/18perf test stat: Update test expectations and eventstools/perf/tests/shell/stat.sh+3 -30 replies
17/18perf test stat csv: Update test expectations and eventstools/perf/tests/shell/lib/stat_output.sh tools/perf/tests/shell/stat+csv_output.sh+2 -20 replies
18/18perf tool_pmu: Make core_wide and target_cpu json eventstools/perf/pmu-events/arch/common/common/tool.json tools/perf/pmu-events/empty-pmu-events.c tools/perf/util/expr.c +3 more+163 -1230 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v4 yet.

Read the v4 thread

Cover letter [PATCH v4 00/18]

Prior to this series stat-shadow would produce hard coded metrics if certain events appeared in the evlist. This series produces equivalent json metrics and cleans up the consequences in tests and display output. A before and after of the default display output on a tigerlake is:

Before:

```
$ perf stat -a sleep 1

 Performance counter stats for 'system wide':

    16,041,816,418      cpu-clock                        #   15.995 CPUs utilized             
             5,749      context-switches                 #  358.376 /sec                      
               121      cpu-migrations                   #    7.543 /sec                      
             1,806      page-faults                      #  112.581 /sec                      
       825,965,204      instructions                     #    0.70  insn per cycle            
     1,180,799,101      cycles                           #    0.074 GHz                       
       168,945,109      branches                         #   10.532 M/sec                     
         4,629,567      branch-misses                    #    2.74% of all branches           
 #     30.2 %  tma_backend_bound      
                                                  #      7.8 %  tma_bad_speculation    
                                                  #     47.1 %  tma_frontend_bound     
 #     14.9 %  tma_retiring           
```

After:

```
$ perf stat -a sleep 1

 Performance counter stats for 'system wide':

             2,890      context-switches                 #    179.9 cs/sec  cs_per_second     
    16,061,923,339      cpu-clock                        #     16.0 CPUs  CPUs_utilized       
                43      cpu-migrations                   #      2.7 migrations/sec  migrations_per_second
             5,645      page-faults                      #    351.5 faults/sec  page_faults_per_second
         5,708,413      branch-misses                    #      1.4 %  branch_miss_rate         (88.83%)
       429,978,120      branches                         #     26.8 M/sec  branch_frequency     (88.85%)
     1,626,915,897      cpu-cycles                       #      0.1 GHz  cycles_frequency       (88.84%)
     2,556,805,534      instructions                     #      1.5 instructions  insn_per_cycle  (88.86%)
                        TopdownL1                 #     20.1 %  tma_backend_bound      
                                                  #     40.5 %  tma_bad_speculation      (88.90%)
                                                  #     17.2 %  tma_frontend_bound       (78.05%)
                                                  #     22.2 %  tma_retiring             (88.89%)

       1.002994394 seconds time elapsed
```

Having the metrics in json brings greater uniformity, allows events to be shared by metrics, and it also allows descriptions like:

```
$ perf list cs_per_second
...
  cs_per_second
       [Context switches per CPU second]
```

A thorn in the side of doing this work was that the hard coded metrics were used by perf script with '-F metric'. This functionality didn't work for me (I was testing perf record -e instructions,cycles with/without leader sampling and then perf script -F metric but saw nothing but empty lines) but anyway I decided to fix it to the best of my ability in this series. So the script side counters were removed and the regular ones associated with the evsel used. The json metrics were all searched looking for ones that have a subset of events matching those in the perf script session, and all metrics are printed. This is kind of weird as the counters are being set by the period of samples, but I carried the behavior forward. I suspect there needs to be follow up work to make this better, but what is in the series is superior to what is currently in the tree. Follow up work could include finding metrics for the machine in the perf.data rather than using the host, allowing multiple metrics even if the metric ids of the events differ, fixing pre-existing perf stat record/report issues, etc.

There is a lot of stat tests that, for example, assume '-e instructions,cycles' will produce an IPC metric. These things needed tidying as now the metric must be explicitly asked for and when doing this ones using software events were preferred to increase compatibility. As the test updates were numerous they are distinct to the patches updating the functionality causing periods in the series where not all tests are passing. If this is undesirable the test fixes can be squashed into the functionality updates, but this will be kind of messy, especially as at some points in the series both the old metrics and the new metrics will be displayed.

v4: K/sec to M/sec on branch frequency (Namhyung), perf script -F
metric to-done a system-wide calculation (Namhyung) and don't crash because of the CPU map index couldn't be found. Regenerate commit messages but the cpu-clock was always yielding 0 on my machine leading to a lot of nan metric values.

v3: Rebase resolving merge conflicts in
tools/perf/pmu-events/empty-pmu-events.c by just regenerating it (Dapeng Mi).
https://lore.kernel.org/lkml/[email protected]/

v2: Drop merged patches, add json to document target_cpu/core_wide and
example to "Add care to picking the evsel for displaying a metric" commit message (Namhyung).
https://lore.kernel.org/lkml/[email protected]/

v1: https://lore.kernel.org/lkml/[email protected]/

Ian Rogers (18):
perf metricgroup: Add care to picking the evsel for displaying a
metric
perf expr: Add #target_cpu literal
perf jevents: Add set of common metrics based on default ones perf jevents: Add metric DefaultShowEvents
perf stat: Add detail -d,-dd,-ddd metrics
perf script: Change metric format to use json metrics perf stat: Remove hard coded shadow metrics
perf stat: Fix default metricgroup display on hybrid perf stat: Sort default events/metrics
perf stat: Remove "unit" workarounds for metric-only perf test stat+json: Improve metric-only testing
perf test stat: Ignore failures in Default[234] metricgroups perf test stat: Update std_output testing metric expectations perf test metrics: Update all metrics for possibly failing default
metrics
perf test stat: Update shadow test to use metrics
perf test stat: Update test expectations and events perf test stat csv: Update test expectations and events perf tool_pmu: Make core_wide and target_cpu json events

 tools/perf/builtin-script.c                   | 251 ++++++++++-
 tools/perf/builtin-stat.c                     | 154 ++-----
 .../arch/common/common/metrics.json           | 151 +++++++
 .../pmu-events/arch/common/common/tool.json   |  12 +
 tools/perf/pmu-events/empty-pmu-events.c      | 229 ++++++----
 tools/perf/pmu-events/jevents.py              |  28 +-
 tools/perf/pmu-events/pmu-events.h            |   2 +
 .../tests/shell/lib/perf_json_output_lint.py  |   4 +-
 tools/perf/tests/shell/lib/stat_output.sh     |   2 +-
 tools/perf/tests/shell/stat+csv_output.sh     |   2 +-
 tools/perf/tests/shell/stat+json_output.sh    |   2 +-
 tools/perf/tests/shell/stat+shadow_stat.sh    |   4 +-
 tools/perf/tests/shell/stat+std_output.sh     |   4 +-
 tools/perf/tests/shell/stat.sh                |   6 +-
 .../perf/tests/shell/stat_all_metricgroups.sh |   3 +
 tools/perf/tests/shell/stat_all_metrics.sh    |   7 +-
 tools/perf/util/evsel.h                       |   1 +
 tools/perf/util/expr.c                        |   8 +-
 tools/perf/util/metricgroup.c                 |  92 +++-
 tools/perf/util/stat-display.c                |  55 +--
 tools/perf/util/stat-shadow.c                 | 404 +-----------------
 tools/perf/util/stat.h                        |   2 +-
 tools/perf/util/tool_pmu.c                    |  24 +-
 tools/perf/util/tool_pmu.h                    |   9 +-
 24 files changed, 769 insertions(+), 687 deletions(-)
 create mode 100644 tools/perf/pmu-events/arch/common/common/metrics.json
Signature · 1 line
2.51.2.1041.gc1ab5b90ca-goog