Skip to contentSkip to conversation

perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output

PATCH Superseded Chun-Tse Shao 4 participants 5 messages Oct 30 – Nov 11 perf events

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

PATCH v7 1/1 perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output

When running perf stat with -A (--no-aggr) and --metric-only in JSON output mode (-j), perf stat evaluates metric expressions across all matching PMUs (including uncore PMUs like uncore_iio_0, uncore_iio_1, etc.).

However, perf stat previously formatted JSON output by printing only "cpu" : "<id>" and grouping all metric values on a single line per CPU without identifying which PMU instance evaluated each metric. As a result, when an uncore event spans multiple PMU boxes, perf stat printed repeated, ambiguous metric keys without PMU names.

Fix this by:
1. Including "pmu" : "<pmu_name>" in print_aggr_id_json when evsel->pmu is a non-core or hybrid PMU in AGGR_NONE mode (-A). Tool events are not labeled as they don't belong to a hardware PMU instance.
2. Starting a new JSON metric line in AGGR_NONE mode (-A) whenever the PMU name changes, so that a line only has metrics from the PMU it is labeled with.
3. Opening a metric line only for events that have a metric, as other events print nothing in metric-only mode. This avoids spurious lines like {"cpu" : "0", "pmu" : "tool"} for events like user_time, while metrics on tool events, like tma_info_system_time (duration_time), are still printed with the line prefix.
4. Updating perf_json_output_lint.py to recognize the new "pmu" key in the JSON test suite.

Before the fix:

  $ perf stat -M iio_bandwidth_read -a -A --metric-only -j -I 1000
  {"interval" : 1.001017947, "cpu" : "0", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "22.5", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "0.2", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "0.0", "MB/s  iio_bandwidth_read" : "0.0"}

There is no way to determine which uncore device generated the metrics.

After:

  $ perf stat -M iio_bandwidth_read -a -A --metric-only -j -I 1000
  {"interval" : 1.000314908, "cpu" : "0", "pmu" : "uncore_iio_0", "MB/s  iio_bandwidth_read" : "0.0"}
  {"interval" : 1.000314908, "cpu" : "0", "pmu" : "uncore_iio_1", "MB/s  iio_bandwidth_read" : "0.1"}
  {"interval" : 1.000314908, "cpu" : "0", "pmu" : "uncore_iio_11", "MB/s  iio_bandwidth_read" : "0.0"}
  ...
  {"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_0", "MB/s  iio_bandwidth_read" : "0.0"}
  {"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_1", "MB/s  iio_bandwidth_read" : "0.0"}
  {"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_11", "MB/s  iio_bandwidth_read" : "0.0"}
Assisted-by Gemini:gemini-3.1-pro-preview

---
v7:
- Pass the evsel to print_metric_begin() explicitly from all callers instead of falling back to evlist__first(). (Ian)
v6: lore.kernel.org/[email protected]
- Open metric lines for events that have a metric instead of skipping tool events, so metrics on tool events (e.g. tma_info_system_time) keep the line prefix and are terminated properly. (Sashiko)
- Split JSON lines only when the shown PMU name changes. Tool event metrics are no longer mislabeled with an uncore PMU name, and the STD and CSV output are no longer split.
- Rebase onto the latest perf-tools-next.
v5: lore.kernel.org/[email protected]
- Fix perf test "perf stat JSON output linter" failure.
v4: lore.kernel.org/[email protected]
- Fix the line splitting issue, causing spurious empty {"pmu": "tool"} objects and order-dependent PMU mislabeling.
v3: lore.kernel.org/[email protected]
- Add before the fix output.
v2: lore.kernel.org/[email protected]
- Fix print_metric_begin() initialization in print_no_aggr_metric() to ensure tool events always open lines cleanly.

 .../tests/shell/lib/perf_json_output_lint.py  |  5 +-
 tools/perf/util/stat-display.c                | 76 ++++++++++++++++---
 2 files changed, 68 insertions(+), 13 deletions(-)
tools/perf/tests/shell/lib/perf_json_output_lint.py +4 -1
index 8cde22ac9d97..982be893bd69 100644
@@ -65,6 +65,7 @@ def check_json_output(expected_count):
       'metricgroup': lambda x: True,
       'node': lambda x: True,
       'pcnt-running': isfloat,
+      'pmu': lambda x: True,
       'socket': lambda x: True,
       'thread': lambda x: True,
       'unit': lambda x: True,
@@ -99,8 +100,10 @@ def check_json_output(expected_count):
 try:
   if args.no_args or args.system_wide or args.event:
     expected_items = [5, 7]
-  elif args.interval or args.per_thread or args.system_wide_no_aggr:
+  elif args.interval or args.per_thread:
     expected_items = [6, 8]
+  elif args.system_wide_no_aggr:
+    expected_items = [6, 7, 8, 9]
   elif args.per_core or args.per_socket or args.per_node or args.per_die or args.per_cluster or args.per_cache:
     expected_items = [7, 9]
   elif args.metric_only:
tools/perf/util/stat-display.c +64 -12

165 lines not on this page · raw message

Signature · 1 line
2.56.0.317.g454f8302e1-goog
#1 lore raw
Sashiko has reviewed this patch and found no issues.
Quoted 19 lines

When running perf stat with -A (--no-aggr) and --metric-only in JSON output mode (-j), perf stat evaluates metric expressions across all matching PMUs (including uncore PMUs like uncore_iio_0, uncore_iio_1, etc.).

However, perf stat previously formatted JSON output by printing only "cpu" : "<id>" and grouping all metric values on a single line per CPU without identifying which PMU instance evaluated each metric. As a result, when an uncore event spans multiple PMU boxes, perf stat printed repeated, ambiguous metric keys without PMU names.

Fix this by:
1. Including "pmu" : "<pmu_name>" in print_aggr_id_json when evsel->pmu
is a non-core or hybrid PMU in AGGR_NONE mode (-A). Tool events are
not labeled as they don't belong to a hardware PMU instance.
[ ... ]
{"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_1", "MB/s iio_bandwidth_read" : "0.0"} {"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_11", "MB/s iio_bandwidth_read" : "0.0"}

Quoted 2 lines
Assisted-by Gemini:gemini-3.1-pro-preview

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Tested-by on v7 1/1
#2 lore raw
In reply to Chun-Tse Shao
Quoted 2 lines · Chun-Tse Shao

When running perf stat with -A (--no-aggr) and --metric-only in JSON output mode (-j), [...]

Tested on AMD Zen 3 on top of perf-tools-next 13a8d580433a. With '-M l3_cache,data_fabric,iotlb -a -A --metric-only -j' the metrics are now on separate lines labeled amd_l3, amd_df and amd_iommu_0. STD, CSV and aggregated JSON output are unchanged, and the stat linters pass.

Two changes the changelog doesn't mention:

- Without -M, 'perf stat -a -A --metric-only -j' now prints two lines per CPU; the first is labeled "pmu" : "software".

- Plain 'perf stat -a -A -j' gets the "pmu" key too.

Are those intended?

(Tested with the help of an AI assistant; I checked the results.)

maintainer
#3 lore raw
Quoted 11 lines · Hang Zhengyang
Quoted 2 lines · Chun-Tse Shao

When running perf stat with -A (--no-aggr) and --metric-only in JSON output mode (-j), [...]

Tested on AMD Zen 3 on top of perf-tools-next 13a8d580433a. With '-M l3_cache,data_fabric,iotlb -a -A --metric-only -j' the metrics are now on separate lines labeled amd_l3, amd_df and amd_iommu_0. STD, CSV and aggregated JSON output are unchanged, and the stat linters pass.

Quoted 8 lines

Two changes the changelog doesn't mention:

- Without -M, 'perf stat -a -A --metric-only -j' now prints two lines per CPU; the first is labeled "pmu" : "software".

- Plain 'perf stat -a -A -j' gets the "pmu" key too.

Are those intended?

I believe it's intended and looks fine.

Ian, are you ok with this?

Thanks,
Na

#4 lore raw
Quoted 11 lines · Namhyung Kim
Quoted 9 lines · Hang Zhengyang
Quoted 2 lines · Chun-Tse Shao

When running perf stat with -A (--no-aggr) and --metric-only in JSON output mode (-j), [...]

Tested on AMD Zen 3 on top of perf-tools-next 13a8d580433a. With '-M l3_cache,data_fabric,iotlb -a -A --metric-only -j' the metrics are now on separate lines labeled amd_l3, amd_df and amd_iommu_0. STD, CSV and aggregated JSON output are unchanged, and the stat linters pass.

Quoted 8 lines
Quoted 8 lines

Two changes the changelog doesn't mention:

- Without -M, 'perf stat -a -A --metric-only -j' now prints two lines per CPU; the first is labeled "pmu" : "software".

- Plain 'perf stat -a -A -j' gets the "pmu" key too.

Thank you for the testing. Yes it needs to be fixed:

```
  $perf stat -a -A -M branch_miss_rate,cs_per_second --metric-only -j -- sleep 1
    {"cpu" : "0", "pmu" : "software", "cs/sec  cs_per_second" : "1211.5"}
    {"cpu" : "0", "%  branch_miss_rate" : "1.3"}
   ...
```

The fix should skip printing software pmu like tool and cpu. Also, PMU printing should be skipped in non-metric-only mode.

v8: lore.kernel.org/[email protected]

-CT

Quoted 9 lines
Quoted 2 lines

Are those intended?

I believe it's intended and looks fine.

Ian, are you ok with this?

Thanks,
Na