perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output
PATCH v3 1/1 perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output
When running perf stat with -A (--no-aggr) and --metric-only in JSON output mode (-j), perf stat evaluates metric expressions across all matching PMUs (including uncore PMUs like uncore_iio_0, uncore_iio_1, etc.).
However, perf stat previously formatted JSON output by printing only "cpu" : "<id>" and grouping all metric values on a single line per CPU without identifying which PMU instance evaluated each metric. As a result, when an uncore event spans multiple PMU boxes, perf stat printed repeated, ambiguous metric keys without PMU names.
Fix this by:
1. Including "pmu" : "<pmu_name>" in print_aggr_id_json when evsel->pmu is a non-core or hybrid PMU in AGGR_NONE mode (-A).
2. Starting a new JSON metric line in AGGR_NONE mode (-A) whenever the underlying PMU instance changes across PMU events.
3. Updating perf_json_output_lint.py to recognize the new "pmu" key in the JSON test suite.
Before the fix:
$ perf stat -M iio_bandwidth_read -a -A --metric-only -j -I 1000
{"interval" : 1.001017947, "cpu" : "0", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "22.5", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "0.2", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "0.0", "MB/s iio_bandwidth_read" : "0.0"}There is no way to determine which uncore device generated the metrics.
After:
$ perf stat -M iio_bandwidth_read -a -A --metric-only -j -I 1000
{"interval" : 1.000314908, "cpu" : "0", "pmu" : "uncore_iio_0", "MB/s iio_bandwidth_read" : "0.0"}
{"interval" : 1.000314908, "cpu" : "0", "pmu" : "uncore_iio_1", "MB/s iio_bandwidth_read" : "0.1"}
{"interval" : 1.000314908, "cpu" : "0", "pmu" : "uncore_iio_11", "MB/s iio_bandwidth_read" : "0.0"}
...
{"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_0", "MB/s iio_bandwidth_read" : "0.0"}
{"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_1", "MB/s iio_bandwidth_read" : "0.0"}
{"interval" : 1.000314908, "cpu" : "56", "pmu" : "uncore_iio_11", "MB/s iio_bandwidth_read" : "0.0"}---
v3:
- Add before the fix output.
v2: lore.kernel.org/[email protected]
- Fix print_metric_begin() initialization in print_no_aggr_metric() to ensure tool events always open lines cleanly.
- Simplify and fix PMU line-split transition tracking when switching between different PMU instances or transitioning from tool events.
- Add bounds check on aggr_map index in print_no_aggr_metric() to prevent out-of-bounds array reads if sysfs CPU topology is incomplete.
v1: lore.kernel.org/[email protected]
.../tests/shell/lib/perf_json_output_lint.py | 1 + tools/perf/util/stat-display.c | 48 ++++++++++++++----- 2 files changed, 37 insertions(+), 12 deletions(-)
tools/perf/tests/shell/lib/perf_json_output_lint.py +1 -0
@@ -64,6 +64,7 @@ def check_json_output(expected_items): 'metric-threshold': lambda x: x in ['unknown', 'good', 'less good', 'nearly bad', 'bad'], 'metricgroup': lambda x: True, 'node': lambda x: True, + 'pmu': lambda x: True, 'pcnt-running': lambda x: isfloat(x), 'socket': lambda x: True, 'thread': lambda x: True,
tools/perf/util/stat-display.c +36 -12
115 lines not on this page · raw message
--
2.55.0.508.g3f0d502094-goog
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Flawed line-split transition logic causes spurious empty JSON objects and incorrect PMU…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Flawed line-split transition logic causes spurious empty JSON objects and incorrect PMU attribution for tool events. --
--- Patch [1]: [PATCH v3] perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output --- commit 20e3cd08fd5e415b458343262ee76bc1f5f6ce85
Author: Chun-Tse Shao <[email protected]>
perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output
This commit modifies perf stat JSON output in --metric-only and -A mode to group metrics by PMU. It achieves this by adding a "pmu" key to the JSON output and starting a new JSON line when the underlying PMU instance changes.
Quoted 4 lines
tools/perf/util/stat-display.c +0
-0
[ ... ]
Quoted 9 lines
tools/perf/util/stat-display.c +2
-0@@ -1146,13 +1149,34 @@ static void print_no_aggr_metric(struct perf_stat_config *config,if (config->aggr_map->map[aggr_idx].cpu.cpu == cpu.cpu)break ; } +if (aggr_idx >= config->aggr_map->nr) +continue ; os->evsel = counter; os->id =aggr_cpu_id__cpu (cpu,/*data=*/ NULL );
tools/perf/util/stat-display.c static void print_no_aggr_metric(struct perf_stat_config *config, · line 1163
Quoted 10 lines
tools/perf/util/stat-display.c +7
-3-if (first) { -print_metric_begin (config, evlist, os, aggr_idx); - first = false; + +if (config->metric_only) { +struct perf_pmu *pmu = counter->pmu; + +if (!evsel__is_tool (counter)) { +if (config->json_output && line_open && + (!last_pmu || pmu != last_pmu)) {
[Severity: Medium]
Does this line-split transition logic handle tool events correctly depending on their order in the event list?
If a tool event is processed first, line_open becomes true, but last_pmu remains NULL. When the next event (a PMU event) is processed, the condition (!last_pmu || pmu != last_pmu) evaluates to true. This prematurely closes the line and creates a spurious empty JSON object for the tool event.
Conversely, if a tool event comes after a PMU event, the else-if block below is taken because !evsel__is_tool(counter) evaluates to false. This bypasses the line-splitting check entirely, causing the tool event to silently merge into the preceding PMU's JSON object and incorrectly inherit its PMU attribution.
Quoted 15 lines
tools/perf/util/stat-display.c +14
-0+print_metric_end (config, os); + line_open = false; + } +if (!line_open) { +print_metric_begin (config, evlist, os, + aggr_idx, counter); + line_open = true; + } + last_pmu = pmu; + }else if (!line_open) { +print_metric_begin (config, evlist, os, + aggr_idx, counter); + line_open = true; + } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Namhyung Kim
maintainerQuoted 45 lines · On Wed, Jul 29, 2026 at 05:17:41PM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Flawed line-split transition logic causes spurious empty JSON objects and incorrect PMU attribution for tool events. ----- Patch [1]: [PATCH v3] perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output --- commit
20e3cd08fd5e415b458343262ee76bc1f5f6ce85
Author: Chun-Tse Shao <[email protected]>perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output
This commit modifies perf stat JSON output in --metric-only and -A mode to group metrics by PMU. It achieves this by adding a "pmu" key to the JSON output and starting a new JSON line when the underlying PMU instance changes.
Quoted 4 lines
tools/perf/util/stat-display.c +0
-0[ ... ]
Quoted 9 lines
tools/perf/util/stat-display.c +2
-0@@ -1146,13 +1149,34 @@ static void print_no_aggr_metric(struct perf_stat_config *config,if (config->aggr_map->map[aggr_idx].cpu.cpu == cpu.cpu)break ; } +if (aggr_idx >= config->aggr_map->nr) +continue ; os->evsel = counter; os->id =aggr_cpu_id__cpu (cpu,/*data=*/ NULL );tools/perf/util/stat-display.c static void print_no_aggr_metric(struct perf_stat_config *config, · line 1163
Quoted 10 lines
tools/perf/util/stat-display.c +7
-3-if (first) { -print_metric_begin (config, evlist, os, aggr_idx); - first = false; + +if (config->metric_only) { +struct perf_pmu *pmu = counter->pmu; + +if (!evsel__is_tool (counter)) { +if (config->json_output && line_open && + (!last_pmu || pmu != last_pmu)) {[Severity: Medium]
Does this line-split transition logic handle tool events correctly depending on their order in the event list?
Quoted 4 lines
If a tool event is processed first,
line_openbecomes true, butlast_pmuremains NULL. When the next event (a PMU event) is processed, the condition (!last_pmu || pmu !=last_pmu) evaluates to true. This prematurely closes the line and creates a spurious empty JSON object for the tool event.
Something like this?
$ sudo ./perf stat -aA -e user_time -M insn_per_cycle -j --metric-only true
{"cpu" : "0", "pmu" : "tool"}
{"cpu" : "0", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.8"}
{"cpu" : "1", "pmu" : "tool"}
{"cpu" : "1", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.1"}
{"cpu" : "2", "pmu" : "tool"}
{"cpu" : "2", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.5"}
{"cpu" : "3", "pmu" : "tool"}
{"cpu" : "3", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.3"}
...Thanks,
Namhyung
Quoted 25 lines
Conversely, if a tool event comes after a PMU event, the else-if block below is taken because !evsel__is_tool(counter) evaluates to false. This bypasses the line-splitting check entirely, causing the tool event to silently merge into the preceding PMU's JSON object and incorrectly inherit its PMU attribution.
Quoted 15 lines
tools/perf/util/stat-display.c +14
-0+print_metric_end (config, os); + line_open = false; + } +if (!line_open) { +print_metric_begin (config, evlist, os, + aggr_idx, counter); + line_open = true; + } + last_pmu = pmu; + }else if (!line_open) { +print_metric_begin (config, evlist, os, + aggr_idx, counter); + line_open = true; + } }Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 65 lines · Namhyung Kim
Quoted 45 lines · On Wed, Jul 29, 2026 at 05:17:41PM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Flawed line-split transition logic causes spurious empty JSON objects and incorrect PMU attribution for tool events. ----- Patch [1]: [PATCH v3] perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output --- commit
20e3cd08fd5e415b458343262ee76bc1f5f6ce85
Author: Chun-Tse Shao <[email protected]>perf stat: Include PMU name and split uncore events per PMU in metric-only JSON output
This commit modifies perf stat JSON output in --metric-only and -A mode to group metrics by PMU. It achieves this by adding a "pmu" key to the JSON output and starting a new JSON line when the underlying PMU instance changes.
Quoted 4 lines
tools/perf/util/stat-display.c +0
-0[ ... ]
Quoted 9 lines
tools/perf/util/stat-display.c +2
-0@@ -1146,13 +1149,34 @@ static void print_no_aggr_metric(struct perf_stat_config *config,if (config->aggr_map->map[aggr_idx].cpu.cpu == cpu.cpu)break ; } +if (aggr_idx >= config->aggr_map->nr) +continue ; os->evsel = counter; os->id =aggr_cpu_id__cpu (cpu,/*data=*/ NULL );tools/perf/util/stat-display.c static void print_no_aggr_metric(struct perf_stat_config *config, · line 1163
Quoted 10 lines
tools/perf/util/stat-display.c +7
-3-if (first) { -print_metric_begin (config, evlist, os, aggr_idx); - first = false; + +if (config->metric_only) { +struct perf_pmu *pmu = counter->pmu; + +if (!evsel__is_tool (counter)) { +if (config->json_output && line_open && + (!last_pmu || pmu != last_pmu)) {[Severity: Medium]
Does this line-split transition logic handle tool events correctly depending on their order in the event list?Quoted 4 lines
If a tool event is processed first,
line_openbecomes true, butlast_pmuremains NULL. When the next event (a PMU event) is processed, the condition (!last_pmu || pmu !=last_pmu) evaluates to true. This prematurely closes the line and creates a spurious empty JSON object for the tool event.Something like this?
$ sudo ./perf stat -aA -e user_time -M insn_per_cycle -j --metric-only true {"cpu" : "0", "pmu" : "tool"} {"cpu" : "0", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.8"} {"cpu" : "1", "pmu" : "tool"} {"cpu" : "1", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.1"} {"cpu" : "2", "pmu" : "tool"} {"cpu" : "2", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.5"} {"cpu" : "3", "pmu" : "tool"} {"cpu" : "3", "pmu" : "cpu_core", "instructions insn_per_cycle" : "0.3"} ...
Quoted 2 lines
Thanks,
Namhyung
Thanks for the test, I sent the v4 patch for fixing this problem:
lore.kernel.org/[email protected]
Test:
$ perf stat -aA -e user_time -M insn_per_cycle -j --metric-only true
{"cpu" : "0", "instructions insn_per_cycle" : "0.9"}
{"cpu" : "1", "instructions insn_per_cycle" : "0.6"}
{"cpu" : "2", "instructions insn_per_cycle" : "0.2"}
{"cpu" : "3", "instructions insn_per_cycle" : "0.1"}
{"cpu" : "4", "instructions insn_per_cycle" : "0.2"}
{"cpu" : "5", "instructions insn_per_cycle" : "0.1"}
{"cpu" : "6", "instructions insn_per_cycle" : "0.1"}
{"cpu" : "7", "instructions insn_per_cycle" : "0.1"}
{"cpu" : "8", "instructions insn_per_cycle" : "0.1"}
{"cpu" : "9", "instructions insn_per_cycle" : "0.1"}
...-CT
Quoted 26 lines
Quoted 26 lines
Conversely, if a tool event comes after a PMU event, the else-if block below is taken because !evsel__is_tool(counter) evaluates to false. This bypasses the line-splitting check entirely, causing the tool event to silently merge into the preceding PMU's JSON object and incorrectly inherit its PMU attribution.
Quoted 5 lines
tools/perf/util/stat-display.c +5
-0+print_metric_end (config, os); + line_open = false; + } +if (!line_open) { + print_metric_begin(config, evlist, os,tools/perf/util/stat-display.c static void print_no_aggr_metric(struct perf_stat_config *config, · line 1178
Quoted 10 lines
tools/perf/util/stat-display.c +9
-0+ aggr_idx, counter); + line_open = true; + } + last_pmu = pmu; + }else if (!line_open) { +print_metric_begin (config, evlist, os, + aggr_idx, counter); + line_open = true; + } }--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1