Skip to contentSkip to conversation

perf stat: Decouple and modularize metrics/events output printing API

RFC PATCH Superseded Ian Rogers 5 participants 98 messages May 22 – Jul 17 perf events Page 2 of 2

lore
Share & tools
lore
#76 lore raw

Re: [PATCH v4 02/14] perf stat: Implement standard console (STD) formatting callbacks

In reply to Ian Rogers
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] Out-of-bounds array access and NULL pointer dereference when printing aggregation headers. - [Low]…

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Out-of-bounds array access and NULL pointer dereference when printing aggregation headers.
- [Low] Uninitialized struct fields in dynamically allocated memory.
- [Low] Redundant and inverted multiplexing scaling logic acts as dead code.
--

--- Patch [2]: [PATCH v4 02/14] perf stat: Implement standard console (STD) formatting callbacks --- commit 42fd504b1ec9248fe9557a917da5fa2928b1bb6b
Author: Ian Rogers <[email protected]>

perf stat: Implement standard console (STD) formatting callbacks

This patch implements standard console formatting callbacks inside util/stat-print-std.c, replacing the empty stubs introduced in Commit 1. Introduces the format-private queued_event and queued_metric DOM nodes to buffer traversal streams.

Quoted 4 lines
tools/perf/util/stat-print-std.c +0 -0
index a1311f950559b..d83410ae3385a 100644

[ ... ]

Quoted 6 lines
tools/perf/util/stat-print-std.c +6 -0
+static int std_print_event(void *ctx, struct perf_stat_config *config,
+			   struct evsel *evsel, int aggr_idx, u64 val, u64 ena,
+			   u64 run, double stdev_pct, const char *cgrp)
+{
+	struct std_print_state *ps = ctx;
+	struct queued_event *ev;

[ ... ]

tools/perf/util/stat-print-std.c line 239

Quoted 5 lines
tools/perf/util/stat-print-std.c +5 -0
+	ev = malloc(sizeof(*ev));
+	if (!ev)
+		return -ENOMEM;
+
+	ev->name = strdup(evsel__name(evsel));

[Severity: Low]
Are all fields in struct queued_event initialized after allocation? It looks like ev->is_metricgroup remains uninitialized here.

[ ... ]

Quoted 7 lines
tools/perf/util/stat-print-std.c +7 -0
+static int std_print_metric(void *ctx, struct perf_stat_config *config __maybe_unused,
+			    struct evsel *evsel __maybe_unused, int aggr_idx __maybe_unused,
+			    const char *name, const char *unit, double val,
+			    enum metric_threshold_classify thresh)
+{
+	struct std_print_state *ps = ctx;
+	struct queued_metric *b;

[ ... ]

tools/perf/util/stat-print-std.c line 290

Quoted 5 lines
tools/perf/util/stat-print-std.c +5 -0
+	b = malloc(sizeof(*b));
+	if (!b)
+		return -ENOMEM;
+
+	b->name = strdup(name);

[Severity: Low]
Similarly, are all fields in struct queued_metric initialized? It appears b->evsel and b->aggr_idx remain uninitialized here.

[ ... ]

Quoted 2 lines
tools/perf/util/stat-print-std.c +2 -0
+static int std_print_end(void *ctx, struct perf_stat_config *config)
+{

[ ... ]

tools/perf/util/stat-print-std.c line 472

Quoted 8 lines
tools/perf/util/stat-print-std.c +8 -0
+			if (ev->run == 0 || ev->ena == 0) {
+				fprintf(out, "%*s ", COUNTS_LEN, bad_count);
+			} else {
+				double scaled = (double)ev->val;
+				double avg;
+
+				if (ev->ena < ev->run)
+					scaled = (double)ev->val * ev->run / ev->ena;

[Severity: Low]
Is this scaling logic correct?

It looks like the condition ev->ena < ev->run is inverted, as an event cannot physically run longer than it was enabled, making this dead code.

Additionally, if the condition were corrected, wouldn't this double-scale the value, as ev->val is already scaled by perf_counts_values__scale() during counter processing?

[ ... ]

Quoted 2 lines
tools/perf/util/stat-print-std.c +2 -0
+static int std_metric_only_print_end(void *ctx, struct perf_stat_config *config)
+{

[ ... ]

tools/perf/util/stat-print-std.c line 679

Quoted 3 lines
tools/perf/util/stat-print-std.c +3 -0
+		if (config->aggr_map && first_aggr >= 0 &&
+		    config->aggr_mode != AGGR_GLOBAL) {
+			int len = aggr_header_lens[config->aggr_mode];

[Severity: High]
Can a corrupted or maliciously crafted perf.data file provide an arbitrary config->aggr_mode here?

Since perf_event__read_stat_config() directly assigns the value to stat_config.aggr_mode without validation, indexing aggr_header_lens with it could lead to an out-of-bounds read or NULL pointer dereference when processing untrusted input.

[ ... ]

Quoted 4 lines
tools/perf/util/stat-print-std.c +4 -0
+int perf_stat__print_std(struct evlist *evlist, struct perf_stat_config *config,
+			 const struct target *target, const struct timespec *ts,
+			 int argc, const char **argv)
+{

[ ... ]

Quoted 7 lines
tools/perf/util/stat-print-std.c +7 -0
+	} else {
+		if (config->interval && !config->headers_printed) {
+			FILE *output = config->output;
+
+			if (config->aggr_mode == AGGR_GLOBAL) {
+				fprintf(output, "#%*s %*s %*s events\n",
+					15 - 2, "time", 18, "counts",

tools/perf/util/stat-print-std.c line 842

Quoted 10 lines
tools/perf/util/stat-print-std.c +10 -0
+					config->unit_width, "unit");
+			} else {
+				fprintf(output,
+					"#%*s %-*s ctrs %*s %*s events\n",
+					15 - 2, "time",
+					aggr_header_lens[config->aggr_mode],
+					aggr_header_std[config->aggr_mode],
+					18, "counts", config->unit_width,
+					"unit");
+			}

[Severity: High]
Could the same out-of-bounds indexing occur here with config->aggr_mode?

Unvalidated indexing into statically sized arrays for interval headers might trigger a crash if aggr_mode exceeds the array bounds.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
maintainer
#77 lore raw

Re: [PATCH v4 02/14] perf stat: Implement standard console (STD) formatting callbacks

In reply to Ian Rogers
Quoted 16 lines · Ian Rogers

This patch implements standard console formatting callbacks inside util/stat-print-std.c, replacing the empty stubs introduced in Commit 1.

Introduces the format-private struct queued_event and struct queued_metric DOM nodes to buffer traversal streams, and fully encapsulates DOM state initialization and queue cleanups inside std_print_start() and std_print_end().

Utilizes the newly centralized unified aggregation helpers to resolve CPU and thread prefixes cleanly, and incorporates full interval-mode timestamp printing support across all rows.

Assisted-by Antigravity:gemini-3.5-flash

---

[SNIP]

Quoted 5 lines
tools/perf/util/stat-print-std.c +5 -0
+static int std_print_event(void *ctx, struct perf_stat_config *config,
+			   struct evsel *evsel, int aggr_idx, u64 val, u64 ena,
+			   u64 run, double stdev_pct, const char *cgrp)
+{
+	struct std_print_state *ps = ctx;

tools/perf/util/stat-print-std.c line 237

Quoted 10 lines
tools/perf/util/stat-print-std.c +10 -0
+	struct queued_event *ev;
+	/* Skip zero counters locally in STD callbacks if they qualify */
+	if (val == 0 && std_should_skip_zero_counter(config, evsel, aggr_idx)) {
+		ps->current_event = NULL;
+		return 0;
+	}
+
+	ev = malloc(sizeof(*ev));
+	if (!ev)
+		return -ENOMEM;

I expected this function prints the event info right away. Why we need to save it and print later?

Quoted 7 lines
tools/perf/util/stat-print-std.c +7 -0
+
+	ev->name = strdup(evsel__name(evsel));
+	if (!ev->name) {
+		free(ev);
+		return -ENOMEM;
+	}
+

tools/perf/util/stat-print-std.c line 254

Quoted 10 lines
tools/perf/util/stat-print-std.c +10 -0
+	if (cgrp && cgrp[0]) {
+		ev->cgrp = strdup(cgrp);
+		if (!ev->cgrp) {
+			free(ev->name);
+			free(ev);
+			return -ENOMEM;
+		}
+	} else {
+		ev->cgrp = NULL;
+	}

Do you really need this? Why not using evsel__name() and evsel->cgrp->name directly?

Quoted 14 lines
tools/perf/util/stat-print-std.c +14 -0
+
+	ev->evsel = evsel;
+	ev->val = val;
+	ev->ena = ena;
+	ev->run = run;
+	ev->stdev_pct = stdev_pct;
+	ev->aggr_idx = aggr_idx;
+	INIT_LIST_HEAD(&ev->metrics_list);
+
+	list_add_tail(&ev->list, &ps->events_list);
+	ps->current_event = ev;
+
+	return 0;
+}

[SNIP]

Quoted 75 lines
tools/perf/util/stat-print-std.c +74 -0
+static int std_print_end(void *ctx, struct perf_stat_config *config)
 {
+	struct std_print_state *ps = ctx;
+	struct queued_event *ev, *tmp_ev;
+	struct queued_metric *met, *tmp_met;
+	FILE *out = ps->fp;
+	bool first;
+	const char *last_mg_name = NULL;
+	const struct perf_pmu *last_pmu = NULL;
+	int last_aggr_idx = -1;
+
+	/* Print the formatted header prefix (only in non-interval mode) */
+	if (!config->interval)
+		print_header_std(config, ps->target, ps->argc, ps->argv);
+
+	list_for_each_entry_safe(ev, tmp_ev, &ps->events_list, list) {
+		struct evsel *evsel = ev->evsel;
+		double sc = evsel->scale;
+		const char *fmt;
+		const char *bad_count = evsel->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+		struct metric_event *me =
+			metricgroup__lookup(&evsel->evlist->metric_events, evsel, false);
+		bool is_metricgroup = false;
+		bool skip_header = false;
+		char full_name[128] = "";
+
+		if (me && me->is_default && !evsel->default_show_events) {
+			struct metric_expr *mexp =
+				list_first_entry(&me->head, struct metric_expr, nd);
+			const char *mg_name = mexp->default_metricgroup_name;
+			bool need_full_name = perf_pmus__num_core_pmus() > 1;
+
+			if (need_full_name && evsel->pmu)
+				scnprintf(full_name, sizeof(full_name), "%s (%s)", mg_name,
+					  evsel->pmu->name);
+			else
+				scnprintf(full_name, sizeof(full_name), "%s", mg_name);
+			is_metricgroup = true;
+
+			if (last_mg_name && !strcmp(last_mg_name, mg_name) &&
+			    last_pmu == evsel->pmu && last_aggr_idx == ev->aggr_idx) {
+				skip_header = true;
+			}
+			last_mg_name = mg_name;
+			last_pmu = evsel->pmu;
+			last_aggr_idx = ev->aggr_idx;
+		}
+
+		/* Print interval timestamp if configured */
+		if (config->interval && ps->timestamp[0] && !skip_header)
+			fprintf(out, "%s", ps->timestamp);
+
+		/* 1. Print aggregation prefix first (if we don't skip header) */
+		if (!skip_header && config->aggr_map && ev->aggr_idx >= 0) {
+			struct aggr_cpu_id id = config->aggr_map->map[ev->aggr_idx];
+			int aggr_nr = 0;
+
+			if (evsel->stats && evsel->stats->aggr)
+				aggr_nr = evsel->stats->aggr[ev->aggr_idx].nr;
+
+			print_aggr_id_std(config, out, evsel, id, aggr_nr);
+		}
+
+		/* 2. Print event value (scaled) or spaces if metricgroup */
+		if (is_metricgroup) {
+			if (!skip_header) {
+				int n = fprintf(out, " %*s", EVNAME_LEN, full_name);
+				int pad = MGROUP_LEN + config->unit_width + 2 - n;
+
+				fprintf(out, "%*s", pad > 0 ? pad : 0, "");
+			}
+		} else {
+			if (config->big_num)
+				fmt = floor(sc) != sc ? "%'*.2f " : "%'*.0f ";
+			else

tools/perf/util/stat-print-std.c line 472

Quoted 10 lines
tools/perf/util/stat-print-std.c +10 -0
+				fmt = floor(sc) != sc ? "%*.2f " : "%*.0f ";
+
+			if (ev->run == 0 || ev->ena == 0) {
+				fprintf(out, "%*s ", COUNTS_LEN, bad_count);
+			} else {
+				double scaled = (double)ev->val;
+				double avg;
+
+				if (ev->ena < ev->run)
+					scaled = (double)ev->val * ev->run / ev->ena;

I think the value is already scaled in process_counter_values().

Thanks,
Namhyung

Quoted 4 lines
tools/perf/util/stat-print-std.c +4 -0
+
+				avg = scaled * sc;
+				fprintf(out, fmt, COUNTS_LEN, avg);
+			}
author reviewer
#78 lore raw

PATCH v4 3/14 perf stat: Extend STD output linter to test basic New API checks

In reply to Ian Rogers

This patch extends the standard console (STD) output linter script tools/perf/tests/shell/stat+std_output.sh to run the basic no-argument check a second time using the --new print flag:

  perf_cmd="--new -o ${stat_output}"
  check_no_args "STD (New API)" "$perf_cmd"

This ensures that standard console outputs produced by the decoupled printing callbacks are formally validated by the test suite.

---
 tools/perf/tests/shell/stat+std_output.sh | 4 ++++
 1 file changed, 4 insertions(+)
tools/perf/tests/shell/stat+std_output.sh +4 -0
index 9c4b92ecf448..233e0a50eb33 100755
@@ -118,5 +118,9 @@ then
 else
 	echo "[Skip] Skipping tests for system_wide_no_aggr, per_core, per_die and per_socket since socket id exposed via topology is invalid"
 fi
+# New API basic checks
+perf_cmd="--new -o ${stat_output}"
+check_no_args "STD (New API)" "$perf_cmd"
+
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#79 lore raw

PATCH v4 4/14 perf stat: Extend STD output linter to test core aggregation checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+std_output.sh to run all standard CPU and thread aggregation checks under the --new print flag:

- check_system_wide
- check_system_wide_no_aggr
- check_interval
- check_per_thread
- check_per_node
- check_per_core
- check_per_socket
- check_per_die

This guarantees that standard console outputs produced by the decoupled printing engine are verified across all core CPU-aggregation modes.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+std_output.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)
tools/perf/tests/shell/stat+std_output.sh +11 -0
index 233e0a50eb33..8dee005a7281 100755
@@ -121,6 +121,17 @@ fi
 # New API basic checks
 perf_cmd="--new -o ${stat_output}"
 check_no_args "STD (New API)" "$perf_cmd"
+check_system_wide "STD (New API)" "$perf_cmd"
+check_interval "STD (New API)" "$perf_cmd"
+check_per_thread "STD (New API)" "$perf_cmd"
+check_per_node "STD (New API)" "$perf_cmd"
+if [ $skip_test -ne 1 ]
+then
+	check_system_wide_no_aggr "STD (New API)" "$perf_cmd"
+	check_per_core "STD (New API)" "$perf_cmd"
+	check_per_die "STD (New API)" "$perf_cmd"
+	check_per_socket "STD (New API)" "$perf_cmd"
+fi
 
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#80 lore raw

PATCH v4 5/14 perf stat: Extend STD output linter to test advanced PMU checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+std_output.sh to run the advanced hardware PMU and topology checks under the --new print flag:

- check_per_cache_instance
- check_per_cluster

This ensures that standard console outputs are verified under advanced topology-aware aggregation environments.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+std_output.sh | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/tests/shell/stat+std_output.sh +2 -0
index 8dee005a7281..0d38a1283967 100755
@@ -131,6 +131,8 @@ then
 	check_per_core "STD (New API)" "$perf_cmd"
 	check_per_die "STD (New API)" "$perf_cmd"
 	check_per_socket "STD (New API)" "$perf_cmd"
+	check_per_cache_instance "STD (New API)" "$perf_cmd"
+	check_per_cluster "STD (New API)" "$perf_cmd"
 fi
 
 cleanup
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#81 lore raw

PATCH v4 6/14 perf stat: Extend STD output linter to test metric-only checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+std_output.sh to run the metric-only check under the --new print flag:

- check_metric_only

This guarantees that standard console metric-only outputs produced by the decoupled printing callbacks are formally validated by the test linter.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+std_output.sh | 1 +
 1 file changed, 1 insertion(+)
tools/perf/tests/shell/stat+std_output.sh +1 -0
index 0d38a1283967..69720b19b908 100755
@@ -134,6 +134,7 @@ then
 	check_per_cache_instance "STD (New API)" "$perf_cmd"
 	check_per_cluster "STD (New API)" "$perf_cmd"
 fi
+check_metric_only "STD (New API)" "$perf_cmd"
 
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
maintainer
#82 lore raw

Re: [PATCH v4 06/14] perf stat: Extend STD output linter to test metric-only checks

Quoted 5 lines · Ian Rogers

This patch extends tools/perf/tests/shell/stat+std_output.sh to run the metric-only check under the --new print flag:

- check_metric_only

Quoted 7 lines

This guarantees that standard console metric-only outputs produced by the decoupled printing callbacks are formally validated by the test linter.

Assisted-by Antigravity:gemini-3.5-flash

Can you please combine test changes into a commit? I guess it's ok to have separate one for std, csv and json. But I'm seeing this as a temporary changes and we will remove it once we switch to the new way.

Thanks,
Namhyung

Quoted 19 lines
---
 tools/perf/tests/shell/stat+std_output.sh | 1 +
 1 file changed, 1 insertion(+)
tools/perf/tests/shell/stat+std_output.sh +1 -0
index 0d38a1283967..69720b19b908 100755
@@ -134,6 +134,7 @@ then
 	check_per_cache_instance "STD (New API)" "$perf_cmd"
 	check_per_cluster "STD (New API)" "$perf_cmd"
 fi
+check_metric_only "STD (New API)" "$perf_cmd"
 
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#83 lore raw

PATCH v4 7/14 perf stat: Implement CSV formatting callbacks

In reply to Ian Rogers

This patch implements CSV output formatting callbacks inside util/stat-print-csv.c, replacing the empty stubs introduced in Commit 1.

Defines the format-private struct queued_event and struct queued_metric DOM nodes to buffer traversal streams, and fully encapsulates CSV queued lists lifecycle and deallocations inside csv_print_start() and csv_print_end().

Utilizes the newly centralized unified aggregation helpers to format CPU and thread column prefixes cleanly, fixes metrics separators padding, and incorporates full interval-mode timestamp printing support.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/util/stat-print-csv.c | 678 ++++++++++++++++++++++++++++++-
 1 file changed, 671 insertions(+), 7 deletions(-)
tools/perf/util/stat-print-csv.c +671 -7
index ee86d07636ea..f93ca1885d46 100644
@@ -1,13 +1,677 @@
 // SPDX-License-Identifier: GPL-2.0
-#include "stat-print.h"
+#include <errno.h>
+#include <inttypes.h>
+#include <math.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <time.h>
+#include <stdbool.h>
 #include <linux/compiler.h>
+#include <linux/list.h>
+
+#include "cpumap.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "stat-print.h"
+#include "stat.h"
+#include "thread_map.h"
+#include "debug.h"
+
+#define COMM_LEN 16
+#define PID_LEN 7
+
+struct queued_metric {
+	struct list_head list;
+	struct evsel *evsel;
+	char *name;
+	char *unit;
+	double val;
+	int aggr_idx;
+};
+
+/**
+ * struct queued_event - In-memory record of a buffered CSV counter event.
+ * @list: Linked list node for queueing.
+ * @evsel: The associated performance event selector.
+ * @name: The uniquely formatted/resolved event name.
+ * @unit: The event's unit (e.g. "msec", "cycles").
+ * @cgrp: Cgroup name (optional).
+ * @val: Raw aggregated counter value.
+ * @ena: Enabled time for multiplexing percentage.
+ * @run: Running time for multiplexing percentage.
+ * @scale: Event scale factor.
+ * @stdev_pct: Standard deviation percentage.
+ * @supported: Event hardware support indicator.
+ * @aggr_idx: Aggregation index.
+ * @metrics_list: Linked list head containing nested queued_metric structures.
+ */
+struct queued_event {
+	struct list_head list;
+	struct evsel *evsel;
+	char *name;
+	char *unit;
+	char *cgrp;
+	u64 val, ena, run;
+	double scale;
+	double stdev_pct;
+	bool supported;
+	int aggr_idx;
+	struct list_head metrics_list;
+};
+
+/**
+ * struct csv_print_state - Print state context for CSV output.
+ * @fp: File descriptor to output to.
+ * @sep: CSV column separator character/string.
+ * @timestamp: Formatted interval timestamp (optional).
+ * @events_list: Linked list head containing queued_event nodes.
+ * @current_event: Pointer to the currently active event being printed.
+ *                 Serves as a temporary bridge to associate streaming metrics back to
+ *                 their parent event node during list buffering. This relies on a
+ *                 strict temporal coupling in the traversal driver: the driver always
+ *                 invokes print_metric() callbacks for a counter synchronously and
+ *                 immediately after its print_event() callback, prior to advancing
+ *                 to the next event or aggregation node. This pointer is completely
+ *                 private to CSV printing, keeping the traversal driver decoupled
+ *                 and preserving strict encapsulation.
+ */
+struct csv_print_state {
+	FILE *fp;
+	const char *sep;
+	char timestamp[64];
+	struct list_head events_list;
+	struct queued_event *current_event;
+};
+
+/**
+ * struct csv_metric_only_print_state - Metric-only print state context for CSV output.
+ * @fp: File descriptor to output to.
+ * @sep: CSV column separator.
+ * @timestamp: Formatted interval timestamp (optional).
+ * @evlist: Evlist to query entries from.
+ * @queued_metrics: Linked list head containing queued_metric nodes.
+ */
+struct csv_metric_only_print_state {
+	FILE *fp;
+	const char *sep;
+	char timestamp[64];
+	struct evlist *evlist;
+	struct list_head queued_metrics;
+};
+
+/**
+ * print_aggr_id_csv - Print the aggregation prefix for CSV format.
+ *
+ * Copied and adapted from stat-display.c.
+ */
+static void print_aggr_id_csv(const struct perf_stat_config *config, FILE *output,
+			      struct evsel *evsel, struct aggr_cpu_id id, int aggr_nr)
+{
+	const char *sep = config->csv_sep;
+
+	switch (config->aggr_mode) {
+	case AGGR_CORE:
+		fprintf(output, "S%d-D%d-C%d%s%d%s", id.socket, id.die, id.core, sep, aggr_nr, sep);
+		break;
+	case AGGR_CACHE:
+		fprintf(output, "S%d-D%d-L%d-ID%d%s%d%s", id.socket, id.die, id.cache_lvl, id.cache,
+			sep, aggr_nr, sep);
+		break;
+	case AGGR_CLUSTER:
+		fprintf(output, "S%d-D%d-CLS%d%s%d%s", id.socket, id.die, id.cluster, sep, aggr_nr,
+			sep);
+		break;
+	case AGGR_DIE:
+		fprintf(output, "S%d-D%d%s%d%s", id.socket, id.die, sep, aggr_nr, sep);
+		break;
+	case AGGR_SOCKET:
+		fprintf(output, "S%d%s%d%s", id.socket, sep, aggr_nr, sep);
+		break;
+	case AGGR_NODE:
+		fprintf(output, "N%d%s%d%s", id.node, sep, aggr_nr, sep);
+		break;
+	case AGGR_NONE:
+		if (evsel->percore && !config->percore_show_thread)
+			fprintf(output, "S%d-D%d-C%d%s", id.socket, id.die, id.core, sep);
+		else if (id.cpu.cpu > -1)
+			fprintf(output, "CPU%d%s", id.cpu.cpu, sep);
+		break;
+	case AGGR_THREAD: {
+		const char *comm = "unknown";
+		int pid = -1;
+
+		if (evsel && evsel->core.threads && id.thread_idx >= 0 &&
+		    id.thread_idx < perf_thread_map__nr(evsel->core.threads)) {
+			comm = perf_thread_map__comm(evsel->core.threads,
+						      id.thread_idx);
+			pid = perf_thread_map__pid(evsel->core.threads,
+						    id.thread_idx);
+		}
+		fprintf(output, "%s-%d%s", comm, pid, sep);
+		break;
+	}
+	case AGGR_GLOBAL:
+	case AGGR_UNSET:
+	case AGGR_MAX:
+	default:
+		break;
+	}
+}
+
+/*
+ * CSV Output Callbacks - Normal Mode
+ */
+
+static int csv_print_start(void *ctx, struct perf_stat_config *config __maybe_unused)
+{
+	struct csv_print_state *ps = ctx;
+
+	INIT_LIST_HEAD(&ps->events_list);
+	ps->current_event = NULL;
+	return 0;
+}
+
+static int csv_print_event(void *ctx, struct perf_stat_config *config,
+			   struct evsel *evsel, int aggr_idx, u64 val, u64 ena,
+			   u64 run, double stdev_pct, const char *cgrp)
+{
+	struct csv_print_state *ps = ctx;
+	struct queued_event *ev;
+	struct aggr_cpu_id id = aggr_cpu_id__empty();
+
+	if (config && config->aggr_map && aggr_idx >= 0 &&
+	    aggr_idx < config->aggr_map->nr)
+		id = config->aggr_map->map[aggr_idx];
+
+	/* Skip zero counters in CSV callbacks if they qualify */
+	if (val == 0 && should_skip_zero_counter(config, evsel, &id)) {
+		ps->current_event = NULL;
+		return 0;
+	}
+
+	ev = malloc(sizeof(*ev));
+	if (!ev)
+		return -ENOMEM;
+
+	ev->name = strdup(evsel__name(evsel));
+	if (!ev->name) {
+		free(ev);
+		return -ENOMEM;
+	}
+
+	if (evsel->unit) {
+		ev->unit = strdup(evsel->unit);
+		if (!ev->unit) {
+			free(ev->name);
+			free(ev);
+			return -ENOMEM;
+		}
+	} else {
+		ev->unit = NULL;
+	}
+
+	if (cgrp && cgrp[0]) {
+		ev->cgrp = strdup(cgrp);
+		if (!ev->cgrp) {
+			free(ev->unit);
+			free(ev->name);
+			free(ev);
+			return -ENOMEM;
+		}
+	} else {
+		ev->cgrp = NULL;
+	}
+
+	ev->evsel = evsel;
+	ev->val = val;
+	ev->ena = ena;
+	ev->run = run;
+	ev->scale = evsel->scale;
+	ev->stdev_pct = stdev_pct;
+	ev->supported = evsel->supported;
+	ev->aggr_idx = aggr_idx;
+	INIT_LIST_HEAD(&ev->metrics_list);
+
+	list_add_tail(&ev->list, &ps->events_list);
+	ps->current_event = ev;
+
+	return 0;
+}
+
+static int csv_print_metric(void *ctx, struct perf_stat_config *config __maybe_unused,
+			    struct evsel *evsel, int aggr_idx,
+			    const char *name, const char *unit, double val,
+			    enum metric_threshold_classify thresh __maybe_unused)
+{
+	struct csv_print_state *ps = ctx;
+	struct queued_metric *b;
+
+	if (!ps->current_event)
+		return 0;
+
+	if (evsel != ps->current_event->evsel) {
+		pr_err("decoupled print engine: temporal coupling violation: evsel mismatch!\n");
+		return -EINVAL;
+	}
+
+	b = malloc(sizeof(*b));
+	if (!b)
+		return -ENOMEM;
+
+	b->evsel = evsel;
+	b->name = strdup(name);
+	if (!b->name) {
+		free(b);
+		return -ENOMEM;
+	}
+
+	if (unit && unit[0]) {
+		b->unit = strdup(unit);
+		if (!b->unit) {
+			free(b->name);
+			free(b);
+			return -ENOMEM;
+		}
+	} else {
+		b->unit = NULL;
+	}
+
+	b->val = val;
+	b->aggr_idx = aggr_idx;
+	list_add_tail(&b->list, &ps->current_event->metrics_list);
+
+	return 0;
+}
+
+static int csv_print_end(void *ctx, struct perf_stat_config *config)
+{
+	struct csv_print_state *ps = ctx;
+	struct queued_event *ev, *tmp_ev;
+	struct queued_metric *met, *tmp_met;
+	FILE *output = ps->fp;
+	const char *sep = ps->sep;
+	bool has_metrics;
+
+	list_for_each_entry_safe(ev, tmp_ev, &ps->events_list, list) {
+		struct evsel *evsel = ev->evsel;
+		bool ok = (ev->run != 0 && ev->ena != 0);
+		const char *bad_count = ev->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+		double enabled_percent = 100;
+
+		/* Print interval timestamp first if configured */
+		if (config->interval && ps->timestamp[0])
+			fprintf(output, "%s", ps->timestamp);
+
+		/* Print aggregation prefix first in CSV normal mode */
+		if (config->aggr_map && ev->aggr_idx >= 0) {
+			struct aggr_cpu_id id = config->aggr_map->map[ev->aggr_idx];
+			int aggr_nr = 0;
+
+			if (evsel->stats && evsel->stats->aggr)
+				aggr_nr = evsel->stats->aggr[ev->aggr_idx].nr;
+
+			print_aggr_id_csv(config, output, evsel, id, aggr_nr);
+		}
+
+		/* 1. Print Value, Unit, Name (Columns 1, 2, 3) */
+		if (ok) {
+			double sc = ev->scale;
+			double avg = ev->val * sc;
+			const char *fmt = floor(sc) != sc ? "%.2f%s" : "%.0f%s";
+
+			fprintf(output, fmt, avg, sep);
+		} else {
+			fprintf(output, "%s%s", bad_count, sep);
+		}
 
-int perf_stat__print_csv(struct evlist *evlist __maybe_unused,
-			 const struct perf_stat_config *config __maybe_unused,
-			 const struct target *target __maybe_unused,
-			 const struct timespec *ts __maybe_unused,
-			 int argc __maybe_unused,
-			 const char **argv __maybe_unused)
+		if (ev->unit)
+			fprintf(output, "%s%s", ev->unit, sep);
+		else
+			fprintf(output, "%s", sep);
+
+		fprintf(output, "%s", ev->name);
+
+		/* 2. Print Runtime and Enabled Percentage (Columns 4, 5) */
+		if (ev->run != ev->ena)
+			enabled_percent = 100.0 * ev->run / ev->ena;
+		fprintf(output, "%s%" PRIu64 "%s%.2f", sep, ev->run, sep,
+			enabled_percent);
+
+		/* 3. Print Metrics (Columns 6, 7) */
+		has_metrics = false;
+		list_for_each_entry_safe(met, tmp_met, &ev->metrics_list, list) {
+			if (!has_metrics) {
+				has_metrics = true;
+			} else {
+				fprintf(output, "\n");
+				if (config->interval && ps->timestamp[0])
+					fprintf(output, "%s", ps->timestamp);
+				if (config->aggr_map && ev->aggr_idx >= 0) {
+					struct aggr_cpu_id id = config->aggr_map->map[ev->aggr_idx];
+					int aggr_nr = 0;
+
+					if (evsel->stats && evsel->stats->aggr)
+						aggr_nr = evsel->stats->aggr[ev->aggr_idx].nr;
+
+					print_aggr_id_csv(config, output, evsel, id, aggr_nr);
+				}
+				/*
+				 * Pad exactly 4 commas (Value, Unit, Name, Run, Enabled)
+				 * to line up with the Metric Value column.
+				 */
+				fprintf(output, "%s%s%s%s", sep, sep, sep, sep);
+			}
+			fprintf(output, "%s%.2f%s", sep, met->val, sep);
+			if (met->name && met->name[0])
+				fprintf(output, "%s", met->name);
+
+			list_del(&met->list);
+			free(met->name);
+			free(met->unit);
+			free(met);
+		}
+		if (!has_metrics)
+			fprintf(output, "%s%s", sep, sep);
+		fprintf(output, "\n");
+
+		list_del(&ev->list);
+		free(ev->name);
+		free(ev->unit);
+		free(ev->cgrp);
+		free(ev);
+	}
+	return 0;
+}
+
+static const struct perf_stat_print_callbacks csv_print_callbacks = {
+	.print_start = csv_print_start,
+	.print_end = csv_print_end,
+	.print_event = csv_print_event,
+	.print_metric = csv_print_metric,
+};
+
+/*
+ * CSV Output Callbacks - Metric-Only Mode
+ */
+
+static int csv_metric_only_print_start(void *ctx,
+				       struct perf_stat_config *config __maybe_unused)
 {
+	struct csv_metric_only_print_state *ps = ctx;
+
+	INIT_LIST_HEAD(&ps->queued_metrics);
 	return 0;
 }
+
+static int csv_metric_only_print_metric(void *ctx,
+					struct perf_stat_config *config __maybe_unused,
+					struct evsel *evsel, int aggr_idx,
+					const char *name, const char *unit, double val,
+					enum metric_threshold_classify thresh __maybe_unused)
+{
+	struct csv_metric_only_print_state *ps = ctx;
+	struct queued_metric *b = malloc(sizeof(*b));
+
+	if (!b)
+		return -ENOMEM;
+
+	b->evsel = evsel;
+	b->name = strdup(name);
+	if (!b->name) {
+		free(b);
+		return -ENOMEM;
+	}
+
+	if (unit && unit[0]) {
+		b->unit = strdup(unit);
+		if (!b->unit) {
+			free(b->name);
+			free(b);
+			return -ENOMEM;
+		}
+	} else {
+		b->unit = NULL;
+	}
+
+	b->val = val;
+	b->aggr_idx = aggr_idx;
+	list_add_tail(&b->list, &ps->queued_metrics);
+
+	return 0;
+}
+
+static int csv_metric_only_print_end(void *ctx, struct perf_stat_config *config)
+{
+	struct csv_metric_only_print_state *ps = ctx;
+	FILE *output = ps->fp;
+	const char *sep = ps->sep;
+	struct queued_metric *b, *tmp;
+	char **unique_names = NULL;
+	size_t num_unique = 0;
+	size_t unique_alloc = 0;
+	int *unique_aggrs = NULL;
+	size_t num_aggrs = 0;
+	size_t aggr_alloc = 0;
+	size_t i, j;
+	int ret = 0;
+	int err;
+
+	if (list_empty(&ps->queued_metrics))
+		return 0;
+
+	/* 1. Build unique metric names and unique aggr_idx lists */
+	list_for_each_entry(b, &ps->queued_metrics, list) {
+		bool found_metric = false;
+		bool found_aggr = false;
+
+		for (i = 0; i < num_unique; i++) {
+			if (!strcmp(unique_names[i], b->name)) {
+				found_metric = true;
+				break;
+			}
+		}
+		if (!found_metric) {
+			if (num_unique >= unique_alloc) {
+				unique_alloc = unique_alloc ? unique_alloc * 2 : 16;
+				unique_names = realloc(unique_names,
+						       unique_alloc * sizeof(char *));
+				if (!unique_names) {
+					ret = -ENOMEM;
+					goto cleanup_grid;
+				}
+			}
+			unique_names[num_unique++] = strdup(b->name);
+			if (!unique_names[num_unique - 1]) {
+				num_unique--;
+				ret = -ENOMEM;
+				goto cleanup_grid;
+			}
+		}
+
+		for (i = 0; i < num_aggrs; i++) {
+			if (unique_aggrs[i] == b->aggr_idx) {
+				found_aggr = true;
+				break;
+			}
+		}
+		if (!found_aggr) {
+			if (num_aggrs >= aggr_alloc) {
+				aggr_alloc = aggr_alloc ? aggr_alloc * 2 : 16;
+				unique_aggrs = realloc(unique_aggrs,
+						       aggr_alloc * sizeof(int));
+				if (!unique_aggrs) {
+					ret = -ENOMEM;
+					goto cleanup_grid;
+				}
+			}
+			unique_aggrs[num_aggrs++] = b->aggr_idx;
+		}
+	}
+
+	/* 2. Print Headers (if not already printed) */
+	if (!config->metric_only_headers_printed) {
+		if (config->interval) {
+			fprintf(output, "#%s%s", "time", sep);
+		} else {
+			if (config->aggr_map && num_aggrs > 0 &&
+			    config->aggr_mode != AGGR_GLOBAL) {
+				const char *header = aggr_header_csv[config->aggr_mode];
+				const char *p = header;
+
+				while (*p) {
+					if (*p == ',')
+						fputs(sep, output);
+					else
+						fputc(*p, output);
+					p++;
+				}
+			}
+		}
+
+		/* Print uniquely collected headers in exact columnar order */
+		for (i = 0; i < num_unique; i++) {
+			char *header_name = NULL;
+			/* Find first matching metric to grab its unit for the header */
+			char *unit = NULL;
+
+			list_for_each_entry(b, &ps->queued_metrics, list) {
+				if (!strcmp(b->name, unique_names[i])) {
+					unit = b->unit;
+					break;
+				}
+			}
+
+			if (unit && unit[0])
+				err = asprintf(&header_name, "%s  %s", unit,
+					       unique_names[i]);
+			else
+				err = asprintf(&header_name, "%s",
+					       unique_names[i]);
+
+			if (err < 0 || !header_name) {
+				ret = -ENOMEM;
+				goto cleanup_grid;
+			}
+			fprintf(output, "%s%s", header_name, sep);
+			free(header_name);
+		}
+		fprintf(output, "\n");
+		config->metric_only_headers_printed = true;
+	}
+
+	/* 3. Print Values for Each Aggregation Node */
+	for (i = 0; i < num_aggrs; i++) {
+		int current_aggr = unique_aggrs[i];
+		struct evsel *last_evsel = NULL;
+
+		/* Print aggregation prefix */
+		if (config->interval && ps->timestamp[0])
+			fprintf(output, "%s", ps->timestamp);
+
+		if (config->aggr_map && current_aggr >= 0 &&
+		    current_aggr < config->aggr_map->nr) {
+			struct aggr_cpu_id id = config->aggr_map->map[current_aggr];
+			int aggr_nr = 0;
+
+			/* Find evsel for this aggr_idx */
+			list_for_each_entry(b, &ps->queued_metrics, list) {
+				if (b->aggr_idx == current_aggr) {
+					last_evsel = b->evsel;
+					break;
+				}
+			}
+
+			if (last_evsel && last_evsel->stats &&
+			    last_evsel->stats->aggr &&
+			    current_aggr < last_evsel->stats->nr_aggr)
+				aggr_nr = last_evsel->stats->aggr[current_aggr].nr;
+
+			print_aggr_id_csv(config, output, last_evsel, id, aggr_nr);
+		}
+
+		/* Print each metric, or pad with empty comma if missing for this node */
+		for (j = 0; j < num_unique; j++) {
+			struct queued_metric *found_metric = NULL;
+
+			list_for_each_entry(b, &ps->queued_metrics, list) {
+				if (b->aggr_idx == current_aggr &&
+				    !strcmp(b->name, unique_names[j])) {
+					found_metric = b;
+					break;
+				}
+			}
+
+			if (found_metric) {
+				if (isnan(found_metric->val))
+					fprintf(output, "%s", sep);
+				else
+					fprintf(output, "%.1f%s",
+						found_metric->val, sep);
+			} else {
+				/* Padding for missing metric to preserve column alignment */
+				fprintf(output, "%s", sep);
+			}
+		}
+		fprintf(output, "\n");
+	}
+
+cleanup_grid:
+	for (i = 0; i < num_unique; i++)
+		free(unique_names[i]);
+	free(unique_names);
+	free(unique_aggrs);
+
+	list_for_each_entry_safe(b, tmp, &ps->queued_metrics, list) {
+		list_del(&b->list);
+		free(b->name);
+		free(b->unit);
+		free(b);
+	}
+	return ret;
+}
+
+static const struct perf_stat_print_callbacks csv_metric_only_print_callbacks = {
+	.print_start = csv_metric_only_print_start,
+	.print_end = csv_metric_only_print_end,
+	.print_event = NULL,
+	.print_metric = csv_metric_only_print_metric,
+};
+
+int perf_stat__print_csv(struct evlist *evlist, struct perf_stat_config *config,
+			 const struct target *target, const struct timespec *ts,
+			 int argc, const char **argv)
+{
+	if (config->metric_only) {
+		struct csv_metric_only_print_state ps = {
+			.fp = config->output,
+			.sep = config->csv_sep,
+			.evlist = evlist,
+		};
+		if (config->interval && ts) {
+			scnprintf(ps.timestamp, sizeof(ps.timestamp), "%lu.%09lu%s",
+				  (unsigned long)ts->tv_sec, ts->tv_nsec, config->csv_sep);
+		} else {
+			ps.timestamp[0] = '\0';
+		}
+		return perf_stat__print_cb(evlist, config, target, ts, argc, argv,
+					   &csv_metric_only_print_callbacks, &ps);
+	} else {
+		struct csv_print_state ps = {
+			.fp = config->output,
+			.sep = config->csv_sep,
+		};
+
+
+
+		if (config->interval && ts) {
+			scnprintf(ps.timestamp, sizeof(ps.timestamp), "%lu.%09lu%s",
+				  (unsigned long)ts->tv_sec, ts->tv_nsec, config->csv_sep);
+		} else {
+			ps.timestamp[0] = '\0';
+		}
+		return perf_stat__print_cb(evlist, config, target, ts, argc, argv,
+					   &csv_print_callbacks, &ps);
+	}
+}
Signature · 1 line
2.55.0.141.g00534a21ce-goog
#84 lore raw

Re: [PATCH v4 07/14] perf stat: Implement CSV formatting callbacks

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] Incorrect `realloc` error handling in `csv_metric_only_print_end` leads to a memory leak and…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Incorrect realloc error handling in csv_metric_only_print_end leads to a memory leak and segmentation fault.
- [Medium] The buffered cgroup name is silently dropped in csv_print_end(), breaking cgroup reporting in CSV mode. --

--- Patch [7]: [PATCH v4 07/14] perf stat: Implement CSV formatting callbacks --- commit 20a60f8e4c4914f0bb470217a8770175ba0dfe51
Author: Ian Rogers <[email protected]>
Subject: perf stat: Implement CSV formatting callbacks

This commit implements CSV output formatting callbacks in util/stat-print-csv.c. It defines structures to buffer traversal streams and encapsulates CSV queued lists lifecycle inside the print start and end functions. It also incorporates full interval-mode timestamp printing support.

Quoted 4 lines
tools/perf/util/stat-print-csv.c +0 -0
index ee86d07636ea6..f93ca1885d468 100644

[ ... ]

Quoted 18 lines
tools/perf/util/stat-print-csv.c +18 -0
+static int csv_print_end(void *ctx, struct perf_stat_config *config)
+{
+	struct csv_print_state *ps = ctx;
+	struct queued_event *ev, *tmp_ev;
+	struct queued_metric *met, *tmp_met;
+	FILE *output = ps->fp;
+	const char *sep = ps->sep;
+	bool has_metrics;
+
+	list_for_each_entry_safe(ev, tmp_ev, &ps->events_list, list) {
+		struct evsel *evsel = ev->evsel;
+		bool ok = (ev->run != 0 && ev->ena != 0);
+		const char *bad_count = ev->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+		double enabled_percent = 100;
+
+		/* Print interval timestamp first if configured */
+		if (config->interval && ps->timestamp[0])
+			fprintf(output, "%s", ps->timestamp);

[ ... ]

tools/perf/util/stat-print-csv.c line 341

Quoted 6 lines
tools/perf/util/stat-print-csv.c +6 -0
+		/* 2. Print Runtime and Enabled Percentage (Columns 4, 5) */
+		if (ev->run != ev->ena)
+			enabled_percent = 100.0 * ev->run / ev->ena;
+		fprintf(output, "%s%" PRIu64 "%s%.2f", sep, ev->run, sep,
+			enabled_percent);
+

[Severity: Medium]
Does this code silently drop the buffered cgroup name from the CSV output?

The cgroup string is correctly duplicated and stored in ev->cgrp during csv_print_event(), but it doesn't appear to be printed here along with the other core event properties before the ev struct is freed at the end of the loop.

Quoted 3 lines
tools/perf/util/stat-print-csv.c +3 -0
+		/* 3. Print Metrics (Columns 6, 7) */
+		has_metrics = false;
+		list_for_each_entry_safe(met, tmp_met, &ev->metrics_list, list) {

[ ... ]

Quoted 3 lines
tools/perf/util/stat-print-csv.c +3 -0
+static int csv_metric_only_print_end(void *ctx, struct perf_stat_config *config)
+{
+	struct csv_metric_only_print_state *ps = ctx;

[ ... ]

tools/perf/util/stat-print-csv.c line 484

Quoted 10 lines
tools/perf/util/stat-print-csv.c +10 -0
+		if (!found_metric) {
+			if (num_unique >= unique_alloc) {
+				unique_alloc = unique_alloc ? unique_alloc * 2 : 16;
+				unique_names = realloc(unique_names,
+						       unique_alloc * sizeof(char *));
+				if (!unique_names) {
+					ret = -ENOMEM;
+					goto cleanup_grid;
+				}
+			}

[Severity: Medium]
Can this cause a memory leak and a segmentation fault if realloc() fails?

If realloc() fails, it returns NULL and directly overwrites the unique_names pointer, which loses the reference to the previously allocated memory block.

Later, at the cleanup_grid label:

tools/perf/util/stat-print-csv.c line 622

Quoted 3 lines
tools/perf/util/stat-print-csv.c +3 -0
+cleanup_grid:
+	for (i = 0; i < num_unique; i++)
+		free(unique_names[i]);

The code will dereference the now-NULL unique_names array, which would cause an immediate crash.

This same pattern also appears just below this for the unique_aggrs allocation:

Quoted 9 lines
tools/perf/util/stat-print-csv.c +9 -0
+			if (num_aggrs >= aggr_alloc) {
+				aggr_alloc = aggr_alloc ? aggr_alloc * 2 : 16;
+				unique_aggrs = realloc(unique_aggrs,
+						       aggr_alloc * sizeof(int));
+				if (!unique_aggrs) {
+					ret = -ENOMEM;
+					goto cleanup_grid;
+				}
+			}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
maintainer
#85 lore raw

Re: [PATCH v4 07/14] perf stat: Implement CSV formatting callbacks

In reply to Ian Rogers
Quoted 16 lines · Ian Rogers

This patch implements CSV output formatting callbacks inside util/stat-print-csv.c, replacing the empty stubs introduced in Commit 1.

Defines the format-private struct queued_event and struct queued_metric DOM nodes to buffer traversal streams, and fully encapsulates CSV queued lists lifecycle and deallocations inside csv_print_start() and csv_print_end().

Utilizes the newly centralized unified aggregation helpers to format CPU and thread column prefixes cleanly, fixes metrics separators padding, and incorporates full interval-mode timestamp printing support.

Assisted-by Antigravity:gemini-3.5-flash

---

[SNIP]

Quoted 41 lines
tools/perf/util/stat-print-csv.c +41 -0
+/**
+ * print_aggr_id_csv - Print the aggregation prefix for CSV format.
+ *
+ * Copied and adapted from stat-display.c.
+ */
+static void print_aggr_id_csv(const struct perf_stat_config *config, FILE *output,
+			      struct evsel *evsel, struct aggr_cpu_id id, int aggr_nr)
+{
+	const char *sep = config->csv_sep;
+
+	switch (config->aggr_mode) {
+	case AGGR_CORE:
+		fprintf(output, "S%d-D%d-C%d%s%d%s", id.socket, id.die, id.core, sep, aggr_nr, sep);
+		break;
+	case AGGR_CACHE:
+		fprintf(output, "S%d-D%d-L%d-ID%d%s%d%s", id.socket, id.die, id.cache_lvl, id.cache,
+			sep, aggr_nr, sep);
+		break;
+	case AGGR_CLUSTER:
+		fprintf(output, "S%d-D%d-CLS%d%s%d%s", id.socket, id.die, id.cluster, sep, aggr_nr,
+			sep);
+		break;
+	case AGGR_DIE:
+		fprintf(output, "S%d-D%d%s%d%s", id.socket, id.die, sep, aggr_nr, sep);
+		break;
+	case AGGR_SOCKET:
+		fprintf(output, "S%d%s%d%s", id.socket, sep, aggr_nr, sep);
+		break;
+	case AGGR_NODE:
+		fprintf(output, "N%d%s%d%s", id.node, sep, aggr_nr, sep);
+		break;
+	case AGGR_NONE:
+		if (evsel->percore && !config->percore_show_thread)
+			fprintf(output, "S%d-D%d-C%d%s", id.socket, id.die, id.core, sep);
+		else if (id.cpu.cpu > -1)
+			fprintf(output, "CPU%d%s", id.cpu.cpu, sep);
+		break;
+	case AGGR_THREAD: {
+		const char *comm = "unknown";
+		int pid = -1;
+

tools/perf/util/stat-print-csv.c line 154

Quoted 10 lines
tools/perf/util/stat-print-csv.c +10 -0
+		if (evsel && evsel->core.threads && id.thread_idx >= 0 &&
+		    id.thread_idx < perf_thread_map__nr(evsel->core.threads)) {
+			comm = perf_thread_map__comm(evsel->core.threads,
+						      id.thread_idx);
+			pid = perf_thread_map__pid(evsel->core.threads,
+						    id.thread_idx);
+		}
+		fprintf(output, "%s-%d%s", comm, pid, sep);
+		break;
+	}

Can you use the type-agnostic helper here?

Quoted 7 lines
tools/perf/util/stat-print-csv.c +7 -0
+	case AGGR_GLOBAL:
+	case AGGR_UNSET:
+	case AGGR_MAX:
+	default:
+		break;
+	}
+}

[SNIP]

Quoted 8 lines
tools/perf/util/stat-print-csv.c +8 -0
+static int csv_print_end(void *ctx, struct perf_stat_config *config)
+{
+	struct csv_print_state *ps = ctx;
+	struct queued_event *ev, *tmp_ev;
+	struct queued_metric *met, *tmp_met;
+	FILE *output = ps->fp;
+	const char *sep = ps->sep;
+	bool has_metrics;

tools/perf/util/stat-print-csv.c line 305

Quoted 10 lines
tools/perf/util/stat-print-csv.c +10 -0
+
+	list_for_each_entry_safe(ev, tmp_ev, &ps->events_list, list) {
+		struct evsel *evsel = ev->evsel;
+		bool ok = (ev->run != 0 && ev->ena != 0);
+		const char *bad_count = ev->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+		double enabled_percent = 100;
+
+		/* Print interval timestamp first if configured */
+		if (config->interval && ps->timestamp[0])
+			fprintf(output, "%s", ps->timestamp);

Doesn't it need a separator?

Quoted 31 lines
tools/perf/util/stat-print-csv.c +24 -6
+
+		/* Print aggregation prefix first in CSV normal mode */
+		if (config->aggr_map && ev->aggr_idx >= 0) {
+			struct aggr_cpu_id id = config->aggr_map->map[ev->aggr_idx];
+			int aggr_nr = 0;
+
+			if (evsel->stats && evsel->stats->aggr)
+				aggr_nr = evsel->stats->aggr[ev->aggr_idx].nr;
+
+			print_aggr_id_csv(config, output, evsel, id, aggr_nr);
+		}
+
+		/* 1. Print Value, Unit, Name (Columns 1, 2, 3) */
+		if (ok) {
+			double sc = ev->scale;
+			double avg = ev->val * sc;
+			const char *fmt = floor(sc) != sc ? "%.2f%s" : "%.0f%s";
+
+			fprintf(output, fmt, avg, sep);
+		} else {
+			fprintf(output, "%s%s", bad_count, sep);
+		}
 
-int perf_stat__print_csv(struct evlist *evlist __maybe_unused,
-			 const struct perf_stat_config *config __maybe_unused,
-			 const struct target *target __maybe_unused,
-			 const struct timespec *ts __maybe_unused,
-			 int argc __maybe_unused,
-			 const char **argv __maybe_unused)
+		if (ev->unit)
+			fprintf(output, "%s%s", ev->unit, sep);

tools/perf/util/stat-print-csv.c line 340

Quoted 10 lines
tools/perf/util/stat-print-csv.c +10 -0
+		else
+			fprintf(output, "%s", sep);
+
+		fprintf(output, "%s", ev->name);
+
+		/* 2. Print Runtime and Enabled Percentage (Columns 4, 5) */
+		if (ev->run != ev->ena)
+			enabled_percent = 100.0 * ev->run / ev->ena;
+		fprintf(output, "%s%" PRIu64 "%s%.2f", sep, ev->run, sep,
+			enabled_percent);

Cgroups?

Thanks,
Namhyung

Quoted 46 lines
tools/perf/util/stat-print-csv.c +46 -0
+
+		/* 3. Print Metrics (Columns 6, 7) */
+		has_metrics = false;
+		list_for_each_entry_safe(met, tmp_met, &ev->metrics_list, list) {
+			if (!has_metrics) {
+				has_metrics = true;
+			} else {
+				fprintf(output, "\n");
+				if (config->interval && ps->timestamp[0])
+					fprintf(output, "%s", ps->timestamp);
+				if (config->aggr_map && ev->aggr_idx >= 0) {
+					struct aggr_cpu_id id = config->aggr_map->map[ev->aggr_idx];
+					int aggr_nr = 0;
+
+					if (evsel->stats && evsel->stats->aggr)
+						aggr_nr = evsel->stats->aggr[ev->aggr_idx].nr;
+
+					print_aggr_id_csv(config, output, evsel, id, aggr_nr);
+				}
+				/*
+				 * Pad exactly 4 commas (Value, Unit, Name, Run, Enabled)
+				 * to line up with the Metric Value column.
+				 */
+				fprintf(output, "%s%s%s%s", sep, sep, sep, sep);
+			}
+			fprintf(output, "%s%.2f%s", sep, met->val, sep);
+			if (met->name && met->name[0])
+				fprintf(output, "%s", met->name);
+
+			list_del(&met->list);
+			free(met->name);
+			free(met->unit);
+			free(met);
+		}
+		if (!has_metrics)
+			fprintf(output, "%s%s", sep, sep);
+		fprintf(output, "\n");
+
+		list_del(&ev->list);
+		free(ev->name);
+		free(ev->unit);
+		free(ev->cgrp);
+		free(ev);
+	}
+	return 0;
+}
author reviewer
#86 lore raw

PATCH v4 8/14 perf stat: Extend CSV output linter to test core aggregation checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+csv_output.sh to run all basic and core aggregation linter checks a second time under the --new CSV print flag:

- check_no_args
- check_system_wide
- check_interval
- check_event
- check_per_thread
- check_per_node
- check_system_wide_no_aggr
- check_per_core
- check_per_socket
- check_per_die

This guarantees that CSV outputs produced by the decoupled printing engine are formally verified and column-valid across standard and interval modes.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+csv_output.sh | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
tools/perf/tests/shell/stat+csv_output.sh +16 -0
index cd6fff597091..e4153a03d716 100755
@@ -88,5 +88,21 @@ then
 else
 	echo "[Skip] Skipping tests for system_wide_no_aggr, per_core, per_die and per_socket since socket id exposed via topology is invalid"
 fi
+# New API CSV checks
+perf_cmd="--new -x$csv_sep -o ${stat_output}"
+check_no_args "CSV (New API)" "$perf_cmd"
+check_system_wide "CSV (New API)" "$perf_cmd"
+check_interval "CSV (New API)" "$perf_cmd"
+check_event "CSV (New API)" "$perf_cmd"
+check_per_thread "CSV (New API)" "$perf_cmd"
+check_per_node "CSV (New API)" "$perf_cmd"
+if [ $skip_test -ne 1 ]
+then
+	check_system_wide_no_aggr "CSV (New API)" "$perf_cmd"
+	check_per_core "CSV (New API)" "$perf_cmd"
+	check_per_die "CSV (New API)" "$perf_cmd"
+	check_per_socket "CSV (New API)" "$perf_cmd"
+fi
+
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#87 lore raw

PATCH v4 9/14 perf stat: Extend CSV output linter to test advanced PMU and metric-only checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+csv_output.sh to run the advanced hardware PMU, topology-aware aggregation, and metric-only checks a second time under the --new CSV print flag:

- check_per_cache_instance
- check_per_cluster
- check_metric_only

This guarantees that CSV outputs produced by the decoupled printing callbacks are verified and column-valid under advanced aggregation modes and metric-only CSV row-column layouts.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+csv_output.sh | 3 +++
 1 file changed, 3 insertions(+)
tools/perf/tests/shell/stat+csv_output.sh +3 -0
index e4153a03d716..c48d9e2b6429 100755
@@ -102,7 +102,10 @@ then
 	check_per_core "CSV (New API)" "$perf_cmd"
 	check_per_die "CSV (New API)" "$perf_cmd"
 	check_per_socket "CSV (New API)" "$perf_cmd"
+	check_per_cache_instance "CSV (New API)" "$perf_cmd"
+	check_per_cluster "CSV (New API)" "$perf_cmd"
 fi
+check_metric_only "CSV (New API)" "$perf_cmd"
 
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#88 lore raw

PATCH v4 10/14 perf stat: Implement streaming JSON formatting callbacks

In reply to Ian Rogers

This patch implements streaming JSON formatting callbacks inside util/stat-print-json.c, replacing the empty stubs introduced in Commit 1.

Delivers a highly optimized, zero-allocation, and 100% streaming print engine for JSON normal and metric-only modes. It bypasses dynamic queue events and metric lists entirely, formatting and streaming JSON objects directly onto the output file descriptor.

Utilizes the newly centralized unified aggregation helpers to format CPU and thread keys inside the JSON objects, and incorporates full interval-mode timestamp printing support.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/util/stat-print-json.c | 340 +++++++++++++++++++++++++++++-
 1 file changed, 333 insertions(+), 7 deletions(-)
tools/perf/util/stat-print-json.c +333 -7
index 4690c4286fd2..7ed00aae89f4 100644
@@ -1,13 +1,339 @@
 // SPDX-License-Identifier: GPL-2.0
-#include "stat-print.h"
+#include <errno.h>
+#include <inttypes.h>
+#include <math.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <time.h>
+#include <stdbool.h>
 #include <linux/compiler.h>
+#include <linux/kernel.h>
+
+#include "cpumap.h"
+#include "evlist.h"
+#include "evsel.h"
+#include "stat-print.h"
+#include "stat.h"
+#include "thread_map.h"
+
+static const char *metric_threshold_classify__str(enum metric_threshold_classify thresh)
+{
+	const char *const strs[] = {
+		"unknown", "bad", "nearly bad", "less good", "good",
+	};
+	_Static_assert(ARRAY_SIZE(strs) - 1 == METRIC_THRESHOLD_GOOD, "missing enum value");
+	return strs[thresh];
+}
+
+/**
+ * struct json_print_state - Print state context for JSON output.
+ * @fp: File descriptor to output to.
+ * @timestamp: Formatted interval timestamp (optional).
+ */
+struct json_print_state {
+	FILE *fp;
+	char timestamp[64];
+};
+
+/**
+ * struct json_metric_only_print_state - Metric-only print state context for JSON output.
+ * @fp: File descriptor to output to.
+ * @timestamp: Formatted interval timestamp (optional).
+ * @evlist: Evlist to query entries from.
+ * @last_aggr_idx: The aggregation index of the last printed metric.
+ * @first_in_group: Whether the current metric is the first in its group.
+ */
+struct json_metric_only_print_state {
+	FILE *fp;
+	char timestamp[64];
+	struct evlist *evlist;
+	int last_aggr_idx;
+	bool first_in_group;
+};
+
+/**
+ * print_aggr_id_json - Print the aggregation prefix for JSON format.
+ *
+ * Copied and adapted from stat-display.c.
+ */
+static void print_aggr_id_json(const struct perf_stat_config *config, FILE *output,
+			       struct evsel *evsel, struct aggr_cpu_id id, int aggr_nr)
+{
+	switch (config->aggr_mode) {
+	case AGGR_CORE:
+		fprintf(output, "\"core\" : \"S%d-D%d-C%d\", \"counters\" : %d, ", id.socket,
+			id.die, id.core, aggr_nr);
+		break;
+	case AGGR_CACHE:
+		fprintf(output, "\"cache\" : \"S%d-D%d-L%d-ID%d\", \"counters\" : %d, ", id.socket,
+			id.die, id.cache_lvl, id.cache, aggr_nr);
+		break;
+	case AGGR_CLUSTER:
+		fprintf(output, "\"cluster\" : \"S%d-D%d-CLS%d\", \"counters\" : %d, ", id.socket,
+			id.die, id.cluster, aggr_nr);
+		break;
+	case AGGR_DIE:
+		fprintf(output, "\"die\" : \"S%d-D%d\", \"counters\" : %d, ", id.socket, id.die,
+			aggr_nr);
+		break;
+	case AGGR_SOCKET:
+		fprintf(output, "\"socket\" : \"S%d\", \"counters\" : %d, ", id.socket, aggr_nr);
+		break;
+	case AGGR_NODE:
+		fprintf(output, "\"node\" : \"N%d\", \"counters\" : %d, ", id.node, aggr_nr);
+		break;
+	case AGGR_NONE:
+		if (evsel->percore && !config->percore_show_thread)
+			fprintf(output, "\"core\" : \"S%d-D%d-C%d\", ", id.socket, id.die, id.core);
+		else if (id.cpu.cpu > -1)
+			fprintf(output, "\"cpu\" : \"%d\", ", id.cpu.cpu);
+		break;
+	case AGGR_THREAD:
+		fprintf(output, "\"thread\" : \"%s-%d\", ",
+			perf_thread_map__comm(evsel->core.threads, id.thread_idx),
+			perf_thread_map__pid(evsel->core.threads, id.thread_idx));
+		break;
+	case AGGR_GLOBAL:
+	case AGGR_UNSET:
+	case AGGR_MAX:
+	default:
+		break;
+	}
+}
+
+/*
+ * JSON Output Callbacks - Normal Mode (100% Streaming & Zero-Allocation)
+ */
+
+static int json_print_start(void *ctx __maybe_unused,
+			    struct perf_stat_config *config __maybe_unused)
+{
+	return 0;
+}
+
+static int json_print_event(void *ctx, struct perf_stat_config *config,
+			    struct evsel *evsel, int aggr_idx, u64 val,
+			    u64 ena, u64 run, double stdev_pct __maybe_unused,
+			    const char *cgrp __maybe_unused)
+{
+	struct json_print_state *ps = ctx;
+	FILE *output = config->output;
+	bool ok = (run != 0 && ena != 0);
+	double enabled_percent = 100.0;
+
+	fprintf(output, "{");
+
+	/* Print interval timestamp first if configured */
+	if (config->interval && ps && ps->timestamp[0])
+		fprintf(output, "%s", ps->timestamp);
+
+	/* Print aggregation JSON fields if configured */
+	if (config->aggr_map && aggr_idx >= 0) {
+		struct aggr_cpu_id id = config->aggr_map->map[aggr_idx];
+		int aggr_nr = 0;
+
+		if (evsel->stats && evsel->stats->aggr)
+			aggr_nr = evsel->stats->aggr[aggr_idx].nr;
+
+		print_aggr_id_json(config, output, evsel, id, aggr_nr);
+	}
+
+	if (ok) {
+		double sc = evsel->scale;
+		double avg = val * sc;
+
+		fprintf(output, "\"counter-value\" : \"%f\"", avg);
+	} else {
+		const char *bad_count = evsel->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+
+		fprintf(output, "\"counter-value\" : \"%s\"", bad_count);
+	}
+
+	fprintf(output, ", \"unit\" : \"%s\"", evsel->unit ?: "");
+	/* Cast away const for legacy evsel__name */
+	fprintf(output, ", \"event\" : \"%s\"", evsel__name((struct evsel *)evsel));
+
+	if (run != ena)
+		enabled_percent = 100.0 * run / ena;
+	fprintf(output, ", \"event-runtime\" : %" PRIu64 ", \"pcnt-running\" : %.2f", run,
+		enabled_percent);
+	fprintf(output, "}\n");
+
+	return 0;
+}
+
+static int json_print_metric(void *ctx, struct perf_stat_config *config,
+			     struct evsel *evsel, int aggr_idx,
+			     const char *name, const char *unit, double val,
+			     enum metric_threshold_classify thresh)
+{
+	struct json_print_state *ps = ctx;
+	FILE *output = config->output;
+	u64 run = 0, ena = 0;
+	double enabled_percent = 100.0;
+	struct perf_stat_evsel *ps_evsel = evsel->stats;
+
+	if (ps_evsel && ps_evsel->aggr && aggr_idx >= 0 &&
+	    aggr_idx < ps_evsel->nr_aggr) {
+		run = ps_evsel->aggr[aggr_idx].counts.run;
+		ena = ps_evsel->aggr[aggr_idx].counts.ena;
+	}
 
-int perf_stat__print_json(struct evlist *evlist __maybe_unused,
-			  const struct perf_stat_config *config __maybe_unused,
-			  const struct target *target __maybe_unused,
-			  const struct timespec *ts __maybe_unused,
-			  int argc __maybe_unused,
-			  const char **argv __maybe_unused)
+	fprintf(output, "{");
+
+	/* Print interval timestamp first if configured */
+	if (config->interval && ps && ps->timestamp[0])
+		fprintf(output, "%s", ps->timestamp);
+
+	/* Print aggregation JSON fields if configured */
+	if (config->aggr_map && aggr_idx >= 0) {
+		struct aggr_cpu_id id = config->aggr_map->map[aggr_idx];
+		int aggr_nr = 0;
+
+		if (evsel->stats && evsel->stats->aggr)
+			aggr_nr = evsel->stats->aggr[aggr_idx].nr;
+
+		print_aggr_id_json(config, output, evsel, id, aggr_nr);
+	}
+
+	if (run != ena)
+		enabled_percent = 100.0 * run / ena;
+	fprintf(output, "\"event-runtime\" : %" PRIu64 ", \"pcnt-running\" : %.2f", run,
+		enabled_percent);
+	fprintf(output, ", \"metric-value\" : \"%f\"", val);
+	if (unit && unit[0] && name && name[0])
+		fprintf(output, ", \"metric-unit\" : \"%s  %s\"", unit, name);
+	else if (name && name[0])
+		fprintf(output, ", \"metric-unit\" : \"%s\"", name);
+	else if (unit && unit[0])
+		fprintf(output, ", \"metric-unit\" : \"%s\"", unit);
+	if (thresh != METRIC_THRESHOLD_UNKNOWN) {
+		fprintf(output, ", \"metric-threshold\" : \"%s\"",
+			metric_threshold_classify__str(thresh));
+	}
+	fprintf(output, "}\n");
+
+	return 0;
+}
+
+static int json_print_end(void *ctx __maybe_unused,
+			  struct perf_stat_config *config __maybe_unused)
 {
 	return 0;
 }
+
+static const struct perf_stat_print_callbacks json_print_callbacks = {
+	.print_start = json_print_start,
+	.print_end = json_print_end,
+	.print_event = json_print_event,
+	.print_metric = json_print_metric,
+};
+
+/*
+ * JSON Output Callbacks - Metric-Only Mode (100% Streaming & Zero-Allocation)
+ */
+
+static int json_metric_only_print_start(void *ctx,
+					struct perf_stat_config *config __maybe_unused)
+{
+	struct json_metric_only_print_state *ps = ctx;
+
+	/* Initialize to -2 to distinguish from -1 (a valid index in AGGR_GLOBAL mode) */
+	ps->last_aggr_idx = -2;
+	ps->first_in_group = true;
+	return 0;
+}
+
+static int json_metric_only_print_metric(void *ctx,
+					 struct perf_stat_config *config __maybe_unused,
+					 struct evsel *evsel, int aggr_idx,
+					 const char *name, const char *unit, double val,
+					 enum metric_threshold_classify thresh __maybe_unused)
+{
+	struct json_metric_only_print_state *ps = ctx;
+	FILE *output = ps->fp;
+
+
+	if (aggr_idx != ps->last_aggr_idx) {
+		if (ps->last_aggr_idx != -2)
+			fprintf(output, "}\n");
+		fprintf(output, "{");
+		if (config->interval && ps->timestamp[0])
+			fprintf(output, "%s", ps->timestamp);
+		if (config->aggr_map && aggr_idx >= 0 &&
+		    aggr_idx < config->aggr_map->nr) {
+			struct aggr_cpu_id id = config->aggr_map->map[aggr_idx];
+			int aggr_nr = 0;
+
+			if (evsel && evsel->stats && evsel->stats->aggr &&
+			    aggr_idx < evsel->stats->nr_aggr)
+				aggr_nr = evsel->stats->aggr[aggr_idx].nr;
+
+			print_aggr_id_json(config, output, evsel, id, aggr_nr);
+		}
+		ps->last_aggr_idx = aggr_idx;
+		ps->first_in_group = true;
+	}
+
+	if (!ps->first_in_group)
+		fprintf(output, ", ");
+	ps->first_in_group = false;
+
+	if (unit && unit[0])
+		fprintf(output, "\"%s  %s\" : \"%.1f\"", unit, name, val);
+	else
+		fprintf(output, "\"%s\" : \"%.1f\"", name, val);
+	return 0;
+}
+
+static int json_metric_only_print_end(void *ctx,
+				      struct perf_stat_config *config __maybe_unused)
+{
+	struct json_metric_only_print_state *ps = ctx;
+	FILE *output = ps->fp;
+
+	if (ps->last_aggr_idx != -2)
+		fprintf(output, "}\n");
+	return 0;
+}
+
+static const struct perf_stat_print_callbacks json_metric_only_print_callbacks = {
+	.print_start = json_metric_only_print_start,
+	.print_end = json_metric_only_print_end,
+	.print_event = NULL,
+	.print_metric = json_metric_only_print_metric,
+};
+
+int perf_stat__print_json(struct evlist *evlist, struct perf_stat_config *config,
+			  const struct target *target, const struct timespec *ts,
+			  int argc, const char **argv)
+{
+	if (config->metric_only) {
+		struct json_metric_only_print_state ps = {
+			.fp = config->output,
+			.evlist = evlist,
+		};
+		if (config->interval && ts) {
+			scnprintf(ps.timestamp, sizeof(ps.timestamp), "\"interval\" : %lu.%09lu, ",
+				  (unsigned long)ts->tv_sec, ts->tv_nsec);
+		} else {
+			ps.timestamp[0] = '\0';
+		}
+		return perf_stat__print_cb(evlist, config, target, ts, argc, argv,
+					   &json_metric_only_print_callbacks, &ps);
+	} else {
+		struct json_print_state ps = {
+			.fp = config->output,
+		};
+		if (config->interval && ts) {
+			scnprintf(ps.timestamp, sizeof(ps.timestamp), "\"interval\" : %lu.%09lu, ",
+				  (unsigned long)ts->tv_sec, ts->tv_nsec);
+		} else {
+			ps.timestamp[0] = '\0';
+		}
+		return perf_stat__print_cb(evlist, config, target, ts, argc, argv,
+					   &json_print_callbacks, &ps);
+	}
+}
Signature · 1 line
2.55.0.141.g00534a21ce-goog
AI review Acked-by Chun-Tse Shao on v4 10/14
#89 lore raw

Re: [PATCH v4 10/14] perf stat: Implement streaming JSON formatting callbacks

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:

New issues:
- [High] The JSON output completely ignores cgroup identification and statistical variance data for events.
- [Low] The state structs contain fields that are initialized but never used by any print callbacks.

Pre-existing issues:
- [High] Missing bounds and NULL check on evsel->core.threads in print_aggr_id_json when formatting AGGR_THREAD JSON output.
- [High] Passing a potentially NULL string from perf_thread_map__comm() to the %s format specifier causes undefined behavior and leads to a crash on musl libc systems. --

--- Patch [10]: [PATCH v4 10/14] perf stat: Implement streaming JSON formatting callbacks --- commit 77f0723329ce102d572ddc7616c0e8a06c6c64f1
Author: Ian Rogers <[email protected]>

perf stat: Implement streaming JSON formatting callbacks

This patch implements streaming JSON formatting callbacks inside util/stat-print-json.c, replacing the empty stubs introduced in Commit 1. It delivers a highly optimized, zero-allocation, and streaming print engine for JSON normal and metric-only modes.

Assisted-by Antigravity:gemini-3.5-flash
Quoted 4 lines
tools/perf/util/stat-print-json.c +0 -0
index 4690c4286fd2a..7ed00aae89f4b 100644

[ ... ]

Quoted 7 lines
tools/perf/util/stat-print-json.c +7 -0
+struct json_print_state {
+	FILE *fp;
+	char timestamp[64];
+};
+
+/**
+ * struct json_metric_only_print_state - Metric-only print state context for JSON output.

tools/perf/util/stat-print-json.c line 51

Quoted 10 lines
tools/perf/util/stat-print-json.c +10 -0
+ * @fp: File descriptor to output to.
+ * @timestamp: Formatted interval timestamp (optional).
+ * @evlist: Evlist to query entries from.
+ * @last_aggr_idx: The aggregation index of the last printed metric.
+ * @first_in_group: Whether the current metric is the first in its group.
+ */
+struct json_metric_only_print_state {
+	FILE *fp;
+	char timestamp[64];
+	struct evlist *evlist;

[Severity: Low]
Are the fp field in json_print_state and the evlist field in json_metric_only_print_state needed? It appears they are initialized in perf_stat__print_json but never actually read or used by any print callbacks, which introduces unnecessary dead state.

Quoted 3 lines
tools/perf/util/stat-print-json.c +3 -0
+	int last_aggr_idx;
+	bool first_in_group;
+};

[ ... ]

tools/perf/util/stat-print-json.c line 96

Quoted 4 lines
tools/perf/util/stat-print-json.c +4 -0
+	case AGGR_THREAD:
+		fprintf(output, "\"thread\" : \"%s-%d\", ",
+			perf_thread_map__comm(evsel->core.threads, id.thread_idx),
+			perf_thread_map__pid(evsel->core.threads, id.thread_idx));

[Severity: High]
This is a pre-existing issue, but is it possible for evsel->core.threads to be NULL or id.thread_idx to be out of bounds here? Calling perf_thread_map__comm without validating the index can lead to a deterministic segmentation fault.

[Severity: High]
This isn't a bug introduced by this patch, but does perf_thread_map__comm safely return a non-NULL string in all cases? If the comm string pointer is NULL, passing it to the %s format specifier in fprintf causes a segmentation fault on systems using musl libc.

Quoted 1 line
tools/perf/util/stat-print-json.c +1 -0
+		break;

[ ... ]

tools/perf/util/stat-print-json.c line 119

Quoted 4 lines
tools/perf/util/stat-print-json.c +4 -0
+static int json_print_event(void *ctx, struct perf_stat_config *config,
+			    struct evsel *evsel, int aggr_idx, u64 val,
+			    u64 ena, u64 run, double stdev_pct __maybe_unused,
+			    const char *cgrp __maybe_unused)

[Severity: High]
Does ignoring the stdev_pct and cgrp parameters result in a regression of the JSON output parity? The baseline code formatted cgroup and variance into the JSON output, so marking these as unused here seems to silently drop standard deviation and cgroup data, which could break automated profiling pipelines.

Quoted 1 line

+{

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
maintainer
#90 lore raw

Re: [PATCH v4 10/14] perf stat: Implement streaming JSON formatting callbacks

In reply to Ian Rogers
Quoted 17 lines · Ian Rogers

This patch implements streaming JSON formatting callbacks inside util/stat-print-json.c, replacing the empty stubs introduced in Commit 1.

Delivers a highly optimized, zero-allocation, and 100% streaming print engine for JSON normal and metric-only modes. It bypasses dynamic queue events and metric lists entirely, formatting and streaming JSON objects directly onto the output file descriptor.

Utilizes the newly centralized unified aggregation helpers to format CPU and thread keys inside the JSON objects, and incorporates full interval-mode timestamp printing support.

Assisted-by Antigravity:gemini-3.5-flash

---

[SNIP]

Quoted 38 lines
tools/perf/util/stat-print-json.c +38 -0
+/**
+ * print_aggr_id_json - Print the aggregation prefix for JSON format.
+ *
+ * Copied and adapted from stat-display.c.
+ */
+static void print_aggr_id_json(const struct perf_stat_config *config, FILE *output,
+			       struct evsel *evsel, struct aggr_cpu_id id, int aggr_nr)
+{
+	switch (config->aggr_mode) {
+	case AGGR_CORE:
+		fprintf(output, "\"core\" : \"S%d-D%d-C%d\", \"counters\" : %d, ", id.socket,
+			id.die, id.core, aggr_nr);
+		break;
+	case AGGR_CACHE:
+		fprintf(output, "\"cache\" : \"S%d-D%d-L%d-ID%d\", \"counters\" : %d, ", id.socket,
+			id.die, id.cache_lvl, id.cache, aggr_nr);
+		break;
+	case AGGR_CLUSTER:
+		fprintf(output, "\"cluster\" : \"S%d-D%d-CLS%d\", \"counters\" : %d, ", id.socket,
+			id.die, id.cluster, aggr_nr);
+		break;
+	case AGGR_DIE:
+		fprintf(output, "\"die\" : \"S%d-D%d\", \"counters\" : %d, ", id.socket, id.die,
+			aggr_nr);
+		break;
+	case AGGR_SOCKET:
+		fprintf(output, "\"socket\" : \"S%d\", \"counters\" : %d, ", id.socket, aggr_nr);
+		break;
+	case AGGR_NODE:
+		fprintf(output, "\"node\" : \"N%d\", \"counters\" : %d, ", id.node, aggr_nr);
+		break;
+	case AGGR_NONE:
+		if (evsel->percore && !config->percore_show_thread)
+			fprintf(output, "\"core\" : \"S%d-D%d-C%d\", ", id.socket, id.die, id.core);
+		else if (id.cpu.cpu > -1)
+			fprintf(output, "\"cpu\" : \"%d\", ", id.cpu.cpu);
+		break;
+	case AGGR_THREAD:

tools/perf/util/stat-print-json.c line 103

Quoted 10 lines
tools/perf/util/stat-print-json.c +10 -0
+		fprintf(output, "\"thread\" : \"%s-%d\", ",
+			perf_thread_map__comm(evsel->core.threads, id.thread_idx),
+			perf_thread_map__pid(evsel->core.threads, id.thread_idx));
+		break;
+	case AGGR_GLOBAL:
+	case AGGR_UNSET:
+	case AGGR_MAX:
+	default:
+		break;
+	}

It'd be great if you can use the format-agnostic helpers.

Quoted 17 lines
tools/perf/util/stat-print-json.c +17 -0
+}
+
+/*
+ * JSON Output Callbacks - Normal Mode (100% Streaming & Zero-Allocation)
+ */
+
+static int json_print_start(void *ctx __maybe_unused,
+			    struct perf_stat_config *config __maybe_unused)
+{
+	return 0;
+}
+
+static int json_print_event(void *ctx, struct perf_stat_config *config,
+			    struct evsel *evsel, int aggr_idx, u64 val,
+			    u64 ena, u64 run, double stdev_pct __maybe_unused,
+			    const char *cgrp __maybe_unused)
+{

tools/perf/util/stat-print-json.c line 130

Quoted 10 lines
tools/perf/util/stat-print-json.c +10 -0
+	struct json_print_state *ps = ctx;
+	FILE *output = config->output;
+	bool ok = (run != 0 && ena != 0);
+	double enabled_percent = 100.0;
+
+	fprintf(output, "{");
+
+	/* Print interval timestamp first if configured */
+	if (config->interval && ps && ps->timestamp[0])
+		fprintf(output, "%s", ps->timestamp);

"Interval" : ?

Quoted 16 lines
tools/perf/util/stat-print-json.c +16 -0
+
+	/* Print aggregation JSON fields if configured */
+	if (config->aggr_map && aggr_idx >= 0) {
+		struct aggr_cpu_id id = config->aggr_map->map[aggr_idx];
+		int aggr_nr = 0;
+
+		if (evsel->stats && evsel->stats->aggr)
+			aggr_nr = evsel->stats->aggr[aggr_idx].nr;
+
+		print_aggr_id_json(config, output, evsel, id, aggr_nr);
+	}
+
+	if (ok) {
+		double sc = evsel->scale;
+		double avg = val * sc;
+

tools/perf/util/stat-print-json.c line 156

Quoted 10 lines
tools/perf/util/stat-print-json.c +10 -0
+		fprintf(output, "\"counter-value\" : \"%f\"", avg);
+	} else {
+		const char *bad_count = evsel->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED;
+
+		fprintf(output, "\"counter-value\" : \"%s\"", bad_count);
+	}
+
+	fprintf(output, ", \"unit\" : \"%s\"", evsel->unit ?: "");
+	/* Cast away const for legacy evsel__name */
+	fprintf(output, ", \"event\" : \"%s\"", evsel__name((struct evsel *)evsel));

Cgroups?

tools/perf/util/stat-print-json.c line 162

Quoted 6 lines
tools/perf/util/stat-print-json.c +6 -0
+
+	if (run != ena)
+		enabled_percent = 100.0 * run / ena;
+	fprintf(output, ", \"event-runtime\" : %" PRIu64 ", \"pcnt-running\" : %.2f", run,
+		enabled_percent);
+	fprintf(output, "}\n");

Now metrics are printed in a separate object/line?

Thanks,
Namhyung

Quoted 3 lines
tools/perf/util/stat-print-json.c +3 -0
+
+	return 0;
+}
author reviewer
#91 lore raw

PATCH v4 11/14 perf stat: Extend JSON output linter to test core aggregation checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+json_output.sh to run all basic and standard CPU/thread aggregation checks under the --new JSON print flag:

- check_no_args
- check_system_wide
- check_interval
- check_event
- check_per_thread
- check_per_node
- check_system_wide_no_aggr
- check_per_core
- check_per_socket
- check_per_die

This guarantees that JSON outputs produced by the decoupled, zero-allocation, and streaming print callbacks are formally validated across standard and interval modes.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+json_output.sh | 71 ++++++++++++++--------
 1 file changed, 45 insertions(+), 26 deletions(-)
tools/perf/tests/shell/stat+json_output.sh +45 -26
index 85d1ad7186c6..8168b861ade7 100755
@@ -13,6 +13,7 @@ shelldir=$(dirname "$0")
 . "${shelldir}"/lib/setup_python.sh
 pythonchecker=$(dirname $0)/lib/perf_json_output_lint.py
 
+perf_new_opt=""
 stat_output=$(mktemp /tmp/__perf_test.stat_output.json.XXXXX)
 
 cleanup() {
@@ -35,42 +36,42 @@ function ParanoidAndNotRoot()
 
 check_no_args()
 {
-	echo -n "Checking json output: no args "
-	perf stat -j -o "${stat_output}" true
+	echo -n "Checking $api_label: no args "
+	perf stat -j $perf_new_opt -o "${stat_output}" true
 	$PYTHON $pythonchecker --no-args --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_system_wide()
 {
-	echo -n "Checking json output: system wide "
+	echo -n "Checking $api_label: system wide "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j -a -o "${stat_output}" true
+	perf stat -j $perf_new_opt -a -o "${stat_output}" true
 	$PYTHON $pythonchecker --system-wide --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_system_wide_no_aggr()
 {
-	echo -n "Checking json output: system wide no aggregation "
+	echo -n "Checking $api_label: system wide no aggregation "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j -A -a --no-merge -o "${stat_output}" true
+	perf stat -j $perf_new_opt -A -a --no-merge -o "${stat_output}" true
 	$PYTHON $pythonchecker --system-wide-no-aggr --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_interval()
 {
-	echo -n "Checking json output: interval "
-	perf stat -j -I 1000 -o "${stat_output}" true
+	echo -n "Checking $api_label: interval "
+	perf stat -j $perf_new_opt -I 1000 -o "${stat_output}" true
 	$PYTHON $pythonchecker --interval --file "${stat_output}"
 	echo "[Success]"
 }
@@ -78,110 +79,110 @@ check_interval()
 
 check_event()
 {
-	echo -n "Checking json output: event "
-	perf stat -j -e cpu-clock -o "${stat_output}" true
+	echo -n "Checking $api_label: event "
+	perf stat -j $perf_new_opt -e cpu-clock -o "${stat_output}" true
 	$PYTHON $pythonchecker --event --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_per_core()
 {
-	echo -n "Checking json output: per core "
+	echo -n "Checking $api_label: per core "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-core -a -o "${stat_output}" true
+	perf stat -j $perf_new_opt --per-core -a -o "${stat_output}" true
 	$PYTHON $pythonchecker --per-core --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_per_thread()
 {
-	echo -n "Checking json output: per thread "
+	echo -n "Checking $api_label: per thread "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-thread -p $$ -o "${stat_output}" true
+	perf stat -j $perf_new_opt --per-thread -p $$ -o "${stat_output}" true
 	$PYTHON $pythonchecker --per-thread --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_per_cache_instance()
 {
-	echo -n "Checking json output: per cache_instance "
+	echo -n "Checking $api_label: per cache_instance "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-cache -a true 2>&1 | $PYTHON $pythonchecker --per-cache
+	perf stat -j $perf_new_opt --per-cache -a true 2>&1 | $PYTHON $pythonchecker --per-cache
 	echo "[Success]"
 }
 
 check_per_cluster()
 {
-	echo -n "Checking json output: per cluster "
+	echo -n "Checking $api_label: per cluster "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-cluster -a true 2>&1 | $PYTHON $pythonchecker --per-cluster
+	perf stat -j $perf_new_opt --per-cluster -a true 2>&1 | $PYTHON $pythonchecker --per-cluster
 	echo "[Success]"
 }
 
 check_per_die()
 {
-	echo -n "Checking json output: per die "
+	echo -n "Checking $api_label: per die "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-die -a -o "${stat_output}" true
+	perf stat -j $perf_new_opt --per-die -a -o "${stat_output}" true
 	$PYTHON $pythonchecker --per-die --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_per_node()
 {
-	echo -n "Checking json output: per node "
+	echo -n "Checking $api_label: per node "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-node -a -o "${stat_output}" true
+	perf stat -j $perf_new_opt --per-node -a -o "${stat_output}" true
 	$PYTHON $pythonchecker --per-node --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_per_socket()
 {
-	echo -n "Checking json output: per socket "
+	echo -n "Checking $api_label: per socket "
 	if ParanoidAndNotRoot 0
 	then
 		echo "[Skip] paranoia and not root"
 		return
 	fi
-	perf stat -j --per-socket -a -o "${stat_output}" true
+	perf stat -j $perf_new_opt --per-socket -a -o "${stat_output}" true
 	$PYTHON $pythonchecker --per-socket --file "${stat_output}"
 	echo "[Success]"
 }
 
 check_metric_only()
 {
-	echo -n "Checking json output: metric only "
+	echo -n "Checking $api_label: metric only "
 	if [ "$(uname -m)" = "s390x" ] && ! grep '^facilities' /proc/cpuinfo  | grep -qw 67
 	then
 		echo "[Skip] CPU-measurement counter facility not installed"
 		return
 	fi
-	perf stat -j --metric-only -M page_faults_per_second -o "${stat_output}" true
+	perf stat -j $perf_new_opt --metric-only -M page_faults_per_second -o "${stat_output}" true
 	$PYTHON $pythonchecker --metric-only --file "${stat_output}"
 	echo "[Success]"
 }
@@ -214,6 +215,7 @@ check_for_topology()
 }
 
 check_for_topology
+api_label="json output"
 check_no_args
 check_system_wide
 check_interval
@@ -232,5 +234,22 @@ then
 else
 	echo "[Skip] Skipping tests for system_wide_no_aggr, per_core, per_die and per_socket since socket id exposed via topology is invalid"
 fi
+# Run New API JSON basic and standard aggregation checks
+perf_new_opt="--new"
+api_label="json (New API)"
+check_no_args
+check_system_wide
+check_interval
+check_event
+check_per_thread
+check_per_node
+if [ $skip_test -ne 1 ]
+then
+	check_system_wide_no_aggr
+	check_per_core
+	check_per_die
+	check_per_socket
+fi
+
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#92 lore raw

PATCH v4 12/14 perf stat: Extend JSON output linter to test advanced PMU and metric-only checks

In reply to Ian Rogers

This patch extends tools/perf/tests/shell/stat+json_output.sh to run the advanced hardware PMU, topology-aware aggregation, and metric-only JSON checks a second time under the --new print flag:

- check_per_cache_instance
- check_per_cluster
- check_metric_only

This guarantees that JSON outputs produced by the decoupled streaming printing callbacks are verified and structurally valid under advanced topology-aware modes and metric-only JSON row-column layouts.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat+json_output.sh | 3 +++
 1 file changed, 3 insertions(+)
tools/perf/tests/shell/stat+json_output.sh +3 -0
index 8168b861ade7..d18cac5dcff1 100755
@@ -249,7 +249,10 @@ then
 	check_per_core
 	check_per_die
 	check_per_socket
+	check_per_cache_instance
+	check_per_cluster
 fi
+check_metric_only
 
 cleanup
 exit 0
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#93 lore raw

PATCH v4 13/14 perf stat: Add --new support to PMU metrics Python validator

In reply to Ian Rogers

This patch extends the performance metrics value Python validation script tools/perf/tests/shell/lib/perf_metric_validation.py to support a new command-line argument -new:

parser.add_argument("-new", help="Use new printing API (--new)", ...)

When set, the Validator class appends the --new option flag to its internally spawned perf stat commands:

command = [tool, 'stat']
if self.new_print:

      command.append('--new')

This enables validating Intel PMU metric mathematical values generated specifically by the decoupled, streaming JSON printing callbacks.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/lib/perf_metric_validation.py | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
tools/perf/tests/shell/lib/perf_metric_validation.py +9 -3
index 3d52f94f22b9..b8397234eebc 100644
@@ -36,7 +36,7 @@ class TestError:
 
 class Validator:
     def __init__(self, rulefname, reportfname='', t=5, debug=False, datafname='', fullrulefname='',
-                 workload='true', metrics='', cputype='cpu'):
+                 workload='true', metrics='', cputype='cpu', new_print=False):
         self.rulefname = rulefname
         self.reportfname = reportfname
         self.rules = None
@@ -68,6 +68,7 @@ class Validator:
         self.datafname = datafname
         self.debug = debug
         self.fullrulefname = fullrulefname
+        self.new_print = new_print
 
     def __set_metrics(self, metrics=''):
         if metrics != '':
@@ -379,7 +380,10 @@ class Validator:
 
     def _run_perf(self, metric, workload: str):
         tool = 'perf'
-        command = [tool, 'stat', '--cputype', self.cputype, '-j', '-M', f"{metric}", "-a"]
+        command = [tool, 'stat']
+        if self.new_print:
+            command.append('--new')
+        command.extend(['--cputype', self.cputype, '-j', '-M', f"{metric}", "-a"])
         wl = workload.split()
         command.extend(wl)
         print(" ".join(command))
@@ -587,6 +591,8 @@ def main() -> None:
     parser.add_argument("-m", help="Metric list to validate", default="")
     parser.add_argument("-cputype", help="Only test metrics for the given CPU/PMU type",
                         default="cpu")
+    parser.add_argument("-new", help="Use new printing API (--new)",
+                        action="store_true", default=False)
     args = parser.parse_args()
     outpath = Path(args.output_dir)
     reportf = Path.joinpath(outpath, 'perf_report.json')
@@ -595,7 +601,7 @@ def main() -> None:
 
     validator = Validator(args.rule, reportf, debug=args.debug,
                           datafname=datafile, fullrulefname=fullrule, workload=args.wl,
-                          metrics=args.m, cputype=args.cputype)
+                          metrics=args.m, cputype=args.cputype, new_print=args.new)
     ret = validator.test()
 
     return ret
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#94 lore raw

PATCH v4 14/14 perf stat: Extend PMU metrics value linter to validate --new outputs

In reply to Ian Rogers

This patch updates tools/perf/tests/shell/stat_metrics_values.sh to run the metrics value validation linter twice: once for the legacy path, and once under the --new print flag using the newly introduced python validator -new argument:

$PYTHON $pythonvalidator -rule $rulefile -output_dir $tmpdir ... -new

This mathematically guarantees that calculated Intel PMU metric values staged inside the decoupled streaming JSON callbacks are parsed and validated cleanly against the formal rules engine.

Assisted-by Antigravity:gemini-3.5-flash
---
 tools/perf/tests/shell/stat_metrics_values.sh | 22 ++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)
tools/perf/tests/shell/stat_metrics_values.sh +19 -3
index 86c5c7f70933..3f545e88b69f 100755
@@ -26,13 +26,29 @@ echo "Output will be stored in: $tmpdir"
 for cputype in /sys/bus/event_source/devices/cpu_*; do
 	cputype=$(basename "$cputype")
 	echo "Testing metrics for: $cputype"
-	$PYTHON $pythonvalidator -rule $rulefile -output_dir $tmpdir -wl "${workload}" \
+	mkdir -p "$tmpdir/legacy"
+	$PYTHON $pythonvalidator -rule $rulefile -output_dir "$tmpdir/legacy" -wl "${workload}" \
 		-cputype "${cputype}"
 	ret=$?
-	rm -rf $tmpdir
 	if [ $ret -ne 0 ]; then
-		echo "Metric validation return with errors. Please check metrics reported with errors."
+		echo "Metric validation return with errors. " \
+			"Please check metrics reported with errors in: " \
+			"$tmpdir/legacy"
+		exit $ret
+	fi
+
+	echo "Testing metrics for: $cputype (New API)"
+	mkdir -p "$tmpdir/new"
+	$PYTHON $pythonvalidator -rule $rulefile -output_dir "$tmpdir/new" -wl "${workload}" \
+		-cputype "${cputype}" -new
+	ret=$?
+	if [ $ret -ne 0 ]; then
+		echo "Metric validation return with errors (New API). " \
+			"Please check metrics reported with errors in: " \
+			"$tmpdir/new"
+		exit $ret
 	fi
 done
+rm -rf "$tmpdir"
 exit $ret
 
Signature · 1 line
2.55.0.141.g00534a21ce-goog
#95 lore raw

Re: [PATCH v4 14/14] perf stat: Extend PMU metrics value linter to validate --new outputs

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The test script intentionally bypasses temporary directory cleanup on failure to aid debugging…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The test script intentionally bypasses temporary directory cleanup on failure to aid debugging, but points the user to an empty directory because it fails to pass the -debug flag to the Python validator. --

--- Patch [14]: [PATCH v4 14/14] perf stat: Extend PMU metrics value linter to validate --new outputs --- commit aa043483360a1985a27fcbbfa6c36e6a6c2a5d00
Author: Ian Rogers <[email protected]>
perf stat: Extend PMU metrics value linter to validate --new outputs

This patch updates the stat_metrics_values.sh script to run the metrics value validation linter twice: once for the legacy path, and once under the --new print flag. It also adds early exits on validation failures so developers can inspect the output directory.

Quoted 4 lines
tools/perf/tests/shell/stat_metrics_values.sh +0 -0
index 30566f0b54279..9823b09589574 100755

[ ... ]

Quoted 6 lines
tools/perf/tests/shell/stat_metrics_values.sh +1 -1
@@ -19,13 +19,29 @@ echo "Output will be stored in: $tmpdir"
 for cputype in /sys/bus/event_source/devices/cpu_*; do
 	cputype=$(basename "$cputype")
 	echo "Testing metrics for: $cputype"
-	$PYTHON $pythonvalidator -rule $rulefile -output_dir $tmpdir -wl "${workload}" \
+	mkdir -p "$tmpdir/legacy"

tools/perf/tests/shell/stat_metrics_values.sh echo "Output will be stored in: $tmpdir" · line 30

Quoted 10 lines
tools/perf/tests/shell/stat_metrics_values.sh +5 -2
+	$PYTHON $pythonvalidator -rule $rulefile -output_dir "$tmpdir/legacy" -wl "${workload}" \
 		-cputype "${cputype}"
 	ret=$?
-	rm -rf $tmpdir
 	if [ $ret -ne 0 ]; then
-		echo "Metric validation return with errors. Please check metrics reported with errors."
+		echo "Metric validation return with errors. " \
+			"Please check metrics reported with errors in: " \
+			"$tmpdir/legacy"
+		exit $ret

[Severity: Medium]
Will this leave the user with an empty directory to inspect?

The script intentionally exits early on failure to bypass the temporary directory cleanup, directing the user to check "$tmpdir/legacy".

However, looking at Validator.create_report() in perf_metric_validation.py, the Python script only generates output files if the debug flag is provided:

tools/perf/tests/shell/lib/perf_metric_validation.py:Validator.create_report() {
    ...
    if self.debug:
        allres = [{"Workload": self.workloads[i], ... }]
        self.json_dump(allres, self.datafname)
    ...
}

Since -debug is not passed to the Python script in this patch, won't this result in an empty directory being leaked in /tmp while leaving the developer with no output data to inspect?

tools/perf/tests/shell/stat_metrics_values.sh echo "Output will be stored in: $tmpdir" · line 49

Quoted 12 lines
tools/perf/tests/shell/stat_metrics_values.sh +12 -0
+	fi
+
+	echo "Testing metrics for: $cputype (New API)"
+	mkdir -p "$tmpdir/new"
+	$PYTHON $pythonvalidator -rule $rulefile -output_dir "$tmpdir/new" -wl "${workload}" \
+		-cputype "${cputype}" -new
+	ret=$?
+	if [ $ret -ne 0 ]; then
+		echo "Metric validation return with errors (New API). " \
+			"Please check metrics reported with errors in: " \
+			"$tmpdir/new"
+		exit $ret

[Severity: Medium]
This appears to have the same issue as above. The script directs the user to inspect "$tmpdir/new", but without passing -debug to the pythonvalidator, won't this directory also be empty?

Quoted 4 lines
tools/perf/tests/shell/stat_metrics_values.sh +1 -0
 	fi
 done
+rm -rf "$tmpdir"
 exit $ret
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
maintainer
#96 lore raw

Re: [PATCH v4 00/14] perf stat: Decouple and modularize display formatting

In reply to Ian Rogers
Quoted 4 lines · Ian Rogers

This patch set decouples and modularizes perf stat metric and event counter display processing into a unified traversal engine supporting pluggable, format-specific callbacks for standard (STD), CSV, and JSON outputs via the new --new flag.

Thanks for working on this!

I'm not sure the goal is to produce the exactly same output or not. If so, we need to verify that by comparing outputs before and after. Probably you can use perf stat record and report.

If not, please show the output here.

Actually I cannot say I like the name of the flag, but it's ok if we want to keep it for a short period time and eventually get rid of it.

Thanks,
Namhyung

Quoted 58 lines

Changes in v4:
- Squash all multi-metric column padding, zero-skip logic, and formatting fixes directly into their introducing commits for a perfectly bisectable, pristine Git history.
- Avoid newlines before signed-off-by, assisted-by, and acked-by tags, organizing all contributor tags into a single, contiguous block for strict checkpatch and LKML compliance.
- Introduce strict, grid-based column padding to perfectly align multi-metric events (e.g. TopdownL1) across both standard and metric-only CSV rows, guaranteeing perfect 6-comma and 7-comma parsing compliance by external DataFrame engines.
- Refactor the PMU and hybrid metric event printer to propagate identical zero-counter skipping semantics in alignment with the legacy abs_printout path.
- Re-run all STD, JSON, and CSV linter test suites to achieve 100% passing status.

Ian Rogers (14):
perf stat: Introduce core generic print traversal engine and header
stubs
perf stat: Implement standard console (STD) formatting callbacks perf stat: Extend STD output linter to test basic New API checks perf stat: Extend STD output linter to test core aggregation checks perf stat: Extend STD output linter to test advanced PMU checks perf stat: Extend STD output linter to test metric-only checks perf stat: Implement CSV formatting callbacks
perf stat: Extend CSV output linter to test core aggregation checks perf stat: Extend CSV output linter to test advanced PMU and
metric-only checks
perf stat: Implement streaming JSON formatting callbacks perf stat: Extend JSON output linter to test core aggregation checks perf stat: Extend JSON output linter to test advanced PMU and
metric-only checks
perf stat: Add --new support to PMU metrics Python validator perf stat: Extend PMU metrics value linter to validate --new outputs

 tools/perf/builtin-stat.c                     |  96 +-
 .../tests/shell/lib/perf_metric_validation.py |  12 +-
 tools/perf/tests/shell/stat+csv_output.sh     |  19 +
 tools/perf/tests/shell/stat+json_output.sh    |  74 +-
 tools/perf/tests/shell/stat+std_output.sh     |  18 +
 tools/perf/tests/shell/stat_metrics_values.sh |  22 +-
 tools/perf/util/Build                         |   4 +
 tools/perf/util/stat-display.c                |  12 +-
 tools/perf/util/stat-print-csv.c              | 677 ++++++++++++++
 tools/perf/util/stat-print-json.c             | 339 +++++++
 tools/perf/util/stat-print-std.c              | 854 ++++++++++++++++++
 tools/perf/util/stat-print.c                  | 500 ++++++++++
 tools/perf/util/stat-print.h                  | 137 +++
 tools/perf/util/stat.h                        |   6 +
 14 files changed, 2688 insertions(+), 82 deletions(-)
 create mode 100644 tools/perf/util/stat-print-csv.c
 create mode 100644 tools/perf/util/stat-print-json.c
 create mode 100644 tools/perf/util/stat-print-std.c
 create mode 100644 tools/perf/util/stat-print.c
 create mode 100644 tools/perf/util/stat-print.h
Signature · 1 line
2.55.0.141.g00534a21ce-goog
author reviewer
#97 lore raw

Re: [PATCH v4 00/14] perf stat: Decouple and modularize display formatting

Quoted 9 lines · Namhyung Kim
Quoted 4 lines · Ian Rogers

This patch set decouples and modularizes perf stat metric and event counter display processing into a unified traversal engine supporting pluggable, format-specific callbacks for standard (STD), CSV, and JSON outputs via the new --new flag.

Thanks for working on this!

Quoted 8 lines

I'm not sure the goal is to produce the exactly same output or not. If so, we need to verify that by comparing outputs before and after. Probably you can use perf stat record and report.

If not, please show the output here.

Actually I cannot say I like the name of the flag, but it's ok if we want to keep it for a short period time and eventually get rid of it.

I don't like the flag name either :-) I wanted the output generated with --new to pass all the existing perf stat tests, which requires the output to be as close as possible. Only the json output changes things as the order of results matters less with json. The csv and std printing buffer events and metrics to make sure the output matches without relying on a particular printing an event, then printing a metric, etc. order from the caller. My hope in doing things this way was that we could land --new, another patch can make it the default, another patch series can delete the old code and another patch can remove the flag. Once we are happy with the --new version of the code we can change what CSV and std output looks like without worrying about the existing stat output code, etc.

Thanks,
Ian

Quoted 61 lines

Thanks,
Namhyung

Quoted 58 lines

Changes in v4:
- Squash all multi-metric column padding, zero-skip logic, and formatting fixes directly into their introducing commits for a perfectly bisectable, pristine Git history.
- Avoid newlines before signed-off-by, assisted-by, and acked-by tags, organizing all contributor tags into a single, contiguous block for strict checkpatch and LKML compliance.
- Introduce strict, grid-based column padding to perfectly align multi-metric events (e.g. TopdownL1) across both standard and metric-only CSV rows, guaranteeing perfect 6-comma and 7-comma parsing compliance by external DataFrame engines.
- Refactor the PMU and hybrid metric event printer to propagate identical zero-counter skipping semantics in alignment with the legacy abs_printout path.
- Re-run all STD, JSON, and CSV linter test suites to achieve 100% passing status.

Ian Rogers (14):
perf stat: Introduce core generic print traversal engine and header
stubs
perf stat: Implement standard console (STD) formatting callbacks perf stat: Extend STD output linter to test basic New API checks perf stat: Extend STD output linter to test core aggregation checks perf stat: Extend STD output linter to test advanced PMU checks perf stat: Extend STD output linter to test metric-only checks perf stat: Implement CSV formatting callbacks
perf stat: Extend CSV output linter to test core aggregation checks perf stat: Extend CSV output linter to test advanced PMU and
metric-only checks
perf stat: Implement streaming JSON formatting callbacks perf stat: Extend JSON output linter to test core aggregation checks perf stat: Extend JSON output linter to test advanced PMU and
metric-only checks
perf stat: Add --new support to PMU metrics Python validator perf stat: Extend PMU metrics value linter to validate --new outputs

 tools/perf/builtin-stat.c                     |  96 +-
 .../tests/shell/lib/perf_metric_validation.py |  12 +-
 tools/perf/tests/shell/stat+csv_output.sh     |  19 +
 tools/perf/tests/shell/stat+json_output.sh    |  74 +-
 tools/perf/tests/shell/stat+std_output.sh     |  18 +
 tools/perf/tests/shell/stat_metrics_values.sh |  22 +-
 tools/perf/util/Build                         |   4 +
 tools/perf/util/stat-display.c                |  12 +-
 tools/perf/util/stat-print-csv.c              | 677 ++++++++++++++
 tools/perf/util/stat-print-json.c             | 339 +++++++
 tools/perf/util/stat-print-std.c              | 854 ++++++++++++++++++
 tools/perf/util/stat-print.c                  | 500 ++++++++++
 tools/perf/util/stat-print.h                  | 137 +++
 tools/perf/util/stat.h                        |   6 +
 14 files changed, 2688 insertions(+), 82 deletions(-)
 create mode 100644 tools/perf/util/stat-print-csv.c
 create mode 100644 tools/perf/util/stat-print-json.c
 create mode 100644 tools/perf/util/stat-print-std.c
 create mode 100644 tools/perf/util/stat-print.c
 create mode 100644 tools/perf/util/stat-print.h

--
2.55.0.141.g00534a21ce-goog