JSON output for perf stat
Ian Rogers
reviewerv4 cover JSON output for perf stat
Parsing the CSV or text output of perf stat can be problematic when new output is added (columns in CSV format). JSON names values and simplifies the job of parsing. Add a JSON output option to perf-stat then add unit test that parses and validates the output.
This is a resend of two v2 patches:
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
with a few formatting changes and improvements to the linter.
The CSV test/linter is also added to ensure that CSV output doesn't regress:
https://lore.kernel.org/lkml/[email protected]/
v4. Does some minor fixes to the json linter.
v3. There is some tidy up of CSV code including a potential memory
over run in the os.nfields set up caught by sanitizers. To facilitate this an AGGR_MAX value is added. v3 also adds the CSV testing.
v2. Fixes the system wide no aggregation test to not run if the
paranoia is wrong. It also makes the counter-value check handle the "<not counted>" and "<not supported>" cases.
Claire Jensen (3):
perf test: Add checking for perf stat CSV output. perf stat: Add JSON output option
perf test: Json format checking
tools/perf/Documentation/perf-stat.txt | 21 + tools/perf/builtin-stat.c | 6 + .../tests/shell/lib/perf_csv_output_lint.py | 48 +++ .../tests/shell/lib/perf_json_output_lint.py | 94 +++++ tools/perf/tests/shell/stat+csv_output.sh | 147 +++++++ tools/perf/tests/shell/stat+json_output.sh | 147 +++++++ tools/perf/util/stat-display.c | 384 +++++++++++++----- tools/perf/util/stat.c | 1 + tools/perf/util/stat.h | 2 + 9 files changed, 744 insertions(+), 106 deletions(-) create mode 100644 tools/perf/tests/shell/lib/perf_csv_output_lint.py create mode 100644 tools/perf/tests/shell/lib/perf_json_output_lint.py create mode 100755 tools/perf/tests/shell/stat+csv_output.sh create mode 100755 tools/perf/tests/shell/stat+json_output.sh
Signature · 1 line
2.36.1.124.g0e6072fb45-goog
Ian Rogers
reviewerPATCH v4 1/3 perf test: Add checking for perf stat CSV output.
From: Claire Jensen <[email protected]>
Counts expected fields for various commands. No testing added for summary mode since it is broken.
An example of the summary output is:
summary,263831,,instructions:u,1435072,100.0,0.46,insn per cycle
,,,,,1.37,stalled cycles per insn
This should be:
summary,263831,,instructions:u,1435072,100.0,0.46,insn per cycle
summary,,,,,,1.37,stalled cycles per insnThe output has 7 fields when it should have 8. Additionally, the newline spacing is wrong, so it was excluded from testing until a fix is made.
--- .../tests/shell/lib/perf_csv_output_lint.py | 48 ++++++ tools/perf/tests/shell/stat+csv_output.sh | 147 ++++++++++++++++++ 2 files changed, 195 insertions(+) create mode 100644 tools/perf/tests/shell/lib/perf_csv_output_lint.py create mode 100755 tools/perf/tests/shell/stat+csv_output.sh
tools/perf/tests/shell/lib/perf_csv_output_lint.py new file +48 -0
@@ -0,0 +1,48 @@ +#!/usr/bin/python +# SPDX-License-Identifier: GPL-2.0 + +import argparse +import sys + +# Basic sanity check of perf CSV output as specified in the man page. +# Currently just checks the number of fields per line in output. + +ap = argparse.ArgumentParser() +ap.add_argument('--no-args', action='store_true') +ap.add_argument('--interval', action='store_true') +ap.add_argument('--system-wide-no-aggr', action='store_true') +ap.add_argument('--system-wide', action='store_true') +ap.add_argument('--event', action='store_true') +ap.add_argument('--per-core', action='store_true') +ap.add_argument('--per-thread', action='store_true') +ap.add_argument('--per-die', action='store_true') +ap.add_argument('--per-node', action='store_true') +ap.add_argument('--per-socket', action='store_true') +ap.add_argument('--separator', default=',', nargs='?') +args = ap.parse_args() + +Lines = sys.stdin.readlines() + +def check_csv_output(exp): + for line in Lines: + if 'failed' not in line: + count = line.count(args.separator) + if count != exp: + sys.stdout.write(''.join(Lines)) + raise RuntimeError(f'wrong number of fields. expected {exp} in {line}') + +try: + if args.no_args or args.system_wide or args.event: + expected_items = 6 + elif args.interval or args.per_thread or args.system_wide_no_aggr: + expected_items = 7 + elif args.per_core or args.per_socket or args.per_node or args.per_die: + expected_items = 8 + else: + ap.print_help() + raise RuntimeError('No checking option specified') + check_csv_output(expected_items) + +except: + sys.stdout.write('Test failed for input: ' + ''.join(Lines)) + raise
tools/perf/tests/shell/stat+csv_output.sh new file +147 -0
147 lines not on this page · raw message
Signature · 1 line
2.36.1.124.g0e6072fb45-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v4 1/3] perf test: Add checking for perf stat CSV output.
Em Tue, May 24, 2022 at 10:38:12PM -0700, Ian Rogers escreveu:
Quoted 10 lines
From: Claire Jensen <[email protected]>
Counts expected fields for various commands. No testing added for summary mode since it is broken.
An example of the summary output is:
summary,263831,,instructions:u,1435072,100.0,0.46,insn per cycle,,,,,1.37,stalled cycles per insn
Quoted 7 lines
This should be:
summary,263831,,instructions:u,1435072,100.0,0.46,insn per cycle summary,,,,,,1.37,stalled cycles per insnThe output has 7 fields when it should have 8. Additionally, the newline spacing is wrong, so it was excluded from testing until a fix is made.
s/parnoia/paranoid/ on the [Skip] lines, or am I missing something? :-)
I'll fix this up here.
- Arnaldo
Quoted 217 lines
Signed-off-by Claire Jensen <[email protected]>--- .../tests/shell/lib/perf_csv_output_lint.py | 48 ++++++ tools/perf/tests/shell/stat+csv_output.sh | 147 ++++++++++++++++++ 2 files changed, 195 insertions(+) create mode 100644 tools/perf/tests/shell/lib/perf_csv_output_lint.py create mode 100755 tools/perf/tests/shell/stat+csv_output.shtools/perf/tests/shell/lib/perf_csv_output_lint.py new file +48
-0@@ -0,0 +1,48 @@ +#!/usr/bin/python +# SPDX-License-Identifier: GPL-2.0 + +import argparse +import sys + +# Basic sanity check of perf CSV output as specified in the man page. +# Currently just checks the number of fields per line in output. + +ap = argparse.ArgumentParser() +ap.add_argument('--no-args', action='store_true') +ap.add_argument('--interval', action='store_true') +ap.add_argument('--system-wide-no-aggr', action='store_true') +ap.add_argument('--system-wide', action='store_true') +ap.add_argument('--event', action='store_true') +ap.add_argument('--per-core', action='store_true') +ap.add_argument('--per-thread', action='store_true') +ap.add_argument('--per-die', action='store_true') +ap.add_argument('--per-node', action='store_true') +ap.add_argument('--per-socket', action='store_true') +ap.add_argument('--separator', default=',', nargs='?') +args = ap.parse_args() + +Lines = sys.stdin.readlines() + +def check_csv_output(exp): + for line in Lines: + if 'failed' not in line: + count = line.count(args.separator) + if count != exp: + sys.stdout.write(''.join(Lines)) + raise RuntimeError(f'wrong number of fields. expected {exp} in {line}') + +try: + if args.no_args or args.system_wide or args.event: + expected_items = 6 + elif args.interval or args.per_thread or args.system_wide_no_aggr: + expected_items = 7 + elif args.per_core or args.per_socket or args.per_node or args.per_die: + expected_items = 8 + else: + ap.print_help() + raise RuntimeError('No checking option specified') + check_csv_output(expected_items) + +except: + sys.stdout.write('Test failed for input: ' + ''.join(Lines)) + raisetools/perf/tests/shell/stat+csv_output.sh new file +147
-0@@ -0,0 +1,147 @@ +#!/bin/bash +# perf stat CSV output linter +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +# Tests various perf stat CSV output commands for the +# correct number of fields and the CSV separator set to ','. + +set -e + +pythonchecker=$(dirname $0)/lib/perf_csv_output_lint.py +if [ "x$PYTHON" == "x" ] +then + if which python3 > /dev/null + then + PYTHON=python3 + elif which python > /dev/null + then + PYTHON=python + else + echo Skipping test, python not detected please set environment variable PYTHON. + exit 2 + fi +fi + +# Return true if perf_event_paranoid is > $1 and not running as root. +function ParanoidAndNotRoot() +{ + [ $(id -u) != 0 ] && [ $(cat /proc/sys/kernel/perf_event_paranoid) -gt $1 ] +} + +check_no_args() +{ + echo -n "Checking CSV output: no args " + perf stat -x, true 2>&1 | $PYTHON $pythonchecker --no-args + echo "[Success]" +} + +check_system_wide() +{ + echo -n "Checking CSV output: system wide " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -x, -a true 2>&1 | $PYTHON $pythonchecker --system-wide + echo "[Success]" +} + +check_system_wide_no_aggr() +{ + echo -n "Checking CSV output: system wide " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + echo -n "Checking CSV output: system wide no aggregation " + perf stat -x, -A -a --no-merge true 2>&1 | $PYTHON $pythonchecker --system-wide-no-aggr + echo "[Success]" +} + +check_interval() +{ + echo -n "Checking CSV output: interval " + perf stat -x, -I 1000 true 2>&1 | $PYTHON $pythonchecker --interval + echo "[Success]" +} + + +check_event() +{ + echo -n "Checking CSV output: event " + perf stat -x, -e cpu-clock true 2>&1 | $PYTHON $pythonchecker --event + echo "[Success]" +} + +check_per_core() +{ + echo -n "Checking CSV output: per core " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -x, --per-core -a true 2>&1 | $PYTHON $pythonchecker --per-core + echo "[Success]" +} + +check_per_thread() +{ + echo -n "Checking CSV output: per thread " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -x, --per-thread -a true 2>&1 | $PYTHON $pythonchecker --per-thread + echo "[Success]" +} + +check_per_die() +{ + echo -n "Checking CSV output: per die " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -x, --per-die -a true 2>&1 | $PYTHON $pythonchecker --per-die + echo "[Success]" +} + +check_per_node() +{ + echo -n "Checking CSV output: per node " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -x, --per-node -a true 2>&1 | $PYTHON $pythonchecker --per-node + echo "[Success]" +} + +check_per_socket() +{ + echo -n "Checking CSV output: per socket " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -x, --per-socket -a true 2>&1 | $PYTHON $pythonchecker --per-socket + echo "[Success]" +} + +check_no_args +check_system_wide +check_system_wide_no_aggr +check_interval +check_event +check_per_core +check_per_thread +check_per_die +check_per_node +check_per_socket +exit 0Signature · 1 line
2.36.1.124.g0e6072fb45-goog
Signature · 1 line
- Arnaldo
Ian Rogers
reviewerPATCH v4 2/3 perf stat: Add JSON output option
In reply to Ian RogersFrom: Claire Jensen <[email protected]>
CSV output is tricky to format and column layout changes are susceptible to breaking parsers. New JSON-formatted output has variable names to identify fields that are consistent and informative, making the output parseable.
CSV output example:
1.20,msec,task-clock:u,1204272,100.00,0.697,CPUs utilized 0,,context-switches:u,1204272,100.00,0.000,/sec 0,,cpu-migrations:u,1204272,100.00,0.000,/sec 70,,page-faults:u,1204272,100.00,58.126,K/sec
JSON output example:
{"counter-value" : "3805.723968", "unit" : "msec", "event" :
"cpu-clock", "event-runtime" : 3805731510100.00, "pcnt-running" : 100.00, "metric-value" : 4.007571, "metric-unit" : "CPUs utilized"} {"counter-value" : "6166.000000", "unit" : "", "event" :
"context-switches", "event-runtime" : 3805723045100.00, "pcnt-running" : 100.00, "metric-value" : 1.620191, "metric-unit" : "K/sec"} {"counter-value" : "466.000000", "unit" : "", "event" :
"cpu-migrations", "event-runtime" : 3805727613100.00, "pcnt-running" : 100.00, "metric-value" : 122.447136, "metric-unit" : "/sec"} {"counter-value" : "208.000000", "unit" : "", "event" :
"page-faults", "event-runtime" : 3805726799100.00, "pcnt-running" : 100.00, "metric-value" : 54.654516, "metric-unit" : "/sec"}
Also added documentation for JSON option.
There is some tidy up of CSV code including a potential memory over run in the os.nfields set up. To facilitate this an AGGR_MAX value is added.
--- tools/perf/Documentation/perf-stat.txt | 21 ++ tools/perf/builtin-stat.c | 6 + tools/perf/util/stat-display.c | 384 ++++++++++++++++++------- tools/perf/util/stat.c | 1 + tools/perf/util/stat.h | 2 + 5 files changed, 308 insertions(+), 106 deletions(-)
tools/perf/Documentation/perf-stat.txt +21 -0
@@ -570,6 +570,27 @@ Additional metrics may be printed with all earlier fields being empty. include::intel-hybrid.txt[] +JSON FORMAT +----------- + +With -j, perf stat is able to print out a JSON format output +that can be used for parsing. + +- timestamp : optional usec time stamp in fractions of second (with -I) +- optional aggregate options: + - core : core identifier (with --per-core) + - die : die identifier (with --per-die) + - socket : socket identifier (with --per-socket) + - node : node identifier (with --per-node) + - thread : thread identifier (with --per-thread) +- counter-value : counter value +- unit : unit of the counter value or empty +- event : event name +- variance : optional variance if multiple values are collected (with -r) +- runtime : run time of counter +- metric-value : optional metric value +- metric-unit : optional unit of metric + SEE ALSO -------- linkperf:perf-top[1], linkperf:perf-list[1]
tools/perf/builtin-stat.c +6 -0
@@ -1259,6 +1259,8 @@ static struct option stat_options[] = { "Merge identical named hybrid events"),OPT_STRING ('x' ,"field-separator" , &stat_config.csv_sep,"separator" ,"print counts with custom separator" ), +OPT_BOOLEAN ('j' ,"json-output" , &stat_config.json_output, +"print counts in JSON format" ),OPT_CALLBACK ('G' ,"cgroup" , &evsel_list,"name" ,"monitor event in cgroup name only" , parse_stat_cgroups), OPT_STRING(0, "for-each-cgroup", &stat_config.cgroup_list, "name",
@@ -1445,6 +1447,7 @@ static aggr_cpu_id_get_t aggr_mode__get_aggr(enum aggr_mode aggr_mode)case AGGR_GLOBAL :case AGGR_THREAD :case AGGR_UNSET : +case AGGR_MAX :default :return NULL ; }
@@ -1469,6 +1472,7 @@ static aggr_get_id_t aggr_mode__get_id(enum aggr_mode aggr_mode)case AGGR_GLOBAL :case AGGR_THREAD :case AGGR_UNSET : +case AGGR_MAX :default :return NULL ; }
@@ -1619,6 +1623,7 @@ static aggr_cpu_id_get_t aggr_mode__get_aggr_file(enum aggr_mode aggr_mode)case AGGR_GLOBAL :case AGGR_THREAD :case AGGR_UNSET : +case AGGR_MAX :default :return NULL ; }
@@ -1639,6 +1644,7 @@ static aggr_get_id_t aggr_mode__get_id_file(enum aggr_mode aggr_mode)case AGGR_GLOBAL :case AGGR_THREAD :case AGGR_UNSET : +case AGGR_MAX :default :return NULL ; }
tools/perf/util/stat-display.c +278 -106
556 lines not on this page · raw message
tools/perf/util/stat.c +1 -0
@@ -401,6 +401,7 @@ process_counter_values(struct perf_stat_config *config, struct evsel *evsel, aggr->ena += count->ena; aggr->run += count->run;case AGGR_UNSET : +case AGGR_MAX :default :break ; }
tools/perf/util/stat.h +2 -0
@@ -57,6 +57,7 @@ enum aggr_mode {AGGR_THREAD ,AGGR_UNSET ,AGGR_NODE , +AGGR_MAX }; enum {
@@ -121,6 +122,7 @@ struct perf_stat_config {bool no_inherit;bool identifier;bool csv_output; +bool json_output;bool interval_clear;bool metric_only;bool null_run;
Signature · 1 line
2.36.1.124.g0e6072fb45-goog
Re: [PATCH v4 2/3] perf stat: Add JSON output option
Hi Ian,
Quoted 30 lines · Ian Rogers
From: Claire Jensen <[email protected]>
CSV output is tricky to format and column layout changes are susceptible to breaking parsers. New JSON-formatted output has variable names to identify fields that are consistent and informative, making the output parseable.
CSV output example:
1.20,msec,task-clock:u,1204272,100.00,0.697,CPUs utilized 0,,context-switches:u,1204272,100.00,0.000,/sec 0,,cpu-migrations:u,1204272,100.00,0.000,/sec 70,,page-faults:u,1204272,100.00,58.126,K/sec
JSON output example:
{"counter-value" : "3805.723968", "unit" : "msec", "event" :
"cpu-clock", "event-runtime" : 3805731510100.00, "pcnt-running" : 100.00, "metric-value" : 4.007571, "metric-unit" : "CPUs utilized"} {"counter-value" : "6166.000000", "unit" : "", "event" :
"context-switches", "event-runtime" : 3805723045100.00, "pcnt-running" : 100.00, "metric-value" : 1.620191, "metric-unit" : "K/sec"} {"counter-value" : "466.000000", "unit" : "", "event" :
"cpu-migrations", "event-runtime" : 3805727613100.00, "pcnt-running" : 100.00, "metric-value" : 122.447136, "metric-unit" : "/sec"} {"counter-value" : "208.000000", "unit" : "", "event" :
"page-faults", "event-runtime" : 3805726799100.00, "pcnt-running" : 100.00, "metric-value" : 54.654516, "metric-unit" : "/sec"}
Quoted 5 lines
Also added documentation for JSON option.
There is some tidy up of CSV code including a potential memory over run in the os.nfields set up. To facilitate this anAGGR_MAXvalue is added.Signed-off-by Claire Jensen <[email protected]>
Your sign-off as well?
Anyway, I think there are places to clean up this part of code more but that's not a part of the work. Maybe I need to find some time to do that later.
Thanks,
Namhyung
Ian Rogers
reviewerRe: [PATCH v4 2/3] perf stat: Add JSON output option
Quoted 34 lines · Namhyung Kim
Hi Ian,
Quoted 30 lines · Ian Rogers
From: Claire Jensen <[email protected]>
CSV output is tricky to format and column layout changes are susceptible to breaking parsers. New JSON-formatted output has variable names to identify fields that are consistent and informative, making the output parseable.
CSV output example:
1.20,msec,task-clock:u,1204272,100.00,0.697,CPUs utilized 0,,context-switches:u,1204272,100.00,0.000,/sec 0,,cpu-migrations:u,1204272,100.00,0.000,/sec 70,,page-faults:u,1204272,100.00,58.126,K/sec
JSON output example:
{"counter-value" : "3805.723968", "unit" : "msec", "event" :
"cpu-clock", "event-runtime" : 3805731510100.00, "pcnt-running" : 100.00, "metric-value" : 4.007571, "metric-unit" : "CPUs utilized"} {"counter-value" : "6166.000000", "unit" : "", "event" :
"context-switches", "event-runtime" : 3805723045100.00, "pcnt-running" : 100.00, "metric-value" : 1.620191, "metric-unit" : "K/sec"} {"counter-value" : "466.000000", "unit" : "", "event" :
"cpu-migrations", "event-runtime" : 3805727613100.00, "pcnt-running" : 100.00, "metric-value" : 122.447136, "metric-unit" : "/sec"} {"counter-value" : "208.000000", "unit" : "", "event" :
"page-faults", "event-runtime" : 3805726799100.00, "pcnt-running" : 100.00, "metric-value" : 54.654516, "metric-unit" : "/sec"}
Quoted 7 lines
Quoted 5 lines
Also added documentation for JSON option.
There is some tidy up of CSV code including a potential memory over run in the os.nfields set up. To facilitate this anAGGR_MAXvalue is added.Signed-off-by Claire Jensen <[email protected]>Your sign-off as well?
Doh:
Thanks,
Ian
Quoted 8 lines
Anyway, I think there are places to clean up this part of code more but that's not a part of the work. Maybe I need to find some time to do that later.
Acked-by Namhyung Kim <[email protected]>Thanks,
Namhyung
Ian Rogers
reviewerRe: [PATCH v4 2/3] perf stat: Add JSON output option
Quoted 42 lines · Ian Rogers
Quoted 40 lines · Namhyung Kim
Hi Ian,
Quoted 35 lines · Ian Rogers
From: Claire Jensen <[email protected]>
CSV output is tricky to format and column layout changes are susceptible to breaking parsers. New JSON-formatted output has variable names to identify fields that are consistent and informative, making the output parseable.
CSV output example:
1.20,msec,task-clock:u,1204272,100.00,0.697,CPUs utilized 0,,context-switches:u,1204272,100.00,0.000,/sec 0,,cpu-migrations:u,1204272,100.00,0.000,/sec 70,,page-faults:u,1204272,100.00,58.126,K/sec
JSON output example:
{"counter-value" : "3805.723968", "unit" : "msec", "event" :
"cpu-clock", "event-runtime" : 3805731510100.00, "pcnt-running" : 100.00, "metric-value" : 4.007571, "metric-unit" : "CPUs utilized"} {"counter-value" : "6166.000000", "unit" : "", "event" :
"context-switches", "event-runtime" : 3805723045100.00, "pcnt-running" : 100.00, "metric-value" : 1.620191, "metric-unit" : "K/sec"} {"counter-value" : "466.000000", "unit" : "", "event" :
"cpu-migrations", "event-runtime" : 3805727613100.00, "pcnt-running" : 100.00, "metric-value" : 122.447136, "metric-unit" : "/sec"} {"counter-value" : "208.000000", "unit" : "", "event" :
"page-faults", "event-runtime" : 3805726799100.00, "pcnt-running" : 100.00, "metric-value" : 54.654516, "metric-unit" : "/sec"}Also added documentation for JSON option.
There is some tidy up of CSV code including a potential memory over run in the os.nfields set up. To facilitate this anAGGR_MAXvalue is added.Signed-off-by Claire Jensen <[email protected]>
Quoted 7 lines
Quoted 1 line
Your sign-off as well?
Doh:
Signed-off-by Ian Rogers <[email protected]>Thanks,
Ian
Actually, this was fixed in V5:
https://lore.kernel.org/lkml/[email protected]/
Thanks,
Ian
Quoted 8 lines
Quoted 8 lines
Anyway, I think there are places to clean up this part of code more but that's not a part of the work. Maybe I need to find some time to do that later.
Acked-by Namhyung Kim <[email protected]>Thanks,
Namhyung
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v4 2/3] perf stat: Add JSON output option
In reply to Ian RogersEm Tue, May 31, 2022 at 04:13:59PM -0700, Ian Rogers escreveu:
Quoted 39 lines · Namhyung Kim
Quoted 38 lines
Hi Ian,
Quoted 34 lines · Ian Rogers
From: Claire Jensen <[email protected]>
CSV output is tricky to format and column layout changes are susceptible to breaking parsers. New JSON-formatted output has variable names to identify fields that are consistent and informative, making the output parseable.
CSV output example:
1.20,msec,task-clock:u,1204272,100.00,0.697,CPUs utilized 0,,context-switches:u,1204272,100.00,0.000,/sec 0,,cpu-migrations:u,1204272,100.00,0.000,/sec 70,,page-faults:u,1204272,100.00,58.126,K/sec
JSON output example:
{"counter-value" : "3805.723968", "unit" : "msec", "event" :
"cpu-clock", "event-runtime" : 3805731510100.00, "pcnt-running" : 100.00, "metric-value" : 4.007571, "metric-unit" : "CPUs utilized"} {"counter-value" : "6166.000000", "unit" : "", "event" :
"context-switches", "event-runtime" : 3805723045100.00, "pcnt-running" : 100.00, "metric-value" : 1.620191, "metric-unit" : "K/sec"} {"counter-value" : "466.000000", "unit" : "", "event" :
"cpu-migrations", "event-runtime" : 3805727613100.00, "pcnt-running" : 100.00, "metric-value" : 122.447136, "metric-unit" : "/sec"} {"counter-value" : "208.000000", "unit" : "", "event" :
"page-faults", "event-runtime" : 3805726799100.00, "pcnt-running" : 100.00, "metric-value" : 54.654516, "metric-unit" : "/sec"}Also added documentation for JSON option.
There is some tidy up of CSV code including a potential memory over run in the os.nfields set up. To facilitate this anAGGR_MAXvalue is added.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Claire Jensen <[email protected]>Your sign-off as well?
Doh:
Signed-off-by Ian Rogers <[email protected]>
Added it (b4 did it really) and also added the fixup below.
- Arnaldo
tools/perf/util/stat-display.c +1 -1
@@ -35,7 +35,7 @@ static void print_running(struct perf_stat_config *config, enabled_percent =100 * run / ena;if (config->json_output)fprintf (config->output, -"\"event-runtime\" : %lu, \"pcnt-running\" : %.2f, " , +"\"event-runtime\" : %" PRIu64", \"pcnt-running\" : %.2f, " , run, enabled_percent);else if (config->csv_output) fprintf(config->output,
Ian Rogers
reviewerPATCH v4 3/3 perf test: Json format checking
In reply to Ian RogersFrom: Claire Jensen <[email protected]>
Add field checking tests for perf stat JSON output. Sanity checks the expected number of fields are present, that the expected keys are present and they have the correct values.
--- .../tests/shell/lib/perf_json_output_lint.py | 94 +++++++++++ tools/perf/tests/shell/stat+json_output.sh | 147 ++++++++++++++++++ 2 files changed, 241 insertions(+) create mode 100644 tools/perf/tests/shell/lib/perf_json_output_lint.py create mode 100755 tools/perf/tests/shell/stat+json_output.sh
tools/perf/tests/shell/lib/perf_json_output_lint.py new file +94 -0
@@ -0,0 +1,94 @@ +#!/usr/bin/python +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +# Basic sanity check of perf JSON output as specified in the man page. + +import argparse +import sys +import json + +ap = argparse.ArgumentParser() +ap.add_argument('--no-args', action='store_true') +ap.add_argument('--interval', action='store_true') +ap.add_argument('--system-wide-no-aggr', action='store_true') +ap.add_argument('--system-wide', action='store_true') +ap.add_argument('--event', action='store_true') +ap.add_argument('--per-core', action='store_true') +ap.add_argument('--per-thread', action='store_true') +ap.add_argument('--per-die', action='store_true') +ap.add_argument('--per-node', action='store_true') +ap.add_argument('--per-socket', action='store_true') +args = ap.parse_args() + +Lines = sys.stdin.readlines() + +def isfloat(num): + try: + float(num) + return True + except ValueError: + return False + + +def isint(num): + try: + int(num) + return True + except ValueError: + return False + +def is_counter_value(num): + return isfloat(num) or num == '<not counted>' or num == '<not supported>' + +def check_json_output(expected_items): + if expected_items != -1: + for line in Lines: + if 'failed' not in line: + count = 0 + count = line.count(',') + if count != expected_items and count == 1 and 'metric-value' in line: + # Events that generate >1 metric may have isolated metric values. + continue + if count != expected_items: + raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}' + f' in \'{line}\'') + checks = { + 'aggregate-number': lambda x: isfloat(x), + 'core': lambda x: True, + 'counter-value': lambda x: is_counter_value(x), + 'cgroup': lambda x: True, + 'cpu': lambda x: isint(x), + 'die': lambda x: True, + 'event': lambda x: True, + 'event-runtime': lambda x: isfloat(x), + 'interval': lambda x: isfloat(x), + 'metric-unit': lambda x: True, + 'metric-value': lambda x: isfloat(x), + 'node': lambda x: True, + 'pcnt-running': lambda x: isfloat(x), + 'socket': lambda x: True, + 'thread': lambda x: True, + 'unit': lambda x: True, + } + input = '[\n' + ','.join(Lines) + '\n]' + for item in json.loads(input): + for key, value in item.items(): + if key not in checks: + raise RuntimeError(f'Unexpected key: key={key} value={value}') + if not checks[key](value): + raise RuntimeError(f'Check failed for: key={key} value={value}') + + +try: + if args.no_args or args.system_wide or args.event: + expected_items = 6 + elif args.interval or args.per_thread or args.system_wide_no_aggr: + expected_items = 7 + elif args.per_core or args.per_socket or args.per_node or args.per_die: + expected_items = 8 + else: + # If no option is specified, don't check the number of items. + expected_items = -1 + check_json_output(expected_items) +except: + print('Test failed for input:\n' + '\n'.join(Lines)) + raise
tools/perf/tests/shell/stat+json_output.sh new file +147 -0
147 lines not on this page · raw message
Signature · 1 line
2.36.1.124.g0e6072fb45-goog
Arnaldo Carvalho de Melo
maintainerEm Tue, May 24, 2022 at 10:38:11PM -0700, Ian Rogers escreveu:
Quoted 5 lines
Parsing the CSV or text output of perf stat can be problematic when new output is added (columns in CSV format). JSON names values and simplifies the job of parsing. Add a JSON output option to perf-stat then add unit test that parses and validates the output.
Quoted 7 lines
This is a resend of two v2 patches:
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
with a few formatting changes and improvements to the linter.The CSV test/linter is also added to ensure that CSV output doesn't regress:
https://lore.kernel.org/lkml/[email protected]/
So, the JSON test is failing:
⬢[acme@toolbox perf]$ perf test -v JSON
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
90: perf stat JSON output linter :
--- start ---
test child forked, pid 2626229
Checking json output: no args [Success]
Checking json output: system wide [Skip] parnoia and not root
Checking json output: system wide [Skip] parnoia and not root
Checking json output: interval Test failed for input:
{"interval" : 0.000506453, "counter-value" : "0.212360", "unit" : "msec", "event" : "task-clock:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 0.000212, "metric-unit" : "CPUs utilized"}
{"interval" : 0.000506453, "counter-value" : "0.000000", "unit" : "", "event" : "context-switches:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 0.000000, "metric-unit" : "/sec"}
{"interval" : 0.000506453, "counter-value" : "0.000000", "unit" : "", "event" : "cpu-migrations:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 0.000000, "metric-unit" : "/sec"}
{"interval" : 0.000506453, "counter-value" : "45.000000", "unit" : "", "event" : "page-faults:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 211.904313, "metric-unit" : "K/sec"}
{"interval" : 0.000506453, "counter-value" : "143761.000000", "unit" : "", "event" : "cycles:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 0.676968, "metric-unit" : "GHz"}
{"interval" : 0.000506453, "counter-value" : "456.000000", "unit" : "", "event" : "stalled-cycles-frontend:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 0.317193, "metric-unit" : "frontend cycles idle"}
{"interval" : 0.000506453, "counter-value" : "11639.000000", "unit" : "", "event" : "stalled-cycles-backend:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 8.096076, "metric-unit" : "backend cycles idle"}
{"interval" : 0.000506453, "counter-value" : "150684.000000", "unit" : "", "event" : "instructions:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 1.048156, "metric-unit" : "insn per cycle"}
{"interval" : 0.000506453, "metric-value" : 0.077241, "metric-unit" : "stalled cycles per insn"}
{"interval" : 0.000506453, "counter-value" : "29735.000000", "unit" : "", "event" : "branches:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 140.021661, "metric-unit" : "M/sec"}
{"interval" : 0.000506453, "counter-value" : "<not counted>", "unit" : "", "event" : "branch-misses:u", "event-runtime" : 0, "pcnt-running" : 0.00, "metric-value" : 0.000000, "metric-unit" : ""}
Traceback (most recent call last):
File "/var/home/acme/git/perf/./tools/perf/tests/shell/lib/perf_json_output_lint.py", line 91, in <module>
check_json_output(expected_items)
File "/var/home/acme/git/perf/./tools/perf/tests/shell/lib/perf_json_output_lint.py", line 52, in check_json_output
raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}'
RuntimeError: wrong number of fields. counted 2 expected 7 in '{"interval" : 0.000506453, "metric-value" : 0.077241, "metric-unit" : "stalled cycles per insn"} '
test child finished with -1
---- end ----
perf stat JSON output linter: FAILED!
⬢[acme@toolbox perf]$
So please check this and resubmit.
My system is a fedora 35 silverblue toolbox.
⬢[acme@toolbox perf]$ rpm -q python3
python3-3.10.4-1.fc35.x86_64
- Arnaldo
Arnaldo Carvalho de Melo
maintainerEm Wed, May 25, 2022 at 08:18:27AM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 59 lines
Em Tue, May 24, 2022 at 10:38:11PM -0700, Ian Rogers escreveu:
Quoted 5 lines
Parsing the CSV or text output of perf stat can be problematic when new output is added (columns in CSV format). JSON names values and simplifies the job of parsing. Add a JSON output option to perf-stat then add unit test that parses and validates the output.
Quoted 7 lines
This is a resend of two v2 patches:
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
with a few formatting changes and improvements to the linter.The CSV test/linter is also added to ensure that CSV output doesn't regress:
https://lore.kernel.org/lkml/[email protected]/So, the JSON test is failing:
⬢[acme@toolbox perf]$ perf test -v JSON
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
90: perf stat JSON output linter :
--- start ---
test child forked, pid 2626229
Checking json output: no args [Success]
Checking json output: system wide [Skip] parnoia and not root
Checking json output: system wide [Skip] parnoia and not root
Checking json output: interval Test failed for input:
{"interval" : 0.000506453, "counter-value" : "0.212360", "unit" : "msec", "event" : "task-clock:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 0.000212, "metric-unit" : "CPUs utilized"}{"interval" : 0.000506453, "counter-value" : "0.000000", "unit" : "", "event" : "context-switches:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 0.000000, "metric-unit" : "/sec"}
{"interval" : 0.000506453, "counter-value" : "0.000000", "unit" : "", "event" : "cpu-migrations:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 0.000000, "metric-unit" : "/sec"}
{"interval" : 0.000506453, "counter-value" : "45.000000", "unit" : "", "event" : "page-faults:u", "event-runtime" : 212360, "pcnt-running" : 100.00, "metric-value" : 211.904313, "metric-unit" : "K/sec"}
{"interval" : 0.000506453, "counter-value" : "143761.000000", "unit" : "", "event" : "cycles:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 0.676968, "metric-unit" : "GHz"}
{"interval" : 0.000506453, "counter-value" : "456.000000", "unit" : "", "event" : "stalled-cycles-frontend:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 0.317193, "metric-unit" : "frontend cycles idle"}
{"interval" : 0.000506453, "counter-value" : "11639.000000", "unit" : "", "event" : "stalled-cycles-backend:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 8.096076, "metric-unit" : "backend cycles idle"}
{"interval" : 0.000506453, "counter-value" : "150684.000000", "unit" : "", "event" : "instructions:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 1.048156, "metric-unit" : "insn per cycle"}
{"interval" : 0.000506453, "metric-value" : 0.077241, "metric-unit" : "stalled cycles per insn"}
{"interval" : 0.000506453, "counter-value" : "29735.000000", "unit" : "", "event" : "branches:u", "event-runtime" : 217290, "pcnt-running" : 100.00, "metric-value" : 140.021661, "metric-unit" : "M/sec"}
{"interval" : 0.000506453, "counter-value" : "<not counted>", "unit" : "", "event" : "branch-misses:u", "event-runtime" : 0, "pcnt-running" : 0.00, "metric-value" : 0.000000, "metric-unit" : ""}
Traceback (most recent call last):
File "/var/home/acme/git/perf/./tools/perf/tests/shell/lib/perf_json_output_lint.py", line 91, in <module>
check_json_output(expected_items)
File "/var/home/acme/git/perf/./tools/perf/tests/shell/lib/perf_json_output_lint.py", line 52, incheck_json_output
raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}'
RuntimeError: wrong number of fields. counted 2 expected 7 in '{"interval" : 0.000506453, "metric-value" : 0.077241, "metric-unit" : "stalled cycles per insn"} '
test child finished with -1
---- end ----
perf stat JSON output linter: FAILED!
⬢[acme@toolbox perf]$
Quoted 1 line
So please check this and resubmit.
I kept the first patch in the series, so please just check the JSON ones.
- Arnaldo
Quoted 6 lines
My system is a fedora 35 silverblue toolbox.
⬢[acme@toolbox perf]$ rpm -q python3
python3-3.10.4-1.fc35.x86_64- Arnaldo
Signature · 1 line
- Arnaldo