perf test: Assorted fixes and updates
Namhyung Kim
maintainercover perf test: Assorted fixes and updates
Hello,
This is a list of random fixes and updates in perf test. I've added a new test case for perf trace and updated perf stat output test to check the --metric-only option. Also make perf probe and trace tests for non-root users as they need priviledged operations.
Maybe we need to fix perf stat --metric-only output not to include bogus metrics like GHz, but let's keep it for later. :)
Thanks,
Namhyung
Namhyung Kim (4):
perf test: Add --metric-only to perf stat output tests perf test: Skip perf probe tests when running as non-root perf test: Skip perf trace tests when running as non-root perf test: Add trace record and replay test
.../tests/shell/lib/perf_json_output_lint.py | 7 +++++++ tools/perf/tests/shell/lib/stat_output.sh | 8 +++++++ .../tests/shell/perftool-testsuite_probe.sh | 1 + tools/perf/tests/shell/probe_vfs_getname.sh | 1 + .../shell/record+probe_libc_inet_pton.sh | 1 + .../shell/record+script_probe_vfs_getname.sh | 1 + tools/perf/tests/shell/stat+csv_output.sh | 2 ++ tools/perf/tests/shell/stat+json_output.sh | 9 ++++++++ tools/perf/tests/shell/stat+std_output.sh | 8 +++++++ .../shell/test_uprobe_from_different_cu.sh | 11 +++++----- .../tests/shell/trace+probe_vfs_getname.sh | 1 + tools/perf/tests/shell/trace_btf_enum.sh | 1 + tools/perf/tests/shell/trace_btf_general.sh | 1 + tools/perf/tests/shell/trace_exit_race.sh | 1 + tools/perf/tests/shell/trace_record_replay.sh | 21 +++++++++++++++++++ 15 files changed, 68 insertions(+), 6 deletions(-) create mode 100755 tools/perf/tests/shell/trace_record_replay.sh
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Namhyung Kim
maintainerPATCH 1/4 perf test: Add --metric-only to perf stat output tests
Add a test case for --metric-only for std, csv, json output mode using shadow IPC metric from instructions and cycles events. It should produce 'insn per cycle' metric.
But currently JSON output has (none) 'GHz' as well. It looks like a bug but I don't have enough time to debug it for now so I made it pass. :(
$ perf stat --metric-only -e instructions,cycles true
Performance counter stats for 'true':
0.56
0.002127319 seconds time elapsed
0.002077000 seconds user
0.000000000 seconds sys$ perf stat -x, --metric-only -e instructions,cycles true
0.55,,
$ perf stat -j --metric-only -e instructions,cycles true
{"insn per cycle" : "0.53", "GHz" : "none"} $ perf test output -v
5: Test data source output : Ok
31: Sort output of hist entries : Ok
88: perf stat CSV output linter : Ok
90: perf stat JSON output linter : Ok
92: perf stat STD output linter : Ok--- tools/perf/tests/shell/lib/perf_json_output_lint.py | 7 +++++++ tools/perf/tests/shell/lib/stat_output.sh | 8 ++++++++ tools/perf/tests/shell/stat+csv_output.sh | 2 ++ tools/perf/tests/shell/stat+json_output.sh | 9 +++++++++ tools/perf/tests/shell/stat+std_output.sh | 8 ++++++++ 5 files changed, 34 insertions(+)
tools/perf/tests/shell/lib/perf_json_output_lint.py +7 -0
@@ -19,6 +19,7 @@ ap.add_argument('--per-cluster', action='store_true') ap.add_argument('--per-die', action='store_true') ap.add_argument('--per-node', action='store_true') ap.add_argument('--per-socket', action='store_true') +ap.add_argument('--metric-only', action='store_true') ap.add_argument('--file', type=argparse.FileType('r'), default=sys.stdin) args = ap.parse_args()
@@ -64,6 +65,8 @@ Lines = args.file.readlines() 'socket': lambda x: True, 'thread': lambda x: True, 'unit': lambda x: True, + 'insn per cycle': lambda x: isfloat(x), + 'GHz': lambda x: True, # FIXME: it seems unintended for --metric-only } input = '[\n' + ','.join(Lines) + '\n]' for item in json.loads(input):
@@ -78,6 +81,8 @@ Lines = args.file.readlines() pass elif count - 1 in expected_items and 'metric-threshold' in item: pass + elif count in expected_items and 'insn per cycle' in item: + pass elif count not in expected_items: raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}' f' in \'{item}\'')
@@ -95,6 +100,8 @@ Lines = args.file.readlines() expected_items = [6, 8] elif args.per_core or args.per_socket or args.per_node or args.per_die or args.per_cluster or args.per_cache: expected_items = [7, 9] + elif args.metric_only: + expected_items = [1, 2] else: # If no option is specified, don't check the number of items. expected_items = -1
tools/perf/tests/shell/lib/stat_output.sh +8 -0
@@ -148,6 +148,14 @@ check_per_socket() echo "[Success]" } +check_metric_only() +{ + echo -n "Checking $1 output: metric only " + perf stat --metric-only $2 -e instructions,cycles true + commachecker --metric-only + echo "[Success]" +} + # The perf stat options for per-socket, per-core, per-die # and -A ( no_aggr mode ) uses the info fetched from this # directory: "/sys/devices/system/cpu/cpu*/topology". For
tools/perf/tests/shell/stat+csv_output.sh +2 -0
@@ -44,6 +44,7 @@ function commachecker() ;; "--per-die") exp=8 ;; "--per-cluster") exp=8 ;; "--per-cache") exp=8 + ;; "--metric-only") exp=2 esac while read line
@@ -75,6 +76,7 @@ check_interval "CSV" "$perf_cmd" check_event "CSV" "$perf_cmd" check_per_thread "CSV" "$perf_cmd" check_per_node "CSV" "$perf_cmd" +check_metric_only "CSV" "$perf_cmd" if [ $skip_test -ne 1 ] then check_system_wide_no_aggr "CSV" "$perf_cmd"
tools/perf/tests/shell/stat+json_output.sh +9 -0
@@ -173,6 +173,14 @@ check_per_socket() echo "[Success]" } +check_metric_only() +{ + echo -n "Checking json output: metric only " + perf stat -j --metric-only -e instructions,cycles -o "${stat_output}" true + $PYTHON $pythonchecker --metric-only --file "${stat_output}" + echo "[Success]" +} + # The perf stat options for per-socket, per-core, per-die # and -A ( no_aggr mode ) uses the info fetched from this # directory: "/sys/devices/system/cpu/cpu*/topology". For
@@ -207,6 +215,7 @@ check_interval check_event check_per_thread check_per_node +check_metric_only if [ $skip_test -ne 1 ] then check_system_wide_no_aggr
tools/perf/tests/shell/stat+std_output.sh +8 -0
@@ -30,6 +30,7 @@ trap trap_cleanup EXIT TERM INT function commachecker() { local prefix=1 + local -i metric_only=0 case "$1" in "--interval") prefix=2
@@ -41,6 +42,7 @@ function commachecker() ;; "--per-die") prefix=3 ;; "--per-cache") prefix=3 ;; "--per-cluster") prefix=3 + ;; "--metric-only") metric_only=1 esac while read line
@@ -60,6 +62,9 @@ function commachecker() x=${main_body%#*} [ "$x" = "" ] && continue + # Check metric only - if it has a non-empty result + [ $metric_only -eq 1 ] && return 0 + # Skip metrics without event name y=${main_body#*#} for i in "${!skip_metric[@]}"; do
@@ -84,6 +89,8 @@ function commachecker() exit 1; } done < "${stat_output}" + + [ $metric_only -eq 1 ] && exit 1 return 0 }
@@ -95,6 +102,7 @@ check_system_wide "STD" "$perf_cmd" check_interval "STD" "$perf_cmd" check_per_thread "STD" "$perf_cmd" check_per_node "STD" "$perf_cmd" +check_metric_only "STD" "$perf_cmd" if [ $skip_test -ne 1 ] then check_system_wide_no_aggr "STD" "$perf_cmd"
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Namhyung Kim
maintainerPATCH 2/4 perf test: Skip perf probe tests when running as non-root
In reply to Namhyung Kimperf trace requires root because it needs to use [ku]probes. Skip those test when it's not run as root.
Before:
$ perf test probe 47: Probe SDT events : Ok 104: test perf probe of function from different CU : FAILED! 115: perftool-testsuite_probe : FAILED! 117: Add vfs_getname probe to get syscall args filenames : FAILED! 118: probe libc's inet_pton & backtrace it with ping : FAILED! 119: Use vfs_getname probe to get syscall args filenames : FAILED!
After:
$ perf test probe 47: Probe SDT events : Ok 104: test perf probe of function from different CU : Skip 115: perftool-testsuite_probe : Skip 117: Add vfs_getname probe to get syscall args filenames : Skip 118: probe libc's inet_pton & backtrace it with ping : Skip 119: Use vfs_getname probe to get syscall args filenames : Skip
--- tools/perf/tests/shell/perftool-testsuite_probe.sh | 1 + tools/perf/tests/shell/probe_vfs_getname.sh | 1 + tools/perf/tests/shell/record+probe_libc_inet_pton.sh | 1 + .../tests/shell/record+script_probe_vfs_getname.sh | 1 + .../perf/tests/shell/test_uprobe_from_different_cu.sh | 11 +++++------ 5 files changed, 9 insertions(+), 6 deletions(-)
tools/perf/tests/shell/perftool-testsuite_probe.sh +1 -0
@@ -2,6 +2,7 @@ # perftool-testsuite_probe (exclusive) # SPDX-License-Identifier: GPL-2.0 +[ "$(id -u)" == 0 ] || exit 2 test -d "$(dirname "$0")/base_probe" || exit 2 cd "$(dirname "$0")/base_probe" || exit 2 status=0
tools/perf/tests/shell/probe_vfs_getname.sh +1 -0
@@ -8,6 +8,7 @@ . "$(dirname $0)"/lib/probe.sh skip_if_no_perf_probe || exit 2 +[ "$(id -u)" == 0 ] || exit 2 # shellcheck source=lib/probe_vfs_getname.sh . "$(dirname $0)"/lib/probe_vfs_getname.sh
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +1 -0
@@ -105,6 +105,7 @@ delete_libc_inet_pton_event() { # Check for IPv6 interface existence ip a sh lo | grep -F -q inet6 || exit 2 +[ "$(id -u)" == 0 ] || exit 2 skip_if_no_perf_probe && \ add_libc_inet_pton_event && \
tools/perf/tests/shell/record+script_probe_vfs_getname.sh +1 -0
@@ -13,6 +13,7 @@ . "$(dirname "$0")/lib/probe.sh" skip_if_no_perf_probe || exit 2 +[ "$(id -u)" == 0 ] || exit 2 # shellcheck source=lib/probe_vfs_getname.sh . "$(dirname "$0")/lib/probe_vfs_getname.sh"
tools/perf/tests/shell/test_uprobe_from_different_cu.sh +5 -6
@@ -4,12 +4,11 @@ set -e -# Skip if there's no probe command. -if ! perf | grep probe -then - echo "Skip: probe command isn't present" - exit 2 -fi +# shellcheck source=lib/probe.sh +. "$(dirname $0)"/lib/probe.sh + +skip_if_no_perf_probe || exit 2 +[ "$(id -u)" == 0 ] || exit 2 # skip if there's no gcc if ! [ -x "$(command -v gcc)" ]; then
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 2/4] perf test: Skip perf probe tests when running as non-root
Quoted 11 lines · Namhyung Kim
perf trace requires root because it needs to use [ku]probes. Skip those test when it's not run as root.
Before:
$ perf test probe 47: Probe SDT events : Ok 104: test perf probe of function from different CU : FAILED! 115: perftool-testsuite_probe : FAILED! 117: Add vfs_getname probe to get syscall args filenames : FAILED! 118: probe libc's inet_pton & backtrace it with ping : FAILED! 119: Use vfs_getname probe to get syscall args filenames : FAILED!
Do you have ShellCheck installed?
TEST /tmp/build/perf-tools-next/tests/shell/probe_vfs_getname.sh.shellcheck_log TEST /tmp/build/perf-tools-next/tests/shell/record+probe_libc_inet_pton.sh.shellcheck_log TEST /tmp/build/perf-tools-next/tests/shell/record+script_probe_vfs_getname.sh.shellcheck_log
In tests/shell/probe_vfs_getname.sh line 11:
[ "$(id -u)" == 0 ] || exit 2
^-- SC3014 (warning): In POSIX sh, == in place of = is undefined.
For more information:
https://www.shellcheck.net/wiki/SC3014 -- In POSIX sh, == in place of = is ...
In tests/shell/record+script_probe_vfs_getname.sh line 16:
[ "$(id -u)" == 0 ] || exit 2
^-- SC3014 (warning): In POSIX sh, == in place of = is undefined.
For more information:
https://www.shellcheck.net/wiki/SC3014 -- In POSIX sh, == in place of = is ...
make[4]: *** [tests/Build:91: /tmp/build/perf-tools-next/tests/shell/probe_vfs_getname.sh.shellcheck_log] Error 1 make[4]: *** Waiting for unfinished jobs....
make[4]: *** [tests/Build:91: /tmp/build/perf-tools-next/tests/shell/record+script_probe_vfs_getname.sh.shellcheck_log] Error 1
In tests/shell/record+probe_libc_inet_pton.sh line 108:
[ "$(id -u)" == 0 ] || exit 2
^-- SC3014 (warning): In POSIX sh, == in place of = is undefined.
For more information:
https://www.shellcheck.net/wiki/SC3014 -- In POSIX sh, == in place of = is ...
make[4]: *** [tests/Build:91: /tmp/build/perf-tools-next/tests/shell/record+probe_libc_inet_pton.sh.shellcheck_log] Error 1 make[3]: *** [/home/acme/git/perf-tools-next/tools/build/Makefile.build:138: tests] Error 2
make[2]: *** [Makefile.perf:810: /tmp/build/perf-tools-next/perf-test-in.o] Error 2
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [Makefile.perf:321: sub-make] Error 2
make: *** [Makefile:119: install-bin] Error 2
make: Leaving directory '/home/acme/git/perf-tools-next/tools/perf'
⬢ [acme@toolbox perf-tools-next]$
The patch below seems to make it happy.
But since it seems to be mixed up, i.e. its not a string comparision, so why the quotes?
Probably it should be
[ $(id -u) -eq 0 ] || exit 2
?
It all works with bash, but ShellCheck being installed stops the build with that POSIX error.
⬢ [acme@toolbox perf-tools-next]$ [ "$(id -u)" == 1000 ] && echo bla bla
⬢ [acme@toolbox perf-tools-next]$ [ "$(id -u)" = 1000 ] && echo bla bla
⬢ [acme@toolbox perf-tools-next]$ [ $(id -u) -eq 1000 ] && echo bla bla
⬢ [acme@toolbox perf-tools-next]$ id
uid=1000(acme) gid=1000(acme) groups=1000(acme),10(wheel) ⬢ [acme@toolbox perf-tools-next]
- Arnaldo
tools/perf/tests/shell/probe_vfs_getname.sh +1 -1
@@ -8,7 +8,7 @@ . "$(dirname $0)"/lib/probe.sh skip_if_no_perf_probe || exit 2 -[ "$(id -u)" == 0 ] || exit 2 +[ "$(id -u)" = 0 ] || exit 2 # shellcheck source=lib/probe_vfs_getname.sh . "$(dirname $0)"/lib/probe_vfs_getname.sh
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +1 -1
@@ -105,7 +105,7 @@ delete_libc_inet_pton_event() { # Check for IPv6 interface existence ip a sh lo | grep -F -q inet6 || exit 2 -[ "$(id -u)" == 0 ] || exit 2 +[ "$(id -u)" = 0 ] || exit 2 skip_if_no_perf_probe && \ add_libc_inet_pton_event && \
tools/perf/tests/shell/record+script_probe_vfs_getname.sh +1 -1
@@ -13,7 +13,7 @@ . "$(dirname "$0")/lib/probe.sh" skip_if_no_perf_probe || exit 2 -[ "$(id -u)" == 0 ] || exit 2 +[ "$(id -u)" = 0 ] || exit 2 # shellcheck source=lib/probe_vfs_getname.sh . "$(dirname "$0")/lib/probe_vfs_getname.sh"
Namhyung Kim
maintainerRe: [PATCH 2/4] perf test: Skip perf probe tests when running as non-root
Hi Arnaldo,
Quoted 19 lines · Arnaldo Carvalho de Melo
Quoted 11 lines · Namhyung Kim
perf trace requires root because it needs to use [ku]probes. Skip those test when it's not run as root.
Before:
$ perf test probe 47: Probe SDT events : Ok 104: test perf probe of function from different CU : FAILED! 115: perftool-testsuite_probe : FAILED! 117: Add vfs_getname probe to get syscall args filenames : FAILED! 118: probe libc's inet_pton & backtrace it with ping : FAILED! 119: Use vfs_getname probe to get syscall args filenames : FAILED!Do you have ShellCheck installed?
TEST /tmp/build/perf-tools-next/tests/shell/probe_vfs_getname.sh.shellcheck_log TEST /tmp/build/perf-tools-next/tests/shell/record+probe_libc_inet_pton.sh.shellcheck_log TEST /tmp/build/perf-tools-next/tests/shell/record+script_probe_vfs_getname.sh.shellcheck_log
Quoted 6 lines
In
tests/shell/probe_vfs_getname.shline 11:
[ "$(id -u)" == 0 ] || exit 2^-- SC3014 (warning): In POSIX sh, == in place of = is undefined.For more information:
https://www.shellcheck.net/wiki/SC3014 -- In POSIX sh, == in place of = is ...
Sorry for the noise, it's fixed in v2.
Thanks,
Namhyung
Namhyung Kim
maintainerPATCH 3/4 perf test: Skip perf trace tests when running as non-root
In reply to Namhyung Kimperf trace requires root because it needs to use tracepoints and BPF. Skip those test when it's not run as root.
Before:
$ perf test trace 15: Parse sched tracepoints fields : Skip (permissions) 80: perf ftrace tests : Skip 105: perf trace enum augmentation tests : FAILED! 106: perf trace BTF general tests : FAILED! 107: perf trace exit race : FAILED! 118: probe libc's inet_pton & backtrace it with ping : Skip 125: Check Arm CoreSight trace data recording and synthesized samples: Skip 127: Check Arm SPE trace data recording and synthesized samples : Skip 132: Check open filename arg using perf trace + vfs_getname : FAILED!
After:
$ perf test trace 15: Parse sched tracepoints fields : Skip (permissions) 80: perf ftrace tests : Skip 105: perf trace enum augmentation tests : Skip 106: perf trace BTF general tests : Skip 107: perf trace exit race : Skip 118: probe libc's inet_pton & backtrace it with ping : Skip 125: Check Arm CoreSight trace data recording and synthesized samples: Skip 127: Check Arm SPE trace data recording and synthesized samples : Skip 132: Check open filename arg using perf trace + vfs_getname : Skip
--- tools/perf/tests/shell/trace+probe_vfs_getname.sh | 1 + tools/perf/tests/shell/trace_btf_enum.sh | 1 + tools/perf/tests/shell/trace_btf_general.sh | 1 + tools/perf/tests/shell/trace_exit_race.sh | 1 + 4 files changed, 4 insertions(+)
tools/perf/tests/shell/trace+probe_vfs_getname.sh +1 -0
@@ -15,6 +15,7 @@ skip_if_no_perf_probe || exit 2 skip_if_no_perf_trace || exit 2 +[ "$(id -u)" == 0 ] || exit 2 . "$(dirname $0)"/lib/probe_vfs_getname.sh
tools/perf/tests/shell/trace_btf_enum.sh +1 -0
@@ -13,6 +13,7 @@ TESTPROG="perf test -w landlock" # shellcheck source=lib/probe.sh . "$(dirname $0)"/lib/probe.sh skip_if_no_perf_trace || exit 2 +[ "$(id -u)" == 0 ] || exit 2 check_vmlinux() { echo "Checking if vmlinux exists"
tools/perf/tests/shell/trace_btf_general.sh +1 -0
@@ -76,6 +76,7 @@ trace_config() { skip_if_no_perf_trace || exit 2 check_vmlinux || exit 2 +[ "$(id -u)" == 0 ] || exit 2 trace_config
tools/perf/tests/shell/trace_exit_race.sh +1 -0
@@ -10,6 +10,7 @@ . "$(dirname $0)"/lib/probe.sh skip_if_no_perf_trace || exit 2 +[ "$(id -u)" == 0 ] || exit 2 if [ "$1" = "-v" ]; then verbose="1"
Signature · 1 line
2.48.1.711.g2feabab25a-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 3/4] perf test: Skip perf trace tests when running as non-root
tools/perf/tests/shell/trace+probe_vfs_getname.sh line 18
Quoted 6 lines · Namhyung Kim
+++
b/tools/perf/tests/shell/trace+probe_vfs_getname.shtools/perf/tests/shell/trace+probe_vfs_getname.sh +1
-0@@ -15,6 +15,7 @@ skip_if_no_perf_probe || exit 2 skip_if_no_perf_trace || exit 2 +[ "$(id -u)" == 0 ] || exit 2
Same issue as with the previous patch.
- Arnaldo
Namhyung Kim
maintainerPATCH 4/4 perf test: Add trace record and replay test
In reply to Namhyung KimIt just check trace record and replay could display correct output. It uses 'sleep' process and sees there's a clock_nanosleep syscall.
$ sudo perf test -vv replay
108: perf trace record and replay:
--- start ---
test child forked, pid 1563219
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.077 MB /tmp/temporary_file.w1ApA (242 samples) ]
0.686 (1000.068 ms): sleep/1563226 clock_nanosleep(rqtp: 0x7ffc20ffee10, rmtp: 0x7ffc20ffee50) = 0
---- end(0) ----
108: perf trace record and replay : Ok--- tools/perf/tests/shell/trace_record_replay.sh | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100755 tools/perf/tests/shell/trace_record_replay.sh
tools/perf/tests/shell/trace_record_replay.sh new file +21 -0
@@ -0,0 +1,21 @@ +#!/bin/sh +# perf trace record and replay +# SPDX-License-Identifier: GPL-2.0 + +# Check that perf trace works with record and replay + +# shellcheck source=lib/probe.sh +. "$(dirname $0)"/lib/probe.sh + +skip_if_no_perf_trace || exit 2 +[ "$(id -u)" == 0 ] || exit 2 + +file=$(mktemp /tmp/temporary_file.XXXXX) + +perf trace record -o ${file} sleep 1 || exit 1 +if ! perf trace -i ${file} 2>&1 | grep nanosleep; then + echo "Failed: cannot find *nanosleep syscall" + exit 1 +fi + +rm -f ${file}
Signature · 1 line
2.48.1.711.g2feabab25a-goog