perf test: adding new testsuite: common files
PATCH 2/9 perf test: adding new testsuite: common files
A new testsuite based on black-box testing of perf's subcommands is located in tools/perf/testsuite. It can be run from the perf's dir by "perf test suite".
The suite aims on testing the subcommands and their sanity from the user's point of view. It is designed to be easily extensible when a new feature needs to be covered.
This commit adds the base part of the testsuite (config, checkers):
common/
init.sh
- to be sourced within the test scripts
patterns.sh
- contains common regexp patterns
settings.sh
- sourced from init.sh
- contains various constants and settings
parametrization.sh
- sourced from settings.sh
- like a config file for enablement and disablement of some features, regulation of depth, etc.
check_*.pl
- various scripts for checking output
- might be extended by some more advanced checking tools and techniques
--- .../testsuite/common/check_all_lines_matched.pl | 38 +++++++ .../testsuite/common/check_all_patterns_found.pl | 33 ++++++ .../testsuite/common/check_any_pattern_found.pl | 14 +++ .../testsuite/common/check_buildids_vs_files.pl | 22 ++++ .../testsuite/common/check_errors_whitelisted.pl | 50 +++++++++ .../testsuite/common/check_exact_pattern_order.pl | 27 +++++ .../testsuite/common/check_kallsyms_vs_probes.pl | 48 +++++++++ .../testsuite/common/check_no_patterns_found.pl | 33 ++++++ tools/perf/testsuite/common/init.sh | 67 ++++++++++++ tools/perf/testsuite/common/parametrization.sh | 32 ++++++ tools/perf/testsuite/common/patterns.sh | 117 +++++++++++++++++++++ tools/perf/testsuite/common/settings.sh | 57 ++++++++++ 12 files changed, 538 insertions(+) create mode 100755 tools/perf/testsuite/common/check_all_lines_matched.pl create mode 100755 tools/perf/testsuite/common/check_all_patterns_found.pl create mode 100755 tools/perf/testsuite/common/check_any_pattern_found.pl create mode 100755 tools/perf/testsuite/common/check_buildids_vs_files.pl create mode 100755 tools/perf/testsuite/common/check_errors_whitelisted.pl create mode 100755 tools/perf/testsuite/common/check_exact_pattern_order.pl create mode 100755 tools/perf/testsuite/common/check_kallsyms_vs_probes.pl create mode 100755 tools/perf/testsuite/common/check_no_patterns_found.pl create mode 100644 tools/perf/testsuite/common/init.sh create mode 100644 tools/perf/testsuite/common/parametrization.sh create mode 100644 tools/perf/testsuite/common/patterns.sh create mode 100644 tools/perf/testsuite/common/settings.sh
tools/perf/testsuite/common/check_all_lines_matched.pl new file +38 -0
@@ -0,0 +1,38 @@ +#!/usr/bin/perl + +@regexps = @ARGV; + +$max_printed_lines = 20; +$max_printed_lines = $ENV{ERROR_MESSAGE_MAX_LINES} if (defined $ENV{ERROR_MESSAGE_MAX_LINES}); + +$quiet = 0; +$quiet = 1 if (defined $ENV{TESTMODE_QUIET} && $ENV{TESTMODE_QUIET} eq "y"); + +$passed = 1; +$lines_printed = 0; + +while (<STDIN>) +{ + s/\n//; + + $line_matched = 0; + for $r (@regexps) + { + if (/$r/) + { + $line_matched = 1; + last; + } + } + + unless ($line_matched) + { + if ($lines_printed++ < $max_printed_lines) + { + print "Line did not match any pattern: \"$_\"\n" unless $quiet; + } + $passed = 0; + } +} + +exit ($passed == 0);
tools/perf/testsuite/common/check_all_patterns_found.pl new file +33 -0
@@ -0,0 +1,33 @@ +#!/usr/bin/perl + +@regexps = @ARGV; + +$quiet = 0; +$quiet = 1 if (defined $ENV{TESTMODE_QUIET} && $ENV{TESTMODE_QUIET} eq "y"); + +%found = (); +$passed = 1; + +while (<STDIN>) +{ + s/\n//; + + for $r (@regexps) + { + if (/$r/) + { + $found{$r} = 1; # FIXME: maybe add counters -- how many times was the regexp matched + } + } +} + +for $r (@regexps) +{ + unless (exists $found{$r}) + { + print "Regexp not found: \"$r\"\n" unless $quiet; + $passed = 0; + } +} + +exit ($passed == 0);
tools/perf/testsuite/common/check_any_pattern_found.pl new file +14 -0
@@ -0,0 +1,14 @@ +#!/usr/bin/perl + +@regexps = @ARGV; + +while (<STDIN>) +{ + s/\n//; + for $r (@regexps) + { + exit 0 if (/$r/); + } +} + +exit 1;
tools/perf/testsuite/common/check_buildids_vs_files.pl new file +22 -0
@@ -0,0 +1,22 @@ +#!/usr/bin/perl + +$quiet = 0; +$quiet = 1 if (defined $ENV{TESTMODE_QUIET} && $ENV{TESTMODE_QUIET} eq "y"); + +$passed = 1; + +while (<STDIN>) +{ + chomp; + ($buildid_from_list, $filepath) = $_ =~ /^(\w{40})\s+((?:\/[\w\+.-]+)+)$/; + $filecmd_output = `file $filepath`; + ($buildid_from_file) = $filecmd_output =~ /BuildID\[sha1\]=(\w{40})/; + + if ($buildid_from_file ne $buildid_from_list) + { + $passed = 0; + print "$filepath has $buildid_from_file buildid but perf shows $buildid_from_list\n" unless $quiet; + } +} + +exit !($passed);
tools/perf/testsuite/common/check_errors_whitelisted.pl new file +50 -0
50 lines not on this page · raw message
tools/perf/testsuite/common/check_exact_pattern_order.pl new file +27 -0
@@ -0,0 +1,27 @@ +#!/usr/bin/perl + +@regexps = @ARGV; + +$quiet = 0; +$quiet = 1 if (defined $ENV{TESTMODE_QUIET} && $ENV{TESTMODE_QUIET} eq "y"); + +$passed = 1; +$r = shift @regexps; + +while (<STDIN>) +{ + s/\n//; + + if (/$r/) + { + $r = shift @regexps; + } +} + +if (defined $r) +{ + print "Pattern not found in the proper order: $r\n" unless $quiet; + exit 1; +} + +exit 0;
tools/perf/testsuite/common/check_kallsyms_vs_probes.pl new file +48 -0
48 lines not on this page · raw message
tools/perf/testsuite/common/check_no_patterns_found.pl new file +33 -0
@@ -0,0 +1,33 @@ +#!/usr/bin/perl + +@regexps = @ARGV; + +$quiet = 0; +$quiet = 1 if (defined $ENV{TESTMODE_QUIET} && $ENV{TESTMODE_QUIET} eq "y"); + +%found = (); +$passed = 1; + +while (<STDIN>) +{ + s/\n//; + + for $r (@regexps) + { + if (/$r/) + { + $found{$r} = 1; + } + } +} + +for $r (@regexps) +{ + if (exists $found{$r}) + { + print "Regexp found: \"$r\"\n" unless $quiet; + $passed = 0; + } +} + +exit ($passed == 0);
tools/perf/testsuite/common/init.sh new file +67 -0
67 lines not on this page · raw message
tools/perf/testsuite/common/parametrization.sh new file +32 -0
@@ -0,0 +1,32 @@ +# +# parametrization.sh +# Author: Michael Petlan <[email protected]> +# +# Description: +# +# This file configures the testcases how deeply they should +# look at things. The parametrization allows you to use the suite +# for both smoke testing and deeper testing. +# + +#### general + +# If set, the '--help' outputs of the commands will be checked. +# Since the '--help' option calls a manpage and manpages are not +# always generated, you may skip the man-page checks. +export PARAM_GENERAL_HELP_TEXT_CHECK=n + + +#### perf_stat + +# If set, the 24x7 events will be tested on all available cores. +# That might make it 'nproc' times longer. Basically it should be +# enough to run each event on one core only. +# Note: POWER8 only +export PARAM_STAT_24x7_ALL_CORES=n + +# If set, all the tracepoint events will be checked for syntax +# errors in their definition. This testcase may take a long time +# and the checks are not that crucial, so it can be turned off +# when you do not want to deep dive. +export PARAM_STAT_TRACEPOINT_EVENTS_SYNTAX=n
tools/perf/testsuite/common/patterns.sh new file +117 -0
117 lines not on this page · raw message
tools/perf/testsuite/common/settings.sh new file +57 -0
57 lines not on this page · raw message