Skip to contentSkip to conversation

perftool-testsuite 2nd batch

PATCH In review Michael Petlan 5 participants 43 messages Jun 24, 2024 – Sep 26, 2024 perf events

lore
Share & tools
lore
#0 lore raw

cover perftool-testsuite 2nd batch

Hello,

posting a second batch of tests from perftool-testsuite [1], as a part of the testsuite upstreaming effort.

This batch brings several perf-probe tests ideated by Masami Hiramatsu and a perf-report test.

We have also refactored settings files and fixed shellcheck warnings.

Finally, I have noticed that the scripts are run twice, once run by the driver (e.g. perftool-testsuite_*.sh) and once by perf-test's testsuite construction feature (that adds all executable shell scripts in all sub directories automatically). This is undesirable, so I have fixed it for now by skipping "base_*" dirs in the search. I see that this approach is not ideal, because the aim of the search seems to be to get rid of various test drivers, unify the way the tests are detected and run, while allowing them to be in subdirectories, like the Carsten Haitzler's coresight tests. However, I still think this is better solution that just having a pile of scripts without hierarchy that are grabbed and ordered into one-level queue of tests. Apart from allowing the tests to share some setup procedure, etc., is helps to keep the structure more clear.

A better design for future might be to have shell tests ordered in directories, such as

  # cd tests/shell/
  # find . -name \*.sh
  coresight/asm_pure_loop.sh
  coresight/memcpy_thread_16k_10.sh
  coresight/thread_loop_check_tid_10.sh
  coresight/thread_loop_check_tid_2.sh
  coresight/unroll_loop_thread_10.sh
  base_probe/test_adding_kernel.sh
  base_probe/test_adding_blacklisted.sh
  base_probe/test_basic.sh
  base_probe/test_invalid_options.sh
  base_probe/test_line_semantics.sh
  base_report/setup.sh
  base_report/test_basic.sh
  [...]  

The directories might also contain some "header" file that would bear the name of the parent testcase for all the tests inside it (e.g. "perf probe shell tests").

Then, no separate drivers would be needed, perf-test testsuite- creation mechanism would search for the shell scripts as it does now, but it would create multilevel structure with subtests, such as:

97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
98: CoreSight tests
98:1: CoreSight / ASM Pure loop
98:2: CoreSight / Thread Loop ...
[...]

... similar to how C-tests can be organized. In case there is a setup.sh file in such a subdirectory, it would be run too. With properly designed names, also tag-like (substring matching) approach could be used to select desired tests (i.e. to override the tree structure and pick tests related to some topic, no matter if they are in base_stat or base_record subdirectory).

This plan also counts with processing the current test scripts and possibly putting them to best matching place in the structure.

There is already a bunch of shell tests and I thinkg it might be good to organize them into a structure.

Thoughts and ideas are welcome.

Thanks and regards,

Michael

[1] https://github.com/rfmvh/perftool-testsuite

Michael Petlan (2):
perf tests shell: Skip base_* dirs in test script search perf testsuite: Install perf-report tests

Veronika Molnarova (9):
perf testsuite: Merge settings files for shell tests perf testsuite: Fix shellcheck warnings
perf testsuite probe: Add test for blacklisted kprobes handling perf testsuite probe: Add test for basic perf-probe options perf testsuite probe: Add test for invalid options
perf testsuite probe: Add test for line semantics
perf testsuite: Add common output checking helper
perf testsuite report: Add test for perf-report basic functionality perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  65 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  78 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  77 +++++++
 .../shell/base_probe/test_line_semantics.sh   |  53 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  30 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 188 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  50 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 627 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/11 perf tests shell: Skip base_* dirs in test script search

The test scripts in base_* directories currently have their own drivers that run them. Before this patch, the shell test-suite generator causes them to run twice. Fix that by skipping them in the generator.

A cleaner solution (for future) will be to use the directory structure idea (introduced by Carsten Haitzler in 7391db645938 ("perf test:
Refactor shell tests allowing subdirs")) to generate test entries with subtests, like:

$ perf test list
[...]
97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
[...]

There is already a lot of shell test scripts and many are about to come, so there is a need for some hierarchy.

---
 tools/perf/tests/tests-scripts.c | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/tests/tests-scripts.c +2 -0
index e2042b368269..01971e997425 100644
@@ -222,6 +222,8 @@ static void append_scripts_in_dir(int dir_fd,
 			if (!S_ISDIR(st.st_mode))
 				continue;
 		}
+		if (strncmp(ent->d_name, "base_", 5) == 0)
+			continue; /* Skip scripts that have a separate driver. */
 		fd = openat(dir_fd, ent->d_name, O_PATH);
 		append_scripts_in_dir(fd, result, result_sz);
 	}
Signature · 1 line
2.43.0
#2 lore raw

PATCH 2/11 perf testsuite: Merge settings files for shell tests

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Merge perf testsuite setting files into common settings to reduce duplicates and prevent errors.

---
 tools/perf/tests/shell/base_probe/settings.sh | 48 -------------------
 .../shell/base_probe/test_adding_kernel.sh    |  3 --
 tools/perf/tests/shell/common/init.sh         | 23 +++++++++
 tools/perf/tests/shell/common/settings.sh     | 23 +++++++++
 4 files changed, 46 insertions(+), 51 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
tools/perf/tests/shell/base_probe/settings.sh deleted +0 -48
deleted file mode 100644
index 123621c7f95e..000000000000
@@ -1,48 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-#	settings.sh of perf_probe test
-#	Author: Michael Petlan <[email protected]>
-#	Author: Masami Hiramatsu <[email protected]>
-#
-
-export TEST_NAME="perf_probe"
-
-export MY_ARCH=`arch`
-
-if [ -n "$PERFSUITE_RUN_DIR" ]; then
-	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
-	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
-	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
-	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
-	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
-	test -d "$MAKE_TARGET_DIR" || mkdir -p "$MAKE_TARGET_DIR"
-	export LOGS_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME/logs"
-	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
-else
-	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
-	export CURRENT_TEST_DIR="."
-	export LOGS_DIR="."
-fi
-
-check_kprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/kprobe_events
-}
-
-check_uprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/uprobe_events
-}
-
-clear_all_probes()
-{
-	echo 0 > /sys/kernel/debug/tracing/events/enable
-	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
-	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
-}
-
-check_sdt_support()
-{
-	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
-}
tools/perf/tests/shell/base_probe/test_adding_kernel.sh +0 -3
index 63bb8974b38e..3363f8c57e46 100755
@@ -15,10 +15,7 @@
 
 # include working environment
 . ../common/init.sh
-. ./settings.sh
 
-# shellcheck disable=SC2034 # the variable is later used after the working environment is included
-THIS_TEST_NAME=`basename $0 .sh`
 TEST_RESULT=0
 
 TEST_PROBE=${TEST_PROBE:-"inode_permission"}
tools/perf/tests/shell/common/init.sh +23 -0
index aadeaf782e03..b99fba50ba94 100644
@@ -115,3 +115,26 @@ detect_amd()
 	# 1 = is not AMD or unknown
 	grep "vendor_id" < /proc/cpuinfo | grep -q "AMD"
 }
+
+# base probe utility
+check_kprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/kprobe_events
+}
+
+check_uprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/uprobe_events
+}
+
+clear_all_probes()
+{
+	echo 0 > /sys/kernel/debug/tracing/events/enable
+	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
+	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
+}
+
+check_sdt_support()
+{
+	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
+}
tools/perf/tests/shell/common/settings.sh +23 -0
index 361641dbaaad..dcdb7d49ac00 100644
@@ -65,6 +65,29 @@ else
 	export MEND=""
 fi
 
+### general info
+DIR_PATH=`dirname "$(readlink -e "$0")"`
+
+export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
+export MY_ARCH=`arch`
+
+# storing logs and temporary files variables
+if [ -n "$PERFSUITE_RUN_DIR" ]; then
+	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
+	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
+	#     dirs will be used for that
+	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
+	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
+	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
+	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
+else
+	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
+	export CURRENT_TEST_DIR="."
+	export LOGS_DIR="."
+fi
+
 
 #### test parametrization
 if [ ! -d ./common ]; then
Signature · 1 line
2.43.0
#3 lore raw

PATCH 3/11 perf testsuite: Fix shellcheck warnings

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Shellcheck is becoming a standard when building perf to prevent any unnecessary mistakes. Fix shellcheck warnings in perf testsuite.

---
 tools/perf/tests/shell/common/init.sh     | 8 ++++----
 tools/perf/tests/shell/common/settings.sh | 9 +++++----
 2 files changed, 9 insertions(+), 8 deletions(-)
tools/perf/tests/shell/common/init.sh +4 -4
index b99fba50ba94..075f17623c8e 100644
@@ -26,8 +26,8 @@ print_results()
 	PERF_RETVAL="$1"; shift
 	CHECK_RETVAL="$1"; shift
 	FAILURE_REASON=""
-	TASK_COMMENT="$@"
-	if [ $PERF_RETVAL -eq 0 -a $CHECK_RETVAL -eq 0 ]; then
+	TASK_COMMENT="$*"
+	if [ $PERF_RETVAL -eq 0 ] && [ $CHECK_RETVAL -eq 0 ]; then
 		_echo "$MPASS-- [ PASS ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT"
 		return 0
 	else
@@ -56,7 +56,7 @@ print_overall_results()
 
 print_testcase_skipped()
 {
-	TASK_COMMENT="$@"
+	TASK_COMMENT="$*"
 	_echo "$MSKIP-- [ SKIP ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT :: testcase skipped"
 	return 0
 }
@@ -69,7 +69,7 @@ print_overall_skipped()
 
 print_warning()
 {
-	WARN_COMMENT="$@"
+	WARN_COMMENT="$*"
 	_echo "$MWARN-- [ WARN ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $WARN_COMMENT"
 	return 0
 }
tools/perf/tests/shell/common/settings.sh +5 -4
index dcdb7d49ac00..2e50dd0bff9b 100644
@@ -45,7 +45,7 @@ export TEST_IGNORE_MISSING_PMU=${TEST_IGNORE_MISSING_PMU:-n}
 export LC_ALL=C
 
 #### colors
-if [ -t 1 -o "$TESTLOG_FORCE_COLOR" = "yes" ]; then
+if [ -t 1 ] || [ "$TESTLOG_FORCE_COLOR" = "yes" ]; then
 	export MPASS="\e[32m"
 	export MALLPASS="\e[1;32m"
 	export MFAIL="\e[31m"
@@ -68,15 +68,15 @@ fi
 ### general info
 DIR_PATH=`dirname "$(readlink -e "$0")"`
 
-export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
-export MY_ARCH=`arch`
+TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`; export TEST_NAME
+MY_ARCH=`arch`; export MY_ARCH
 
 # storing logs and temporary files variables
 if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
 	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
 	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`; export PERFSUITE_RUN_DIR
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
@@ -93,6 +93,7 @@ fi
 if [ ! -d ./common ]; then
 	# set parameters based on runmode
 	if [ -f ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh ]; then
+		# shellcheck source=/dev/null
 		. ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh
 	fi
 	# if some parameters haven't been set until now, set them to default
Signature · 1 line
2.43.0
#4 lore raw

PATCH 4/11 perf testsuite probe: Add test for blacklisted kprobes handling

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Test perf probe interface. Blacklisted functions should be rejected when there is an attempt to set a kprobe to them.

---
 .../base_probe/test_adding_blacklisted.sh     | 67 +++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh new file +67 -0
new file mode 100755
index 000000000000..b5dc10b2a738
@@ -0,0 +1,67 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_adding_blacklisted of perf_probe test
+#	Author: Masami Hiramatsu <[email protected]>
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		Blacklisted functions should not be added successfully as probes,
+#	they must be skipped.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+# skip if not supported
+BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
+if [ -z "$BLACKFUNC" ]; then
+	print_overall_skipped
+	exit 0
+fi
+
+# remove all previously added probes
+clear_all_probes
+
+
+### adding blacklisted function
+
+# functions from blacklist should be skipped by perf probe
+! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
+PERF_EXIT_CODE=$?
+
+REGEX_SCOPE_FAIL="Failed to find scope of probe point"
+REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
+REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
+REGEX_ERROR_MESSAGE="Error: Failed to add events."
+REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
+REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
+REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
+(( TEST_RESULT += $? ))
+
+
+### listing not-added probe
+
+# blacklisted probes should NOT appear in perf-list output
+$CMD_PERF list probe:\* > $LOGS_DIR/adding_blacklisted_list.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_lines_matched.pl "$RE_LINE_EMPTY" "List of pre-defined events" "Metric Groups:" < $LOGS_DIR/adding_blacklisted_list.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "listing blacklisted probe (should NOT be listed)"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#5 lore raw

PATCH 5/11 perf testsuite probe: Add test for basic perf-probe options

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Test basic behavior of perf-probe subcommand. It is run as a part of perftool-testsuite_probe test case.

---
 .../perf/tests/shell/base_probe/test_basic.sh | 80 +++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
tools/perf/tests/shell/base_probe/test_basic.sh new file +80 -0
new file mode 100755
index 000000000000..09669ec479f2
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_basic of perf_probe test
+#	Author: Michael Petlan <[email protected]>
+#	Author: Masami Hiramatsu <[email protected]>
+#
+#	Description:
+#
+#		This test tests basic functionality of perf probe command.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### help message
+
+if [ "$PARAM_GENERAL_HELP_TEXT_CHECK" = "y" ]; then
+	# test that a help message is shown and looks reasonable
+	$CMD_PERF probe --help > $LOGS_DIR/basic_helpmsg.log 2> $LOGS_DIR/basic_helpmsg.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "PERF-PROBE" "NAME" "SYNOPSIS" "DESCRIPTION" "OPTIONS" "PROBE\s+SYNTAX" "PROBE\s+ARGUMENT" "LINE\s+SYNTAX" < $LOGS_DIR/basic_helpmsg.log
+	CHECK_EXIT_CODE=$?
+	../common/check_all_patterns_found.pl "LAZY\s+MATCHING" "FILTER\s+PATTERN" "EXAMPLES" "SEE\s+ALSO" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "vmlinux" "module=" "source=" "verbose" "quiet" "add=" "del=" "list.*EVENT" "line=" "vars=" "externs" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "no-inlines" "funcs.*FILTER" "filter=FILTER" "force" "dry-run" "max-probes" "exec=" "demangle-kernel" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_no_patterns_found.pl "No manual entry for" < $LOGS_DIR/basic_helpmsg.err
+	(( CHECK_EXIT_CODE += $? ))
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "help message"
+	(( TEST_RESULT += $? ))
+else
+	print_testcase_skipped "help message"
+fi
+
+
+### usage message
+
+# without any args perf-probe should print usage
+$CMD_PERF probe 2> $LOGS_DIR/basic_usage.log > /dev/null
+
+../common/check_all_patterns_found.pl "[Uu]sage" "perf probe" "verbose" "quiet" "add" "del" "force" "line" "vars" "externs" "range" < $LOGS_DIR/basic_usage.log
+CHECK_EXIT_CODE=$?
+
+print_results 0 $CHECK_EXIT_CODE "usage message"
+(( TEST_RESULT += $? ))
+
+
+### quiet switch
+
+# '--quiet' should mute all output
+$CMD_PERF probe --quiet --add vfs_read > $LOGS_DIR/basic_quiet01.log 2> $LOGS_DIR/basic_quiet01.err
+PERF_EXIT_CODE=$?
+$CMD_PERF probe --quiet --del vfs_read > $LOGS_DIR/basic_quiet03.log 2> $LOGS_DIR/basic_quiet02.err
+(( PERF_EXIT_CODE += $? ))
+
+test "`cat $LOGS_DIR/basic_quiet*log $LOGS_DIR/basic_quiet*err | wc -l`" -eq 0
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "quiet switch"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#6 lore raw

PATCH 6/11 perf testsuite probe: Add test for invalid options

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Test if various incompatible options are correctly handled-rejected. It is run as a part of perftool-testsuite_probe test case.

---
 .../shell/base_probe/test_invalid_options.sh  | 79 +++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
tools/perf/tests/shell/base_probe/test_invalid_options.sh new file +79 -0
new file mode 100755
index 000000000000..1fedfd8b0d0d
@@ -0,0 +1,79 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_invalid_options of perf_probe test
+#	Author: Masami Hiramatsu <[email protected]>
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		This test checks whether the invalid and incompatible options are reported
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### missing argument
+
+# some options require an argument
+for opt in '-a' '-d' '-L' '-V'; do
+	! $CMD_PERF probe $opt 2> $LOGS_DIR/invalid_options_missing_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .* requires a value" < $LOGS_DIR/invalid_options_missing_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unnecessary argument
+
+# some options may omit the argument
+for opt in '-F' '-l'; do
+	$CMD_PERF probe -F > /dev/null 2> $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	test ! -s $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "unnecessary argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### mutually exclusive options
+
+# some options are mutually exclusive
+test -e $LOGS_DIR/invalid_options_mutually_exclusive.log && rm -f $LOGS_DIR/invalid_options_mutually_exclusive.log
+for opt in '-a xxx -d xxx' '-a xxx -L foo' '-a xxx -V foo' '-a xxx -l' '-a xxx -F' \
+		'-d xxx -L foo' '-d xxx -V foo' '-d xxx -l' '-d xxx -F' \
+		'-L foo -V bar' '-L foo -l' '-L foo -F' '-V foo -l' '-V foo -F' '-l -F'; do
+	! $CMD_PERF probe $opt > /dev/null 2> $LOGS_DIR/aux.log
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .+ cannot be used with switch .+" < $LOGS_DIR/aux.log
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "mutually exclusive options :: $opt"
+	(( TEST_RESULT += $? ))
+
+	# gather the logs
+	cat $LOGS_DIR/aux.log | grep "Error" >> $LOGS_DIR/invalid_options_mutually_exclusive.log
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#7 lore raw

PATCH 7/11 perf testsuite probe: Add test for line semantics

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

The perf-probe command uses a specific semantics to describe probes. Test some patterns that are known to be both valid and invalid if they are handled appropriately.

This test is run as a part of perftool-testsuite_probe test case.

---
 .../shell/base_probe/test_line_semantics.sh   | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
tools/perf/tests/shell/base_probe/test_line_semantics.sh new file +55 -0
new file mode 100755
index 000000000000..d8f4bde0f585
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_line_semantics of perf_probe test
+#	Author: Masami Hiramatsu <[email protected]>
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		This test checks whether the semantic errors of line option's
+#		arguments are properly reported.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### acceptable --line descriptions
+
+# testing acceptance of valid patterns for the '--line' option
+VALID_PATTERNS="func func:10 func:0-10 func:2+10 [email protected] [email protected]:1 source.c:1 source.c:1+1 source.c:1-10"
+for desc in $VALID_PATTERNS; do
+	! ( $CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error" )
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "acceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unacceptable --line descriptions
+
+# testing handling of invalid patterns for the '--line' option
+INVALID_PATTERNS="func:foo func:1-foo func:1+foo func;lazy\*pattern"
+for desc in $INVALID_PATTERNS; do
+	$CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error"
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "unacceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#8 lore raw

PATCH 8/11 perf testsuite: Add common output checking helper

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

As a form of validation, it is a common practice to check the outputs of commands whether they contain expected patterns or match a certain regular expression.

This output checking helper is designed to allow checking stderr output of perf commands for unexpected messages, while ignoring messages that are known to be harmless, e.g.:
"Lowering default frequency rate to \d+\."
"\d+ out of order events recorded."
etc.

---
 .../shell/common/check_errors_whitelisted.pl  | 51 +++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
tools/perf/tests/shell/common/check_errors_whitelisted.pl new file +51 -0
new file mode 100755
index 000000000000..c57d355dd76e
@@ -0,0 +1,51 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0
+
+$whitelist_file = shift;
+
+if (defined $whitelist_file)
+{
+	open (INFILE, $whitelist_file) or die "Checker error: Unable to open the whitelist file: $whitelist_file\n";
+	@regexps = <INFILE>;
+	close INFILE or die "Checker error: Unable to close the whitelist file: $whitelist_file\n";
+}
+else
+{
+	@regexps = ();
+}
+
+$max_printed_lines = 20;
+$max_printed_lines = $ENV{TESTLOG_ERR_MSG_MAX_LINES} if (defined $ENV{TESTLOG_ERR_MSG_MAX_LINES});
+
+$quiet = 1;
+$quiet = 0 if (defined $ENV{TESTLOG_VERBOSITY} && $ENV{TESTLOG_VERBOSITY} ge 2);
+
+$passed = 1;
+$lines_printed = 0;
+
+while (<STDIN>)
+{
+	s/\n//;
+
+	$line_matched = 0;
+	for $r (@regexps)
+	{
+		chomp $r;
+		if (/$r/)
+		{
+			$line_matched = 1;
+			last;
+		}
+	}
+
+	unless ($line_matched)
+	{
+		if ($lines_printed++ < $max_printed_lines)
+		{
+			print "Line did not match any pattern: \"$_\"\n" unless $quiet;
+		}
+		$passed = 0;
+	}
+}
+
+exit ($passed == 0);
Signature · 1 line
2.43.0
#9 lore raw

PATCH 9/11 perf testsuite report: Add test for perf-report basic functionality

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Test basic execution and some options of perf-report subcommand, like show-nr-samples, header, showcpuutilization, pid and symbol filtering.

---
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 tools/perf/tests/shell/common/settings.sh     |   2 +
 4 files changed, 229 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
tools/perf/tests/shell/base_report/setup.sh new file +32 -0
new file mode 100755
index 000000000000..4caa496660c6
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	setup.sh of perf report test
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		We need some sample data for perf-report testing
+#
+#
+
+# include working environment
+. ../common/init.sh
+
+test -d "$HEADER_TAR_DIR" || mkdir -p "$HEADER_TAR_DIR"
+
+SW_EVENT="cpu-clock"
+
+$CMD_PERF record -asdg -e $SW_EVENT -o $CURRENT_TEST_DIR/perf.data -- $CMD_LONGER_SLEEP 2> $LOGS_DIR/setup.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_patterns_found.pl "$RE_LINE_RECORD1" "$RE_LINE_RECORD2" < $LOGS_DIR/setup.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "prepare the perf.data file"
+TEST_RESULT=$?
+
+print_overall_results $TEST_RESULT
+exit $?
tools/perf/tests/shell/base_report/stderr-whitelist.txt new file +5 -0
new file mode 100644
index 000000000000..e3341401b47c
@@ -0,0 +1,5 @@
+no symbols found in .*, maybe install a debug package
+was updated .*is prelink enabled.+ Restart the long running apps that use it
+Warning:
+\d+ out of order events recorded.
+detected invalid bpf_prog_info
tools/perf/tests/shell/base_report/test_basic.sh new file +190 -0

190 lines not on this page · raw message

tools/perf/tests/shell/common/settings.sh +2 -0
index 2e50dd0bff9b..cba1b338f96f 100644
@@ -80,12 +80,14 @@ if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	export HEADER_TAR_DIR="$CURRENT_TEST_DIR/header_tar"
 	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
 	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
 else
 	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
 	export CURRENT_TEST_DIR="."
 	export LOGS_DIR="."
+	export HEADER_TAR_DIR="./header_tar"
 fi
 
 
Signature · 1 line
2.43.0
#10 lore raw

PATCH 10/11 perf testsuite report: Add test case for perf report

In reply to Michael Petlan

From: Veronika Molnarova <[email protected]>

Add new perf report test case that acts as an entry element in perf test list. Runs multiple subtests from directory "base_report", which can be expanded without further editing.

---
 .../tests/shell/perftool-testsuite_report.sh  | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
tools/perf/tests/shell/perftool-testsuite_report.sh new file +23 -0
new file mode 100755
index 000000000000..973012ce92a7
@@ -0,0 +1,23 @@
+#!/bin/bash
+# perftool-testsuite_report
+# SPDX-License-Identifier: GPL-2.0
+
+test -d "$(dirname "$0")/base_report" || exit 2
+cd "$(dirname "$0")/base_report" || exit 2
+status=0
+
+PERFSUITE_RUN_DIR=$(mktemp -d /tmp/"$(basename "$0" .sh)".XXX)
+export PERFSUITE_RUN_DIR
+
+for testcase in setup.sh test_*; do                  # skip setup.sh if not present or not executable
+     test -x "$testcase" || continue
+     ./"$testcase"
+     (( status += $? ))
+done
+
+if ! [ "$PERFTEST_KEEP_LOGS" = "y" ]; then
+	rm -rf "$PERFSUITE_RUN_DIR"
+fi
+
+test $status -ne 0 && exit 1
+exit 0
Signature · 1 line
2.43.0
#11 lore raw

PATCH 11/11 perf testsuite: Install perf-report tests

In reply to Michael Petlan
---
 tools/perf/Makefile.perf | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/Makefile.perf +2 -0
index 5c35c0d89306..a7eb1fa97b1c 100644
@@ -1080,6 +1080,8 @@ install-tests: all install-gtk
 		$(INSTALL) tests/shell/common/*.pl '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/common'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
 		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
+		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight' ; \
 		$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
 	$(Q)$(MAKE) -C tests/shell/coresight install-tests
Signature · 1 line
2.43.0
maintainer
#12 lore raw
In reply to Michael Petlan

Hello,

Quoted 71 lines · Michael Petlan

Hello,

posting a second batch of tests from perftool-testsuite [1], as a part of the testsuite upstreaming effort.

This batch brings several perf-probe tests ideated by Masami Hiramatsu and a perf-report test.

We have also refactored settings files and fixed shellcheck warnings.

Finally, I have noticed that the scripts are run twice, once run by the driver (e.g. perftool-testsuite_*.sh) and once by perf-test's testsuite construction feature (that adds all executable shell scripts in all sub directories automatically). This is undesirable, so I have fixed it for now by skipping "base_*" dirs in the search. I see that this approach is not ideal, because the aim of the search seems to be to get rid of various test drivers, unify the way the tests are detected and run, while allowing them to be in subdirectories, like the Carsten Haitzler's coresight tests. However, I still think this is better solution that just having a pile of scripts without hierarchy that are grabbed and ordered into one-level queue of tests. Apart from allowing the tests to share some setup procedure, etc., is helps to keep the structure more clear.

A better design for future might be to have shell tests ordered in directories, such as

  # cd tests/shell/
  # find . -name \*.sh
  coresight/asm_pure_loop.sh
  coresight/memcpy_thread_16k_10.sh
  coresight/thread_loop_check_tid_10.sh
  coresight/thread_loop_check_tid_2.sh
  coresight/unroll_loop_thread_10.sh
  base_probe/test_adding_kernel.sh
  base_probe/test_adding_blacklisted.sh
  base_probe/test_basic.sh
  base_probe/test_invalid_options.sh
  base_probe/test_line_semantics.sh
  base_report/setup.sh
  base_report/test_basic.sh
  [...]  

The directories might also contain some "header" file that would bear the name of the parent testcase for all the tests inside it (e.g. "perf probe shell tests").

Then, no separate drivers would be needed, perf-test testsuite- creation mechanism would search for the shell scripts as it does now, but it would create multilevel structure with subtests, such as:

97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
98: CoreSight tests
98:1: CoreSight / ASM Pure loop
98:2: CoreSight / Thread Loop ...
[...]

... similar to how C-tests can be organized. In case there is a setup.sh file in such a subdirectory, it would be run too. With properly designed names, also tag-like (substring matching) approach could be used to select desired tests (i.e. to override the tree structure and pick tests related to some topic, no matter if they are in base_stat or base_record subdirectory).

This plan also counts with processing the current test scripts and possibly putting them to best matching place in the structure.

Quoted 8 lines

There is already a bunch of shell tests and I thinkg it might be good to organize them into a structure.

Thoughts and ideas are welcome.

Thanks and regards,

Michael

Thanks for the work, but it doesn't apply to the current perf-tools-next. Can you please rebase?

Thanks,
Namhyung

Quoted 50 lines

[1] https://github.com/rfmvh/perftool-testsuite

Michael Petlan (2):
perf tests shell: Skip base_* dirs in test script search perf testsuite: Install perf-report tests

Veronika Molnarova (9):
perf testsuite: Merge settings files for shell tests perf testsuite: Fix shellcheck warnings
perf testsuite probe: Add test for blacklisted kprobes handling perf testsuite probe: Add test for basic perf-probe options perf testsuite probe: Add test for invalid options
perf testsuite probe: Add test for line semantics
perf testsuite: Add common output checking helper
perf testsuite report: Add test for perf-report basic functionality perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  65 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  78 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  77 +++++++
 .../shell/base_probe/test_line_semantics.sh   |  53 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  30 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 188 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  50 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 627 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
Signature · 1 line
2.43.0
#13 lore raw

v2 cover perftool-testsuite 2nd batch

From: Veronika Molnarova <[email protected]>

Hello,

rebased the patches for the current perf-tools-next branch in v2 as it was not applying.

Thanks for the notice,
Veronika

Michael Petlan (2):
perf tests shell: Skip base_* dirs in test script search perf testsuite: Install perf-report tests

Veronika Molnarova (9):
perf testsuite: Merge settings files for shell tests perf testsuite: Fix shellcheck warnings
perf testsuite probe: Add test for blacklisted kprobes handling perf testsuite probe: Add test for basic perf-probe options perf testsuite probe: Add test for invalid options
perf testsuite probe: Add test for line semantics
perf testsuite: Add common output checking helper
perf testsuite report: Add test for perf-report basic functionality perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  67 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  80 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  79 ++++++++
 .../shell/base_probe/test_line_semantics.sh   |  55 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  51 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 640 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
Signature · 1 line
2.43.0
#14 lore raw

PATCH v2 1/11 perf tests shell: Skip base_* dirs in test script search

From: Michael Petlan <[email protected]>

The test scripts in base_* directories currently have their own drivers that run them. Before this patch, the shell test-suite generator causes them to run twice. Fix that by skipping them in the generator.

A cleaner solution (for future) will be to use the directory structure idea (introduced by Carsten Haitzler in 7391db645938 ("perf test:
Refactor shell tests allowing subdirs")) to generate test entries with subtests, like:

$ perf test list
[...]
97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
[...]

There is already a lot of shell test scripts and many are about to come, so there is a need for some hierarchy.

---
 tools/perf/tests/tests-scripts.c | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/tests/tests-scripts.c +2 -0
index e2042b3682..01971e9974 100644
@@ -222,6 +222,8 @@ static void append_scripts_in_dir(int dir_fd,
 			if (!S_ISDIR(st.st_mode))
 				continue;
 		}
+		if (strncmp(ent->d_name, "base_", 5) == 0)
+			continue; /* Skip scripts that have a separate driver. */
 		fd = openat(dir_fd, ent->d_name, O_PATH);
 		append_scripts_in_dir(fd, result, result_sz);
 	}
Signature · 1 line
2.43.0
#15 lore raw

PATCH v2 2/11 perf testsuite: Merge settings files for shell tests

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Merge perf testsuite setting files into common settings to reduce duplicates and prevent errors.

---
 tools/perf/tests/shell/base_probe/settings.sh | 48 -------------------
 .../shell/base_probe/test_adding_kernel.sh    |  3 --
 tools/perf/tests/shell/common/init.sh         | 23 +++++++++
 tools/perf/tests/shell/common/settings.sh     | 23 +++++++++
 4 files changed, 46 insertions(+), 51 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
tools/perf/tests/shell/base_probe/settings.sh deleted +0 -48
deleted file mode 100644
index 123621c7f9..0000000000
@@ -1,48 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-#	settings.sh of perf_probe test
-#	Author: Michael Petlan <[email protected]>
-#	Author: Masami Hiramatsu <[email protected]>
-#
-
-export TEST_NAME="perf_probe"
-
-export MY_ARCH=`arch`
-
-if [ -n "$PERFSUITE_RUN_DIR" ]; then
-	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
-	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
-	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
-	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
-	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
-	test -d "$MAKE_TARGET_DIR" || mkdir -p "$MAKE_TARGET_DIR"
-	export LOGS_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME/logs"
-	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
-else
-	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
-	export CURRENT_TEST_DIR="."
-	export LOGS_DIR="."
-fi
-
-check_kprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/kprobe_events
-}
-
-check_uprobes_available()
-{
-	test -e /sys/kernel/debug/tracing/uprobe_events
-}
-
-clear_all_probes()
-{
-	echo 0 > /sys/kernel/debug/tracing/events/enable
-	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
-	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
-}
-
-check_sdt_support()
-{
-	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
-}
tools/perf/tests/shell/base_probe/test_adding_kernel.sh +0 -3
index 187dc8d4b1..d541ffd44a 100755
@@ -15,10 +15,7 @@
 
 # include working environment
 . ../common/init.sh
-. ./settings.sh
 
-# shellcheck disable=SC2034 # the variable is later used after the working environment is included
-THIS_TEST_NAME=`basename $0 .sh`
 TEST_RESULT=0
 
 # shellcheck source=lib/probe_vfs_getname.sh
tools/perf/tests/shell/common/init.sh +23 -0
index aadeaf782e..b99fba50ba 100644
@@ -115,3 +115,26 @@ detect_amd()
 	# 1 = is not AMD or unknown
 	grep "vendor_id" < /proc/cpuinfo | grep -q "AMD"
 }
+
+# base probe utility
+check_kprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/kprobe_events
+}
+
+check_uprobes_available()
+{
+	test -e /sys/kernel/debug/tracing/uprobe_events
+}
+
+clear_all_probes()
+{
+	echo 0 > /sys/kernel/debug/tracing/events/enable
+	check_kprobes_available && echo > /sys/kernel/debug/tracing/kprobe_events
+	check_uprobes_available && echo > /sys/kernel/debug/tracing/uprobe_events
+}
+
+check_sdt_support()
+{
+	$CMD_PERF list sdt | grep sdt > /dev/null 2> /dev/null
+}
tools/perf/tests/shell/common/settings.sh +23 -0
index 361641dbaa..dcdb7d49ac 100644
@@ -65,6 +65,29 @@ else
 	export MEND=""
 fi
 
+### general info
+DIR_PATH=`dirname "$(readlink -e "$0")"`
+
+export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
+export MY_ARCH=`arch`
+
+# storing logs and temporary files variables
+if [ -n "$PERFSUITE_RUN_DIR" ]; then
+	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
+	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
+	#     dirs will be used for that
+	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
+	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
+	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
+	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
+else
+	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
+	export CURRENT_TEST_DIR="."
+	export LOGS_DIR="."
+fi
+
 
 #### test parametrization
 if [ ! -d ./common ]; then
Signature · 1 line
2.43.0
#16 lore raw

PATCH v2 3/11 perf testsuite: Fix shellcheck warnings

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Shellcheck is becoming a standard when building perf to prevent any unnecessary mistakes. Fix shellcheck warnings in perf testsuite.

---
 tools/perf/tests/shell/common/init.sh     | 8 ++++----
 tools/perf/tests/shell/common/settings.sh | 9 +++++----
 2 files changed, 9 insertions(+), 8 deletions(-)
tools/perf/tests/shell/common/init.sh +4 -4
index b99fba50ba..075f17623c 100644
@@ -26,8 +26,8 @@ print_results()
 	PERF_RETVAL="$1"; shift
 	CHECK_RETVAL="$1"; shift
 	FAILURE_REASON=""
-	TASK_COMMENT="$@"
-	if [ $PERF_RETVAL -eq 0 -a $CHECK_RETVAL -eq 0 ]; then
+	TASK_COMMENT="$*"
+	if [ $PERF_RETVAL -eq 0 ] && [ $CHECK_RETVAL -eq 0 ]; then
 		_echo "$MPASS-- [ PASS ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT"
 		return 0
 	else
@@ -56,7 +56,7 @@ print_overall_results()
 
 print_testcase_skipped()
 {
-	TASK_COMMENT="$@"
+	TASK_COMMENT="$*"
 	_echo "$MSKIP-- [ SKIP ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $TASK_COMMENT :: testcase skipped"
 	return 0
 }
@@ -69,7 +69,7 @@ print_overall_skipped()
 
 print_warning()
 {
-	WARN_COMMENT="$@"
+	WARN_COMMENT="$*"
 	_echo "$MWARN-- [ WARN ] --$MEND $TEST_NAME :: $THIS_TEST_NAME :: $WARN_COMMENT"
 	return 0
 }
tools/perf/tests/shell/common/settings.sh +5 -4
index dcdb7d49ac..2e50dd0bff 100644
@@ -45,7 +45,7 @@ export TEST_IGNORE_MISSING_PMU=${TEST_IGNORE_MISSING_PMU:-n}
 export LC_ALL=C
 
 #### colors
-if [ -t 1 -o "$TESTLOG_FORCE_COLOR" = "yes" ]; then
+if [ -t 1 ] || [ "$TESTLOG_FORCE_COLOR" = "yes" ]; then
 	export MPASS="\e[32m"
 	export MALLPASS="\e[1;32m"
 	export MFAIL="\e[31m"
@@ -68,15 +68,15 @@ fi
 ### general info
 DIR_PATH=`dirname "$(readlink -e "$0")"`
 
-export TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`
-export MY_ARCH=`arch`
+TEST_NAME=`basename $DIR_PATH | sed 's/base/perf/'`; export TEST_NAME
+MY_ARCH=`arch`; export MY_ARCH
 
 # storing logs and temporary files variables
 if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	# when $PERFSUITE_RUN_DIR is set to something, all the logs and temp files will be placed there
 	# --> the $PERFSUITE_RUN_DIR/perf_something/examples and $PERFSUITE_RUN_DIR/perf_something/logs
 	#     dirs will be used for that
-	export PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`
+	PERFSUITE_RUN_DIR=`readlink -f $PERFSUITE_RUN_DIR`; export PERFSUITE_RUN_DIR
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
@@ -93,6 +93,7 @@ fi
 if [ ! -d ./common ]; then
 	# set parameters based on runmode
 	if [ -f ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh ]; then
+		# shellcheck source=/dev/null
 		. ../common/parametrization.$PERFTOOL_TESTSUITE_RUNMODE.sh
 	fi
 	# if some parameters haven't been set until now, set them to default
Signature · 1 line
2.43.0
#17 lore raw

PATCH v2 4/11 perf testsuite probe: Add test for blacklisted kprobes handling

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Test perf probe interface. Blacklisted functions should be rejected when there is an attempt to set a kprobe to them.

---
 .../base_probe/test_adding_blacklisted.sh     | 67 +++++++++++++++++++
 1 file changed, 67 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh new file +67 -0
new file mode 100755
index 0000000000..b5dc10b2a7
@@ -0,0 +1,67 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_adding_blacklisted of perf_probe test
+#	Author: Masami Hiramatsu <[email protected]>
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		Blacklisted functions should not be added successfully as probes,
+#	they must be skipped.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+# skip if not supported
+BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
+if [ -z "$BLACKFUNC" ]; then
+	print_overall_skipped
+	exit 0
+fi
+
+# remove all previously added probes
+clear_all_probes
+
+
+### adding blacklisted function
+
+# functions from blacklist should be skipped by perf probe
+! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
+PERF_EXIT_CODE=$?
+
+REGEX_SCOPE_FAIL="Failed to find scope of probe point"
+REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
+REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
+REGEX_ERROR_MESSAGE="Error: Failed to add events."
+REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
+REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
+REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
+(( TEST_RESULT += $? ))
+
+
+### listing not-added probe
+
+# blacklisted probes should NOT appear in perf-list output
+$CMD_PERF list probe:\* > $LOGS_DIR/adding_blacklisted_list.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_lines_matched.pl "$RE_LINE_EMPTY" "List of pre-defined events" "Metric Groups:" < $LOGS_DIR/adding_blacklisted_list.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "listing blacklisted probe (should NOT be listed)"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#18 lore raw

PATCH v2 5/11 perf testsuite probe: Add test for basic perf-probe options

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Test basic behavior of perf-probe subcommand. It is run as a part of perftool-testsuite_probe test case.

---
 .../perf/tests/shell/base_probe/test_basic.sh | 80 +++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
tools/perf/tests/shell/base_probe/test_basic.sh new file +80 -0
new file mode 100755
index 0000000000..09669ec479
@@ -0,0 +1,80 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_basic of perf_probe test
+#	Author: Michael Petlan <[email protected]>
+#	Author: Masami Hiramatsu <[email protected]>
+#
+#	Description:
+#
+#		This test tests basic functionality of perf probe command.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### help message
+
+if [ "$PARAM_GENERAL_HELP_TEXT_CHECK" = "y" ]; then
+	# test that a help message is shown and looks reasonable
+	$CMD_PERF probe --help > $LOGS_DIR/basic_helpmsg.log 2> $LOGS_DIR/basic_helpmsg.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "PERF-PROBE" "NAME" "SYNOPSIS" "DESCRIPTION" "OPTIONS" "PROBE\s+SYNTAX" "PROBE\s+ARGUMENT" "LINE\s+SYNTAX" < $LOGS_DIR/basic_helpmsg.log
+	CHECK_EXIT_CODE=$?
+	../common/check_all_patterns_found.pl "LAZY\s+MATCHING" "FILTER\s+PATTERN" "EXAMPLES" "SEE\s+ALSO" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "vmlinux" "module=" "source=" "verbose" "quiet" "add=" "del=" "list.*EVENT" "line=" "vars=" "externs" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_all_patterns_found.pl "no-inlines" "funcs.*FILTER" "filter=FILTER" "force" "dry-run" "max-probes" "exec=" "demangle-kernel" < $LOGS_DIR/basic_helpmsg.log
+	(( CHECK_EXIT_CODE += $? ))
+	../common/check_no_patterns_found.pl "No manual entry for" < $LOGS_DIR/basic_helpmsg.err
+	(( CHECK_EXIT_CODE += $? ))
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "help message"
+	(( TEST_RESULT += $? ))
+else
+	print_testcase_skipped "help message"
+fi
+
+
+### usage message
+
+# without any args perf-probe should print usage
+$CMD_PERF probe 2> $LOGS_DIR/basic_usage.log > /dev/null
+
+../common/check_all_patterns_found.pl "[Uu]sage" "perf probe" "verbose" "quiet" "add" "del" "force" "line" "vars" "externs" "range" < $LOGS_DIR/basic_usage.log
+CHECK_EXIT_CODE=$?
+
+print_results 0 $CHECK_EXIT_CODE "usage message"
+(( TEST_RESULT += $? ))
+
+
+### quiet switch
+
+# '--quiet' should mute all output
+$CMD_PERF probe --quiet --add vfs_read > $LOGS_DIR/basic_quiet01.log 2> $LOGS_DIR/basic_quiet01.err
+PERF_EXIT_CODE=$?
+$CMD_PERF probe --quiet --del vfs_read > $LOGS_DIR/basic_quiet03.log 2> $LOGS_DIR/basic_quiet02.err
+(( PERF_EXIT_CODE += $? ))
+
+test "`cat $LOGS_DIR/basic_quiet*log $LOGS_DIR/basic_quiet*err | wc -l`" -eq 0
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "quiet switch"
+(( TEST_RESULT += $? ))
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#19 lore raw

PATCH v2 6/11 perf testsuite probe: Add test for invalid options

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Test if various incompatible options are correctly handled-rejected. It is run as a part of perftool-testsuite_probe test case.

---
 .../shell/base_probe/test_invalid_options.sh  | 79 +++++++++++++++++++
 1 file changed, 79 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
tools/perf/tests/shell/base_probe/test_invalid_options.sh new file +79 -0
new file mode 100755
index 0000000000..1fedfd8b0d
@@ -0,0 +1,79 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_invalid_options of perf_probe test
+#	Author: Masami Hiramatsu <[email protected]>
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		This test checks whether the invalid and incompatible options are reported
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### missing argument
+
+# some options require an argument
+for opt in '-a' '-d' '-L' '-V'; do
+	! $CMD_PERF probe $opt 2> $LOGS_DIR/invalid_options_missing_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .* requires a value" < $LOGS_DIR/invalid_options_missing_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "missing argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unnecessary argument
+
+# some options may omit the argument
+for opt in '-F' '-l'; do
+	$CMD_PERF probe -F > /dev/null 2> $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	PERF_EXIT_CODE=$?
+
+	test ! -s $LOGS_DIR/invalid_options_unnecessary_argument$opt.err
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "unnecessary argument for $opt"
+	(( TEST_RESULT += $? ))
+done
+
+
+### mutually exclusive options
+
+# some options are mutually exclusive
+test -e $LOGS_DIR/invalid_options_mutually_exclusive.log && rm -f $LOGS_DIR/invalid_options_mutually_exclusive.log
+for opt in '-a xxx -d xxx' '-a xxx -L foo' '-a xxx -V foo' '-a xxx -l' '-a xxx -F' \
+		'-d xxx -L foo' '-d xxx -V foo' '-d xxx -l' '-d xxx -F' \
+		'-L foo -V bar' '-L foo -l' '-L foo -F' '-V foo -l' '-V foo -F' '-l -F'; do
+	! $CMD_PERF probe $opt > /dev/null 2> $LOGS_DIR/aux.log
+	PERF_EXIT_CODE=$?
+
+	../common/check_all_patterns_found.pl "Error: switch .+ cannot be used with switch .+" < $LOGS_DIR/aux.log
+	CHECK_EXIT_CODE=$?
+
+	print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "mutually exclusive options :: $opt"
+	(( TEST_RESULT += $? ))
+
+	# gather the logs
+	cat $LOGS_DIR/aux.log | grep "Error" >> $LOGS_DIR/invalid_options_mutually_exclusive.log
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#20 lore raw

PATCH v2 7/11 perf testsuite probe: Add test for line semantics

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

The perf-probe command uses a specific semantics to describe probes. Test some patterns that are known to be both valid and invalid if they are handled appropriately.

This test is run as a part of perftool-testsuite_probe test case.

---
 .../shell/base_probe/test_line_semantics.sh   | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
tools/perf/tests/shell/base_probe/test_line_semantics.sh new file +55 -0
new file mode 100755
index 0000000000..d8f4bde0f5
@@ -0,0 +1,55 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	test_line_semantics of perf_probe test
+#	Author: Masami Hiramatsu <[email protected]>
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		This test checks whether the semantic errors of line option's
+#		arguments are properly reported.
+#
+
+# include working environment
+. ../common/init.sh
+
+TEST_RESULT=0
+
+if ! check_kprobes_available; then
+	print_overall_skipped
+	exit 0
+fi
+
+
+### acceptable --line descriptions
+
+# testing acceptance of valid patterns for the '--line' option
+VALID_PATTERNS="func func:10 func:0-10 func:2+10 [email protected] [email protected]:1 source.c:1 source.c:1+1 source.c:1-10"
+for desc in $VALID_PATTERNS; do
+	! ( $CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error" )
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "acceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+### unacceptable --line descriptions
+
+# testing handling of invalid patterns for the '--line' option
+INVALID_PATTERNS="func:foo func:1-foo func:1+foo func;lazy\*pattern"
+for desc in $INVALID_PATTERNS; do
+	$CMD_PERF probe --line $desc 2>&1 | grep -q "Semantic error"
+	CHECK_EXIT_CODE=$?
+
+	print_results 0 $CHECK_EXIT_CODE "unacceptable descriptions :: $desc"
+	(( TEST_RESULT += $? ))
+done
+
+
+# print overall results
+print_overall_results "$TEST_RESULT"
+exit $?
Signature · 1 line
2.43.0
#21 lore raw

PATCH v2 8/11 perf testsuite: Add common output checking helper

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

As a form of validation, it is a common practice to check the outputs of commands whether they contain expected patterns or match a certain regular expression.

This output checking helper is designed to allow checking stderr output of perf commands for unexpected messages, while ignoring messages that are known to be harmless, e.g.:
"Lowering default frequency rate to \d+\."
"\d+ out of order events recorded."
etc.

---
 .../shell/common/check_errors_whitelisted.pl  | 51 +++++++++++++++++++
 1 file changed, 51 insertions(+)
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
tools/perf/tests/shell/common/check_errors_whitelisted.pl new file +51 -0
new file mode 100755
index 0000000000..c57d355dd7
@@ -0,0 +1,51 @@
+#!/usr/bin/perl
+# SPDX-License-Identifier: GPL-2.0
+
+$whitelist_file = shift;
+
+if (defined $whitelist_file)
+{
+	open (INFILE, $whitelist_file) or die "Checker error: Unable to open the whitelist file: $whitelist_file\n";
+	@regexps = <INFILE>;
+	close INFILE or die "Checker error: Unable to close the whitelist file: $whitelist_file\n";
+}
+else
+{
+	@regexps = ();
+}
+
+$max_printed_lines = 20;
+$max_printed_lines = $ENV{TESTLOG_ERR_MSG_MAX_LINES} if (defined $ENV{TESTLOG_ERR_MSG_MAX_LINES});
+
+$quiet = 1;
+$quiet = 0 if (defined $ENV{TESTLOG_VERBOSITY} && $ENV{TESTLOG_VERBOSITY} ge 2);
+
+$passed = 1;
+$lines_printed = 0;
+
+while (<STDIN>)
+{
+	s/\n//;
+
+	$line_matched = 0;
+	for $r (@regexps)
+	{
+		chomp $r;
+		if (/$r/)
+		{
+			$line_matched = 1;
+			last;
+		}
+	}
+
+	unless ($line_matched)
+	{
+		if ($lines_printed++ < $max_printed_lines)
+		{
+			print "Line did not match any pattern: \"$_\"\n" unless $quiet;
+		}
+		$passed = 0;
+	}
+}
+
+exit ($passed == 0);
Signature · 1 line
2.43.0
#22 lore raw

PATCH v2 9/11 perf testsuite report: Add test for perf-report basic functionality

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Test basic execution and some options of perf-report subcommand, like show-nr-samples, header, showcpuutilization, pid and symbol filtering.

---
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 tools/perf/tests/shell/common/settings.sh     |   2 +
 4 files changed, 229 insertions(+)
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
tools/perf/tests/shell/base_report/setup.sh new file +32 -0
new file mode 100755
index 0000000000..4caa496660
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0
+
+#
+#	setup.sh of perf report test
+#	Author: Michael Petlan <[email protected]>
+#
+#	Description:
+#
+#		We need some sample data for perf-report testing
+#
+#
+
+# include working environment
+. ../common/init.sh
+
+test -d "$HEADER_TAR_DIR" || mkdir -p "$HEADER_TAR_DIR"
+
+SW_EVENT="cpu-clock"
+
+$CMD_PERF record -asdg -e $SW_EVENT -o $CURRENT_TEST_DIR/perf.data -- $CMD_LONGER_SLEEP 2> $LOGS_DIR/setup.log
+PERF_EXIT_CODE=$?
+
+../common/check_all_patterns_found.pl "$RE_LINE_RECORD1" "$RE_LINE_RECORD2" < $LOGS_DIR/setup.log
+CHECK_EXIT_CODE=$?
+
+print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "prepare the perf.data file"
+TEST_RESULT=$?
+
+print_overall_results $TEST_RESULT
+exit $?
tools/perf/tests/shell/base_report/stderr-whitelist.txt new file +5 -0
new file mode 100644
index 0000000000..e3341401b4
@@ -0,0 +1,5 @@
+no symbols found in .*, maybe install a debug package
+was updated .*is prelink enabled.+ Restart the long running apps that use it
+Warning:
+\d+ out of order events recorded.
+detected invalid bpf_prog_info
tools/perf/tests/shell/base_report/test_basic.sh new file +190 -0

190 lines not on this page · raw message

tools/perf/tests/shell/common/settings.sh +2 -0
index 2e50dd0bff..cba1b338f9 100644
@@ -80,12 +80,14 @@ if [ -n "$PERFSUITE_RUN_DIR" ]; then
 	export CURRENT_TEST_DIR="$PERFSUITE_RUN_DIR/$TEST_NAME"
 	export MAKE_TARGET_DIR="$CURRENT_TEST_DIR/examples"
 	export LOGS_DIR="$CURRENT_TEST_DIR/logs"
+	export HEADER_TAR_DIR="$CURRENT_TEST_DIR/header_tar"
 	test -d "$CURRENT_TEST_DIR" || mkdir -p "$CURRENT_TEST_DIR"
 	test -d "$LOGS_DIR" || mkdir -p "$LOGS_DIR"
 else
 	# when $PERFSUITE_RUN_DIR is not set, logs will be placed here
 	export CURRENT_TEST_DIR="."
 	export LOGS_DIR="."
+	export HEADER_TAR_DIR="./header_tar"
 fi
 
 
Signature · 1 line
2.43.0
#23 lore raw

PATCH v2 10/11 perf testsuite report: Add test case for perf report

In reply to [email protected]

From: Veronika Molnarova <[email protected]>

Add new perf report test case that acts as an entry element in perf test list. Runs multiple subtests from directory "base_report", which can be expanded without further editing.

---
 .../tests/shell/perftool-testsuite_report.sh  | 23 +++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
tools/perf/tests/shell/perftool-testsuite_report.sh new file +23 -0
new file mode 100755
index 0000000000..973012ce92
@@ -0,0 +1,23 @@
+#!/bin/bash
+# perftool-testsuite_report
+# SPDX-License-Identifier: GPL-2.0
+
+test -d "$(dirname "$0")/base_report" || exit 2
+cd "$(dirname "$0")/base_report" || exit 2
+status=0
+
+PERFSUITE_RUN_DIR=$(mktemp -d /tmp/"$(basename "$0" .sh)".XXX)
+export PERFSUITE_RUN_DIR
+
+for testcase in setup.sh test_*; do                  # skip setup.sh if not present or not executable
+     test -x "$testcase" || continue
+     ./"$testcase"
+     (( status += $? ))
+done
+
+if ! [ "$PERFTEST_KEEP_LOGS" = "y" ]; then
+	rm -rf "$PERFSUITE_RUN_DIR"
+fi
+
+test $status -ne 0 && exit 1
+exit 0
Signature · 1 line
2.43.0
#24 lore raw

PATCH v2 11/11 perf testsuite: Install perf-report tests

In reply to [email protected]

From: Michael Petlan <[email protected]>

---
 tools/perf/Makefile.perf | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/Makefile.perf +2 -0
index e6d56b5553..b921b19c39 100644
@@ -1081,6 +1081,8 @@ install-tests: all install-gtk
 		$(INSTALL) tests/shell/common/*.pl '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/common'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
 		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
+		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight' ; \
 		$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
 	$(Q)$(MAKE) -C tests/shell/coresight install-tests
Signature · 1 line
2.43.0
#25 lore raw
In reply to [email protected]

Ping

Quoted 52 lines · On 7/2/24 13:08, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Hello,

rebased the patches for the current perf-tools-next branch in v2 as it was not applying.

Thanks for the notice,
Veronika

Michael Petlan (2):
perf tests shell: Skip base_* dirs in test script search perf testsuite: Install perf-report tests

Veronika Molnarova (9):
perf testsuite: Merge settings files for shell tests perf testsuite: Fix shellcheck warnings
perf testsuite probe: Add test for blacklisted kprobes handling perf testsuite probe: Add test for basic perf-probe options perf testsuite probe: Add test for invalid options
perf testsuite probe: Add test for line semantics
perf testsuite: Add common output checking helper
perf testsuite report: Add test for perf-report basic functionality perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  67 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  80 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  79 ++++++++
 .../shell/base_probe/test_line_semantics.sh   |  55 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  51 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 640 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
#26 lore raw
Quoted 1 line · Veronika Molnarova

Ping

Since this v2 series came from you, we need to have your Signed-off-by in Michael's patch, please provide them in response to this message.

I'm adding them tentatively to my local branch, but I need your public confirmation to follow the rules.

- Arnaldo

Quoted 53 lines · On 7/2/24 13:08, [email protected] wrote:
Quoted 52 lines

From: Veronika Molnarova <[email protected]>

Hello,

rebased the patches for the current perf-tools-next branch in v2 as it was not applying.

Thanks for the notice,
Veronika

Michael Petlan (2):
perf tests shell: Skip base_* dirs in test script search perf testsuite: Install perf-report tests

Veronika Molnarova (9):
perf testsuite: Merge settings files for shell tests perf testsuite: Fix shellcheck warnings
perf testsuite probe: Add test for blacklisted kprobes handling perf testsuite probe: Add test for basic perf-probe options perf testsuite probe: Add test for invalid options
perf testsuite probe: Add test for line semantics
perf testsuite: Add common output checking helper
perf testsuite report: Add test for perf-report basic functionality perf testsuite report: Add test case for perf report

 tools/perf/Makefile.perf                      |   2 +
 tools/perf/tests/shell/base_probe/settings.sh |  48 -----
 .../base_probe/test_adding_blacklisted.sh     |  67 ++++++
 .../shell/base_probe/test_adding_kernel.sh    |   3 -
 .../perf/tests/shell/base_probe/test_basic.sh |  80 ++++++++
 .../shell/base_probe/test_invalid_options.sh  |  79 ++++++++
 .../shell/base_probe/test_line_semantics.sh   |  55 +++++
 tools/perf/tests/shell/base_report/setup.sh   |  32 +++
 .../shell/base_report/stderr-whitelist.txt    |   5 +
 .../tests/shell/base_report/test_basic.sh     | 190 ++++++++++++++++++
 .../shell/common/check_errors_whitelisted.pl  |  51 +++++
 tools/perf/tests/shell/common/init.sh         |  31 ++-
 tools/perf/tests/shell/common/settings.sh     |  28 ++-
 .../tests/shell/perftool-testsuite_report.sh  |  23 +++
 tools/perf/tests/tests-scripts.c              |   2 +
 15 files changed, 640 insertions(+), 56 deletions(-)
 delete mode 100644 tools/perf/tests/shell/base_probe/settings.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_basic.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_invalid_options.sh
 create mode 100755 tools/perf/tests/shell/base_probe/test_line_semantics.sh
 create mode 100755 tools/perf/tests/shell/base_report/setup.sh
 create mode 100644 tools/perf/tests/shell/base_report/stderr-whitelist.txt
 create mode 100755 tools/perf/tests/shell/base_report/test_basic.sh
 create mode 100755 tools/perf/tests/shell/common/check_errors_whitelisted.pl
 create mode 100755 tools/perf/tests/shell/perftool-testsuite_report.sh
#27 lore raw

PATCH v2 1/11 perf tests shell: Skip base_* dirs in test script search

From: Michael Petlan <[email protected]>

The test scripts in base_* directories currently have their own drivers that run them. Before this patch, the shell test-suite generator causes them to run twice. Fix that by skipping them in the generator.

A cleaner solution (for future) will be to use the directory structure idea (introduced by Carsten Haitzler in 7391db645938 ("perf test:
Refactor shell tests allowing subdirs")) to generate test entries with subtests, like:

$ perf test list
[...]
97: perf probe shell tests
97:1: perf probe basic functionality
97:2: perf probe tests with arguments
97:3: perf probe invalid options handling
[...]

There is already a lot of shell test scripts and many are about to come, so there is a need for some hierarchy.

---
 tools/perf/tests/tests-scripts.c | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/tests/tests-scripts.c +2 -0
index e2042b3682..01971e9974 100644
@@ -222,6 +222,8 @@ static void append_scripts_in_dir(int dir_fd,
 			if (!S_ISDIR(st.st_mode))
 				continue;
 		}
+		if (strncmp(ent->d_name, "base_", 5) == 0)
+			continue; /* Skip scripts that have a separate driver. */
 		fd = openat(dir_fd, ent->d_name, O_PATH);
 		append_scripts_in_dir(fd, result, result_sz);
 	}
Signature · 1 line
2.43.0
#28 lore raw

PATCH v2 11/11 perf testsuite: Install perf-report tests

In reply to Arnaldo Carvalho de Melo

From: Michael Petlan <[email protected]>

---
 tools/perf/Makefile.perf | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/Makefile.perf +2 -0
index e6d56b5553..b921b19c39 100644
@@ -1081,6 +1081,8 @@ install-tests: all install-gtk
 		$(INSTALL) tests/shell/common/*.pl '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/common'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
 		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_probe'; \
+		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
+		$(INSTALL) tests/shell/base_probe/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/base_report'; \
 		$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight' ; \
 		$(INSTALL) tests/shell/coresight/*.sh '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/tests/shell/coresight'
 	$(Q)$(MAKE) -C tests/shell/coresight install-tests
Signature · 1 line
2.43.0
#29 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 3 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Veronika Molnarova

Ping

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

- Arnaldo

root@x1:~# perf test -vvvvv 88
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 2780578
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: -a
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: --add
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf list
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf probe -l
-- [ PASS ] -- perf_probe :: test_adding_kernel :: using added probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: deleting added probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing removed probe (should NOT be listed)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: dry run :: adding probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: first probe adding
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (without force)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (with force)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: using doubled probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: removing multiple probes
-- [ PASS ] -- perf_probe :: test_adding_kernel :: wildcard adding support
-- [ PASS ] -- perf_probe :: test_adding_kernel :: non-existing variable
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: add
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: record
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function argument probing :: script
## [ PASS ] ## perf_probe :: test_adding_kernel SUMMARY
-- [ SKIP ] -- perf_probe :: test_basic :: help message :: testcase skipped
-- [ PASS ] -- perf_probe :: test_basic :: usage message
-- [ PASS ] -- perf_probe :: test_basic :: quiet switch
## [ PASS ] ## perf_probe :: test_basic SUMMARY
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -a
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -d
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -L
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -V
-- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -d xxx
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -L foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -V foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -L foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -V foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -V bar
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -l -F
## [ PASS ] ## perf_probe :: test_invalid_options SUMMARY
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:0-10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:2+10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: [email protected]
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: [email protected]:1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1+1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1-10
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1-foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1+foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func;lazy\*pattern
## [ PASS ] ## perf_probe :: test_line_semantics SUMMARY
---- end(-1) ----
 88: perftool-testsuite_probe                                        : FAILED!
root@x1:~# 
#30 lore raw
Quoted 10 lines · Arnaldo Carvalho de Melo
Quoted 3 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Veronika Molnarova

Ping

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

- Arnaldo

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

If the variable "PERFTEST_KEEP_LOGS=y" is set to keep debug logs located in "/tmp/perftool-testsuite_probe.XXX", those could provide some more info.

Would it be possible to get system specification for the possible issue reproduction?

Thanks,
Veronika

Quoted 77 lines
root@x1:~# perf test -vvvvv 88
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 2780578
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: -a
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: --add
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf list
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing added probe :: perf probe -l
-- [ PASS ] -- perf_probe :: test_adding_kernel :: using added probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: deleting added probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: listing removed probe (should NOT be listed)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: dry run :: adding probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: first probe adding
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (without force)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: force-adding probes :: second probe adding (with force)
-- [ PASS ] -- perf_probe :: test_adding_kernel :: using doubled probe
-- [ PASS ] -- perf_probe :: test_adding_kernel :: removing multiple probes
-- [ PASS ] -- perf_probe :: test_adding_kernel :: wildcard adding support
-- [ PASS ] -- perf_probe :: test_adding_kernel :: non-existing variable
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: add
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function with retval :: record
-- [ PASS ] -- perf_probe :: test_adding_kernel :: function argument probing :: script
## [ PASS ] ## perf_probe :: test_adding_kernel SUMMARY
-- [ SKIP ] -- perf_probe :: test_basic :: help message :: testcase skipped
-- [ PASS ] -- perf_probe :: test_basic :: usage message
-- [ PASS ] -- perf_probe :: test_basic :: quiet switch
## [ PASS ] ## perf_probe :: test_basic SUMMARY
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -a
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -d
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -L
-- [ PASS ] -- perf_probe :: test_invalid_options :: missing argument for -V
-- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: unnecessary argument for -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -d xxx
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -L foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -V foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -a xxx -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -L foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -V foo
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -d xxx -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -V bar
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -L foo -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -l
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -V foo -F
-- [ PASS ] -- perf_probe :: test_invalid_options :: mutually exclusive options :: -l -F
## [ PASS ] ## perf_probe :: test_invalid_options SUMMARY
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:0-10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: func:2+10
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: [email protected]
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: [email protected]:1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1+1
-- [ PASS ] -- perf_probe :: test_line_semantics :: acceptable descriptions :: source.c:1-10
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1-foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func:1+foo
-- [ PASS ] -- perf_probe :: test_line_semantics :: unacceptable descriptions :: func;lazy\*pattern
## [ PASS ] ## perf_probe :: test_line_semantics SUMMARY
---- end(-1) ----
 88: perftool-testsuite_probe                                        : FAILED!
root@x1:~# 
#31 lore raw
Quoted 17 lines · Veronika Molnarova
Quoted 8 lines · Arnaldo Carvalho de Melo
Quoted 2 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Veronika Molnarova

Ping

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

Quoted 5 lines

If the variable "PERFTEST_KEEP_LOGS=y" is set to keep debug logs located in "/tmp/perftool-testsuite_probe.XXX", those could provide some more info.

Would it be possible to get system specification for the possible issue reproduction?

root@x1:~# head /etc/os-release 
NAME="Fedora Linux"
VERSION="40 (Workstation Edition)"
ID=fedora
VERSION_ID=40
VERSION_CODENAME=""
PLATFORM_ID="platform:f40"
PRETTY_NAME="Fedora Linux 40 (Workstation Edition)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:40"
root@x1:~# uname -a
Linux x1 6.10.4-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Aug 11 15:32:50 UTC 2024 x86_64 GNU/Linux
root@x1:~#
root@x1:~# perf probe warn_thunk_thunk
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~# perf probe -vvv -L warn_thunk_thunk
capget syscall failed (No such file or directory - 2) fall back on root check
Looking at the vmlinux_path (8 entries long)
Using /usr/lib/debug/lib/modules/6.10.4-200.fc40.x86_64/vmlinux for symbols
Open Debuginfo file: /usr/lib/debug/.build-id/32/6993e6ef36099bc55f5d9c9d3c6cefe2c9a5fb.debug
fname: (null), lineno:0
New line range: 0 to 2147483647
Failed to get the declared file name of warn_thunk_thunk
path: (null)
Debuginfo analysis failed.
  Error: Failed to show lines. Reason: Invalid argument (Code: -22)
root@x1:~#
root@x1:~# perf probe -L warn_thunk_thunk
Debuginfo analysis failed.
  Error: Failed to show lines.
root@x1:~# perf probe icmp_rcv
Added new event:
  probe:icmp_rcv       (on icmp_rcv)

You can now use it in all perf tools, such as:

perf record -e probe:icmp_rcv -aR sleep 1

root@x1:~# perf probe -L icmp_rcv | head
<icmp_rcv@/usr/src/debug/kernel-6.10.4/linux-6.10.4-200.fc40.x86_64/net/ipv4/icmp.c:0>
      0  int icmp_rcv(struct sk_buff *skb)
         {
      2  	enum skb_drop_reason reason = SKB_DROP_REASON_NOT_SPECIFIED;
         	struct rtable *rt = skb_rtable(skb);
         	struct net *net = dev_net(rt->dst.dev);
         	struct icmphdr *icmph;
         	if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb)) {
      8  		struct sec_path *sp = skb_sec_path(skb);

root@x1:~#

root@x1:~# grep warn_thunk_thunk /proc/kallsyms 
ffffffff97004ae0 T __pfx_warn_thunk_thunk
ffffffff97004af0 T warn_thunk_thunk
root@x1:~#
root@x1:~# rm -f /tmp/perftool-testsuite_probe.*
rm: cannot remove '/tmp/perftool-testsuite_probe.C62': Is a directory
rm: cannot remove '/tmp/perftool-testsuite_probe.cyR': Is a directory
rm: cannot remove '/tmp/perftool-testsuite_probe.XOQ': Is a directory
root@x1:~# rm -rf /tmp/perftool-testsuite_probe.*
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : FAILED!
root@x1:~#
root@x1:~# ls -la /tmp/perftool-testsuite_probe.RJh/
total 0
drwx------.  3 root root   60 Sep  2 11:24 .
drwxrwxrwt. 30 root root 1900 Sep  2 11:24 ..
drwxr-xr-x.  3 root root   80 Sep  2 11:24 perf_probe
root@x1:~# ls -la /tmp/perftool-testsuite_probe.RJh/perf_probe/
total 80
drwxr-xr-x. 3 root root    80 Sep  2 11:24 .
drwx------. 3 root root    60 Sep  2 11:24 ..
drwxr-xr-x. 2 root root   740 Sep  2 11:24 logs
-rw-------. 1 root root 80170 Sep  2 11:24 perf.data
root@x1:~# ls -la /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/
total 108
drwxr-xr-x. 2 root root  740 Sep  2 11:24 .
drwxr-xr-x. 3 root root   80 Sep  2 11:24 ..
-rw-r--r--. 1 root root  191 Sep  2 11:24 adding_blacklisted.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 adding_blacklisted_list.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 adding_blacklisted.log
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_add--add.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_add-a.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_add.err
-rw-r--r--. 1 root root 7280 Sep  2 11:24 adding_kernel_adding_wildcard.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_dryrun.err
-rw-r--r--. 1 root root  165 Sep  2 11:24 adding_kernel_forceadd_01.err
-rw-r--r--. 1 root root  214 Sep  2 11:24 adding_kernel_forceadd_02.err
-rw-r--r--. 1 root root  169 Sep  2 11:24 adding_kernel_forceadd_03.err
-rw-r--r--. 1 root root  201 Sep  2 11:24 adding_kernel_func_retval_add.err
-rw-r--r--. 1 root root  161 Sep  2 11:24 adding_kernel_func_retval_record.err
-rw-r--r--. 1 root root 1651 Sep  2 11:24 adding_kernel_func_retval_script.log
-rw-r--r--. 1 root root   58 Sep  2 11:24 adding_kernel_list-l.log
-rw-r--r--. 1 root root   72 Sep  2 11:24 adding_kernel_list.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 adding_kernel_list_removed.log
-rw-r--r--. 1 root root  328 Sep  2 11:24 adding_kernel_nonexisting.err
-rw-r--r--. 1 root root   38 Sep  2 11:24 adding_kernel_removing.err
-rw-r--r--. 1 root root   78 Sep  2 11:24 adding_kernel_removing_wildcard.err
-rw-r--r--. 1 root root  294 Sep  2 11:24 adding_kernel_using_probe.log
-rw-r--r--. 1 root root  131 Sep  2 11:24 adding_kernel_using_two.log
-rw-r--r--. 1 root root  589 Sep  2 11:24 aux.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet01.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet01.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet02.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 basic_quiet03.log
-rw-r--r--. 1 root root 3064 Sep  2 11:24 basic_usage.log
-rw-r--r--. 1 root root  938 Sep  2 11:24 invalid_options_missing_argument-a.err
-rw-r--r--. 1 root root  485 Sep  2 11:24 invalid_options_missing_argument-d.err
-rw-r--r--. 1 root root  517 Sep  2 11:24 invalid_options_missing_argument-L.err
-rw-r--r--. 1 root root  535 Sep  2 11:24 invalid_options_missing_argument-V.err
-rw-r--r--. 1 root root  750 Sep  2 11:24 invalid_options_mutually_exclusive.log
-rw-r--r--. 1 root root    0 Sep  2 11:24 invalid_options_unnecessary_argument-F.err
-rw-r--r--. 1 root root    0 Sep  2 11:24 invalid_options_unnecessary_argument-l.err
root@x1:~#
root@x1:~# grep "decl_line" /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/*
/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

- Arnaldo

#32 lore raw
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 7 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Quoted 6 lines
Quoted 6 lines

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

<SNIP>

Quoted 11 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#
Quoted 2 lines

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

- Arnaldo

tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh +2 -1
index b5dc10b2a73810b3..14290d08a4a96eaf 100755
@@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
 REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
 REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
 REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
-../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
 CHECK_EXIT_CODE=$?
 
 print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
#33 lore raw
Quoted 37 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 7 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Quoted 6 lines
Quoted 6 lines

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

<SNIP>

Quoted 11 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#
Quoted 2 lines

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Quoted 8 lines

Did the patch below, now it passes, Ack?

root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

The logs continue to point out that problem:

root@x1:~# cat /tmp/perftool-testsuite_probe.Fu4/perf_probe/logs/adding_blacklisted.err 
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
Quoted 16 lines

- Arnaldo

tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh +2 -1
index b5dc10b2a73810b3..14290d08a4a96eaf 100755
@@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
 REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
 REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
 REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
-../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
 CHECK_EXIT_CODE=$?
 
 print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
#34 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 39 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 7 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Quoted 6 lines
Quoted 6 lines

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

<SNIP>

Quoted 11 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#
Quoted 2 lines

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

Quoted 8 lines
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

- Arnaldo

Yeah, that would fix the test case, the question is whether we should accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

Quoted 16 lines
tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh +2 -1
index b5dc10b2a73810b3..14290d08a4a96eaf 100755
@@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
 REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
 REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
 REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
-../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
 CHECK_EXIT_CODE=$?
 
 print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
#35 lore raw
Quoted 47 lines · Veronika Molnarova
Quoted 44 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 7 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Quoted 6 lines
Quoted 6 lines

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

<SNIP>

Quoted 11 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.
  Error: Failed to add events.
root@x1:~#
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#
Quoted 2 lines

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#
Quoted 5 lines
Quoted 3 lines

Or am I missing the point? :-)

- Arnaldo

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

Thanks in advance!

- Arnaldo

Quoted 16 lines
Quoted 16 lines
tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh +2 -1
index b5dc10b2a73810b3..14290d08a4a96eaf 100755
@@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
 REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
 REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
 REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
-../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
 CHECK_EXIT_CODE=$?
 
 print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
maintainer Acked-by Masami Hiramatsu (Google)
#36 lore raw
Quoted 22 lines · Arnaldo Carvalho de Melo
Quoted 9 lines · Veronika Molnarova
Quoted 7 lines · Arnaldo Carvalho de Melo
Quoted 4 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Veronika Molnarova

On 8/30/24 17:27, Arnaldo Carvalho de Melo wrote:

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

Quoted 8 lines
Quoted 6 lines
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 6 lines
Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

And b4 doesn't like it, I'm trying to find it manually

⬢[acme@toolbox perf-tools-next]$ b4 am -ctsl --cc-trailers [email protected] Grabbing thread from lore.kernel.org/all/[email protected]/t.mbox.gz Checking for newer revisions
Grabbing search results from lore.kernel.org
Analyzing 21 messages in the thread
WARNING: duplicate messages found at index 1
Subject 1: tools/perf: Fix to avoid crashing if DW_AT_decl_file is NULL
Subject 2: perf probe: add test for regression introduced by switch to die_get_decl_file
2 is not a reply... assume additional patch
WARNING: duplicate messages found at index 2
Subject 1: tools/perf: Fix to use dwarf_attr_integrate for generic attr accessor Subject 2: perf probe: add test for regression introduced by switch to die_get_decl_file
2 is not a reply... assume additional patch
^CTraceback (most recent call last):
File "/home/acme/git/b4/src/b4/command.py", line 428, in <module>

    cmd()

File "/home/acme/git/b4/src/b4/command.py", line 410, in cmd

And then I noticed that his _was_ already merged, see below, so we're back to the previous square, can you try to continue this analysis, I'm rather busy with preparing for the upcoming conferences, so any help I can get is appreciated :-)

- Arnaldo

commit c66e1c68c13b872505f25ab641c44b77313ee7fe

Author: Georg Müller <[email protected]>
Date:   Wed Jun 28 10:45:51 2023 +0200

perf probe: Read DWARF files from the correct CU

After switching from dwarf_decl_file() to die_get_decl_file(), it is not possible to add probes for certain functions:

$ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed A function DIE doesn't have decl_line. Maybe broken DWARF? A function DIE doesn't have decl_line. Maybe broken DWARF? Probe point 'match_unit_removed' not found.
Error: Failed to add events.

The problem is that die_get_decl_file() uses the wrong CU to search for the file. elfutils commit e1db5cdc9f has some good explanation for this:

dwarf_decl_file uses dwarf_attr_integrate to get the DW_AT_decl_file attribute. This means the attribute might come from a different DIE in a different CU. If so, we need to use the CU associated with the attribute, not the original DIE, to resolve the file name.

This patch uses the same source of information as elfutils: use attribute DW_AT_decl_file and use this CU to search for the file.

Fixes dc9a5d2ccd5c823c ("perf probe: Fix to get declared file name from clang DWARF5")
Cc Adrian Hunter <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Ian Rogers <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
#37 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 21 lines · On Mon, 2 Sep 2024 15:46:42 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Quoted 20 lines · Veronika Molnarova
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines · Arnaldo Carvalho de Melo
Quoted 14 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

right, thunk functions should not be probed.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

<SNIP>

Quoted 5 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.

Yeah, thunk functions are defined by asm code. No DWARF is available.

Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
Quoted 8 lines

Error: Failed to add events.
root@x1:~#

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#
Quoted 5 lines
Quoted 3 lines

Or am I missing the point? :-)

- Arnaldo

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 6 lines
Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

Thanks for update my email. Yeah, it looks good to me.

Thanks,

Quoted 21 lines

Thanks in advance!

- Arnaldo

Quoted 16 lines
Quoted 16 lines
tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh +2 -1
index b5dc10b2a73810b3..14290d08a4a96eaf 100755
@@ -42,7 +42,8 @@ REGEX_ERROR_MESSAGE="Error: Failed to add events."
 REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
 REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
 REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
-../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err
+REGEX_MISSING_DECL_LINE="A function DIE doesn't have decl_line. Maybe broken DWARF?"
+../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" "$REGEX_MISSING_DECL_LINE" < $LOGS_DIR/adding_blacklisted.err
 CHECK_EXIT_CODE=$?
 
 print_results $PERF_EXIT_CODE $CHECK_EXIT_CODE "adding blacklisted function $BLACKFUNC"
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
#38 lore raw
Quoted 29 lines · Masami Hiramatsu
Quoted 19 lines · On Mon, 2 Sep 2024 15:46:42 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Quoted 18 lines · Veronika Molnarova
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines · Arnaldo Carvalho de Melo
Quoted 14 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

right, thunk functions should not be probed.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<SNIP>

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.

Yeah, thunk functions are defined by asm code. No DWARF is available.

Sure, but then, how would that be detected by the test?

To recap, the ./tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh test does:

# skip if not supported
BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
if [ -z "$BLACKFUNC" ]; then
        print_overall_skipped
        exit 0
fi
# remove all previously added probes
clear_all_probes

### adding blacklisted function

# functions from blacklist should be skipped by perf probe
! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
PERF_EXIT_CODE=$?

REGEX_SCOPE_FAIL="Failed to find scope of probe point"
REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
REGEX_ERROR_MESSAGE="Error: Failed to add events."
REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err CHECK_EXIT_CODE=$?

So it _tries_ to probe blacklisted functions and check what is that the tooling does when asked to probe things it shouldn't probe.

So, I proposed that if we get that error message (A function DIE doesn't have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did the right thing and asked if that wasn't really papering over some problem and pointed to a patch that addressed a problem that lead to that exact same message being produced, but that didn't help with our current case, with another blacklisted function, warn_thunk_thunk.

You mentioned that this is an asm function, so no DWARF, let alone correct DWARF, so the message is kinda ok, but we can't use that message as it could paper over real problems like the one fixed by the patch that Veronika pointed out.

How to figure out if a function has valid DWARF? Maybe for the moment we should have _another_ blacklist, one for functions in the blacklist that have no valid DWARF? A blacklist's blacklist?

Turtles all the way down? :-)

- Arnaldo

Quoted 27 lines
Quoted 26 lines
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines

Error: Failed to add events.
root@x1:~#

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

Quoted 8 lines
Quoted 6 lines
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

Quoted 6 lines
Quoted 2 lines

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

Thanks for update my email. Yeah, it looks good to me.

Unfortunately that cset is already there:

commit c66e1c68c13b872505f25ab641c44b77313ee7fe

Author: Georg Müller <[email protected]>
Date:   Wed Jun 28 10:45:51 2023 +0200

perf probe: Read DWARF files from the correct CU

After switching from dwarf_decl_file() to die_get_decl_file(), it is not possible to add probes for certain functions:

$ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed A function DIE doesn't have decl_line. Maybe broken DWARF? A function DIE doesn't have decl_line. Maybe broken DWARF? Probe point 'match_unit_removed' not found.
Error: Failed to add events.

⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1 v6.10
v6.10-rc1
v6.10-rc2
v6.10-rc3
v6.10-rc4
v6.10-rc5
v6.10-rc6
v6.10-rc7
v6.11-rc1
v6.11-rc2
v6.11-rc3
v6.11-rc4
v6.11-rc5
v6.11-rc6
⬢[acme@toolbox perf-tools-next]$

But..

root@number:~# perf test -vvvvv testsuite_probe |& head
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 511749
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
root@number:~# perf -v
perf version 6.11.rc6.g69e90b02e611
root@number:~#
#39 lore raw
Quoted 87 lines · Arnaldo Carvalho de Melo
Quoted 29 lines · Masami Hiramatsu
Quoted 19 lines · On Mon, 2 Sep 2024 15:46:42 -0300 Arnaldo Carvalho de Melo <[email protected]>…
Quoted 18 lines · Veronika Molnarova
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines · Arnaldo Carvalho de Melo
Quoted 14 lines · Veronika Molnarova
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Wed, Aug 28, 2024 at 04:10:55PM +0200, Veronika Molnarova wrote:

So, is there some other knob to get further output from that FAIL case? Like the command that is failing? I'll try to take a look at the sources, but are you seeing this problem with what is in perf-tools-next/perf-tools-next?

Couldn't reproduce the issue on multiple systems. There is no further way of getting more logs from the test case. The output of the failure is from the regex checking when a line wasn't matched to any possible regex. Looking at the test case "adding blacklisted function warn_thunk_thunk", the failure is caused by command 'perf probe warn_thunk_thunk', which is a blacklisted function taken from "/sys/kernel/debug/kprobes/blacklist".

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
root@x1:~# grep thunk_thunk /sys/kernel/debug/kprobes/blacklist
0xffffffff97004af0-0xffffffff97004b20	warn_thunk_thunk
root@x1:~#

right, thunk functions should not be probed.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<SNIP>

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

/tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err:A function DIE doesn't have decl_line. Maybe broken DWARF?

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.err
A function DIE doesn't have decl_line. Maybe broken DWARF?
A function DIE doesn't have decl_line. Maybe broken DWARF?
Probe point 'warn_thunk_thunk' not found.

Yeah, thunk functions are defined by asm code. No DWARF is available.

Sure, but then, how would that be detected by the test?

To recap, the ./tools/perf/tests/shell/base_probe/test_adding_blacklisted.sh test does:

# skip if not supported
BLACKFUNC=`head -n 1 /sys/kernel/debug/kprobes/blacklist 2> /dev/null | cut -f2`
if [ -z "$BLACKFUNC" ]; then
        print_overall_skipped
        exit 0
fi
# remove all previously added probes
clear_all_probes

### adding blacklisted function

# functions from blacklist should be skipped by perf probe
! $CMD_PERF probe $BLACKFUNC > $LOGS_DIR/adding_blacklisted.log 2> $LOGS_DIR/adding_blacklisted.err
PERF_EXIT_CODE=$?

REGEX_SCOPE_FAIL="Failed to find scope of probe point"
REGEX_SKIP_MESSAGE=" is blacklisted function, skip it\."
REGEX_NOT_FOUND_MESSAGE="Probe point \'$BLACKFUNC\' not found."
REGEX_ERROR_MESSAGE="Error: Failed to add events."
REGEX_INVALID_ARGUMENT="Failed to write event: Invalid argument"
REGEX_SYMBOL_FAIL="Failed to find symbol at $RE_ADDRESS"
REGEX_OUT_SECTION="$BLACKFUNC is out of \.\w+, skip it"
../common/check_all_lines_matched.pl "$REGEX_SKIP_MESSAGE" "$REGEX_NOT_FOUND_MESSAGE" "$REGEX_ERROR_MESSAGE" "$REGEX_SCOPE_FAIL" "$REGEX_INVALID_ARGUMENT" "$REGEX_SYMBOL_FAIL" "$REGEX_OUT_SECTION" < $LOGS_DIR/adding_blacklisted.err CHECK_EXIT_CODE=$?

So it _tries_ to probe blacklisted functions and check what is that the tooling does when asked to probe things it shouldn't probe.

So, I proposed that if we get that error message (A function DIE doesn't have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did the right thing and asked if that wasn't really papering over some problem and pointed to a patch that addressed a problem that lead to that exact same message being produced, but that didn't help with our current case, with another blacklisted function, warn_thunk_thunk.

You mentioned that this is an asm function, so no DWARF, let alone correct DWARF, so the message is kinda ok, but we can't use that message as it could paper over real problems like the one fixed by the patch that Veronika pointed out.

Quoted 5 lines

How to figure out if a function has valid DWARF? Maybe for the moment we should have _another_ blacklist, one for functions in the blacklist that have no valid DWARF? A blacklist's blacklist?

Turtles all the way down? :-)

We have the needed info to provide a more informative warning:

<1><da258>: Abbrev Number: 0
Compilation Unit @ offset 0xda259:

   Length:        0x3f (32-bit)
   Version:       5
   Unit Type:     DW_UT_compile (1)
   Abbrev Offset: 0x74c8
   Pointer Size:  8
 <0><da265>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <da266>   DW_AT_stmt_list   : 0x12cbd
    <da26a>   DW_AT_ranges      : 0x2d6e
    <da26e>   DW_AT_name        : (indirect string, offset: 0x1ec2f): /home/acme/git/linux/arch/x86/entry/entry.S
    <da272>   DW_AT_comp_dir    : (indirect string, offset: 0x1be93): /home/acme/git/build/v6.11-rc5+
    <da276>   DW_AT_producer    : (indirect string, offset: 0x1beb3): GNU AS 2.41
    <da27a>   DW_AT_language    : 32769 (MIPS assembler)
 <1><da27c>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da27d>   DW_AT_name        : (indirect string, offset: 0x1ec5b): entry_ibpb
    <da281>   DW_AT_external    : 1
    <da281>   DW_AT_type        : <0xda29a>
    <da282>   DW_AT_low_pc      : 0xffffffff82062790
    <da28a>   DW_AT_high_pc     : 23
 <1><da28b>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
    <da290>   DW_AT_external    : 1
    <da290>   DW_AT_type        : <0xda29a>
    <da291>   DW_AT_low_pc      : 0xffffffff81008060
    <da299>   DW_AT_high_pc     : 45

<1><da29a>: Abbrev Number: 3 (DW_TAG_unspecified_type)

root@number:~# readelf -wi `pahole --running_kernel_vmlinux` | grep warn_thunk_thunk
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
root@number:~# pahole --running_kernel_vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux
root@number:~# perf buildid-list -k
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~# file /lib/modules/6.11.0-rc5+/build/vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=4dd7f9c4507b82a5bf90671d524e5bb308104ffa, with debug_info, not stripped
root@number:~# perf buildid-list -i /lib/modules/6.11.0-rc5+/build/vmlinux
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~#

- Arnaldo

Quoted 89 lines

- Arnaldo

Quoted 27 lines
Quoted 26 lines
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines

Error: Failed to add events.
root@x1:~#

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

Quoted 8 lines
Quoted 6 lines
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

Quoted 6 lines
Quoted 2 lines

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

Thanks for update my email. Yeah, it looks good to me.

Unfortunately that cset is already there:

commit c66e1c68c13b872505f25ab641c44b77313ee7fe

Author: Georg Müller <[email protected]>
Date:   Wed Jun 28 10:45:51 2023 +0200

perf probe: Read DWARF files from the correct CU

After switching from dwarf_decl_file() to die_get_decl_file(), it is not possible to add probes for certain functions:

$ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed A function DIE doesn't have decl_line. Maybe broken DWARF? A function DIE doesn't have decl_line. Maybe broken DWARF? Probe point 'match_unit_removed' not found.
Error: Failed to add events.

⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1 v6.10
v6.10-rc1
v6.10-rc2
v6.10-rc3
v6.10-rc4
v6.10-rc5
v6.10-rc6
v6.10-rc7
v6.11-rc1
v6.11-rc2
v6.11-rc3
v6.11-rc4
v6.11-rc5
v6.11-rc6
⬢[acme@toolbox perf-tools-next]$

But..

root@number:~# perf test -vvvvv testsuite_probe |& head
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 511749
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
root@number:~# perf -v
perf version 6.11.rc6.g69e90b02e611
root@number:~#
#40 lore raw
Quoted 13 lines · Arnaldo Carvalho de Melo
Quoted 11 lines · Arnaldo Carvalho de Melo

So, I proposed that if we get that error message (A function DIE doesn't have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did the right thing and asked if that wasn't really papering over some problem and pointed to a patch that addressed a problem that lead to that exact same message being produced, but that didn't help with our current case, with another blacklisted function, warn_thunk_thunk.

You mentioned that this is an asm function, so no DWARF, let alone correct DWARF, so the message is kinda ok, but we can't use that message as it could paper over real problems like the one fixed by the patch that Veronika pointed out.

Quoted 5 lines
Quoted 3 lines

How to figure out if a function has valid DWARF? Maybe for the moment we should have _another_ blacklist, one for functions in the blacklist that have no valid DWARF? A blacklist's blacklist?

We have the needed info to provide a more informative warning:

So, perhaps we should, in the test, when we get that "broken dwarf" output, look at the debuginfo file for the kernel to see if the function is one in a DW_UT_compile (CU) that has

     <da27a>   DW_AT_language    : 32769 (MIPS assembler)

And then just have a -vv output stating that that function is being skipped because it is DWARF from assembler?

Veronika, would you be willing to try to do that?

Thanks,

- Arnaldo

Quoted 131 lines

<1><da258>: Abbrev Number: 0
Compilation Unit @ offset 0xda259:

   Length:        0x3f (32-bit)
   Version:       5
   Unit Type:     DW_UT_compile (1)
   Abbrev Offset: 0x74c8
   Pointer Size:  8
 <0><da265>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <da266>   DW_AT_stmt_list   : 0x12cbd
    <da26a>   DW_AT_ranges      : 0x2d6e
    <da26e>   DW_AT_name        : (indirect string, offset: 0x1ec2f): /home/acme/git/linux/arch/x86/entry/entry.S
    <da272>   DW_AT_comp_dir    : (indirect string, offset: 0x1be93): /home/acme/git/build/v6.11-rc5+
    <da276>   DW_AT_producer    : (indirect string, offset: 0x1beb3): GNU AS 2.41
    <da27a>   DW_AT_language    : 32769 (MIPS assembler)
 <1><da27c>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da27d>   DW_AT_name        : (indirect string, offset: 0x1ec5b): entry_ibpb
    <da281>   DW_AT_external    : 1
    <da281>   DW_AT_type        : <0xda29a>
    <da282>   DW_AT_low_pc      : 0xffffffff82062790
    <da28a>   DW_AT_high_pc     : 23
 <1><da28b>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
    <da290>   DW_AT_external    : 1
    <da290>   DW_AT_type        : <0xda29a>
    <da291>   DW_AT_low_pc      : 0xffffffff81008060
    <da299>   DW_AT_high_pc     : 45

<1><da29a>: Abbrev Number: 3 (DW_TAG_unspecified_type)

root@number:~# readelf -wi `pahole --running_kernel_vmlinux` | grep warn_thunk_thunk
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
root@number:~# pahole --running_kernel_vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux
root@number:~# perf buildid-list -k
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~# file /lib/modules/6.11.0-rc5+/build/vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=4dd7f9c4507b82a5bf90671d524e5bb308104ffa, with debug_info, not stripped
root@number:~# perf buildid-list -i /lib/modules/6.11.0-rc5+/build/vmlinux
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~#

- Arnaldo

Quoted 89 lines

- Arnaldo

Quoted 27 lines
Quoted 26 lines
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines

Error: Failed to add events.
root@x1:~#

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

Quoted 8 lines
Quoted 6 lines
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

Quoted 6 lines
Quoted 2 lines

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

Thanks for update my email. Yeah, it looks good to me.

Unfortunately that cset is already there:

commit c66e1c68c13b872505f25ab641c44b77313ee7fe

Author: Georg Müller <[email protected]>
Date:   Wed Jun 28 10:45:51 2023 +0200

perf probe: Read DWARF files from the correct CU

After switching from dwarf_decl_file() to die_get_decl_file(), it is not possible to add probes for certain functions:

$ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed A function DIE doesn't have decl_line. Maybe broken DWARF? A function DIE doesn't have decl_line. Maybe broken DWARF? Probe point 'match_unit_removed' not found.
Error: Failed to add events.

⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1 v6.10
v6.10-rc1
v6.10-rc2
v6.10-rc3
v6.10-rc4
v6.10-rc5
v6.10-rc6
v6.10-rc7
v6.11-rc1
v6.11-rc2
v6.11-rc3
v6.11-rc4
v6.11-rc5
v6.11-rc6
⬢[acme@toolbox perf-tools-next]$

But..

root@number:~# perf test -vvvvv testsuite_probe |& head
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 511749
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
root@number:~# perf -v
perf version 6.11.rc6.g69e90b02e611
root@number:~#
#41 lore raw
Quoted 24 lines · Arnaldo Carvalho de Melo
Quoted 13 lines · Arnaldo Carvalho de Melo
Quoted 11 lines · Arnaldo Carvalho de Melo

So, I proposed that if we get that error message (A function DIE doesn't have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did the right thing and asked if that wasn't really papering over some problem and pointed to a patch that addressed a problem that lead to that exact same message being produced, but that didn't help with our current case, with another blacklisted function, warn_thunk_thunk.

You mentioned that this is an asm function, so no DWARF, let alone correct DWARF, so the message is kinda ok, but we can't use that message as it could paper over real problems like the one fixed by the patch that Veronika pointed out.

Quoted 5 lines
Quoted 3 lines

How to figure out if a function has valid DWARF? Maybe for the moment we should have _another_ blacklist, one for functions in the blacklist that have no valid DWARF? A blacklist's blacklist?

We have the needed info to provide a more informative warning:

So, perhaps we should, in the test, when we get that "broken dwarf" output, look at the debuginfo file for the kernel to see if the function is one in a DW_UT_compile (CU) that has

Quoted 6 lines
     <da27a>   DW_AT_language    : 32769 (MIPS assembler)

And then just have a -vv output stating that that function is being skipped because it is DWARF from assembler?

Veronika, would you be willing to try to do that?

Okay, I can try to parse the debuginfo to check the DWARF.

Thanks,
Veronika

Quoted 137 lines

Thanks,

- Arnaldo

Quoted 131 lines

<1><da258>: Abbrev Number: 0
Compilation Unit @ offset 0xda259:

   Length:        0x3f (32-bit)
   Version:       5
   Unit Type:     DW_UT_compile (1)
   Abbrev Offset: 0x74c8
   Pointer Size:  8
 <0><da265>: Abbrev Number: 1 (DW_TAG_compile_unit)
    <da266>   DW_AT_stmt_list   : 0x12cbd
    <da26a>   DW_AT_ranges      : 0x2d6e
    <da26e>   DW_AT_name        : (indirect string, offset: 0x1ec2f): /home/acme/git/linux/arch/x86/entry/entry.S
    <da272>   DW_AT_comp_dir    : (indirect string, offset: 0x1be93): /home/acme/git/build/v6.11-rc5+
    <da276>   DW_AT_producer    : (indirect string, offset: 0x1beb3): GNU AS 2.41
    <da27a>   DW_AT_language    : 32769 (MIPS assembler)
 <1><da27c>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da27d>   DW_AT_name        : (indirect string, offset: 0x1ec5b): entry_ibpb
    <da281>   DW_AT_external    : 1
    <da281>   DW_AT_type        : <0xda29a>
    <da282>   DW_AT_low_pc      : 0xffffffff82062790
    <da28a>   DW_AT_high_pc     : 23
 <1><da28b>: Abbrev Number: 2 (DW_TAG_subprogram)
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
    <da290>   DW_AT_external    : 1
    <da290>   DW_AT_type        : <0xda29a>
    <da291>   DW_AT_low_pc      : 0xffffffff81008060
    <da299>   DW_AT_high_pc     : 45

<1><da29a>: Abbrev Number: 3 (DW_TAG_unspecified_type)

root@number:~# readelf -wi `pahole --running_kernel_vmlinux` | grep warn_thunk_thunk
    <da28c>   DW_AT_name        : (indirect string, offset: 0x1ec66): warn_thunk_thunk
root@number:~# pahole --running_kernel_vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux
root@number:~# perf buildid-list -k
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~# file /lib/modules/6.11.0-rc5+/build/vmlinux
/lib/modules/6.11.0-rc5+/build/vmlinux: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=4dd7f9c4507b82a5bf90671d524e5bb308104ffa, with debug_info, not stripped
root@number:~# perf buildid-list -i /lib/modules/6.11.0-rc5+/build/vmlinux
4dd7f9c4507b82a5bf90671d524e5bb308104ffa
root@number:~#

- Arnaldo

Quoted 89 lines

- Arnaldo

Quoted 27 lines
Quoted 26 lines
Quoted 11 lines
Quoted 11 lines
Quoted 8 lines

Error: Failed to add events.
root@x1:~#

root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted_list.log 
root@x1:~# cat /tmp/perftool-testsuite_probe.RJh/perf_probe/logs/adding_blacklisted.log 
root@x1:~#

So is it just a matter of adding this output to the expected outputs? In test_adding_blacklisted.sh?

Did the patch below, now it passes, Ack?

Quoted 8 lines
Quoted 6 lines
root@x1:~# export PERFTEST_KEEP_LOGS=y
root@x1:~# perf test 88
 88: perftool-testsuite_probe                                        : Ok
root@x1:~#

Or am I missing the point? :-)

Yeah, that would fix the test case, the question is whether we should

I was lazy, but yeah, that thought crossed my mind, good thing you did the due diligence and found that patch! Looking at it now.

Quoted 3 lines

accept it as a possible output. Found some old patch that should have fixed the mentioned issue:
https://lore.kernel.org/all/[email protected]/

Quoted 6 lines
Quoted 2 lines

I updated Masami's e-mail to a more recent one, Masami-san, wdyt? Can I have your opinion on the patch pointed out by Veronika?

Thanks for update my email. Yeah, it looks good to me.

Unfortunately that cset is already there:

commit c66e1c68c13b872505f25ab641c44b77313ee7fe

Author: Georg Müller <[email protected]>
Date:   Wed Jun 28 10:45:51 2023 +0200

perf probe: Read DWARF files from the correct CU

After switching from dwarf_decl_file() to die_get_decl_file(), it is not possible to add probes for certain functions:

$ perf probe -x /usr/lib/systemd/systemd-logind match_unit_removed A function DIE doesn't have decl_line. Maybe broken DWARF? A function DIE doesn't have decl_line. Maybe broken DWARF? Probe point 'match_unit_removed' not found.
Error: Failed to add events.

⬢[acme@toolbox perf-tools-next]$ git tag --contains c66e1c68c13b872505f25ab641c44b77313ee7fe | grep ^v6.1 v6.10
v6.10-rc1
v6.10-rc2
v6.10-rc3
v6.10-rc4
v6.10-rc5
v6.10-rc6
v6.10-rc7
v6.11-rc1
v6.11-rc2
v6.11-rc3
v6.11-rc4
v6.11-rc5
v6.11-rc6
⬢[acme@toolbox perf-tools-next]$

But..

root@number:~# perf test -vvvvv testsuite_probe |& head
capget syscall failed (No such file or directory - 2) fall back on root check
 88: perftool-testsuite_probe:
--- start ---
test child forked, pid 511749
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
Line did not match any pattern: "A function DIE doesn't have decl_line. Maybe broken DWARF?"
-- [ FAIL ] -- perf_probe :: test_adding_blacklisted :: adding blacklisted function warn_thunk_thunk (output regexp parsing)
-- [ PASS ] -- perf_probe :: test_adding_blacklisted :: listing blacklisted probe (should NOT be listed)
## [ FAIL ] ## perf_probe :: test_adding_blacklisted SUMMARY :: 1 failures found
-- [ PASS ] -- perf_probe :: test_adding_kernel :: adding probe inode_permission :: 
root@number:~# perf -v
perf version 6.11.rc6.g69e90b02e611
root@number:~#
maintainer
#42 lore raw
Quoted 27 lines · Veronika Molnarova
Quoted 24 lines · Arnaldo Carvalho de Melo
Quoted 13 lines · Arnaldo Carvalho de Melo
Quoted 11 lines · Arnaldo Carvalho de Melo

So, I proposed that if we get that error message (A function DIE doesn't have decl_line. Maybe broken DWARF?) we should skip it, but Veronika did the right thing and asked if that wasn't really papering over some problem and pointed to a patch that addressed a problem that lead to that exact same message being produced, but that didn't help with our current case, with another blacklisted function, warn_thunk_thunk.

You mentioned that this is an asm function, so no DWARF, let alone correct DWARF, so the message is kinda ok, but we can't use that message as it could paper over real problems like the one fixed by the patch that Veronika pointed out.

Quoted 5 lines
Quoted 3 lines

How to figure out if a function has valid DWARF? Maybe for the moment we should have _another_ blacklist, one for functions in the blacklist that have no valid DWARF? A blacklist's blacklist?

We have the needed info to provide a more informative warning:

So, perhaps we should, in the test, when we get that "broken dwarf" output, look at the debuginfo file for the kernel to see if the function is one in a DW_UT_compile (CU) that has

Quoted 8 lines
Quoted 6 lines
     <da27a>   DW_AT_language    : 32769 (MIPS assembler)

And then just have a -vv output stating that that function is being skipped because it is DWARF from assembler?

Veronika, would you be willing to try to do that?

Okay, I can try to parse the debuginfo to check the DWARF.

I'll wait for v3 then.

Thanks,
Namhyung