Skip to contentSkip to conversation

perf test: Fix inet_pton probe failure due to multiple probes

PATCH New Ian Rogers 4 participants 7 messages Apr 10 – Apr 14 perf events

lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH 1/1 perf test: Fix inet_pton probe failure due to multiple probes

When adding a probe for libc's inet_pton, perf probe may create multiple probe points (e.g., due to inlining or multiple symbol resolutions), resulting in multiple identical event names being output (e.g., probe_libc:inet_pton_1).

The script previously used a brittle pipeline (tail -n +2 | head -n -5) and an awk script to extract the event name. When multiple probes were added, awk would output the event name multiple times, which expanded to multiple words in bash. This broke the subsequent perf record and perf probe -d commands, causing the test to fail with:
Error: another command except --add is set.

Fix this by removing the brittle tail/head commands and appending | sort -u to the awk extraction. This ensures that only a single, unique event name is captured, regardless of how many probe points are created.

Assisted-by Gemini:gemini-3.1-pro-preview
---
 tools/perf/tests/shell/record+probe_libc_inet_pton.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +2 -2
index ab99bef556bf..b717063f70c2 100755
@@ -22,9 +22,9 @@ event_pattern='probe_libc:inet_pton(_[[:digit:]]+)?'
 
 add_libc_inet_pton_event() {
 
-	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | tail -n +2 | head -n -5 | \
+	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | \
 			awk -v ep="$event_pattern" -v l="$libc" '$0 ~ ep && $0 ~ \
-			("\\(on inet_pton in " l "\\)") {print $1}')
+			("\\(on inet_pton in " l "\\)") {print $1}' | sort -u)
 
 	if [ $? -ne 0 ] || [ -z "$event_name" ] ; then
 		printf "FAIL: could not add event\n"
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
#1 lore raw
Sashiko AI review found 1 potential issue(s): - [Medium] The use of `sort -u` fails to guarantee a single event name on systems without uprobe multiprobe support, leaving the test broken in those…

Sashiko AI review found 1 potential issue(s):
- [Medium] The use of sort -u fails to guarantee a single event name on systems without uprobe multiprobe support, leaving the test broken in those environments. --

commit 82ddf08abe58056272425532334245db796f2020
Author: Ian Rogers <[email protected]>

perf test: Fix inet_pton probe failure due to multiple probes

This commit attempts to fix a test failure caused by perf probe outputting multiple probe points. It replaces a tail/head pipeline with sort -u to try and extract a single, unique event name from the awk script output.

tools/perf/tests/shell/record+probe_libc_inet_pton.sh event_pattern='probe_libc:inet_pton(_[[:digit:]]+)?' · line 27

Quoted 13 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +2 -2
index ab99bef556bf0..b717063f70c27 100755
@@ -22,9 +22,9 @@ event_pattern='probe_libc:inet_pton(_[[:digit:]]+)?'
 
 add_libc_inet_pton_event() {
 
-	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | tail -n +2 | head -n -5 | \
+	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | \
 			awk -v ep="$event_pattern" -v l="$libc" '$0 ~ ep && $0 ~ \
-			("\\(on inet_pton in " l "\\)") {print $1}')
+			("\\(on inet_pton in " l "\\)") {print $1}' | sort -u)

Does sort -u guarantee a single event name is captured on kernels without uprobe multiprobe support?

On systems without multiprobe support, perf probe actively prevents name collisions by appending indices to subsequent probe points, outputting names like probe_libc:inet_pton and probe_libc:inet_pton_1.

Because these strings are textually distinct, sort -u will output all of them rather than deduplicating them.

This leaves the event_name variable containing multiple words separated by newlines. When expanded unquoted in the subsequent perf record and perf probe commands, bash word splitting causes the same syntax errors the patch intends to fix.

Could this use head -n 1 or awk '{print $1; exit}' instead to robustly guarantee a single name is captured across all environments?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author reviewer
#2 lore raw

PATCH v2 1/1 perf test: Fix inet_pton probe failure and unroll call graph

In reply to Ian Rogers

When adding a probe for libc's inet_pton, perf probe may create multiple probe points (e.g., due to inlining or multiple symbol resolutions), resulting in multiple identical event names being output (e.g., probe_libc:inet_pton_1).

The script previously used a brittle pipeline (tail -n +2 | head -n -5) and an awk script to extract the event name. When multiple probes were added, awk would output the event name multiple times, which expanded to multiple words in bash. This broke the subsequent perf record and perf probe -d commands, causing the test to fail with:
Error: another command except --add is set.

Fix this by removing the brittle tail/head commands and appending | head -n 1 to the awk extraction. This ensures that only a single, unique event name is captured, regardless of how many probe points are created.

Additionally, the test artificially limited the backtrace size via max-stack=4 and did not specify dwarf call graphs for non-s390x architectures. In newer libc versions where inet_pton is nested deeper or compiled without frame pointers, perf script failed to resolve the backtrace up to /bin/ping. Fix this by explicitly collecting dwarf call-graphs for all architectures and increasing max-stack to 8.

Assisted-by Gemini:gemini-3.1-pro-preview
---
 tools/perf/tests/shell/record+probe_libc_inet_pton.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +4 -4
index ab99bef556bf..eca629ee83f0 100755
@@ -22,9 +22,9 @@ event_pattern='probe_libc:inet_pton(_[[:digit:]]+)?'
 
 add_libc_inet_pton_event() {
 
-	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | tail -n +2 | head -n -5 | \
+	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | \
 			awk -v ep="$event_pattern" -v l="$libc" '$0 ~ ep && $0 ~ \
-			("\\(on inet_pton in " l "\\)") {print $1}')
+			("\\(on inet_pton in " l "\\)") {print $1}' | head -n 1)
 
 	if [ $? -ne 0 ] || [ -z "$event_name" ] ; then
 		printf "FAIL: could not add event\n"
@@ -40,12 +40,12 @@ trace_libc_inet_pton_backtrace() {
 	echo ".*inet_pton\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
 	case "$(uname -m)" in
 	s390x)
-		eventattr='call-graph=dwarf,max-stack=4'
+		eventattr='call-graph=dwarf,max-stack=8'
 		echo "((__GI_)?getaddrinfo|text_to_binary_address)\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
 		echo "(gaih_inet|main)\+0x[[:xdigit:]]+[[:space:]]\(inlined|.*/bin/ping.*\)$" >> $expected
 		;;
 	*)
-		eventattr='max-stack=4'
+		eventattr='call-graph=dwarf,max-stack=8'
 		echo ".*(\+0x[[:xdigit:]]+|\[unknown\])[[:space:]]\(.*/bin/ping.*\)$" >> $expected
 		;;
 	esac
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
#3 lore raw

Re: [PATCH v2] perf test: Fix inet_pton probe failure and unroll call graph

Sashiko AI review found 1 potential issue(s): - [Medium] Unconditionally requiring DWARF call-graphs causes the test to falsely fail on perf builds lacking DWARF support. --

Sashiko AI review found 1 potential issue(s):
- [Medium] Unconditionally requiring DWARF call-graphs causes the test to falsely fail on perf builds lacking DWARF support. --

commit dbec7073d294917f7866e8ea3a6b39004a7407ad
Author: Ian Rogers <[email protected]>

perf test: Fix inet_pton probe failure and unroll call graph

This commit fixes an issue where multiple identical event names were output by perf probe when creating multiple probe points, which broke subsequent commands. It also updates the backtrace collection to explicitly request dwarf call-graphs and increases max-stack to 8 for all architectures.

Quoted 4 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +0 -0
index ab99bef556bf0..eca629ee83f03 100755

[ ... ]

tools/perf/tests/shell/record+probe_libc_inet_pton.sh trace_libc_inet_pton_backtrace() { · line 48

Quoted 12 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +2 -2
@@ -40,12 +40,12 @@ trace_libc_inet_pton_backtrace() {
 	echo ".*inet_pton\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
 	case "$(uname -m)" in
 	s390x)
-		eventattr='call-graph=dwarf,max-stack=4'
+		eventattr='call-graph=dwarf,max-stack=8'
 		echo "((__GI_)?getaddrinfo|text_to_binary_address)\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
 		echo "(gaih_inet|main)\+0x[[:xdigit:]]+[[:space:]]\(inlined|.*/bin/ping.*\)$" >> $expected
 		;;
 	*)
-		eventattr='max-stack=4'
+		eventattr='call-graph=dwarf,max-stack=8'

Will this cause the test to falsely fail on perf builds lacking DWARF support?

If perf is compiled without libunwind or libdw, running perf record with call-graph=dwarf will exit with an error. Since the test script doesn't appear to verify DWARF support before running, the test will report a hard failure rather than gracefully skipping on minimal perf builds.

Quoted 3 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +0 -0
 		echo ".*(\+0x[[:xdigit:]]+|\[unknown\])[[:space:]]\(.*/bin/ping.*\)$" >> $expected
 		;;
 	esac
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author reviewer
#4 lore raw

Re: [PATCH v2] perf test: Fix inet_pton probe failure and unroll call graph

Quoted 35 lines · On Sat, Apr 11, 2026 at 12:54 PM <[email protected]> wrote:

Sashiko AI review found 1 potential issue(s):
- [Medium] Unconditionally requiring DWARF call-graphs causes the test to falsely fail on perf builds lacking DWARF support. --

commit dbec7073d294917f7866e8ea3a6b39004a7407ad
Author: Ian Rogers <[email protected]>

perf test: Fix inet_pton probe failure and unroll call graph

This commit fixes an issue where multiple identical event names were output by perf probe when creating multiple probe points, which broke subsequent commands. It also updates the backtrace collection to explicitly request dwarf call-graphs and increases max-stack to 8 for all architectures.

Quoted 4 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +0 -0
index ab99bef556bf0..eca629ee83f03 100755

[ ... ]

tools/perf/tests/shell/record+probe_libc_inet_pton.sh trace_libc_inet_pton_backtrace() { · line 48

Quoted 12 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +2 -2
@@ -40,12 +40,12 @@ trace_libc_inet_pton_backtrace() {
      echo ".*inet_pton\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
      case "$(uname -m)" in
      s390x)
-             eventattr='call-graph=dwarf,max-stack=4'
+             eventattr='call-graph=dwarf,max-stack=8'
              echo "((__GI_)?getaddrinfo|text_to_binary_address)\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
              echo "(gaih_inet|main)\+0x[[:xdigit:]]+[[:space:]]\(inlined|.*/bin/ping.*\)$" >> $expected
              ;;
      *)
-             eventattr='max-stack=4'
+             eventattr='call-graph=dwarf,max-stack=8'

Will this cause the test to falsely fail on perf builds lacking DWARF support?

Quoted 4 lines

If perf is compiled without libunwind or libdw, running perf record with call-graph=dwarf will exit with an error. Since the test script doesn't appear to verify DWARF support before running, the test will report a hard failure rather than gracefully skipping on minimal perf builds.

Since perf probe requires libdw and libunwind is deprecated this isn't a concern.

Thanks,
Ian

Quoted 6 lines

tools/perf/tests/shell/record+probe_libc_inet_pton.sh trace_libc_inet_pton_backtrace() { · line 51

Quoted 3 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +0 -0
              echo ".*(\+0x[[:xdigit:]]+|\[unknown\])[[:space:]]\(.*/bin/ping.*\)$" >> $expected
              ;;
      esac

--
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1

Tested-by on v2 1/1
#5 lore raw

Re: [PATCH v2] perf test: Fix inet_pton probe failure and unroll call graph

In reply to Ian Rogers
Quoted 51 lines · Ian Rogers

When adding a probe for libc's inet_pton, perf probe may create multiple probe points (e.g., due to inlining or multiple symbol resolutions), resulting in multiple identical event names being output (e.g., probe_libc:inet_pton_1).

The script previously used a brittle pipeline (tail -n +2 | head -n -5) and an awk script to extract the event name. When multiple probes were added, awk would output the event name multiple times, which expanded to multiple words in bash. This broke the subsequent perf record and perf probe -d commands, causing the test to fail with:
Error: another command except --add is set.

Fix this by removing the brittle tail/head commands and appending | head -n 1 to the awk extraction. This ensures that only a single, unique event name is captured, regardless of how many probe points are created.

Additionally, the test artificially limited the backtrace size via max-stack=4 and did not specify dwarf call graphs for non-s390x architectures. In newer libc versions where inet_pton is nested deeper or compiled without frame pointers, perf script failed to resolve the backtrace up to /bin/ping. Fix this by explicitly collecting dwarf call-graphs for all architectures and increasing max-stack to 8.

Assisted-by Gemini:gemini-3.1-pro-preview
---
 tools/perf/tests/shell/record+probe_libc_inet_pton.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +2 -3
index ab99bef556bf..eca629ee83f0 100755
@@ -22,9 +22,9 @@ event_pattern='probe_libc:inet_pton(_[[:digit:]]+)?'
 
 add_libc_inet_pton_event() {
 
-	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | tail -n +2 | head -n -5 | \
+	event_name=$(perf probe -f -x $libc -a inet_pton 2>&1 | \
 			awk -v ep="$event_pattern" -v l="$libc" '$0 ~ ep && $0 ~ \
-			("\\(on inet_pton in " l "\\)") {print $1}')
+			("\\(on inet_pton in " l "\\)") {print $1}' | head -n 1)
 
 	if [ $? -ne 0 ] || [ -z "$event_name" ] ; then
 		printf "FAIL: could not add event\n"
@@ -40,12 +40,12 @@ trace_libc_inet_pton_backtrace() {
 	echo ".*inet_pton\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
 	case "$(uname -m)" in
 	s390x)
-		eventattr='call-graph=dwarf,max-stack=4'

tools/perf/tests/shell/record+probe_libc_inet_pton.sh trace_libc_inet_pton_backtrace() { · line 51

Quoted 10 lines
tools/perf/tests/shell/record+probe_libc_inet_pton.sh +2 -1
+		eventattr='call-graph=dwarf,max-stack=8'
 		echo "((__GI_)?getaddrinfo|text_to_binary_address)\+0x[[:xdigit:]]+[[:space:]]\($libc|inlined\)$" >> $expected
 		echo "(gaih_inet|main)\+0x[[:xdigit:]]+[[:space:]]\(inlined|.*/bin/ping.*\)$" >> $expected
 		;;
 	*)
-		eventattr='max-stack=4'
+		eventattr='call-graph=dwarf,max-stack=8'
 		echo ".*(\+0x[[:xdigit:]]+|\[unknown\])[[:space:]]\(.*/bin/ping.*\)$" >> $expected
 		;;
 	esac

for s390:

Signature · 9 lines
Thomas Richter, Dept 3303, IBM s390 Linux Development, Boeblingen, Germany
--
IBM Deutschland Research & Development GmbH

Vorsitzender des Aufsichtsrats: Wolfgang Wendt

Geschäftsführung: David Faller

Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
maintainer Applied
#6 lore raw

Re: [PATCH v2] perf test: Fix inet_pton probe failure and unroll call graph

In reply to Ian Rogers
Quoted 5 lines · Ian Rogers

When adding a probe for libc's inet_pton, perf probe may create multiple probe points (e.g., due to inlining or multiple symbol resolutions), resulting in multiple identical event names being output (e.g., probe_libc:inet_pton_1).

Quoted 8 lines

The script previously used a brittle pipeline (tail -n +2 | head -n -5) and an awk script to extract the event name. When multiple probes were added, awk would output the event name multiple times, which expanded to multiple words in bash. This broke the subsequent perf record and perf probe -d commands, causing the test to fail with:
Error: another command except --add is set.

[...]

Applied to perf-tools-next, thanks!

Best regards,
Namhyung