Skip to contentSkip to conversation

perf test: Fix non-bash issue with stat bpf counters

PATCH Applied Ian Rogers 3 participants 13 messages Jun 17, 2021 – Jun 21, 2021

Subsystems (2)
lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH 1/4 perf test: Fix non-bash issue with stat bpf counters

$(( .. )) is a bash feature but the test's interpreter is !/bin/sh, switch the code to use expr.

---
 tools/perf/tests/shell/stat_bpf_counters.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/tests/shell/stat_bpf_counters.sh +2 -2
index 22eb31e48ca7..2f9948b3d943 100755
@@ -11,9 +11,9 @@ compare_number()
        second_num=$2
 
        # upper bound is first_num * 110%
-       upper=$(( $first_num + $first_num / 10 ))
+       upper=$(expr $first_num + $first_num / 10 )
        # lower bound is first_num * 90%
-       lower=$(( $first_num - $first_num / 10 ))
+       lower=$(expr $first_num - $first_num / 10 )
 
        if [ $second_num -gt $upper ] || [ $second_num -lt $lower ]; then
                echo "The difference between $first_num and $second_num are greater than 10%."
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
author reviewer
#1 lore raw

PATCH 2/4 perf test: Pass the verbose option to shell tests

Having a verbose option will allow shell tests to provide extra failure details when the fail or skip.

---
 tools/perf/tests/builtin-test.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/tests/builtin-test.c +4 -1
index cbbfe48ab802..a8160b1684de 100644
@@ -577,11 +577,14 @@ struct shell_test {
 static int shell_test__run(struct test *test, int subdir __maybe_unused)
 {
 	int err;
-	char script[PATH_MAX];
+	char script[PATH_MAX + 3];
 	struct shell_test *st = test->priv;
 
 	path__join(script, sizeof(script), st->dir, st->file);
 
+	if (verbose)
+		strncat(script, " -v", sizeof(script));
+
 	err = system(script);
 	if (!err)
 		return TEST_OK;
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
maintainer Applied
#2 lore raw

Re: [PATCH 2/4] perf test: Pass the verbose option to shell tests

Em Thu, Jun 17, 2021 at 11:42:14AM -0700, Ian Rogers escreveu:

Quoted 2 lines

Having a verbose option will allow shell tests to provide extra failure details when the fail or skip.

Thanks, applied to perf/core.

- Arnaldo

Quoted 28 lines
---
 tools/perf/tests/builtin-test.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/tests/builtin-test.c +4 -1
index cbbfe48ab802..a8160b1684de 100644
@@ -577,11 +577,14 @@ struct shell_test {
 static int shell_test__run(struct test *test, int subdir __maybe_unused)
 {
 	int err;
-	char script[PATH_MAX];
+	char script[PATH_MAX + 3];
 	struct shell_test *st = test->priv;
 
 	path__join(script, sizeof(script), st->dir, st->file);
 
+	if (verbose)
+		strncat(script, " -v", sizeof(script));
+
 	err = system(script);
 	if (!err)
 		return TEST_OK;
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
Signature · 1 line
- Arnaldo
#3 lore raw

Re: [PATCH 2/4] perf test: Pass the verbose option to shell tests

Em Thu, Jun 17, 2021 at 04:19:06PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 26 lines

Em Thu, Jun 17, 2021 at 11:42:14AM -0700, Ian Rogers escreveu:

Quoted 2 lines

Having a verbose option will allow shell tests to provide extra failure details when the fail or skip.

Thanks, applied to perf/core.

- Arnaldo

Quoted 17 lines
---
 tools/perf/tests/builtin-test.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/tests/builtin-test.c +1 -1
index cbbfe48ab802..a8160b1684de 100644
@@ -577,11 +577,14 @@ struct shell_test {
 static int shell_test__run(struct test *test, int subdir __maybe_unused)
 {
 	int err;
-	char script[PATH_MAX];
+	char script[PATH_MAX + 3];
 	struct shell_test *st = test->priv;
 
Quoted 1 line

tools/perf/tests/builtin-test.c struct shell_test { · line 583

Quoted 1 line
tools/perf/tests/builtin-test.c +0 -0
 	path__join(script, sizeof(script), st->dir, st->file);

probably you need to add a '- 3' after the sizeof above, right?

Quoted 4 lines

tools/perf/tests/builtin-test.c struct shell_test { · line 587

Quoted 4 lines
tools/perf/tests/builtin-test.c +3 -0
+	if (verbose)
+		strncat(script, " -v", sizeof(script));
+

Seemed simple enough, but gcc knows better, I'm removing this one:

    tests/builtin-test.c:586:26: error: the value of the size argument in 'strncat' is too large, might lead to a buffer overflow [-Werror,-Wstrncat-size]
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
    tests/builtin-test.c:586:26: note: change the argument to be the free space in the destination buffer minus the terminating null byte
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
                                           sizeof(script) - strlen(script) - 1
    1 error generated.
    make[3]: *** [/git/perf-5.13.0-rc4/tools/build/Makefile.build:139: tests] Error 2
  77    31.98 ubuntu:21.04                  : FAIL gcc version 10.3.0 (Ubuntu 10.3.0-1ubuntu1)
    tests/builtin-test.c:586:26: error: the value of the size argument in 'strncat' is too large, might lead to a buffer overflow [-Werror,-Wstrncat-size]
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
    tests/builtin-test.c:586:26: note: change the argument to be the free space in the destination buffer minus the terminating null byte
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
                                           sizeof(script) - strlen(script) - 1
    1 error generated.
    make[3]: *** [/git/perf-5.13.0-rc4/tools/build/Makefile.build:139: tests] Error 2
Quoted 10 lines
Quoted 6 lines
tools/perf/tests/builtin-test.c +0 -0
 	err = system(script);
 	if (!err)
 		return TEST_OK;
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
Signature · 1 line
- Arnaldo
Signature · 1 line
- Arnaldo
author reviewer
#4 lore raw

Re: [PATCH 2/4] perf test: Pass the verbose option to shell tests

Quoted 28 lines · Arnaldo Carvalho de Melo

Em Thu, Jun 17, 2021 at 04:19:06PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 26 lines

Em Thu, Jun 17, 2021 at 11:42:14AM -0700, Ian Rogers escreveu:

Quoted 2 lines

Having a verbose option will allow shell tests to provide extra failure details when the fail or skip.

Thanks, applied to perf/core.

- Arnaldo

Quoted 17 lines
---
 tools/perf/tests/builtin-test.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/tests/builtin-test.c +1 -1
index cbbfe48ab802..a8160b1684de 100644
@@ -577,11 +577,14 @@ struct shell_test {
 static int shell_test__run(struct test *test, int subdir __maybe_unused)
 {
    int err;
-   char script[PATH_MAX];
+   char script[PATH_MAX + 3];
    struct shell_test *st = test->priv;
 
Quoted 3 lines
Quoted 1 line

tools/perf/tests/builtin-test.c struct shell_test { · line 583

Quoted 1 line
tools/perf/tests/builtin-test.c +0 -0
    path__join(script, sizeof(script), st->dir, st->file);

probably you need to add a '- 3' after the sizeof above, right?

Either way is fine, but -3 is ok with me.

Quoted 18 lines
Quoted 4 lines

tools/perf/tests/builtin-test.c struct shell_test { · line 587

Quoted 4 lines
tools/perf/tests/builtin-test.c +3 -0
+   if (verbose)
+           strncat(script, " -v", sizeof(script));
+

Seemed simple enough, but gcc knows better, I'm removing this one:

    tests/builtin-test.c:586:26: error: the value of the size argument in 'strncat' is too large, might lead to a buffer overflow [-Werror,-Wstrncat-size]
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
    tests/builtin-test.c:586:26: note: change the argument to be the free space in the destination buffer minus the terminating null byte
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
                                           sizeof(script) - strlen(script) - 1
    1 error generated.
    make[3]: *** [/git/perf-5.13.0-rc4/tools/build/Makefile.build:139: tests] Error 2
  77    31.98 ubuntu:21.04                  : FAIL gcc version 10.3.0 (Ubuntu 10.3.0-1ubuntu1)
    tests/builtin-test.c:586:26: error: the value of the size argument in 'strncat' is too large, might lead to a buffer overflow [-Werror,-Wstrncat-size]
Quoted 8 lines
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
    tests/builtin-test.c:586:26: note: change the argument to be the free space in the destination buffer minus the terminating null byte
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
                                           sizeof(script) - strlen(script) - 1
    1 error generated.
    make[3]: *** [/git/perf-5.13.0-rc4/tools/build/Makefile.build:139: tests] Error 2

Thanks gcc :-) Do you want me to resend the patch?

Ian

Quoted 14 lines
Quoted 10 lines
Quoted 6 lines
tools/perf/tests/builtin-test.c +0 -1
    err = system(script);
    if (!err)
            return TEST_OK;
--

2.32.0.288.g62a8d224e6-goog

--

- Arnaldo

--

- Arnaldo

#5 lore raw

Re: [PATCH 2/4] perf test: Pass the verbose option to shell tests

Em Fri, Jun 18, 2021 at 09:45:05PM -0700, Ian Rogers escreveu:

Quoted 63 lines
Quoted 28 lines · Arnaldo Carvalho de Melo

Em Thu, Jun 17, 2021 at 04:19:06PM -0300, Arnaldo Carvalho de Melo escreveu:

Quoted 26 lines

Em Thu, Jun 17, 2021 at 11:42:14AM -0700, Ian Rogers escreveu:

Quoted 2 lines

Having a verbose option will allow shell tests to provide extra failure details when the fail or skip.

Thanks, applied to perf/core.

- Arnaldo

Quoted 17 lines
---
 tools/perf/tests/builtin-test.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/tests/builtin-test.c +1 -1
index cbbfe48ab802..a8160b1684de 100644
@@ -577,11 +577,14 @@ struct shell_test {
 static int shell_test__run(struct test *test, int subdir __maybe_unused)
 {
    int err;
-   char script[PATH_MAX];
+   char script[PATH_MAX + 3];
    struct shell_test *st = test->priv;
 
Quoted 3 lines
Quoted 1 line

tools/perf/tests/builtin-test.c struct shell_test { · line 583

Quoted 1 line
tools/perf/tests/builtin-test.c +0 -0
    path__join(script, sizeof(script), st->dir, st->file);

probably you need to add a '- 3' after the sizeof above, right?

Either way is fine, but -3 is ok with me.

Quoted 18 lines
Quoted 4 lines

tools/perf/tests/builtin-test.c struct shell_test { · line 587

Quoted 4 lines
tools/perf/tests/builtin-test.c +3 -0
+   if (verbose)
+           strncat(script, " -v", sizeof(script));
+

Seemed simple enough, but gcc knows better, I'm removing this one:

    tests/builtin-test.c:586:26: error: the value of the size argument in 'strncat' is too large, might lead to a buffer overflow [-Werror,-Wstrncat-size]
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
    tests/builtin-test.c:586:26: note: change the argument to be the free space in the destination buffer minus the terminating null byte
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
                                           sizeof(script) - strlen(script) - 1
    1 error generated.
    make[3]: *** [/git/perf-5.13.0-rc4/tools/build/Makefile.build:139: tests] Error 2
  77    31.98 ubuntu:21.04                  : FAIL gcc version 10.3.0 (Ubuntu 10.3.0-1ubuntu1)
    tests/builtin-test.c:586:26: error: the value of the size argument in 'strncat' is too large, might lead to a buffer overflow [-Werror,-Wstrncat-size]
Quoted 8 lines
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
    tests/builtin-test.c:586:26: note: change the argument to be the free space in the destination buffer minus the terminating null byte
                    strncat(script, " -v", sizeof(script));
                                           ^~~~~~~~~~~~~~
                                           sizeof(script) - strlen(script) - 1
    1 error generated.
    make[3]: *** [/git/perf-5.13.0-rc4/tools/build/Makefile.build:139: tests] Error 2
Quoted 1 line

Thanks gcc :-) Do you want me to resend the patch?

In such cases please do,

- Arnaldo

author reviewer
#6 lore raw

PATCH 3/4 perf test: Add verbose skip output for bpf counters

In reply to Ian Rogers

Provide additional context for when the stat bpf counters test skips.

---
 tools/perf/tests/shell/stat_bpf_counters.sh | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tools/perf/tests/shell/stat_bpf_counters.sh +7 -1
index 2f9948b3d943..81d61b6e1208 100755
@@ -22,7 +22,13 @@ compare_number()
 }
 
 # skip if --bpf-counters is not supported
-perf stat --bpf-counters true > /dev/null 2>&1 || exit 2
+if ! perf stat --bpf-counters true > /dev/null 2>&1; then
+	if [ "$1" == "-v" ]; then
+		echo "Skipping: --bpf-counters not supported"
+		perf --no-pager stat --bpf-counters true || true
+	fi
+	exit 2
+fi
 
 base_cycles=$(perf stat --no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
 bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
maintainer Applied
#7 lore raw

Re: [PATCH 3/4] perf test: Add verbose skip output for bpf counters

Em Thu, Jun 17, 2021 at 11:42:15AM -0700, Ian Rogers escreveu:

Quoted 1 line

Provide additional context for when the stat bpf counters test skips.

Thanks, applied.

- Arnaldo

Quoted 27 lines
---
 tools/perf/tests/shell/stat_bpf_counters.sh | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
tools/perf/tests/shell/stat_bpf_counters.sh +7 -1
index 2f9948b3d943..81d61b6e1208 100755
@@ -22,7 +22,13 @@ compare_number()
 }
 
 # skip if --bpf-counters is not supported
-perf stat --bpf-counters true > /dev/null 2>&1 || exit 2
+if ! perf stat --bpf-counters true > /dev/null 2>&1; then
+	if [ "$1" == "-v" ]; then
+		echo "Skipping: --bpf-counters not supported"
+		perf --no-pager stat --bpf-counters true || true
+	fi
+	exit 2
+fi
 
 base_cycles=$(perf stat --no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
 bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
Signature · 1 line
- Arnaldo
author reviewer
#8 lore raw

PATCH 4/4 perf test: Make stat bpf counters test more robust

In reply to Ian Rogers

If the test is run on a hypervisor then the cycles event may not be counted, skip the test in this situation. Fail the test if cycles are not counted in the subsequent bpf counter run.

---
 tools/perf/tests/shell/stat_bpf_counters.sh | 8 ++++++++
 1 file changed, 8 insertions(+)
tools/perf/tests/shell/stat_bpf_counters.sh +8 -0
index 81d61b6e1208..2aed20dc2262 100755
@@ -31,7 +31,15 @@ if ! perf stat --bpf-counters true > /dev/null 2>&1; then
 fi
 
 base_cycles=$(perf stat --no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
+if [ "$base_cycles" == "<not" ]; then
+	echo "Skipping: cycles event not counted"
+	exit 2
+fi
 bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
+if [ "$bpf_cycles" == "<not" ]; then
+	echo "Failed: cycles not counted with --bpf-counters"
+	exit 1
+fi
 
 compare_number $base_cycles $bpf_cycles
 exit 0
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
maintainer Applied
#9 lore raw

Re: [PATCH 4/4] perf test: Make stat bpf counters test more robust

Em Thu, Jun 17, 2021 at 11:42:16AM -0700, Ian Rogers escreveu:

Quoted 3 lines

If the test is run on a hypervisor then the cycles event may not be counted, skip the test in this situation. Fail the test if cycles are not counted in the subsequent bpf counter run.

Thanks, applied.

- Arnaldo

Quoted 28 lines
---
 tools/perf/tests/shell/stat_bpf_counters.sh | 8 ++++++++
 1 file changed, 8 insertions(+)
tools/perf/tests/shell/stat_bpf_counters.sh +8 -0
index 81d61b6e1208..2aed20dc2262 100755
@@ -31,7 +31,15 @@ if ! perf stat --bpf-counters true > /dev/null 2>&1; then
 fi
 
 base_cycles=$(perf stat --no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
+if [ "$base_cycles" == "<not" ]; then
+	echo "Skipping: cycles event not counted"
+	exit 2
+fi
 bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- perf bench sched messaging -g 1 -l 100 -t 2>&1 | awk '/cycles/ {print $1}')
+if [ "$bpf_cycles" == "<not" ]; then
+	echo "Failed: cycles not counted with --bpf-counters"
+	exit 1
+fi
 
 compare_number $base_cycles $bpf_cycles
 exit 0
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
Signature · 1 line
- Arnaldo
maintainer Applied
#10 lore raw
In reply to Ian Rogers

Em Thu, Jun 17, 2021 at 11:42:13AM -0700, Ian Rogers escreveu:

Quoted 2 lines

$(( .. )) is a bash feature but the test's interpreter is !/bin/sh, switch the code to use expr.

Thanks, applied to perf/urgent.

- Arnaldo

Quoted 24 lines
---
 tools/perf/tests/shell/stat_bpf_counters.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/tests/shell/stat_bpf_counters.sh +2 -2
index 22eb31e48ca7..2f9948b3d943 100755
@@ -11,9 +11,9 @@ compare_number()
        second_num=$2
 
        # upper bound is first_num * 110%
-       upper=$(( $first_num + $first_num / 10 ))
+       upper=$(expr $first_num + $first_num / 10 )
        # lower bound is first_num * 90%
-       lower=$(( $first_num - $first_num / 10 ))
+       lower=$(expr $first_num - $first_num / 10 )
 
        if [ $second_num -gt $upper ] || [ $second_num -lt $lower ]; then
                echo "The difference between $first_num and $second_num are greater than 10%."
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
Signature · 1 line
- Arnaldo
#11 lore raw
In reply to Ian Rogers

Hi Ian,

Quoted 3 lines · On Thu, 17 Jun 2021 11:42:13 -0700 Ian Rogers <[email protected]> wrote:

$(( .. )) is a bash feature but the test's interpreter is !/bin/sh, switch the code to use expr.

The $(( .. )) syntax is specified in POSIX (see
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04),
so unless this caused an actual problem, this change is unnecessary.

Signature · 2 lines
Cheers,
Stephen Rothwell
author reviewer
#12 lore raw
Quoted 11 lines · Stephen Rothwell

Hi Ian,

Quoted 3 lines · On Thu, 17 Jun 2021 11:42:13 -0700 Ian Rogers <[email protected]> wrote:

$(( .. )) is a bash feature but the test's interpreter is !/bin/sh, switch the code to use expr.

The $(( .. )) syntax is specified in POSIX (see
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04),
so unless this caused an actual problem, this change is unnecessary.

Agreed. The issue I was seeing was:

./tests/shell/stat_bpf_counters.sh: line 14: <not + <not / 10 : syntax
error: operand expected (error token is "<not + <not / 10 ")

but that syntax error is caused by running the test within a hypervisor. I'll resend the patch set with this one dropped.

Thanks,
Ian

Quoted 3 lines

--
Cheers,
Stephen Rothwell