Skip to contentSkip to conversation

perf test stat_all_pmu.sh: Correctly check 'perf stat' result

PATCH Applied [email protected] 3 participants 10 messages Nov 22, 2024 – Mar 14, 2025 perf events

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 perf test stat_all_pmu.sh: Correctly check 'perf stat' result

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Fixes 7e73ea40295620e7 ("perf test: Ignore security failures in all PMU test")
---
 tools/perf/tests/shell/stat_all_pmu.sh | 48 ++++++++++++++++++--------
 1 file changed, 34 insertions(+), 14 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +34 -14
index 8b148b300be1131e..9c466c0efa857fc2 100755
@@ -2,7 +2,6 @@
 # perf all PMU test (exclusive)
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
 err=0
 result=""
 
@@ -16,34 +15,55 @@ trap trap_cleanup EXIT TERM INT
 # Test all PMU events; however exclude parameterized ones (name contains '?')
 for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
 do
-  echo "Testing $p"
-  result=$(perf stat -e "$p" true 2>&1)
-  if echo "$result" | grep -q "$p"
+  echo -n "Testing $p -- "
+  output=$(perf stat -e "$p" true 2>&1)
+  stat_result=$?
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
-    continue
-  fi
-  if echo "$result" | grep -q "<not supported>"
-  then
-    # Event not supported, so ignore.
-    continue
+    if [ $stat_result -eq 0 ] && ! echo "$output" | grep -q "<not supported>"
+    then
+      # Event supported.
+      echo "supported"
+      continue
+    elif echo "$output" | grep -q "<not supported>"
+    then
+      # Event not supported, so ignore.
+      echo "not supported"
+      continue
+    elif echo "$output" | grep -q "No permission to enable"
+    then
+      # No permissions, so ignore.
+      echo "no permission to enable"
+      continue
+    elif echo "$output" | grep -q "Bad event name"
+    then
+      # Non-existent event.
+      echo "Error: Bad event name"
+      echo "$output"
+      err=1
+      continue
+    fi
   fi
-  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+
+  if echo "$output" | grep -q "Access to performance monitoring and observability operations is limited."
   then
     # Access is limited, so ignore.
+    echo "access limited"
     continue
   fi
 
   # We failed to see the event and it is supported. Possibly the workload was
   # too small so retry with something longer.
-  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-  if echo "$result" | grep -q "$p"
+  output=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
+    echo "supported"
     continue
   fi
   echo "Error: event '$p' not printed in:"
-  echo "$result"
+  echo "$output"
   err=1
 done
 
Signature · 1 line
2.43.0
#1 lore raw
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Thanks,
Veronika

Quoted 88 lines
Fixes 7e73ea40295620e7 ("perf test: Ignore security failures in all PMU test")
---
 tools/perf/tests/shell/stat_all_pmu.sh | 48 ++++++++++++++++++--------
 1 file changed, 34 insertions(+), 14 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +34 -14
index 8b148b300be1131e..9c466c0efa857fc2 100755
@@ -2,7 +2,6 @@
 # perf all PMU test (exclusive)
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
 err=0
 result=""
 
@@ -16,34 +15,55 @@ trap trap_cleanup EXIT TERM INT
 # Test all PMU events; however exclude parameterized ones (name contains '?')
 for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
 do
-  echo "Testing $p"
-  result=$(perf stat -e "$p" true 2>&1)
-  if echo "$result" | grep -q "$p"
+  echo -n "Testing $p -- "
+  output=$(perf stat -e "$p" true 2>&1)
+  stat_result=$?
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
-    continue
-  fi
-  if echo "$result" | grep -q "<not supported>"
-  then
-    # Event not supported, so ignore.
-    continue
+    if [ $stat_result -eq 0 ] && ! echo "$output" | grep -q "<not supported>"
+    then
+      # Event supported.
+      echo "supported"
+      continue
+    elif echo "$output" | grep -q "<not supported>"
+    then
+      # Event not supported, so ignore.
+      echo "not supported"
+      continue
+    elif echo "$output" | grep -q "No permission to enable"
+    then
+      # No permissions, so ignore.
+      echo "no permission to enable"
+      continue
+    elif echo "$output" | grep -q "Bad event name"
+    then
+      # Non-existent event.
+      echo "Error: Bad event name"
+      echo "$output"
+      err=1
+      continue
+    fi
   fi
-  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+
+  if echo "$output" | grep -q "Access to performance monitoring and observability operations is limited."
   then
     # Access is limited, so ignore.
+    echo "access limited"
     continue
   fi
 
   # We failed to see the event and it is supported. Possibly the workload was
   # too small so retry with something longer.
-  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-  if echo "$result" | grep -q "$p"
+  output=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
+    echo "supported"
     continue
   fi
   echo "Error: event '$p' not printed in:"
-  echo "$result"
+  echo "$output"
   err=1
 done
 
#2 lore raw
Quoted 28 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Quoted 8 lines

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Thanks,
Veronika

Quoted 88 lines
Quoted 88 lines
Fixes 7e73ea40295620e7 ("perf test: Ignore security failures in all PMU test")
---
 tools/perf/tests/shell/stat_all_pmu.sh | 48 ++++++++++++++++++--------
 1 file changed, 34 insertions(+), 14 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +34 -14
index 8b148b300be1131e..9c466c0efa857fc2 100755
@@ -2,7 +2,6 @@
 # perf all PMU test (exclusive)
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
 err=0
 result=""
 
@@ -16,34 +15,55 @@ trap trap_cleanup EXIT TERM INT
 # Test all PMU events; however exclude parameterized ones (name contains '?')
 for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
 do
-  echo "Testing $p"
-  result=$(perf stat -e "$p" true 2>&1)
-  if echo "$result" | grep -q "$p"
+  echo -n "Testing $p -- "
+  output=$(perf stat -e "$p" true 2>&1)
+  stat_result=$?
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
-    continue
-  fi
-  if echo "$result" | grep -q "<not supported>"
-  then
-    # Event not supported, so ignore.
-    continue
+    if [ $stat_result -eq 0 ] && ! echo "$output" | grep -q "<not supported>"
+    then
+      # Event supported.
+      echo "supported"
+      continue
+    elif echo "$output" | grep -q "<not supported>"
+    then
+      # Event not supported, so ignore.
+      echo "not supported"
+      continue
+    elif echo "$output" | grep -q "No permission to enable"
+    then
+      # No permissions, so ignore.
+      echo "no permission to enable"
+      continue
+    elif echo "$output" | grep -q "Bad event name"
+    then
+      # Non-existent event.
+      echo "Error: Bad event name"
+      echo "$output"
+      err=1
+      continue
+    fi
   fi
-  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+
+  if echo "$output" | grep -q "Access to performance monitoring and observability operations is limited."
   then
     # Access is limited, so ignore.
+    echo "access limited"
     continue
   fi
 
   # We failed to see the event and it is supported. Possibly the workload was
   # too small so retry with something longer.
-  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-  if echo "$result" | grep -q "$p"
+  output=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
+    echo "supported"
     continue
   fi
   echo "Error: event '$p' not printed in:"
-  echo "$result"
+  echo "$output"
   err=1
 done
 
maintainer
#3 lore raw

Hello,

Quoted 37 lines · Veronika Molnarova
Quoted 34 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Quoted 6 lines
Quoted 2 lines

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Ian, are you ok with this?

Thanks,
Namhyung

Quoted 89 lines
Quoted 88 lines
Quoted 88 lines
Fixes 7e73ea40295620e7 ("perf test: Ignore security failures in all PMU test")
---
 tools/perf/tests/shell/stat_all_pmu.sh | 48 ++++++++++++++++++--------
 1 file changed, 34 insertions(+), 14 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +34 -14
index 8b148b300be1131e..9c466c0efa857fc2 100755
@@ -2,7 +2,6 @@
 # perf all PMU test (exclusive)
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
 err=0
 result=""
 
@@ -16,34 +15,55 @@ trap trap_cleanup EXIT TERM INT
 # Test all PMU events; however exclude parameterized ones (name contains '?')
 for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
 do
-  echo "Testing $p"
-  result=$(perf stat -e "$p" true 2>&1)
-  if echo "$result" | grep -q "$p"
+  echo -n "Testing $p -- "
+  output=$(perf stat -e "$p" true 2>&1)
+  stat_result=$?
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
-    continue
-  fi
-  if echo "$result" | grep -q "<not supported>"
-  then
-    # Event not supported, so ignore.
-    continue
+    if [ $stat_result -eq 0 ] && ! echo "$output" | grep -q "<not supported>"
+    then
+      # Event supported.
+      echo "supported"
+      continue
+    elif echo "$output" | grep -q "<not supported>"
+    then
+      # Event not supported, so ignore.
+      echo "not supported"
+      continue
+    elif echo "$output" | grep -q "No permission to enable"
+    then
+      # No permissions, so ignore.
+      echo "no permission to enable"
+      continue
+    elif echo "$output" | grep -q "Bad event name"
+    then
+      # Non-existent event.
+      echo "Error: Bad event name"
+      echo "$output"
+      err=1
+      continue
+    fi
   fi
-  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+
+  if echo "$output" | grep -q "Access to performance monitoring and observability operations is limited."
   then
     # Access is limited, so ignore.
+    echo "access limited"
     continue
   fi
 
   # We failed to see the event and it is supported. Possibly the workload was
   # too small so retry with something longer.
-  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-  if echo "$result" | grep -q "$p"
+  output=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
+    echo "supported"
     continue
   fi
   echo "Error: event '$p' not printed in:"
-  echo "$result"
+  echo "$output"
   err=1
 done
 
Tested-by
#4 lore raw
Quoted 41 lines · Namhyung Kim

Hello,

Quoted 37 lines · Veronika Molnarova
Quoted 34 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Ian, are you ok with this?

Just ping again for this patch, can this fix be acked?

And I already tested this patch and it works well on the ppc64le system.

- Qiao

Quoted 93 lines

Thanks,
Namhyung

Quoted 89 lines
Quoted 88 lines
Quoted 88 lines
Fixes 7e73ea40295620e7 ("perf test: Ignore security failures in all PMU test")
---
 tools/perf/tests/shell/stat_all_pmu.sh | 48 ++++++++++++++++++--------
 1 file changed, 34 insertions(+), 14 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +34 -14
index 8b148b300be1131e..9c466c0efa857fc2 100755
@@ -2,7 +2,6 @@
 # perf all PMU test (exclusive)
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
 err=0
 result=""
 
@@ -16,34 +15,55 @@ trap trap_cleanup EXIT TERM INT
 # Test all PMU events; however exclude parameterized ones (name contains '?')
 for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
 do
-  echo "Testing $p"
-  result=$(perf stat -e "$p" true 2>&1)
-  if echo "$result" | grep -q "$p"
+  echo -n "Testing $p -- "
+  output=$(perf stat -e "$p" true 2>&1)
+  stat_result=$?
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
-    continue
-  fi
-  if echo "$result" | grep -q "<not supported>"
-  then
-    # Event not supported, so ignore.
-    continue
+    if [ $stat_result -eq 0 ] && ! echo "$output" | grep -q "<not supported>"
+    then
+      # Event supported.
+      echo "supported"
+      continue
+    elif echo "$output" | grep -q "<not supported>"
+    then
+      # Event not supported, so ignore.
+      echo "not supported"
+      continue
+    elif echo "$output" | grep -q "No permission to enable"
+    then
+      # No permissions, so ignore.
+      echo "no permission to enable"
+      continue
+    elif echo "$output" | grep -q "Bad event name"
+    then
+      # Non-existent event.
+      echo "Error: Bad event name"
+      echo "$output"
+      err=1
+      continue
+    fi
   fi
-  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+
+  if echo "$output" | grep -q "Access to performance monitoring and observability operations is limited."
   then
     # Access is limited, so ignore.
+    echo "access limited"
     continue
   fi
 
   # We failed to see the event and it is supported. Possibly the workload was
   # too small so retry with something longer.
-  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-  if echo "$result" | grep -q "$p"
+  output=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
+    echo "supported"
     continue
   fi
   echo "Error: event '$p' not printed in:"
-  echo "$result"
+  echo "$output"
   err=1
 done
 
#5 lore raw
Quoted 50 lines · Qiao Zhao
Quoted 48 lines · Namhyung Kim

Hello,

Quoted 43 lines · Veronika Molnarova
Quoted 28 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Quoted 8 lines

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Quoted 8 lines
Quoted 2 lines

Ian, are you ok with this?

Just ping again for this patch, can this fix be acked?

And I already tested this patch and it works well on the ppc64le system.

Revert the Tested-by, and when more tests on Intel EMR system, and show other "Bad event name" and " value too big for format (umask)" errors. This patch needs more tests and rewrites.

- Qiao

Quoted 96 lines

- Qiao

Quoted 93 lines

Thanks,
Namhyung

Quoted 89 lines
Quoted 88 lines
Quoted 88 lines
Fixes 7e73ea40295620e7 ("perf test: Ignore security failures in all PMU test")
---
 tools/perf/tests/shell/stat_all_pmu.sh | 48 ++++++++++++++++++--------
 1 file changed, 34 insertions(+), 14 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +34 -14
index 8b148b300be1131e..9c466c0efa857fc2 100755
@@ -2,7 +2,6 @@
 # perf all PMU test (exclusive)
 # SPDX-License-Identifier: GPL-2.0
 
-set -e
 err=0
 result=""
 
@@ -16,34 +15,55 @@ trap trap_cleanup EXIT TERM INT
 # Test all PMU events; however exclude parameterized ones (name contains '?')
 for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
 do
-  echo "Testing $p"
-  result=$(perf stat -e "$p" true 2>&1)
-  if echo "$result" | grep -q "$p"
+  echo -n "Testing $p -- "
+  output=$(perf stat -e "$p" true 2>&1)
+  stat_result=$?
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
-    continue
-  fi
-  if echo "$result" | grep -q "<not supported>"
-  then
-    # Event not supported, so ignore.
-    continue
+    if [ $stat_result -eq 0 ] && ! echo "$output" | grep -q "<not supported>"
+    then
+      # Event supported.
+      echo "supported"
+      continue
+    elif echo "$output" | grep -q "<not supported>"
+    then
+      # Event not supported, so ignore.
+      echo "not supported"
+      continue
+    elif echo "$output" | grep -q "No permission to enable"
+    then
+      # No permissions, so ignore.
+      echo "no permission to enable"
+      continue
+    elif echo "$output" | grep -q "Bad event name"
+    then
+      # Non-existent event.
+      echo "Error: Bad event name"
+      echo "$output"
+      err=1
+      continue
+    fi
   fi
-  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+
+  if echo "$output" | grep -q "Access to performance monitoring and observability operations is limited."
   then
     # Access is limited, so ignore.
+    echo "access limited"
     continue
   fi
 
   # We failed to see the event and it is supported. Possibly the workload was
   # too small so retry with something longer.
-  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-  if echo "$result" | grep -q "$p"
+  output=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$output" | grep -q "$p"
   then
     # Event seen in output.
+    echo "supported"
     continue
   fi
   echo "Error: event '$p' not printed in:"
-  echo "$result"
+  echo "$output"
   err=1
 done
 
maintainer
#6 lore raw

Hello,

Quoted 56 lines · Qiao Zhao
Quoted 55 lines · Qiao Zhao
Quoted 41 lines · Namhyung Kim

Hello,

Quoted 37 lines · Veronika Molnarova
Quoted 34 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Ian, are you ok with this?

Just ping again for this patch, can this fix be acked?

Quoted 7 lines
Quoted 3 lines

And I already tested this patch and it works well on the ppc64le system.

Revert the Tested-by, and when more tests on Intel EMR system, and show other "Bad event name" and " value too big for format (umask)" errors. This patch needs more tests and rewrites.

Thanks for the test, though!
Namhyung

#7 lore raw

Some more tests with Veronika's patch:

Pick one failed event and use '-vv' to show more debug info like this:

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
# ./perf stat -vv -e 'unc_iio_req_from_pcie_pass_cmpl.wr' true
Using CPUID GenuineIntel-6-CF-2
Attempt to add: uncore_iio_0/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_0/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_2/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_2/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_6/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_6/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_8/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_8/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_1/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_1/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_3/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_3/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_7/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_7/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
                     \___ Bad event name

Unable to find event on a PMU of 'unc_iio_req_from_pcie_pass_cmpl.wr'

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255
Run 'perf list' for a list of valid events

Usage: perf stat [<options>] [<command>]

-e, --event <event> event selector. use 'perf list' to list
available events

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

And the event umask shows "umask=0x70ff010", this value is higher than maximum value "255".

After talking with Michael, this "Bad event name" issue should be fixed by Ian Rogers's patch
https://lore.kernel.org/lkml/[email protected]/#r
perf vendor events: Update Sapphirerapids events/metrics

Will try to test Veronika's patch again after Roger's patch merges.

- Qiao

- Qiao

Quoted 72 lines · Namhyung Kim

Hello,

Quoted 56 lines · Qiao Zhao
Quoted 55 lines · Qiao Zhao
Quoted 41 lines · Namhyung Kim

Hello,

Quoted 37 lines · Veronika Molnarova
Quoted 34 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Ian, are you ok with this?

Just ping again for this patch, can this fix be acked?

Quoted 7 lines
Quoted 3 lines

And I already tested this patch and it works well on the ppc64le system.

Revert the Tested-by, and when more tests on Intel EMR system, and show other "Bad event name" and " value too big for format (umask)" errors. This patch needs more tests and rewrites.

Thanks for the test, though!
Namhyung

Tested-by
#8 lore raw

Hi Namhyung,

Sorry for causing some trouble and confusion.
Veronika's patch fixed the "check 'perf stat' result" issue, and the test passed.
About the "Bad event name" issue, it is another EMR/SPR uncore issue, not caused by Veronike's patch.
So if no other is missing, can we merge this issue?

- Qiao

Quoted 150 lines · Qiao Zhao

Some more tests with Veronika's patch:

Pick one failed event and use '-vv' to show more debug info like this:

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
# ./perf stat -vv -e 'unc_iio_req_from_pcie_pass_cmpl.wr' true
Using CPUID GenuineIntel-6-CF-2
Attempt to add: uncore_iio_0/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_0/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_2/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_2/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_6/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_6/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_8/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_8/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_1/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_1/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_3/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_3/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
Attempt to add: uncore_iio_7/unc_iio_req_from_pcie_pass_cmpl.wr/
..after resolving event:
uncore_iio_7/event=0x90,ch_mask=0xff,fc_mask=0x7,umask=0x70ff010/
event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
                     \___ Bad event name

Unable to find event on a PMU of 'unc_iio_req_from_pcie_pass_cmpl.wr'

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255

event syntax error: 'unc_iio_req_from_pcie_pass_cmpl.wr'
\___ value too big for format (umask), maximum is 255
Run 'perf list' for a list of valid events

Usage: perf stat [<options>] [<command>]

-e, --event <event> event selector. use 'perf list' to list
available events

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

And the event umask shows "umask=0x70ff010", this value is higher than maximum value "255".

After talking with Michael, this "Bad event name" issue should be fixed by Ian Rogers's patch
https://lore.kernel.org/lkml/[email protected]/#r
perf vendor events: Update Sapphirerapids events/metrics

Will try to test Veronika's patch again after Roger's patch merges.

- Qiao

- Qiao

Quoted 72 lines · Namhyung Kim

Hello,

Quoted 56 lines · Qiao Zhao
Quoted 55 lines · Qiao Zhao
Quoted 41 lines · Namhyung Kim

Hello,

Quoted 37 lines · Veronika Molnarova
Quoted 34 lines · Veronika Molnarova
Quoted 20 lines · On 11/23/24 00:12, [email protected] wrote:

From: Veronika Molnarova <[email protected]>

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

    $ perf test -vv "perf all pmu"
    Testing i915/actual-frequency/
    Unexpected signal in main
    Error:
    Access to performance monitoring and observability operations is limited.

Secondly, the if-else branches are not exclusive as the checking if the event is present in the output log covers also the "<not supported>" events, which should be accepted, and also the "Bad name events", which should be rejected.

Quoted 4 lines

Remove the "set -e" option from the test case, correctly parse the "perf stat" output log and check its return value. Add the missing outputs for the 'perf stat' result and also add logs messages to report the branch that parsed the event for more info.

Hi Ian,

is there anything that I am missing? If so would be great to know the idea of your patch. This issue is getting quite old so would be great to get a proper fix together.

Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Thanks,
Veronika

Hello,

just wanted to ping the patch. Would be great to sort it out and close the issue.

Ian, are you ok with this?

Just ping again for this patch, can this fix be acked?

Quoted 7 lines
Quoted 3 lines

And I already tested this patch and it works well on the ppc64le system.

Revert the Tested-by, and when more tests on Intel EMR system, and show other "Bad event name" and " value too big for format (umask)" errors. This patch needs more tests and rewrites.

Thanks for the test, though!
Namhyung

maintainer Applied
#9 lore raw
In reply to [email protected]
Quoted 8 lines · On Sat, 23 Nov 2024 00:12:33 +0100, [email protected] wrote:

Test case "stat_all_pmu.sh" is not correctly checking 'perf stat' output due to a poor design. Firstly, having the 'set -e' option with a trap catching the sigexit causes the shell to exit immediately if 'perf stat' ends with any non-zero value, which is then caught by the trap reporting an unexpected signal. This causes events that should be parsed by the if-else statement to be caught by the trap handler and are reported as errors:

[...]

Applied to perf-tools-next, thanks!

Best regards,
Namhyung