Skip to contentSkip to conversation

perf test: Ignore security failures in all PMU test

PATCH Applied Ian Rogers 2 participants 5 messages Sep 25, 2024 – Sep 27, 2024 perf events

lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH 1/1 perf test: Ignore security failures in all PMU test

Refactor code to have some more error diagnosis on traps, etc. and to do less work on each line. Add an ignore situation for security failures.

---
 tools/perf/tests/shell/stat_all_pmu.sh | 52 ++++++++++++++++++++------
 1 file changed, 40 insertions(+), 12 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +40 -12
index d2a3506e0d19..42456d89c5da 100755
@@ -1,23 +1,51 @@
-#!/bin/sh
+#!/bin/bash
 # perf all PMU test
 # SPDX-License-Identifier: GPL-2.0
 
 set -e
+err=0
+result=""
+
+trap_cleanup() {
+  echo "Unexpected signal in ${FUNCNAME[1]}"
+  echo "$result"
+  exit 1
+}
+trap trap_cleanup EXIT TERM INT
 
 # Test all PMU events; however exclude parameterized ones (name contains '?')
-for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g'); do
+for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
+do
   echo "Testing $p"
   result=$(perf stat -e "$p" true 2>&1)
-  if ! echo "$result" | grep -q "$p" && ! echo "$result" | grep -q "<not supported>" ; then
-    # We failed to see the event and it is supported. Possibly the workload was
-    # too small so retry with something longer.
-    result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-    if ! echo "$result" | grep -q "$p" ; then
-      echo "Event '$p' not printed in:"
-      echo "$result"
-      exit 1
-    fi
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
+  fi
+  if echo "$result" | grep -q "<not supported>"
+  then
+    # Event not supported, so ignore.
+    continue
+  fi
+  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+  then
+    # Access is limited, so ignore.
+    continue
+  fi
+
+  # We failed to see the event and it is supported. Possibly the workload was
+  # too small so retry with something longer.
+  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
   fi
+  echo "Error: event '$p' not printed in:"
+  echo "$result"
+  err=1
 done
 
-exit 0
+trap - EXIT TERM INT
+exit $err
Signature · 1 line
2.46.0.792.g87dc391469-goog
maintainer
#1 lore raw
Quoted 43 lines · Ian Rogers

Refactor code to have some more error diagnosis on traps, etc. and to do less work on each line. Add an ignore situation for security failures.

---
 tools/perf/tests/shell/stat_all_pmu.sh | 52 ++++++++++++++++++++------
 1 file changed, 40 insertions(+), 12 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +12 -10
index d2a3506e0d19..42456d89c5da 100755
@@ -1,23 +1,51 @@
-#!/bin/sh
+#!/bin/bash
 # perf all PMU test
 # SPDX-License-Identifier: GPL-2.0
 
 set -e
+err=0
+result=""
+
+trap_cleanup() {
+  echo "Unexpected signal in ${FUNCNAME[1]}"
+  echo "$result"
+  exit 1
+}
+trap trap_cleanup EXIT TERM INT
 
 # Test all PMU events; however exclude parameterized ones (name contains '?')
-for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g'); do
+for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
+do
   echo "Testing $p"
   result=$(perf stat -e "$p" true 2>&1)
-  if ! echo "$result" | grep -q "$p" && ! echo "$result" | grep -q "<not supported>" ; then
-    # We failed to see the event and it is supported. Possibly the workload was
-    # too small so retry with something longer.
-    result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-    if ! echo "$result" | grep -q "$p" ; then
-      echo "Event '$p' not printed in:"
-      echo "$result"
-      exit 1

tools/perf/tests/shell/stat_all_pmu.sh line 29

Quoted 10 lines
tools/perf/tests/shell/stat_all_pmu.sh +9 -1
-    fi
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
+  fi
+  if echo "$result" | grep -q "<not supported>"
+  then
+    # Event not supported, so ignore.
+    continue

I'm curious about this case. It'll show up even if it's not supported right? Then the first condition would match and it doesn't reach here.

  $ perf stat -e LLC-loads true

Performance counter stats for 'true':

     <not supported>      LLC-loads                                                             
  
         0.001213558 seconds time elapsed
  
         0.001373000 seconds user
         0.000000000 seconds sys

tools/perf/tests/shell/stat_all_pmu.sh line 38

Quoted 9 lines
tools/perf/tests/shell/stat_all_pmu.sh +9 -0
+  fi
+  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+  then
+    # Access is limited, so ignore.
+    continue
+  fi
+
+  # We failed to see the event and it is supported. Possibly the workload was
+  # too small so retry with something longer.

Ok, maybe we want to check "<not counted>" events specifically.

Thanks,
Namhyung

Quoted 17 lines
tools/perf/tests/shell/stat_all_pmu.sh +10 -1
+  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
   fi
+  echo "Error: event '$p' not printed in:"
+  echo "$result"
+  err=1
 done
 
-exit 0
+trap - EXIT TERM INT
+exit $err
Signature · 1 line
2.46.0.792.g87dc391469-goog
author reviewer
#2 lore raw
Quoted 61 lines · Namhyung Kim
Quoted 43 lines · Ian Rogers

Refactor code to have some more error diagnosis on traps, etc. and to do less work on each line. Add an ignore situation for security failures.

---
 tools/perf/tests/shell/stat_all_pmu.sh | 52 ++++++++++++++++++++------
 1 file changed, 40 insertions(+), 12 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +12 -10
index d2a3506e0d19..42456d89c5da 100755
@@ -1,23 +1,51 @@
-#!/bin/sh
+#!/bin/bash
 # perf all PMU test
 # SPDX-License-Identifier: GPL-2.0
 
 set -e
+err=0
+result=""
+
+trap_cleanup() {
+  echo "Unexpected signal in ${FUNCNAME[1]}"
+  echo "$result"
+  exit 1
+}
+trap trap_cleanup EXIT TERM INT
 
 # Test all PMU events; however exclude parameterized ones (name contains '?')
-for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g'); do
+for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
+do
   echo "Testing $p"
   result=$(perf stat -e "$p" true 2>&1)
-  if ! echo "$result" | grep -q "$p" && ! echo "$result" | grep -q "<not supported>" ; then
-    # We failed to see the event and it is supported. Possibly the workload was
-    # too small so retry with something longer.
-    result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-    if ! echo "$result" | grep -q "$p" ; then
-      echo "Event '$p' not printed in:"
-      echo "$result"
-      exit 1

tools/perf/tests/shell/stat_all_pmu.sh line 29

Quoted 10 lines
tools/perf/tests/shell/stat_all_pmu.sh +9 -1
-    fi
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
+  fi
+  if echo "$result" | grep -q "<not supported>"
+  then
+    # Event not supported, so ignore.
+    continue

I'm curious about this case. It'll show up even if it's not supported right? Then the first condition would match and it doesn't reach here.

  $ perf stat -e LLC-loads true
Quoted 8 lines
   Performance counter stats for 'true':

     <not supported>      LLC-loads

         0.001213558 seconds time elapsed

         0.001373000 seconds user
         0.000000000 seconds sys

We carry this change at Google for an issue on older series kernels. I was resending the patch as a courtesy in the migration of development work to:
https://github.com/googleprodkernel/linux-perf
I'm okay with this being a Google only change.

Thanks,
Ian

Quoted 33 lines

tools/perf/tests/shell/stat_all_pmu.sh line 38

Quoted 9 lines
tools/perf/tests/shell/stat_all_pmu.sh +9 -0
+  fi
+  if echo "$result" | grep -q "Access to performance monitoring and observability operations is limited."
+  then
+    # Access is limited, so ignore.
+    continue
+  fi
+
+  # We failed to see the event and it is supported. Possibly the workload was
+  # too small so retry with something longer.

Ok, maybe we want to check "<not counted>" events specifically.

Thanks,
Namhyung

Quoted 17 lines
tools/perf/tests/shell/stat_all_pmu.sh +10 -2
+  result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
   fi
+  echo "Error: event '$p' not printed in:"
+  echo "$result"
+  err=1
 done
 
-exit 0
+trap - EXIT TERM INT
+exit $err
--

2.46.0.792.g87dc391469-goog

maintainer
#3 lore raw
Quoted 68 lines · Ian Rogers
Quoted 67 lines · Namhyung Kim
Quoted 43 lines · Ian Rogers

Refactor code to have some more error diagnosis on traps, etc. and to do less work on each line. Add an ignore situation for security failures.

---
 tools/perf/tests/shell/stat_all_pmu.sh | 52 ++++++++++++++++++++------
 1 file changed, 40 insertions(+), 12 deletions(-)
tools/perf/tests/shell/stat_all_pmu.sh +12 -10
index d2a3506e0d19..42456d89c5da 100755
@@ -1,23 +1,51 @@
-#!/bin/sh
+#!/bin/bash
 # perf all PMU test
 # SPDX-License-Identifier: GPL-2.0
 
 set -e
+err=0
+result=""
+
+trap_cleanup() {
+  echo "Unexpected signal in ${FUNCNAME[1]}"
+  echo "$result"
+  exit 1
+}
+trap trap_cleanup EXIT TERM INT
 
 # Test all PMU events; however exclude parameterized ones (name contains '?')
-for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g'); do
+for p in $(perf list --raw-dump pmu | sed 's/[[:graph:]]\+?[[:graph:]]\+[[:space:]]//g')
+do
   echo "Testing $p"
   result=$(perf stat -e "$p" true 2>&1)
-  if ! echo "$result" | grep -q "$p" && ! echo "$result" | grep -q "<not supported>" ; then
-    # We failed to see the event and it is supported. Possibly the workload was
-    # too small so retry with something longer.
-    result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
-    if ! echo "$result" | grep -q "$p" ; then
-      echo "Event '$p' not printed in:"
-      echo "$result"
-      exit 1

tools/perf/tests/shell/stat_all_pmu.sh line 29

Quoted 10 lines
tools/perf/tests/shell/stat_all_pmu.sh +9 -1
-    fi
+  if echo "$result" | grep -q "$p"
+  then
+    # Event seen in output.
+    continue
+  fi
+  if echo "$result" | grep -q "<not supported>"
+  then
+    # Event not supported, so ignore.
+    continue

I'm curious about this case. It'll show up even if it's not supported right? Then the first condition would match and it doesn't reach here.

  $ perf stat -e LLC-loads true

Performance counter stats for 'true':

<not supported> LLC-loads

0.001213558 seconds time elapsed

Quoted 8 lines
Quoted 2 lines
         0.001373000 seconds user
         0.000000000 seconds sys

We carry this change at Google for an issue on older series kernels. I was resending the patch as a courtesy in the migration of development work to:
https://github.com/googleprodkernel/linux-perf
I'm okay with this being a Google only change.

I'm not sure if it's a kernel issue. IIUC you want to run perf stat again if some events are not counted, right?

I understand your patch just converted the existing logic and added the check for security related failures. So I'll pick it up for now but we may want to revisit this later.

Thanks,
Namhyung

maintainer Applied
#4 lore raw
In reply to Ian Rogers
Quoted 4 lines · Ian Rogers

Refactor code to have some more error diagnosis on traps, etc. and to do less work on each line. Add an ignore situation for security failures.

Applied to perf-tools-next, thanks!

Best regards,
Namhyung