Skip to contentSkip to conversation

perf test: Workload test of metric and metricgroups

PATCH Applied Ian Rogers 4 participants 10 messages Sep 17, 2021 – Jan 12, 2022 perf events

lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH v3 1/2 perf test: Workload test of metric and metricgroups

Test every metric and metricgroup with 'true' as a workload. For metrics, check that we see the metric printed or get unsupported. If the 'true' workload executes too quickly retry with 'perf bench internals synthesize'.

v3. Fix test condition (thanks to Paul A. Clarke <[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

---
 .../perf/tests/shell/stat_all_metricgroups.sh | 12 ++++++++++
 tools/perf/tests/shell/stat_all_metrics.sh    | 22 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100755 tools/perf/tests/shell/stat_all_metricgroups.sh
 create mode 100755 tools/perf/tests/shell/stat_all_metrics.sh
tools/perf/tests/shell/stat_all_metricgroups.sh new file +12 -0
new file mode 100755
index 000000000000..de24d374ce24
@@ -0,0 +1,12 @@
+#!/bin/sh
+# perf all metricgroups test
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+for m in $(perf list --raw-dump metricgroups); do
+  echo "Testing $m"
+  perf stat -M "$m" true
+done
+
+exit 0
tools/perf/tests/shell/stat_all_metrics.sh new file +22 -0
new file mode 100755
index 000000000000..7f4ba3cad632
@@ -0,0 +1,22 @@
+#!/bin/sh
+# perf all metrics test
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+for m in $(perf list --raw-dump metrics); do
+  echo "Testing $m"
+  result=$(perf stat -M "$m" true 2>&1)
+  if [[ ! "$result" =~ "$m" ]] && [[ ! "$result" =~ "<not supported>" ]]; then
+    # We failed to see the metric and the events are support. Possibly the
+    # workload was too small so retry with something longer.
+    result=$(perf stat -M "$m" perf bench internals synthesize 2>&1)
+    if [[ ! "$result" =~ "$m" ]]; then
+      echo "Metric '$m' not printed in:"
+      echo "$result"
+      exit 1
+    fi
+  fi
+done
+
+exit 0
Signature · 1 line
2.33.0.464.g1972c5931b-goog
author reviewer
#1 lore raw

PATCH v3 2/2 perf test: Workload test of all PMUs

Iterate over the list of PMUs and run the 'true' workload on them. If the event isn't printed then run the large 'perf bench internals synthesize' workload and check the event is counted.

On a Skylake this test takes 1m15s mainly running the 'true' workload.

---
 tools/perf/tests/shell/stat_all_pmu.sh | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100755 tools/perf/tests/shell/stat_all_pmu.sh
tools/perf/tests/shell/stat_all_pmu.sh new file +22 -0
new file mode 100755
index 000000000000..2de7fd0394fd
@@ -0,0 +1,22 @@
+#!/bin/sh
+# perf all PMU test
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+for p in $(perf list --raw-dump pmu); do
+  echo "Testing $p"
+  result=$(perf stat -e "$p" true 2>&1)
+  if [[ ! "$result" =~ "$p" ]] && [[ ! "$result" =~ "<not supported>" ]]; then
+    # We failed to see the event and it is supported. Possibly the workload was
+    # too small so retry with something longer.
+    result=$(perf stat -e "$p" perf bench internals synthesize 2>&1)
+    if [[ ! "$result" =~ "$p" ]]; then
+      echo "Event '$p' not printed in:"
+      echo "$result"
+      exit 1
+    fi
+  fi
+done
+
+exit 0
Signature · 1 line
2.33.0.464.g1972c5931b-goog
#2 lore raw
In reply to Ian Rogers

Em Fri, Sep 17, 2021 at 11:42:39AM -0700, Ian Rogers escreveu:

Quoted 9 lines

Test every metric and metricgroup with 'true' as a workload. For metrics, check that we see the metric printed or get unsupported. If the 'true' workload executes too quickly retry with 'perf bench internals synthesize'.

v3. Fix test condition (thanks to Paul A. Clarke <[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Hi John, does your Reviewed-by stands for v3 too?

- Arnaldo

Quoted 56 lines
---
 .../perf/tests/shell/stat_all_metricgroups.sh | 12 ++++++++++
 tools/perf/tests/shell/stat_all_metrics.sh    | 22 +++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100755 tools/perf/tests/shell/stat_all_metricgroups.sh
 create mode 100755 tools/perf/tests/shell/stat_all_metrics.sh
tools/perf/tests/shell/stat_all_metricgroups.sh new file +12 -0
new file mode 100755
index 000000000000..de24d374ce24
@@ -0,0 +1,12 @@
+#!/bin/sh
+# perf all metricgroups test
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+for m in $(perf list --raw-dump metricgroups); do
+  echo "Testing $m"
+  perf stat -M "$m" true
+done
+
+exit 0
tools/perf/tests/shell/stat_all_metrics.sh new file +22 -0
new file mode 100755
index 000000000000..7f4ba3cad632
@@ -0,0 +1,22 @@
+#!/bin/sh
+# perf all metrics test
+# SPDX-License-Identifier: GPL-2.0
+
+set -e
+
+for m in $(perf list --raw-dump metrics); do
+  echo "Testing $m"
+  result=$(perf stat -M "$m" true 2>&1)
+  if [[ ! "$result" =~ "$m" ]] && [[ ! "$result" =~ "<not supported>" ]]; then
+    # We failed to see the metric and the events are support. Possibly the
+    # workload was too small so retry with something longer.
+    result=$(perf stat -M "$m" perf bench internals synthesize 2>&1)
+    if [[ ! "$result" =~ "$m" ]]; then
+      echo "Metric '$m' not printed in:"
+      echo "$result"
+      exit 1
+    fi
+  fi
+done
+
+exit 0
Signature · 1 line
2.33.0.464.g1972c5931b-goog
Signature · 1 line
- Arnaldo
Reviewed-by
#3 lore raw
Quoted 6 lines · Arnaldo Carvalho de Melo

Em Fri, Sep 17, 2021 at 11:42:39AM -0700, Ian Rogers escreveu:

Quoted 5 lines

Test every metric and metricgroup with 'true' as a workload. For metrics, check that we see the metric printed or get unsupported. If the 'true' workload executes too quickly retry with 'perf bench internals synthesize'.

Quoted 8 lines
Quoted 4 lines

v3. Fix test condition (thanks to Paul A. Clarke<[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Hi John, does your Reviewed-by stands for v3 too?

- Arnaldo

Yeah,

Thanks

maintainer Applied
#4 lore raw

Em Fri, Sep 24, 2021 at 08:39:18PM +0100, John Garry escreveu:

Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 10 lines

Em Fri, Sep 17, 2021 at 11:42:39AM -0700, Ian Rogers escreveu:

Quoted 9 lines

Test every metric and metricgroup with 'true' as a workload. For metrics, check that we see the metric printed or get unsupported. If the 'true' workload executes too quickly retry with 'perf bench internals synthesize'.

v3. Fix test condition (thanks to Paul A. Clarke<[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Quoted 5 lines
Quoted 1 line

Hi John, does your Reviewed-by stands for v3 too?

Yeah,

Thanks, applied.

- Arnaldo

#5 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 10 lines · Arnaldo Carvalho de Melo

Em Fri, Sep 17, 2021 at 11:42:39AM -0700, Ian Rogers escreveu:

Quoted 9 lines

Test every metric and metricgroup with 'true' as a workload. For metrics, check that we see the metric printed or get unsupported. If the 'true' workload executes too quickly retry with 'perf bench internals synthesize'.

v3. Fix test condition (thanks to Paul A. Clarke<[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Hi Ian,

I just noticed that this test fails on my broadwell machine.

I am using acme perf/core @ 09dd3c22daaf

metricgroup Memory_Bw fails, and it seems because of the "true" argument to "perf stat" (or any argument, like sleep 1):

john@localhost:~/kernel-dev9/tools/perf> sudo ./perf stat -M Memory_BW ^C
Performance counter stats for 'system wide':

             2,184      arb/event=0x84,umask=0x1/ #     0.26 
DRAM_BW_Use
         2,954,938      arb/event=0x81,umask=0x1/ 

       736,368,852 ns   duration_time 

        58,202,980      l1d_pend_miss.pending_cycles #     2.34 MLP 
                 (80.11%)
       136,293,194      l1d_pend_miss.pending 
              (19.89%)
       736,368,852 ns   duration_time 

         1,065,656      longest_lat_cache.miss    #     0.09 
L3_Cache_Fill_BW         (39.71%)
       736,368,852 ns   duration_time 

         5,365,477      l2_lines_in.all           #     0.47 
L2_Cache_Fill_BW         (59.80%)
       736,368,852 ns   duration_time 

         3,557,362      l1d.replacement           #     0.31 
L1D_Cache_Fill_BW        (79.90%)
       736,368,852 ns   duration_time 


       0.736368852 seconds time elapsed

john@localhost:~/kernel-dev9/tools/perf> sudo ./perf stat -M Memory_BW true
Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (arb/event=0x84,umask=0x1/). /bin/dmesg | grep -i perf may provide additional information.

john@localhost:~/kernel-dev9/tools/perf>

Anyone any idea on this before I start digging?

Thanks,
John

#6 lore raw
Quoted 12 lines · John Garry
Quoted 10 lines · Arnaldo Carvalho de Melo

Em Fri, Sep 17, 2021 at 11:42:39AM -0700, Ian Rogers escreveu:

Quoted 9 lines

Test every metric and metricgroup with 'true' as a workload. For metrics, check that we see the metric printed or get unsupported. If the 'true' workload executes too quickly retry with 'perf bench internals synthesize'.

v3. Fix test condition (thanks to Paul A. Clarke<[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Quoted 5 lines

Hi Ian,

I just noticed that this test fails on my broadwell machine.

I am using acme perf/core @ 09dd3c22daaf

Hi,

Can you try with tmp.perf/perf_cpu instead?

There's a patch there that maybe fixes this.

- Arnaldo

Quoted 48 lines

metricgroup Memory_Bw fails, and it seems because of the "true" argument to "perf stat" (or any argument, like sleep 1):

john@localhost:~/kernel-dev9/tools/perf> sudo ./perf stat -M Memory_BW ^C
Performance counter stats for 'system wide':

             2,184      arb/event=0x84,umask=0x1/ #     0.26 
DRAM_BW_Use
         2,954,938      arb/event=0x81,umask=0x1/ 

       736,368,852 ns   duration_time 

        58,202,980      l1d_pend_miss.pending_cycles #     2.34 MLP 
                 (80.11%)
       136,293,194      l1d_pend_miss.pending 
              (19.89%)
       736,368,852 ns   duration_time 

         1,065,656      longest_lat_cache.miss    #     0.09 
L3_Cache_Fill_BW         (39.71%)
       736,368,852 ns   duration_time 

         5,365,477      l2_lines_in.all           #     0.47 
L2_Cache_Fill_BW         (59.80%)
       736,368,852 ns   duration_time 

         3,557,362      l1d.replacement           #     0.31 
L1D_Cache_Fill_BW        (79.90%)
       736,368,852 ns   duration_time 


       0.736368852 seconds time elapsed

john@localhost:~/kernel-dev9/tools/perf> sudo ./perf stat -M Memory_BW true Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (arb/event=0x84,umask=0x1/).
/bin/dmesg | grep -i perf may provide additional information.

john@localhost:~/kernel-dev9/tools/perf>

Anyone any idea on this before I start digging?

Thanks,
John

#7 lore raw
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

v3. Fix test condition (thanks to Paul A. Clarke<[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Hi Ian,

Quoted 8 lines
Quoted 3 lines

I just noticed that this test fails on my broadwell machine.

I am using acme perf/core @ 09dd3c22daaf

Hi,

Can you try with tmp.perf/perf_cpu instead?

There's a patch there that maybe fixes this.

Yeah, that (metricgroups) test passes on that branch. I assume it's the test script -a change...

Cheers,
john

#8 lore raw

Em Wed, Jan 12, 2022 at 05:32:56PM +0000, John Garry escreveu:

Quoted 10 lines · Arnaldo Carvalho de Melo
Quoted 9 lines
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines

v3. Fix test condition (thanks to Paul A. Clarke<[email protected]>). Add a
fallback case of a larger workload so that we don't ignore "<not counted>".
v2. Switched the workload to something faster.

Hi Ian,

I just noticed that this test fails on my broadwell machine.

I am using acme perf/core @ 09dd3c22daaf

Quoted 8 lines
Quoted 5 lines

Hi,

Can you try with tmp.perf/perf_cpu instead?

There's a patch there that maybe fixes this.

Yeah, that (metricgroups) test passes on that branch. I assume it's the test script -a change...

Can I get that as a Tested-by: you tag?

- Arnaldo

Tested-by
#9 lore raw
Quoted 4 lines · Arnaldo Carvalho de Melo
Quoted 3 lines
Quoted 1 line

There's a patch there that maybe fixes this.

Yeah, that (metricgroups) test passes on that branch. I assume it's the test script -a change...

Can I get that as a Tested-by: you tag?