Skip to contentSkip to conversation

perf test: Improve perf record tests (v1)

PATCH In review Namhyung Kim 2 participants 19 messages Sep 7, 2022 – Sep 13, 2022 perf events

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

cover perf test: Improve perf record tests (v1)

Hello,

This patchset improves the perf record tests to check more cases so that it can find problems early. The motivation is a failure in per-thread mmap with multi-threaded targets which Adrian is working on the fix.

I added a custom test program and more combinations like system-wide and command line workload (in per-process mode) testing with multi-threaded recording mode.

Currently it fails on per-thread and register capture tests. The system-wide test was skipped since I ran it as a normal user. We can use this to verify Adirian's fix and future works.

  $ ./perf test -v 86
   86: perf record tests                                               :
  --- start ---
  test child forked, pid 1190747
  Build a test program
  Basic --per-thread mode test
  Per-thread record [Failed record]
  Register capture test
  Register capture test [Failed missing output]
  Basic --system-wide mode test
  System-wide record [Skipped not supported]
  Basic target workload test
  Basic target workload test [Success]
  test child finished with -1
  ---- end ----
  perf record tests: FAILED!

You can find it in 'perf/record-test-v1' branch in

git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git

Thanks,
Namhyung

Namhyung Kim (6):
perf test: Do not use instructions:u explicitly perf test: Use a test program in perf record tests perf test: Test record with --threads option perf test: Add system-wide mode in perf record tests perf test: Add target workload test in perf record tests perf test: Do not set TEST_SKIP for record subtests

 tools/perf/tests/shell/record.sh | 150 +++++++++++++++++++++++++++----
 1 file changed, 133 insertions(+), 17 deletions(-)

base-commit: 6c3bd8d3e01d9014312caa52e4ef1c29d5249648

Signature · 1 line
2.37.2.789.g6183377224-goog
author maintainer
#1 lore raw

PATCH 1/6 perf test: Do not use instructions:u explicitly

I think it's to support non-root user tests. But perf record can handle the case and fall back to a software event (cpu-clock). Practically this would affect when it's run on a VM, but it seems no reason to prevent running the test in the guest.

---
 tools/perf/tests/shell/record.sh | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/tests/shell/record.sh +5 -5
index 00c7285ce1ac..40b087bfdb76 100755
@@ -21,18 +21,18 @@ trap trap_cleanup exit term int
 
 test_per_thread() {
   echo "Basic --per-thread mode test"
-  if ! perf record -e instructions:u -o ${perfdata} --quiet true 2> /dev/null
+  if ! perf record -o /dev/null --quiet true 2> /dev/null
   then
-    echo "Per-thread record [Skipped instructions:u not supported]"
+    echo "Per-thread record [Skipped event not supported]"
     if [ $err -ne 1 ]
     then
       err=2
     fi
     return
   fi
-  if ! perf record -e instructions:u --per-thread -o ${perfdata} true 2> /dev/null
+  if ! perf record --per-thread -o ${perfdata} true 2> /dev/null
   then
-    echo "Per-thread record of instructions:u [Failed]"
+    echo "Per-thread record [Failed record]"
     err=1
     return
   fi
@@ -49,7 +49,7 @@ test_register_capture() {
   echo "Register capture test"
   if ! perf list | egrep -q 'br_inst_retired.near_call'
   then
-    echo "Register capture test [Skipped missing instruction]"
+    echo "Register capture test [Skipped missing event]"
     if [ $err -ne 1 ]
     then
       err=2
Signature · 1 line
2.37.2.789.g6183377224-goog
reviewer Reviewed-by on v1 1/6
#2 lore raw

Re: [PATCH 1/6] perf test: Do not use instructions:u explicitly

Quoted 6 lines · Namhyung Kim

I think it's to support non-root user tests. But perf record can handle the case and fall back to a software event (cpu-clock). Practically this would affect when it's run on a VM, but it seems no reason to prevent running the test in the guest.

Quoted 40 lines
---
 tools/perf/tests/shell/record.sh | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/tests/shell/record.sh +5 -5
index 00c7285ce1ac..40b087bfdb76 100755
@@ -21,18 +21,18 @@ trap trap_cleanup exit term int
 
 test_per_thread() {
   echo "Basic --per-thread mode test"
-  if ! perf record -e instructions:u -o ${perfdata} --quiet true 2> /dev/null
+  if ! perf record -o /dev/null --quiet true 2> /dev/null
   then
-    echo "Per-thread record [Skipped instructions:u not supported]"
+    echo "Per-thread record [Skipped event not supported]"
     if [ $err -ne 1 ]
     then
       err=2
     fi
     return
   fi
-  if ! perf record -e instructions:u --per-thread -o ${perfdata} true 2> /dev/null
+  if ! perf record --per-thread -o ${perfdata} true 2> /dev/null
   then
-    echo "Per-thread record of instructions:u [Failed]"
+    echo "Per-thread record [Failed record]"
     err=1
     return
   fi
@@ -49,7 +49,7 @@ test_register_capture() {
   echo "Register capture test"
   if ! perf list | egrep -q 'br_inst_retired.near_call'
   then
-    echo "Register capture test [Skipped missing instruction]"
+    echo "Register capture test [Skipped missing event]"
     if [ $err -ne 1 ]
     then
       err=2
author maintainer
#3 lore raw

PATCH 2/6 perf test: Use a test program in perf record tests

In reply to Namhyung Kim

If the system has cc it could build a test program with two threads and then use it for more detailed testing. Also it adds initial delay of 3ms to profile a multi-threaded target. This change make the test failing but that's what we want to check for now.

If cc is not found, it falls back to use the default value 'true'.

---
 tools/perf/tests/shell/record.sh | 58 +++++++++++++++++++++++++++++---
 1 file changed, 54 insertions(+), 4 deletions(-)
tools/perf/tests/shell/record.sh +54 -4
index 40b087bfdb76..cea3c7b7e2cd 100755
@@ -6,10 +6,18 @@ set -e
 
 err=0
 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+testprog=$(mktemp /tmp/__perf_test.prog.XXXXXX)
+testsym="test_loop"
+testopt="-D 3"
 
 cleanup() {
   rm -f ${perfdata}
   rm -f ${perfdata}.old
+
+  if [ "${testprog}" != "true" ]; then
+    rm -f ${testprog}
+  fi
+
   trap - exit term int
 }
 
@@ -19,9 +27,49 @@ trap_cleanup() {
 }
 trap trap_cleanup exit term int
 
+build_test_program() {
+  if ! [ -x "$(command -v cc)" ]; then
+    # No CC found. Fall back to 'true'
+    testprog=true
+    testsym=true
+    testopt=''
+    return
+  fi
+
+  echo "Build a test program"
+  cat <<EOF | cc -o ${testprog} -xc - -pthread
+#include <stdio.h>
+#include <unistd.h>
+#include <pthread.h>
+
+void test_loop(void) {
+  volatile int count = 1000000;
+
+  // wait for perf record
+  usleep(5000);
+
+  while (count--)
+    continue;
+}
+
+void *thfunc(void *arg) {
+  test_loop();
+  return NULL;
+}
+
+int main(void) {
+  pthread_t th;
+  pthread_create(&th, NULL, thfunc, NULL);
+  test_loop();
+  pthread_join(th, NULL);
+  return 0;
+}
+EOF
+}
+
 test_per_thread() {
   echo "Basic --per-thread mode test"
-  if ! perf record -o /dev/null --quiet true 2> /dev/null
+  if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Skipped event not supported]"
     if [ $err -ne 1 ]
@@ -30,13 +78,13 @@ test_per_thread() {
     fi
     return
   fi
-  if ! perf record --per-thread -o ${perfdata} true 2> /dev/null
+  if ! perf record --per-thread ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Failed record]"
     err=1
     return
   fi
-  if ! perf report -i ${perfdata} -q | egrep -q true
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
   then
     echo "Per-thread record [Failed missing output]"
     err=1
@@ -62,7 +110,7 @@ test_register_capture() {
     return
   fi
   if ! perf record -o - --intr-regs=di,r8,dx,cx -e cpu/br_inst_retired.near_call/p \
-    -c 1000 --per-thread true 2> /dev/null \
+    -c 1000 --per-thread ${testopt} ${testprog} 2> /dev/null \
     | perf script -F ip,sym,iregs -i - 2> /dev/null \
     | egrep -q "DI:"
   then
@@ -73,6 +121,8 @@ test_register_capture() {
   echo "Register capture test [Success]"
 }
 
+build_test_program
+
 test_per_thread
 test_register_capture
 
Signature · 1 line
2.37.2.789.g6183377224-goog
reviewer
#4 lore raw

Re: [PATCH 2/6] perf test: Use a test program in perf record tests

Quoted 4 lines · Namhyung Kim

If the system has cc it could build a test program with two threads and then use it for more detailed testing. Also it adds initial delay of 3ms to profile a multi-threaded target. This change make the test failing but that's what we want to check for now.

So the delay is just to get a separate dummy event?

I hit the issue from this patch:

https://lore.kernel.org/lkml/[email protected]/

That is, if I apply the patch above then the test passes, otherwise it fails always.

Quoted 116 lines

If cc is not found, it falls back to use the default value 'true'.

---
 tools/perf/tests/shell/record.sh | 58 +++++++++++++++++++++++++++++---
 1 file changed, 54 insertions(+), 4 deletions(-)
tools/perf/tests/shell/record.sh +54 -4
index 40b087bfdb76..cea3c7b7e2cd 100755
@@ -6,10 +6,18 @@ set -e
 
 err=0
 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+testprog=$(mktemp /tmp/__perf_test.prog.XXXXXX)
+testsym="test_loop"
+testopt="-D 3"
 
 cleanup() {
   rm -f ${perfdata}
   rm -f ${perfdata}.old
+
+  if [ "${testprog}" != "true" ]; then
+    rm -f ${testprog}
+  fi
+
   trap - exit term int
 }
 
@@ -19,9 +27,49 @@ trap_cleanup() {
 }
 trap trap_cleanup exit term int
 
+build_test_program() {
+  if ! [ -x "$(command -v cc)" ]; then
+    # No CC found. Fall back to 'true'
+    testprog=true
+    testsym=true
+    testopt=''
+    return
+  fi
+
+  echo "Build a test program"
+  cat <<EOF | cc -o ${testprog} -xc - -pthread
+#include <stdio.h>
+#include <unistd.h>
+#include <pthread.h>
+
+void test_loop(void) {
+  volatile int count = 1000000;
+
+  // wait for perf record
+  usleep(5000);
+
+  while (count--)
+    continue;
+}
+
+void *thfunc(void *arg) {
+  test_loop();
+  return NULL;
+}
+
+int main(void) {
+  pthread_t th;
+  pthread_create(&th, NULL, thfunc, NULL);
+  test_loop();
+  pthread_join(th, NULL);
+  return 0;
+}
+EOF
+}
+
 test_per_thread() {
   echo "Basic --per-thread mode test"
-  if ! perf record -o /dev/null --quiet true 2> /dev/null
+  if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Skipped event not supported]"
     if [ $err -ne 1 ]
@@ -30,13 +78,13 @@ test_per_thread() {
     fi
     return
   fi
-  if ! perf record --per-thread -o ${perfdata} true 2> /dev/null
+  if ! perf record --per-thread ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Failed record]"
     err=1
     return
   fi
-  if ! perf report -i ${perfdata} -q | egrep -q true
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
   then
     echo "Per-thread record [Failed missing output]"
     err=1
@@ -62,7 +110,7 @@ test_register_capture() {
     return
   fi
   if ! perf record -o - --intr-regs=di,r8,dx,cx -e cpu/br_inst_retired.near_call/p \
-    -c 1000 --per-thread true 2> /dev/null \
+    -c 1000 --per-thread ${testopt} ${testprog} 2> /dev/null \
     | perf script -F ip,sym,iregs -i - 2> /dev/null \
     | egrep -q "DI:"
   then
@@ -73,6 +121,8 @@ test_register_capture() {
   echo "Register capture test [Success]"
 }
 
+build_test_program
+
 test_per_thread
 test_register_capture
 
author maintainer
#5 lore raw

Re: [PATCH 2/6] perf test: Use a test program in perf record tests

Hi Adrian,

Quoted 8 lines · Adrian Hunter
Quoted 4 lines · Namhyung Kim

If the system has cc it could build a test program with two threads and then use it for more detailed testing. Also it adds initial delay of 3ms to profile a multi-threaded target. This change make the test failing but that's what we want to check for now.

So the delay is just to get a separate dummy event?

No, it's actually to wait for the new thread.

Quoted 6 lines

I hit the issue from this patch:

https://lore.kernel.org/lkml/[email protected]/

That is, if I apply the patch above then the test passes, otherwise it fails always.

It's good to find/verify the bug with this :)

Thanks,
Namhyung

reviewer
#6 lore raw

Re: [PATCH 2/6] perf test: Use a test program in perf record tests

In reply to Namhyung Kim
Quoted 103 lines · Namhyung Kim

If the system has cc it could build a test program with two threads and then use it for more detailed testing. Also it adds initial delay of 3ms to profile a multi-threaded target. This change make the test failing but that's what we want to check for now.

If cc is not found, it falls back to use the default value 'true'.

---
 tools/perf/tests/shell/record.sh | 58 +++++++++++++++++++++++++++++---
 1 file changed, 54 insertions(+), 4 deletions(-)
tools/perf/tests/shell/record.sh +51 -3
index 40b087bfdb76..cea3c7b7e2cd 100755
@@ -6,10 +6,18 @@ set -e
 
 err=0
 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+testprog=$(mktemp /tmp/__perf_test.prog.XXXXXX)
+testsym="test_loop"
+testopt="-D 3"
 
 cleanup() {
   rm -f ${perfdata}
   rm -f ${perfdata}.old
+
+  if [ "${testprog}" != "true" ]; then
+    rm -f ${testprog}
+  fi
+
   trap - exit term int
 }
 
@@ -19,9 +27,49 @@ trap_cleanup() {
 }
 trap trap_cleanup exit term int
 
+build_test_program() {
+  if ! [ -x "$(command -v cc)" ]; then
+    # No CC found. Fall back to 'true'
+    testprog=true
+    testsym=true
+    testopt=''
+    return
+  fi
+
+  echo "Build a test program"
+  cat <<EOF | cc -o ${testprog} -xc - -pthread
+#include <stdio.h>
+#include <unistd.h>
+#include <pthread.h>
+
+void test_loop(void) {
+  volatile int count = 1000000;
+
+  // wait for perf record
+  usleep(5000);
+
+  while (count--)
+    continue;
+}
+
+void *thfunc(void *arg) {
+  test_loop();
+  return NULL;
+}
+
+int main(void) {
+  pthread_t th;
+  pthread_create(&th, NULL, thfunc, NULL);
+  test_loop();
+  pthread_join(th, NULL);
+  return 0;
+}
+EOF
+}
+
 test_per_thread() {
   echo "Basic --per-thread mode test"
-  if ! perf record -o /dev/null --quiet true 2> /dev/null
+  if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Skipped event not supported]"
     if [ $err -ne 1 ]
@@ -30,13 +78,13 @@ test_per_thread() {
     fi
     return
   fi
-  if ! perf record --per-thread -o ${perfdata} true 2> /dev/null
+  if ! perf record --per-thread ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Failed record]"
     err=1
     return
   fi
-  if ! perf report -i ${perfdata} -q | egrep -q true
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
   then
     echo "Per-thread record [Failed missing output]"
     err=1

tools/perf/tests/shell/record.sh test_register_capture() { · line 114

Quoted 6 lines
tools/perf/tests/shell/record.sh +1 -1
@@ -62,7 +110,7 @@ test_register_capture() {
     return
   fi
   if ! perf record -o - --intr-regs=di,r8,dx,cx -e cpu/br_inst_retired.near_call/p \
-    -c 1000 --per-thread true 2> /dev/null \
+    -c 1000 --per-thread ${testopt} ${testprog} 2> /dev/null \
     | perf script -F ip,sym,iregs -i - 2> /dev/null \

With the kernel patch:

https://lore.kernel.org/lkml/[email protected]/

I get:

Samples for 'dummy:HG' event do not have IREGS attribute set. Cannot print 'iregs' field.

Quoted 11 lines
tools/perf/tests/shell/record.sh +2 -0
     | egrep -q "DI:"
   then
@@ -73,6 +121,8 @@ test_register_capture() {
   echo "Register capture test [Success]"
 }
 
+build_test_program
+
 test_per_thread
 test_register_capture
 
reviewer
#7 lore raw

Re: [PATCH 2/6] perf test: Use a test program in perf record tests

Quoted 113 lines · Adrian Hunter
Quoted 103 lines · Namhyung Kim

If the system has cc it could build a test program with two threads and then use it for more detailed testing. Also it adds initial delay of 3ms to profile a multi-threaded target. This change make the test failing but that's what we want to check for now.

If cc is not found, it falls back to use the default value 'true'.

---
 tools/perf/tests/shell/record.sh | 58 +++++++++++++++++++++++++++++---
 1 file changed, 54 insertions(+), 4 deletions(-)
tools/perf/tests/shell/record.sh +51 -3
index 40b087bfdb76..cea3c7b7e2cd 100755
@@ -6,10 +6,18 @@ set -e
 
 err=0
 perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX)
+testprog=$(mktemp /tmp/__perf_test.prog.XXXXXX)
+testsym="test_loop"
+testopt="-D 3"
 
 cleanup() {
   rm -f ${perfdata}
   rm -f ${perfdata}.old
+
+  if [ "${testprog}" != "true" ]; then
+    rm -f ${testprog}
+  fi
+
   trap - exit term int
 }
 
@@ -19,9 +27,49 @@ trap_cleanup() {
 }
 trap trap_cleanup exit term int
 
+build_test_program() {
+  if ! [ -x "$(command -v cc)" ]; then
+    # No CC found. Fall back to 'true'
+    testprog=true
+    testsym=true
+    testopt=''
+    return
+  fi
+
+  echo "Build a test program"
+  cat <<EOF | cc -o ${testprog} -xc - -pthread
+#include <stdio.h>
+#include <unistd.h>
+#include <pthread.h>
+
+void test_loop(void) {
+  volatile int count = 1000000;
+
+  // wait for perf record
+  usleep(5000);
+
+  while (count--)
+    continue;
+}
+
+void *thfunc(void *arg) {
+  test_loop();
+  return NULL;
+}
+
+int main(void) {
+  pthread_t th;
+  pthread_create(&th, NULL, thfunc, NULL);
+  test_loop();
+  pthread_join(th, NULL);
+  return 0;
+}
+EOF
+}
+
 test_per_thread() {
   echo "Basic --per-thread mode test"
-  if ! perf record -o /dev/null --quiet true 2> /dev/null
+  if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Skipped event not supported]"
     if [ $err -ne 1 ]
@@ -30,13 +78,13 @@ test_per_thread() {
     fi
     return
   fi
-  if ! perf record --per-thread -o ${perfdata} true 2> /dev/null
+  if ! perf record --per-thread ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Failed record]"
     err=1
     return
   fi
-  if ! perf report -i ${perfdata} -q | egrep -q true
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
   then
     echo "Per-thread record [Failed missing output]"
     err=1

tools/perf/tests/shell/record.sh test_register_capture() { · line 114

Quoted 6 lines
tools/perf/tests/shell/record.sh +1 -1
@@ -62,7 +110,7 @@ test_register_capture() {
     return
   fi
   if ! perf record -o - --intr-regs=di,r8,dx,cx -e cpu/br_inst_retired.near_call/p \
-    -c 1000 --per-thread true 2> /dev/null \
+    -c 1000 --per-thread ${testopt} ${testprog} 2> /dev/null \
     | perf script -F ip,sym,iregs -i - 2> /dev/null \

With the kernel patch:

Quoted 5 lines

https://lore.kernel.org/lkml/[email protected]/

I get:

Samples for 'dummy:HG' event do not have IREGS attribute set. Cannot print 'iregs' field.

We seem to need:

tools/perf/builtin-script.c +1 -1
index 13580a9c50b8d..959291903936a 100644
@@ -411,7 +411,7 @@ static int evsel__do_check_stype(struct evsel *evsel, u64 sample_type, const cha
       int type = output_type(attr->type);
       const char *evname;
 
-       if (attr->sample_type & sample_type)
+       if (evsel__is_dummy_event(evsel) || attr->sample_type & sample_type)
               return 0;
 
       if (output[type].user_set_fields & field) {
tools/perf/tests/shell/record.sh +0 -0
Quoted 13 lines
Quoted 12 lines
tools/perf/tests/shell/record.sh +2 -0
     | egrep -q "DI:"
   then
@@ -73,6 +121,8 @@ test_register_capture() {
   echo "Register capture test [Success]"
 }
 
+build_test_program
+
 test_per_thread
 test_register_capture
 
author maintainer
#8 lore raw

Re: [PATCH 2/6] perf test: Use a test program in perf record tests

Quoted 10 lines · Adrian Hunter
Quoted 8 lines · Adrian Hunter

Review

Quoted 4 lines · Namhyung Kim
   if ! perf record -o - --intr-regs=di,r8,dx,cx -e cpu/br_inst_retired.near_call/p \
(quoted diff) +1 -1
-    -c 1000 --per-thread true 2> /dev/null \
+    -c 1000 --per-thread ${testopt} ${testprog} 2> /dev/null \
     | perf script -F ip,sym,iregs -i - 2> /dev/null \

With the kernel patch:

Quoted 7 lines
Quoted 5 lines

https://lore.kernel.org/lkml/[email protected]/

I get:

Samples for 'dummy:HG' event do not have IREGS attribute set. Cannot print 'iregs' field.

We seem to need:

Yeah, I think we have a discussion already.

Thanks,
Namhyung

Quoted 15 lines
tools/perf/builtin-script.c +1 -1
index 13580a9c50b8d..959291903936a 100644
@@ -411,7 +411,7 @@ static int evsel__do_check_stype(struct evsel *evsel, u64 sample_type, const cha
       int type = output_type(attr->type);
       const char *evname;
 
-       if (attr->sample_type & sample_type)
+       if (evsel__is_dummy_event(evsel) || attr->sample_type & sample_type)
               return 0;
 
       if (output[type].user_set_fields & field) {
tools/perf/tests/shell/record.sh +0 -0
author maintainer
#9 lore raw

PATCH 3/6 perf test: Test record with --threads option

In reply to Namhyung Kim

To verify per-thread mode is working with multi-thread recording. Use two software events for testing to check event set-output handling. Also update the cleanup routine because threads recording produces data in a directory.

---
 tools/perf/tests/shell/record.sh | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
tools/perf/tests/shell/record.sh +15 -2
index cea3c7b7e2cd..3331fb092654 100755
@@ -11,8 +11,8 @@ testsym="test_loop"
 testopt="-D 3"
 
 cleanup() {
-  rm -f ${perfdata}
-  rm -f ${perfdata}.old
+  rm -rf ${perfdata}
+  rm -rf ${perfdata}.old
 
   if [ "${testprog}" != "true" ]; then
     rm -f ${testprog}
@@ -90,6 +90,19 @@ test_per_thread() {
     err=1
     return
   fi
+  if ! perf record -e cpu-clock,cs --per-thread --threads=core ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Per-thread record with --threads [Failed]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Per-thread record with --threads [Failed missing output]"
+    err=1
+    return
+  fi
   echo "Basic --per-thread mode test [Success]"
 }
 
Signature · 1 line
2.37.2.789.g6183377224-goog
reviewer
#10 lore raw

Re: [PATCH 3/6] perf test: Test record with --threads option

Quoted 26 lines · Namhyung Kim

To verify per-thread mode is working with multi-thread recording. Use two software events for testing to check event set-output handling. Also update the cleanup routine because threads recording produces data in a directory.

---
 tools/perf/tests/shell/record.sh | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
tools/perf/tests/shell/record.sh +2 -2
index cea3c7b7e2cd..3331fb092654 100755
@@ -11,8 +11,8 @@ testsym="test_loop"
 testopt="-D 3"
 
 cleanup() {
-  rm -f ${perfdata}
-  rm -f ${perfdata}.old
+  rm -rf ${perfdata}
+  rm -rf ${perfdata}.old
 
   if [ "${testprog}" != "true" ]; then
     rm -f ${testprog}

tools/perf/tests/shell/record.sh test_per_thread() { · line 94

Quoted 5 lines
tools/perf/tests/shell/record.sh +2 -0
@@ -90,6 +90,19 @@ test_per_thread() {
     err=1
     return
   fi
+  if ! perf record -e cpu-clock,cs --per-thread --threads=core ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null

This does not seem to be possible, because it gives the error:

--per-thread option is mutually exclusive to parallel streaming mode. Failed to initialize parallel data streaming masks

Quoted 14 lines
tools/perf/tests/shell/record.sh +11 -0
+  then
+    echo "Per-thread record with --threads [Failed]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Per-thread record with --threads [Failed missing output]"
+    err=1
+    return
+  fi
   echo "Basic --per-thread mode test [Success]"
 }
 
author maintainer
#11 lore raw

Re: [PATCH 3/6] perf test: Test record with --threads option

Quoted 11 lines · Adrian Hunter

tools/perf/tests/shell/record.sh test_per_thread() { · line 94

Quoted 6 lines · Namhyung Kim
tools/perf/tests/shell/record.sh +2 -0
@@ -90,6 +90,19 @@ test_per_thread() {
     err=1
     return
   fi
+  if ! perf record -e cpu-clock,cs --per-thread --threads=core ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null

This does not seem to be possible, because it gives the error:

Quoted 2 lines

--per-thread option is mutually exclusive to parallel streaming mode. Failed to initialize parallel data streaming masks

Oh, ok. Let's drop this change then.

Thanks,
Namhyung

author maintainer
#12 lore raw

PATCH 4/6 perf test: Add system-wide mode in perf record tests

In reply to Namhyung Kim

Add system wide recording test with the same pattern. It'd skip the test when it failes to run perf record. For system-wide mode, it needs to avoid build-id collection and synthesis because the test only cares about the test program and kernel would generates necessary events as the process starts.

---
 tools/perf/tests/shell/record.sh | 34 ++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
tools/perf/tests/shell/record.sh +34 -0
index 3331fb092654..bd4ef60948bd 100755
@@ -134,10 +134,44 @@ test_register_capture() {
   echo "Register capture test [Success]"
 }
 
+test_system_wide() {
+  echo "Basic --system-wide mode test"
+  if ! perf record -aB --synth=no ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "System-wide record [Skipped not supported]"
+    if [ $err -ne 1 ]
+    then
+      err=2
+    fi
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "System-wide record [Failed missing output]"
+    err=1
+    return
+  fi
+  if ! perf record -aB --synth=no -e cpu-clock,cs --threads=cpu ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "System-wide test [Failed recording with threads]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "System-wide record [Failed missing output]"
+    err=1
+    return
+  fi
+  echo "Basic --system-wide mode test [Success]"
+}
+
 build_test_program
 
 test_per_thread
 test_register_capture
+test_system_wide
 
 cleanup
 exit $err
Signature · 1 line
2.37.2.789.g6183377224-goog
reviewer Reviewed-by on v1 4/6
#13 lore raw

Re: [PATCH 4/6] perf test: Add system-wide mode in perf record tests

Quoted 7 lines · Namhyung Kim

Add system wide recording test with the same pattern. It'd skip the test when it failes to run perf record. For system-wide mode, it needs to avoid build-id collection and synthesis because the test only cares about the test program and kernel would generates necessary events as the process starts.

Quoted 53 lines
---
 tools/perf/tests/shell/record.sh | 34 ++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
tools/perf/tests/shell/record.sh +34 -0
index 3331fb092654..bd4ef60948bd 100755
@@ -134,10 +134,44 @@ test_register_capture() {
   echo "Register capture test [Success]"
 }
 
+test_system_wide() {
+  echo "Basic --system-wide mode test"
+  if ! perf record -aB --synth=no ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "System-wide record [Skipped not supported]"
+    if [ $err -ne 1 ]
+    then
+      err=2
+    fi
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "System-wide record [Failed missing output]"
+    err=1
+    return
+  fi
+  if ! perf record -aB --synth=no -e cpu-clock,cs --threads=cpu ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "System-wide test [Failed recording with threads]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "System-wide record [Failed missing output]"
+    err=1
+    return
+  fi
+  echo "Basic --system-wide mode test [Success]"
+}
+
 build_test_program
 
 test_per_thread
 test_register_capture
+test_system_wide
 
 cleanup
 exit $err
author maintainer
#14 lore raw

PATCH 5/6 perf test: Add target workload test in perf record tests

In reply to Namhyung Kim

Add a subtest which profiles the given workload on the command line. As it's a minimal requirement, test should run ok so it doesn't skip the test even if it failed to run the perf record.

---
 tools/perf/tests/shell/record.sh | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
tools/perf/tests/shell/record.sh +31 -0
index bd4ef60948bd..ff66e58f3a26 100755
@@ -167,11 +167,42 @@ test_system_wide() {
   echo "Basic --system-wide mode test [Success]"
 }
 
+test_workload() {
+  echo "Basic target workload test"
+  if ! perf record ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Workload record [Failed record]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Workload record [Failed missing output]"
+    err=1
+    return
+  fi
+  if ! perf record -e cpu-clock,cs --threads=package ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Workload record [Failed recording with threads]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Workload record [Failed missing output]"
+    err=1
+    return
+  fi
+  echo "Basic target workload test [Success]"
+}
+
 build_test_program
 
 test_per_thread
 test_register_capture
 test_system_wide
+test_workload
 
 cleanup
 exit $err
Signature · 1 line
2.37.2.789.g6183377224-goog
reviewer
#15 lore raw

Re: [PATCH 5/6] perf test: Add target workload test in perf record tests

Quoted 31 lines · Namhyung Kim

Add a subtest which profiles the given workload on the command line. As it's a minimal requirement, test should run ok so it doesn't skip the test even if it failed to run the perf record.

---
 tools/perf/tests/shell/record.sh | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
tools/perf/tests/shell/record.sh +14 -0
index bd4ef60948bd..ff66e58f3a26 100755
@@ -167,11 +167,42 @@ test_system_wide() {
   echo "Basic --system-wide mode test [Success]"
 }
 
+test_workload() {
+  echo "Basic target workload test"
+  if ! perf record ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Workload record [Failed record]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Workload record [Failed missing output]"
+    err=1
+    return
+  fi

tools/perf/tests/shell/record.sh test_system_wide() { · line 193

Quoted 10 lines
tools/perf/tests/shell/record.sh +10 -0
+  if ! perf record -e cpu-clock,cs --threads=package ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Workload record [Failed recording with threads]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Workload record [Failed missing output]"

When testing, this was happening always until changing the delay from 3 to 1 ms. It might be a bit racy to work consistently on different machines.

Another approach is to wait for threads to appear in /proc/N/task like wait_for_threads() here:

	https://lore.kernel.org/linux-perf-users/[email protected]/T/#u
Quoted 15 lines
tools/perf/tests/shell/record.sh +7 -0
+    err=1
+    return
+  fi
+  echo "Basic target workload test [Success]"
+}
+
 build_test_program
 
 test_per_thread
 test_register_capture
 test_system_wide
+test_workload
 
 cleanup
 exit $err
author maintainer
#16 lore raw

Re: [PATCH 5/6] perf test: Add target workload test in perf record tests

Hi Adrian,

Quoted 48 lines · Adrian Hunter
Quoted 31 lines · Namhyung Kim

Add a subtest which profiles the given workload on the command line. As it's a minimal requirement, test should run ok so it doesn't skip the test even if it failed to run the perf record.

---
 tools/perf/tests/shell/record.sh | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
tools/perf/tests/shell/record.sh +14 -0
index bd4ef60948bd..ff66e58f3a26 100755
@@ -167,11 +167,42 @@ test_system_wide() {
   echo "Basic --system-wide mode test [Success]"
 }
 
+test_workload() {
+  echo "Basic target workload test"
+  if ! perf record ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Workload record [Failed record]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Workload record [Failed missing output]"
+    err=1
+    return
+  fi

tools/perf/tests/shell/record.sh test_system_wide() { · line 193

Quoted 10 lines
tools/perf/tests/shell/record.sh +10 -0
+  if ! perf record -e cpu-clock,cs --threads=package ${testopt} \
+    -o ${perfdata} ${testprog} 2> /dev/null
+  then
+    echo "Workload record [Failed recording with threads]"
+    err=1
+    return
+  fi
+  if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
+  then
+    echo "Workload record [Failed missing output]"

When testing, this was happening always until changing the delay from 3 to 1 ms. It might be a bit racy to work consistently on different machines.

Quoted 4 lines

Another approach is to wait for threads to appear in /proc/N/task like wait_for_threads() here:

        https://lore.kernel.org/linux-perf-users/[email protected]/T/#u

Looks good. Maybe we can move it to tests/shell/lib ?

Thanks,
Namhyung

author maintainer
#17 lore raw

PATCH 6/6 perf test: Do not set TEST_SKIP for record subtests

In reply to Namhyung Kim

It now has 4 sub tests and one of them should run at least. But once TEST_SKIP (= 2) return value is set, it won't be overwritten unless there's a failure. I think we should return success when one or more tested are skipped but the remaining subtests are passed.

So update the test code not to set the err variable when it skips the test.

---
 tools/perf/tests/shell/record.sh | 12 ------------
 1 file changed, 12 deletions(-)
tools/perf/tests/shell/record.sh +0 -12
index ff66e58f3a26..487981767455 100755
@@ -72,10 +72,6 @@ test_per_thread() {
   if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Skipped event not supported]"
-    if [ $err -ne 1 ]
-    then
-      err=2
-    fi
     return
   fi
   if ! perf record --per-thread ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
@@ -111,10 +107,6 @@ test_register_capture() {
   if ! perf list | egrep -q 'br_inst_retired.near_call'
   then
     echo "Register capture test [Skipped missing event]"
-    if [ $err -ne 1 ]
-    then
-      err=2
-    fi
     return
   fi
   if ! perf record --intr-regs=\? 2>&1 | egrep -q 'available registers: AX BX CX DX SI DI BP SP IP FLAGS CS SS R8 R9 R10 R11 R12 R13 R14 R15'
@@ -139,10 +131,6 @@ test_system_wide() {
   if ! perf record -aB --synth=no ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
   then
     echo "System-wide record [Skipped not supported]"
-    if [ $err -ne 1 ]
-    then
-      err=2
-    fi
     return
   fi
   if ! perf report -i ${perfdata} -q | egrep -q ${testsym}
Signature · 1 line
2.37.2.789.g6183377224-goog
reviewer Reviewed-by on v1 6/6
#18 lore raw

Re: [PATCH 6/6] perf test: Do not set TEST_SKIP for record subtests

Quoted 9 lines · Namhyung Kim

It now has 4 sub tests and one of them should run at least. But once TEST_SKIP (= 2) return value is set, it won't be overwritten unless there's a failure. I think we should return success when one or more tested are skipped but the remaining subtests are passed.

So update the test code not to set the err variable when it skips the test.

I tend to agree that the perf's default event should work.

Quoted 41 lines
---
 tools/perf/tests/shell/record.sh | 12 ------------
 1 file changed, 12 deletions(-)
tools/perf/tests/shell/record.sh +0 -12
index ff66e58f3a26..487981767455 100755
@@ -72,10 +72,6 @@ test_per_thread() {
   if ! perf record -o /dev/null --quiet ${testprog} 2> /dev/null
   then
     echo "Per-thread record [Skipped event not supported]"
-    if [ $err -ne 1 ]
-    then
-      err=2
-    fi
     return
   fi
   if ! perf record --per-thread ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
@@ -111,10 +107,6 @@ test_register_capture() {
   if ! perf list | egrep -q 'br_inst_retired.near_call'
   then
     echo "Register capture test [Skipped missing event]"
-    if [ $err -ne 1 ]
-    then
-      err=2
-    fi
     return
   fi
   if ! perf record --intr-regs=\? 2>&1 | egrep -q 'available registers: AX BX CX DX SI DI BP SP IP FLAGS CS SS R8 R9 R10 R11 R12 R13 R14 R15'
@@ -139,10 +131,6 @@ test_system_wide() {
   if ! perf record -aB --synth=no ${testopt} -o ${perfdata} ${testprog} 2> /dev/null
   then
     echo "System-wide record [Skipped not supported]"
-    if [ $err -ne 1 ]
-    then
-      err=2
-    fi
     return
   fi
   if ! perf report -i ${perfdata} -q | egrep -q ${testsym}