Skip to contentSkip to conversation

perf bench sched pipe: Add -G/--cgroups option

PATCH Superseded Namhyung Kim 2 participants 6 messages Oct 4, 2023 – Oct 13, 2023 perf events

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

PATCH 1/1 perf bench sched pipe: Add -G/--cgroups option

The -G/--cgroups option is to put sender and receiver in different cgroups in order to measure cgroup context switch overheads.

Users need to make sure the cgroups exist and accessible.

  # perf stat -e context-switches,cgroup-switches \
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 > /dev/null

Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000':

              20,001      context-switches
                   2      cgroup-switches

         0.053449651 seconds time elapsed

         0.011286000 seconds user
         0.041869000 seconds sys
  # perf stat -e context-switches,cgroup-switches \
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB > /dev/null

Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB':

              20,001      context-switches
              20,001      cgroup-switches

         0.052768627 seconds time elapsed

         0.006284000 seconds user
         0.046266000 seconds sys
---
 tools/perf/Documentation/perf-bench.txt | 19 +++++
 tools/perf/bench/sched-pipe.c           | 93 +++++++++++++++++++++++++
 2 files changed, 112 insertions(+)
tools/perf/Documentation/perf-bench.txt +19 -0
index ca5789625cd2..8331bd28b10e 100644
@@ -124,6 +124,14 @@ Options of *pipe*
 --loop=::
 Specify number of loops.
 
+-G::
+--cgroups=::
+Names of cgroups for sender and receiver, separated by a comma.
+This is useful to check cgroup context switching overhead.
+Note that perf doesn't create nor delete the cgroups, so users should
+make sure that the cgroups exist and are accessible before use.
+
+
 Example of *pipe*
 ^^^^^^^^^^^^^^^^^
 
@@ -141,6 +149,17 @@ Example of *pipe*
         Total time:0.016 sec
                 16.948000 usecs/op
                 59004 ops/sec
+
+% perf bench sched pipe -G AAA,BBB
+(executing 1000000 pipe operations between cgroups)
+# Running 'sched/pipe' benchmark:
+# Executed 1000000 pipe operations between two processes
+
+     Total time: 6.886 [sec]
+
+       6.886208 usecs/op
+         145217 ops/sec
+
 ---------------------
 
 SUITES FOR 'syscall'
tools/perf/bench/sched-pipe.c +93 -0

132 lines not on this page · raw message

Signature · 1 line
2.42.0.609.gbb76f46606-goog
#1 lore raw

Em Wed, Oct 04, 2023 at 01:47:41PM -0700, Namhyung Kim escreveu:

Quoted 19 lines

The -G/--cgroups option is to put sender and receiver in different cgroups in order to measure cgroup context switch overheads.

Users need to make sure the cgroups exist and accessible.

  # perf stat -e context-switches,cgroup-switches \
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 > /dev/null

   Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000':

              20,001      context-switches
                   2      cgroup-switches

         0.053449651 seconds time elapsed

         0.011286000 seconds user
         0.041869000 seconds sys
  # perf stat -e context-switches,cgroup-switches \
Quoted 8 lines
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB > /dev/null

   Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB':

              20,001      context-switches
              20,001      cgroup-switches

         0.052768627 seconds time elapsed

So I tried with:

[root@quaco ~]# perf bench sched pipe -G system.slice,user.slice # Running 'sched/pipe' benchmark:
cannot enter to cgroup: system.slice
cannot enter to cgroup: user.slice
# Executed 1000000 pipe operations between two processes

     Total time: 6.301 [sec]

       6.301478 usecs/op
         158692 ops/sec

[root@quaco ~]#

Should't it bail out when not managing to enter the cgroups?

Also:

[root@quaco ~]# mkdir /sys/fs/cgroup/AAA [root@quaco ~]# mkdir /sys/fs/cgroup/BBB [root@quaco ~]#
[root@quaco ~]# perf bench sched pipe -G AAA,BBB # Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA
cannot enter to cgroup: BBB
# Executed 1000000 pipe operations between two processes

     Total time: 6.397 [sec]

       6.397830 usecs/op
         156302 ops/sec

[root@quaco ~]#

-rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.procs [root@quaco ~]# ls -la /sys/fs/cgroup/AAA/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.procs -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.procs [root@quaco ~]#

[root@quaco ~]# perf bench sched pipe -G AAA,BBB
# Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA (written=-1, len=6, errno=Operation not supported) cannot enter to cgroup: BBB (written=-1, len=6, errno=Operation not supported) # Executed 1000000 pipe operations between two processes

     Total time: 6.303 [sec]

       6.303221 usecs/op
         158649 ops/sec

[root@quaco ~]#

I'm certainly missing something here :-\

- Arnaldo

author maintainer
#2 lore raw

Hi Arnaldo,

Quoted 38 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 04, 2023 at 01:47:41PM -0700, Namhyung Kim escreveu:

Quoted 19 lines

The -G/--cgroups option is to put sender and receiver in different cgroups in order to measure cgroup context switch overheads.

Users need to make sure the cgroups exist and accessible.

  # perf stat -e context-switches,cgroup-switches \
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 > /dev/null

   Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000':

              20,001      context-switches
                   2      cgroup-switches

         0.053449651 seconds time elapsed

         0.011286000 seconds user
         0.041869000 seconds sys
  # perf stat -e context-switches,cgroup-switches \
Quoted 8 lines
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB > /dev/null

   Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB':

              20,001      context-switches
              20,001      cgroup-switches

         0.052768627 seconds time elapsed

So I tried with:

[root@quaco ~]# perf bench sched pipe -G system.slice,user.slice # Running 'sched/pipe' benchmark:
cannot enter to cgroup: system.slice
cannot enter to cgroup: user.slice
# Executed 1000000 pipe operations between two processes

Quoted 7 lines
     Total time: 6.301 [sec]

       6.301478 usecs/op
         158692 ops/sec

[root@quaco ~]#

Should't it bail out when not managing to enter the cgroups?

Hmm.. right. Will fix.

Quoted 33 lines

Also:

[root@quaco ~]# mkdir /sys/fs/cgroup/AAA [root@quaco ~]# mkdir /sys/fs/cgroup/BBB [root@quaco ~]#
[root@quaco ~]# perf bench sched pipe -G AAA,BBB # Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA
cannot enter to cgroup: BBB
# Executed 1000000 pipe operations between two processes

     Total time: 6.397 [sec]

       6.397830 usecs/op
         156302 ops/sec

[root@quaco ~]#

-rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.procs [root@quaco ~]# ls -la /sys/fs/cgroup/AAA/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.procs -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.procs [root@quaco ~]#

[root@quaco ~]# perf bench sched pipe -G AAA,BBB
# Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA (written=-1, len=6, errno=Operation not supported) cannot enter to cgroup: BBB (written=-1, len=6, errno=Operation not supported) # Executed 1000000 pipe operations between two processes

Quoted 7 lines
     Total time: 6.303 [sec]

       6.303221 usecs/op
         158649 ops/sec

[root@quaco ~]#

I'm certainly missing something here :-\

Try to enable some cgroup controllers first. Like

  # echo +cpu > /sys/fs/cgroup/AAA/cgroup.subtree_control
  # echo +cpu > /sys/fs/cgroup/BBB/cgroup.subtree_control

Thanks,
Namhyung

#3 lore raw

Em Fri, Oct 13, 2023 at 03:57:27PM -0700, Namhyung Kim escreveu:

Quoted 91 lines

Hi Arnaldo,

Quoted 38 lines · Arnaldo Carvalho de Melo

Em Wed, Oct 04, 2023 at 01:47:41PM -0700, Namhyung Kim escreveu:

Quoted 19 lines

The -G/--cgroups option is to put sender and receiver in different cgroups in order to measure cgroup context switch overheads.

Users need to make sure the cgroups exist and accessible.

  # perf stat -e context-switches,cgroup-switches \
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 > /dev/null

   Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000':

              20,001      context-switches
                   2      cgroup-switches

         0.053449651 seconds time elapsed

         0.011286000 seconds user
         0.041869000 seconds sys
  # perf stat -e context-switches,cgroup-switches \
Quoted 8 lines
Quoted 1 line

taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB > /dev/null

   Performance counter stats for 'taskset -c 0 perf bench sched pipe -l 10000 -G AAA,BBB':

              20,001      context-switches
              20,001      cgroup-switches

         0.052768627 seconds time elapsed

So I tried with:

[root@quaco ~]# perf bench sched pipe -G system.slice,user.slice # Running 'sched/pipe' benchmark:
cannot enter to cgroup: system.slice
cannot enter to cgroup: user.slice
# Executed 1000000 pipe operations between two processes

Quoted 7 lines
     Total time: 6.301 [sec]

       6.301478 usecs/op
         158692 ops/sec

[root@quaco ~]#

Should't it bail out when not managing to enter the cgroups?

Hmm.. right. Will fix.

Quoted 39 lines

Also:

[root@quaco ~]# mkdir /sys/fs/cgroup/AAA [root@quaco ~]# mkdir /sys/fs/cgroup/BBB [root@quaco ~]#
[root@quaco ~]# perf bench sched pipe -G AAA,BBB # Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA
cannot enter to cgroup: BBB
# Executed 1000000 pipe operations between two processes

     Total time: 6.397 [sec]

       6.397830 usecs/op
         156302 ops/sec

[root@quaco ~]#

-rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.procs [root@quaco ~]# ls -la /sys/fs/cgroup/AAA/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.procs -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.procs [root@quaco ~]#

[root@quaco ~]# perf bench sched pipe -G AAA,BBB
# Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA (written=-1, len=6, errno=Operation not supported) cannot enter to cgroup: BBB (written=-1, len=6, errno=Operation not supported) # Executed 1000000 pipe operations between two processes

     Total time: 6.303 [sec]

       6.303221 usecs/op
         158649 ops/sec

[root@quaco ~]#

Quoted 6 lines
Quoted 1 line

I'm certainly missing something here :-\

Try to enable some cgroup controllers first. Like

  # echo +cpu > /sys/fs/cgroup/AAA/cgroup.subtree_control
  # echo +cpu > /sys/fs/cgroup/BBB/cgroup.subtree_control

[root@quaco cgroup]# echo +cpu > /sys/fs/cgroup/AAA/cgroup.subtree_control [root@quaco cgroup]# echo +cpu > /sys/fs/cgroup/BBB/cgroup.subtree_control [root@quaco cgroup]# perf bench sched pipe -G AAA,BBB
# Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA (written=-1, len=6, errno=Operation not supported) cannot enter to cgroup: BBB (written=-1, len=6, errno=Operation not supported) # Executed 1000000 pipe operations between two processes

     Total time: 6.267 [sec]

       6.267680 usecs/op
         159548 ops/sec

[root@quaco cgroup]# cat /sys/fs/cgroup/AAA/cgroup.subtree_control
cpu
[root@quaco cgroup]# cat /sys/fs/cgroup/BBB/cgroup.subtree_control
cpu
[root@quaco cgroup]# uname -a
Linux quaco 6.5.6-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Oct 6 19:02:35 UTC 2023 x86_64 GNU/Linux [root@quaco cgroup]#

author maintainer
#4 lore raw
Quoted 46 lines · Arnaldo Carvalho de Melo

Em Fri, Oct 13, 2023 at 03:57:27PM -0700, Namhyung Kim escreveu:

Quoted 25 lines

Hi Arnaldo,

Quoted 21 lines · Arnaldo Carvalho de Melo

-rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.procs [root@quaco ~]# ls -la /sys/fs/cgroup/AAA/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/AAA/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.threads -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.threads [root@quaco ~]# ls -la /sys/fs/cgroup/BBB/cgroup.procs -rw-r--r--. 1 root root 0 Oct 13 19:22 /sys/fs/cgroup/BBB/cgroup.procs [root@quaco ~]#

[root@quaco ~]# perf bench sched pipe -G AAA,BBB
# Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA (written=-1, len=6, errno=Operation not supported) cannot enter to cgroup: BBB (written=-1, len=6, errno=Operation not supported) # Executed 1000000 pipe operations between two processes

     Total time: 6.303 [sec]

       6.303221 usecs/op
         158649 ops/sec

[root@quaco ~]#

Quoted 6 lines
Quoted 1 line

I'm certainly missing something here :-\

Try to enable some cgroup controllers first. Like

  # echo +cpu > /sys/fs/cgroup/AAA/cgroup.subtree_control
  # echo +cpu > /sys/fs/cgroup/BBB/cgroup.subtree_control

[root@quaco cgroup]# echo +cpu > /sys/fs/cgroup/AAA/cgroup.subtree_control [root@quaco cgroup]# echo +cpu > /sys/fs/cgroup/BBB/cgroup.subtree_control [root@quaco cgroup]# perf bench sched pipe -G AAA,BBB
# Running 'sched/pipe' benchmark:
cannot enter to cgroup: AAA (written=-1, len=6, errno=Operation not supported) cannot enter to cgroup: BBB (written=-1, len=6, errno=Operation not supported) # Executed 1000000 pipe operations between two processes

     Total time: 6.267 [sec]

       6.267680 usecs/op
         159548 ops/sec
Quoted 7 lines

[root@quaco cgroup]# cat /sys/fs/cgroup/AAA/cgroup.subtree_control
cpu
[root@quaco cgroup]# cat /sys/fs/cgroup/BBB/cgroup.subtree_control
cpu
[root@quaco cgroup]# uname -a
Linux quaco 6.5.6-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Oct 6 19:02:35 UTC 2023 x86_64 GNU/Linux [root@quaco cgroup]#

Hmm... I don't know, I've tested it on v1 only ;-p I'll take a look.

Also I found a bug, will fix too.

Thanks,
Namhyung

tools/perf/bench/sched-pipe.c +2 -2
index 1103fd658d5d..25236f339b90 100644
@@ -118,9 +118,9 @@ static void enter_cgroup(struct cgroup *cgrp)
 
        /* try cgroup v2 interface first */
        if (threaded)
-               fd = openat(cgrp->fd, "cgroup.procs", O_WRONLY);
-       else
                fd = openat(cgrp->fd, "cgroup.threads", O_WRONLY);
+       else
+               fd = openat(cgrp->fd, "cgroup.progs", O_WRONLY);
 
        /* try cgroup v1 if failed */
        if (fd < 0)
author maintainer
#5 lore raw
Quoted 6 lines · Namhyung Kim
tools/perf/bench/sched-pipe.c +0 -0
index 1103fd658d5d..25236f339b90 100644
@@ -118,9 +118,9 @@ static void enter_cgroup(struct cgroup *cgrp)
 

tools/perf/bench/sched-pipe.c static void enter_cgroup(struct cgroup *cgrp) · line 126

Quoted 10 lines
tools/perf/bench/sched-pipe.c +2 -2
        /* try cgroup v2 interface first */
        if (threaded)
-               fd = openat(cgrp->fd, "cgroup.procs", O_WRONLY);
-       else
                fd = openat(cgrp->fd, "cgroup.threads", O_WRONLY);
+       else
+               fd = openat(cgrp->fd, "cgroup.progs", O_WRONLY);
 
        /* try cgroup v1 if failed */
        if (fd < 0)

Oh, actually it should be 'procs' instead of 'progs'.

Thanks,
Namhyung