perf/core: only check cpuctx with cgroup events during cgroup switch
PATCH 1/1 perf/core: only check cpuctx with cgroup events during cgroup switch
Following conversation in CQM/CMT last series about speeding up the context switch for cgroup events:
https://patchwork.kernel.org/patch/9478617/
This is a low-hanging fruit optimization. It replaces the iteration over the "pmus" list in cgroup switch by an iteration over a new list that contains only cpuctxs with at least one cgroup event.
The iteration over pmus has become increasingly costly as the number of pmus have increased. It is also heavily affected by cache contention.
Below some instrumentation measurements with 10, 50 and 90 percentiles of the total cost of context switch at distinct levels of cache contenton for a simple array read/write microbenchark.
Contention
Level Nr events Before (us) After (us) Median
L2 L3 types (10%, 50%, 90%) (10%, 50%, 90% Speedup
--------------------------------------------------------------------------
Low Low 1 (1.72, 2.42, 5.85) (1.35, 1.64, 5.46) 29% High Low 1 (2.08, 4.56, 19.8) (1720, 2.20, 13.7) 51% High High 1 (2.86, 10.4, 12.7) (2.54, 4.32, 12.1) 58%
Low Low 2 (1.98, 3.20, 6.89) (1.68, 2.41, 8.89) 24% High Low 2 (2.48, 5.28, 22.4) (2150, 3.69, 14.6) 30% High High 2 (3.32, 8.09, 13.9) (2.80, 5.15, 13.7) 36%
where:
1 event type = cycles
2 event types = cycles,intel_cqm/llc_occupancy/
Contetion L2 Low: workset < L2 cache size.
High: " >> L2 " " .
Contetion L3 Low: workset of task on all sockets < L3 cache size.
High: " " " " " " >> L3 " " .
Median Speed up is (50%ile Before - 50%ile After) / 50%ile Before
--- include/linux/perf_event.h | 1 + kernel/events/core.c | 96 +++++++++++++++++++++------------------------- 2 files changed, 44 insertions(+), 53 deletions(-)
include/linux/perf_event.h +1 -0
@@ -788,6 +788,7 @@ struct perf_cpu_context {struct pmu *unique_pmu;#ifdef CONFIG_CGROUP_PERF struct perf_cgroup *cgrp; +struct list_head cgrp_cpuctx_entry;#endif struct list_head sched_cb_entry;
kernel/events/core.c +43 -53
137 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
kbuild test robot
build boturl: https://github.com/0day-ci/linux/commits/David-Carrillo-Cisneros/perf-core-only-check-cpuctx-with-cgroup-events-during-cgroup-switch/20161231-103433 config: i386-randconfig-x005-201652…
Hi David,
[auto build test ERROR on tip/perf/core]
[also build test ERROR on v4.10-rc1 next-20161224]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/David-Carrillo-Cisneros/perf-core-only-check-cpuctx-with-cgroup-events-during-cgroup-switch/20161231-103433
config: i386-randconfig-x005-201652 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901 reproduce:
# save the attached .config to linux build tree
make ARCH=i386 All error/warnings (new ones prefixed by >>):
In file included from include/asm-generic/percpu.h:6:0,
from arch/x86/include/asm/percpu.h:542,
from arch/x86/include/asm/preempt.h:5,
from include/linux/preempt.h:59,
from include/linux/spinlock.h:50,
from include/linux/wait.h:8,
from include/linux/fs.h:5,
from kernel/events/core.c:12:
kernel/events/core.c: In function 'perf_event_init_all_cpus':
>> kernel/events/core.c:10587:27: error: 'cgrp_cpuctx_list' undeclared (first use in this function)
INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu));
^
include/linux/percpu-defs.h:206:47: note: in definition of macro '__verify_pcpu_ptr'
const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL; \
^~~
include/linux/percpu-defs.h:250:47: note: in expansion of macro 'VERIFY_PERCPU_PTR'
#define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR(ptr); })
^~~~~~~~~~~~~~~~~
include/linux/percpu-defs.h:256:29: note: in expansion of macro 'per_cpu_ptr'
#define per_cpu(var, cpu) (*per_cpu_ptr(&(var), cpu))
^~~~~~~~~~~
>> kernel/events/core.c:10587:19: note: in expansion of macro 'per_cpu'
INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu));
^~~~~~~
kernel/events/core.c:10587:27: note: each undeclared identifier is reported only once for each function it appears in
INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu));
^
include/linux/percpu-defs.h:206:47: note: in definition of macro '__verify_pcpu_ptr'
const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL; \
^~~
include/linux/percpu-defs.h:250:47: note: in expansion of macro 'VERIFY_PERCPU_PTR'
#define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR(ptr); })
^~~~~~~~~~~~~~~~~
include/linux/percpu-defs.h:256:29: note: in expansion of macro 'per_cpu_ptr'
#define per_cpu(var, cpu) (*per_cpu_ptr(&(var), cpu))
^~~~~~~~~~~
>> kernel/events/core.c:10587:19: note: in expansion of macro 'per_cpu'
INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu));
^~~~~~~vim +/cgrp_cpuctx_list +10587 kernel/events/core.c
10581 mutex_init(&swhash->hlist_mutex);
10582 INIT_LIST_HEAD(&per_cpu(active_ctx_list, cpu));
10583
10584 INIT_LIST_HEAD(&per_cpu(pmu_sb_events.list, cpu));
10585 raw_spin_lock_init(&per_cpu(pmu_sb_events.lock, cpu));
10586
10587 INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu));
10588 INIT_LIST_HEAD(&per_cpu(sched_cb_list, cpu));
10589 }
10590 }
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
kbuild test robot
build boturl: https://github.com/0day-ci/linux/commits/David-Carrillo-Cisneros/perf-core-only-check-cpuctx-with-cgroup-events-during-cgroup-switch/20161231-103433 config: x86_64-randconfig-s1-12311108…
Hi David,
[auto build test ERROR on tip/perf/core]
[also build test ERROR on v4.10-rc1 next-20161224]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/David-Carrillo-Cisneros/perf-core-only-check-cpuctx-with-cgroup-events-during-cgroup-switch/20161231-103433
config: x86_64-randconfig-s1-12311108 (attached as .config)
compiler: gcc-4.4 (Debian 4.4.7-8) 4.4.7 reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64 All error/warnings (new ones prefixed by >>):
kernel/events/core.c: In function 'perf_event_init_all_cpus':
kernel/events/core.c:10587: error: 'cgrp_cpuctx_list' undeclared (first use in this function) kernel/events/core.c:10587: error: (Each undeclared identifier is reported only once kernel/events/core.c:10587: error: for each function it appears in.)
>> kernel/events/core.c:10587: error: type defaults to 'int' in declaration of 'type name' >> kernel/events/core.c:10587: warning: cast from pointer to integer of different size >> kernel/events/core.c:10587: error: type defaults to 'int' in declaration of 'type name'
vim +10587 kernel/events/core.c
10581 mutex_init(&swhash->hlist_mutex);
10582 INIT_LIST_HEAD(&per_cpu(active_ctx_list, cpu));
10583
10584 INIT_LIST_HEAD(&per_cpu(pmu_sb_events.list, cpu));
10585 raw_spin_lock_init(&per_cpu(pmu_sb_events.lock, cpu));
10586
10587 INIT_LIST_HEAD(&per_cpu(cgrp_cpuctx_list, cpu));
10588 INIT_LIST_HEAD(&per_cpu(sched_cb_list, cpu));
10589 }
10590 }
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation