perf_event: fix a cgroup switch warning
Subsystems (2)
RFC PATCH 1/1 perf_event: fix a cgroup switch warning
We have been consistently triggering the warning WARN_ON_ONCE(cpuctx->cgrp) in perf_cgroup_switch() for a rather long time, although we still have no clue on how to reproduce it.
Looking into the code, it seems the only possibility here is that the process calling perf_event_open() with a cgroup target exits before the process in the target cgroup exits but after it gains CPU to run. This is because we use the atomic counter perf_cgroup_events as an indication of whether cgroup perf event has enabled or not, which is inaccurate, illustrated as below:
CPU 0 CPU 1
// open perf events with a cgroup
// target for all CPU'sperf_event_open():
account_event_cpu()
// perf_cgroup_events == 1
// Schedule in a process in the target cgroup perf_cgroup_switch() perf_event_release_kernel(): unaccount_event_cpu() // perf_cgroup_events == 0 // schedule out // but perf_cgroup_sched_out() is skipped // cpuctx->cgrp left as non-NULL // schedule in another process perf_cgroup_switch() // WARN triggerred
The proposed fix is kinda ugly, as it adds a flag in each process to indicate whether this process has to go through perf_cgroup_sched_out() when perf_cgroup_events is false negative. The other possible fix is to force a reschedule on each target CPU before decreasing the counter perf_cgroup_events, but this is expensive.
Suggestions? Thoughts?
--- include/linux/sched.h | 3 +++ kernel/events/core.c | 5 ++++- 2 files changed, 7 insertions(+), 1 deletion(-)
include/linux/sched.h +3 -0
@@ -733,6 +733,9 @@ struct task_struct {/* to be used once the psi infrastructure lands upstream. */ unsigned use_memdelay:1;#endif +#ifdef CONFIG_PERF_EVENTS + unsigned perf_cgroup_sched_in:1; +#endif unsigned long atomic_flags;/* Flags requiring atomic access. */
kernel/events/core.c +4 -1
@@ -817,6 +817,7 @@ static void perf_cgroup_switch(struct task_struct *task, int mode)* to event_filter_match() in event_sched_out() */ cpuctx->cgrp =NULL ; + task->perf_cgroup_sched_in =0 ; }if (mode &PERF_CGROUP_SWIN ) {
@@ -831,6 +832,7 @@ static void perf_cgroup_switch(struct task_struct *task, int mode) cpuctx->cgrp =perf_cgroup_from_task (task, &cpuctx->ctx);cpu_ctx_sched_in (cpuctx,EVENT_ALL , task); + task->perf_cgroup_sched_in =1 ; }perf_pmu_enable (cpuctx->ctx.pmu);perf_ctx_unlock (cpuctx, cpuctx->task_ctx);
@@ -3233,7 +3235,8 @@ void __perf_event_task_sched_out(struct task_struct *task,* to check if we have to switch out PMU state. * cgroup event are system-wide mode only */ -if (atomic_read (this_cpu_ptr (&perf_cgroup_events))) +if (atomic_read (this_cpu_ptr (&perf_cgroup_events)) || + task->perf_cgroup_sched_in)perf_cgroup_sched_out (task, next); }
Signature · 1 line
2.21.0
Peter Zijlstra
maintainerQuoted 17 lines · Cong Wang
We have been consistently triggering the warning
WARN_ON_ONCE(cpuctx->cgrp)inperf_cgroup_switch()for a rather long time, although we still have no clue on how to reproduce it.Looking into the code, it seems the only possibility here is that the process calling
perf_event_open()with a cgroup target exits before the process in the target cgroup exits but after it gains CPU to run. This is because we use the atomic counterperf_cgroup_eventsas an indication of whether cgroup perf event has enabled or not, which is inaccurate, illustrated as below:CPU 0 CPU 1
// open perf events with a cgroup
// target for all CPU'sperf_event_open():
account_event_cpu()
//perf_cgroup_events== 1
Quoted 8 lines
// Schedule in a process in the target cgroup perf_cgroup_switch() perf_event_release_kernel(): unaccount_event_cpu() // perf_cgroup_events == 0 // schedule out // but perf_cgroup_sched_out() is skipped // cpuctx->cgrp left as non-NULL
which implies we observed: 'perf_cgroup_events == 0'
Quoted 2 lines
// schedule in another process perf_cgroup_switch() // WARN triggerred
which implies we observed: 'perf_cgroup_events == 1'
Which is impossible. It _might_ have been possible if the out and in happened on different CPUs. But then I'm not sure that is enough to trigger the problem.
Quoted 1 line
The proposed fix is kinda ugly,
Yes :-)
Quoted 1 line
Suggestions? Thoughts?
At perf_event_release time, when it is the last cgroup event, there should not be any cgroup events running anymore, so ideally perf_cgroup_switch() would not set state.
Furthermore; list_update_cgroup_event() will actually clear cpuctx->cgrp on removal of the last cgroup event.
Also; perf_cgroup_switch() will WARN when there are not in fact any cgroup events at all. I would expect that WARN to trigger too in your scneario. But you're not seeing that?
I do however note that that check seems racy; we do that without holding the ctx_lock.
Quoted 34 lines · Peter Zijlstra
Quoted 17 lines · Cong Wang
We have been consistently triggering the warning
WARN_ON_ONCE(cpuctx->cgrp)inperf_cgroup_switch()for a rather long time, although we still have no clue on how to reproduce it.Looking into the code, it seems the only possibility here is that the process calling
perf_event_open()with a cgroup target exits before the process in the target cgroup exits but after it gains CPU to run. This is because we use the atomic counterperf_cgroup_eventsas an indication of whether cgroup perf event has enabled or not, which is inaccurate, illustrated as below:CPU 0 CPU 1
// open perf events with a cgroup
// target for all CPU'sperf_event_open():
account_event_cpu()
//perf_cgroup_events== 1Quoted 8 lines
// Schedule in a process in the target cgroup perf_cgroup_switch() perf_event_release_kernel(): unaccount_event_cpu() // perf_cgroup_events == 0 // schedule out // but perf_cgroup_sched_out() is skipped // cpuctx->cgrp left as non-NULLwhich implies we observed:
'perf_cgroup_events== 0'Quoted 2 lines
// schedule in another process
perf_cgroup_switch()// WARN triggerred
Quoted 7 lines
which implies we observed:
'perf_cgroup_events== 1'Which is impossible. It _might_ have been possible if the out and in happened on different CPUs. But then I'm not sure that is enough to trigger the problem.
Good catch, but this just needs one more perf_event_open(), right? :)
Quoted 5 lines
Quoted 2 lines
The proposed fix is kinda ugly,
Yes :-)
Quoted 8 lines
Quoted 1 line
Suggestions? Thoughts?
At
perf_event_releasetime, when it is the last cgroup event, there should not be any cgroup events running anymore, so ideallyperf_cgroup_switch()would not set state.Furthermore;
list_update_cgroup_event()will actually clearcpuctx->cgrpon removal of the last cgroup event.
Ah, yes, this probably explains why it is harder to trigger than I expected.
Quoted 4 lines
Also;
perf_cgroup_switch()will WARN when there are not in fact any cgroup events at all. I would expect that WARN to trigger too in your scneario. But you're not seeing that?
Not sure if I follow you, but if there is no cgroup event, cgrp_cpuctx_list should be empty, right?
From the stack traces I can't tell, what I can tell is that we use cgroup events in most cases.
Quoted 3 lines
I do however note that that check seems racy; we do that without holding the
ctx_lock.
Hmm? perf_ctx_lock() is taken in perf_cgroup_switch(), so I think locking is fine.
Thanks.