perf_event: Fix oops triggered by cpu offline/online
PATCH 1/1 perf_event: Fix oops triggered by cpu offline/online
Anton Blanchard found that he could reliably make the kernel hit a BUG_ON in the slab allocator by taking a cpu offline and then online while a system-wide perf record session was running.
The reason is that when the cpu comes up, we completely reinitialize the ctx field of the struct perf_cpu_context for the cpu. If there is a system-wide perf record session running, then there will be a struct perf_event that has a reference to the context, so its refcount will be 2. (The perf_event has been removed from the context's group_entry and event_entry lists by perf_event_exit_cpu(), but that doesn't remove the perf_event's reference to the context and doesn't decrement the context's refcount.)
When the cpu comes up, perf_event_init_cpu() gets called, and it calls __perf_event_init_context() on the cpu's context. That resets the refcount to 1. Then when the perf record session finishes and the perf_event is closed, the refcount gets decremented to 0 and the context gets kfreed after an RCU grace period. Since the context wasn't kmalloced -- it's part of a per-cpu variable -- bad things happen.
In fact we don't need to completely reinitialize the context when the cpu comes up. It's sufficient to initialize the context once at boot, but we need to do it for all possible cpus.
This moves the context initialization to happen at boot time. With this, we don't trash the refcount and the context never gets kfreed, and we don't hit the BUG_ON.
---
This bug appears to go back to May 2009, so this patch should get backported to 2.6.31, .32 and .33.
kernel/perf_event.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
kernel/perf_event.c +12 -1
@@ -5392,12 +5392,22 @@ int perf_event_init_task(struct task_struct *child)return ret; } +static void __init perf_event_init_all_cpus (void ) +{ +int cpu; +struct perf_cpu_context *cpuctx; + +for_each_possible_cpu (cpu) { + cpuctx = &per_cpu (perf_cpu_context, cpu); +__perf_event_init_context (&cpuctx->ctx,NULL ); + } +} +static void __cpuinit perf_event_init_cpu (int cpu) {struct perf_cpu_context *cpuctx; cpuctx = &per_cpu (perf_cpu_context, cpu); -__perf_event_init_context (&cpuctx->ctx,NULL );spin_lock (&perf_resource_lock); cpuctx->max_pertask = perf_max_events - perf_reserved_percpu;
@@ -5474,6 +5484,7 @@ static struct notifier_block __cpuinitdata perf_cpu_nb = {void __init perf_event_init (void ) { +perf_event_init_all_cpus ();perf_cpu_notify (&perf_cpu_nb, (unsigned long )CPU_UP_PREPARE , (void *)(long )smp_processor_id ()); perf_cpu_notify(&perf_cpu_nb, (unsigned long)CPU_ONLINE,
Quoted 25 lines · Paul Mackerras
Anton Blanchard found that he could reliably make the kernel hit a
BUG_ONin the slab allocator by taking a cpu offline and then online while a system-wide perf record session was running.The reason is that when the cpu comes up, we completely reinitialize the ctx field of the struct
perf_cpu_contextfor the cpu. If there is a system-wide perf record session running, then there will be a structperf_eventthat has a reference to the context, so its refcount will be 2. (Theperf_eventhas been removed from the context'sgroup_entryandevent_entrylists byperf_event_exit_cpu(), but that doesn't remove the perf_event's reference to the context and doesn't decrement the context's refcount.)When the cpu comes up,
perf_event_init_cpu()gets called, and it calls__perf_event_init_context()on the cpu's context. That resets the refcount to 1. Then when the perf record session finishes and theperf_eventis closed, the refcount gets decremented to 0 and the context gets kfreed after an RCU grace period. Since the context wasn't kmalloced -- it's part of a per-cpu variable -- bad things happen.In fact we don't need to completely reinitialize the context when the cpu comes up. It's sufficient to initialize the context once at boot, but we need to do it for all possible cpus.
Quoted 7 lines
This moves the context initialization to happen at boot time. With this, we don't trash the refcount and the context never gets kfreed, and we don't hit the
BUG_ON.Reported-by Anton Blanchard <[email protected]>Signed-off-by Paul Mackerras <[email protected]>Tested-by Anton Blanchard <[email protected]>
Very nice catch!