Skip to contentSkip to conversation

perf_counter: Fix race in attaching counters to tasks

RFC PATCH Superseded Paul Mackerras 2 participants 3 messages May 26, 2009 – May 27, 2009

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 perf_counter: Fix race in attaching counters to tasks

Commit 564c2b21 ("perf_counter: Optimize context switch between identical inherited contexts") introduced a race where it is possible that a counter being attached to a task could get attached to the wrong task, if the task is one that has inherited its context from another task via fork. This happens because the optimized context switch could switch the context to another task after find_get_context has read task->perf_counter_ctxp. In fact, it's possible that the context could then get freed, if the other task then exits.

This fixes the problem by protecting both the context switch and the critical code in find_get_context with a spinlock. We use the ctx->lock of the parent context for this because it will be common between any pair of contexts that might get swapped. Thus perf_counter_task_sched_out only needs to take one lock to exclude find_get_context from getting the wrong context for either the old task or the new task.

To make sure that none of the contexts being looked at in find_get_context can get freed, this changes the context freeing code to use RCU. Thus an rcu_read_lock() is sufficient to ensure that no contexts can get freed. This part of the patch is lifted from a patch posted by Peter Zijlstra.

This also adds a check to make sure that we can't add a counter to a task that is exiting. This solves a race between perf_counter_exit_task and find_get_context; it ensures that find_get_context can't attach a new context to a task after perf_counter_exit_task has disposed of the task's context.

With this, we are now doing the unclone in find_get_context rather than when a counter was added to or removed from a context (actually, we were missing the unclone_ctx() call when adding a counter to a context). We don't need to unclone when removing a counter from a context because we have no way to remove a counter from a cloned context.

This also takes out the smp_wmb() in find_get_context, which Peter Zijlstra pointed out was unnecessary because the cmpxchg implies a full barrier anyway.

---

include/linux/perf_counter.h +3 -2
index 2b16ed3..35dc996 100644
@@ -541,8 +541,9 @@ struct perf_counter_context {
 	 * been cloned (inherited) from a common ancestor.
 	 */
 	struct perf_counter_context *parent_ctx;
-	u32			parent_gen;
-	u32			generation;
+	u64			parent_gen;
+	u64			generation;
+	struct rcu_head		rcu_head;
 };
 
 /**
kernel/perf_counter.c +95 -39

245 lines not on this page · raw message

#1 lore raw
Quoted 9 lines · Paul Mackerras

Commit 564c2b21 ("perf_counter: Optimize context switch between identical inherited contexts") introduced a race where it is possible that a counter being attached to a task could get attached to the wrong task, if the task is one that has inherited its context from another task via fork. This happens because the optimized context switch could switch the context to another task after find_get_context has read task->perf_counter_ctxp. In fact, it's possible that the context could then get freed, if the other task then exits.

Quoted 7 lines

This fixes the problem by protecting both the context switch and the critical code in find_get_context with a spinlock. We use the ctx->lock of the parent context for this because it will be common between any pair of contexts that might get swapped. Thus perf_counter_task_sched_out only needs to take one lock to exclude find_get_context from getting the wrong context for either the old task or the new task.

I'm not sure this is the better approach (as opposed to your previous patch that used a per counter/task lock), since the parent context lock will serialize the full process tree over all cpus, whereas the per counter/task lock will be lock to the current cpu.

Quoted 11 lines

To make sure that none of the contexts being looked at in find_get_context can get freed, this changes the context freeing code to use RCU. Thus an rcu_read_lock() is sufficient to ensure that no contexts can get freed. This part of the patch is lifted from a patch posted by Peter Zijlstra.

This also adds a check to make sure that we can't add a counter to a task that is exiting. This solves a race between perf_counter_exit_task and find_get_context; it ensures that find_get_context can't attach a new context to a task after perf_counter_exit_task has disposed of the task's context.

Right, this leaves us with the flush_old_exec() race. I tried solving that by dropping the ctx reference count to 0 before iterating the ctx counters and destroying them vs atomic_inc_not_zero() reference acquire in find_get_context().

Quoted 4 lines

With this, we are now doing the unclone in find_get_context rather than when a counter was added to or removed from a context (actually, we were missing the unclone_ctx() call when adding a counter to a context).

We destroy the parent_ctx pointer under the lock, but suppose we're trying to attach to the parent context itself, doesn't that mean we need to increment the generatoin count under the lock as well?

Quoted 3 lines

We don't need to unclone when removing a counter from a
context because we have no way to remove a counter from a cloned context.

Not without removing it from the whole hierarchy indeed.

kernel/perf_counter.c static void __perf_counter_remove_from_context(void *info) · line 280

Quoted 13 lines

This also takes out the smp_wmb() in find_get_context, which Peter Zijlstra pointed out was unnecessary because the cmpxchg implies a full barrier anyway.

kernel/perf_counter.c +4 -0
@@ -281,13 +273,16 @@ static void __perf_counter_remove_from_context(void *info)
  *
  * CPU counters are removed with a smp call. For task counters we only
  * call when the task is on a CPU.
+ *
+ * If counter->ctx is a cloned context, callers must make sure that
+ * every task struct that counter->ctx->task could possibly point to
+ * remains valid.
  */

This comment (and its replicas) confuse me in that they only require something but then don't expand on how this is accomplished.

Quoted 20 lines
kernel/perf_counter.c +6 -1
@@ -1437,6 +1491,12 @@ static void perf_counter_for_each_sibling(struct perf_counter *counter,
 	mutex_unlock(&ctx->mutex);
 }
 
+/*
+ * Holding the top-level counter's child_mutex means that any
+ * descendant process that has inherited this counter will block
+ * in sync_child_counter if it goes to exit, thus satisfying the
+ * task existence requirements of perf_counter_enable/disable.
+ */
 static void perf_counter_for_each_child(struct perf_counter *counter,
 					void (*func)(struct perf_counter *))
 {
@@ -3449,17 +3509,13 @@ void perf_counter_exit_task(struct task_struct *child)
 {
 	struct perf_counter *child_counter, *tmp;
 	struct perf_counter_context *child_ctx;
-	unsigned long flags;
 
 	child_ctx = child->perf_counter_ctxp;

kernel/perf_counter.c void perf_counter_exit_task(struct task_struct *child) · line 3520

Quoted 10 lines
kernel/perf_counter.c +0 -3
-
 	if (likely(!child_ctx))
 		return;
 
-	local_irq_save(flags);
 	__perf_counter_task_sched_out(child_ctx);
 	child->perf_counter_ctxp = NULL;
-	local_irq_restore(flags);
 
 	mutex_lock(&child_ctx->mutex);

This change looks wrong, __perf_counter_task_sched_out() -> __perf_counter_sched_out() -> spin_lock(&ctx->lock) wants IRQs disabled..

#2 lore raw
Quoted 4 lines · Peter Zijlstra

I'm not sure this is the better approach (as opposed to your previous patch that used a per counter/task lock), since the parent context lock will serialize the full process tree over all cpus, whereas the per counter/task lock will be lock to the current cpu.

OK, see the patch below, which goes back to the 2-locks approach, except now they are just the ctx->lock of each context rather than being an extra lock in the task_struct. We can do that because we can use RCU to guarantee the existence of the context in find_get_context.

Quoted 4 lines

Right, this leaves us with the flush_old_exec() race. I tried solving that by dropping the ctx reference count to 0 before iterating the ctx counters and destroying them vs atomic_inc_not_zero() reference acquire in find_get_context().

I think I have solved that race now; the key observation being that it doesn't matter if a context has some top-level (non-inherited) counters in it after perf_counter_exit_task has finished, or if top-level counters are added. They won't do anything and they'll get removed when their fd is closed.

Quoted 3 lines

We destroy the parent_ctx pointer under the lock, but suppose we're trying to attach to the parent context itself, doesn't that mean we need to increment the generatoin count under the lock as well?

OK, we now increment it under the lock.

Quoted 5 lines
Quoted 3 lines

We don't need to unclone when removing a counter from a
context because we have no way to remove a counter from a cloned context.

Not without removing it from the whole hierarchy indeed.

Well, I can't even see any way to remove it from the hierarchy. You can close the fd for the top-level counter but we won't get perf_release called until all the fput's have been done, which only happens as descendant processes exit.

Quoted 8 lines
Quoted 5 lines

+ *
+ * If counter->ctx is a cloned context, callers must make sure that
+ * every task struct that counter->ctx->task could possibly point to
+ * remains valid.
*/

This comment (and its replicas) confuse me in that they only require something but then don't expand on how this is accomplished.

OK, I added some stuff to explain that.

Quoted 10 lines

Review

Quoted 6 lines
(quoted diff) +0 -2
-	local_irq_save(flags);
 	__perf_counter_task_sched_out(child_ctx);
 	child->perf_counter_ctxp = NULL;
-	local_irq_restore(flags);
 
 	mutex_lock(&child_ctx->mutex);

This change looks wrong, __perf_counter_task_sched_out() -> __perf_counter_sched_out() -> spin_lock(&ctx->lock) wants IRQs disabled..

Right, I took that change out.

If this all looks OK to you, I'll submit it properly with a patch description and signoff...

Paul.

include/linux/perf_counter.h +3 -2
index 2b16ed3..35dc996 100644
@@ -541,8 +541,9 @@ struct perf_counter_context {
 	 * been cloned (inherited) from a common ancestor.
 	 */
 	struct perf_counter_context *parent_ctx;
-	u32			parent_gen;
-	u32			generation;
+	u64			parent_gen;
+	u64			generation;
+	struct rcu_head		rcu_head;
 };
 
 /**
kernel/perf_counter.c +143 -60

353 lines not on this page · raw message