Skip to contentSkip to conversation

sched: Fix PROVE_RCU vs cpu_cgroup

tip:sched/core tip-bot for Peter Zijlstra 1 participant 1 message Jun 8, 2010 cgroup

lore
Share & tools
lore
#0 lore raw
Commit-ID:  dc61b1d65e353d638b2445f71fb8e5b5630f2415
Gitweb:     http://git.kernel.org/tip/dc61b1d65e353d638b2445f71fb8e5b5630f2415
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Tue, 8 Jun 2010 11:40:42 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 8 Jun 2010 18:44:04 +0200

sched: Fix PROVE_RCU vs cpu_cgroup

PROVE_RCU has a few issues with the cpu_cgroup because the scheduler typically holds rq->lock around the css rcu derefs but the generic cgroup code doesn't (and can't) know about that lock.

Provide means to add extra checks to the css dereference and use that in the scheduler to annotate its users.

The addition of rq->lock to these checks is correct because the cgroup_subsys::attach() method takes the rq->lock for each task it moves, therefore by holding that lock, we ensure the task is pinned to the current cgroup and the RCU derefence is valid.

That leaves one genuine race in __sched_setscheduler() where we used task_group() without holding any of the required locks and thus raced with the cgroup code. Solve this by moving the check under the appropriate lock.

Cc "Paul E. McKenney" <[email protected]>

LKML-Reference: <new-submission>

---
 include/linux/cgroup.h |   20 ++++++---
 kernel/sched.c         |  115 ++++++++++++++++++++++++-----------------------
 2 files changed, 73 insertions(+), 62 deletions(-)
include/linux/cgroup.h +14 -6
index 0c62160..e3d00fd 100644
@@ -525,13 +525,21 @@ static inline struct cgroup_subsys_state *cgroup_subsys_state(
 	return cgrp->subsys[subsys_id];
 }
 
-static inline struct cgroup_subsys_state *task_subsys_state(
-	struct task_struct *task, int subsys_id)
+/*
+ * function to get the cgroup_subsys_state which allows for extra
+ * rcu_dereference_check() conditions, such as locks used during the
+ * cgroup_subsys::attach() methods.
+ */
+#define task_subsys_state_check(task, subsys_id, __c)			\
+	rcu_dereference_check(task->cgroups->subsys[subsys_id],		\
+			      rcu_read_lock_held() ||			\
+			      lockdep_is_held(&task->alloc_lock) ||	\
+			      cgroup_lock_is_held() || (__c))
+
+static inline struct cgroup_subsys_state *
+task_subsys_state(struct task_struct *task, int subsys_id)
 {
-	return rcu_dereference_check(task->cgroups->subsys[subsys_id],
-				     rcu_read_lock_held() ||
-				     lockdep_is_held(&task->alloc_lock) ||
-				     cgroup_lock_is_held());
+	return task_subsys_state_check(task, subsys_id, false);
 }
 
 static inline struct cgroup* task_cgroup(struct task_struct *task,
kernel/sched.c +59 -56

139 lines not on this page · raw message