Skip to contentSkip to conversation

Cpuset: oops in exit on null cpuset fix

PATCH In review Paul Jackson 2 participants 3 messages Feb 15, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Cpuset: oops in exit on null cpuset fix

From: Paul Jackson <[email protected]>

Fix a latent bug in cpuset_exit() handling. If a task tried to allocate memory after calling cpuset_exit(), it oops'd in cpuset_update_task_memory_state() on a NULL cpuset pointer.

So set the exiting tasks cpuset to the root cpuset instead of to NULL.

A distro kernel hit this with an added kernel package that had just such a hook (allocating memory) in the exit code path.

---

 kernel/cpuset.c |   35 ++++++++++++++++++++++++++++++++++-
 1 files changed, 34 insertions(+), 1 deletion(-)
kernel/cpuset.c +34 -1
@@ -2022,6 +2022,39 @@ void cpuset_fork(struct task_struct *chi
  * We don't need to task_lock() this reference to tsk->cpuset,
  * because tsk is already marked PF_EXITING, so attach_task() won't
  * mess with it, or task is a failed fork, never visible to attach_task.
+ *
+ * Hack:
+ *
+ *    Set the exiting tasks cpuset to the root cpuset (top_cpuset).
+ *
+ *    Don't leave a task unable to allocate memory, as that is an
+ *    accident waiting to happen should someone add a callout in
+ *    do_exit() after the cpuset_exit() call that might allocate.
+ *    If a task tries to allocate memory with an invalid cpuset,
+ *    it will oops in cpuset_update_task_memory_state().
+ *
+ *    We call cpuset_exit() while the task is still competent to
+ *    handle notify_on_release(), then leave the task attached to
+ *    the root cpuset (top_cpuset) for the remainder of its exit.
+ *
+ *    To do this properly, we would increment the reference count on
+ *    top_cpuset, and near the very end of the kernel/exit.c do_exit()
+ *    code we would add a second cpuset function call, to drop that
+ *    reference.  This would just create an unnecessary hot spot on
+ *    the top_cpuset reference count, to no avail.
+ *
+ *    Normally, holding a reference to a cpuset without bumping its
+ *    count is unsafe.   The cpuset could go away, or someone could
+ *    attach us to a different cpuset, decrementing the count on
+ *    the first cpuset that we never incremented.  But in this case,
+ *    top_cpuset isn't going away, and either task has PF_EXITING set,
+ *    which wards off any attach_task() attempts, or task is a failed
+ *    fork, never visible to attach_task.
+ *
+ *    Another way to do this would be to set the cpuset pointer
+ *    to NULL here, and check in cpuset_update_task_memory_state()
+ *    for a NULL pointer.  This hack avoids that NULL check, for no
+ *    cost (other than this way too long comment ;).
  **/
 
 void cpuset_exit(struct task_struct *tsk)
@@ -2029,7 +2062,7 @@ void cpuset_exit(struct task_struct *tsk
 	struct cpuset *cs;
 
 	cs = tsk->cpuset;
-	tsk->cpuset = NULL;
+	tsk->cpuset = &top_cpuset;	/* Hack - see comment above */
 
 	if (notify_on_release(cs)) {
 		char *pathbuf = NULL;
Signature · 3 lines
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <[email protected]> 1.650.933.1373
#1 lore raw
Quoted 10 lines · Paul Jackson

Fix a latent bug in cpuset_exit() handling. If a task tried to allocate memory after calling cpuset_exit(), it oops'd in cpuset_update_task_memory_state() on a NULL cpuset pointer.

So set the exiting tasks cpuset to the root cpuset instead of to NULL.

A distro kernel hit this with an added kernel package that had just such a hook (allocating memory) in the exit code path.

Seems strange to patch the kernel for this. Can't the add-on patch do it? Or can we just move the cpuset_exit()s to later in the exit() paths?

#2 lore raw
Quoted 2 lines · Andrew

Seems strange to patch the kernel for this. Can't the add-on patch do it? Or can we just move the cpuset_exit()s to later in the exit() paths?

Sure, the add-on could have handled it, or had its exit routine called earlier, or had cpuset_exit called later, ...

But it would still have been an accident waiting to happen again, when someone adds another hook, or gets the order in cpuset_exit wrong, or some such.

Heck, for all I know, your *-mm kernel might even have such a bug now (before you took this patch). Perhaps there was some code path that would try to allocate memory after the cpuset_exit call, and no one has hit on that path yet.

This patch was a zero runtime cost way to reduce the risks. With this patch, a task doesn't have a blackout period, shortly before exit, when memory allocations will oops the kernel.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401