Skip to contentSkip to conversation

mm_struct leak on cpu hotplug (s390/ppc64)

BUG New Heiko Carstens 3 participants 8 messages Jan 4, 2005 – Jan 7, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

there is an mm_struct memory leak when using cpu hotplug. Appearently start_secondary in smp.c initializes active_mm of the cpu's idle task and increases init_mm's mm_count. But on cpu_die the idle task's active_mm doesn't get dropped and therefore on the next cpu_up event (->start_secondary) it gets overwritten and the result is a forgotten reference count to whatever mm_struct was active when the cpu was taken down previously.

The patch below should fix this for s390 (at least it works fine for me), but I'm not sure if it's ok to call mmdrop from __cpu_die.

Also this very same leak exists for ppc64 as well.

Any opinions?

Thanks,
Heiko

arch/s390/kernel/smp.c +5 -0
diff -urN linux-2.6.10/arch/s390/kernel/smp.c linux-2.6.10-patched/arch/s390/kernel/smp.c
@@ -728,9 +728,14 @@
 void
 __cpu_die(unsigned int cpu)
 {
+	struct task_struct *p;
+
 	/* Wait until target cpu is down */
 	while (!cpu_stopped(cpu));
 	printk("Processor %d spun down\n", cpu);
+	p = current_set[cpu];
+	mmdrop(p->active_mm);
+	p->active_mm = NULL;
 }
 
 void
#1 lore raw
Quoted 10 lines · Heiko Carstens

Hi,

there is an mm_struct memory leak when using cpu hotplug. Appearently start_secondary in smp.c initializes active_mm of the cpu's idle task and increases init_mm's mm_count. But on cpu_die the idle task's active_mm doesn't get dropped and therefore on the next cpu_up event (->start_secondary) it gets overwritten and the result is a forgotten reference count to whatever mm_struct was active when the cpu was taken down previously.

Quoted 6 lines

The patch below should fix this for s390 (at least it works fine for me), but I'm not sure if it's ok to call mmdrop from __cpu_die.

Also this very same leak exists for ppc64 as well.

Any opinions?

Wouldn't it be better to fix this in generic code instead of duplicating it in each architecture? It looks like the same thing would occur on ia64 also.

What about something like this? Tested on ppc64.

kernel/sched.c +3 -0
Index: 2.6.10/kernel/sched.c
===================================================================
@@ -4088,6 +4088,9 @@
 		migrate_nr_uninterruptible(rq);
 		BUG_ON(rq->nr_running != 0);
 
+		/* Must manually drop reference to avoid leaking mm_structs. */
+		mmdrop(rq->idle->active_mm);
+
 		/* No need to migrate the tasks: it was best-effort if
 		 * they didn't do lock_cpu_hotplug().  Just wake up
 		 * the requestors. */
#2 lore raw

kernel/sched.c line 4096

Quoted 11 lines · Nathan Lynch

What about something like this? Tested on ppc64.

kernel/sched.c +3 -0
 		migrate_nr_uninterruptible(rq);
 		BUG_ON(rq->nr_running != 0);
 
+		/* Must manually drop reference to avoid leaking mm_structs. */
+		mmdrop(rq->idle->active_mm);
+
 		/* No need to migrate the tasks: it was best-effort if
 		 * they didn't do lock_cpu_hotplug().  Just wake up
 		 * the requestors. */

this doesnt look correct to me, because we might end up pulling the rug (the pagetables) from under the idle task on that CPU. This can happen in two ways: 1) there's no direct synchronization between a dead CPU having called into cpu_die() and the downing CPU doing the mmdrop(), so we might end up dropping it before the idle has entered the final loop and is still executing kernel code, 2) even when the dead idle task is already in its final loop there's no generic guarantee that an mmdrop() can be done - e.g. on x86 the kernel pagetables are mixed up with the user pagetables and an mmdrop() in case of lazy-TLB might end up zapping the idle task's pagetables which might break in subtle ways.

the correct solution i think would be to call back into the scheduler from cpu_die():

void cpu_die(void)

{
        if (ppc_md.cpu_die)
                ppc_md.cpu_die();
+	idle_task_exit();
        local_irq_disable();
        for (;;);
}

and then in idle_task_exit(), do something like:

void idle_task_exit(void)

{
	struct mm_struct *mm = current->active_mm;

	if (mm != &init_mm)
		switch_mm(mm, &init_mm, current);
	mmdrop(mm);
}

(completely untested.) This makes sure that the idle task uses the init_mm (which always has valid pagetables), and also ensures correct reference-counting. Hm?

Ingo

#3 lore raw
Quoted 23 lines

the correct solution i think would be to call back into the scheduler from cpu_die():

void cpu_die(void)

{
        if (ppc_md.cpu_die)
                ppc_md.cpu_die();
+   idle_task_exit();
        local_irq_disable();
        for (;;);
}

and then in idle_task_exit(), do something like:

void idle_task_exit(void)

{
   struct mm_struct *mm = current->active_mm;

   if (mm != &init_mm)
      switch_mm(mm, &init_mm, current);
   mmdrop(mm);
}
Quoted 3 lines

(completely untested.) This makes sure that the idle task uses the init_mm (which always has valid pagetables), and also ensures correct reference-counting. Hm?

Looks good and works for me.

Thanks,
Heiko

#4 lore raw
In reply to Ingo Molnar
Quoted 23 lines · Ingo Molnar

the correct solution i think would be to call back into the scheduler from cpu_die():

void cpu_die(void)

{
        if (ppc_md.cpu_die)
                ppc_md.cpu_die();
(quoted diff) +1 -0
+	idle_task_exit();
        local_irq_disable();
        for (;;);

}

and then in idle_task_exit(), do something like:

void idle_task_exit(void)

{
	struct mm_struct *mm = current->active_mm;

	if (mm != &init_mm)
		switch_mm(mm, &init_mm, current);
	mmdrop(mm);
}
Quoted 3 lines

(completely untested.) This makes sure that the idle task uses the init_mm (which always has valid pagetables), and also ensures correct reference-counting. Hm?

OK, how's this? I'll submit the sched and ppc64 bits separately if there are no objections. I assume Heiko can take care of s390.

Note that in the ppc64 cpu_die function we must call idle_task_exit before calling ppc_md.cpu_die, because the latter does not return.

Nathan

include/linux/sched.h +1 -0
Index: 2.6.10/include/linux/sched.h
===================================================================
@@ -735,6 +735,7 @@
 #endif
 
 extern void sched_idle_next(void);
+extern void idle_task_exit(void);
 extern void set_user_nice(task_t *p, long nice);
 extern int task_prio(const task_t *p);
 extern int task_nice(const task_t *p);
kernel/sched.c +14 -0
Index: 2.6.10/kernel/sched.c
===================================================================
@@ -3995,6 +3995,20 @@
 	spin_unlock_irqrestore(&rq->lock, flags);
 }
 
+/* Ensures that the idle task is using init_mm right before its cpu goes
+ * offline.
+ */
+void idle_task_exit(void)
+{
+	struct mm_struct *mm = current->active_mm;
+
+	BUG_ON(cpu_online(smp_processor_id()));
+
+	if (mm != &init_mm)
+		switch_mm(mm, &init_mm, current);
+	mmdrop(mm);
+}
+
 static void migrate_dead(unsigned int dead_cpu, task_t *tsk)
 {
 	struct runqueue *rq = cpu_rq(dead_cpu);
arch/ppc64/kernel/setup.c +1 -0
Index: 2.6.10/arch/ppc64/kernel/setup.c
===================================================================
@@ -1337,6 +1337,7 @@
 
 void cpu_die(void)
 {
+	idle_task_exit();
 	if (ppc_md.cpu_die)
 		ppc_md.cpu_die();
 	local_irq_disable();
#5 lore raw
Quoted 5 lines · Nathan Lynch

OK, how's this? I'll submit the sched and ppc64 bits separately if there are no objections. I assume Heiko can take care of s390.

Note that in the ppc64 cpu_die function we must call idle_task_exit before calling ppc_md.cpu_die, because the latter does not return.

Ingo

#6 lore raw

PATCH 1/1 introduce idle_task_exit

Quoted 9 lines · Ingo Molnar
Quoted 5 lines · Nathan Lynch

OK, how's this? I'll submit the sched and ppc64 bits separately if there are no objections. I assume Heiko can take care of s390.

Note that in the ppc64 cpu_die function we must call idle_task_exit before calling ppc_md.cpu_die, because the latter does not return.

Heiko Carstens figured out that offlining a cpu can leak mm_structs because the dying cpu's idle task fails to switch to init_mm and mmdrop its active_mm before the cpu is down. This patch introduces idle_task_exit, which allows the idle task to do this as Ingo suggested.

I will follow this up with a patch for ppc64 which calls idle_task_exit from cpu_die.

---

kernel/sched.c renamed from kernel/sched.c~sched-idle_task_exit +14 -0
diff -puN kernel/sched.c~sched-idle_task_exit kernel/sched.c
@@ -3996,6 +3996,20 @@ void sched_idle_next(void)
 	spin_unlock_irqrestore(&rq->lock, flags);
 }
 
+/* Ensures that the idle task is using init_mm right before its cpu goes
+ * offline.
+ */
+void idle_task_exit(void)
+{
+	struct mm_struct *mm = current->active_mm;
+
+	BUG_ON(cpu_online(smp_processor_id()));
+
+	if (mm != &init_mm)
+		switch_mm(mm, &init_mm, current);
+	mmdrop(mm);
+}
+
 static void migrate_dead(unsigned int dead_cpu, task_t *tsk)
 {
 	struct runqueue *rq = cpu_rq(dead_cpu);
include/linux/sched.h renamed from include/linux/sched.h~sched-idle_task_exit +1 -0
diff -puN include/linux/sched.h~sched-idle_task_exit include/linux/sched.h
@@ -757,6 +757,7 @@ extern void sched_exec(void);
 #endif
 
 extern void sched_idle_next(void);
+extern void idle_task_exit(void);
 extern void set_user_nice(task_t *p, long nice);
 extern int task_prio(const task_t *p);
 extern int task_nice(const task_t *p);

_

#7 lore raw

PATCH 1/1 ppc64: call idle_task_exit from cpu_die

Call idle_task_exit from cpu_die to avoid mm_struct leak.

---

arch/ppc64/kernel/setup.c renamed from arch/ppc64/kernel/setup.c~ppc64-call-idle_task_exit-from-cpu_die +1 -0
diff -puN arch/ppc64/kernel/setup.c~ppc64-call-idle_task_exit-from-cpu_die arch/ppc64/kernel/setup.c
@@ -1345,6 +1345,7 @@ early_param("xmon", early_xmon);
 
 void cpu_die(void)
 {
+	idle_task_exit();
 	if (ppc_md.cpu_die)
 		ppc_md.cpu_die();
 	local_irq_disable();

_