Skip to contentSkip to conversation

A couple of OOM killer races

DISCUSSION Richard Hayden 3 participants 3 messages Sep 1, 2005 – Sep 8, 2005

lore
Share & tools
lore
#0 lore raw

Hi all,

It appears there is no protection in badness() (called by out_of_memory() for each process) when it reads p->mm->total_vm. Another processor (or a kernel preemption) could presumably run do_exit and then exit_mm, freeing the process in question's reference to its mm just after the (!p->mm) check but before it reads p->mm->total_vm, making the latter reference a null pointer reference.

Also there appears to be no protection when we set p->time_slice in __oom_kill_task(). Am I right in thinking that this field should be protected by the appropriate runqueue lock, at least this is what scheduler_tick() seems to use?

Have I missed anything?

Best regards,

Richard Hayden.

___________________________________________________________ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com

#1 lore raw
Quoted 8 lines · Richard Hayden

Hi all,

It appears there is no protection in badness() (called by out_of_memory() for each process) when it reads p->mm->total_vm. Another processor (or a kernel preemption) could presumably run do_exit and then exit_mm, freeing the process in question's reference to its mm just after the (!p->mm) check but before it reads p->mm->total_vm, making the latter reference a null pointer reference.

In badness, the tasklist_lock has been hold. And when an exit signal
delived to it because other thread call do_group_exit, it need to hold the tasklist_lock first, so we are protected.

Signature · 1 line
Yingchao Zhou
#2 lore raw
In reply to Richard Hayden
Quoted 8 lines · Richard Hayden

Hi all,

It appears there is no protection in badness() (called by out_of_memory() for each process) when it reads p->mm->total_vm. Another processor (or a kernel preemption) could presumably run do_exit and then exit_mm, freeing the process in question's reference to its mm just after the (!p->mm) check but before it reads p->mm->total_vm, making the latter reference a null pointer reference.

We have read_lock(&tasklist_lock); .

Quoted 5 lines

Also there appears to be no protection when we set p->time_slice in __oom_kill_task(). Am I right in thinking that this field should be protected by the appropriate runqueue lock, at least this is what scheduler_tick() seems to use?

ditto

Signature · 2 lines
Coywolf Qi Hunt
http://sosdg.org/~coywolf/