2.6.21 frozen for a few minutes, swapping to disk
Subsystems (2)
Hi all,
today, with 2.6.21, my laptop had a really odd behaviour. It started writing to disk for a few minutes with no interactivity at all (no redraw on screen, only hdd led on). It's the first time i noticed OOM-killer started do kill programs.
It was totally unresponsive for minutes, after back to life it had a load of ~19.0, and 300+ MB on swap (first time i saw this).
It's an HP pavillon core duo 2.0 GHz, 1 GB RAM
kern.log details: http://www.debianpt.org/~elmig/pool/kernel/20070429/kern.log
.config: http://www.debianpt.org/~elmig/pool/kernel/20070429/2.6.21.config
dmesg: http://www.debianpt.org/~elmig/pool/kernel/20070429/dmesg
As this is the first time it happened and it felt odd i am reporting.
If aditional info is needed please CC me as i am not on the list.
Signature · 4 lines
Com os melhores cumprimentos/Best regards, Miguel Figueiredo http://www.DebianPT.org
Andrew Morton
maintainerQuoted 12 lines · On Sun, 29 Apr 2007 11:28:05 +0100 Miguel Figueiredo <[email protected]> wrote:
Hi all,
today, with 2.6.21, my laptop had a really odd behaviour. It started writing to disk for a few minutes with no interactivity at all (no redraw on screen, only hdd led on). It's the first time i noticed OOM-killer started do kill programs.
It was totally unresponsive for minutes, after back to life it had a load of ~19.0, and 300+ MB on swap (first time i saw this).
It's an HP pavillon core duo 2.0 GHz, 1 GB RAM
Quoted 9 lines
kern.log details:
http://www.debianpt.org/~elmig/pool/kernel/20070429/kern.log
.config: http://www.debianpt.org/~elmig/pool/kernel/20070429/2.6.21.config
dmesg: http://www.debianpt.org/~elmig/pool/kernel/20070429/dmesgAs this is the first time it happened and it felt odd i am reporting.
If aditional info is needed please CC me as i am not on the list.
hm, a genuine oom on an all-ext3 data=ordered i386 system, just like a million other people. How very weird.
I assume all those pages on the LRU are pagecache pages which for some reason we're unable to reclaim.
If some privileged application went berzerk mlock()ing everything then that might explain it. It sounds improbable, but then, something improbable has happened.
We cleverly managed to not display the pagecache totals in the oom-killer output. Could you please take a copy of /proc/meminfo after an oom-killing, send that? And /proc/vmstat, I guess.
If you're keen, we could eliminate the mlock possibility by adding this:
mm/mlock.c renamed from mm/mlock.c~a +4 -0
@@ -127,6 +127,8 @@ asmlinkage long sys_mlock(unsigned longunsigned long lock_limit;int error = -ENOMEM ; +return 0 ; +if (!can_do_mlock ())return -EPERM ;
@@ -151,6 +153,8 @@ asmlinkage long sys_munlock(unsigned lon {int ret; +return 0 ; +down_write (¤t->mm->mmap_sem); len =PAGE_ALIGN (len + (start & ~PAGE_MASK )); start &=PAGE_MASK ;
_
Quoted 21 lines · Andrew Morton
Quoted 8 lines · On Sun, 29 Apr 2007 11:28:05 +0100 Miguel Figueiredo <[email protected]> wrote:
Hi all,
today, with 2.6.21, my laptop had a really odd behaviour. It started writing to disk for a few minutes with no interactivity at all (no redraw on screen, only hdd led on). It's the first time i noticed OOM-killer started do kill programs.
It was totally unresponsive for minutes, after back to life it had a load of ~19.0, and 300+ MB on swap (first time i saw this).
It's an HP pavillon core duo 2.0 GHz, 1 GB RAM
Quoted 8 lines
kern.log details: http://www.debianpt.org/~elmig/pool/kernel/20070429/kern.log
.config: http://www.debianpt.org/~elmig/pool/kernel/20070429/2.6.21.config
dmesg: http://www.debianpt.org/~elmig/pool/kernel/20070429/dmesgAs this is the first time it happened and it felt odd i am reporting.
If aditional info is needed please CC me as i am not on the list.
Quoted 5 lines
hm, a genuine oom on an all-ext3 data=ordered i386 system, just like a million other people. How very weird.
I assume all those pages on the LRU are pagecache pages which for some reason we're unable to reclaim.
It looks like it used up all swap? I'd guess a memory leak in some application, or maybe a page refcount leak somewhere.
Quoted 33 lines
If some privileged application went berzerk mlock()ing everything then that might explain it. It sounds improbable, but then, something improbable has happened.
We cleverly managed to not display the pagecache totals in the oom-killer output. Could you please take a copy of
/proc/meminfoafter an oom-killing, send that? And/proc/vmstat, I guess.If you're keen, we could eliminate the mlock possibility by adding this:
mm/mlock.c renamed from mm/mlock.c~a +4
-0@@ -127,6 +127,8 @@ asmlinkage long sys_mlock(unsigned longunsigned long lock_limit;int error = -ENOMEM ; +return 0 ; +if (!can_do_mlock ())return -EPERM ;@@ -151,6 +153,8 @@ asmlinkage long sys_munlock(unsigned lon {int ret; +return 0 ; +down_write (¤t->mm->mmap_sem); len =PAGE_ALIGN (len + (start & ~PAGE_MASK )); start &=PAGE_MASK ;_
Signature · 1 line
SUSE Labs, Novell Inc.
Andrew Morton
maintainerQuoted 8 lines · On Tue, 01 May 2007 15:42:30 +1000 Nick Piggin <[email protected]> wrote:
Quoted 5 lines
hm, a genuine oom on an all-ext3 data=ordered i386 system, just like a million other people. How very weird.
I assume all those pages on the LRU are pagecache pages which for some reason we're unable to reclaim.
It looks like it used up all swap? I'd guess a memory leak in some application, or maybe a page refcount leak somewhere.
yes, I missed that. The number of mapped pages is tiny so the thing has
been trying to swap out like.
The question is: how much memory is free after the oom-killing storm? If it's "lots" then it's probably an application problem. If it's "not much" then perhaps there's a kernel leak.
Quoted 9 lines · Andrew Morton
Quoted 6 lines · On Tue, 01 May 2007 15:42:30 +1000 Nick Piggin <[email protected]> wrote:
Quoted 5 lines
hm, a genuine oom on an all-ext3 data=ordered i386 system, just like a million other people. How very weird.
I assume all those pages on the LRU are pagecache pages which for some reason we're unable to reclaim.
Quoted 6 lines
Quoted 2 lines
It looks like it used up all swap? I'd guess a memory leak in some application, or maybe a page refcount leak somewhere.
yes, I missed that. The number of mapped pages is tiny so the thing has
been trying to swap out like.
I didn't quite parse this :)
If the memory is leaking slowly, it could be eventually pushing everything out to swap without having a large amount of mapped pages.
Or if something is slowly writing stuff to tmpfs, that may not show up in mapped pages either.
Quoted 3 lines
The question is: how much memory is free after the oom-killing storm? If it's "lots" then it's probably an application problem. If it's "not much" then perhaps there's a kernel leak.
Yeah, or a tmpfs filesystem being filled up (what does df say?).
Signature · 1 line
SUSE Labs, Novell Inc.