OOM killer in 2.4.15pre1 still not 100% ok
Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):
I reconfigured my text box for NFS root fs operation and turned off swap. The box has 128M RAM.
At the time of OOM I was in Midnight Commander on normal vc (not xterm). X+KDE (Kmail and a couple of Konquerors) was loaded but I didn't work in X a that moment.
OOM killed top. I have top permanently running on vc10.
I presume it was neither big nor newly fired process so I don't think it was right candidate for kill.
Last top screen is below.
(How nice: OOM killer taking snapshots of processes at kill time, i.e. aids in its own debugging! :-)
(Oh, nice idea: dump top-like info in syslog after each OOM kill?) --
vda
5:01pm up 1:08, 3 users, load average: 0.18, 0.10, 0.06 61 processes: 58 sleeping, 2 running, 1 zombie, 0 stopped
CPU states: 1.9% user, 15.6% system, 0.0% nice, 82.3% idle Mem: 126272K av, 123428K used, 2844K free, 0K shrd, 16K buff Swap: 0K av, 0K used, 0K free 47748K cached
PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND 4 root 15 0 0 0 0 SW 0 13.8 0.0 0:02 kswapd 974 root 9 0 4852 4852 4724 S 0 2.3 3.8 0:17 mpg123 693 user0 9 0 1440 1440 1232 R 0 0.3 1.1 0:19 top c s 790 root 9 0 5052 5052 4756 S 0 0.1 4.0 0:08 kdeinit: kded 816 root 9 0 9428 9428 8080 S 0 0.1 7.4 0:01 kdeinit: kdesktop 975 root 9 0 1708 1708 1592 S 0 0.1 1.3 0:00 mpg123 1 root 8 0 188 188 160 S 0 0.0 0.1 0:05 init 2 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 keventd 3 root 19 19 0 0 0 SWN 0 0.0 0.0 0:00 ksoftirqd_CPU0 5 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 bdflush 6 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 kupdated 7 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 eth0 8 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 rpciod 14 root 9 0 580 580 496 S 0 0.0 0.4 0:00 devfsd /dev 687 root 9 0 624 624 528 S 0 0.0 0.4 0:00 syslogd 690 root 9 0 1120 1120 444 S 0 0.0 0.8 0:00 klogd -c 3 702 root 9 0 124 124 96 S 0 0.0 0.0 0:00 dhcpcd -t 20 -R -d eth1 734 rpc 9 0 644 644 540 S 0 0.0 0.5 0:00 rpc.portmap 736 root 9 0 556 556 484 S 0 0.0 0.4 0:00 inetd 738 root 9 0 664 664 564 S 0 0.0 0.5 0:00 automount
--timeout 5 /mnt/auto
740 root 9 0 488 488 420 S 0 0.0 0.3 0:00 gpm -2 -m/dev/psaux -t ps2
741 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty1 linux
743 root 9 0 1168 1168 892 S 0 0.0 0.9 0:00 -bash 745 root 9 0 1168 1168 892 S 0 0.0 0.9 0:00 -bash 748 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty4 linux
749 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty5 linux
752 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty6 linux
753 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty7 linux
754 root 9 0 1116 1116 820 S 0 0.0 0.8 0:00 nmbd
-l/var/log/samba/nmbd.log -
755 root 9 0 1168 1168 1060 S 0 0.0 0.9 0:00 -bash
Quoted 5 lines · vda
Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):
I reconfigured my text box for NFS root fs operation and turned off swap. The box has 128M RAM.
...
Quoted 5 lines
5:01pm up 1:08, 3 users, load average: 0.18, 0.10, 0.06 61 processes: 58 sleeping, 2 running, 1 zombie, 0 stopped
CPU states: 1.9% user, 15.6% system, 0.0% nice, 82.3% idle Mem: 126272K av, 123428K used, 2844K free, 0K shrd, 16K buff Swap: 0K av, 0K used, 0K free 47748K cached
Er, with almost 3megs of free memory and -47megs- of cache, the problem isn't with the OOM killer's selection, but the fact it was triggered with nearly half the RAM still usable. Do you actually know the OOM killer was triggered? Or did top just mysteriously exit?
Personally, I've found the new 2.4.14+ OOM killer to be highly accurate, I've run ext3's shared mappings torture test on my 384meg RAM (256meg swap) box repeatedly, and it pushes my computer to OOM every 60 seconds or so. It always kills the correct process, with my box remaining absolutely stable, even with less swap space than RAM.
-Ryan
Quoted 15 lines · Ryan Cumming
Quoted 5 lines · vda
Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):
I reconfigured my text box for NFS root fs operation and turned off swap. The box has 128M RAM.
...
Quoted 5 lines
5:01pm up 1:08, 3 users, load average: 0.18, 0.10, 0.06 61 processes: 58 sleeping, 2 running, 1 zombie, 0 stopped
CPU states: 1.9% user, 15.6% system, 0.0% nice, 82.3% idle Mem: 126272K av, 123428K used, 2844K free, 0K shrd, 16K buff Swap: 0K av, 0K used, 0K free 47748K cached
Quoted 4 lines
Er, with almost 3megs of free memory and -47megs- of cache, the problem isn't with the OOM killer's selection, but the fact it was triggered with nearly half the RAM still usable. Do you actually know the OOM killer was triggered? Or did top just mysteriously exit?
I have no swap at all.
There was a message "Out of Memory: killing process top" or something such on screen.
--
vda
Quoted 2 lines · vda
Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):
With no swap, seems pretty frisky in pre9. I can't start X/KDE with a 64MB ram boot. Touching the end of ram is very deadly atm with no swap enabled.. can't say it's a wrong behavior though, as I am touching the end. (I think I'd _rather_ it die than thrash)
-Mike
Quoted 5 lines · Ryan Cumming
Personally, I've found the new 2.4.14+ OOM killer to be highly accurate, I've run ext3's shared mappings torture test on my 384meg RAM (256meg swap) box repeatedly, and it pushes my computer to OOM every 60 seconds or so. It always kills the correct process, with my box remaining absolutely stable, even with less swap space than RAM.
OOM did it again, this time I ran glib 1.2.10 configure script and while it checked whether I have working -static, OOM again killed my top (looks like it really likes my top :-). You can see newly started ld which are about to trigger OOM within 5 secs in last screen of poor top.
Maybe I misunderstand something, but why OOM chose top? Is it how it is intended to work?
PS. I thought 128 Megs of RAM is enough for X+KDE+gcc... Do we all have to play Gig'o'Rama these days?
[Yes I know that swap will help, let's not start a thread...] --
vda
Out of Memory: Killed process 726 (top)
12:20pm up 1:12, 4 users, load average: 0.39, 0.20, 0.11 67 processes: 62 sleeping, 5 running, 0 zombie, 0 stopped CPU states: 12.4% user, 46.8% system, 0.0% nice, 40.7% idle
Mem: 126272K av, 124856K used, 1416K free, 0K shrd, 0K buff
Swap: 0K av, 0K used, 0K free 45664K cached
PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND 4 root 17 0 0 0 0 SW 0 14.5 0.0 0:05 kswapd 2071 root 14 0 2340 2340 728 R 0 11.0 1.8 0:00 /usr/bin/ld -m elf_i386 -static 900 root 13 0 1916 1916 1788 R 0 3.5 1.5 0:35 mpg123 1964 root 9 0 968 968 856 S 0 2.1 0.7 0:00 /bin/sh
../ltconfig --no-reexec
868 root 9 0 7008 7008 6212 S 0 1.9 5.5 0:00 kdeinit:
klipper -icon klipper -
1779 root 9 0 992 992 740 S 0 1.7 0.7 0:00 /bin/sh
../configure --prefix=/u
2067 root 9 0 496 496 400 S 0 1.1 0.3 0:00 gcc -o
conftest -g -O2 -static c
8 root 10 0 0 0 0 SW 0 0.7 0.0 0:00 rpciod 847 root 13 0 2272 2268 1580 R 0 0.7 1.7 0:00 artsd -F 10 -S 4096 726 user0 9 0 1340 1340 1128 R 0 0.5 1.0 0:21 top c s 2070 root 9 0 456 456 388 S 0 0.5 0.3 0:00 /usr/lib/gcc301/lib/gcc-lib/i686 820 root 9 0 47012 29M 1156 S 0 0.3 24.1 1:56 X :0 -layout mga 901 root 10 0 1684 1684 1568 S 0 0.3 1.3 0:01 mpg123 844 root 9 0 5016 5016 4720 S 0 0.1 3.9 0:08 kdeinit: kded 860 root 9 0 9804 9804 8368 S 0 0.1 7.7 0:01 kdeinit: kdesktop 1 root 8 0 52 52 24 S 0 0.0 0.0 0:05 init 2 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 keventd 3 root 19 19 0 0 0 SWN 0 0.0 0.0 0:00 ksoftirqd_CPU0 5 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 bdflush 6 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 kupdated 7 root 9 0 0 0 0 SW 0 0.0 0.0 0:00 eth0 40 root 9 0 560 560 476 S 0 0.0 0.4 0:00 devfsd /dev 720 root 9 0 624 624 528 S 0 0.0 0.4 0:00 syslogd 723 root 9 0 1100 1100 424 S 0 0.0 0.8 0:00 klogd -c 3 735 root 9 0 124 124 96 S 0 0.0 0.0 0:00 dhcpcd -t 20 -R -d eth1 767 rpc 9 0 644 644 540 S 0 0.0 0.5 0:00 rpc.portmap 769 root 9 0 544 544 472 S 0 0.0 0.4 0:00 inetd 773 root 9 0 472 472 404 S 0 0.0 0.3 0:00 gpm -2 -m
/dev/psaux -t ps2
790 root 9 0 484 484 424 S 0 0.0 0.3 0:00/sbin/agetty 38400 tty1 linux
791 root 9 0 1168 1168 892 S 0 0.0 0.9 0:00 -bash
Quoted 2 lines · vda
Maybe I misunderstand something, but why OOM chose top? Is it how it is intended to work?
It is intended to do the least possible damage when killing something. I'd say it does nicely when killing "top", you surely don't loose much work that way. :-)
Helge Hafting