Skip to contentSkip to conversation

OOM killer in 2.4.15pre1 still not 100% ok

DISCUSSION vda 4 participants 6 messages Nov 22, 2001 – Nov 26, 2001

lore
Share & tools
lore

vda

author
#0 lore raw

Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):

I reconfigured my text box for NFS root fs operation and turned off swap. The box has 128M RAM.

At the time of OOM I was in Midnight Commander on normal vc (not xterm). X+KDE (Kmail and a couple of Konquerors) was loaded but I didn't work in X a that moment.

OOM killed top. I have top permanently running on vc10.
I presume it was neither big nor newly fired process so I don't think it was right candidate for kill.

Last top screen is below.
(How nice: OOM killer taking snapshots of processes at kill time, i.e. aids in its own debugging! :-)
(Oh, nice idea: dump top-like info in syslog after each OOM kill?) --
vda

5:01pm up 1:08, 3 users, load average: 0.18, 0.10, 0.06 61 processes: 58 sleeping, 2 running, 1 zombie, 0 stopped

CPU states:  1.9% user, 15.6% system,  0.0% nice, 82.3% idle
Mem:  126272K av, 123428K used,   2844K free,      0K shrd,     16K buff
Swap:      0K av,      0K used,      0K free                 47748K cached
PID USER     PRI  NI  SIZE  RSS SHARE STAT  LIB %CPU %MEM   TIME COMMAND
  4 root      15   0     0    0     0 SW      0 13.8  0.0   0:02 kswapd
974 root       9   0  4852 4852  4724 S       0  2.3  3.8   0:17 mpg123
693 user0      9   0  1440 1440  1232 R       0  0.3  1.1   0:19 top c s
790 root       9   0  5052 5052  4756 S       0  0.1  4.0   0:08 kdeinit: kded
816 root       9   0  9428 9428  8080 S       0  0.1  7.4   0:01 kdeinit: 
kdesktop
975 root       9   0  1708 1708  1592 S       0  0.1  1.3   0:00 mpg123
  1 root       8   0   188  188   160 S       0  0.0  0.1   0:05 init
  2 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 keventd
  3 root      19  19     0    0     0 SWN     0  0.0  0.0   0:00 
ksoftirqd_CPU0
  5 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 bdflush
  6 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 kupdated
  7 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 eth0
  8 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 rpciod
 14 root       9   0   580  580   496 S       0  0.0  0.4   0:00 devfsd /dev
687 root       9   0   624  624   528 S       0  0.0  0.4   0:00 syslogd
690 root       9   0  1120 1120   444 S       0  0.0  0.8   0:00 klogd -c 3
702 root       9   0   124  124    96 S       0  0.0  0.0   0:00 dhcpcd -t 20 
-R -d eth1
734 rpc        9   0   644  644   540 S       0  0.0  0.5   0:00 rpc.portmap
736 root       9   0   556  556   484 S       0  0.0  0.4   0:00 inetd
738 root       9   0   664  664   564 S       0  0.0  0.5   0:00 automount 

--timeout 5 /mnt/auto
740 root 9 0 488 488 420 S 0 0.0 0.3 0:00 gpm -2 -m
/dev/psaux -t ps2
741 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty1 linux

743 root       9   0  1168 1168   892 S       0  0.0  0.9   0:00 -bash
745 root       9   0  1168 1168   892 S       0  0.0  0.9   0:00 -bash
748 root       9   0   484  484   424 S       0  0.0  0.3   0:00 /sbin/agetty 

38400 tty4 linux
749 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty5 linux
752 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty6 linux
753 root 9 0 484 484 424 S 0 0.0 0.3 0:00 /sbin/agetty
38400 tty7 linux
754 root 9 0 1116 1116 820 S 0 0.0 0.8 0:00 nmbd
-l/var/log/samba/nmbd.log -
755 root 9 0 1168 1168 1060 S 0 0.0 0.9 0:00 -bash

#1 lore raw
Quoted 5 lines · vda

Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):

I reconfigured my text box for NFS root fs operation and turned off swap. The box has 128M RAM.

...

Quoted 5 lines

5:01pm up 1:08, 3 users, load average: 0.18, 0.10, 0.06 61 processes: 58 sleeping, 2 running, 1 zombie, 0 stopped

CPU states:  1.9% user, 15.6% system,  0.0% nice, 82.3% idle
Mem:  126272K av, 123428K used,   2844K free,      0K shrd,     16K buff
Swap:      0K av,      0K used,      0K free                 47748K cached

Er, with almost 3megs of free memory and -47megs- of cache, the problem isn't with the OOM killer's selection, but the fact it was triggered with nearly half the RAM still usable. Do you actually know the OOM killer was triggered? Or did top just mysteriously exit?

Personally, I've found the new 2.4.14+ OOM killer to be highly accurate, I've run ext3's shared mappings torture test on my 384meg RAM (256meg swap) box repeatedly, and it pushes my computer to OOM every 60 seconds or so. It always kills the correct process, with my box remaining absolutely stable, even with less swap space than RAM.

-Ryan

author
#2 lore raw
In reply to vda
Quoted 15 lines · Ryan Cumming
Quoted 5 lines · vda

Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):

I reconfigured my text box for NFS root fs operation and turned off swap. The box has 128M RAM.

...

Quoted 5 lines

5:01pm up 1:08, 3 users, load average: 0.18, 0.10, 0.06 61 processes: 58 sleeping, 2 running, 1 zombie, 0 stopped

CPU states:  1.9% user, 15.6% system,  0.0% nice, 82.3% idle
Mem:  126272K av, 123428K used,   2844K free,      0K shrd,     16K buff
Swap:      0K av,      0K used,      0K free                 47748K cached
Quoted 4 lines

Er, with almost 3megs of free memory and -47megs- of cache, the problem isn't with the OOM killer's selection, but the fact it was triggered with nearly half the RAM still usable. Do you actually know the OOM killer was triggered? Or did top just mysteriously exit?

I have no swap at all.
There was a message "Out of Memory: killing process top" or something such on screen.
--
vda

#3 lore raw
In reply to vda
Quoted 2 lines · vda

Today I saw OOM killer in action for the very fist time. Just want to inform that it still not 100% ok (IMHO):

With no swap, seems pretty frisky in pre9. I can't start X/KDE with a 64MB ram boot. Touching the end of ram is very deadly atm with no swap enabled.. can't say it's a wrong behavior though, as I am touching the end. (I think I'd _rather_ it die than thrash)

-Mike

author
#4 lore raw
In reply to vda
Quoted 5 lines · Ryan Cumming

Personally, I've found the new 2.4.14+ OOM killer to be highly accurate, I've run ext3's shared mappings torture test on my 384meg RAM (256meg swap) box repeatedly, and it pushes my computer to OOM every 60 seconds or so. It always kills the correct process, with my box remaining absolutely stable, even with less swap space than RAM.

OOM did it again, this time I ran glib 1.2.10 configure script and while it checked whether I have working -static, OOM again killed my top (looks like it really likes my top :-). You can see newly started ld which are about to trigger OOM within 5 secs in last screen of poor top.

Maybe I misunderstand something, but why OOM chose top? Is it how it is intended to work?

PS. I thought 128 Megs of RAM is enough for X+KDE+gcc... Do we all have to play Gig'o'Rama these days?
[Yes I know that swap will help, let's not start a thread...] --
vda

Out of Memory: Killed process 726 (top)

12:20pm up 1:12, 4 users, load average: 0.39, 0.20, 0.11 67 processes: 62 sleeping, 5 running, 0 zombie, 0 stopped CPU states: 12.4% user, 46.8% system, 0.0% nice, 40.7% idle
Mem: 126272K av, 124856K used, 1416K free, 0K shrd, 0K buff
Swap: 0K av, 0K used, 0K free 45664K cached

 PID USER     PRI  NI  SIZE  RSS SHARE STAT  LIB %CPU %MEM   TIME COMMAND
   4 root      17   0     0    0     0 SW      0 14.5  0.0   0:05 kswapd
2071 root      14   0  2340 2340   728 R       0 11.0  1.8   0:00 /usr/bin/ld 
-m elf_i386 -static
 900 root      13   0  1916 1916  1788 R       0  3.5  1.5   0:35 mpg123
1964 root       9   0   968  968   856 S       0  2.1  0.7   0:00 /bin/sh 

../ltconfig --no-reexec
868 root 9 0 7008 7008 6212 S 0 1.9 5.5 0:00 kdeinit:
klipper -icon klipper -
1779 root 9 0 992 992 740 S 0 1.7 0.7 0:00 /bin/sh
../configure --prefix=/u
2067 root 9 0 496 496 400 S 0 1.1 0.3 0:00 gcc -o
conftest -g -O2 -static c

   8 root      10   0     0    0     0 SW      0  0.7  0.0   0:00 rpciod
 847 root      13   0  2272 2268  1580 R       0  0.7  1.7   0:00 artsd -F 10 
-S 4096
 726 user0      9   0  1340 1340  1128 R       0  0.5  1.0   0:21 top c s
2070 root       9   0   456  456   388 S       0  0.5  0.3   0:00 
/usr/lib/gcc301/lib/gcc-lib/i686
 820 root       9   0 47012  29M  1156 S       0  0.3 24.1   1:56 X :0 
-layout mga
 901 root      10   0  1684 1684  1568 S       0  0.3  1.3   0:01 mpg123
 844 root       9   0  5016 5016  4720 S       0  0.1  3.9   0:08 kdeinit: 
kded
 860 root       9   0  9804 9804  8368 S       0  0.1  7.7   0:01 kdeinit: 
kdesktop
   1 root       8   0    52   52    24 S       0  0.0  0.0   0:05 init
   2 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 keventd
   3 root      19  19     0    0     0 SWN     0  0.0  0.0   0:00 
ksoftirqd_CPU0
   5 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 bdflush
   6 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 kupdated
   7 root       9   0     0    0     0 SW      0  0.0  0.0   0:00 eth0
  40 root       9   0   560  560   476 S       0  0.0  0.4   0:00 devfsd /dev
 720 root       9   0   624  624   528 S       0  0.0  0.4   0:00 syslogd
 723 root       9   0  1100 1100   424 S       0  0.0  0.8   0:00 klogd -c 3
 735 root       9   0   124  124    96 S       0  0.0  0.0   0:00 dhcpcd -t 
20 -R -d eth1
 767 rpc        9   0   644  644   540 S       0  0.0  0.5   0:00 rpc.portmap
 769 root       9   0   544  544   472 S       0  0.0  0.4   0:00 inetd
 773 root       9   0   472  472   404 S       0  0.0  0.3   0:00 gpm -2 -m 

/dev/psaux -t ps2
790 root 9 0 484 484 424 S 0 0.0 0.3 0:00
/sbin/agetty 38400 tty1 linux
791 root 9 0 1168 1168 892 S 0 0.0 0.9 0:00 -bash

#5 lore raw
In reply to vda
Quoted 2 lines · vda

Maybe I misunderstand something, but why OOM chose top? Is it how it is intended to work?

It is intended to do the least possible damage when killing something. I'd say it does nicely when killing "top", you surely don't loose much work that way. :-)

Helge Hafting