Strange behavior in out-of-memory situation

I'm using Red Hat kernel 2.4.18-26.7.x.

I ran a program which is trying to suck up all of memory. I would like to kill the process, but "top", "vmstat", and "ps" all hang when I try to use them. Also, pressing ctrl-c in the terminal where I can the program won't kill it.

To an extent, however, the system was still usable, albeit EXTREMELY unresponsive. Eventually, the program dumped core, and everything returned to norma.

Is this a known problem?

Your system seems to be in normal working order. There is no problem here. A program using up all memory _will_ make the machine very unresponsive as everything goes into swap (or discards all executable code for those who think they can fool the system by not having swap).

You may indeed be unable to run other programs like top, because there aren't enough memory left to run them. Your program probably dumped core when it ran the system completely out of memory and trigged the Out-Of-Memory killer.

The only unusual here is that ctrl+c didn't work, but some programs block that signal. Perhaps your program does too.

You can use ulimit if you don't want your machine to become sluggish. The effect is that your memory eater will be killed earlier, before it uses up so much memory that nothing else will run.

Helge Hafting

I wrote the program, and it's a simple data processor. It does nothing to block ctrl-c. In fact, although I couldn't run ps, top, or vmstat, I could run other things like ls and vi.