BUG: oom-killer active when overcommit_memory contains '2', and more...
I see this as 2 bugs, personally, and will say right up front that none of these problems exist in 2.4.10. I don't mean to sound flippant, but also have to say that the reality of this is that I do not have the time to test things further for anyone. Filing this bug report is as much as I have time for (in addition to all of the time I have spent on this already before this message).
1. Filling up the memory on a 2.6.10 box renders it completely and permanently useless even after the memory-filling process exits due to oom-killer's havoc.
I came across this problem when running Iozone NFSv3 benchmarks. The client Linux box in question is running 2.6.10 (Fedora Core 3 with latest updates) and has 4GB physical memory and 4GB in a swap partition. To perform a proper Iozone benchmark for NFS testing, one needs to specify a test file size that is larger than the amount of memory in the client. In my case, 4224MB is being specified:
./iozone -a -g 4224m -f /sol9server/testfile
As soon as it finally gets to trying a file size of 4194304, oom-killer steps in and starts blasting processes off my machine. NFS stops functioning, RPC calls to the box fail, SSH connections are no longer accepted, and I generally have to hard powercycle the box. That's pretty poor :)
2. Andries Brouwer looked into this and asked me to try turning off oom-killer with the following command:
echo 2 > /proc/sys/vm/overcommit_memory
Re-running the 4GB benchmark with Iozone, this did nothing and oom-killer walked through the door blasting processes off of my machine, rendering it totally broken in the end.