Prevent OOM from killing init
[to various people]
No, ulimit does not work. (But it helps a little.) No, /proc/sys/vm/overcommit_memory does not work.
[to Alan]
Quoted 1 line
Nobody feels its very important because nobody has implemented it.
Yes, that is the right response.
What can one say? One can only do.
Andries
Quoted 6 lines · [email protected] wrote:
[to various people]
No, ulimit does not work. (But it helps a little.) No,
/proc/sys/vm/overcommit_memorydoes not work.
Quoted 6 lines
[to Alan]
Quoted 1 line
Nobody feels its very important because nobody has implemented it.
Yes, that is the right response.
What can one say? One can only do.
Please just expect a patch for tomorrow ;-).
The only thing I have currently to do is testing.
I will be using the installation process of the ORACLE iAS 9i for linux on my notebook, becouse it used to trigger oom for me VERY frequently. So far all things BEHAVE...
Quoted 4 lines
[to various people]
No, ulimit does not work. (But it helps a little.) No,
/proc/sys/vm/overcommit_memorydoes not work.
Entirely correct. ulimit certainly makes it much harder for a single runaway process to take down important parts of the system - now why doesn't $(MAJOR_DISTRO_VENDOR) set it up by default? NetBSD does. It's not an infallible solution by any means, but it sure does help.
I just asked a friend to run my test program on his NetBSD box - it ran into ulimit and malloc() returned 0. Setting ulimit on my RH 6.2 box - which defaults to unlimited - also caused it to fail gracefully.
Quoted 6 lines
[to Alan]
Quoted 1 line
Nobody feels its very important because nobody has implemented it.
Yes, that is the right response.
What can one say? One can only do.
Ah, but what does one do? Badger major distro vendors to set ulimit properly by default? Improve the OOM-killer so it gives less "badness" to low-UID processes? Implement an early-failure mechanism for malloc(), so hard OOM is not hit except by an extremely determined process (or set of processes)?
Personally, I think all of the above. Your views may differ.
Hmm... "if ( freemem < (size_of_mallocing_process / 20) ) fail_to_allocate;"
Seems like a reasonable soft limit - processes which have already got lots of RAM can probably stand not to have that little bit more and can be curbed more quickly. Processes with less probably don't deserve to die and furthermore are less likely to be engineered to handle malloc() failure, so failure only occurs closer to the mark. In this scenario OOM killing (which is, after all, a last resort) should trigger rarely and simple malloc() failure (which userspace apps can cope with more easily) is an early-warning and prevention system.
Comments?
--------------------------------------------------------------
from: Jonathan "Chromatix" Morton mail: [email protected] (not for attachments) big-mail: [email protected] uni-mail: [email protected]
The key to knowledge is not to rely on people to teach you it.
Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/
-----BEGIN GEEK CODE BLOCK-----
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -----END GEEK CODE BLOCK-----
Quoted 5 lines · Jonathan Morton
Hmm... "if ( freemem < (
size_of_mallocing_process/ 20) )fail_to_allocate;"Seems like a reasonable soft limit - processes which have already got lots of RAM can probably stand not to have that little bit more and can be curbed more quickly.
This looks like it could nicely in preventing a single process from getting out of hand and gobbling up all memory.
It won't prevent the system from a mongolian horde of processes, but nobody should expect your one-liner to fix world piece ;)
I like it, now lets test it ;)
regards,
Rik
--
Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...
http://www.surriel.com/
Quoted 9 lines · Rik van Riel
Quoted 6 lines · Jonathan Morton
Hmm... "if ( freemem < (
size_of_mallocing_process/ 20) )fail_to_allocate;"Seems like a reasonable soft limit - processes which have already got lots of RAM can probably stand not to have that little bit more and can be curbed more quickly.
Quoted 7 lines
This looks like it could nicely in preventing a single process from getting out of hand and gobbling up all memory.
It won't prevent the system from a mongolian horde of processes, but nobody should expect your one-liner to fix world piece ;)
I like it, now lets test it ;)
I thought of some things which could break it, which I want to try and deal with before releasing a patch. Specifically, I want to make freepages.min sacrosanct, so that malloc() *never* tries to use it. This should be fairly easy to implement - simply subtract freepages.min from the freemem part. An even nicer way would be to subtract freepages.low (or some similar value) instead of freepages.min for non-root or non-privileged processes.
BTW, is the 'current' pointer always valid when vm_enough_memory() is called? If so, I can remove one redundant check.
My NetBSD friend appears to have found code in the BSD kernel which sets up ulimit values sensibly by default - eg. it's not handled by the boot scripts. Presumably a root process is capable of changing the limits, but I'm guessing that sensible defaults in the kernel have to be a Good Thing�. Comments?
--------------------------------------------------------------
from: Jonathan "Chromatix" Morton mail: [email protected] (not for attachments) big-mail: [email protected] uni-mail: [email protected]
The key to knowledge is not to rely on people to teach you it.
Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/
-----BEGIN GEEK CODE BLOCK-----
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -----END GEEK CODE BLOCK-----
Quoted 7 lines
I thought of some things which could break it, which I want to try and deal with before releasing a patch. Specifically, I want to make freepages.min sacrosanct, so that
malloc()*never* tries to use it. This should be fairly easy to implement - simply subtract freepages.min from the freemem part. An even nicer way would be to subtract freepages.low (or some similar value) instead of freepages.min for non-root or non-privileged processes.
Hmm, interesting. Even with my modification - which means that vm_enough_memory() will always return false if the allocation would clobber freepages.min - I can still trigger OOM quite easily. Even with no swap on my box, there's a lot of disk activity, probably due to there being virtually no disk cache left - could the generation of disk buffer and cache pages be bypassing vm_enough_memory()? If so, would using freepages.low as the threshold rather than freepages.min help at all? (or have I got everything muddled...)
--------------------------------------------------------------
from: Jonathan "Chromatix" Morton mail: [email protected] (not for attachments) big-mail: [email protected] uni-mail: [email protected]
The key to knowledge is not to rely on people to teach you it.
Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/
-----BEGIN GEEK CODE BLOCK-----
Version 3.12
GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -----END GEEK CODE BLOCK-----
Quoted 1 line · On Sat, 24 Mar 2001 [email protected] wrote:
No, ulimit does not work. (But it helps a little.)
no, not perfect, i very much agree. but in daily usage it reduces chance of OOM to close to 0.
Quoted 1 line
No,
/proc/sys/vm/overcommit_memorydoes not work.
that's because it disables the very rough resource checking that linux has. it makes OOM even easier to achieve:
mm/mmap.c::vm_enough_memory():
/* Sometimes we want to use more memory than we have. */
if (sysctl_overcommit_memory)
return 1;it doesn't make linux go into a 'non-overcommit' mode, cause linux does not have the accounting to cover it...
solution according to more knowledgable folks than i, sysadmin, is better accounting so that vm_enough_memory can be more accurate rather than developing an all-seeing oom_killer().
Quoted 1 line
Andries
regards,
Signature · 6 lines
Paul Jakma [email protected] [email protected] PGP5 key: http://www.clubi.ie/jakma/publickey.txt ------------------------------------------- Fortune: "We are on the verge: Today our program proved Fermat's next-to-last theorem." -- Epigrams in Programming, ACM SIGPLAN Sept. 1982