[2.4.26] overcommit_memory documentation clarification
Hi,
The documentation of overcommit_memory in Documentation/sysctl/vm.txt should be clarified, as with the following simple program, malloc() never returns 0 on an official 2.4.26 kernel, even if overcommit_memory has been set to 0. Running it has the effect of having random processes killed, and eventually this process itself.
/* $Id: malloc.c 2753 2004-03-16 15:23:09Z lefevre $
*
* malloc() test
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define ONEMB 1048576
int main (void)
{
char *p;
int i;
for (i = 1; (p = malloc(ONEMB)) != NULL; i++)
{
printf ("Got %d MB\n", i);
memset (p, 0, ONEMB);
} printf ("malloc() failed - OK\n");
return 0;
}
(After some discussions, it appears not to be a bug in the glibc
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=247300>.)
Signature · 4 lines
Vincent Lefèvre <[email protected]> - Web: <http://www.vinc17.org/> 100% validated (X)HTML - Acorn / RISC OS / ARM, free software, YP17, Championnat International des Jeux Mathématiques et Logiques, etc. Work: CR INRIA - computer arithmetic / SPACES project at LORIA
Quoted 7 lines · Vincent Lefevre
Hi,
The documentation of
overcommit_memoryinDocumentation/sysctl/vm.txtshould be clarified, as with the following simple program,malloc()never returns 0 on an official 2.4.26 kernel, even ifovercommit_memoryhas been set to 0. Running it has the effect of having random processes killed, and eventually this process itself.
What made you think that malloc would return 0 if the system was "out of memory??" Malloc will return NULL, which is not 0 BTW, if you are out of address-space or have corrupted it by writing past a previous allocation. Malloc's return value is a void *. It should be compared against NULL, not zero.
When malloc() needs new "memory". It just asks the kernel to set the new break address or, in the case of mmap() mallocs, asks to extend a mapped region. Until somebody actually uses those regions, you haven't used any memory. So there is no way for malloc() to "know" ahead of time.
If you run a malloc() bomb from the root account you should end up killing off a lot of processes. If you run it from a normal user account, and you have set the user's resource quotas properly, only the user should get into trouble.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.26 on an i686 machine (5557.45 BogoMips).
Note 96.31% of all statistics are fiction.
Quoted 5 lines · Richard B. Johnson
What made you think that malloc would return 0 if the system was "out of memory??" Malloc will return NULL, which is not 0 BTW, if you are out of address-space or have corrupted it by writing past a previous allocation. Malloc's return value is a void *. It should be compared against NULL, not zero.
You are wrong. They are the same value (this is required by the ISO C standard), i.e. both NULL == (void *) 0 and NULL == 0 must be true.
Quoted 5 lines
When
malloc()needs new "memory". It just asks the kernel to set the new break address or, in the case ofmmap()mallocs, asks to extend a mapped region. Until somebody actually uses those regions, you haven't used any memory. So there is no way formalloc()to "know" ahead of time.
Again, you are wrong. The goal of malloc is to reserve memory. This can be seen as used memory. If the implementation behaves differently, then it is broken.
Quoted 4 lines
If you run a
malloc()bomb from the root account you should end up killing off a lot of processes. If you run it from a normal user account, and you have set the user's resource quotas properly, only the user should get into trouble.
The quotas are set properly (i.e. there are no quotas).
Signature · 4 lines
Vincent Lefèvre <[email protected]> - Web: <http://www.vinc17.org/> 100% validated (X)HTML - Acorn / RISC OS / ARM, free software, YP17, Championnat International des Jeux Mathématiques et Logiques, etc. Work: CR INRIA - computer arithmetic / SPACES project at LORIA
Quoted 1 line · Vincent
NULL == (void *) 0 and NULL == 0 must be true
Yes - NULL is compares equal to both (void *)0 and 0. No - not necessarily the _same_ value - one could be on a system with 32 bit ints, 64 bit pointers, for example.
Quoted 1 line
The goal of malloc is to reserve memory.
It's up to the kernel whether sbrk (used by malloc to obtain virtual address space) reserves memory or not.
Check out:
/proc/sys/vm/overcommit_memory Documentation/sysctl/vm.txt - overcommit_memory
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373Quoted 5 lines · Paul Jackson
Quoted 1 line · Vincent
NULL == (void *) 0 and NULL == 0 must be true
Yes - NULL is compares equal to both (void *)0 and 0. No - not necessarily the _same_ value - one could be on a system with 32 bit ints, 64 bit pointers, for example.
And so?
Quoted 3 lines
Quoted 1 line
The goal of malloc is to reserve memory.
It's up to the kernel whether sbrk (used by malloc to obtain virtual address space) reserves memory or not.
More old_mmap than brk (BTW, I forgot to say that this was on an x86 machine, I don't know if this matters...).
Quoted 3 lines
Check out:
/proc/sys/vm/overcommit_memoryDocumentation/sysctl/vm.txt-overcommit_memory
But the documentation is wrong (on an official 2.4.26 kernel). It seems that there is no way to get malloc() always return 0 when there isn't enough memory, even in simple cases (see my program posted in the first message).
Signature · 4 lines
Vincent Lefèvre <[email protected]> - Web: <http://www.vinc17.org/> 100% validated (X)HTML - Acorn / RISC OS / ARM, free software, YP17, Championnat International des Jeux Mathématiques et Logiques, etc. Work: CR INRIA - computer arithmetic / SPACES project at LORIA
Quoted 16 lines · Vincent Lefevre
Quoted 5 lines · Paul Jackson
Quoted 1 line · Vincent
NULL == (void *) 0 and NULL == 0 must be true
Yes - NULL is compares equal to both (void *)0 and 0. No - not necessarily the _same_ value - one could be on a system with 32 bit ints, 64 bit pointers, for example.
And so?
Quoted 3 lines
Quoted 1 line
The goal of malloc is to reserve memory.
It's up to the kernel whether sbrk (used by malloc to obtain virtual address space) reserves memory or not.
More
old_mmapthan brk (BTW, I forgot to say that this was on an x86 machine, I don't know if this matters...).
Quoted 8 lines
Quoted 3 lines
Check out:
/proc/sys/vm/overcommit_memoryDocumentation/sysctl/vm.txt-overcommit_memoryBut the documentation is wrong (on an official 2.4.26 kernel). It seems that there is no way to get
malloc()always return 0 when there isn't enough memory, even in simple cases (see my program posted in the first message).
Right.
We should or merge Alan's strict-overcommit patches (from RH's tree), or fix the documentation.
Marc-Christian Petersen has a patch to fixup the documentation.
On Thursday 27 May 2004 14:20, Marcelo Tosatti wrote:
Hi Marcelo,
Quoted 2 lines
We should or merge Alan's strict-overcommit patches (from RH's tree), or fix the documentation.
I vote for the strict-overcommit thing. Do you have that handy for your 2.4-bk?
Alan, prolly you? Or do we have to extract it from linux-2.4.21-selected-ac-bits.patch? ;)
ciao, Marc
Quoted 3 lines · Marc-Christian Petersen
On Thursday 27 May 2004 14:20, Marcelo Tosatti wrote:
Hi Marcelo,
Hi Marc,
Quoted 8 lines
Quoted 2 lines
We should or merge Alan's strict-overcommit patches (from RH's tree), or fix the documentation.
I vote for the strict-overcommit thing. Do you have that handy for your 2.4-bk?
Alan, prolly you? Or do we have to extract it from linux-2.4.21-selected-ac-bits.patch? ;)
Alan is busy with other things.
The strict overcommit fixes need to be extraced and tested.
Dave Jones told me about it the other day.
Still haven't found the time to download RH's srpm.
Quoted 11 lines · Marcelo Tosatti
Quoted 6 lines
Quoted 2 lines
We should or merge Alan's strict-overcommit patches (from RH's tree), or fix the documentation.
I vote for the strict-overcommit thing. Do you have that handy for your 2.4-bk?
Alan, prolly you? Or do we have to extract it from linux-2.4.21-selected-ac-bits.patch? ;)Alan is busy with other things.
The strict overcommit fixes need to be extraced and tested. Dave Jones told me about it the other day.
Still haven't found the time to download RH's srpm.
The overcommit bits in the Fedora SRPM come directly from the 2.4.22-ac1 patch. You're better off just grabbing Bero's -pac patch. He has a 2.4.26 patch, which should save you rediffing, just chop out the relevant bits.
http://kernel.org/pub/linux/kernel/people/bero/2.4/2.4.27/
Dave
Quoted 1 line · Marcelo Tosatti
The strict overcommit fixes need to be extraced and tested.
Bero's tree has them. If you merge it it is probably good to make the small changes so it works like 2.6