Repost: could ia32 mmap() allocations grow downward?
Quoted 7 lines · Wayne Whitney
BTW, if one were trying to port some code that uses
brk()directly and even frees memory that way, then it seems that with glibc'smalloc(), one could make it work by instructingmalloc()always to usemmap().
P.S. I am 100% sure that the particular application of mine that started me thinking about this, MAGMA, uses its own allocator built on top ofbrk()and never callsmalloc()itself.
If you have legacy app, how it comes that it uses mmap? And if I do not use mmap, I have nothing at 1GB:
void main() { sleep(10); brk((void*)0xBF000000); pause(); }
/proc/pidof x/maps says during sleep(10):
08048000-080a1000 r-xp 00000000 03:03 230941 /usr/src/linus/x
080a1000-080a5000 rw-p 00058000 03:03 230941 /usr/src/linus/x
080a5000-080a6000 rwxp 00000000 00:00 0
bffff000-c0000000 rwxp 00000000 00:00 0
and after brk() (which suceeded after I did ulimit -d unlimited and 'echo 1 >/proc/sys/vm/overcommit_memory') I see:
08048000-080a1000 r-xp 00000000 03:03 230941 /usr/src/linus/x
080a1000-080a5000 rw-p 00058000 03:03 230941 /usr/src/linus/x
080a5000-bf000000 rwxp 00000000 00:00 0
bffff000-c0000000 rwxp 00000000 00:00 0
So maybe MAGMA uses some API which it should not use under any circumstances... Such as that you linked it with libc6 stdio.
Best regards,
Petr Vandrovec [email protected]
Quoted 1 line · Petr Vandrovec
If you have legacy app, how it comes that it uses mmap?
Very good question. The app per se does not call mmap(), but mmap() is called once when I execute it. So it must be something from libc:
[whitney@mf1 whitney]$ ldd which magma
not a dynamic executable
[whitney@mf1 whitney]$ magma
[ . . .]
[2]+ Stopped magma
[whitney@mf1 whitney]$ cat /proc/pidof magma/maps 08048000-08afb000 r-xp 00000000 21:07 64318 magma 08afb000-08c3e000 rw-p 00ab2000 21:07 64318 magma 08c3e000-0bc54000 rwxp 00000000 00:00 0 40000000-40001000 rw-p 00000000 00:00 0 bfffd000-c0000000 rwxp ffffe000 00:00 0
Quoted 2 lines
So maybe MAGMA uses some API which it should not use under any circumstances... Such as that you linked it with libc6 stdio.
Indeed. How can I avoid the map at 0x40000000? Must I avoid using certain glibc2 functions, and then link the executable carefully to leave out their initialization routines? Or can I set some magic environment variable to tell glibc2 to mmap() the single map with MAP_FIXED at a higher addresss? Of course I could modify glibc2 so that it does all (or most) of its mmap()'s with MAP_FIXED at a higher address. Is there an alternative libc that might work out of the box or require less modification?
So it seems like for MAGMA I should be able to work around the fact that mmap()'s start at 0x40000000. But as difficulties with other programs come up here fairly regularly, I still think it makes sense to fully understand the downside of modifying the kernel to allocate mmap() VMAs going downward. If the downside is small, I think it is a good tradeoff.
Cheers, Wayne
Quoted 2 lines · Wayne Whitney
The app per se does not call
mmap(), butmmap()is called once when I execute it.
Correction: strace shows that it is called many times during startup, but only once without a corresponding munmap()>
Wayne
Quoted 5 lines · Wayne Whitney
So it seems like for MAGMA I should be able to work around the fact that mmap()'s start at
0x40000000. But as difficulties with other programs come up here fairly regularly, I still think it makes sense to fully understand the downside of modifying the kernel to allocatemmap()VMAs going downward. If the downside is small, I think it is a good tradeoff.
My fear is that you may encounter an indefinite number of buggy apps, which expect an mmap() to follow the mmap() before: easy bug to commit, and to go unnoticed, until you reverse the layout.
As to where to place your stack: I don't know what assumptions are made elsewhere, but a seemingly good place is just below the program's text at 0x08048000. People sometimes ask why i386 ELF text is usually placed
there: I think it's a convention of some other UNIX implementations, which used to put stack below text and data above it, all sharing the one page table (if it's a smallish process).
Hugh
Quoted 3 lines · Hugh Dickins
My fear is that you may encounter an indefinite number of buggy apps, which expect an
mmap()to follow themmap()before: easy bug to commit, and to go unnoticed, until you reverse the layout.
Hmm, so which is more important to support, buggy users of (unguaranteed side effects of) the new interface, or users of the legacy interface? I can see the argument that that the buggy users of the new interface are more important. Maybe CONFIG_MMAP_GROWS_DOWNWARDS, or a /proc entry?
Wayne
Quoted 10 lines · Wayne Whitney
Quoted 3 lines · Hugh Dickins
My fear is that you may encounter an indefinite number of buggy apps, which expect an
mmap()to follow themmap()before: easy bug to commit, and to go unnoticed, until you reverse the layout.Hmm, so which is more important to support, buggy users of (unguaranteed side effects of) the new interface, or users of the legacy interface? I can see the argument that that the buggy users of the new interface are more important. Maybe
CONFIG_MMAP_GROWS_DOWNWARDS, or a /proc entry?
Hard to know until you try it: my fear may prove groundless, or experience may discourage you from the exercise completely.
Quick guess is that what you'd really want in the end is not a CONFIG option or /proc tunable, but some mark in an ELF section for what behaviour that particular executable wants.
I'm reluctant to call wanting a large virtual address space buggy; but expecting contiguous ascending mmaps (without MAP_FIXED) is buggy.
Hugh