/proc/pid/maps containg anonymous maps that have PROT_NONE
Hi,
I noticed that there are (quite large) entries in /proc/pid/maps that have PROT_NONE, right after an existing mapping:
7fffe4000000-7fffe406a000 rw-p 7fffe4000000 00:00 0 7fffe406a000-7fffe8000000 ---p 7fffe406a000 00:00 0 7ffff76d1000-7ffff76e0000 r-xp 00000000 09:03 260750 /lib/libbz2.so.1.0.4
7ffff76e0000-7ffff78df000 ---p 0000f000 09:03 260750 /lib/libbz2.so.1.0.4
I don't mind that 2Mb map, but what is 7fffe406a000-7fffe8000000 ---p ? (63M)
Is it coming from glibc mapping memory as PROT_NONE, and using mprotect/madvise to make it writable, and then caching the mappings for future use, rather than freeing them?
/proc/pid/smaps shows:
7fffe406a000-7fffe8000000 ---p 7fffe406a000 00:00 0
Size: 65112 kB Rss: 0 kB Pss: 0 kB Shared_Clean: 0 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 0 kB Referenced: 0 kB Swap: 0 kB
I straced the program creating these, and I couldn't find anything with 7fffe406a000, but only before that address:
[pid 31928] mprotect(0x7fffe4000000, 135168, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4021000, 131072, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4041000, 32768, PROT_READ|PROT_WRITE) = 0
[pid 31928] mprotect(0x7fffe4049000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe404a000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe404b000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe404c000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe404d000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe404e000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe404f000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4050000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4051000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4052000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4053000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4054000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4055000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4056000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4057000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4058000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe4059000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe405a000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe405b000, 4096, PROT_READ|PROT_WRITE) = 0 [pid 31928] mprotect(0x7fffe405c000, 4096, PROT_READ|PROT_WRITE) = 0
[pid 31928] mprotect(0x7fffe405d000, 53248, PROT_READ|PROT_WRITE) = 0 [pid 31928] madvise(0x7fffe4069000, 4096, 0x4 /* MADV_??? */) = 0 [pid 31928] madvise(0x7fffe4021000, 294912, 0x4 /* MADV_??? */) = 0 [pid 31928] madvise(0x7fffe4021000, 4096, 0x4 /* MADV_??? */) = 0
There is an mprotect and madvise that end at 0x7fffe406a000. Those mprotects and madvise are coming from glibc. Its strange that I don't see the mmap only the mprotect, but I used strace -f.
This happens on:
Linux debian 2.6.26-1-amd64 #1 SMP Thu Oct 9 14:16:53 UTC 2008 x86_64 GNU/Linux
strace is here:
http://edwintorok.googlepages.com/log2.bz2
Best regards,
--Edwin
Quoted 10 lines · Török Edwin
I noticed that there are (quite large) entries in
/proc/pid/mapsthat havePROT_NONE, right after an existing mapping:
7fffe4000000-7fffe406a000 rw-p7fffe400000000:00 0 7fffe406a000-7fffe8000000 ---p7fffe406a00000:00 0 7ffff76d1000-7ffff76e0000 r-xp 00000000 09:03 260750/lib/libbz2.so.1.0.4
7ffff76e0000-7ffff78df000 ---p0000f00009:03 260750/lib/libbz2.so.1.0.4
Quoted 6 lines
I don't mind that 2Mb map, but what is 7fffe406a000-7fffe8000000 ---p ? (63M)
Is it coming from glibc mapping memory as
PROT_NONE, and using mprotect/madvise to make it writable, and then caching the mappings for future use, rather than freeing them?
mmap PROT_NONE to reserve an arena, munmap to trim off top and bottom, mprotect to make areas read+writable, madvise 0x4 to say MADV_DONTNEED on some parts. gcc? Or the application itself (clamd) and its libs?
Quoted 4 lines
I straced the program creating these, and I couldn't find anything with
7fffe406a000, but only before that address:[pid 31928]
mprotect(0x7fffe4000000, 135168, PROT_READ|PROT_WRITE)= 0
[snipped]
Quoted 6 lines
[pid 31928]
madvise(0x7fffe4021000, 4096, 0x4 /* MADV_??? */)= 0There is an mprotect and madvise that end at
0x7fffe406a000. Those mprotects and madvise are coming from glibc. Its strange that I don't see the mmap only the mprotect, but I used strace -f.
Quoted 6 lines
This happens on:
Linux debian 2.6.26-1-amd64 #1 SMP Thu Oct 9 14:16:53 UTC 2008x86_64GNU/Linuxstrace is here:
http://edwintorok.googlepages.com/log2.bz2
Just before your first mprotect above there's:
[pid 31928] mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0 <unfinished ...> [pid 31938] futex(0x7ffff6e489e0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 31928] <... mmap resumed> ) = 0x7fffe2bd3000
which maps 7fffe2bd3000-7fffeabd3000; then
[pid 31928] munmap(0x7fffe2bd3000, 21155840 <unfinished ...>
[pid 31938] <... futex resumed> ) = 0
[pid 31928] <... munmap resumed> ) = 0
which unmaps 7fffe2bd3000-7fffe4000000; and then
[pid 31928] munmap(0x7fffe8000000, 45953024) = 0
which unmaps 7fffe8000000-7fffeabd3000.
So it's trimming off the rough edges to leave 7fffe4000000-7fffe8000000 mapped PROT_NONE, then mprotecting what it needs of that.
Why does it mmap too much then trim it down? Perhaps it's trying to minimize pagetable usage, perhaps it's internally convenient to base on rounded addresses, I don't know.
But the mmap is there: just easily overlooked because of the way it munmaps too (with strace showing hex addresses but decimal sizes).
Hugh
Quoted 57 lines · Hugh Dickins
Quoted 9 lines · Török Edwin
I noticed that there are (quite large) entries in
/proc/pid/mapsthat havePROT_NONE, right after an existing mapping:
7fffe4000000-7fffe406a000 rw-p7fffe400000000:00 0 7fffe406a000-7fffe8000000 ---p7fffe406a00000:00 0 7ffff76d1000-7ffff76e0000 r-xp 00000000 09:03 260750/lib/libbz2.so.1.0.4
7ffff76e0000-7ffff78df000 ---p0000f00009:03 260750/lib/libbz2.so.1.0.4Quoted 7 lines
I don't mind that 2Mb map, but what is 7fffe406a000-7fffe8000000 ---p ? (63M)
Is it coming from glibc mapping memory as
PROT_NONE, and using mprotect/madvise to make it writable, and then caching the mappings for future use, rather than freeing them?mmap
PROT_NONEto reserve an arena, munmap to trim off top and bottom, mprotect to make areas read+writable, madvise0x4to sayMADV_DONTNEEDon some parts. gcc? Or the application itself (clamd) and its libs?Quoted 5 lines
I straced the program creating these, and I couldn't find anything with
7fffe406a000, but only before that address:[pid 31928]
mprotect(0x7fffe4000000, 135168, PROT_READ|PROT_WRITE)= 0[snipped]
Quoted 6 lines
[pid 31928]
madvise(0x7fffe4021000, 4096, 0x4 /* MADV_??? */)= 0There is an mprotect and madvise that end at
0x7fffe406a000. Those mprotects and madvise are coming from glibc. Its strange that I don't see the mmap only the mprotect, but I used strace -f.Quoted 7 lines
This happens on:
Linux debian 2.6.26-1-amd64 #1 SMP Thu Oct 9 14:16:53 UTC 2008x86_64GNU/Linuxstrace is here:
http://edwintorok.googlepages.com/log2.bz2Just before your first mprotect above there's:
[pid 31928] mmap(NULL, 134217728,
PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0 <unfinished ...> [pid 31938] futex(0x7ffff6e489e0,FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 31928] <... mmap resumed> ) =0x7fffe2bd3000which maps 7fffe2bd3000-7fffeabd3000; then
[pid 31928] munmap(0x7fffe2bd3000, 21155840 <unfinished ...>
[pid 31938] <... futex resumed> ) = 0
[pid 31928] <... munmap resumed> ) = 0
Quoted 10 lines
which unmaps 7fffe2bd3000-7fffe4000000; and then
[pid 31928]
munmap(0x7fffe8000000, 45953024)= 0which unmaps 7fffe8000000-7fffeabd3000.
So it's trimming off the rough edges to leave 7fffe4000000-7fffe8000000 mapped
PROT_NONE, then mprotecting what it needs of that.
Thanks I completely missed the mmap, because I grepped for 7fffe4000000.
Quoted 6 lines
Why does it mmap too much then trim it down? Perhaps it's trying to minimize pagetable usage, perhaps it's internally convenient to base on rounded addresses, I don't know.
But the mmap is there: just easily overlooked because of the way it munmaps too (with strace showing hex addresses but decimal sizes).
I will get some stacktraces and figure out, know that I know which mmap to look for (the one with MAP_NORESERVE).
Best regards,
--Edwin
Quoted 15 lines · Török Edwin
Quoted 13 lines · Hugh Dickins
Quoted 2 lines
mmap
PROT_NONEto reserve an arena, munmap to trim off top and bottom, mprotect to make areas read+writable, madvise0x4to sayMADV_DONTNEEDon some parts. gcc? Or the application itself (clamd) and its libs?Why does it mmap too much then trim it down? Perhaps it's trying to minimize pagetable usage, perhaps it's internally convenient to base on rounded addresses, I don't know.
Quoted 7 lines
Quoted 3 lines
But the mmap is there: just easily overlooked because of the way it munmaps too (with strace showing hex addresses but decimal sizes).
I will get some stacktraces and figure out, know that I know which mmap to look for (the one with
MAP_NORESERVE).
I found it, glibc: arena.c:669 does it, there's a comment explaining why:
/* If consecutive mmap (0, HEAP_MAX_SIZE << 1, ...) calls return decreasing
addresses as opposed to increasing, new_heap would badly fragment the address space. In that case remember the second HEAP_MAX_SIZE part aligned to HEAP_MAX_SIZE from last mmap (0, HEAP_MAX_SIZE << 1, ...) call (if it is already aligned) and try to reuse it next time. We need no locking for it, as kernel ensures the atomicity for us - worst case we'll call mmap (addr, HEAP_MAX_SIZE, ...) for some value of addr in multiple threads, but only one will succeed. */
Anyway it is MAP_NORESERVE, and PROT_NONE so it doesn't waste physical or swap memory.
Best regards,
--Edwin
Quoted 14 lines
[pid 31928] mmap(NULL, 134217728,
PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0 <unfinished ...> [pid 31938] futex(0x7ffff6e489e0,FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 31928] <... mmap resumed> ) =0x7fffe2bd3000which maps 7fffe2bd3000-7fffeabd3000; then
[pid 31928] munmap(0x7fffe2bd3000, 21155840 <unfinished ...>
[pid 31938] <... futex resumed> ) = 0
[pid 31928] <... munmap resumed> ) = 0which unmaps 7fffe2bd3000-7fffe4000000; and then
[pid 31928]
munmap(0x7fffe8000000, 45953024)= 0
Quoted 8 lines
which unmaps 7fffe8000000-7fffeabd3000.
So it's trimming off the rough edges to leave 7fffe4000000-7fffe8000000 mapped
PROT_NONE, then mprotecting what it needs of that.Why does it mmap too much then trim it down? Perhaps it's trying to minimize pagetable usage, perhaps it's internally convenient to base on rounded addresses, I don't know.
Yup.
glibc malloc's arena need heap size alignment.
it cause that struct arena address can calcurate "arena = any_ptr & ~(HEAP_SIZE-1)" as old linux's "current" macro calculation.
Quoted 2 lines
But the mmap is there: just easily overlooked because of the way it munmaps too (with strace showing hex addresses but decimal sizes).