Skip to contentSkip to conversation

mprotect() failed: Cannot allocate memory

DISCUSSION Paweł Sikora 4 participants 9 messages Apr 20, 2010 – Apr 21, 2010

lore
Share & tools
lore
#1 lore raw
In reply to Paweł Sikora
Quoted 1 line · On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:

i'm trying to debug an ugly application with ElectricFence.

electricfence does a lot of ugly memory tricks to do its thing, including, but not limited to, overriding memory related symbols. best to seek help from the electricfence authors.

unless of course, things still fail when you dont use electricfence. -mike

#0 lore raw

hi,

i'm trying to debug an ugly application with ElectricFence.
in fact, on x86-64 box with 8GB ram and 16GB swap i'm getting following error:

"ElectricFence Exiting: mprotect() failed: Cannot allocate memory"

the program has been compiled with gcc-4.5, glibc-2.11.1, kernel-2.6.32. did you ever come across such (kernel/glibc) limitations?

here's a simple testcase which triggs -ENOMEM in mprotect().

#define N 100
#include<stdlib.h>

double **bm;

int main(){
    int i;
    long NN = 4*N*N;
    int kmax=100;

        bm = (double **)malloc((time_t)NN*sizeof(double *));
        for(i=0; i<NN; ++i){
                bm[i] = (double*)malloc((time_t)kmax*sizeof(double));
        }

        for(i=0; i<NN; ++i){
                free(bm[i]);
        }
        free(bm);
        return 0;
}

thanks for any hint,

BR,
Pawel.

#2 lore raw
In reply to Mike Frysinger
Quoted 6 lines · On Wednesday 21 April 2010 01:17:22 Mike Frysinger wrote:
Quoted 1 line · On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:

i'm trying to debug an ugly application with ElectricFence.

electricfence does a lot of ugly memory tricks to do its thing, including, but not limited to, overriding memory related symbols. best to seek help from the electricfence authors.

so, let's avoid EF and run following test:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

void* my_alloc( size_t n )

{
        size_t ps = getpagesize();
        printf( "request for %Zd bytes => ", n );
        /* alloc PAGE_SIZE + n */
        char* p = mmap( 0, ps + n, PROT_READ | PROT_WRITE, MAP_SHARED | 
MAP_ANONYMOUS, -1, 0 );
        if ( p == MAP_FAILED )
                __builtin_abort();
        /* block guard page */
        int rc = mprotect( p, ps, PROT_NONE );
        if ( rc != 0 )
                __builtin_abort();
        char* q = p + ps;
        printf( "guard page @ %p, allocated region @ %p\n", p, q );
        return q;
}

int main()

{
        #define N 100  
        size_t NN = 4*100*100;
        size_t kmax = 100;
        int i;

        double **bm = (double **)my_alloc( NN * sizeof( double* ) );
        for( i = 0; i < NN; ++i )
        {
                bm[ i ] = (double*)my_alloc( kmax * sizeof( double ) );
        }
        // leak...
        return 0;
}

and the result is...

(...)
mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x7f5fd97df000
mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

#3 lore raw
Quoted 1 line · On Tuesday 20 April 2010 19:44:18 Paweł Sikora wrote:

so, let's avoid EF and run following test:

good, the info is useful now

Quoted 4 lines

mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x7f5fd97df000
mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

since you're using funcs that go straight to the kernel, this most likely isnt related to glibc at all, so sticking to LKML would be best. the next step would be to look at the linux internals and find out why this is failing. -mike

#4 lore raw
In reply to Paweł Sikora
Quoted 49 lines · Paweł Sikora
Quoted 6 lines · On Wednesday 21 April 2010 01:17:22 Mike Frysinger wrote:
Quoted 1 line · On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:

i'm trying to debug an ugly application with ElectricFence.

electricfence does a lot of ugly memory tricks to do its thing, including, but not limited to, overriding memory related symbols. best to seek help from the electricfence authors.

so, let's avoid EF and run following test:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

void* my_alloc( size_t n )

{
        size_t ps = getpagesize();
        printf( "request for %Zd bytes => ", n );
        /* alloc PAGE_SIZE + n */
        char* p = mmap( 0, ps + n, PROT_READ | PROT_WRITE, MAP_SHARED | 
MAP_ANONYMOUS, -1, 0 );
        if ( p == MAP_FAILED )
                __builtin_abort();
        /* block guard page */
        int rc = mprotect( p, ps, PROT_NONE );
        if ( rc != 0 )
                __builtin_abort();
        char* q = p + ps;
        printf( "guard page @ %p, allocated region @ %p\n", p, q );
        return q;
}

int main()

{
        #define N 100  
        size_t NN = 4*100*100;
        size_t kmax = 100;
        int i;

        double **bm = (double **)my_alloc( NN * sizeof( double* ) );
        for( i = 0; i < NN; ++i )
        {
                bm[ i ] = (double*)my_alloc( kmax * sizeof( double ) );
        }
        // leak...
        return 0;
}
Quoted 6 lines

and the result is...

(...)
mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x7f5fd97df000
mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

Have you checked available memory on your system ? Or user limit ?

You test program is going to allocate
79 + 1 pages for bm
1 + 1 for each double arrays (x 40000)

So in the end your program is allocating 80080 pages, so about 312MBytes.

It not that big for a 64bits system.

Check limits such as
-d the maximum size of a process's data segment
-l the maximum size a process may lock into memory
-m the maximum resident set size

Regards.

Signature · 1 line
Yann Droneaud
#5 lore raw

Dnia 21-04-2010 o 11:22:19 Yann Droneaud <[email protected]> napisał(a):

Quoted 71 lines · Paweł Sikora
Quoted 51 lines · On Wednesday 21 April 2010 01:17:22 Mike Frysinger wrote:
Quoted 4 lines · On Tuesday 20 April 2010 19:05:20 Paweł Sikora wrote:
Quoted 1 line

i'm trying to debug an ugly application with ElectricFence.

electricfence does a lot of ugly memory tricks to do its thing,

including,

Quoted 1 line

but not limited to, overriding memory related symbols. best to seek

help

Quoted 1 line

from the electricfence authors.

so, let's avoid EF and run following test:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

void* my_alloc( size_t n )

{
        size_t ps = getpagesize();
        printf( "request for %Zd bytes => ", n );
        /* alloc PAGE_SIZE + n */
        char* p = mmap( 0, ps + n, PROT_READ | PROT_WRITE, MAP_SHARED |
MAP_ANONYMOUS, -1, 0 );
        if ( p == MAP_FAILED )
                __builtin_abort();
        /* block guard page */
        int rc = mprotect( p, ps, PROT_NONE );
        if ( rc != 0 )
                __builtin_abort();
        char* q = p + ps;
        printf( "guard page @ %p, allocated region @ %p\n", p, q );
        return q;
}

int main()

{
        #define N 100
        size_t NN = 4*100*100;
        size_t kmax = 100;
        int i;

        double **bm = (double **)my_alloc( NN * sizeof( double* ) );
        for( i = 0; i < NN; ++i )
        {
                bm[ i ] = (double*)my_alloc( kmax * sizeof( double ) );
        }
        // leak...
        return 0;
}
Quoted 6 lines

and the result is...

(...)
mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x7f5fd97df000
mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

Have you checked available memory on your system ? Or user limit ?

You test program is going to allocate
79 + 1 pages for bm
1 + 1 for each double arrays (x 40000)

So in the end your program is allocating 80080 pages, so about 312MBytes.

It not that big for a 64bits system.

Check limits such as

Review

Quoted 3 lines
(quoted diff) +0 -3
-d	the maximum size of a process's data segment
-l	the maximum size a process may lock into memory
-m	the maximum resident set size
$ ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        8192
-c: core file size (blocks)    0
-m: resident set size (kbytes) unlimited
-u: processes                  unlimited
-n: file descriptors           1024
-l: locked-in-memory size (kb) 64
-v: address space (kb)         unlimited
-x: file locks                 unlimited
-i: pending signals            64024
-q: bytes in POSIX msg queues  819200
-e: max nice                   0
-r: max rt priority            0
$ free -m
             total       used       free     shared    buffers     cached
Mem:          8004       1779       6224          0         36       1228
-/+ buffers/cache:        514       7489
Swap:        15625          0      15625

imho the testcase has enough hardware resources.

#6 lore raw
In reply to Yann Droneaud

Dnia 21-04-2010 o 11:22:19 Yann Droneaud <[email protected]> napisał(a):

Quoted 7 lines
Quoted 4 lines

(...)
mmap(NULL, 4896, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x7f5fd97df000
mprotect(0x7f5fd97df000, 4096, PROT_NONE) = -1 ENOMEM (Cannot allocate memory)

Have you checked available memory on your system ? Or user limit ?

Quoted 8 lines

You test program is going to allocate
79 + 1 pages for bm
1 + 1 for each double arrays (x 40000)

So in the end your program is allocating 80080 pages, so about 312MBytes.

It not that big for a 64bits system.

afaics in gdb, the mprotect fails at i=32756. it's near to 2^15. maybe some kernel data structures are full?

#7 lore raw
In reply to Paweł Sikora
Quoted 11 lines · Paweł Sikora

hi,

i'm trying to debug an ugly application with ElectricFence.
in fact, on x86-64 box with 8GB ram and 16GB swap i'm getting following error:

"ElectricFence Exiting: mprotect() failed: Cannot allocate memory"

the program has been compiled with gcc-4.5, glibc-2.11.1, kernel-2.6.32. did you ever come across such (kernel/glibc) limitations?

here's a simple testcase which triggs -ENOMEM in mprotect().

You probably depleted the max map count, see:
/proc/sys/vm/max_map_count

We have a limit on the number of maps you can have, those mprotect() calls split you maps like crazy, see also /proc/$pid/maps.

eg. change your second test program to include something like:

  char buf[128];
  snprintf(buf, sizeof(buf), "cat /proc/%d/maps", (int)getpid());
  system(buf);

at the end after lowering your NN count to fit, and observe the result of those mprotect() calls.

#8 lore raw

Dnia 21-04-2010 o 13:16:50 Peter Zijlstra <[email protected]> napisał(a):

Quoted 8 lines · Paweł Sikora
Quoted 7 lines

hi,

i'm trying to debug an ugly application with ElectricFence.
in fact, on x86-64 box with 8GB ram and 16GB swap i'm getting following error:

"ElectricFence Exiting: mprotect() failed: Cannot allocate memory"

Quoted 7 lines
Quoted 4 lines

the program has been compiled with gcc-4.5, glibc-2.11.1, kernel-2.6.32. did you ever come across such (kernel/glibc) limitations?

here's a simple testcase which triggs -ENOMEM in mprotect().

You probably depleted the max map count, see:
/proc/sys/vm/max_map_count

yes, that is the clue :)

the limit in /proc/sys/vm/max_map_count was set to 65530. with echo 128000 > /proc/sys/vm/max_map_count the testcase passes.

thanks for hint.