Skip to contentSkip to conversation

mmap vs. real memory

DISCUSSION Brian McGrew 3 participants 3 messages Sep 10, 2009 – Sep 11, 2009

lore
Share & tools
lore
#0 lore raw

Good afternoon all,

Right now, I'm in what appears [to me] to be a very tricky situation. We have several programs that all need access to the same memory. Currently, we use mmap and map to a file on disk like this:

const u_long    kCommsegSize            = 1024L * 1024L * 1024L;
const u_long    kCommsegMappedAddress   = 0xb0000000;

And then the code is:

    char *cseg_addr = reinterpret_cast<char *>(mmap(
            reinterpret_cast<char *>(kCommsegMappedAddress),
            size,
            PROT_READ  | PROT_WRITE,
            MAP_SHARED | MAP_FIXED,
            fd,
            0));
    gShmemOffset = kCommsegMappedAddress - kCommseg;
    return(cseg_addr);

And, all of this has worked great ever since we went from a physical shared memory board on a vme bus to emulating shared memory on the computer. In the beginning, kCommsegSize was only 4MB (4L * 1024L * 1024L).

Now that we are needing more and more shared memory, we're seeing random performance issues. With 16MB, 64MB and 256MB (in recent past) all was good. But now, trying to allocate 1GB of shared memory, we see it taking anywhere from .6 to 9 SEONDS to access the file. No good!

What I'm wondering and needed to do is map a chunk of mymoery (1GB today, maybe 2GB later on) so that all my apps can access it. Short of creating a ramdisk and moving the mmap'ed file to ramdisk, what is the best way to do this???

Thanks,

-brian

#1 lore raw
Quoted 9 lines · Brian McGrew

Now that we are needing more and more shared memory, we're seeing random performance issues. With 16MB, 64MB and 256MB (in recent past) all was good. But now, trying to allocate 1GB of shared memory, we see it taking anywhere from .6 to 9 SEONDS to access the file. No good!

What I'm wondering and needed to do is map a chunk of mymoery (1GB today, maybe 2GB later on) so that all my apps can access it. Short of creating a ramdisk and moving the mmap'ed file to ramdisk, what is the best way to do this???

Maybe make a ramfs partition, mount it somewhere, and create your file there? This is more efficient and simpler than creating a ramdisk.

Alternately, have you considered using hugetlbfs and mapping the shared memory with large pages?

Chris

#2 lore raw
In reply to Brian McGrew
Quoted 30 lines · Brian McGrew

Good afternoon all,

Right now, I'm in what appears [to me] to be a very tricky situation. We have several programs that all need access to the same memory. Currently, we use mmap and map to a file on disk like this:

const u_long    kCommsegSize            = 1024L * 1024L * 1024L;
const u_long    kCommsegMappedAddress   = 0xb0000000;

And then the code is:

    char *cseg_addr = reinterpret_cast<char *>(mmap(
            reinterpret_cast<char *>(kCommsegMappedAddress),
            size,
            PROT_READ  | PROT_WRITE,
            MAP_SHARED | MAP_FIXED,
            fd,
            0));
    gShmemOffset = kCommsegMappedAddress - kCommseg;
    return(cseg_addr);

And, all of this has worked great ever since we went from a physical shared memory board on a vme bus to emulating shared memory on the computer. In the beginning, kCommsegSize was only 4MB (4L * 1024L * 1024L).

Now that we are needing more and more shared memory, we're seeing random performance issues. With 16MB, 64MB and 256MB (in recent past) all was good. But now, trying to allocate 1GB of shared memory, we see it taking anywhere from .6 to 9 SEONDS to access the file. No good!

Quoted 4 lines

What I'm wondering and needed to do is map a chunk of mymoery (1GB today, maybe 2GB later on) so that all my apps can access it. Short of creating a ramdisk and moving the mmap'ed file to ramdisk, what is the best way to do this???

Were are you storing the file? For memory-only, you should use /dev/shm. Another more portable solution is to use shm_open(3), that will open the file were the system wants it.

Signature · 2 lines
Luciano Rocha <[email protected]>
Eurotux Informática, S.A. <http://www.eurotux.com/>