Skip to contentSkip to conversation

distributed shared memory / mmap

DISCUSSION Martin Uecker 5 participants 5 messages Aug 20, 2008

lore
Share & tools
lore
#0 lore raw

Hi!

Is there any easy way to get shared memory between to processes on different machines? It seems possible using mmap on a file system like GFS on top of a distributed block device.

Sincerly,
Martin Uecker

#1 lore raw

Hi,

Quoted 11 lines · Martin Uecker

Hi!

Is there any easy way to get shared memory between to processes on different machines? It seems possible using mmap on a file system like GFS on top of a distributed block device.

Sincerly,
Martin Uecker

Yes, you can do it using GFS/GFS2 and it should work. On the other hand, don't do it. Its probably not what you want and unlikely to give you good performance since the locking is "per inode" and thus you'll generate a lot of cache flushing if there is any appreciable amount of writing to the shared region. If you have a read mostly (or only) workload, then it will work much better.

There are often better solutions though, depending on the application in question,

Steve.

#2 lore raw
Quoted 14 lines · Steven Whitehouse

Hi,

Quoted 11 lines · Martin Uecker

Hi!

Is there any easy way to get shared memory between to processes on different machines? It seems possible using mmap on a file system like GFS on top of a distributed block device.

Sincerly,
Martin Uecker

Quoted 6 lines

Yes, you can do it using GFS/GFS2 and it should work. On the other hand, don't do it. Its probably not what you want and unlikely to give you good performance since the locking is "per inode" and thus you'll generate a lot of cache flushing if there is any appreciable amount of writing to the shared region. If you have a read mostly (or only) workload, then it will work much better.

Alternatively, you can use kernel-level DSM like provided by Kerrighed. Kerrighed provides cluster-wide SYSV shm segments, which sharing is handled at the page level -> far better than the inode level.

You may find this solution not-so-easy though...

Regards,

Louis

Signature · 4 lines
Dr Louis Rilling			Kerlabs
Skype: louis.rilling			Batiment Germanium
Phone: (+33|0) 6 80 89 08 23		80 avenue des Buttes de Coesmes
http://www.kerlabs.com/			35700 Rennes
#3 lore raw
In reply to Martin Uecker
Quoted 8 lines · Martin Uecker

Hi!

Is there any easy way to get shared memory between to processes on different machines? It seems possible using mmap on a file system like GFS on top of a distributed block device.

There is hardware available, designed just for this. Reflective Memory pci cards.

Mark

#4 lore raw
Quoted 10 lines · Mark Hounschell
Quoted 8 lines · Martin Uecker

Hi!

Is there any easy way to get shared memory between to processes on different machines? It seems possible using mmap on a file system like GFS on top of a distributed block device.

Quoted 5 lines

There is hardware available, designed just for this. Reflective Memory pci cards.

Mark
--

Check out "Fabric." There is even an InfiniBand driver in the kernel. Fabric attached memory is used for HPC (High Performance Computing). It's been around for a couple of years and is fairly mature.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.22.1 on an i686 machine (4785.87 BogoMips). My book : http://www.AbominableFirebug.com/
_

****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.