pohmelfs: call for inclusion
Hi
I'm please to announce new and completely rewritten distributed filesystem - POHMELFS
It went a long way from parallel NFS design which lived in drivers/staging/pohmelfs for years effectively without usage case - that design was dead.
New pohmelfs uses elliptics network [1] as its storage backend, which was proved as effective distributed system. Elliptics is used in production in Yandex search company for several years now and clusters range from small (like 6 nodes in 3 datacenters to host 15 billions of small files or hundred of nodes to scale to 1 Pb used for streaming).
We start to cook up 2 small clusters for production pohmelfs testing - one of them is largest in East Europe mirror site, another one is used for internal package storage.
Pohmelfs is just a POSIX frontend to elliptics. It supports hardlinks, symlinks, data checksums, multiple copies and so on. Pohmelfs uses local cache (synced on timely basis) for all operations, and only sync (or close with sync_on_close mount option) or writeback will flush data to remote nodes. There is also background work to flush data to storage like commit in ext3
Directory objects are synced to the storage when they are created on pohmelfs node, file data is being sent from cache later.
All recovery process is handled by elliptics and is performed without pohmelfs clients every noticing that. In particular reads are always directed to the replica with the latest data (determined according to ellptics metadata checked at file open time).
Writes in pohmelfs can be configured to succeed when quorum commits it or when specified in 'successful_write_count=' number of writes return ok. This is useful when you do not care much about number of active replicas, since you know that your data is safe. Or when you create new storage and want to copy to single datacenter to save bandwidth.
But that's enough for advertisement. Here is the code.
I do not know whether it is a good idea to ask it for inclusion into mainline tree right now skipping drivers/staging part. But anyway, I post patch to remove drivers/staging/pohmelfs and add fs/pohmelfs If it is not the way to go, I will resubmit.
1. Elliptics network
http://www.ioremap.net/projects/elliptics
http://www.elliptics.ru (russian support forum - we understand english and will answer pohmelfs questions too)
2. Pohmelfs
http://www.ioremap.net/projects/pohmelfs (old and not yep updated site)
http://www.ioremap.net/taxonomy/term/4 (development blog section)
Signature · 1 line
Evgeniy Polyakov
fs/Kconfig +1 -0
@@ -259,6 +259,7 @@ config NFS_COMMON source "net/sunrpc/Kconfig" source "fs/ceph/Kconfig" source "fs/cifs/Kconfig" +source "fs/pohmelfs/Kconfig" source "fs/ncpfs/Kconfig" source "fs/coda/Kconfig" source "fs/afs/Kconfig"
fs/Makefile +1 -0
@@ -123,3 +123,4 @@ obj-$(CONFIG_GFS2_FS) += gfs2/ obj-$(CONFIG_EXOFS_FS) += exofs/ obj-$(CONFIG_CEPH_FS) += ceph/ obj-$(CONFIG_PSTORE) += pstore/ +obj-$(CONFIG_POHMELFS) += pohmelfs/
fs/pohmelfs/Kconfig new file +11 -0
@@ -0,0 +1,11 @@ +config POHMELFS + tristate "POHMELFS distributed filesystem" + depends on INET && EXPERIMENTAL + select CRYPTO_HASH + help + POHMELFS is a POSIX frontend to Elliptics network + + Elliptics is a key/value storage, which by default imlpements + distributed hash table structure. + + More information can be found at http://www.ioremap.net/projects/elliptics
fs/pohmelfs/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +# +# Makefile for the linux ext2-filesystem routines. +# + +obj-$(CONFIG_POHMELFS) += pohmelfs.o + +pohmelfs-y := dir.o file.o inode.o net.o route.o super.o trans.o symlink.o
fs/pohmelfs/Module.symvers new file +0 -0
fs/pohmelfs/dir.c new file +844 -0
844 lines not on this page · raw message
fs/pohmelfs/file.c new file +470 -0
470 lines not on this page · raw message
fs/pohmelfs/inode.c new file +900 -0
900 lines not on this page · raw message
fs/pohmelfs/net.c new file +611 -0
611 lines not on this page · raw message
fs/pohmelfs/packet.h new file +752 -0
146 lines not on this page · raw message
fs/pohmelfs/pohmelfs.h new file +461 -0
461 lines not on this page · raw message
fs/pohmelfs/route.c new file +279 -0
279 lines not on this page · raw message
fs/pohmelfs/super.c new file +905 -0
905 lines not on this page · raw message
fs/pohmelfs/trans.c new file +384 -0
384 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
commit 0a1cf67228562021535f3d2d35ff8aa198af0eba
Author: Evgeniy Polyakov <[email protected]> Date: Wed Feb 8 23:12:08 2012 +0400
Removed drivers/staging/pohmelfs
drivers/staging/pohmelfs/Kconfig deleted +0 -28
Lines of this file are not shown. View the raw message.
drivers/staging/pohmelfs/Makefile deleted +0 -3
Lines of this file are not shown. View the raw message.
drivers/staging/pohmelfs/config.c deleted +0 -611
611 lines not on this page · raw message
drivers/staging/pohmelfs/crypto.c deleted +0 -878
878 lines not on this page · raw message
drivers/staging/pohmelfs/dir.c deleted +0 -1101
1,101 lines not on this page · raw message
drivers/staging/pohmelfs/inode.c deleted +0 -2056
2,056 lines not on this page · raw message
drivers/staging/pohmelfs/lock.c deleted +0 -182
182 lines not on this page · raw message
drivers/staging/pohmelfs/mcache.c deleted +0 -171
171 lines not on this page · raw message
drivers/staging/pohmelfs/net.c deleted +0 -1209
1,209 lines not on this page · raw message
drivers/staging/pohmelfs/netfs.h deleted +0 -919
919 lines not on this page · raw message
drivers/staging/pohmelfs/path_entry.c deleted +0 -120
120 lines not on this page · raw message
drivers/staging/pohmelfs/trans.c deleted +0 -706
706 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Greg KH
maintainerQuoted 5 lines · Evgeniy Polyakov
But that's enough for advertisement. Here is the code.
I do not know whether it is a good idea to ask it for inclusion into mainline tree right now skippingdrivers/stagingpart. But anyway, I post patch to removedrivers/staging/pohmelfsand add fs/pohmelfs If it is not the way to go, I will resubmit.
It would be easiest to just send a patch deleting the old drivers/staging/pohmelfs, or just ask me to do it, which I can, and then send a new patch just with the new code, as it is so different.
That should make it easier to review, and also small enough to fit through the mailing list size limits :)
thanks,
greg k-h
Quoted 9 lines · Greg KH
Quoted 5 lines
But that's enough for advertisement. Here is the code.
I do not know whether it is a good idea to ask it for inclusion into mainline tree right now skippingdrivers/stagingpart. But anyway, I post patch to removedrivers/staging/pohmelfsand add fs/pohmelfs If it is not the way to go, I will resubmit.It would be easiest to just send a patch deleting the old
drivers/staging/pohmelfs, or just ask me to do it, which I can, and then send a new patch just with the new code, as it is so different.
pohmelfs-staging-remove.diff does hust that - it removes drivers/staging/pohmelfs
pohmelfs.diff adds new code into fs/pohmelfs
Or did they fail to leak into maillist because of the size? If so, they are also accessible via
http://www.ioremap.net/tmp/pohmelfs.diff (142 Kb)
http://www.ioremap.net/tmp/pohmelfs-staging-remove.diff (200 Kb)
I also embedd pohmelfs.diff here:
fs/Kconfig +1 -0
@@ -259,6 +259,7 @@ config NFS_COMMON source "net/sunrpc/Kconfig" source "fs/ceph/Kconfig" source "fs/cifs/Kconfig" +source "fs/pohmelfs/Kconfig" source "fs/ncpfs/Kconfig" source "fs/coda/Kconfig" source "fs/afs/Kconfig"
fs/Makefile +1 -0
@@ -123,3 +123,4 @@ obj-$(CONFIG_GFS2_FS) += gfs2/ obj-$(CONFIG_EXOFS_FS) += exofs/ obj-$(CONFIG_CEPH_FS) += ceph/ obj-$(CONFIG_PSTORE) += pstore/ +obj-$(CONFIG_POHMELFS) += pohmelfs/
fs/pohmelfs/Kconfig new file +11 -0
@@ -0,0 +1,11 @@ +config POHMELFS + tristate "POHMELFS distributed filesystem" + depends on INET && EXPERIMENTAL + select CRYPTO_HASH + help + POHMELFS is a POSIX frontend to Elliptics network + + Elliptics is a key/value storage, which by default imlpements + distributed hash table structure. + + More information can be found at http://www.ioremap.net/projects/elliptics
fs/pohmelfs/Makefile new file +7 -0
@@ -0,0 +1,7 @@ +# +# Makefile for the linux ext2-filesystem routines. +# + +obj-$(CONFIG_POHMELFS) += pohmelfs.o + +pohmelfs-y := dir.o file.o inode.o net.o route.o super.o trans.o symlink.o
fs/pohmelfs/Module.symvers new file +0 -0
fs/pohmelfs/dir.c new file +844 -0
844 lines not on this page · raw message
fs/pohmelfs/file.c new file +470 -0
470 lines not on this page · raw message
fs/pohmelfs/inode.c new file +900 -0
900 lines not on this page · raw message
fs/pohmelfs/net.c new file +611 -0
611 lines not on this page · raw message
fs/pohmelfs/packet.h new file +752 -0
146 lines not on this page · raw message
fs/pohmelfs/pohmelfs.h new file +461 -0
461 lines not on this page · raw message
fs/pohmelfs/route.c new file +279 -0
279 lines not on this page · raw message
fs/pohmelfs/super.c new file +905 -0
905 lines not on this page · raw message
fs/pohmelfs/trans.c new file +384 -0
384 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
Evgeniy Polyakov
Part of the diff is not shown here. View the raw message.
Greg KH
maintainerQuoted 11 lines · Evgeniy Polyakov
Quoted 9 lines · Greg KH
Quoted 5 lines
But that's enough for advertisement. Here is the code.
I do not know whether it is a good idea to ask it for inclusion into mainline tree right now skippingdrivers/stagingpart. But anyway, I post patch to removedrivers/staging/pohmelfsand add fs/pohmelfs If it is not the way to go, I will resubmit.It would be easiest to just send a patch deleting the old
drivers/staging/pohmelfs, or just ask me to do it, which I can, and then send a new patch just with the new code, as it is so different.
Quoted 8 lines
pohmelfs-staging-remove.diff does hust that - it removes
drivers/staging/pohmelfs
pohmelfs.diff adds new code into fs/pohmelfsOr did they fail to leak into maillist because of the size? If so, they are also accessible via
http://www.ioremap.net/tmp/pohmelfs.diff (142 Kb)
http://www.ioremap.net/tmp/pohmelfs-staging-remove.diff (200 Kb)
I need a signed-off-by: for any patch :)
Care to resend me just the remove patch, with the proper changelog and
signed-off-by: so that I can queue it up?
thanks,
greg k-h
Quoted 4 lines · Greg KH
I need a signed-off-by: for any patch :)
Care to resend me just the remove patch, with the proper changelog and
signed-off-by: so that I can queue it up?
Done
Signature · 1 line
Evgeniy Polyakov
fs/pohmelfs/Kconfig line 8
Quoted 7 lines · Evgeniy Polyakov
fs/pohmelfs/Kconfig new file +7
-0+ tristate "POHMELFS distributed filesystem" + depends on INET && EXPERIMENTAL + select CRYPTO_HASH + help + POHMELFS is a POSIX frontend to Elliptics network + + Elliptics is a key/value storage, which by default imlpements
typo fix: implements
Quoted 1 line
fs/pohmelfs/Kconfig new file +1
-0+ distributed hash table structure.
Quoted 9 lines · On Thu, Feb 09, 2012 at 07:54:15PM -0500, [email protected]…
fs/pohmelfs/Kconfig line 8
Quoted 7 lines
fs/pohmelfs/Kconfig new file +7
-0+ tristate "POHMELFS distributed filesystem" + depends on INET && EXPERIMENTAL + select CRYPTO_HASH + help + POHMELFS is a POSIX frontend to Elliptics network + + Elliptics is a key/value storage, which by default imlpementstypo fix: implements
fs/pohmelfs/Kconfig line 9
Quoted 1 line
fs/pohmelfs/Kconfig new file +1
-0+ distributed hash table structure.
Fixed, thank you :)
Signature · 1 line
Evgeniy Polyakov