Developing a Distributed File System
Hello All,
Our team want to develop a distributed file system, a file system like NFS or in other words a remote file access system.
In our design we have multiple clients and a single server. I want to identify main issues involved. What I found after doing some study is this:
(1) Handling of multiple clients (concurrent read/write problem)
(2) Continuation of (1) i.e. the distributed lock management
(3) For performance reasons caching on the client side and the cache coherency problem
(this can be avoided for our prototype release)
(4) Data integrity problem.
To solve these issues, I think I have to study NFS, AFS and coda. Also I am doing general googling for finding out distributed file system issues.
Since both linux and fs (especially a distributed one) are new to me, I want help in this regard that how I start this development of a prototype and which documents/books to study for this purpose.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Quoted 3 lines · UZAIR LAKHANI
Our team want to develop a distributed file system, a file system like NFS or in other words a remote file access system.
First you need to explain why you want to develop a new distributed file system? What is wrong with any of the existing filesystems?
Quoted 4 lines
Since both linux and fs (especially a distributed one) are new to me, I want help in this regard that how I start this development of a prototype and which documents/books to study for this purpose.
This is by no means an easy task to start with. How about fixing some bug or simple feature to implement on an existing filesystem (even a local one) before tackling the hardest problem first?
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
Quoted 10 lines · --- Andreas Dilger
Quoted 1 line · UZAIR LAKHANI
Our team want to develop a distributed file
system, a
Quoted 1 line
file system like NFS or in other words a remote
file
Quoted 1 line
access system.
First you need to explain why you want to develop a new distributed file system? What is wrong with any of the existing filesystems?
Hello All,
Thanks for the reply. Basically we want to move towards developing a fail-over and high performance cluster file system. Although in the simplest design in which there will be a master server and a backup server.
Therefore we first have to figure out the issues involved in a distributed file system (single server and multiple clients).
You may have noticed earlier that I had asked questions regarding the definition of a cluster file system. This was for the same reason.
Quoted 15 lines
Quoted 2 lines
Since both linux and fs (especially a distributed
one)
Quoted 1 line
are new to me, I want help in this regard that how
I
Quoted 2 lines
start this development of a prototype and which
documents/books to study for this purpose.This is by no means an easy task to start with. How about
fixing some bug or simple feature to implement on an existing filesystem (even a local one) before tackling
the hardest problem first?
Quoted 5 lines
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
Quoted 8 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Quoted 9 lines · UZAIR LAKHANI
Thanks for the reply. Basically we want to move towards developing a fail-over and high performance cluster file system. Although in the simplest design in which there will be a master server and a backup server.
Therefore we first have to figure out the issues involved in a distributed file system (single server and multiple clients).
I hate to state the obvious, but this is a VERY HARD PROBLEM.
I may be biased in sugessing this, but why not spend your time and efforts in improving some existing cluster filesystem instead of spending 20 work years of effort just to get a decently usable system? That way, instead of the ever-popular process of having 20 mediocre implementations of something Linux instead gains a small number of very good implementations?
Possible targets of your efforts might include OCFS2, Lustre, GFS2.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.