Distributed storage. Mirror algo extension for automatic recovery.
Subsystems (3)
cover Distributed storage. Mirror algo extension for automatic recovery.
Hi.
I'm pleased to announce sixth release of the distributed storage subsystem, which allows to form a storage on top of remote and local nodes, which in turn can be exported to another storage as a node to form tree-like storages.
This release includes mirroring algorithm extension, which allows to store 'age' of the given node on the underlying media.
In this case, if failed node gets new media, which does not contain correct 'age' (unique id assigned to the whole storage during initialization time), the whole node will be marked as dirty and eventually resynced.
This allows to have completely transparent failure recovery - failed node can be just turned off, its hardware fixed and then turned on. DST core will detect connection reset and automatically reconnect when node is ready and resync if needed without any special administrator's steps.
This patchset has been split into 4 parts:
0 - this introduction
1 - core files
2 - network state machine
3 - documentation and algorithms
Hope they all will find its way into mail lists.
Further TODO list includes:
* new redundancy algorithm (complex, low priority)
* some thoughts about distributed filesystem tightly connected to DST
Thank you.
Homepage:
http://tservice.net.ru/~s0mbre/old/?section=projects&item=dst
Signature · 1 line
Evgeniy Polyakov
Quoted 4 lines · On Thu, 18 Oct 2007 23:17:41 +0400 Evgeniy Polyakov <[email protected]> wrote:
I'm pleased to announce sixth release of the distributed storage subsystem, which allows to form a storage on top of remote and local nodes, which in turn can be exported to another storage as a node to form tree-like storages.
I went back and re-read last month's discussion and I'm not seeing any reason why we shouldn't start thinking about merging this.
How close is it to that stage? A peek at your development blog indicates that things are still changing at a moderate rate?
Hi Andrew.
Quoted 10 lines · Andrew Morton
Quoted 4 lines
I'm pleased to announce sixth release of the distributed storage subsystem, which allows to form a storage on top of remote and local nodes, which in turn can be exported to another storage as a node to form tree-like storages.
I went back and re-read last month's discussion and I'm not seeing any reason why we shouldn't start thinking about merging this.
How close is it to that stage? A peek at your development blog indicates that things are still changing at a moderate rate?
I completed storage layer development itself, the only remaining todo item is to implement new redundancy algorithm, but I did not see major demand on that, so it will stay for now with low priority.
I will use DST as a transport layer for distributed filesystem, and probably that will require additional features, I have no clean design so far, but right now I have nothing in the pipe to commit to DST.
Signature · 1 line
Evgeniy Polyakov