What is holding back clustered snapshotting?
Hi all,
I know that, currently, this isn't supported. Would someone be able
to explain or point me at a place to read up on what is holding this feature back? What are the difficulties? Is it just a question of time, or are there certain technical hurdles in the way?
Thanks for any input!
Madi
Quoted 4 lines · Madison Kelly
I know that, currently, this isn't supported. Would someone be able to
explain or point me at a place to read up on what is holding this feature back? What are the difficulties? Is it just a question of time, or are there certain technical hurdles in the way?
Setting up the shapshot is just a matter of locking and coordination. However, writes to the origin or snapshot (may) require allocating a cluster, copying the origin data, then writing the origin. All of this coordinated with all the machines using the VG. Apart from some cleven invention, this requires global locking on many writes. This is just too inefficient.
However, you can obtain the same effect using a SAN. Have one machine run LVM (and raid, etc), and export LVs via AoE or iSCSI. Of course, that LVM machine now becomes a single point of failure...
Here's an idea (someone probably already thought of this, but..), have one machine in a cluster elected "master" for a VG, and have all reads/writes from other machines go through the master via AoE or iSCSI. When failure of the "master" is detected, elect another machine to take over as master. Sort of a rotating SAN server.
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.
Quoted 9 lines · Madison Kelly
Hi all,
I know that, currently, this isn't supported. Would someone be able
to explain or point me at a place to read up on what is holding this feature back? What are the difficulties? Is it just a question of time, or are there certain technical hurdles in the way?Thanks for any input!
I'm sure there are some technical hurdles... but perhaps customers need to put more of a priority on it.
I keep forgetting to open an RFE SR for this feature. That sorta stuff makes a difference in prioritization for RH's engineering though for sure!
Ray
Quoted 7 lines · Stuart D. Gathman
Quoted 4 lines · Madison Kelly
I know that, currently, this isn't supported. Would someone be able to
explain or point me at a place to read up on what is holding this feature back? What are the difficulties? Is it just a question of time, or are there certain technical hurdles in the way?
Quoted 6 lines
Setting up the shapshot is just a matter of locking and coordination. However, writes to the origin or snapshot (may) require allocating a cluster, copying the origin data, then writing the origin. All of this coordinated with all the machines using the VG. Apart from some cleven invention, this requires global locking on many writes. This is just too inefficient.
Writeable snapshots I guess would be a challenge. But even read only snapshots would be great as it would theoretically make backing up large, clustered filesystems simpler.
Quoted 11 lines
However, you can obtain the same effect using a SAN. Have one machine run LVM (and raid, etc), and export LVs via AoE or iSCSI. Of course, that LVM machine now becomes a single point of failure...
Here's an idea (someone probably already thought of this, but..), have one machine in a cluster elected "master" for a VG, and have all reads/writes from other machines go through the master via AoE or iSCSI. When failure of the "master" is detected, elect another machine to take over as master. Sort of a rotating SAN server.
Interesting idea. :)
Ray
Quoted 10 lines · Ray Van Dolson
Quoted 6 lines
Setting up the shapshot is just a matter of locking and coordination. However, writes to the origin or snapshot (may) require allocating a cluster, copying the origin data, then writing the origin. All of this coordinated with all the machines using the VG. Apart from some cleven invention, this requires global locking on many writes. This is just too inefficient.
Writeable snapshots I guess would be a challenge. But even read only snapshots would be great as it would theoretically make backing up large, clustered filesystems simpler.
Even read-only snapshots have to handle writes to the origin. And anytime any machine writes a new cluster to the origin, all machines need their cluster maps updated so that reads don't pick up the wrong cluster.
Signature · 4 lines
Stuart D. Gathman <[email protected]> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154 "Confutatis maledictis, flammis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial.
Quoted 6 lines · Madison Kelly
Hi all,
I know that, currently, this isn't supported. Would someone be able
to explain or point me at a place to read up on what is holding this feature back? What are the difficulties? Is it just a question of time, or are there certain technical hurdles in the way?Thanks for any input!
The solution isn't ridiculously hard. It has actually been implemented a couple times by me and at least once by Mikulas leveraging a locking interface I had proposed. (You can search the various dm-devel archives for them.)
Our current priority is to improve the snapshot implementation (shared snapshots), but cluster snapshots is on the todo list. It does tend to be a matter of time. As I said, a couple implementations have come forward, but it requires a team effort to push it upstream - the priority hasn't been high enough for it to surpass other things on the list.
brassow