3/5: Device-mapper: snapshots
A message that is not in the archive (sent off-list or not yet received): [email protected]
Review
Quoted 5 lines · Alasdair G Kergon
(quoted diff) +5
-0+ +/*----------------------------------------------------------------- + * Persistent snapshots, by persistent we mean that the snapshot + * will survive a reboot. + *---------------------------------------------------------------*/
Is this target supposed to be crash safe? What happens when the computer crashes while writing to such a volume?
I suppose it would need barriers for that at least, which it doesn't seem to use.
-Andi
Quoted 5 lines · Andi Kleen
Quoted 4 lines · Alasdair G Kergon
+/*-----------------------------------------------------------------
+ * Persistent snapshots, by persistent we mean that the snapshot
+ * will survive a reboot.
+ *---------------------------------------------------------------*/
As opposed to the transient snapshot which is lost on power down.
Quoted 2 lines
Is this target supposed to be crash safe? What happens when the computer crashes while writing to such a volume?
The intention is for snapshot & origin still to be consistent with each other on disk. (No barriers yet, but some synchronous writes.)
Alasdair
Signature · 1 line
[email protected]
Quoted 5 lines
Quoted 2 lines
Is this target supposed to be crash safe? What happens when the computer crashes while writing to such a volume?
The intention is for snapshot & origin still to be consistent with each other on disk. (No barriers yet, but some synchronous writes.)
So it's supposed to be crash safe?
How about documenting this somewhere in the source?
-Andi
Quoted 1 line · Andi Kleen
So it's supposed to be crash safe?
Depends on the type of crash: updates are sequenced and synced (in co-ordinated batches) so it should cope with events like freezing/stopping (but not necessarily memory corruption).
As Kevin indicated, we'll certainly be generating more documentation in the near future. [Some of this will be a by-product of the forthcoming LVM2/device-mapper tutorial at LinuxTag.]
Alasdair
Signature · 1 line
[email protected]
Quoted 10 lines · Andi Kleen
Review
Quoted 5 lines · Alasdair G Kergon
(quoted diff) +5
-0+ +/*----------------------------------------------------------------- + * Persistent snapshots, by persistent we mean that the snapshot + * will survive a reboot. + *---------------------------------------------------------------*/Is this target supposed to be crash safe? What happens when the computer crashes while writing to such a volume?
Quoted 2 lines
I suppose it would need barriers for that at least, which it doesn't seem to use.
Hi Andi,
Please pardon the one month lag, I must bug Zack about getting Kernel Traffic out faster ;)
It is designed to be crash-safe:
- Each snapshot exception is logged to disk by overwriting the last sector of a grow-only list of snapshot exceptions.
- Write completion is not handed back up the chain until:
- the data to be overwritten has been copied to a new exception
- the new exception has been logged to the snapshot store as above
As far as I can see, the concept is leak-proof, except for being sensitive to random garbage in the last few sector writes. I suspect that doesn't happen on modern disk drives. If it does, I hope somebody will shout.
I am not sure what you mean about barriers, perhaps you were thinking of synchronous waiting. This snapshot driver does wait for completions, but it pipelines the waits so throughput is not affected much (snapshot overhead is dominated by copyouts).
Incidently, there is an entirely new snapshot design coming down the pipe that uses a more traditional, journalling design to achieve the necessary hardware-like durability.
Regards,
Daniel
Quoted 5 lines · Daniel Phillips
It is designed to be crash-safe:
- Each snapshot exception is logged to disk by overwriting the last sector of a grow-only list of snapshot exceptions.
Quoted 8 lines
- Write completion is not handed back up the chain until:
- the data to be overwritten has been copied to a new exception
- the new exception has been logged to the snapshot store as aboveAs far as I can see, the concept is leak-proof, except for being sensitive to random garbage in the last few sector writes. I suspect that doesn't happen on modern disk drives. If it does, I hope somebody will shout.
I am not sure what you mean about barriers, perhaps you were thinking of synchronous waiting. This snapshot driver does wait for completions, but it pipelines the waits so throughput is not affected much (snapshot overhead is dominated by copyouts).
Barriers as discussed on lkml ensure your data is committed to stable storage, not simply completed requests. In SCSI this means ordered tags, FUA, or cache flushing. Ditto ATA (cache flushing, mostly).
Jeff
Quoted 16 lines · Jeff Garzik
Quoted 15 lines · Daniel Phillips
It is designed to be crash-safe:
- Each snapshot exception is logged to disk by overwriting the last
sector of a grow-only list of snapshot exceptions.- Write completion is not handed back up the chain until:
- the data to be overwritten has been copied to a new exception
- the new exception has been logged to the snapshot store as aboveAs far as I can see, the concept is leak-proof, except for being sensitive to random garbage in the last few sector writes. I suspect that doesn't happen on modern disk drives. If it does, I hope somebody will shout.
Quoted 8 lines
Quoted 4 lines
I am not sure what you mean about barriers, perhaps you were thinking of synchronous waiting. This snapshot driver does wait for completions, but it pipelines the waits so throughput is not affected much (snapshot overhead is dominated by copyouts).
Barriers as discussed on lkml ensure your data is committed to stable storage, not simply completed requests. In SCSI this means ordered tags, FUA, or cache flushing. Ditto ATA (cache flushing, mostly).
I meant, I didn't know why he thought barriers might apply in this case, but now that you mention it, yes we risk the same bugs with certain hardware as, say, a journal commit does. We need to do something about that at some point. (I see that the barrier patch hasn't made it to mainline yet, and actually, that's good because it needs to be looked at critically.)
Anyway (reading Andi's mind) it seems the snapshot durability strategy just wasn't obvious on a light reading. It certainly wasn't obvious to me without clarification from Joe Thornber, from whose fertile imagination this clever hack apparently sprang. Yes, these details need to be documented.
Regards,
Daniel