Livebackup feature for qemu/qemu-kvm
Hello All,
I have been working on a feature called Livebackup for qemu-kvm (actually it is applicable to qemu). This is a feature that enables a system administrator or a backup program to take full and incremental backups of running VMs. It is explained in more detail here:
http://wiki.qemu.org/Features/Livebackup
I had initiated communication with Stefan a week or so ago regarding this feature and we had some discussion comparing it with two other proposals - Snapshot and Snapshot2.
I have further updated my documentation, and I have been testing my code more intensively over the last week or so. I have two git source trees based on qemu and qemu-kvm for this codebase. I will continue to maintain both till it is necessary.
At this time I would like to restart the discussion around Livebackup and related features.
Having a live backup feature for kvm will make it a better solution for IaaS clouds compared to xen. I would like to solicit feedback from all of you folks involved in the block subsystem of qemu. Stefan mentioned that Jes is the person most intimately involved in the block subsystem, so Jes - your feedback is particularly important.
Thanks,
Jagane
Quoted 5 lines · Jagane Sundar
Having a live backup feature for kvm will make it a better solution for IaaS clouds compared to xen. I would like to solicit feedback from all of you folks involved in the block subsystem of qemu. Stefan mentioned that Jes is the person most intimately involved in the block subsystem, so Jes - your feedback is particularly important.
Jagane,
Reading your page, the first thing I stumble upon under 'Use Cases' is the reference to EBS storage. What is that?
Under details, I think it is not a good idea to rely on QEMU looking for any files with specific file name suffixes. It really should be specified on the command line by the user or admin tool.
Other questions, how do you plan to handle crashes or loss of network connection in the middle of a livebackup? How about handling corrupted livebackup files?
In general it looks interesting, you could consider submitting a presentation about Livebackup to the KVM Forum 2011.
Cheers,
Jes
Quoted 10 lines · Jes Sorensen
Quoted 5 lines · Jagane Sundar
Having a live backup feature for kvm will make it a better solution for IaaS clouds compared to xen. I would like to solicit feedback from all of you folks involved in the block subsystem of qemu. Stefan mentioned that Jes is the person most intimately involved in the block subsystem, so Jes - your feedback is particularly important.
Jagane,
Reading your page, the first thing I stumble upon under 'Use Cases' is the reference to EBS storage. What is that?
EBS stands for Elastic Block Storage - Amazon EC2's shared storage solution. This is the storage that comes with guarantees, since it is replicated across machines.
Quoted 3 lines
Under details, I think it is not a good idea to rely on QEMU looking for any files with specific file name suffixes. It really should be specified on the command line by the user or admin tool.
That's a good idea. Perhaps another attribute in the drive description list, just like type=virtio, maybe backup=livebackup.
Quoted 3 lines
Other questions, how do you plan to handle crashes or loss of network connection in the middle of a livebackup? How about handling corrupted livebackup files?
Crashes of various software:
1. qemu crashes during normal operation of the VM:
When this happens, the livebackup_client is forced to do a full
backup the next
time around. Here's how: livebackup writes out the in-memory dirty
bitmap
to a dirty bitmap file only at the time of orderly shutdown of
qemu. Hence,
the mtime of the virtual disk file is later than the mtime of the
livebackup
dirty bitmap file. This causes livebackup to consider the dirty
bitmap invalid,
and forces the livebackup_client to do a full backup next time around.
2. qemu crashes while livebackup is in progress:
In this case also, the livebackup_client is forced to do a full
backup the next
time around. The dirty bitmap file, the COW file used to store
blocks written
while a livebackup is in progress, are all deleted, and the
livebackup client
is forced to do a full backup next time around.
3. livebackup_client crashes while livebackup is in progress:
In this case, a new livebackup_client may be started, and it can
redo the last
type of backup it was doing - an incremental backup or a full
backup. Note
all the blocks of the last backup type need to be transferred over
again, the
qemu livebackup code does not keep track of what block the client
was at.
It does not need to be a forced full backup.
Quoted 2 lines
In general it looks interesting, you could consider submitting a presentation about Livebackup to the KVM Forum 2011.
Glad to know that you think it is interesting. Also, thanks for the pointer to KVM Forum 2011, Jes. It looks like I have a few more weeks to get an abstract in for KVM forum 2011. I will do so.
Jagane
Quoted 6 lines · Jagane Sundar
Quoted 2 lines · Jes Sorensen
Reading your page, the first thing I stumble upon under 'Use Cases' is the reference to EBS storage. What is that?
EBS stands for Elastic Block Storage - Amazon EC2's shared storage solution. This is the storage that comes with guarantees, since it is replicated across machines.
I see, it would be good if you made that more explicit in the document for those who aren't as experienced with EC2, like me.
Quoted 6 lines
Quoted 3 lines
Under details, I think it is not a good idea to rely on QEMU looking for any files with specific file name suffixes. It really should be specified on the command line by the user or admin tool.
That's a good idea. Perhaps another attribute in the drive description list,
just like type=virtio, maybe backup=livebackup.
I think that is the right way to go.
Quoted 6 lines
Quoted 2 lines
In general it looks interesting, you could consider submitting a presentation about Livebackup to the KVM Forum 2011.
Glad to know that you think it is interesting. Also, thanks for the pointer to KVM Forum 2011, Jes. It looks like I have a few more weeks to get an abstract in for KVM forum 2011. I will do so.
Excellent, please go ahead and submit it!
Cheers,
Jes
Quoted 10 lines · Jes Sorensen
Quoted 6 lines · Jagane Sundar
Quoted 2 lines · Jes Sorensen
Reading your page, the first thing I stumble upon under 'Use Cases' is the reference to EBS storage. What is that?
EBS stands for Elastic Block Storage - Amazon EC2's shared storage solution. This is the storage that comes with guarantees, since it is replicated across machines.
I see, it would be good if you made that more explicit in the document for those who aren't as experienced with EC2, like me.
Done. I also added a failure scenarios section. I will ping you once I get the synopsis in for kvm forum 2011.
Thanks,
Jagane