incremental backup a running vm
Hi,
Does drive_mirror support incremental backup a running vm? Or other mechanism does?
incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to destination, then a copied bitmap will be saved to a file, then the bitmap file will log dirty for the changed data.
Next time backup, only the dirty data will be mirrored to destination. Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Thanks,
Zhang Haoyu
Paolo Bonzini
maintainerQuoted 5 lines · Zhang Haoyu
Hi,
Does
drive_mirrorsupport incremental backup a running vm? Or other mechanism does?
Quoted 8 lines
incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to destination, then a copied bitmap will be saved to a file, then the bitmap file will log dirty for the changed data.
Next time backup, only the dirty data will be mirrored to destination. Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Drive-mirror is for storage migration. For backup there is another job, drive-backup. drive-backup copies a point-in-time snapshot of one or more disks corresponding to when the backup was started.
Incremental backup is being worked on. You can see patches on the list.
Paolo
Quoted 17 lines · Paolo Bonzini
Quoted 5 lines · Zhang Haoyu
Hi,
Does
drive_mirrorsupport incremental backup a running vm? Or other mechanism does?Quoted 8 lines
incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to destination, then a copied bitmap will be saved to a file, then the bitmap file will log dirty for the changed data.
Next time backup, only the dirty data will be mirrored to destination. Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Quoted 3 lines
Drive-mirror is for storage migration. For backup there is another job, drive-backup. drive-backup copies a point-in-time snapshot of one or more disks corresponding to when the backup was started.
Zhang, I've been testing the drive-backup command via QMP for a little while, and it works reasonably well. If you'd like to test it you can quickly try as below, once you have a QEMU instance runing with QMP (I invoke my QEMU instances with '-qmp tcp:localhost:4444,server').
The below script invokes the 'drive-backup' QMP command (Ensure you're using the correct disk, your disk ID might be 'drive-virtio-disk1' :-) )
-----------------
#!/bin/bash
set -x
exec 3<>/dev/tcp/localhost/4444
echo -e "{ 'execute': 'qmp_capabilities' }" >&3
read response <&3
echo $response
echo -e "{ 'execute': 'drive-backup', 'arguments':
{ 'device': 'drive-virtio-disk0', 'sync': 'full', 'target':
'/export/backup-of-vm1.qcow2', 'mode': 'absolute-paths', 'format': 'qcow2' } }" >&3read response <&3
echo $response
-----------------
Once the above is invoked succesfully, you can see the success of the
command on the shell where your QMP server is running:
-----------------
[. . .]
char device redirected to /dev/pts/8 (label charserial0)
QEMU waiting for connection on: disconnected:tcp:localhost:4444,server
Formatting '/export/backup-of-vm1.qcow2', fmt=qcow2 size=53687091200 encryption=off cluster_size=65536 lazy_refcounts=off
-----------------
Quoted 1 line
Incremental backup is being worked on. You can see patches on the list.
Signature · 1 line
/kashyap
Quoted 17 lines · Paolo Bonzini
Quoted 5 lines · Zhang Haoyu
Hi,
Does
drive_mirrorsupport incremental backup a running vm? Or other mechanism does?Quoted 8 lines
incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to destination, then a copied bitmap will be saved to a file, then the bitmap file will log dirty for the changed data.
Next time backup, only the dirty data will be mirrored to destination. Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Quoted 3 lines
Drive-mirror is for storage migration. For backup there is another job, drive-backup. drive-backup copies a point-in-time snapshot of one or more disks corresponding to when the backup was started.
Zhang, I've been testing the drive-backup command via QMP for a little while, and it works reasonably well. If you'd like to test it you can quickly try as below, once you have a QEMU instance runing with QMP (I invoke my QEMU instances with '-qmp tcp:localhost:4444,server').
The below script invokes the 'drive-backup' QMP command (Ensure you're using the correct disk, your disk ID might be 'drive-virtio-disk1' :-) )
-----------------
#!/bin/bash
set -x
exec 3<>/dev/tcp/localhost/4444
echo -e "{ 'execute': 'qmp_capabilities' }" >&3
read response <&3
echo $response
echo -e "{ 'execute': 'drive-backup', 'arguments':
{ 'device': 'drive-virtio-disk0', 'sync': 'full', 'target':
'/export/backup-of-vm1.qcow2', 'mode': 'absolute-paths', 'format': 'qcow2' } }" >&3read response <&3
echo $response
-----------------
Once the above is invoked succesfully, you can see the success of the
command on the shell where your QMP server is running:
-----------------
[. . .]
char device redirected to /dev/pts/8 (label charserial0)
QEMU waiting for connection on: disconnected:tcp:localhost:4444,server
Formatting '/export/backup-of-vm1.qcow2', fmt=qcow2 size=53687091200 encryption=off cluster_size=65536 lazy_refcounts=off
-----------------
Quoted 5 lines
Incremental backup is being worked on. You can see patches on the list.
Paolo
Signature · 1 line
/kashyap
Quoted 18 lines · Kashyap Chamarthy
Quoted 17 lines · Paolo Bonzini
Quoted 13 lines · Zhang Haoyu
Hi,
Does
drive_mirrorsupport incremental backup a running vm? Or other mechanism does?incremental backup a running vm requirements:
First time backup, all of the allocated data will be mirrored to destination, then a copied bitmap will be saved to a file, then the bitmap file will log dirty for the changed data.
Next time backup, only the dirty data will be mirrored to destination. Even the VM shutdown and start after several days,
the bitmap will be loaded while starting vm.
Any ideas?
Quoted 9 lines
Quoted 3 lines
Drive-mirror is for storage migration. For backup there is another job, drive-backup. drive-backup copies a point-in-time snapshot of one or more disks corresponding to when the backup was started.
Zhang, I've been testing the
drive-backupcommand via QMP for a little while, and it works reasonably well. If you'd like to test it you can quickly try as below, once you have a QEMU instance runing with QMP (I invoke my QEMU instances with '-qmp tcp:localhost:4444,server').
Hi, Kashyap
I've tried ‘drive_backup’ via QMP,
but the snapshots were missed to backup to destination, I think the reason is that backup_run() only copy the guest data regarding qcow2 image.
Thanks,
Zhang Haoyu
Quoted 31 lines
The below script invokes the 'drive-backup' QMP command (Ensure you're using the correct disk, your disk ID might be 'drive-virtio-disk1' :-) )
-----------------
#!/bin/bash
set -x
exec 3<>/dev/tcp/localhost/4444
echo -e "{ 'execute': 'qmp_capabilities' }" >&3
read response <&3
echo $response
echo -e "{ 'execute': 'drive-backup', 'arguments':{ 'device': 'drive-virtio-disk0', 'sync': 'full', 'target': '/export/backup-of-vm1.qcow2', 'mode': 'absolute-paths', 'format': 'qcow2' } }" >&3read response <&3
echo $response
-----------------Once the above is invoked succesfully, you can see the success of the
command on the shell where your QMP server is running:
-----------------
[. . .]
char device redirected to/dev/pts/8(label charserial0)
QEMU waiting for connection on: disconnected:tcp:localhost:4444,server
Formatting '/export/backup-of-vm1.qcow2', fmt=qcow2 size=53687091200 encryption=off cluster_size=65536 lazy_refcounts=off
-----------------Quoted 1 line
Incremental backup is being worked on. You can see patches on the list.
Signature · 1 line
/kashyap
Paolo Bonzini
maintainerQuoted 5 lines · Zhang Haoyu
Hi, Kashyap
I've tried ‘drive_backup’ via QMP,
but the snapshots were missed to backup to destination, I think the reason is thatbackup_run()only copy the guest data regarding qcow2 image.
Yes, that's the case.
QEMU cannot still access internal snapshots while the file is open. External snapshots are opened read-only, and can be copied with "cp" while QEMU is running.
Paolo
Quoted 8 lines · Paolo Bonzini
Quoted 5 lines · Zhang Haoyu
Hi, Kashyap
I've tried ‘drive_backup’ via QMP,
but the snapshots were missed to backup to destination, I think the reason is thatbackup_run()only copy the guest data regarding qcow2 image.
Quoted 5 lines
Yes, that's the case.
QEMU cannot still access internal snapshots while the file is open. External snapshots are opened read-only, and can be copied with "cp" while QEMU is running.
Thanks, Paolo,
but too many internal snapshots were saved by customers, switching to external snapshot mechanism has significant impaction on subsequent upgrade.
Another problem:drive_backup just implement one time backup,
but I want VMWare's VDP-like backup mechanism.
The initial backup of a virtual machine takes comparatively more time, because all of the data for that virtual machine is being backed up. Subsequent backups of the same virtual machine take less time, because changed block tracking (log dirty) mechanism is used to only backup the dirty data. After inittial backup done, even the VM shutdown, but subsequent backup also only copy the changed data.
Thanks,
Zhang Haoyu
Quoted 2 lines
Paolo
Paolo Bonzini
maintainerQuoted 4 lines · Zhang Haoyu
Thanks, Paolo,
but too many internal snapshots were saved by customers, switching to external snapshot mechanism has significant impaction on subsequent upgrade.
In that case, patches are welcome. :)
Quoted 9 lines
Another problem:
drive_backupjust implement one time backup,
but I want VMWare's VDP-like backup mechanism.
The initial backup of a virtual machine takes comparatively more time, because all of the data for that virtual machine is being backed up. Subsequent backups of the same virtual machine take less time, because changed block tracking (log dirty) mechanism is used to only backup the dirty data. After inittial backup done, even the VM shutdown, but subsequent backup also only copy the changed data.
As mentioned before, patches for this are on the list.
Paolo
Quoted 19 lines · Paolo Bonzini
Quoted 4 lines · Zhang Haoyu
Thanks, Paolo,
but too many internal snapshots were saved by customers, switching to external snapshot mechanism has significant impaction on subsequent upgrade.In that case, patches are welcome. :)
Quoted 9 lines
Another problem:
drive_backupjust implement one time backup,
but I want VMWare's VDP-like backup mechanism.
The initial backup of a virtual machine takes comparatively more time, because all of the data for that virtual machine is being backed up. Subsequent backups of the same virtual machine take less time, because changed block tracking (log dirty) mechanism is used to only backup the dirty data. After inittial backup done, even the VM shutdown, but subsequent backup also only copy the changed data.
Quoted 2 lines
As mentioned before, patches for this are on the list.
I see, thanks, Paolo.
Quoted 1 line
Paolo