Skip to contentSkip to conversation

Live migration without shared storage

PATCH Superseded [email protected] 3 participants 5 messages Oct 12, 2009 – Oct 13, 2009

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover Live migration without shared storage

This series adds support for live migration without shared storage, means copy the storage while migrating. It was tested with KVM. Supports 2 ways to replicate the storage during migration:
1. Complete copy of storage to destination
2. Assuming the storage is cow based, copy only the allocated data, time of the migration will be linear with the amount of allocated data (user responsibility to verify that the same backend file reside on src and destination).

The code is being used by the Reservoir project: http://www.reservoir-fp7.eu/

Live migration will work as follows:
(qemu) migrate -d tcp:0:4444 # for ordinary live migration
(qemu) migrate -d blk tcp:0:4444 # for live migration with complete storage copy
(qemu) migrate -d blk inc tcp:0:4444 # for live migration with incremental storage copy, storage is cow based.

These patches addresses all previous comments.

---
 Makefile          |    1 +
 block-migration.c |  624 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 block-migration.h |   30 +++
 block.c           |   27 +++
 block.h           |    6 +-
 block_int.h       |    1 +
 buffered_file.c   |   10 +-
 hw/hw.h           |    9 +-
 migration-exec.c  |    9 +-
 migration-fd.c    |    7 +-
 migration-tcp.c   |    7 +-
 migration-unix.c  |    7 +-
 migration.c       |   21 ++-
 migration.h       |   20 ++-
 monitor.c         |   38 +++-
 qemu-monitor.hx   |   13 +-
 savevm.c          |   62 ++++--
 sysemu.h          |    2 +-
 vl.c              |    6 +-
 19 files changed, 851 insertions(+), 49 deletions(-)
#1 lore raw
Quoted 6 lines · [email protected] wrote:

This series adds support for live migration without shared storage, means copy the storage while migrating. It was tested with KVM. Supports 2 ways to replicate the storage during migration:
1. Complete copy of storage to destination
2. Assuming the storage is cow based, copy only the allocated
data, time of the migration will be linear with the amount of allocated
data (user responsibility to verify that the same backend file reside
on src and destination).

Hmm. How about something like:

migrate exec:"rsync file1 file2 ... remote: && nc remote port"

?

/mjt

#2 lore raw
Quoted 9 lines · Michael Tokarev
Quoted 7 lines · [email protected] wrote:

This series adds support for live migration without shared storage, means copy the storage while migrating. It was tested with KVM. Supports 2 ways to replicate the storage during migration:
1. Complete copy of storage to destination 2. Assuming the storage is cow based, copy only the allocated data, time of the migration will be linear with the amount of allocated data (user responsibility to verify that the same backend file reside on src and destination).

Quoted 3 lines

Hmm. How about something like:

migrate exec:"rsync file1 file2 ... remote: && nc remote port"

That can surely work to copy the base files, but it is a task for an upper level like libvirt. For qemu, copying the copy-on-write data is enough.

Paolo

#3 lore raw

[email protected] wrote on 12/10/2009 18:51:32:

Quoted 10 lines · Michael Tokarev
Quoted 9 lines
Quoted 7 lines · [email protected] wrote:

This series adds support for live migration without shared storage, means copy the storage while migrating. It was tested with KVM. Supports 2 ways to replicate the storage during migration:
1. Complete copy of storage to destination 2. Assuming the storage is cow based, copy only the allocated data, time of the migration will be linear with the amount of allocated data (user responsibility to verify that the same backend file reside on src and destination).

Quoted 8 lines
Quoted 3 lines

Hmm. How about something like:

migrate exec:"rsync file1 file2 ... remote: && nc remote port"

That can surely work to copy the base files, but it is a task for an upper level like libvirt. For qemu, copying the copy-on-write data is enough.

I agree that it can be useful to let an upper level component to copy the storage
but you will need a way to synchronize with that layer: 1. qemu --> upper layer:
vm suspended,no more writes to disk.
2. upper layer --> qemu: storage replication is done.
I don't see how without it you will have a valid copy of the storage.

However make use of the copy on write data and cut the amount of transferred data
can be done only from inside qemu.

- Liran

#4 lore raw
Quoted 16 lines
Quoted 7 lines
Quoted 3 lines

Hmm. How about something like:

migrate exec:"rsync file1 file2 ... remote:&& nc remote port"

That can surely work to copy the base files, but it is a task for an upper level like libvirt. For qemu, copying the copy-on-write data is enough.

I agree that it can be useful to let an upper level component to copy the storage
but you will need a way to synchronize with that layer: 1. qemu --> upper layer:
vm suspended,no more writes to disk.
2. upper layer --> qemu: storage replication is done.
I don't see how without it you will have a valid copy of the storage.

Quoted 3 lines

However make use of the copy on write data and cut the amount of transferred data
can be done only from inside qemu.

Yes, I understand the reason for this feature. What I meant was that you could use rsync to copy the base files from an upper level component, before the actual migration is started and qemu transfers the copy-on-write data live.

If Michael was saying instead "why is this needed at all", then you answered him.

Paolo