Skip to contentSkip to conversation

postcopy live migration

PATCH Isaku Yamahata 2 participants 3 messages Jun 4, 2012 1 message not in the archive KVM

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Isaku Yamahata

author
#1 lore raw
Quoted 1 line · Chegu Vinod

Hello Isaku Yamahata,

Hi.

Quoted 2 lines

I just saw your patches..Would it be possible to email me a tar bundle of these patches (makes it easier to apply the patches to a copy of the upstream qemu.git)

I uploaded them to github for those who are interested in it.

git://github.com/yamahata/qemu.git qemu-postcopy-june-04-2012
git://github.com/yamahata/linux-umem.git linux-umem-june-04-2012

Quoted 2 lines

BTW, I am also curious if you have considered using any kind of RDMA features for optimizing the page-faults during postcopy ?

Yes, RDMA is interesting topic. Can we share your use case/concern/issues? Thus we can collaborate.
You may want to see Benoit's results. As long as I know, he has not published his code yet.

thanks,

Quoted 271 lines

Thanks
Vinod

----------------------------------------------------------------------

Message: 1
Date: Mon, 4 Jun 2012 18:57:02 +0900
From: Isaku Yamahata<[email protected]>
To: [email protected], [email protected]
Cc: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected]
Subject: [Qemu-devel] [PATCH v2 00/41] postcopy live migration Message-ID:<[email protected]>

After the long time, we have v2. This is qemu part. The linux kernel part is sent separatedly.

Changes v1 -> v2:
- split up patches for review
- buffered file refactored
- many bug fixes
Espcially PV drivers can work with postcopy
- optimization/heuristic

Patches
1 - 30: refactoring exsiting code and preparation 31 - 37: implement postcopy itself (essential part) 38 - 41: some optimization/heuristic for postcopy

Intro
=====
This patch series implements postcopy live migration.[1]
As discussed at KVM forum 2011, dedicated character device is used for distributed shared memory between migration source and destination. Now we can discuss/benchmark/compare with precopy. I believe there are much rooms for improvement.

[1] http://wiki.qemu.org/Features/PostCopyLiveMigration

Usage
=====
You need load umem character device on the host before starting migration. Postcopy can be used for tcg and kvm accelarator. The implementation depend on only linux umem character device. But the driver dependent code is split into a file.
I tested only host page size == guest page size case, but the implementation allows host page size != guest page size case.

The following options are added with this patch series.
- incoming part
command line options
-postcopy [-postcopy-flags<flags>]
where flags is for changing behavior for benchmark/debugging Currently the following flags are available
0: default
1: enable touching page request

example:
qemu -postcopy -incoming tcp:0:4444 -monitor stdio -machine accel=kvm

- outging part
options for migrate command
migrate [-p [-n] [-m]] URI [<prefault forward> [<prefault backword>]] -p: indicate postcopy migration
-n: disable background transferring pages: This is for benchmark/debugging -m: move background transfer of postcopy mode
<prefault forward>: The number of forward pages which is sent with on-demand <prefault backward>: The number of backward pages which is sent with
on-demand

example:
migrate -p -n tcp:<dest ip address>:4444 migrate -p -n -m tcp:<dest ip address>:4444 32 0

TODO
====
- benchmark/evaluation. Especially how async page fault affects the result.
- improve/optimization
At the moment at least what I'm aware of is
- making incoming socket non-blocking with thread
As page compression is comming, it is impractical to non-blocking read and check if the necessary data is read.
- touching pages in incoming qemu process by fd handler seems suboptimal. creating dedicated thread?
- outgoing handler seems suboptimal causing latency.
- consider on FUSE/CUSE possibility
- don't fork umemd, but create thread?

basic postcopy work flow
========================
qemu on the destination
|
V
open(/dev/umem)
|
V
UMEM_INIT
|
V
Here we have two file descriptors to
umem device and shmem file
|
| umemd
| daemon on the destination
|
V create pipe to communicate
fork()---------------------------------------,
| |
V |
close(socket) V
close(shmem) mmap(shmem file)
| |
V V
mmap(umem device) for guest RAM close(shmem file)
| |
close(umem device) |
| |
V |
wait for ready from daemon<----pipe-----send ready message
| |
| Here the daemon takes over
send ok------------pipe---------------> the owner of the socket
| to the source
V |
entering post copy stage |
start guest execution |
| |
V V
access guest RAM read() to get faulted pages
| |
V V
page fault ------------------------------>page offset is returned
block |
V
pull page from the source write the page contents to the shmem.
|
V
unblock<-----------------------------write() to tell served pages the fault handler returns the page
page fault is resolved
|
| pages can be sent
| backgroundly
| |
| V
| write()
| |
V V
The specified pages<-----pipe------------request to touch pages
are made present by |
touching guest RAM. |
| |
V V
reply-------------pipe-------------> release the cached page
| madvise(MADV_REMOVE)
| |
V V

all the pages are pulled from the source

| |
V V
the vma becomes anonymous<----------------UMEM_MAKE_VMA_ANONYMOUS
(note: I'm not sure if this can be implemented or not)
| |
V V
migration completes exit()

Isaku Yamahata (41):
arch_init: export sort_ram_list() and ram_save_block()
arch_init: export RAM_SAVE_xxx flags for postcopy arch_init/ram_save: introduce constant for ram save version = 4
arch_init: refactor host_from_stream_offset() arch_init/ram_save_live: factor out RAM_SAVE_FLAG_MEM_SIZE case
arch_init: refactor ram_save_block() arch_init/ram_save_live: factor out ram_save_limit arch_init/ram_load: refactor ram_load
arch_init: introduce helper function to find ram block with id string
arch_init: simplify a bit by ram_find_block()
arch_init: factor out counting transferred bytes
arch_init: factor out setting last_block, last_offset exec.c: factor out qemu_get_ram_ptr()
exec.c: export last_ram_offset()
savevm: export qemu_peek_buffer, qemu_peek_byte, qemu_file_skip
savevm: qemu_pending_size() to return pending buffered size savevm, buffered_file: introduce method to drain buffer of buffered
file
QEMUFile: add qemu_file_fd() for later use savevm/QEMUFile: drop qemu_stdio_fd
savevm/QEMUFileSocket: drop duplicated member fd
savevm: rename QEMUFileSocket to QEMUFileFD, socket_close to fd_close savevm/QEMUFile: introduce qemu_fopen_fd
migration.c: remove redundant line in migrate_init()
migration: export migrate_fd_completed() and migrate_fd_cleanup()
migration: factor out parameters into MigrationParams
buffered_file: factor out buffer management logic
buffered_file: Introduce QEMUFileNonblock for nonblock write
buffered_file: add qemu_file to read/write to buffer in memory umem.h: import Linux umem.h
update-linux-headers.sh: teach umem.h to update-linux-headers.sh
configure: add CONFIG_POSTCOPY option
savevm: add new section that is used by postcopy
postcopy: introduce -postcopy and -postcopy-flags option postcopy outgoing: add -p and -n option to migrate command
postcopy: introduce helper functions for postcopy
postcopy: implement incoming part of postcopy live migration
postcopy: implement outgoing part of postcopy live migration postcopy/outgoing: add forward, backward option to specify the size
of prefault
postcopy/outgoing: implement prefault
migrate: add -m (movebg) option to migrate command migration/postcopy: add movebg mode

 Makefile.target                 |    5 +
 arch_init.c                     |  298 ++++---
 arch_init.h                     |   20 +
 block-migration.c               |    8 +-
 buffered_file.c                 |  322 ++++++--
 buffered_file.h                 |   32 +
 configure                       |   12 +
 cpu-all.h                       |    9 +
 exec-obsolete.h                 |    1 +
 exec.c                          |   87 ++-
 hmp-commands.hx                 |   18 +-
 hmp.c                           |   10 +-
 linux-headers/linux/umem.h      |   42 +
 migration-exec.c                |   12 +-
 migration-fd.c                  |   25 +-
 migration-postcopy-stub.c       |   77 ++
 migration-postcopy.c            | 1771 +++++++++++++++++++++++++++++++++++++++
 migration-tcp.c                 |   25 +-
 migration-unix.c                |   26 +-
 migration.c                     |   97 ++-
 migration.h                     |   47 +-
 qapi-schema.json                |    4 +-
 qemu-common.h                   |    2 +
 qemu-file.h                     |    8 +-
 qemu-options.hx                 |   25 +
 qmp-commands.hx                 |    4 +-
 savevm.c                        |  177 ++++-
 scripts/update-linux-headers.sh |    2 +-
 sysemu.h                        |    4 +-
 umem.c                          |  364 ++++++++
 umem.h                          |  101 +++
 vl.c                            |   16 +-
 vmstate.h                       |    2 +-
 33 files changed, 3373 insertions(+), 280 deletions(-)
 create mode 100644 linux-headers/linux/umem.h
 create mode 100644 migration-postcopy-stub.c
 create mode 100644 migration-postcopy.c
 create mode 100644 umem.c
 create mode 100644 umem.h

------------------------------

Signature · 1 line
yamahata
#2 lore raw
Quoted 11 lines · Isaku Yamahata
Quoted 1 line · Chegu Vinod

Hello Isaku Yamahata,

Hi.

Quoted 2 lines

I just saw your patches..Would it be possible to email me a tar bundle of these patches (makes it easier to apply the patches to a copy of the upstream qemu.git)

I uploaded them to github for those who are interested in it.

git://github.com/yamahata/qemu.git qemu-postcopy-june-04-2012
git://github.com/yamahata/linux-umem.git linux-umem-june-04-2012

Thanks for the pointer...

Quoted 3 lines
Quoted 2 lines

BTW, I am also curious if you have considered using any kind of RDMA features for optimizing the page-faults during postcopy ?

Yes, RDMA is interesting topic. Can we share your use case/concern/issues?

Looking at large sized guests (256GB and higher) running cpu/memory intensive enterprise workloads.
The concerns are the same...i.e. having a predictable total migration time, minimal downtime/freeze-time and of course minimal service degradation to the workload(s) in the VM or the co-located VM's...

How large of a guest have you tested your changes with and what kind of workloads have you used so far ?

Quoted 2 lines

Thus we can collaborate.
You may want to see Benoit's results.

Yes. 'have already seen some of Benoit's results.

Hence the question about use of RDMA techniques for post copy.

Quoted 2 lines

As long as I know, he has not published
his code yet.

Thanks
Vinod

Quoted 274 lines

thanks,

Quoted 271 lines

Thanks
Vinod

----------------------------------------------------------------------

Message: 1
Date: Mon, 4 Jun 2012 18:57:02 +0900
From: Isaku Yamahata<[email protected]>
To: [email protected], [email protected]
Cc: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected]
Subject: [Qemu-devel] [PATCH v2 00/41] postcopy live migration Message-ID:<[email protected]>

After the long time, we have v2. This is qemu part. The linux kernel part is sent separatedly.

Changes v1 -> v2:
- split up patches for review
- buffered file refactored
- many bug fixes
Espcially PV drivers can work with postcopy
- optimization/heuristic

Patches
1 - 30: refactoring exsiting code and preparation 31 - 37: implement postcopy itself (essential part) 38 - 41: some optimization/heuristic for postcopy

Intro
=====
This patch series implements postcopy live migration.[1]
As discussed at KVM forum 2011, dedicated character device is used for distributed shared memory between migration source and destination. Now we can discuss/benchmark/compare with precopy. I believe there are much rooms for improvement.

[1] http://wiki.qemu.org/Features/PostCopyLiveMigration

Usage
=====
You need load umem character device on the host before starting migration. Postcopy can be used for tcg and kvm accelarator. The implementation depend on only linux umem character device. But the driver dependent code is split into a file.
I tested only host page size == guest page size case, but the implementation allows host page size != guest page size case.

The following options are added with this patch series.
- incoming part
command line options
-postcopy [-postcopy-flags<flags>]
where flags is for changing behavior for benchmark/debugging Currently the following flags are available
0: default
1: enable touching page request

example:
qemu -postcopy -incoming tcp:0:4444 -monitor stdio -machine accel=kvm

- outging part
options for migrate command
migrate [-p [-n] [-m]] URI [<prefault forward> [<prefault backword>]]
-p: indicate postcopy migration
-n: disable background transferring pages: This is for benchmark/debugging -m: move background transfer of postcopy mode
<prefault forward>: The number of forward pages which is sent with on-demand <prefault backward>: The number of backward pages which is sent with
on-demand

example:
migrate -p -n tcp:<dest ip address>:4444 migrate -p -n -m tcp:<dest ip address>:4444 32 0

TODO
====
- benchmark/evaluation. Especially how async page fault affects the result.
- improve/optimization
At the moment at least what I'm aware of is
- making incoming socket non-blocking with thread
As page compression is comming, it is impractical to non-blocking read and check if the necessary data is read.
- touching pages in incoming qemu process by fd handler seems suboptimal. creating dedicated thread?
- outgoing handler seems suboptimal causing latency.
- consider on FUSE/CUSE possibility
- don't fork umemd, but create thread?

basic postcopy work flow
========================
qemu on the destination
|
V
open(/dev/umem)
|
V
UMEM_INIT
|
V
Here we have two file descriptors to
umem device and shmem file
|
| umemd
| daemon on the destination
|
V create pipe to communicate
fork()---------------------------------------,
| |
V |
close(socket) V
close(shmem) mmap(shmem file)
| |
V V
mmap(umem device) for guest RAM close(shmem file)
| |
close(umem device) |
| |
V |
wait for ready from daemon<----pipe-----send ready message
| |
| Here the daemon takes over
send ok------------pipe---------------> the owner of the socket
| to the source
V |
entering post copy stage |
start guest execution |
| |
V V
access guest RAM read() to get faulted pages
| |
V V
page fault ------------------------------>page offset is returned
block |
V
pull page from the source write the page contents to the shmem.
|
V
unblock<-----------------------------write() to tell served pages the fault handler returns the page
page fault is resolved
|
| pages can be sent
| backgroundly
| |
| V
| write()
| |
V V
The specified pages<-----pipe------------request to touch pages
are made present by |
touching guest RAM. |
| |
V V
reply-------------pipe-------------> release the cached page
| madvise(MADV_REMOVE)
| |
V V

all the pages are pulled from the source

| |
V V
the vma becomes anonymous<----------------UMEM_MAKE_VMA_ANONYMOUS
(note: I'm not sure if this can be implemented or not)
| |
V V
migration completes exit()

Isaku Yamahata (41):
arch_init: export sort_ram_list() and ram_save_block()
arch_init: export RAM_SAVE_xxx flags for postcopy arch_init/ram_save: introduce constant for ram save version = 4
arch_init: refactor host_from_stream_offset() arch_init/ram_save_live: factor out RAM_SAVE_FLAG_MEM_SIZE case
arch_init: refactor ram_save_block() arch_init/ram_save_live: factor out ram_save_limit arch_init/ram_load: refactor ram_load
arch_init: introduce helper function to find ram block with id string
arch_init: simplify a bit by ram_find_block()
arch_init: factor out counting transferred bytes
arch_init: factor out setting last_block, last_offset exec.c: factor out qemu_get_ram_ptr()
exec.c: export last_ram_offset()
savevm: export qemu_peek_buffer, qemu_peek_byte, qemu_file_skip
savevm: qemu_pending_size() to return pending buffered size savevm, buffered_file: introduce method to drain buffer of buffered
file
QEMUFile: add qemu_file_fd() for later use savevm/QEMUFile: drop qemu_stdio_fd savevm/QEMUFileSocket: drop duplicated member fd
savevm: rename QEMUFileSocket to QEMUFileFD, socket_close to fd_close savevm/QEMUFile: introduce qemu_fopen_fd
migration.c: remove redundant line in migrate_init()
migration: export migrate_fd_completed() and migrate_fd_cleanup()
migration: factor out parameters into MigrationParams
buffered_file: factor out buffer management logic
buffered_file: Introduce QEMUFileNonblock for nonblock write
buffered_file: add qemu_file to read/write to buffer in memory umem.h: import Linux umem.h
update-linux-headers.sh: teach umem.h to update-linux-headers.sh
configure: add CONFIG_POSTCOPY option
savevm: add new section that is used by postcopy
postcopy: introduce -postcopy and -postcopy-flags option postcopy outgoing: add -p and -n option to migrate command
postcopy: introduce helper functions for postcopy
postcopy: implement incoming part of postcopy live migration
postcopy: implement outgoing part of postcopy live migration postcopy/outgoing: add forward, backward option to specify the size
of prefault
postcopy/outgoing: implement prefault
migrate: add -m (movebg) option to migrate command migration/postcopy: add movebg mode

  Makefile.target                 |    5 +
  arch_init.c                     |  298 ++++---
  arch_init.h                     |   20 +
  block-migration.c               |    8 +-
  buffered_file.c                 |  322 ++++++--
  buffered_file.h                 |   32 +
  configure                       |   12 +
  cpu-all.h                       |    9 +
  exec-obsolete.h                 |    1 +
  exec.c                          |   87 ++-
  hmp-commands.hx                 |   18 +-
  hmp.c                           |   10 +-
  linux-headers/linux/umem.h      |   42 +
  migration-exec.c                |   12 +-
  migration-fd.c                  |   25 +-
  migration-postcopy-stub.c       |   77 ++
  migration-postcopy.c            | 1771 +++++++++++++++++++++++++++++++++++++++
  migration-tcp.c                 |   25 +-
  migration-unix.c                |   26 +-
  migration.c                     |   97 ++-
  migration.h                     |   47 +-
  qapi-schema.json                |    4 +-
  qemu-common.h                   |    2 +
  qemu-file.h                     |    8 +-
  qemu-options.hx                 |   25 +
  qmp-commands.hx                 |    4 +-
  savevm.c                        |  177 ++++-
  scripts/update-linux-headers.sh |    2 +-
  sysemu.h                        |    4 +-
  umem.c                          |  364 ++++++++
  umem.h                          |  101 +++
  vl.c                            |   16 +-
  vmstate.h                       |    2 +-
  33 files changed, 3373 insertions(+), 280 deletions(-)

create mode 100644 linux-headers/linux/umem.h
create mode 100644 migration-postcopy-stub.c
create mode 100644 migration-postcopy.c
create mode 100644 umem.c
create mode 100644 umem.h

------------------------------

author
#3 lore raw
Quoted 19 lines · Chegu Vinod
Quoted 11 lines · Isaku Yamahata
Quoted 1 line · Chegu Vinod

Hello Isaku Yamahata,

Hi.

Quoted 2 lines

I just saw your patches..Would it be possible to email me a tar bundle of these patches (makes it easier to apply the patches to a copy of the upstream qemu.git)

I uploaded them to github for those who are interested in it.

git://github.com/yamahata/qemu.git qemu-postcopy-june-04-2012
git://github.com/yamahata/linux-umem.git linux-umem-june-04-2012

Thanks for the pointer...

Quoted 3 lines
Quoted 2 lines

BTW, I am also curious if you have considered using any kind of RDMA features for optimizing the page-faults during postcopy ?

Yes, RDMA is interesting topic. Can we share your use case/concern/issues?

Quoted 8 lines

Looking at large sized guests (256GB and higher) running cpu/memory intensive enterprise workloads.
The concerns are the same...i.e. having a predictable total migration time, minimal downtime/freeze-time and of course minimal service degradation to the workload(s) in the VM or the co-located VM's...

How large of a guest have you tested your changes with and what kind of workloads have you used so far ?

Only up to several GB VM. Off course We'd like to benchmark with real huge VM (several hundred GB), but it's somewhat difficult.

Quoted 4 lines
Quoted 2 lines

Thus we can collaborate.
You may want to see Benoit's results.

Yes. 'have already seen some of Benoit's results.

Great.

Quoted 1 line

Hence the question about use of RDMA techniques for post copy.

So far my implementation doesn't used RDMA.

Quoted 281 lines
Quoted 2 lines

As long as I know, he has not published
his code yet.

Thanks
Vinod

Quoted 274 lines

thanks,

Quoted 271 lines

Thanks
Vinod

----------------------------------------------------------------------

Message: 1
Date: Mon, 4 Jun 2012 18:57:02 +0900
From: Isaku Yamahata<[email protected]>
To: [email protected], [email protected]
Cc: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected],
[email protected]
Subject: [Qemu-devel] [PATCH v2 00/41] postcopy live migration Message-ID:<[email protected]>

After the long time, we have v2. This is qemu part. The linux kernel part is sent separatedly.

Changes v1 -> v2:
- split up patches for review
- buffered file refactored
- many bug fixes
Espcially PV drivers can work with postcopy
- optimization/heuristic

Patches
1 - 30: refactoring exsiting code and preparation 31 - 37: implement postcopy itself (essential part) 38 - 41: some optimization/heuristic for postcopy

Intro
=====
This patch series implements postcopy live migration.[1]
As discussed at KVM forum 2011, dedicated character device is used for distributed shared memory between migration source and destination. Now we can discuss/benchmark/compare with precopy. I believe there are much rooms for improvement.

[1] http://wiki.qemu.org/Features/PostCopyLiveMigration

Usage
=====
You need load umem character device on the host before starting migration. Postcopy can be used for tcg and kvm accelarator. The implementation depend on only linux umem character device. But the driver dependent code is split into a file.
I tested only host page size == guest page size case, but the implementation allows host page size != guest page size case.

The following options are added with this patch series.
- incoming part
command line options
-postcopy [-postcopy-flags<flags>]
where flags is for changing behavior for benchmark/debugging Currently the following flags are available
0: default
1: enable touching page request

example:
qemu -postcopy -incoming tcp:0:4444 -monitor stdio -machine accel=kvm

- outging part
options for migrate command
migrate [-p [-n] [-m]] URI [<prefault forward> [<prefault backword>]]
-p: indicate postcopy migration
-n: disable background transferring pages: This is for benchmark/debugging -m: move background transfer of postcopy mode
<prefault forward>: The number of forward pages which is sent with on-demand <prefault backward>: The number of backward pages which is sent with
on-demand

example:
migrate -p -n tcp:<dest ip address>:4444 migrate -p -n -m tcp:<dest ip address>:4444 32 0

TODO
====
- benchmark/evaluation. Especially how async page fault affects the result.
- improve/optimization
At the moment at least what I'm aware of is
- making incoming socket non-blocking with thread
As page compression is comming, it is impractical to non-blocking read and check if the necessary data is read.
- touching pages in incoming qemu process by fd handler seems suboptimal. creating dedicated thread?
- outgoing handler seems suboptimal causing latency.
- consider on FUSE/CUSE possibility
- don't fork umemd, but create thread?

basic postcopy work flow
========================
qemu on the destination
|
V
open(/dev/umem)
|
V
UMEM_INIT
|
V
Here we have two file descriptors to
umem device and shmem file
|
| umemd
| daemon on the destination
|
V create pipe to communicate
fork()---------------------------------------,
| |
V |
close(socket) V
close(shmem) mmap(shmem file)
| |
V V
mmap(umem device) for guest RAM close(shmem file)
| |
close(umem device) |
| |
V |
wait for ready from daemon<----pipe-----send ready message
| |
| Here the daemon takes over
send ok------------pipe---------------> the owner of the socket
| to the source
V |
entering post copy stage |
start guest execution |
| |
V V
access guest RAM read() to get faulted pages
| |
V V
page fault ------------------------------>page offset is returned
block |
V
pull page from the source write the page contents to the shmem.
|
V
unblock<-----------------------------write() to tell served pages the fault handler returns the page
page fault is resolved
|
| pages can be sent
| backgroundly
| |
| V
| write()
| |
V V
The specified pages<-----pipe------------request to touch pages
are made present by |
touching guest RAM. |
| |
V V
reply-------------pipe-------------> release the cached page
| madvise(MADV_REMOVE)
| |
V V

all the pages are pulled from the source

| |
V V
the vma becomes anonymous<----------------UMEM_MAKE_VMA_ANONYMOUS
(note: I'm not sure if this can be implemented or not)
| |
V V
migration completes exit()

Isaku Yamahata (41):
arch_init: export sort_ram_list() and ram_save_block()
arch_init: export RAM_SAVE_xxx flags for postcopy arch_init/ram_save: introduce constant for ram save version = 4
arch_init: refactor host_from_stream_offset() arch_init/ram_save_live: factor out RAM_SAVE_FLAG_MEM_SIZE case
arch_init: refactor ram_save_block() arch_init/ram_save_live: factor out ram_save_limit arch_init/ram_load: refactor ram_load
arch_init: introduce helper function to find ram block with id string
arch_init: simplify a bit by ram_find_block()
arch_init: factor out counting transferred bytes
arch_init: factor out setting last_block, last_offset exec.c: factor out qemu_get_ram_ptr()
exec.c: export last_ram_offset()
savevm: export qemu_peek_buffer, qemu_peek_byte, qemu_file_skip
savevm: qemu_pending_size() to return pending buffered size savevm, buffered_file: introduce method to drain buffer of buffered
file
QEMUFile: add qemu_file_fd() for later use savevm/QEMUFile: drop qemu_stdio_fd savevm/QEMUFileSocket: drop duplicated member fd
savevm: rename QEMUFileSocket to QEMUFileFD, socket_close to fd_close savevm/QEMUFile: introduce qemu_fopen_fd
migration.c: remove redundant line in migrate_init()
migration: export migrate_fd_completed() and migrate_fd_cleanup()
migration: factor out parameters into MigrationParams
buffered_file: factor out buffer management logic
buffered_file: Introduce QEMUFileNonblock for nonblock write
buffered_file: add qemu_file to read/write to buffer in memory umem.h: import Linux umem.h
update-linux-headers.sh: teach umem.h to update-linux-headers.sh
configure: add CONFIG_POSTCOPY option
savevm: add new section that is used by postcopy
postcopy: introduce -postcopy and -postcopy-flags option postcopy outgoing: add -p and -n option to migrate command
postcopy: introduce helper functions for postcopy
postcopy: implement incoming part of postcopy live migration
postcopy: implement outgoing part of postcopy live migration postcopy/outgoing: add forward, backward option to specify the size
of prefault
postcopy/outgoing: implement prefault
migrate: add -m (movebg) option to migrate command migration/postcopy: add movebg mode

  Makefile.target                 |    5 +
  arch_init.c                     |  298 ++++---
  arch_init.h                     |   20 +
  block-migration.c               |    8 +-
  buffered_file.c                 |  322 ++++++--
  buffered_file.h                 |   32 +
  configure                       |   12 +
  cpu-all.h                       |    9 +
  exec-obsolete.h                 |    1 +
  exec.c                          |   87 ++-
  hmp-commands.hx                 |   18 +-
  hmp.c                           |   10 +-
  linux-headers/linux/umem.h      |   42 +
  migration-exec.c                |   12 +-
  migration-fd.c                  |   25 +-
  migration-postcopy-stub.c       |   77 ++
  migration-postcopy.c            | 1771 +++++++++++++++++++++++++++++++++++++++
  migration-tcp.c                 |   25 +-
  migration-unix.c                |   26 +-
  migration.c                     |   97 ++-
  migration.h                     |   47 +-
  qapi-schema.json                |    4 +-
  qemu-common.h                   |    2 +
  qemu-file.h                     |    8 +-
  qemu-options.hx                 |   25 +
  qmp-commands.hx                 |    4 +-
  savevm.c                        |  177 ++++-
  scripts/update-linux-headers.sh |    2 +-
  sysemu.h                        |    4 +-
  umem.c                          |  364 ++++++++
  umem.h                          |  101 +++
  vl.c                            |   16 +-
  vmstate.h                       |    2 +-
  33 files changed, 3373 insertions(+), 280 deletions(-)

create mode 100644 linux-headers/linux/umem.h
create mode 100644 migration-postcopy-stub.c
create mode 100644 migration-postcopy.c
create mode 100644 umem.c
create mode 100644 umem.h

------------------------------

Signature · 1 line
yamahata