Skip to content

COLO: integrate colo frame with block replication and COLO proxy

Zhang Chen qemu-devel 12 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v12 Latest19 patches 10 repliesIn review
  2. v1120 patches 14 repliesSuperseded
  3. v1020 patches 12 repliesSuperseded
Earlier versions · 9
VersionPostedPatchesRepliesStatus
v920 patches7 repliesSuperseded
v817 patches28 repliesSuperseded
v7 posting 2/217 patches18 repliesSuperseded
v7 posting 1/217 patches6 repliesSuperseded
v6 posting 2/20 of 17 patches0 repliesSuperseded
v6 posting 1/217 patches2 repliesSuperseded
v517 patches2 repliesSuperseded
v416 patches8 repliesSuperseded
v316 patches2 repliesSuperseded

Changes in v12 from the cover letter

- Rebased on upstream.
- Removed the patch 15/20 for feature work as Jason's comments.
- Fixed failover bugs in patch 16/19.
- Renamed the dummy_handle_event to default_handle_event in patch 15/19.
- Cleaned needless check job.

Patches in v12 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/19filter-rewriter: Add TCP state machine and fix memory leak in connection_track_tablenet/colo.c net/colo.h net/filter-rewriter.c+100 -162 replies
2/19colo-compare: implement the process of checkpointinclude/migration/colo.h net/colo-compare.c net/colo-compare.h+106 -00 replies
3/19colo-compare: use notifier to notify packets comparing resultnet/colo-compare.c net/colo-compare.h+28 -110 replies
4/19COLO: integrate colo compare with colo framemigration/colo.c migration/migration.c+42 -60 replies
5/19COLO: Add block replication into colo processmigration/colo.c migration/migration.c+53 -03 replies
6/19COLO: Remove colo_state migration structinclude/migration/colo.h migration/Makefile.objs migration/colo-comm.c +6 more+57 -830 replies
7/19COLO: Load dirty pages into SVM's RAM cache firstlyinclude/exec/ram_addr.h migration/migration.c migration/ram.c +2 more+94 -30 replies
8/19ram/COLO: Record the dirty pages that SVM receivedmigration/ram.c+40 -30 replies
9/19COLO: Flush memory data from ram cachemigration/ram.c migration/trace-events+39 -00 replies
10/19qmp event: Add COLO_EXIT event to notify users while exited COLOmigration/colo.c qapi/migration.json+69 -00 replies
11/19qapi/migration.json: Rename COLO unknown mode to none mode.migration/colo-failover.c migration/colo.c qapi/migration.json+7 -70 replies
12/19qapi: Add new command to query colo statusmigration/colo.c qapi/migration.json+53 -00 replies
13/19savevm: split the process of different stages for loadvm/savevmmigration/colo.c migration/savevm.c migration/savevm.h+65 -160 replies
14/19COLO: flush host dirty ram from cachemigration/ram.c+9 -00 replies
15/19filter: Add handle_event method for NetFilterClassinclude/net/filter.h net/filter.c net/net.c+41 -00 replies
16/19filter-rewriter: handle checkpoint and failover eventnet/colo-compare.c net/colo.c net/colo.h +1 more+73 -60 replies
17/19COLO: notify net filters about checkpoint/failover eventmigration/colo.c+15 -00 replies
18/19COLO: quick failover process by kick COLO threadmigration/colo.c+8 -00 replies
19/19docs: Add COLO status diagram to COLO-FT.txtdocs/COLO-FT.txt+34 -01 reply

Reviews trailers per patch

12 of 19 patches have a Reviewed-by or Acked-by.

PatchDr. David Alan GilbertEric BlakeMarkus Armbruster
1/19
2/19
3/19
4/19R-b
5/19R-b
6/19R-b
7/19R-b
8/19R-b
9/19R-b
10/19R-b R-b
11/19R-b R-b
12/19
13/19R-b
14/19R-b
15/19
16/19
17/19R-b
18/19R-b
19/19

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v12 thread

Cover letter [Qemu-devel] [PATCH V12 00/19] COLO: integrate colo frame with block replication and COLO proxy

Hi~ All~

COLO Frame, block replication and COLO proxy(colo-compare,filter-mirror, filter-redirector,filter-rewriter) have been exist in qemu
for long time, it's time to integrate these three parts to make COLO really works.

In this series, we have some optimizations for COLO frame, including separating the process of saving ram and device state, using an COLO_EXIT event to notify users that VM exits COLO, for these parts, most of them have been reviewed long time ago in old version, but since this series have just rebased on upstream which had merged a new series of migration, parts of pathes in this series deserve review again.

We use notifier/callback method for COLO compare to notify COLO frame about net packets inconsistent event, and add a handle_event method for NetFilterClass to help COLO frame to notify filters and colo-compare about checkpoint/failover event, it is flexible.

For the neweset version, please refer to:
https://github.com/zhangckid/qemu/tree/qemu-colo-18sep1

Please review, thanks.

V12:
- Rebased on upstream.
- Removed the patch 15/20 for feature work as Jason's comments.
- Fixed failover bugs in patch 16/19.
- Renamed the dummy_handle_event to default_handle_event in patch 15/19.
- Cleaned needless check job.

V11:
- Rebased on upstream.
- Used "RAMBLOCK_FOREACH_MIGRATABLE()" to replace "QLIST_FOREACH_RCU()" in patch 08/20.
- Fixed COLO related qapi command's since version in patch 10/20.

V10:
- Rebased on upstream.
- Removed the "active" in COLOState.
- Fixed some comments.

V9:
- Rebased on upstream codes.
- Addressed Jason's comments add TCP state machine track in filter-rewriter.
- Fix some bug in colo-compare.
- Fix typo.
- Add filter-rewriter failover handle.
- Add net client type check in colo-compare.
- Add COLO state diagram.
- Addressed Markus and Daive's comments.

V8:
- Rebased on upstream codes.
- Addressed Markus's comments in patch 10/17.
- Addressed Markus's comments in patch 11/17.
- Removed some comments in patch 4/17.
- Moved the "migration_bitmap_clear_dirty()" to suitable position in patch 9/17.
- Rewrote the patch 07/17 to address Davie's comments.
- Moved the "qemu_savevm_live_state" out of the qemu_mutex_lock_iothread.
- Fixed the bug that in some status COLO vm crash with segmentation fault.

V7:
- Addressed Markus's comments in 11/17.
- Rebased on upstream.

V6:
- Addressed Eric Blake's comments, use the enum to feedback in patch 11/17.
- Fixed QAPI command separator problem in patch 11/17.

Zhang Chen (15):
filter-rewriter: Add TCP state machine and fix memory leak in
connection_track_table
colo-compare: implement the process of checkpoint
colo-compare: use notifier to notify packets comparing result
COLO: integrate colo compare with colo frame
COLO: Add block replication into colo process
COLO: Remove colo_state migration struct
COLO: Load dirty pages into SVM's RAM cache firstly ram/COLO: Record the dirty pages that SVM received
COLO: Flush memory data from ram cache qapi/migration.json: Rename COLO unknown mode to none mode.
qapi: Add new command to query colo status
savevm: split the process of different stages for loadvm/savevm
filter: Add handle_event method for NetFilterClass
filter-rewriter: handle checkpoint and failover event
docs: Add COLO status diagram to COLO-FT.txt

zhanghailiang (4):
qmp event: Add COLO_EXIT event to notify users while exited COLO
COLO: flush host dirty ram from cache
COLO: notify net filters about checkpoint/failover event
COLO: quick failover process by kick COLO thread

 docs/COLO-FT.txt          |  34 ++++++
 include/exec/ram_addr.h   |   1 +
 include/migration/colo.h  |  11 +-
 include/net/filter.h      |   5 +
 migration/Makefile.objs   |   2 +-
 migration/colo-comm.c     |  76 --------------
 migration/colo-failover.c |   2 +-
 migration/colo.c          | 212 ++++++++++++++++++++++++++++++++++++--
 migration/migration.c     |  46 ++++++++-
 migration/ram.c           | 166 ++++++++++++++++++++++++++++-
 migration/ram.h           |   4 +
 migration/savevm.c        |  53 ++++++++--
 migration/savevm.h        |   5 +
 migration/trace-events    |   3 +
 net/colo-compare.c        | 115 +++++++++++++++++++--
 net/colo-compare.h        |  24 +++++
 net/colo.c                |  10 +-
 net/colo.h                |  11 +-
 net/filter-rewriter.c     | 162 +++++++++++++++++++++++++++--
 net/filter.c              |  17 +++
 net/net.c                 |  19 ++++
 qapi/migration.json       |  80 +++++++++++++-
 vl.c                      |   2 -
 23 files changed, 921 insertions(+), 139 deletions(-)
 delete mode 100644 migration/colo-comm.c
 create mode 100644 net/colo-compare.h
Signature · 1 line
2.17.GIT