Block replication for continuous checkpoints

---
 block.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

block.c +17 -0

index 4bdc6b3..1e5a4fd 100644
@@ -1255,6 +1255,23 @@ void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd)
     /* Otherwise we won't be able to commit due to check in bdrv_commit */
     bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_COMMIT_TARGET,
                     bs->backing_blocker);
+    /*
+     * We do backup in 3 ways:
+     * 1. drive backup
+     *    The target bs is new opened, and the source is top BDS
+     * 2. blockdev backup
+     *    Both the source and the target are top BDSes.
+     * 3. internal backup(used for block replication)
+     *    Both the source and the target are backing file
+     *
+     * In case 1 and 2, neither the source nor the target is the backing file.
+     * In case 3, we will block the top BDS, so there is only one block job
+     * for the top BDS and its backing chain.
+     */
+    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_SOURCE,
+                    bs->backing_blocker);
+    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_TARGET,
+                    bs->backing_blocker);
 out:
     bdrv_refresh_limits(bs, NULL);
 }

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #2 lore raw

PATCH v18 2/8 Backup: clear all bitmap when doing block checkpoint

From: Wen Congyang <[email protected]>

Signed-off-by Gonglei <[email protected]>

---
 block/backup.c            | 17 +++++++++++++++++
 include/block/block_int.h |  2 ++
 2 files changed, 19 insertions(+)

block/backup.c +17 -0

index 491fd14..d5ffc32 100644
@@ -259,6 +259,23 @@ static void backup_abort(BlockJob *job)
     }
 }
 
+void backup_do_checkpoint(BlockJob *job, Error **errp)
+{
+    BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+    int64_t len;
+
+    assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP);
+
+    if (backup_job->sync_mode != MIRROR_SYNC_MODE_NONE) {
+        error_setg(errp, "The backup job only supports block checkpoint in"
+                   " sync=none mode");
+        return;
+    }
+
+    len = DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size);
+    bitmap_zero(backup_job->done_bitmap, len);
+}
+
 static const BlockJobDriver backup_job_driver = {
     .instance_size  = sizeof(BackupBlockJob),
     .job_type       = BLOCK_JOB_TYPE_BACKUP,

include/block/block_int.h +2 -0

index 7e238a0..3c3caa7 100644
@@ -718,6 +718,8 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
                                   const BdrvChildRole *child_role);
 void bdrv_root_unref_child(BdrvChild *child);
 
+void backup_do_checkpoint(BlockJob *job, Error **errp);
+
 void blk_dev_change_media_cb(BlockBackend *blk, bool load);
 bool blk_dev_has_removable_media(BlockBackend *blk);
 bool blk_dev_has_tray(BlockBackend *blk);

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #3 lore raw

PATCH v18 3/8 Link backup into block core

From: Wen Congyang <[email protected]>

Some programs that add a dependency on it will use the block layer directly.

Signed-off-by Gonglei <[email protected]>

Reviewed-by Stefan Hajnoczi <[email protected]>

Reviewed-by Jeff Cody <[email protected]>

---
 block/Makefile.objs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

block/Makefile.objs +1 -1

index 44a5416..fbfe647 100644
@@ -22,12 +22,12 @@ block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o
 block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
+block-obj-y += backup.o
 
 block-obj-y += crypto.o
 
 common-obj-y += stream.o
 common-obj-y += commit.o
-common-obj-y += backup.o
 
 iscsi.o-cflags     := $(LIBISCSI_CFLAGS)
 iscsi.o-libs       := $(LIBISCSI_LIBS)

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #4 lore raw

PATCH v18 4/8 docs: block replication's description

From: Wen Congyang <[email protected]>

Signed-off-by Gonglei <[email protected]>

---
 docs/block-replication.txt | 239 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 239 insertions(+)
 create mode 100644 docs/block-replication.txt

docs/block-replication.txt new file +239 -0

new file mode 100644
index 0000000..c5fc18b
@@ -0,0 +1,239 @@
+Block replication
+----------------------------------------
+Copyright Fujitsu, Corp. 2016
+Copyright (c) 2016 Intel Corporation
+Copyright (c) 2016 HUAWEI TECHNOLOGIES CO., LTD.
+
+This work is licensed under the terms of the GNU GPL, version 2 or later.
+See the COPYING file in the top-level directory.
+
+Block replication is used for continuous checkpoints. It is designed
+for COLO (COarse-grain LOck-stepping) where the Secondary VM is running.
+It can also be applied for FT/HA (Fault-tolerance/High Assurance) scenario,
+where the Secondary VM is not running.
+
+This document gives an overview of block replication's design.
+
+== Background ==
+High availability solutions such as micro checkpoint and COLO will do
+consecutive checkpoints. The VM state of the Primary and Secondary VM is
+identical right after a VM checkpoint, but becomes different as the VM
+executes till the next checkpoint. To support disk contents checkpoint,
+the modified disk contents in the Secondary VM must be buffered, and are
+only dropped at next checkpoint time. To reduce the network transportation
+effort during a vmstate checkpoint, the disk modification operations of
+the Primary disk are asynchronously forwarded to the Secondary node.
+
+== Workflow ==
+The following is the image of block replication workflow:
+
+        +----------------------+            +------------------------+
+        |Primary Write Requests|            |Secondary Write Requests|
+        +----------------------+            +------------------------+
+                  |                                       |
+                  |                                      (4)
+                  |                                       V
+                  |                              /-------------\
+                  |      Copy and Forward        |             |
+                  |---------(1)----------+       | Disk Buffer |
+                  |                      |       |             |
+                  |                     (3)      \-------------/
+                  |                 speculative      ^
+                  |                write through    (2)
+                  |                      |           |
+                  V                      V           |
+           +--------------+           +----------------+
+           | Primary Disk |           | Secondary Disk |
+           +--------------+           +----------------+
+
+    1) Primary write requests will be copied and forwarded to Secondary
+       QEMU.
+    2) Before Primary write requests are written to Secondary disk, the
+       original sector content will be read from Secondary disk and
+       buffered in the Disk buffer, but it will not overwrite the existing
+       sector content (it could be from either "Secondary Write Requests" or
+       previous COW of "Primary Write Requests") in the Disk buffer.
+    3) Primary write requests will be written to Secondary disk.
+    4) Secondary write requests will be buffered in the Disk buffer and it
+       will overwrite the existing sector content in the buffer.
+
+== Architecture ==
+We are going to implement block replication from many basic
+blocks that are already in QEMU.
+
+         virtio-blk       ||
+             ^            ||                            .----------
+             |            ||                            | Secondary
+        1 Quorum          ||                            '----------
+         /      \         ||
+        /        \        ||
+   Primary    2 filter
+     disk         ^                                                             virtio-blk
+                  |                                                                  ^
+                3 NBD  ------->  3 NBD                                               |
+                client    ||     server                                          2 filter
+                          ||        ^                                                ^
+--------.                 ||        |                                                |
+Primary |                 ||  Secondary disk <--------- hidden-disk 5 <--------- active-disk 4
+--------'                 ||        |          backing        ^       backing
+                          ||        |                         |
+                          ||        |                         |
+                          ||        '-------------------------'
+                          ||           drive-backup sync=none 6
+
+1) The disk on the primary is represented by a block device with two
+children, providing replication between a primary disk and the host that
+runs the secondary VM. The read pattern (fifo) for quorum can be extended
+to make the primary always read from the local disk instead of going through
+NBD.
+
+2) The new block filter (the name is replication) will control the block
+replication.
+
+3) The secondary disk receives writes from the primary VM through QEMU's
+embedded NBD server (speculative write-through).
+
+4) The disk on the secondary is represented by a custom block device
+(called active-disk). It should start as an empty disk, and the format
+should support bdrv_make_empty() and backing file.
+
+5) The hidden-disk is created automatically. It buffers the original content
+that is modified by the primary VM. It should also start as an empty disk,
+and the driver supports bdrv_make_empty() and backing file.
+
+6) The drive-backup job (sync=none) is run to allow hidden-disk to buffer
+any state that would otherwise be lost by the speculative write-through
+of the NBD server into the secondary disk. So before block replication,
+the primary disk and secondary disk should contain the same data.
+
+== Failure Handling ==
+There are 7 internal errors when block replication is running:
+1. I/O error on primary disk
+2. Forwarding primary write requests failed
+3. Backup failed
+4. I/O error on secondary disk
+5. I/O error on active disk
+6. Making active disk or hidden disk empty failed
+7. Doing failover failed
+In case 1 and 5, we just report the error to the disk layer. In case 2, 3,
+4 and 6, we just report block replication's error to FT/HA manager (which
+decides when to do a new checkpoint, when to do failover).
+In case 7, if active commit failed, we use replication failover failed state
+in Secondary's write operation (what decides which target to write).
+
+== New block driver interface ==
+We add three block driver interfaces to control block replication:
+a. replication_start_all()
+   Start block replication, called in migration/checkpoint thread.
+   We must call block_replication_start_all() in secondary QEMU before
+   calling block_replication_start_all() in primary QEMU. The caller
+   must hold the I/O mutex lock if it is in migration/checkpoint
+   thread.
+b. replication_do_checkpoint_all()
+   This interface is called after all VM state is transferred to
+   Secondary QEMU. The Disk buffer will be dropped in this interface.
+   The caller must hold the I/O mutex lock if it is in migration/checkpoint
+   thread.
+c. replication_get_error_all()
+   This interface is called to check if error happened in replication.
+   The caller must hold the I/O mutex lock if it is in migration/checkpoint
+   thread.
+d. replication_stop_all()
+   It is called on failover. We will flush the Disk buffer into
+   Secondary Disk and stop block replication. The vm should be stopped
+   before calling it if you use this API to shutdown the guest, or other
+   things except failover. The caller must hold the I/O mutex lock if it is
+   in migration/checkpoint thread.
+
+== Usage ==
+Primary:
+  -drive if=xxx,driver=quorum,read-pattern=fifo,id=colo1,vote-threshold=1,\
+         children.0.file.filename=1.raw,\
+         children.0.driver=raw
+
+  Run qmp command in primary qemu:
+    { 'execute': 'human-monitor-command',
+      'arguments': {
+          'command-line': 'drive_add -n buddy driver=replication,mode=primary,file.driver=nbd,file.host=xxxx,file.port=xxxx,file.export=colo1,node-name=nbd_client1'
+      }
+    }
+    { 'execute': 'x-blockdev-change',
+      'arguments': {
+          'parent': 'colo1',
+          'node': 'nbd_client1'
+      }
+    }
+  Note:
+  1. There should be only one NBD Client for each primary disk.
+  2. host is the secondary physical machine's hostname or IP
+  3. Each disk must have its own export name.
+  4. It is all a single argument to -drive and you should ignore the
+     leading whitespace.
+  5. The qmp command line must be run after running qmp command line in
+     secondary qemu.
+  6. After failover we need remove children.1 (replication driver).
+
+Secondary:
+  -drive if=none,driver=raw,file.filename=1.raw,id=colo1 \
+  -drive if=xxx,id=topxxx,driver=replication,mode=secondary,top-id=topxxx\
+         file.file.filename=active_disk.qcow2,\
+         file.driver=qcow2,\
+         file.backing.file.filename=hidden_disk.qcow2,\
+         file.backing.driver=qcow2,\
+         file.backing.backing=colo1
+
+  Then run qmp command in secondary qemu:
+    { 'execute': 'nbd-server-start',
+      'arguments': {
+          'addr': {
+              'type': 'inet',
+              'data': {
+                  'host': 'xxx',
+                  'port': 'xxx'
+              }
+          }
+      }
+    }
+    { 'execute': 'nbd-server-add',
+      'arguments': {
+          'device': 'colo1',
+          'writable': true
+      }
+    }
+
+  Note:
+  1. The export name in secondary QEMU command line is the secondary
+     disk's id.
+  2. The export name for the same disk must be the same
+  3. The qmp command nbd-server-start and nbd-server-add must be run
+     before running the qmp command migrate on primary QEMU
+  4. Active disk, hidden disk and nbd target's length should be the
+     same.
+  5. It is better to put active disk and hidden disk in ramdisk.
+  6. It is all a single argument to -drive, and you should ignore
+     the leading whitespace.
+
+After Failover:
+Primary:
+  The secondary host is down, so we should run the following qmp command
+  to remove the nbd child from the quorum:
+  { 'execute': 'x-blockdev-change',
+    'arguments': {
+        'parent': 'colo1',
+        'child': 'children.1'
+    }
+  }
+  { 'execute': 'human-monitor-command',
+    'arguments': {
+        'command-line': 'drive_del xxxx'
+    }
+  }
+  Note: there is no qmp command to remove the blockdev now
+
+Secondary:
+  The primary host is down, so we should do the following thing:
+  { 'execute': 'nbd-server-stop' }
+
+TODO:
+1. Continuous block replication
+2. Shared disk

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #5 lore raw

PATCH v18 5/8 auto complete active commit

From: Wen Congyang <[email protected]>

Auto complete mirror job in background to prevent from blocking synchronously

---
 block/mirror.c            | 13 +++++++++----
 blockdev.c                |  2 +-
 include/block/block_int.h |  3 ++-
 qemu-img.c                |  2 +-
 4 files changed, 13 insertions(+), 7 deletions(-)

block/mirror.c +9 -4

index c2cfc1a..1c4aeae 100644
@@ -790,7 +790,8 @@ static void mirror_start_job(BlockDriverState *bs, BlockDriverState *target,
                              BlockCompletionFunc *cb,
                              void *opaque, Error **errp,
                              const BlockJobDriver *driver,
-                             bool is_none_mode, BlockDriverState *base)
+                             bool is_none_mode, BlockDriverState *base,
+                             bool auto_complete)
 {
     MirrorBlockJob *s;
     BlockDriverState *replaced_bs;

@@ -846,6 +847,9 @@ static void mirror_start_job(BlockDriverState *bs, BlockDriverState *target,
     s->granularity = granularity;
     s->buf_size = ROUND_UP(buf_size, granularity);
     s->unmap = unmap;
+    if (auto_complete) {
+        s->should_complete = true;
+    }
 
     s->dirty_bitmap = bdrv_create_dirty_bitmap(bs, granularity, NULL, errp);
     if (!s->dirty_bitmap) {

@@ -886,14 +890,15 @@ void mirror_start(BlockDriverState *bs, BlockDriverState *target,
     mirror_start_job(bs, target, replaces,
                      speed, granularity, buf_size,
                      on_source_error, on_target_error, unmap, cb, opaque, errp,
-                     &mirror_job_driver, is_none_mode, base);
+                     &mirror_job_driver, is_none_mode, base, false);
 }
 
 void commit_active_start(BlockDriverState *bs, BlockDriverState *base,
                          int64_t speed,
                          BlockdevOnError on_error,
                          BlockCompletionFunc *cb,
-                         void *opaque, Error **errp)
+                         void *opaque, Error **errp,
+                         bool auto_complete)
 {
     int64_t length, base_length;
     int orig_base_flags;

@@ -934,7 +939,7 @@ void commit_active_start(BlockDriverState *bs, BlockDriverState *base,
     bdrv_ref(base);
     mirror_start_job(bs, base, NULL, speed, 0, 0,
                      on_error, on_error, false, cb, opaque, &local_err,
-                     &commit_active_job_driver, false, base);
+                     &commit_active_job_driver, false, base, auto_complete);
     if (local_err) {
         error_propagate(errp, local_err);
         goto error_restore_flags;

blockdev.c +1 -1

index d16d449..5457fdf 100644
@@ -3150,7 +3150,7 @@ void qmp_block_commit(const char *device,
             goto out;
         }
         commit_active_start(bs, base_bs, speed, on_error, block_job_cb,
-                            bs, &local_err);
+                            bs, &local_err, false);
     } else {
         commit_start(bs, base_bs, top_bs, speed, on_error, block_job_cb, bs,
                      has_backing_file ? backing_file : NULL, &local_err);

include/block/block_int.h +2 -1

index 3c3caa7..05469f7 100644
@@ -649,13 +649,14 @@ void commit_start(BlockDriverState *bs, BlockDriverState *base,
  * @cb: Completion function for the job.
  * @opaque: Opaque pointer value passed to @cb.
  * @errp: Error object.
+ * @auto_complete: Auto complete the job.
  *
  */
 void commit_active_start(BlockDriverState *bs, BlockDriverState *base,
                          int64_t speed,
                          BlockdevOnError on_error,
                          BlockCompletionFunc *cb,
-                         void *opaque, Error **errp);
+                         void *opaque, Error **errp, bool auto_complete);
 /*
  * mirror_start:
  * @bs: Block device to operate on.

qemu-img.c +1 -1

index 1697762..326640b 100644
@@ -915,7 +915,7 @@ static int img_commit(int argc, char **argv)
     };
 
     commit_active_start(bs, base_bs, 0, BLOCKDEV_ON_ERROR_REPORT,
-                        common_block_job_cb, &cbi, &local_err);
+                        common_block_job_cb, &cbi, &local_err, false);
     if (local_err) {
         goto done;
     }

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #6 lore raw

PATCH v18 6/8 Introduce new APIs to do replication operation

Signed-off-by Gonglei <[email protected]>

---
 Makefile.objs        |   1 +
 qapi/block-core.json |  13 ++++
 replication.c        | 105 ++++++++++++++++++++++++++++++
 replication.h        | 176 +++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 295 insertions(+)
 create mode 100644 replication.c
 create mode 100644 replication.h

Makefile.objs +1 -0

index 8f705f6..30d403f 100644
@@ -15,6 +15,7 @@ block-obj-$(CONFIG_POSIX) += aio-posix.o
 block-obj-$(CONFIG_WIN32) += aio-win32.o
 block-obj-y += block/
 block-obj-y += qemu-io-cmds.o
+block-obj-y += replication.o
 
 block-obj-m = block/

qapi/block-core.json +13 -0

index 98a20d2..e56cdf4 100644
@@ -2032,6 +2032,19 @@
             '*read-pattern': 'QuorumReadPattern' } }
 
 ##
+# @ReplicationMode
+#
+# An enumeration of replication modes.
+#
+# @primary: Primary mode, the vm's state will be sent to secondary QEMU.
+#
+# @secondary: Secondary mode, receive the vm's state from primary QEMU.
+#
+# Since: 2.7
+##
+{ 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ] }
+
+##
 # @BlockdevOptions
 #
 # Options for creating a block device.  Many options are available for all

replication.c new file +105 -0

105 lines not on this page · raw message

replication.h new file +176 -0

176 lines not on this page · raw message

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #7 lore raw

PATCH v18 7/8 Implement new driver for block replication

From: Wen Congyang <[email protected]>

Signed-off-by Gonglei <[email protected]>

---
 block/Makefile.objs      |   1 +
 block/replication.c      | 625 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/.gitignore         |   1 +
 tests/Makefile           |   4 +
 tests/test-replication.c | 505 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 1136 insertions(+)
 create mode 100644 block/replication.c
 create mode 100644 tests/test-replication.c

block/Makefile.objs +1 -0

index fbfe647..5e28b45 100644
@@ -23,6 +23,7 @@ block-obj-$(CONFIG_LIBSSH2) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 block-obj-y += backup.o
+block-obj-y += replication.o
 
 block-obj-y += crypto.o

block/replication.c new file +625 -0

625 lines not on this page · raw message

tests/.gitignore +1 -0

index 9eed229..a303aaa 100644
@@ -57,6 +57,7 @@ test-qmp-introspect.[ch]
 test-qmp-marshal.c
 test-qmp-output-visitor
 test-rcu-list
+test-replication
 test-rfifolock
 test-string-input-visitor
 test-string-output-visitor

tests/Makefile +4 -0

index 9de9598..babd2de 100644
@@ -99,6 +99,7 @@ check-unit-y += tests/test-crypto-xts$(EXESUF)
 check-unit-y += tests/test-crypto-block$(EXESUF)
 gcov-files-test-logging-y = tests/test-logging.c
 check-unit-y += tests/test-logging$(EXESUF)
+check-unit-y += tests/test-replication$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh

@@ -442,6 +443,9 @@ tests/test-base64$(EXESUF): tests/test-base64.o \
 
 tests/test-logging$(EXESUF): tests/test-logging.o $(test-util-obj-y)
 
+tests/test-replication$(EXESUF): tests/test-replication.o $(test-util-obj-y) \
+	$(test-block-obj-y)
+
 tests/test-qapi-types.c tests/test-qapi-types.h :\
 $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
 	$(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-types.py \

tests/test-replication.c new file +505 -0

505 lines not on this page · raw message

Signature · 1 line

1.9.3

Changlong Xie

author

2016-05-06 09:10 UTC #8 lore raw

Re: [Qemu-devel] [PATCH v18 7/8] Implement new driver for block replication

Quoted 6 lines · Changlong Xie

From: Wen Congyang<[email protected]>
Signed-off-by Wen Congyang<[email protected]>
Signed-off-by zhanghailiang<[email protected]>
Signed-off-by Gonglei<[email protected]>
Signed-off-by Changlong Xie<[email protected]>

Quoted 6 lines

---
  block/Makefile.objs      |   1 +
  block/replication.c      | 625 +++++++++++++++++++++++++++++++++++++++++++++++
  tests/.gitignore         |   1 +
  tests/Makefile           |   4 +
  tests/test-replication.c | 505 ++++++++++++++++++++++++++++++++++++++

I'll split test-replication.c out after v2.6 release for better review

Quoted 3 lines

  5 files changed, 1136 insertions(+)
create mode 100644 block/replication.c
create mode 100644 tests/test-replication.c

Stefan Hajnoczi

2016-05-06 15:47 UTC #9 lore raw

Re: [Qemu-devel] [PATCH v18 7/8] Implement new driver for block replication

In reply to Stefan Hajnoczi

block/replication.c line 137

Quoted 11 lines · Changlong Xie

block/replication.c new file +11 -0

+static void replication_close(BlockDriverState *bs)
+{
+    BDRVReplicationState *s = bs->opaque;
+
+    if (s->mode == REPLICATION_MODE_SECONDARY) {
+        g_free(s->top_id);
+    }
+
+    if (s->replication_state == BLOCK_REPLICATION_RUNNING) {
+        replication_stop(s->rs, false, NULL);
+    }

There is a possible use-after-free with s->top_id. If we free it above then replication_stop() must not call backup_job_cleanup(). I think it could call it from replication_stop().

It would be safer to call replication_stop() before freeing s->top_id.

Quoted 1 line

+ top_bs = bdrv_lookup_bs(s->top_id, s->top_id, errp);

Please check that bs is a child of top_bs. If it is not a child then strange things could happen, for example the AioContexts might not match (meaning it's not thread-safe) so this should be forbidden.

block/replication.c line 381

Quoted 4 lines

block/replication.c new file +4 -0

+        if (!top_bs) {
+            aio_context_release(aio_context);
+            return;
+        }

Error return paths after reopen_backing_file(s, true, &local_err) should undo the operation.

block/replication.c line 465

Quoted 13 lines

block/replication.c new file +13 -0

+        bdrv_op_block_all(top_bs, s->blocker);
+        bdrv_op_unblock(top_bs, BLOCK_OP_TYPE_DATAPLANE, s->blocker);
+
+        /*
+         * Must protect backup target if backup job was stopped/cancelled
+         * unexpectedly
+         */
+        bdrv_ref(s->hidden_disk->bs);
+
+        backup_start(s->secondary_disk->bs, s->hidden_disk->bs, 0,
+                     MIRROR_SYNC_MODE_NONE, NULL, BLOCKDEV_ON_ERROR_REPORT,
+                     BLOCKDEV_ON_ERROR_REPORT, backup_job_completed,
+                     s, NULL, &local_err);

Did you run stress tests where the primary is writing to the disk while the secondary reads from the same sectors?

I thought about this some more and I'm wondering about the following scenario:

NBD writes to secondary_disk and the guest reads from the disk at the same time. There is a coroutine mutex in qcow2.c that protects both read and write requests, but only until they perform the data I/O. It may be possible that the read request from the Secondary VM could be started before the NBD write but the preadv() syscall isn't entered because of CPU scheduling decisions. In the meantime the secondary_disk->hidden_disk backup operation takes place. With some unlucky timing it may be possible for the Secondary VM to read the new contents from secondary_disk instead of the old contents that were backed up into hidden_disk.

Extra serialization would be needed. block/backup.c:wait_for_overlapping_requests() and block/io.c:mark_request_serialising() are good starting points for solving this.

Quoted 1 line

+ cleanup_imgs();

Please use qtest_add_abrt_handler() so cleanup happens even when SIGABRT is received.

Changlong Xie

author

2016-05-10 09:46 UTC #10 lore raw

Re: [Qemu-devel] [PATCH v18 7/8] Implement new driver for block replication

Quoted 17 lines · Stefan Hajnoczi

block/replication.c line 137
Quoted 11 lines · Changlong Xie
block/replication.c new file +11 -0
+static void replication_close(BlockDriverState *bs)
+{
+    BDRVReplicationState *s = bs->opaque;
+
+    if (s->mode == REPLICATION_MODE_SECONDARY) {
+        g_free(s->top_id);
+    }
+
+    if (s->replication_state == BLOCK_REPLICATION_RUNNING) {
+        replication_stop(s->rs, false, NULL);
+    }
There is a possible use-after-free with s->top_id. If we free it above then replication_stop() must not call backup_job_cleanup(). I think it could call it from replication_stop().

Quoted 2 lines

It would be safer to call replication_stop() before freeing s->top_id.

Yes, you are right.

Quoted 6 lines

Quoted 1 line
+ top_bs = bdrv_lookup_bs(s->top_id, s->top_id, errp);
Please check that bs is a child of top_bs. If it is not a child then strange things could happen, for example the AioContexts might not match (meaning it's not thread-safe) so this should be forbidden.

Will fix in next version

Quoted 7 lines

block/replication.c line 381
Quoted 4 lines
block/replication.c new file +4 -0
+        if (!top_bs) {
+            aio_context_release(aio_context);
+            return;
+        }
Error return paths after reopen_backing_file(s, true, &local_err) should undo the operation.

Will do.

Quoted 21 lines

Quoted 4 lines

block/replication.c new file +3 -0

+        bdrv_op_block_all(top_bs, s->blocker);
+        bdrv_op_unblock(top_bs, BLOCK_OP_TYPE_DATAPLANE, s->blocker);
+

block/replication.c line 465

Quoted 10 lines

block/replication.c new file +10 -0

+        /*
+         * Must protect backup target if backup job was stopped/cancelled
+         * unexpectedly
+         */
+        bdrv_ref(s->hidden_disk->bs);
+
+        backup_start(s->secondary_disk->bs, s->hidden_disk->bs, 0,
+                     MIRROR_SYNC_MODE_NONE, NULL, BLOCKDEV_ON_ERROR_REPORT,
+                     BLOCKDEV_ON_ERROR_REPORT, backup_job_completed,
+                     s, NULL, &local_err);

Did you run stress tests where the primary is writing to the disk while the secondary reads from the same sectors?

I thought about this some more and I'm wondering about the following scenario:

Quoted 10 lines

NBD writes to secondary_disk and the guest reads from the disk at the same time. There is a coroutine mutex in qcow2.c that protects both read and write requests, but only until they perform the data I/O. It may be possible that the read request from the Secondary VM could be started before the NBD write but the preadv() syscall isn't entered because of CPU scheduling decisions. In the meantime the secondary_disk->hidden_disk backup operation takes place. With some unlucky timing it may be possible for the Secondary VM to read the new contents from secondary_disk instead of the old contents that were backed up into hidden_disk.

Thanks for your catch. I'll think about this scenario carefully.

Quoted 11 lines

Extra serialization would be needed. block/backup.c:wait_for_overlapping_requests() and block/io.c:mark_request_serialising() are good starting points for solving this.
Quoted 1 line
+ cleanup_imgs();
Please use qtest_add_abrt_handler() so cleanup happens even when SIGABRT is received.

Surely.

Thanks
-Xie

Changlong Xie

author

2016-05-18 09:15 UTC #11 lore raw

Re: [Qemu-devel] [PATCH v18 7/8] Implement new driver for block replication

Quoted 69 lines · Stefan Hajnoczi

block/replication.c line 137
Quoted 11 lines · Changlong Xie
block/replication.c new file +11 -0
+static void replication_close(BlockDriverState *bs)
+{
+    BDRVReplicationState *s = bs->opaque;
+
+    if (s->mode == REPLICATION_MODE_SECONDARY) {
+        g_free(s->top_id);
+    }
+
+    if (s->replication_state == BLOCK_REPLICATION_RUNNING) {
+        replication_stop(s->rs, false, NULL);
+    }
There is a possible use-after-free with s->top_id. If we free it above then replication_stop() must not call backup_job_cleanup(). I think it could call it from replication_stop().
It would be safer to call replication_stop() before freeing s->top_id.
Quoted 1 line
+ top_bs = bdrv_lookup_bs(s->top_id, s->top_id, errp);
Please check that bs is a child of top_bs. If it is not a child then strange things could happen, for example the AioContexts might not match (meaning it's not thread-safe) so this should be forbidden.
block/replication.c line 381
Quoted 4 lines
block/replication.c new file +4 -0
+        if (!top_bs) {
+            aio_context_release(aio_context);
+            return;
+        }
Error return paths after reopen_backing_file(s, true, &local_err) should undo the operation.
block/replication.c line 465
Quoted 13 lines
block/replication.c new file +13 -0
+        bdrv_op_block_all(top_bs, s->blocker);
+        bdrv_op_unblock(top_bs, BLOCK_OP_TYPE_DATAPLANE, s->blocker);
+
+        /*
+         * Must protect backup target if backup job was stopped/cancelled
+         * unexpectedly
+         */
+        bdrv_ref(s->hidden_disk->bs);
+
+        backup_start(s->secondary_disk->bs, s->hidden_disk->bs, 0,
+                     MIRROR_SYNC_MODE_NONE, NULL, BLOCKDEV_ON_ERROR_REPORT,
+                     BLOCKDEV_ON_ERROR_REPORT, backup_job_completed,
+                     s, NULL, &local_err);
Did you run stress tests where the primary is writing to the disk while the secondary reads from the same sectors?
I thought about this some more and I'm wondering about the following scenario:
NBD writes to secondary_disk and the guest reads from the disk at the same time. There is a coroutine mutex in qcow2.c that protects both read and write requests, but only until they perform the data I/O. It may be possible that the read request from the Secondary VM could be started before the NBD write but the preadv() syscall isn't entered because of CPU scheduling decisions. In the meantime the secondary_disk->hidden_disk backup operation takes place. With some unlucky timing it may be possible for the Secondary VM to read the new contents from secondary_disk instead of the old contents that were backed up into hidden_disk.
Extra serialization would be needed. block/backup.c:wait_for_overlapping_requests() and block/io.c:mark_request_serialising() are good starting points for solving this.

Quoted 5 lines

Quoted 1 line
+ cleanup_imgs();
Please use qtest_add_abrt_handler() so cleanup happens even when SIGABRT is received.

It seems unit tests can't use this interface, so i'll implement a similar function.

Thanks
-Xie

Changlong Xie

author

2016-05-18 09:19 UTC #12 lore raw

RFC 1/1 backup: export interfaces for extra serialization

In reply to Stefan Hajnoczi

Quoted 17 lines · Stefan Hajnoczi

Did you run stress tests where the primary is writing to the disk while the secondary reads from the same sectors?
I thought about this some more and I'm wondering about the following scenario:
NBD writes to secondary_disk and the guest reads from the disk at the same time. There is a coroutine mutex in qcow2.c that protects both read and write requests, but only until they perform the data I/O. It may be possible that the read request from the Secondary VM could be started before the NBD write but the preadv() syscall isn't entered because of CPU scheduling decisions. In the meantime the secondary_disk->hidden_disk backup operation takes place. With some unlucky timing it may be possible for the Secondary VM to read the new contents from secondary_disk instead of the old contents that were backed up into hidden_disk.

Quoted 4 lines

Extra serialization would be needed. block/backup.c:wait_for_overlapping_requests() and block/io.c:mark_request_serialising() are good starting points for solving this.

I'm worried about if this patch introduce unexpect deadlock, and would like ask for RFC here.

From 753d9a151351fb14ea774e36a2899f229a7e26ac Mon Sep 17 00:00:00 2001
From: Changlong Xie <[email protected]>
Date: Wed, 18 May 2016 16:19:51 +0800
Subject: [PATCH] [RFC] backup: export interfaces for extra serialization

Normal backup(sync='none') workflow:
step 1. NBD peformance I/O write from client to server
qcow2_co_writev

    bdrv_co_writev
     ...
       bdrv_aligned_pwritev
        notifier_with_return_list_notify -> backup_do_cow
         bdrv_driver_pwritev // write new contents

step 2. drive-backup sync=none
backup_do_cow
{
wait_for_overlapping_requests
cow_request_begin
for(; start < end; start++) {
bdrv_co_readv_no_serialising //read old contents from
Secondary disk
bdrv_co_writev // write old contents to hidden-disk
}
cow_request_end
}

step 3. Then roll back to "step 1" to write new contents to Secondary disk.

And for replication, we must make sure that we only read the old contents from Secondary disk in order to keep contents consistent.

1) Replication workflow of Secondary

                                                         virtio-blk
                                                              ^
------->  1 NBD                                               |
   ||     server                                       3 replication
   ||        ^                                                ^
   ||        |           backing                 backing      |
   ||  Secondary disk 6<-------- hidden-disk 5 <-------- active-disk 4
   ||        |                         ^
   ||        '-------------------------'
   ||           drive-backup sync=none 2

Hence, we need these interfaces to implement coarse-grained serialization between COW of Secondary disk and the read operation of replication.

Example codes about how to use them:

*#include "block/block_backup.h"

static coroutine_fn int xxx_co_readv()

{
        CowRequest req;
        BlockJob *job = secondary_disk->bs->job;

        if (job) {
              backup_wait_for_overlapping_requests(job, start, end);
              backup_cow_request_begin(&req, job, start, end);
              ret = bdrv_co_readv();
              backup_cow_request_end(&req);
              goto out;
        }
        ret = bdrv_co_readv();
out:
        return ret;
}

---
 block/backup.c               | 42 
+++++++++++++++++++++++++++++++++++-------
 include/block/block_backup.h | 13 +++++++++++++
 2 files changed, 48 insertions(+), 7 deletions(-)
 create mode 100644 include/block/block_backup.h

block/backup.c +35 -7

index d5ffc32..424d29d 100644
@@ -17,6 +17,7 @@
 #include "block/block.h"
 #include "block/block_int.h"
 #include "block/blockjob.h"
+#include "block/block_backup.h"
 #include "qapi/error.h"
 #include "qapi/qmp/qerror.h"
 #include "qemu/ratelimit.h"

@@ -27,13 +28,6 @@
 #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
 #define SLICE_TIME 100000000ULL /* ns */
 
-typedef struct CowRequest {
-    int64_t start;
-    int64_t end;
-    QLIST_ENTRY(CowRequest) list;
-    CoQueue wait_queue; /* coroutines blocked on this request */
-} CowRequest;
-
 typedef struct BackupBlockJob {
     BlockJob common;
     BlockDriverState *target;

@@ -276,6 +270,40 @@ void backup_do_checkpoint(BlockJob *job, Error **errp)
     bitmap_zero(backup_job->done_bitmap, len);
 }
 
+void backup_wait_for_overlapping_requests(BlockJob *job, int64_t sector_num,
+                                          int nb_sectors)
+{
+    BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+    int64_t sectors_per_cluster = cluster_size_sectors(backup_job);
+    int64_t start, end;
+
+    assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP);
+
+    start = sector_num / sectors_per_cluster;
+    end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster);
+    wait_for_overlapping_requests(backup_job, start, end);
+}
+
+void backup_cow_request_begin(CowRequest *req, BlockJob *job,
+                              int64_t sector_num,
+                              int nb_sectors)
+{
+    BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common);
+    int64_t sectors_per_cluster = cluster_size_sectors(backup_job);
+    int64_t start, end;
+
+    assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP);
+
+    start = sector_num / sectors_per_cluster;
+    end = DIV_ROUND_UP(sector_num + nb_sectors, sectors_per_cluster);
+    cow_request_begin(req, backup_job, start, end);
+}
+
+void backup_cow_request_end(CowRequest *req)
+{
+    cow_request_end(req);
+}
+
 static const BlockJobDriver backup_job_driver = {
     .instance_size  = sizeof(BackupBlockJob),
     .job_type       = BLOCK_JOB_TYPE_BACKUP,

include/block/block_backup.h new file +13 -0

new file mode 100644
index 0000000..80f5c5c
@@ -0,0 +1,13 @@
+typedef struct CowRequest {
+    int64_t start;
+    int64_t end;
+    QLIST_ENTRY(CowRequest) list;
+    CoQueue wait_queue; /* coroutines blocked on this request */
+} CowRequest;
+
+void backup_wait_for_overlapping_requests(BlockJob *job, int64_t sector_num,
+                                          int nb_sectors);
+void backup_cow_request_begin(CowRequest *req, BlockJob *job,
+                              int64_t sector_num,
+                              int nb_sectors);
+void backup_cow_request_end(CowRequest *req);

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-15 08:08 UTC #13 lore raw

PATCH v18 8/8 support replication driver in blockdev-add

From: Wen Congyang <[email protected]>

Signed-off-by Gonglei <[email protected]>

Reviewed-by Eric Blake <[email protected]>

---
 qapi/block-core.json | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

qapi/block-core.json +18 -2

index e56cdf4..b9f9839 100644
@@ -248,6 +248,7 @@
 #       2.3: 'host_floppy' deprecated
 #       2.5: 'host_floppy' dropped
 #       2.6: 'luks' added
+#       2.7: 'replication' added
 #
 # @backing_file: #optional the name of the backing file (for copy-on-write)
 #

@@ -1632,6 +1633,7 @@
 # Drivers that are supported in block device operations.
 #
 # @host_device, @host_cdrom: Since 2.1
+# @replication: Since 2.7
 #
 # Since: 2.0
 ##

@@ -1639,8 +1641,8 @@
   'data': [ 'archipelago', 'blkdebug', 'blkverify', 'bochs', 'cloop',
             'dmg', 'file', 'ftp', 'ftps', 'host_cdrom', 'host_device',
             'http', 'https', 'luks', 'null-aio', 'null-co', 'parallels',
-            'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'tftp', 'vdi', 'vhdx',
-            'vmdk', 'vpc', 'vvfat' ] }
+            'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'replication', 'tftp',
+            'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
 
 ##
 # @BlockdevOptionsFile

@@ -2045,6 +2047,19 @@
 { 'enum' : 'ReplicationMode', 'data' : [ 'primary', 'secondary' ] }
 
 ##
+# @BlockdevOptionsReplication
+#
+# Driver specific block device options for replication
+#
+# @mode: the replication mode
+#
+# Since: 2.7
+##
+{ 'struct': 'BlockdevOptionsReplication',
+  'base': 'BlockdevOptionsGenericFormat',
+  'data': { 'mode': 'ReplicationMode'  } }
+
+##
 # @BlockdevOptions
 #
 # Options for creating a block device.  Many options are available for all

@@ -2125,6 +2140,7 @@
       'quorum':     'BlockdevOptionsQuorum',
       'raw':        'BlockdevOptionsGenericFormat',
 # TODO rbd: Wait for structured options
+      'replication':'BlockdevOptionsReplication',
 # TODO sheepdog: Wait for structured options
 # TODO ssh: Should take InetSocketAddress for 'host'?
       'tftp':       'BlockdevOptionsFile',

Signature · 1 line

1.9.3

Changlong Xie

author

2016-04-25 09:38 UTC #14 lore raw

The main migration code for COLO looks OK now，and they depend on my block part. So i'd like to ping here.

Thanks
-Xie

Quoted 139 lines · Changlong Xie

Block replication is a very important feature which is used for continuous checkpoints(for example: COLO).
You can get the detailed information about block replication from here:
http://wiki.qemu.org/Features/BlockReplication
Usage:
Please refer to docs/block-replication.txt
This patch series is based on the following patch series:
http://lists.nongnu.org/archive/html/qemu-devel/2016-04/msg02093.html
You can get the patch here:
https://github.com/Pating/qemu/tree/changlox/block-replication-v18
You can get the patch with framework here:
https://github.com/Pating/qemu/tree/changlox/colo_framework_v17
TODO:
1. Continuous block replication. It will be started after basic functions are accepted.
Changs Log:
V18:
p6: add local_err in all replication callbacks to prevent "errp == NULL"
p7: add missing qemu_iovec_destroy(xxx)
V17:
1. Rebase to the lastest codes
p2: refactor backup_do_checkpoint addressed comments from Jeff Cody
p4: fix bugs in "drive_add buddy xxx" hmp commands
p6: add "since: 2.7"
p7: fix bug in replication_close(), add missing "qapi/error.h", add test-replication
p8: add "since: 2.7"
V16:
1. Rebase to the newest codes
2. Address comments from Stefan & hailiang
p3: we don't need this patch now
p4: add "top-id" parameters for secondary
p6: fix NULL pointer in replication callbacks, remove unnecessary typedefs, add doc comments that explain the semantics of Replication
p7: Refactor AioContext for thread-safe, remove unnecessary get_top_bs() *Note*: I'm working on replication testcase now, will send out in V17 V15:
1. Rebase to the newest codes
2. Fix typos and coding style addresed Eric's comments
3. Address Stefan's comments
1) Make backup_do_checkpoint public, drop the changes on BlockJobDriver
2) Update the message and description for [PATCH 4/9]
3) Make replication_(start/stop/do_checkpoint)_all as global interfaces
4) Introduce AioContext lock to protect start/stop/do_checkpoint callbacks
5) Use BdrvChild instead of holding on to BlockDriverState * pointers
4. Clear BDRV_O_INACTIVE for hidden disk's open_flags since commit 09e0c771
5. Introduce replication_get_error_all to check replication status
6. Remove useless discard interface
V14:
1. Implement auto complete active commit
2. Implement active commit block job for replication.c
3. Address the comments from Stefan, add replication-specific API and data structure, also remove old block layer APIs
V13:
1. Rebase to the newest codes
2. Remove redundant marcos and semicolon in replication.c
3. Fix typos in block-replication.txt
V12:
1. Rebase to the newest codes
2. Use backing reference to replcace 'allow-write-backing-file'
V11:
1. Reopen the backing file when starting blcok replication if it is not opened in R/W mode
2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET when opening backing file
3. Block the top BDS so there is only one block job for the top BDS and its backing chain.
V10:
1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing reference.
2. Address the comments from Eric Blake
V9:
1. Update the error messages
2. Rebase to the newest qemu
3. Split child add/delete support. These patches are sent in another patchset. V8:
1. Address Alberto Garcia's comments
V7:
1. Implement adding/removing quorum child. Remove the option non-connect.
2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion V6:
1. Rebase to the newest qemu.
V5:
1. Address the comments from Gong Lei
2. Speed the failover up. The secondary vm can take over very quickly even if there are too many I/O requests.
V4:
1. Introduce a new driver replication to avoid touch nbd and qcow2. V3:
1: use error_setg() instead of error_set()
2. Add a new block job API
3. Active disk, hidden disk and nbd target uses the same AioContext
4. Add a testcase to test new hbitmap API
V2:
1. Redesign the secondary qemu(use image-fleecing)
2. Use Error objects to return error message
3. Address the comments from Max Reitz and Eric Blake
Changlong Xie (1):
Introduce new APIs to do replication operation
Wen Congyang (7):
unblock backup operations in backing file
Backup: clear all bitmap when doing block checkpoint Link backup into block core
docs: block replication's description
auto complete active commit
Implement new driver for block replication support replication driver in blockdev-add
  Makefile.objs              |   1 +
  block.c                    |  17 ++
  block/Makefile.objs        |   3 +-
  block/backup.c             |  17 ++
  block/mirror.c             |  13 +-
  block/replication.c        | 625 +++++++++++++++++++++++++++++++++++++++++++++
  blockdev.c                 |   2 +-
  docs/block-replication.txt | 239 +++++++++++++++++
  include/block/block_int.h  |   5 +-
  qapi/block-core.json       |  33 ++-
  qemu-img.c                 |   2 +-
  replication.c              | 105 ++++++++
  replication.h              | 176 +++++++++++++
  tests/.gitignore           |   1 +
  tests/Makefile             |   4 +
  tests/test-replication.c   | 505 ++++++++++++++++++++++++++++++++++++
  16 files changed, 1738 insertions(+), 10 deletions(-)
create mode 100644 block/replication.c
create mode 100644 docs/block-replication.txt
create mode 100644 replication.c
create mode 100644 replication.h
create mode 100644 tests/test-replication.c

Hailiang Zhang

2016-05-05 07:21 UTC #15 lore raw

ping^2 ...

Sorry for the noise, but this series is the prerequisite of COLO and it got no feedback for almost three weeks ...

Quoted 139 lines · Changlong Xie

Block replication is a very important feature which is used for continuous checkpoints(for example: COLO).
You can get the detailed information about block replication from here:
http://wiki.qemu.org/Features/BlockReplication
Usage:
Please refer to docs/block-replication.txt
This patch series is based on the following patch series:
http://lists.nongnu.org/archive/html/qemu-devel/2016-04/msg02093.html
You can get the patch here:
https://github.com/Pating/qemu/tree/changlox/block-replication-v18
You can get the patch with framework here:
https://github.com/Pating/qemu/tree/changlox/colo_framework_v17
TODO:
1. Continuous block replication. It will be started after basic functions are accepted.
Changs Log:
V18:
p6: add local_err in all replication callbacks to prevent "errp == NULL"
p7: add missing qemu_iovec_destroy(xxx)
V17:
1. Rebase to the lastest codes
p2: refactor backup_do_checkpoint addressed comments from Jeff Cody
p4: fix bugs in "drive_add buddy xxx" hmp commands
p6: add "since: 2.7"
p7: fix bug in replication_close(), add missing "qapi/error.h", add test-replication
p8: add "since: 2.7"
V16:
1. Rebase to the newest codes
2. Address comments from Stefan & hailiang
p3: we don't need this patch now
p4: add "top-id" parameters for secondary
p6: fix NULL pointer in replication callbacks, remove unnecessary typedefs, add doc comments that explain the semantics of Replication
p7: Refactor AioContext for thread-safe, remove unnecessary get_top_bs() *Note*: I'm working on replication testcase now, will send out in V17 V15:
1. Rebase to the newest codes
2. Fix typos and coding style addresed Eric's comments
3. Address Stefan's comments
1) Make backup_do_checkpoint public, drop the changes on BlockJobDriver
2) Update the message and description for [PATCH 4/9]
3) Make replication_(start/stop/do_checkpoint)_all as global interfaces
4) Introduce AioContext lock to protect start/stop/do_checkpoint callbacks
5) Use BdrvChild instead of holding on to BlockDriverState * pointers
4. Clear BDRV_O_INACTIVE for hidden disk's open_flags since commit 09e0c771
5. Introduce replication_get_error_all to check replication status
6. Remove useless discard interface
V14:
1. Implement auto complete active commit
2. Implement active commit block job for replication.c
3. Address the comments from Stefan, add replication-specific API and data structure, also remove old block layer APIs
V13:
1. Rebase to the newest codes
2. Remove redundant marcos and semicolon in replication.c
3. Fix typos in block-replication.txt
V12:
1. Rebase to the newest codes
2. Use backing reference to replcace 'allow-write-backing-file'
V11:
1. Reopen the backing file when starting blcok replication if it is not opened in R/W mode
2. Unblock BLOCK_OP_TYPE_BACKUP_SOURCE and BLOCK_OP_TYPE_BACKUP_TARGET when opening backing file
3. Block the top BDS so there is only one block job for the top BDS and its backing chain.
V10:
1. Use blockdev-remove-medium and blockdev-insert-medium to replace backing reference.
2. Address the comments from Eric Blake
V9:
1. Update the error messages
2. Rebase to the newest qemu
3. Split child add/delete support. These patches are sent in another patchset. V8:
1. Address Alberto Garcia's comments
V7:
1. Implement adding/removing quorum child. Remove the option non-connect.
2. Simplify the backing refrence option according to Stefan Hajnoczi's suggestion V6:
1. Rebase to the newest qemu.
V5:
1. Address the comments from Gong Lei
2. Speed the failover up. The secondary vm can take over very quickly even if there are too many I/O requests.
V4:
1. Introduce a new driver replication to avoid touch nbd and qcow2. V3:
1: use error_setg() instead of error_set()
2. Add a new block job API
3. Active disk, hidden disk and nbd target uses the same AioContext
4. Add a testcase to test new hbitmap API
V2:
1. Redesign the secondary qemu(use image-fleecing)
2. Use Error objects to return error message
3. Address the comments from Max Reitz and Eric Blake
Changlong Xie (1):
Introduce new APIs to do replication operation
Wen Congyang (7):
unblock backup operations in backing file
Backup: clear all bitmap when doing block checkpoint Link backup into block core
docs: block replication's description
auto complete active commit
Implement new driver for block replication support replication driver in blockdev-add
  Makefile.objs              |   1 +
  block.c                    |  17 ++
  block/Makefile.objs        |   3 +-
  block/backup.c             |  17 ++
  block/mirror.c             |  13 +-
  block/replication.c        | 625 +++++++++++++++++++++++++++++++++++++++++++++
  blockdev.c                 |   2 +-
  docs/block-replication.txt | 239 +++++++++++++++++
  include/block/block_int.h  |   5 +-
  qapi/block-core.json       |  33 ++-
  qemu-img.c                 |   2 +-
  replication.c              | 105 ++++++++
  replication.h              | 176 +++++++++++++
  tests/.gitignore           |   1 +
  tests/Makefile             |   4 +
  tests/test-replication.c   | 505 ++++++++++++++++++++++++++++++++++++
  16 files changed, 1738 insertions(+), 10 deletions(-)
create mode 100644 block/replication.c
create mode 100644 docs/block-replication.txt
create mode 100644 replication.c
create mode 100644 replication.h
create mode 100644 tests/test-replication.c

Stefan Hajnoczi

2016-05-06 15:51 UTC #16 lore raw