Skip to contentSkip to conversation

backup-top filter driver for backup

PATCH Superseded Vladimir Sementsov-Ogievskiy 4 participants 32 messages May 29, 2019 – Jun 18, 2019 Block layer

v8 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v8 cover backup-top filter driver for backup

Hi all!

These series introduce backup-top driver. It's a filter-node, which do copy-before-write operation. Mirror uses filter-node for handling guest writes, let's move to filter-node (from write-notifiers) for backup too

v8:
01-03: new
05: add Max's r-b
others changed, change description in each patch mail in Notes section.

v6-v7 was preparing refactoring, which now is in Max's pull request, and these series based on it:
Based-on: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06526.html

Vladimir Sementsov-Ogievskiy (7):
block: teach bdrv_debug_breakpoint skip filters with backing
block: swap operation order in bdrv_append
block: allow not one child for implicit node
block: introduce backup-top filter driver block/io: refactor wait_serialising_requests
block: add lock/unlock range functions block/backup: use backup-top instead of write notifiers

 block/backup-top.h         |  64 ++++++++
 include/block/block_int.h  |   4 +
 block.c                    |  60 +++++--
 block/backup-top.c         | 322 +++++++++++++++++++++++++++++++++++++
 block/backup.c             | 171 ++++++++------------
 block/io.c                 |  68 ++++++--
 block/Makefile.objs        |   2 +
 tests/qemu-iotests/056     |   2 +-
 tests/qemu-iotests/085.out |   2 +-
 tests/qemu-iotests/129     |   1 -
 tests/qemu-iotests/141.out |   2 +-
 11 files changed, 567 insertions(+), 131 deletions(-)
 create mode 100644 block/backup-top.h
 create mode 100644 block/backup-top.c
Signature · 1 line
2.18.0
#1 lore raw

PATCH v8 1/7 block: teach bdrv_debug_breakpoint skip filters with backing

Teach bdrv_debug_breakpoint and bdrv_debug_remove_breakpoint skip filters with backing. This is needed to implement and use in backup job it's own backup_top filter driver (like mirror already has one), and without this improvement, breakpoint removal will fail at least in 55 iotest.

---
 block.c | 34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)
block.c +26 -8
index 1a73e310c1..e6e9770704 100644
@@ -4982,14 +4982,35 @@ void bdrv_debug_event(BlockDriverState *bs, BlkdebugEvent event)
     bs->drv->bdrv_debug_event(bs, event);
 }
 
-int bdrv_debug_breakpoint(BlockDriverState *bs, const char *event,
-                          const char *tag)
+static BlockDriverState *bdrv_find_debug_node(BlockDriverState *bs)
 {
     while (bs && bs->drv && !bs->drv->bdrv_debug_breakpoint) {
-        bs = bs->file ? bs->file->bs : NULL;
+        if (bs->file) {
+            bs = bs->file->bs;
+            continue;
+        }
+
+        if (bs->drv->is_filter && bs->backing) {
+            bs = bs->backing->bs;
+            continue;
+        }
+
+        break;
     }
 
     if (bs && bs->drv && bs->drv->bdrv_debug_breakpoint) {
+        assert(bs->drv->bdrv_debug_remove_breakpoint);
+        return bs;
+    }
+
+    return NULL;
+}
+
+int bdrv_debug_breakpoint(BlockDriverState *bs, const char *event,
+                          const char *tag)
+{
+    bs = bdrv_find_debug_node(bs);
+    if (bs) {
         return bs->drv->bdrv_debug_breakpoint(bs, event, tag);
     }
 
@@ -4998,11 +5019,8 @@ int bdrv_debug_breakpoint(BlockDriverState *bs, const char *event,
 
 int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag)
 {
-    while (bs && bs->drv && !bs->drv->bdrv_debug_remove_breakpoint) {
-        bs = bs->file ? bs->file->bs : NULL;
-    }
-
-    if (bs && bs->drv && bs->drv->bdrv_debug_remove_breakpoint) {
+    bs = bdrv_find_debug_node(bs);
+    if (bs) {
         return bs->drv->bdrv_debug_remove_breakpoint(bs, tag);
     }
 
Signature · 1 line
2.18.0
#2 lore raw

Re: [Qemu-devel] [PATCH v8 1/7] block: teach bdrv_debug_breakpoint skip filters with backing

Quoted 10 lines · Vladimir Sementsov-Ogievskiy

Teach bdrv_debug_breakpoint and bdrv_debug_remove_breakpoint skip filters with backing. This is needed to implement and use in backup job it's own backup_top filter driver (like mirror already has one), and without this improvement, breakpoint removal will fail at least in 55 iotest.

---
 block.c | 34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)

Well, it would work in the meantime, but the real fix of course is to use bdrv_primary_bs().

Max

#3 lore raw

PATCH v8 2/7 block: swap operation order in bdrv_append

In reply to Vladimir Sementsov-Ogievskiy

bs_top parents may conflict with bs_new backing child permissions, so let's do bdrv_replace_node first, it covers more possible cases.

It is needed for further implementation of backup-top filter, which don't want to share write permission on its backing child.

Side effect is that we may set backing hd when device name is already available, so 085 iotest output is changed.

---
 block.c                    | 11 ++++++++---
 tests/qemu-iotests/085.out |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
block.c +8 -3
index e6e9770704..57216f4115 100644
@@ -4088,22 +4088,27 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
 {
     Error *local_err = NULL;
 
-    bdrv_set_backing_hd(bs_new, bs_top, &local_err);
+    bdrv_ref(bs_top);
+
+    bdrv_replace_node(bs_top, bs_new, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
+        error_prepend(errp, "Failed to replace node: ");
         goto out;
     }
 
-    bdrv_replace_node(bs_top, bs_new, &local_err);
+    bdrv_set_backing_hd(bs_new, bs_top, &local_err);
     if (local_err) {
+        bdrv_replace_node(bs_new, bs_top, &error_abort);
         error_propagate(errp, local_err);
-        bdrv_set_backing_hd(bs_new, NULL, &error_abort);
+        error_prepend(errp, "Failed to set backing: ");
         goto out;
     }
 
     /* bs_new is now referenced by its new parents, we don't need the
      * additional reference any more. */
 out:
+    bdrv_unref(bs_top);
     bdrv_unref(bs_new);
 }
 
tests/qemu-iotests/085.out +1 -1
index 6edf107f55..e5a2645bf5 100644
@@ -74,7 +74,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/
 
 === Invalid command - snapshot node used as backing hd ===
 
-{"error": {"class": "GenericError", "desc": "Node 'snap_11' is busy: node is used as backing hd of 'snap_12'"}}
+{"error": {"class": "GenericError", "desc": "Node 'snap_11' is busy: node is used as backing hd of 'virtio0'"}}
 
 === Invalid command - snapshot node has a backing image ===
 
Signature · 1 line
2.18.0
#4 lore raw

Re: [Qemu-devel] [PATCH v8 2/7] block: swap operation order in bdrv_append

Quoted 27 lines · Vladimir Sementsov-Ogievskiy

bs_top parents may conflict with bs_new backing child permissions, so let's do bdrv_replace_node first, it covers more possible cases.

It is needed for further implementation of backup-top filter, which don't want to share write permission on its backing child.

Side effect is that we may set backing hd when device name is already available, so 085 iotest output is changed.

---
 block.c                    | 11 ++++++++---
 tests/qemu-iotests/085.out |  2 +-
 2 files changed, 9 insertions(+), 4 deletions(-)
block.c +3 -1
index e6e9770704..57216f4115 100644
@@ -4088,22 +4088,27 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
 {
     Error *local_err = NULL;
 
-    bdrv_set_backing_hd(bs_new, bs_top, &local_err);
+    bdrv_ref(bs_top);
+
+    bdrv_replace_node(bs_top, bs_new, &local_err);

block.c void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, · line 4102

Quoted 10 lines
block.c +3 -1
     if (local_err) {
         error_propagate(errp, local_err);
+        error_prepend(errp, "Failed to replace node: ");
         goto out;
     }
 
-    bdrv_replace_node(bs_top, bs_new, &local_err);
+    bdrv_set_backing_hd(bs_new, bs_top, &local_err);
     if (local_err) {
+        bdrv_replace_node(bs_new, bs_top, &error_abort);

Hm. I see the need for switching this stuff around, but this &error_abort is much more difficult to verify than the previous one for bdrv_set_backing_hd(..., NULL, ...). I think it at least warrants a comment why the order has to be this way (using git blame has the disadvantage of fading over time as other people modify a piece of code), and why this &error_abort is safe.

Max

Quoted 27 lines
block.c +2 -1
         error_propagate(errp, local_err);
-        bdrv_set_backing_hd(bs_new, NULL, &error_abort);
+        error_prepend(errp, "Failed to set backing: ");
         goto out;
     }
 
     /* bs_new is now referenced by its new parents, we don't need the
      * additional reference any more. */
 out:
+    bdrv_unref(bs_top);
     bdrv_unref(bs_new);
 }
 
tests/qemu-iotests/085.out +1 -1
index 6edf107f55..e5a2645bf5 100644
@@ -74,7 +74,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/
 
 === Invalid command - snapshot node used as backing hd ===
 
-{"error": {"class": "GenericError", "desc": "Node 'snap_11' is busy: node is used as backing hd of 'snap_12'"}}
+{"error": {"class": "GenericError", "desc": "Node 'snap_11' is busy: node is used as backing hd of 'virtio0'"}}
 
 === Invalid command - snapshot node has a backing image ===
 
#5 lore raw

Re: [Qemu-devel] [PATCH v8 2/7] block: swap operation order in bdrv_append

Quoted 34 lines · Max Reitz
Quoted 33 lines · Vladimir Sementsov-Ogievskiy

bs_top parents may conflict with bs_new backing child permissions, so let's do bdrv_replace_node first, it covers more possible cases.

It is needed for further implementation of backup-top filter, which don't want to share write permission on its backing child.

Side effect is that we may set backing hd when device name is already available, so 085 iotest output is changed.

---
  block.c                    | 11 ++++++++---
  tests/qemu-iotests/085.out |  2 +-
  2 files changed, 9 insertions(+), 4 deletions(-)
block.c +4 -1
index e6e9770704..57216f4115 100644
@@ -4088,22 +4088,27 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
  {
      Error *local_err = NULL;
 
-    bdrv_set_backing_hd(bs_new, bs_top, &local_err);
+    bdrv_ref(bs_top);
+
+    bdrv_replace_node(bs_top, bs_new, &local_err);
      if (local_err) {
          error_propagate(errp, local_err);
+        error_prepend(errp, "Failed to replace node: ");
          goto out;
      }
 
Quoted 10 lines

block.c void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, · line 4102

Quoted 4 lines
block.c +2 -1
-    bdrv_replace_node(bs_top, bs_new, &local_err);
+    bdrv_set_backing_hd(bs_new, bs_top, &local_err);
      if (local_err) {
+        bdrv_replace_node(bs_new, bs_top, &error_abort);

Hm. I see the need for switching this stuff around, but this &error_abort is much more difficult to verify than the previous one for bdrv_set_backing_hd(..., NULL, ...). I think it at least warrants a comment why the order has to be this way (using git blame has the disadvantage of fading over time as other people modify a piece of

so, I always use git log -p -- <filepath> instead, and search through it)

Quoted 1 line

code), and why this &error_abort is safe.

ok, will add

Quoted 32 lines

Max

Quoted 27 lines
block.c +2 -1
          error_propagate(errp, local_err);
-        bdrv_set_backing_hd(bs_new, NULL, &error_abort);
+        error_prepend(errp, "Failed to set backing: ");
          goto out;
      }
 
      /* bs_new is now referenced by its new parents, we don't need the
       * additional reference any more. */
  out:
+    bdrv_unref(bs_top);
      bdrv_unref(bs_new);
  }
 
tests/qemu-iotests/085.out +1 -1
index 6edf107f55..e5a2645bf5 100644
@@ -74,7 +74,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/
 
  === Invalid command - snapshot node used as backing hd ===
 
-{"error": {"class": "GenericError", "desc": "Node 'snap_11' is busy: node is used as backing hd of 'snap_12'"}}
+{"error": {"class": "GenericError", "desc": "Node 'snap_11' is busy: node is used as backing hd of 'virtio0'"}}
 
  === Invalid command - snapshot node has a backing image ===
 
Signature · 2 lines
Best regards,
Vladimir
#6 lore raw

PATCH v8 3/7 block: allow not one child for implicit node

In reply to Vladimir Sementsov-Ogievskiy

Upcoming backup-top filter wants to operate like usual implicit filter node with fall-through to backing child. But also needs additional target child, let's support that.

On the other hand, after backup completion (before job dismiss) filter is still attached to job blk, but don't have any children. Support this too.

---
 block.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
block.c +13 -2
index 57216f4115..3f4de3ae32 100644
@@ -6200,9 +6200,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
     }
 
     if (bs->implicit) {
-        /* For implicit nodes, just copy everything from the single child */
+        /*
+         * For implicit nodes, just copy everything from the single child or
+         * from backing, if there are several children.
+         * If there are no children for some reason (filter is still attached
+         * to block-job blk, but already removed from backing chain of device)
+         * do nothing.
+         */
         child = QLIST_FIRST(&bs->children);
-        assert(QLIST_NEXT(child, next) == NULL);
+        if (!child) {
+            return;
+        } else if (QLIST_NEXT(child, next)) {
+            assert(bs->backing);
+            child = bs->backing;
+        }
 
         pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
                 child->bs->exact_filename);
Signature · 1 line
2.18.0
Reviewed-by on v8 3/7
#7 lore raw

Re: [Qemu-devel] [PATCH v8 3/7] block: allow not one child for implicit node

Quoted 30 lines · Vladimir Sementsov-Ogievskiy

Upcoming backup-top filter wants to operate like usual implicit filter node with fall-through to backing child. But also needs additional target child, let's support that.

On the other hand, after backup completion (before job dismiss) filter is still attached to job blk, but don't have any children. Support this too.

---
 block.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
block.c +7 -1
index 57216f4115..3f4de3ae32 100644
@@ -6200,9 +6200,20 @@ void bdrv_refresh_filename(BlockDriverState *bs)
     }
 
     if (bs->implicit) {
-        /* For implicit nodes, just copy everything from the single child */
+        /*
+         * For implicit nodes, just copy everything from the single child or
+         * from backing, if there are several children.
+         * If there are no children for some reason (filter is still attached
+         * to block-job blk, but already removed from backing chain of device)
+         * do nothing.
+         */
         child = QLIST_FIRST(&bs->children);

block.c void bdrv_refresh_filename(BlockDriverState *bs) · line 6219

Quoted 10 lines
block.c +6 -1
-        assert(QLIST_NEXT(child, next) == NULL);
+        if (!child) {
+            return;
+        } else if (QLIST_NEXT(child, next)) {
+            assert(bs->backing);
+            child = bs->backing;
+        }
 
         pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
                 child->bs->exact_filename);

(To be changed to bdrv_filtered_rw_bs(), of course)

#8 lore raw

PATCH v8 4/7 block: introduce backup-top filter driver

In reply to Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

    +-------+
    | Guest |
    +-------+
        |r,w
        v
    +------------+  target   +---------------+
    | backup_top |---------->| target(qcow2) |
    +------------+   CBW     +---------------+
        |
backing |r,w
        v
    +-------------+
    | Active disk |
    +-------------+

The driver will be used in backup instead of write-notifiers.

---
 block/backup-top.h  |  64 +++++++++
 block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
 block/Makefile.objs |   2 +
 3 files changed, 388 insertions(+)
 create mode 100644 block/backup-top.h
 create mode 100644 block/backup-top.c
block/backup-top.h new file +64 -0
new file mode 100644
index 0000000000..788e18c358
@@ -0,0 +1,64 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef BACKUP_TOP_H
+#define BACKUP_TOP_H
+
+#include "qemu/osdep.h"
+
+#include "block/block_int.h"
+
+typedef void (*BackupTopProgressCallback)(uint64_t done, void *opaque);
+typedef struct BDRVBackupTopState {
+    HBitmap *copy_bitmap; /* what should be copied to @target on guest write. */
+    BdrvChild *target;
+
+    BackupTopProgressCallback progress_cb;
+    void *progress_opaque;
+} BDRVBackupTopState;
+
+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.
+ *
+ * @copy_bitmap selects regions which needs CBW. Furthermore, backup_top will
+ * use exactly this bitmap, so it may be used to control backup_top behavior
+ * dynamically. Caller should not release @copy_bitmap during life-time of
+ * backup_top. Progress is tracked by calling @progress_cb function.
+ */
+BlockDriverState *bdrv_backup_top_append(
+        BlockDriverState *source, BlockDriverState *target,
+        HBitmap *copy_bitmap, Error **errp);
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque);
+void bdrv_backup_top_drop(BlockDriverState *bs);
+
+#endif /* BACKUP_TOP_H */
block/backup-top.c new file +322 -0

322 lines not on this page · raw message

block/Makefile.objs +2 -0
index ae11605c9f..dfbdfe6ab4 100644
@@ -40,6 +40,8 @@ block-obj-y += throttle.o copy-on-read.o
 
 block-obj-y += crypto.o
 
+block-obj-y += backup-top.o
+
 common-obj-y += stream.o
 
 nfs.o-libs         := $(LIBNFS_LIBS)
Signature · 1 line
2.18.0
#9 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 75 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

    +-------+
    | Guest |
    +-------+
        |r,w
        v
    +------------+  target   +---------------+
    | backup_top |---------->| target(qcow2) |
    +------------+   CBW     +---------------+
        |
backing |r,w
        v
    +-------------+
    | Active disk |
    +-------------+

The driver will be used in backup instead of write-notifiers.

---
 block/backup-top.h  |  64 +++++++++
 block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
 block/Makefile.objs |   2 +
 3 files changed, 388 insertions(+)
 create mode 100644 block/backup-top.h
 create mode 100644 block/backup-top.c
block/backup-top.h new file +39 -0
new file mode 100644
index 0000000000..788e18c358
@@ -0,0 +1,64 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef BACKUP_TOP_H
+#define BACKUP_TOP_H
+
+#include "qemu/osdep.h"
+
+#include "block/block_int.h"
+
+typedef void (*BackupTopProgressCallback)(uint64_t done, void *opaque);
+typedef struct BDRVBackupTopState {
+    HBitmap *copy_bitmap; /* what should be copied to @target on guest write. */
+    BdrvChild *target;
+
+    BackupTopProgressCallback progress_cb;
+    void *progress_opaque;

block/backup-top.h line 49

Quoted 10 lines
block/backup-top.h new file +10 -0
+} BDRVBackupTopState;
+
+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Quoted 72 lines
block/backup-top.h new file +15 -0
+ *
+ * @copy_bitmap selects regions which needs CBW. Furthermore, backup_top will
+ * use exactly this bitmap, so it may be used to control backup_top behavior
+ * dynamically. Caller should not release @copy_bitmap during life-time of
+ * backup_top. Progress is tracked by calling @progress_cb function.
+ */
+BlockDriverState *bdrv_backup_top_append(
+        BlockDriverState *source, BlockDriverState *target,
+        HBitmap *copy_bitmap, Error **errp);
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque);
+void bdrv_backup_top_drop(BlockDriverState *bs);
+
+#endif /* BACKUP_TOP_H */
block/backup-top.c new file +51 -0
new file mode 100644
index 0000000000..1daa02f539
@@ -0,0 +1,322 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+#include "block/block_int.h"
+#include "block/qdict.h"
+
+#include "block/backup-top.h"
+
+static coroutine_fn int backup_top_co_preadv(
+        BlockDriverState *bs, uint64_t offset, uint64_t bytes,
+        QEMUIOVector *qiov, int flags)
+{
+    /*
+     * Features to be implemented:
+     * F1. COR. save read data to fleecing target for fast access
+     *     (to reduce reads). This possibly may be done with use of copy-on-read
+     *     filter, but we need an ability to make COR requests optional: for
+     *     example, if target is a ram-cache, and if it is full now, we should
+     *     skip doing COR request, as it is actually not necessary.
+     *
+     * F2. Feature for guest: read from fleecing target if data is in ram-cache
+     *     and is unchanged
+     */
+
+    return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);

block/backup-top.c line 61

Quoted 10 lines
block/backup-top.c new file +10 -0
+}
+
+static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
+                                       uint64_t bytes)
+{
+    int ret = 0;
+    BDRVBackupTopState *s = bs->opaque;
+    uint64_t gran = 1UL << hbitmap_granularity(s->copy_bitmap);
+    uint64_t end = QEMU_ALIGN_UP(offset + bytes, gran);
+    uint64_t off = QEMU_ALIGN_DOWN(offset, gran), len;

The “, len” is a bit weirdly placed there, why not define it on a separate line as "uint64_t len = end - off"?

block/backup-top.c line 73

Quoted 12 lines
block/backup-top.c new file +12 -0
+    void *buf = qemu_blockalign(bs, end - off);
+
+    /*
+     * Features to be implemented:
+     * F3. parallelize copying loop
+     * F4. detect zeroes ? or, otherwise, drop detect zeroes from backup code
+     *     and just enable zeroes detecting on target
+     * F5. use block_status ?
+     * F6. don't copy clusters which are already cached by COR [see F1]
+     * F7. if target is ram-cache and it is full, there should be a possibility
+     *     to drop not necessary data (cached by COR [see F1]) to handle CBW
+     *     fast.

Also “drop BDRV_REQ_SERIALISING from writes to s->target unless necessary”?

Quoted 81 lines
block/backup-top.c new file +81 -0
+     */
+
+    len = end - off;
+    while (hbitmap_next_dirty_area(s->copy_bitmap, &off, &len)) {
+        hbitmap_reset(s->copy_bitmap, off, len);
+
+        ret = bdrv_co_pread(bs->backing, off, len, buf,
+                            BDRV_REQ_NO_SERIALISING);
+        if (ret < 0) {
+            hbitmap_set(s->copy_bitmap, off, len);
+            goto out;
+        }
+
+        ret = bdrv_co_pwrite(s->target, off, len, buf, BDRV_REQ_SERIALISING);
+        if (ret < 0) {
+            hbitmap_set(s->copy_bitmap, off, len);
+            goto out;
+        }
+
+        if (s->progress_cb) {
+            s->progress_cb(len, s->progress_opaque);
+        }
+        off += len;
+        if (off >= end) {
+            break;
+        }
+        len = end - off;
+    }
+
+out:
+    qemu_vfree(buf);
+
+    /*
+     * F8. we fail guest request in case of error. We can alter it by
+     * possibility to fail copying process instead, or retry several times, or
+     * may be guest pause, etc.
+     */
+    return ret;
+}
+
+static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
+                                               int64_t offset, int bytes)
+{
+    int ret = backup_top_cbw(bs, offset, bytes);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /*
+     * Features to be implemented:
+     * F9. possibility of lazy discard: just defer the discard after fleecing
+     *     completion. If write (or new discard) occurs to the same area, just
+     *     drop deferred discard.
+     */
+
+    return bdrv_co_pdiscard(bs->backing, offset, bytes);
+}
+
+static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs,
+        int64_t offset, int bytes, BdrvRequestFlags flags)
+{
+    int ret = backup_top_cbw(bs, offset, bytes);
+    if (ret < 0) {
+        return ret;
+    }
+
+    return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags);
+}
+
+static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs,
+                                              uint64_t offset,
+                                              uint64_t bytes,
+                                              QEMUIOVector *qiov, int flags)
+{
+    if (!(flags & BDRV_REQ_WRITE_UNCHANGED)) {
+        int ret = backup_top_cbw(bs, offset, bytes);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+

block/backup-top.c line 164

Quoted 10 lines
block/backup-top.c new file +10 -0
+    return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags);
+}
+
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

block/backup-top.c line 176

Quoted 12 lines
block/backup-top.c new file +12 -0
+}
+
+static void backup_top_refresh_filename(BlockDriverState *bs)
+{
+    if (bs->backing == NULL) {
+        /*
+         * we can be here after failed bdrv_attach_child in
+         * bdrv_set_backing_hd
+         */
+        return;
+    }
+    bdrv_refresh_filename(bs->backing->bs);

bdrv_refresh_filename() should have done this already.

Quoted 12 lines
block/backup-top.c new file +12 -0
+    pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
+            bs->backing->bs->filename);
+}
+
+static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c,
+                                  const BdrvChildRole *role,
+                                  BlockReopenQueue *reopen_queue,
+                                  uint64_t perm, uint64_t shared,
+                                  uint64_t *nperm, uint64_t *nshared)
+{
+    /*
+     * We have HBitmap in the state, its size is fixed, so we never allow

block/backup-top.c line 198

Quoted 10 lines
block/backup-top.c new file +10 -0
+     * resize.
+     */
+    uint64_t rw = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED |
+                  BLK_PERM_WRITE;
+
+    bdrv_filter_default_perms(bs, c, role, reopen_queue, perm, shared,
+                              nperm, nshared);
+
+    *nperm = *nperm & rw;
+    *nshared = *nshared & rw;

Somehow, that bdrv_filter_default_perm() call doesn’t make this function easier for me.

It seems like this is basically just “*nperm = perm & rw” and “*nshared = shared & rw”.

block/backup-top.c line 208

Quoted 10 lines
block/backup-top.c new file +10 -0
+
+    if (role == &child_file) {
+        /*
+         * Target child
+         *
+         * Share write to target (child_file), to not interfere
+         * with guest writes to its disk which may be in target backing chain.
+         */
+        if (perm & BLK_PERM_WRITE) {
+            *nshared = *nshared | BLK_PERM_WRITE;

Why not always share WRITE on the target?

block/backup-top.c line 212

Quoted 4 lines
block/backup-top.c new file +4 -0
+        }
+    } else {
+        /* Source child */
+        if (perm & BLK_PERM_WRITE) {

Or WRITE_UNCHANGED, no?

block/backup-top.c line 216

Quoted 4 lines
block/backup-top.c new file +4 -0
+            *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+        }
+        *nshared =
+            *nshared & (BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED);

And here it isn’t “*nshared = shared & rw” but “rw & ~WRITE”.

I feel like this function would be simpler if you just set *nperm and *nshared separately in the two branches of this condition, without setting a “default” first.

Quoted 27 lines
block/backup-top.c new file +27 -0
+    }
+}
+
+BlockDriver bdrv_backup_top_filter = {
+    .format_name = "backup-top",
+    .instance_size = sizeof(BDRVBackupTopState),
+
+    .bdrv_co_preadv             = backup_top_co_preadv,
+    .bdrv_co_pwritev            = backup_top_co_pwritev,
+    .bdrv_co_pwrite_zeroes      = backup_top_co_pwrite_zeroes,
+    .bdrv_co_pdiscard           = backup_top_co_pdiscard,
+    .bdrv_co_flush              = backup_top_co_flush,
+
+    .bdrv_co_block_status       = bdrv_co_block_status_from_backing,
+
+    .bdrv_refresh_filename      = backup_top_refresh_filename,
+
+    .bdrv_child_perm            = backup_top_child_perm,
+
+    .is_filter = true,
+};
+
+BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
+                                         BlockDriverState *target,
+                                         HBitmap *copy_bitmap,
+                                         Error **errp)
+{

block/backup-top.c line 253

Quoted 10 lines
block/backup-top.c new file +10 -0
+    Error *local_err = NULL;
+    BDRVBackupTopState *state;
+    BlockDriverState *top = bdrv_new_open_driver(&bdrv_backup_top_filter,
+                                                 NULL, BDRV_O_RDWR, errp);
+
+    if (!top) {
+        return NULL;
+    }
+
+    top->implicit = true;

As I said above, I think the caller should set this.

Quoted 6 lines
block/backup-top.c new file +6 -0
+    top->total_sectors = source->total_sectors;
+    top->bl.opt_mem_alignment = MAX(bdrv_opt_mem_align(source),
+                                    bdrv_opt_mem_align(target));
+    top->opaque = state = g_new0(BDRVBackupTopState, 1);
+    state->copy_bitmap = copy_bitmap;
+

block/backup-top.c line 269

Quoted 10 lines
block/backup-top.c new file +10 -0
+    bdrv_ref(target);
+    state->target = bdrv_attach_child(top, target, "target", &child_file, errp);
+    if (!state->target) {
+        bdrv_unref(target);
+        bdrv_unref(top);
+        return NULL;
+    }
+
+    bdrv_set_aio_context(top, bdrv_get_aio_context(source));
+    bdrv_set_aio_context(target, bdrv_get_aio_context(source));

I suppose these calls aren’t necessary anymore (compare d0ee0204f4009).
(I’m not fully sure, though. In any case, they would need to be
bdrv_try_set_aio_context() if they were still necessary.)

Quoted 10 lines
block/backup-top.c new file +10 -0
+
+    bdrv_drained_begin(source);
+
+    bdrv_ref(top);
+    bdrv_append(top, source, &local_err);
+    if (local_err) {
+        error_prepend(&local_err, "Cannot append backup-top filter: ");
+    }
+
+    bdrv_drained_end(source);

block/backup-top.c line 289

Quoted 10 lines
block/backup-top.c new file +10 -0
+
+    if (local_err) {
+        bdrv_unref_child(top, state->target);
+        bdrv_unref(top);
+        error_propagate(errp, local_err);
+        return NULL;
+    }
+
+    return top;
+}

I guess it would be nice if it users could blockdev-add a backup-top node to basically get a backup with sync=none.

(The bdrv_open implementation should then create a new bitmap and initialize it to be fully set.)

But maybe it wouldn’t be so nice and I just have feverish dreams.

Quoted 10 lines
block/backup-top.c new file +10 -0
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque)
+{
+    BDRVBackupTopState *s = bs->opaque;
+
+    s->progress_cb = progress_cb;
+    s->progress_opaque = progress_opaque;
+}
+

block/backup-top.c line 310

Quoted 10 lines
block/backup-top.c new file +10 -0
+void bdrv_backup_top_drop(BlockDriverState *bs)
+{
+    BDRVBackupTopState *s = bs->opaque;
+    AioContext *aio_context = bdrv_get_aio_context(bs);
+
+    aio_context_acquire(aio_context);
+
+    bdrv_drained_begin(bs);
+
+    bdrv_child_try_set_perm(bs->backing, 0, BLK_PERM_ALL, &error_abort);

Pre-existing in other jobs, but I think calling this function is dangerous. (Which is why I remove it in my “block: Ignore loosening perm restrictions failures” series.)

block/backup-top.c line 312

Quoted 2 lines
block/backup-top.c new file +2 -0
+    bdrv_replace_node(bs, backing_bs(bs), &error_abort);
+    bdrv_set_backing_hd(bs, NULL, &error_abort);

I think some of this function should be in a .bdrv_close() implementation, for example this bdrv_set_backing_hd() call.

block/backup-top.c line 317

Quoted 4 lines
block/backup-top.c new file +4 -0
+    bdrv_drained_end(bs);
+
+    if (s->target) {
+        bdrv_unref_child(bs, s->target);

And this. Well, neither needs to be in a .bdrv_close() implementation, actually, because bdrv_close() will just do it by itself.

I suppose you’re trying to remove the children so the node is no longer usable after this point; but that isn’t quite right, then. The filter functions still refer to s->target and bs->backing, so you need to stop them from doing anything then.

At this point, you might as well add a variable to BDRVBackupTopState that says whether the filter is still supposed to be usable (like the “stop” variable I added to mirror in “block/mirror: Fix child permissions”). If so, the permission function should signal 0/ALL for the permissions on backing (and probably target), and all filter functions always immediately return an error.

So I don’t think backing and target need to be removed here. We can wait for that until bdrv_close(), but we should ensure that the filter really is unusable after this function has been called.

Max

Quoted 19 lines
block/backup-top.c new file +5 -0
+    }
+    bdrv_unref(bs);
+
+    aio_context_release(aio_context);
+}
block/Makefile.objs +2 -0
index ae11605c9f..dfbdfe6ab4 100644
@@ -40,6 +40,8 @@ block-obj-y += throttle.o copy-on-read.o
 
 block-obj-y += crypto.o
 
+block-obj-y += backup-top.o
+
 common-obj-y += stream.o
 
 nfs.o-libs         := $(LIBNFS_LIBS)
#10 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 81 lines · Max Reitz
Quoted 80 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

     +-------+
     | Guest |
     +-------+
         |r,w
         v
     +------------+  target   +---------------+
     | backup_top |---------->| target(qcow2) |
     +------------+   CBW     +---------------+
         |
backing |r,w
         v
     +-------------+
     | Active disk |
     +-------------+

The driver will be used in backup instead of write-notifiers.

---
  block/backup-top.h  |  64 +++++++++
  block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
  block/Makefile.objs |   2 +
  3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +44 -0
new file mode 100644
index 0000000000..788e18c358
@@ -0,0 +1,64 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef BACKUP_TOP_H
+#define BACKUP_TOP_H
+
+#include "qemu/osdep.h"
+
+#include "block/block_int.h"
+
+typedef void (*BackupTopProgressCallback)(uint64_t done, void *opaque);
+typedef struct BDRVBackupTopState {
+    HBitmap *copy_bitmap; /* what should be copied to @target on guest write. */
+    BdrvChild *target;
+
+    BackupTopProgressCallback progress_cb;
+    void *progress_opaque;
+} BDRVBackupTopState;
+
+/*
+ * bdrv_backup_top_append
+ *
Quoted 8 lines

block/backup-top.h line 49

Quoted 5 lines
block/backup-top.h new file +5 -0
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Quoted 100 lines
Quoted 73 lines
block/backup-top.h new file +15 -0
+ *
+ * @copy_bitmap selects regions which needs CBW. Furthermore, backup_top will
+ * use exactly this bitmap, so it may be used to control backup_top behavior
+ * dynamically. Caller should not release @copy_bitmap during life-time of
+ * backup_top. Progress is tracked by calling @progress_cb function.
+ */
+BlockDriverState *bdrv_backup_top_append(
+        BlockDriverState *source, BlockDriverState *target,
+        HBitmap *copy_bitmap, Error **errp);
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque);
+void bdrv_backup_top_drop(BlockDriverState *bs);
+
+#endif /* BACKUP_TOP_H */
block/backup-top.c new file +51 -0
new file mode 100644
index 0000000000..1daa02f539
@@ -0,0 +1,322 @@
+/*
+ * backup-top filter driver
+ *
+ * The driver performs Copy-Before-Write (CBW) operation: it is injected above
+ * some node, and before each write it copies _old_ data to the target node.
+ *
+ * Copyright (c) 2018 Virtuozzo International GmbH. All rights reserved.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+
+#include "qemu/cutils.h"
+#include "qapi/error.h"
+#include "block/block_int.h"
+#include "block/qdict.h"
+
+#include "block/backup-top.h"
+
+static coroutine_fn int backup_top_co_preadv(
+        BlockDriverState *bs, uint64_t offset, uint64_t bytes,
+        QEMUIOVector *qiov, int flags)
+{
+    /*
+     * Features to be implemented:
+     * F1. COR. save read data to fleecing target for fast access
+     *     (to reduce reads). This possibly may be done with use of copy-on-read
+     *     filter, but we need an ability to make COR requests optional: for
+     *     example, if target is a ram-cache, and if it is full now, we should
+     *     skip doing COR request, as it is actually not necessary.
+     *
+     * F2. Feature for guest: read from fleecing target if data is in ram-cache
+     *     and is unchanged
+     */
+
+    return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);

block/backup-top.c line 61

Quoted 10 lines
block/backup-top.c new file +10 -0
+}
+
+static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
+                                       uint64_t bytes)
+{
+    int ret = 0;
+    BDRVBackupTopState *s = bs->opaque;
+    uint64_t gran = 1UL << hbitmap_granularity(s->copy_bitmap);
+    uint64_t end = QEMU_ALIGN_UP(offset + bytes, gran);
+    uint64_t off = QEMU_ALIGN_DOWN(offset, gran), len;

The “, len” is a bit weirdly placed there, why not define it on a separate line as "uint64_t len = end - off"?

block/backup-top.c line 73

Quoted 12 lines
block/backup-top.c new file +12 -0
+    void *buf = qemu_blockalign(bs, end - off);
+
+    /*
+     * Features to be implemented:
+     * F3. parallelize copying loop
+     * F4. detect zeroes ? or, otherwise, drop detect zeroes from backup code
+     *     and just enable zeroes detecting on target
+     * F5. use block_status ?
+     * F6. don't copy clusters which are already cached by COR [see F1]
+     * F7. if target is ram-cache and it is full, there should be a possibility
+     *     to drop not necessary data (cached by COR [see F1]) to handle CBW
+     *     fast.
Quoted 1 line

Also “drop BDRV_REQ_SERIALISING from writes to s->target unless necessary”?

hmm, yes. It may be a filter parameter, serialize writes or not.

Quoted 86 lines
Quoted 86 lines
block/backup-top.c new file +85 -0
+     */
+
+    len = end - off;
+    while (hbitmap_next_dirty_area(s->copy_bitmap, &off, &len)) {
+        hbitmap_reset(s->copy_bitmap, off, len);
+
+        ret = bdrv_co_pread(bs->backing, off, len, buf,
+                            BDRV_REQ_NO_SERIALISING);
+        if (ret < 0) {
+            hbitmap_set(s->copy_bitmap, off, len);
+            goto out;
+        }
+
+        ret = bdrv_co_pwrite(s->target, off, len, buf, BDRV_REQ_SERIALISING);
+        if (ret < 0) {
+            hbitmap_set(s->copy_bitmap, off, len);
+            goto out;
+        }
+
+        if (s->progress_cb) {
+            s->progress_cb(len, s->progress_opaque);
+        }
+        off += len;
+        if (off >= end) {
+            break;
+        }
+        len = end - off;
+    }
+
+out:
+    qemu_vfree(buf);
+
+    /*
+     * F8. we fail guest request in case of error. We can alter it by
+     * possibility to fail copying process instead, or retry several times, or
+     * may be guest pause, etc.
+     */
+    return ret;
+}
+
+static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
+                                               int64_t offset, int bytes)
+{
+    int ret = backup_top_cbw(bs, offset, bytes);
+    if (ret < 0) {
+        return ret;
+    }
+
+    /*
+     * Features to be implemented:
+     * F9. possibility of lazy discard: just defer the discard after fleecing
+     *     completion. If write (or new discard) occurs to the same area, just
+     *     drop deferred discard.
+     */
+
+    return bdrv_co_pdiscard(bs->backing, offset, bytes);
+}
+
+static int coroutine_fn backup_top_co_pwrite_zeroes(BlockDriverState *bs,
+        int64_t offset, int bytes, BdrvRequestFlags flags)
+{
+    int ret = backup_top_cbw(bs, offset, bytes);
+    if (ret < 0) {
+        return ret;
+    }
+
+    return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags);
+}
+
+static coroutine_fn int backup_top_co_pwritev(BlockDriverState *bs,
+                                              uint64_t offset,
+                                              uint64_t bytes,
+                                              QEMUIOVector *qiov, int flags)
+{
+    if (!(flags & BDRV_REQ_WRITE_UNCHANGED)) {
+        int ret = backup_top_cbw(bs, offset, bytes);
+        if (ret < 0) {
+            return ret;
+        }
+    }
+
+    return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags);
+}
+
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
Quoted 8 lines

block/backup-top.c line 164

Quoted 6 lines
block/backup-top.c new file +6 -0
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

Hm, you've asked it already, on previous version :) Backup don't do it, so I just keep old behavior. And what is the reason to flush backup target on any guest flush?

Quoted 56 lines

block/backup-top.c line 176

Quoted 13 lines
block/backup-top.c new file +12 -0
+}
+
+static void backup_top_refresh_filename(BlockDriverState *bs)
+{
+    if (bs->backing == NULL) {
+        /*
+         * we can be here after failed bdrv_attach_child in
+         * bdrv_set_backing_hd
+         */
+        return;
+    }
+    bdrv_refresh_filename(bs->backing->bs);

bdrv_refresh_filename() should have done this already.

Quoted 12 lines
block/backup-top.c new file +12 -0
+    pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
+            bs->backing->bs->filename);
+}
+
+static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c,
+                                  const BdrvChildRole *role,
+                                  BlockReopenQueue *reopen_queue,
+                                  uint64_t perm, uint64_t shared,
+                                  uint64_t *nperm, uint64_t *nshared)
+{
+    /*
+     * We have HBitmap in the state, its size is fixed, so we never allow

block/backup-top.c line 198

Quoted 10 lines
block/backup-top.c new file +10 -0
+     * resize.
+     */
+    uint64_t rw = BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED |
+                  BLK_PERM_WRITE;
+
+    bdrv_filter_default_perms(bs, c, role, reopen_queue, perm, shared,
+                              nperm, nshared);
+
+    *nperm = *nperm & rw;
+    *nshared = *nshared & rw;

Somehow, that bdrv_filter_default_perm() call doesn’t make this function easier for me.

It seems like this is basically just “*nperm = perm & rw” and “*nshared = shared & rw”.

block/backup-top.c line 208

Quoted 10 lines
block/backup-top.c new file +10 -0
+
+    if (role == &child_file) {
+        /*
+         * Target child
+         *
+         * Share write to target (child_file), to not interfere
+         * with guest writes to its disk which may be in target backing chain.
+         */
+        if (perm & BLK_PERM_WRITE) {
+            *nshared = *nshared | BLK_PERM_WRITE;
Quoted 1 line

Why not always share WRITE on the target?

Hmm, it's a bad thing to share writes on target, so I'm trying to reduce number of cases when we have share it.

Quoted 7 lines

block/backup-top.c line 212

Quoted 5 lines
block/backup-top.c new file +4 -0
+        }
+    } else {
+        /* Source child */
+        if (perm & BLK_PERM_WRITE) {

Or WRITE_UNCHANGED, no?

Why? We don't need doing CBW for unchanged write.

Quoted 11 lines

block/backup-top.c line 216

Quoted 5 lines
block/backup-top.c new file +4 -0
+            *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+        }
+        *nshared =
+            *nshared & (BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED);

And here it isn’t “*nshared = shared & rw” but “rw & ~WRITE”.

I feel like this function would be simpler if you just set *nperm and *nshared separately in the two branches of this condition, without setting a “default” first.

Ok, I'll try

Quoted 51 lines
Quoted 28 lines
block/backup-top.c new file +27 -0
+    }
+}
+
+BlockDriver bdrv_backup_top_filter = {
+    .format_name = "backup-top",
+    .instance_size = sizeof(BDRVBackupTopState),
+
+    .bdrv_co_preadv             = backup_top_co_preadv,
+    .bdrv_co_pwritev            = backup_top_co_pwritev,
+    .bdrv_co_pwrite_zeroes      = backup_top_co_pwrite_zeroes,
+    .bdrv_co_pdiscard           = backup_top_co_pdiscard,
+    .bdrv_co_flush              = backup_top_co_flush,
+
+    .bdrv_co_block_status       = bdrv_co_block_status_from_backing,
+
+    .bdrv_refresh_filename      = backup_top_refresh_filename,
+
+    .bdrv_child_perm            = backup_top_child_perm,
+
+    .is_filter = true,
+};
+
+BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
+                                         BlockDriverState *target,
+                                         HBitmap *copy_bitmap,
+                                         Error **errp)
+{

block/backup-top.c line 253

Quoted 10 lines
block/backup-top.c new file +10 -0
+    Error *local_err = NULL;
+    BDRVBackupTopState *state;
+    BlockDriverState *top = bdrv_new_open_driver(&bdrv_backup_top_filter,
+                                                 NULL, BDRV_O_RDWR, errp);
+
+    if (!top) {
+        return NULL;
+    }
+
+    top->implicit = true;

As I said above, I think the caller should set this.

Quoted 10 lines
block/backup-top.c new file +10 -0
+    top->total_sectors = source->total_sectors;
+    top->bl.opt_mem_alignment = MAX(bdrv_opt_mem_align(source),
+                                    bdrv_opt_mem_align(target));
+    top->opaque = state = g_new0(BDRVBackupTopState, 1);
+    state->copy_bitmap = copy_bitmap;
+
+    bdrv_ref(target);
+    state->target = bdrv_attach_child(top, target, "target", &child_file, errp);
+    if (!state->target) {
+        bdrv_unref(target);
Quoted 8 lines

block/backup-top.c line 269

Quoted 6 lines
block/backup-top.c new file +6 -0
+        bdrv_unref(top);
+        return NULL;
+    }
+
+    bdrv_set_aio_context(top, bdrv_get_aio_context(source));
+    bdrv_set_aio_context(target, bdrv_get_aio_context(source));

I suppose these calls aren’t necessary anymore (compare d0ee0204f4009).

Hmm, see it, agree.

Quoted 27 lines

(I’m not fully sure, though. In any case, they would need to be
bdrv_try_set_aio_context() if they were still necessary.)

Quoted 10 lines
block/backup-top.c new file +10 -0
+
+    bdrv_drained_begin(source);
+
+    bdrv_ref(top);
+    bdrv_append(top, source, &local_err);
+    if (local_err) {
+        error_prepend(&local_err, "Cannot append backup-top filter: ");
+    }
+
+    bdrv_drained_end(source);

block/backup-top.c line 289

Quoted 10 lines
block/backup-top.c new file +10 -0
+
+    if (local_err) {
+        bdrv_unref_child(top, state->target);
+        bdrv_unref(top);
+        error_propagate(errp, local_err);
+        return NULL;
+    }
+
+    return top;
+}

I guess it would be nice if it users could blockdev-add a backup-top node to basically get a backup with sync=none.

Quoted 4 lines

(The bdrv_open implementation should then create a new bitmap and initialize it to be fully set.)

But maybe it wouldn’t be so nice and I just have feverish dreams.

When series begun, I was trying to make exactly this - user-available filter, which may be used in separate, but you was against)

Maybe, not totally against, but I decided not to argue long, and instead make filter implicit and drop public api (like mirror and commit filters), but place it in a separate file (no one will argue against putting large enough and complete new feature, represented by object into a separate file :). And this actually makes it easy to publish this filter at some moment. And now I think it was good decision anyway, as we postponed arguing around API and around shared dirty bitmaps.

So publishing the filter is future step.

Quoted 17 lines
Quoted 17 lines
block/backup-top.c new file +16 -0
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque)
+{
+    BDRVBackupTopState *s = bs->opaque;
+
+    s->progress_cb = progress_cb;
+    s->progress_opaque = progress_opaque;
+}
+
+void bdrv_backup_top_drop(BlockDriverState *bs)
+{
+    BDRVBackupTopState *s = bs->opaque;
+    AioContext *aio_context = bdrv_get_aio_context(bs);
+
+    aio_context_acquire(aio_context);
Quoted 8 lines

block/backup-top.c line 310

Quoted 4 lines
block/backup-top.c new file +4 -0
+
+    bdrv_drained_begin(bs);
+
+    bdrv_child_try_set_perm(bs->backing, 0, BLK_PERM_ALL, &error_abort);

Pre-existing in other jobs, but I think calling this function is dangerous. (Which is why I remove it in my “block: Ignore loosening perm restrictions failures” series.)

Hmm, good thing.. Should I rebase on it?

Quoted 6 lines

block/backup-top.c line 312

Quoted 3 lines
block/backup-top.c new file +2 -0
+    bdrv_replace_node(bs, backing_bs(bs), &error_abort);
+    bdrv_set_backing_hd(bs, NULL, &error_abort);

I think some of this function should be in a .bdrv_close() implementation, for example this bdrv_set_backing_hd() call.

Why? We don't have .bdrv_open, so why to have .bdrv_close? I think, when we publish this filter most of _add() and _drop() will be refactored to open() and close(). Is there a real reason to implement .close() now?

Quoted 21 lines

block/backup-top.c line 317

Quoted 5 lines
block/backup-top.c new file +4 -0
+    bdrv_drained_end(bs);
+
+    if (s->target) {
+        bdrv_unref_child(bs, s->target);

And this. Well, neither needs to be in a .bdrv_close() implementation, actually, because bdrv_close() will just do it by itself.

I suppose you’re trying to remove the children so the node is no longer usable after this point; but that isn’t quite right, then. The filter functions still refer to s->target and bs->backing, so you need to stop them from doing anything then.

At this point, you might as well add a variable to BDRVBackupTopState that says whether the filter is still supposed to be usable (like the “stop” variable I added to mirror in “block/mirror: Fix child permissions”). If so, the permission function should signal 0/ALL for the permissions on backing (and probably target), and all filter functions always immediately return an error.

Quoted 3 lines

So I don’t think backing and target need to be removed here. We can wait for that until bdrv_close(), but we should ensure that the filter really is unusable after this function has been called.

Ok, thank you for reviewing!

Quoted 24 lines

Max

Quoted 19 lines
block/backup-top.c new file +5 -0
+    }
+    bdrv_unref(bs);
+
+    aio_context_release(aio_context);
+}
block/Makefile.objs +2 -0
index ae11605c9f..dfbdfe6ab4 100644
@@ -40,6 +40,8 @@ block-obj-y += throttle.o copy-on-read.o
 
  block-obj-y += crypto.o
 
+block-obj-y += backup-top.o
+
  common-obj-y += stream.o
 
  nfs.o-libs         := $(LIBNFS_LIBS)
Signature · 2 lines
Best regards,
Vladimir
#11 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

     +-------+
     | Guest |
     +-------+
         |r,w
         v
     +------------+  target   +---------------+
     | backup_top |---------->| target(qcow2) |
     +------------+   CBW     +---------------+
         |
backing |r,w
         v
     +-------------+
     | Active disk |
     +-------------+

The driver will be used in backup instead of write-notifiers.

---
  block/backup-top.h  |  64 +++++++++
  block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
  block/Makefile.objs |   2 +
  3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

[...]

Quoted 11 lines
Quoted 9 lines

block/backup-top.h

Quoted 7 lines
block/backup-top.h new file +7 -0
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

Hm, you've asked it already, on previous version :)

I wasn’t sure...

Quoted 3 lines

Backup don't do it,
so I just keep old behavior. And what is the reason to flush backup target on any guest flush?

Hm, well, what’s the reason not to do it?
Also, there are not only guest flushes. bdrv_flush_all() exists, which is called when the guest is stopped. So who is going to flush the target if not its parent?

[...]

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
block/backup-top.h new file +10 -0
+
+    if (role == &child_file) {
+        /*
+         * Target child
+         *
+         * Share write to target (child_file), to not interfere
+         * with guest writes to its disk which may be in target backing chain.
+         */
+        if (perm & BLK_PERM_WRITE) {
+            *nshared = *nshared | BLK_PERM_WRITE;
Quoted 4 lines
Quoted 1 line

Why not always share WRITE on the target?

Hmm, it's a bad thing to share writes on target, so I'm trying to reduce number of cases when we have share it.

Is it? First of all, this filter doesn’t care. It doesn’t even read from the target (related note: we probably don’t need CONSISTENT_READ on the target).

Second, there is generally going to be a parent on backup-top that has the WRITE permission taken. So this does not really effectively reduce that number of cases.

Quoted 8 lines
Quoted 6 lines

block/backup-top.h

Quoted 4 lines
block/backup-top.h new file +4 -0
+        }
+    } else {
+        /* Source child */
+        if (perm & BLK_PERM_WRITE) {

Or WRITE_UNCHANGED, no?

Why? We don't need doing CBW for unchanged write.

But we will do it still, won’t we?

(If an unchanging write comes in, this driver will handle it just like a normal write, will it not?)

[...]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
block/backup-top.h new file +6 -0
+
+BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
+                                         BlockDriverState *target,
+                                         HBitmap *copy_bitmap,
+                                         Error **errp)
+{

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

+}

I guess it would be nice if it users could blockdev-add a backup-top node to basically get a backup with sync=none.

Quoted 7 lines
Quoted 4 lines

(The bdrv_open implementation should then create a new bitmap and initialize it to be fully set.)

But maybe it wouldn’t be so nice and I just have feverish dreams.

When series begun, I was trying to make exactly this - user-available filter, which may be used in separate, but you was against)

Past me is stupid.

Quoted 9 lines

Maybe, not totally against, but I decided not to argue long, and instead make filter implicit and drop public api (like mirror and commit filters), but place it in a separate file (no one will argue against putting large enough and complete new feature, represented by object into a separate file :). And this actually makes it easy to publish this filter at some moment. And now I think it was good decision anyway, as we postponed arguing around API and around shared dirty bitmaps.

So publishing the filter is future step.

OK, sure.

Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
block/backup-top.h new file +16 -0
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque)
+{
+    BDRVBackupTopState *s = bs->opaque;
+
+    s->progress_cb = progress_cb;
+    s->progress_opaque = progress_opaque;
+}
+
+void bdrv_backup_top_drop(BlockDriverState *bs)
+{
+    BDRVBackupTopState *s = bs->opaque;
+    AioContext *aio_context = bdrv_get_aio_context(bs);
+
+    aio_context_acquire(aio_context);
Quoted 8 lines

block/backup-top.h

Quoted 4 lines
block/backup-top.h new file +4 -0
+
+    bdrv_drained_begin(bs);
+
+    bdrv_child_try_set_perm(bs->backing, 0, BLK_PERM_ALL, &error_abort);

Pre-existing in other jobs, but I think calling this function is dangerous. (Which is why I remove it in my “block: Ignore loosening perm restrictions failures” series.)

Quoted 1 line

Hmm, good thing.. Should I rebase on it?

It would help me at least.

Quoted 9 lines
Quoted 5 lines

block/backup-top.h

Quoted 2 lines
block/backup-top.h new file +2 -0
+    bdrv_replace_node(bs, backing_bs(bs), &error_abort);
+    bdrv_set_backing_hd(bs, NULL, &error_abort);

I think some of this function should be in a .bdrv_close() implementation, for example this bdrv_set_backing_hd() call.

Why? We don't have .bdrv_open, so why to have .bdrv_close? I think, when we publish this filter most of _add() and _drop() will be refactored to open() and close(). Is there a real reason to implement .close() now?

Not really if it isn’t a usable block driver yet, no.

Max

#12 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 57 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

      +-------+
      | Guest |
      +-------+
          |r,w
          v
      +------------+  target   +---------------+
      | backup_top |---------->| target(qcow2) |
      +------------+   CBW     +---------------+
          |
backing |r,w
          v
      +-------------+
      | Active disk |
      +-------------+

The driver will be used in backup instead of write-notifiers.

---
   block/backup-top.h  |  64 +++++++++
   block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
   block/Makefile.objs |   2 +
   3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Quoted 8 lines

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

Quoted 14 lines

[...]

Quoted 11 lines
Quoted 9 lines

block/backup-top.h

Quoted 7 lines
block/backup-top.h new file +7 -0
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

Hm, you've asked it already, on previous version :)

Quoted 7 lines

I wasn’t sure...

Quoted 3 lines

Backup don't do it,
so I just keep old behavior. And what is the reason to flush backup target on any guest flush?

Hm, well, what’s the reason not to do it?

guest flushes will be slowed down?

Quoted 5 lines

Also, there are not only guest flushes. bdrv_flush_all() exists, which is called when the guest is stopped. So who is going to flush the target if not its parent?

[...]

Backup job? But filter should not relay on it..

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

Quoted 17 lines
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
block/backup-top.h new file +10 -0
+
+    if (role == &child_file) {
+        /*
+         * Target child
+         *
+         * Share write to target (child_file), to not interfere
+         * with guest writes to its disk which may be in target backing chain.
+         */
+        if (perm & BLK_PERM_WRITE) {
+            *nshared = *nshared | BLK_PERM_WRITE;
Quoted 4 lines
Quoted 1 line

Why not always share WRITE on the target?

Hmm, it's a bad thing to share writes on target, so I'm trying to reduce number of cases when we have share it.

Quoted 7 lines

Is it? First of all, this filter doesn’t care. It doesn’t even read from the target (related note: we probably don’t need CONSISTENT_READ on the target).

Second, there is generally going to be a parent on backup-top that has the WRITE permission taken. So this does not really effectively reduce that number of cases.

Ok.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
block/backup-top.h new file +4 -0
+        }
+    } else {
+        /* Source child */
+        if (perm & BLK_PERM_WRITE) {
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Or WRITE_UNCHANGED, no?

Why? We don't need doing CBW for unchanged write.

But we will do it still, won’t we?

(If an unchanging write comes in, this driver will handle it just like a normal write, will it not?)

No, it will not, I check this flag in backup_top_co_pwritev

Quoted 64 lines

[...]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
block/backup-top.h new file +6 -0
+
+BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
+                                         BlockDriverState *target,
+                                         HBitmap *copy_bitmap,
+                                         Error **errp)
+{

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

+}

I guess it would be nice if it users could blockdev-add a backup-top node to basically get a backup with sync=none.

Quoted 7 lines
Quoted 4 lines

(The bdrv_open implementation should then create a new bitmap and initialize it to be fully set.)

But maybe it wouldn’t be so nice and I just have feverish dreams.

When series begun, I was trying to make exactly this - user-available filter, which may be used in separate, but you was against)

Past me is stupid.

Quoted 9 lines

Maybe, not totally against, but I decided not to argue long, and instead make filter implicit and drop public api (like mirror and commit filters), but place it in a separate file (no one will argue against putting large enough and complete new feature, represented by object into a separate file :). And this actually makes it easy to publish this filter at some moment. And now I think it was good decision anyway, as we postponed arguing around API and around shared dirty bitmaps.

So publishing the filter is future step.

OK, sure.

Quoted 25 lines
Quoted 24 lines
Quoted 10 lines
block/backup-top.h new file +10 -0
+void bdrv_backup_top_set_progress_callback(
+        BlockDriverState *bs, BackupTopProgressCallback progress_cb,
+        void *progress_opaque)
+{
+    BDRVBackupTopState *s = bs->opaque;
+
+    s->progress_cb = progress_cb;
+    s->progress_opaque = progress_opaque;
+}
+

block/backup-top.h

Quoted 10 lines
block/backup-top.h new file +10 -0
+void bdrv_backup_top_drop(BlockDriverState *bs)
+{
+    BDRVBackupTopState *s = bs->opaque;
+    AioContext *aio_context = bdrv_get_aio_context(bs);
+
+    aio_context_acquire(aio_context);
+
+    bdrv_drained_begin(bs);
+
+    bdrv_child_try_set_perm(bs->backing, 0, BLK_PERM_ALL, &error_abort);

Pre-existing in other jobs, but I think calling this function is dangerous. (Which is why I remove it in my “block: Ignore loosening perm restrictions failures” series.)

Quoted 3 lines
Quoted 1 line

Hmm, good thing.. Should I rebase on it?

It would help me at least.

And now it's already staged, so I should.

Quoted 15 lines
Quoted 10 lines
Quoted 6 lines

block/backup-top.h

Quoted 3 lines
block/backup-top.h new file +2 -0
+    bdrv_replace_node(bs, backing_bs(bs), &error_abort);
+    bdrv_set_backing_hd(bs, NULL, &error_abort);

I think some of this function should be in a .bdrv_close() implementation, for example this bdrv_set_backing_hd() call.

Why? We don't have .bdrv_open, so why to have .bdrv_close? I think, when we publish this filter most of _add() and _drop() will be refactored to open() and close(). Is there a real reason to implement .close() now?

Not really if it isn’t a usable block driver yet, no.

Max

Signature · 2 lines
Best regards,
Vladimir
#13 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 63 lines · Vladimir Sementsov-Ogievskiy
Quoted 62 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

      +-------+
      | Guest |
      +-------+
          |r,w
          v
      +------------+  target   +---------------+
      | backup_top |---------->| target(qcow2) |
      +------------+   CBW     +---------------+
          |
backing |r,w
          v
      +-------------+
      | Active disk |
      +-------------+

The driver will be used in backup instead of write-notifiers.

---
   block/backup-top.h  |  64 +++++++++
   block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
   block/Makefile.objs |   2 +
   3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

Quoted 8 lines
Quoted 3 lines

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

Well, we try, at least. At least we hide them from query-block.

Quoted 3 lines

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

The theory is that qemu may decide to add nodes at any point, but at least when managing chains etc., they may not be visible to the user. I don’t think we can get rid of them so easily.

One example that isn’t implemented yet is copy-on-read. In theory, specifying copy-on-read=on for -drive should create an implicit COR node on top. The user shouldn’t see that node when inspecting the drive or when issuing block jobs on it, etc. And this node has to stay there when the user does e.g. an active commit somewhere down the chain.

That sounds like a horrible ordeal to implement, so it hasn’t been done yet. Maybe it never will. It isn’t that bad for the job filters, because they generally freeze the block graph, so there is no problem with potential modifications.

All in all I do think having implicit nodes makes sense. Maybe not so much now, but in the future (if someone implements converting -drive COR and throttle options to implicit nodes...).

Quoted 16 lines
Quoted 16 lines

[...]

Quoted 11 lines
Quoted 9 lines

block/backup-top.h

Quoted 7 lines
block/backup-top.h new file +7 -0
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

Hm, you've asked it already, on previous version :)

I wasn’t sure...

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Backup don't do it,
so I just keep old behavior. And what is the reason to flush backup target on any guest flush?

Hm, well, what’s the reason not to do it?

guest flushes will be slowed down?

Hm, the user could specify cache=unsafe if they don’t care. Which gives me second thoughs... [1]

Quoted 7 lines
Quoted 5 lines

Also, there are not only guest flushes. bdrv_flush_all() exists, which is called when the guest is stopped. So who is going to flush the target if not its parent?

[...]

Backup job? But filter should not relay on it..

Hm. Backup job actually doesn’t sound that wrong.

Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

But there is also the other case of image fleecing. In this case, the target will have another parent, so bdrv_flush_all() will be done by someone. Still, it intuitively makes sense to me that in this case, the backup-top filter should flush the target if the guest decides to flush the device (so that the target stays consistent on disk).

backup-top currently cannot differentiate between the cases, but maybe that is generally a useful hint to give to it? (That is, whether the target has a consistent state or not.)

[...]

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

block/backup-top.h

Quoted 4 lines
block/backup-top.h new file +4 -0
+        }
+    } else {
+        /* Source child */
+        if (perm & BLK_PERM_WRITE) {

Or WRITE_UNCHANGED, no?

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Why? We don't need doing CBW for unchanged write.

But we will do it still, won’t we?

(If an unchanging write comes in, this driver will handle it just like a normal write, will it not?)

No, it will not, I check this flag in backup_top_co_pwritev

Oops. My bad.

Max

#14 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 83 lines · Max Reitz
Quoted 63 lines · Vladimir Sementsov-Ogievskiy
Quoted 62 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

       +-------+
       | Guest |
       +-------+
           |r,w
           v
       +------------+  target   +---------------+
       | backup_top |---------->| target(qcow2) |
       +------------+   CBW     +---------------+
           |
backing |r,w
           v
       +-------------+
       | Active disk |
       +-------------+

The driver will be used in backup instead of write-notifiers.

---
    block/backup-top.h  |  64 +++++++++
    block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
    block/Makefile.objs |   2 +
    3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

Quoted 8 lines
Quoted 3 lines

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

Well, we try, at least. At least we hide them from query-block.

Quoted 3 lines

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

The theory is that qemu may decide to add nodes at any point, but at least when managing chains etc., they may not be visible to the user. I don’t think we can get rid of them so easily.

Quoted 5 lines

One example that isn’t implemented yet is copy-on-read. In theory, specifying copy-on-read=on for -drive should create an implicit COR node on top. The user shouldn’t see that node when inspecting the drive or when issuing block jobs on it, etc. And this node has to stay there when the user does e.g. an active commit somewhere down the chain.

Why instead not to just write in doc that yes, filter is created when you enable copy-on-read? And do same for all operations which may create filter, we don't have a lot of them? And add optional parameter to set node-name for created filter.

Quoted 9 lines

That sounds like a horrible ordeal to implement, so it hasn’t been done yet. Maybe it never will. It isn’t that bad for the job filters, because they generally freeze the block graph, so there is no problem with potential modifications.

All in all I do think having implicit nodes makes sense. Maybe not so much now, but in the future (if someone implements converting -drive COR and throttle options to implicit nodes...).

But do we have at least strong definition of how implicit nodes should behave on any graph-modifying operations around them? Should new implicit/explicit filters be created above or under them? We should specify it in doc about all such operations, otherwise effect of implicit nodes may change unpredictably for the user. I'm afraid, that implicit filters will just continue my list of features which should not be used if we want to be able to maintain all this mess..

1. If you something around filenames don't work, use node-names, and better don't use filename-based APIs

2. If you qemu-img can't do what you need, use paused qemu process, and better, don't use qemu-img

and a new one:

3. If something is broken around jobs and filters and any block operations, set filter-node-name in parameters to make filter node explicit. And better, always set it..

So at least, I think we should make it possible for all filters to be explicit if user wants, by setting its name like in mirror.

(and then, I will not really care of implicit-filters related logic, like I don't really care about filename-api related logic)

Also, now, implict filters are actually available for direct manipulation by user, as their node-names are exported in query-named-block-nodes and nothing prevents using these names in differet APIs.

Quoted 56 lines
Quoted 17 lines
Quoted 17 lines

[...]

Quoted 11 lines
Quoted 9 lines

block/backup-top.h

Quoted 7 lines
block/backup-top.h new file +7 -0
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

Hm, you've asked it already, on previous version :)

I wasn’t sure...

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Backup don't do it,
so I just keep old behavior. And what is the reason to flush backup target on any guest flush?

Hm, well, what’s the reason not to do it?

guest flushes will be slowed down?

Hm, the user could specify cache=unsafe if they don’t care. Which gives me second thoughs... [1]

Quoted 7 lines
Quoted 5 lines

Also, there are not only guest flushes. bdrv_flush_all() exists, which is called when the guest is stopped. So who is going to flush the target if not its parent?

[...]

Backup job? But filter should not relay on it..

Hm. Backup job actually doesn’t sound that wrong.

Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

But there is also the other case of image fleecing. In this case, the target will have another parent, so bdrv_flush_all() will be done by someone. Still, it intuitively makes sense to me that in this case, the backup-top filter should flush the target if the guest decides to flush the device (so that the target stays consistent on disk).

Quoted 3 lines

backup-top currently cannot differentiate between the cases, but maybe that is generally a useful hint to give to it? (That is, whether the target has a consistent state or not.)

Hmm, for fleecing we don't need consistent state of temporary image. If something fails, the whole operation is considered to be failed. And anyway, I don't see relations between consistency of temporary fleecing image and guest flushes, why should we bind them?

Quoted 23 lines

[...]

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

block/backup-top.h

Quoted 4 lines
block/backup-top.h new file +4 -0
+        }
+    } else {
+        /* Source child */
+        if (perm & BLK_PERM_WRITE) {

Or WRITE_UNCHANGED, no?

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Why? We don't need doing CBW for unchanged write.

But we will do it still, won’t we?

(If an unchanging write comes in, this driver will handle it just like a normal write, will it not?)

No, it will not, I check this flag in backup_top_co_pwritev

Oops. My bad.

Max

Signature · 2 lines
Best regards,
Vladimir
#15 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 84 lines · Vladimir Sementsov-Ogievskiy
Quoted 83 lines · Max Reitz
Quoted 63 lines · Vladimir Sementsov-Ogievskiy
Quoted 62 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

       +-------+
       | Guest |
       +-------+
           |r,w
           v
       +------------+  target   +---------------+
       | backup_top |---------->| target(qcow2) |
       +------------+   CBW     +---------------+
           |
backing |r,w
           v
       +-------------+
       | Active disk |
       +-------------+

The driver will be used in backup instead of write-notifiers.

---
    block/backup-top.h  |  64 +++++++++
    block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
    block/Makefile.objs |   2 +
    3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

Quoted 8 lines
Quoted 3 lines

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

Well, we try, at least. At least we hide them from query-block.

Quoted 3 lines

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

The theory is that qemu may decide to add nodes at any point, but at least when managing chains etc., they may not be visible to the user. I don’t think we can get rid of them so easily.

Quoted 8 lines
Quoted 5 lines

One example that isn’t implemented yet is copy-on-read. In theory, specifying copy-on-read=on for -drive should create an implicit COR node on top. The user shouldn’t see that node when inspecting the drive or when issuing block jobs on it, etc. And this node has to stay there when the user does e.g. an active commit somewhere down the chain.

Why instead not to just write in doc that yes, filter is created when you enable copy-on-read?

Because the problem is that existing users are not aware of that.

If they were, they could simply create a COR filter already.

I suppose we could interpret the deprecation policy in a way that we could change the behavior of -drive copy-on-read=on, but as I already said, what’s the point of supporting -drive copy-on-read=on, when you can simply explicitly create a COR filter on top?

Quoted 3 lines

And do same for all operations which may create filter,
we don't have a lot of them? And add optional parameter to set node-name for created filter.

That optional parameter exists, and if it is given, the user shows that they are aware of the node. Hence the node becomes explicit then.

The problem is with legacy users that use an existing interface and suddenly the externally visible interface of qemu changes in a way that could break them. I suppose additional entries in query-named-block-nodes is not breakage. Maybe it is, that would be a bug then.

(If nobody has noticed so far, that may be because these legacy applications didn’t use query-named-block-nodes. But now maybe libvirt does, and so if we converted copy-on-read=on to a COR node, it would notice now and thus break. So just because our handling of implicit nodes is broken right now does not mean we can abandon the concept altogether.)

Quoted 11 lines
Quoted 8 lines

That sounds like a horrible ordeal to implement, so it hasn’t been done yet. Maybe it never will. It isn’t that bad for the job filters, because they generally freeze the block graph, so there is no problem with potential modifications.

All in all I do think having implicit nodes makes sense. Maybe not so much now, but in the future (if someone implements converting -drive COR and throttle options to implicit nodes...).

But do we have at least strong definition of how implicit nodes should behave on any graph-modifying operations around them?

No. Depends on the node.

Quoted 2 lines

Should new implicit/explicit
filters be created above or under them?

That was always the most difficult question we had when we introduced filters.

The problem is that we never answered it in our code base.

One day, we just added filters (“let’s see what happens”), and in the beginning, they were all explicit, so we still didn’t have to answer this question (in code). Then job filters were added, and we still didn’t have to, because they set blockers so the graph couldn’t be reorganized with them in place anyway.

If we converted copy-on-read=on to a COR node, we would have to answer that question.

Quoted 4 lines

We should specify it in doc about all
such operations, otherwise effect of implicit nodes may change unpredictably for the user. I'm afraid, that implicit filters will just continue my list of features which should not be used if we want to be able to maintain all this mess..

Yeah, well, that is nothing new at all. For example, note the “Split I/O throttling off into own BDS – Requires some care with snapshots etc.” here:
https://wiki.qemu.org/ToDo/Block#Dynamic_graph_reconfiguration_.28e.g._adding_block_filters.2C_taking_snapshots.2C_etc..29

This was always a problem. It was always the biggest problem with filters. We never answered it because it never become an acute problem, as I described above.

We just said “For explicit filters, they just stay where they are, and that’s OK because the user can take care of them.” We never did add the implicit filters that were actually going to become a problem (COR and throttle).

So as I said above, the behavior of implicit nodes needs to be examined on a case-by-case basis. Well, actually, this is only about COR and throttle, really, and both would behave the same: Always stay at the BlockBackend, because that is the behavior when you currently use BB-level throttling or COR.

The jobs that now use implicit nodes freeze the graph, so there is no problem there.

(Also, by the way, we do not need the option to give COR/throttle nodes created through BB options a node name. As I said above, at that point you can just create the node yourself.)

Quoted 2 lines

1. If you something around filenames don't work, use node-names, and better don't use filename-based APIs

I think we agree that filename-based APIs were a mistake. (Though only in hindsight. They were added when there where no node names, so it did make sense then.)

So now you should not use them, correct.

Quoted 2 lines

2. If you qemu-img can't do what you need, use paused qemu process, and better, don't use qemu-img

There was always a plan to give qemu-img feature parity to qemu proper.
But often the interface is just a problem. Node names really don’t
make much sense with qemu-img, I think.

(I mean, feel free to add something that automatically names the backing nodes e.g. backing.0, backing.1, etc., and then add an interface to qemu-img commit to use node names, but I find that weird.)

So I don’t quite see the problem. Of course qemu has more functionality than qemu-img. qemu-img is fine as long as you have a simple set-up, but once it gets more complicated, you need management software; and that software shouldn’t care much whether it uses qemu-img or qemu itself.

Quoted 5 lines

and a new one:

3. If something is broken around jobs and filters and any block operations, set filter-node-name in parameters to make filter node explicit. And better, always set it..

Uh, yeah? I don’t see the problem here.

This is like saying “If I want to use new features, I should use -blockdev and not -hda”. Yes, that is correct. You should set filter-node-name if you care about the exact state of the graph.

Quoted 2 lines

So at least, I think we should make it possible for all filters to be explicit if user wants, by setting its name like in mirror.

For all filters that the user has no other way of creating them, yes. (That is, the filters created by the block jobs.)

Quoted 6 lines

(and then, I will not really care of implicit-filters related logic, like I don't really care about filename-api related logic)

Also, now, implict filters are actually available for direct manipulation by user, as their node-names are exported in query-named-block-nodes and nothing prevents using these names in differet APIs.

Hm, yes. Is that a problem?

Quoted 55 lines
Quoted 55 lines
Quoted 16 lines
Quoted 16 lines

[...]

Quoted 11 lines
Quoted 9 lines

block/backup-top.h

Quoted 7 lines
block/backup-top.h new file +7 -0
+static int coroutine_fn backup_top_co_flush(BlockDriverState *bs)
+{
+    if (!bs->backing) {
+        return 0;
+    }
+
+    return bdrv_co_flush(bs->backing->bs);

Should we flush the target, too?

Hm, you've asked it already, on previous version :)

I wasn’t sure...

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Backup don't do it,
so I just keep old behavior. And what is the reason to flush backup target on any guest flush?

Hm, well, what’s the reason not to do it?

guest flushes will be slowed down?

Hm, the user could specify cache=unsafe if they don’t care. Which gives me second thoughs... [1]

Quoted 7 lines
Quoted 5 lines

Also, there are not only guest flushes. bdrv_flush_all() exists, which is called when the guest is stopped. So who is going to flush the target if not its parent?

[...]

Backup job? But filter should not relay on it..

Hm. Backup job actually doesn’t sound that wrong.

Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

But there is also the other case of image fleecing. In this case, the target will have another parent, so bdrv_flush_all() will be done by someone. Still, it intuitively makes sense to me that in this case, the backup-top filter should flush the target if the guest decides to flush the device (so that the target stays consistent on disk).

Quoted 8 lines
Quoted 3 lines

backup-top currently cannot differentiate between the cases, but maybe that is generally a useful hint to give to it? (That is, whether the target has a consistent state or not.)

Hmm, for fleecing we don't need consistent state of temporary image. If something fails, the whole operation is considered to be failed. And anyway, I don't see relations between consistency of temporary fleecing image and guest flushes, why should we bind them?

Guest flushes are about clearing the cache and keeping the disk state consistent in case of e.g. power failure. So if the target should be consistent, it too should then survive a power failure.

Hm, but then again, if we don’t enforce on the host that the target is always flushed before the source, that consistency is quickly lost when the host decides to flush the cache just because it wants to.

Hmmm. I guess we could say that the target is always inconsistent on e.g. power failure.

My problem is that the target is also, well, a backup, you know. So it’s kind of a pain if we don’t care about it. The user probably wanted to save that old data, and now it may be lost just because we didn’t flush it when the user entered “sync” in the VM. I don’t know.

Max

#16 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Am 17.06.2019 um 16:56 hat Max Reitz geschrieben:

Quoted 77 lines · Vladimir Sementsov-Ogievskiy
Quoted 76 lines
Quoted 75 lines · Max Reitz
Quoted 63 lines · Vladimir Sementsov-Ogievskiy
Quoted 62 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

       +-------+
       | Guest |
       +-------+
           |r,w
           v
       +------------+  target   +---------------+
       | backup_top |---------->| target(qcow2) |
       +------------+   CBW     +---------------+
           |
backing |r,w
           v
       +-------------+
       | Active disk |
       +-------------+

The driver will be used in backup instead of write-notifiers.

---
    block/backup-top.h  |  64 +++++++++
    block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
    block/Makefile.objs |   2 +
    3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

Quoted 8 lines
Quoted 3 lines

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

Well, we try, at least. At least we hide them from query-block.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

The theory is that qemu may decide to add nodes at any point, but at least when managing chains etc., they may not be visible to the user. I don’t think we can get rid of them so easily.

I think the important point to understand here is that implicit filters are a concept to maintain compatibility with legacy (pre-blockdev) management tools which simply don't manage stuff on the node level. So changing the structure of the graph is not a problem for them and we just need to make sure that when they do something with a BlockBackend, we perform the action on the node that they actually mean.

Modern management tools are expected to manage the graph on a node level and to assign node names to everything.

Note that libvirt is close to becoming a modern management tool, so it's probably a good idea to now make all block jobs add filters where we'll need them in the long run.

Quoted 9 lines
Quoted 8 lines
Quoted 5 lines

One example that isn’t implemented yet is copy-on-read. In theory, specifying copy-on-read=on for -drive should create an implicit COR node on top. The user shouldn’t see that node when inspecting the drive or when issuing block jobs on it, etc. And this node has to stay there when the user does e.g. an active commit somewhere down the chain.

Why instead not to just write in doc that yes, filter is created when you enable copy-on-read?

Quoted 8 lines

Because the problem is that existing users are not aware of that.

If they were, they could simply create a COR filter already.

I suppose we could interpret the deprecation policy in a way that we could change the behavior of -drive copy-on-read=on, but as I already said, what’s the point of supporting -drive copy-on-read=on, when you can simply explicitly create a COR filter on top?

I actually changed my opinion on how to do this since we introduced implicit filters. I know think that the right way to move forward is to make sure that the copy-on-read filter can do everything it needs to do, and then just completely deprecate -drive copy-on-read=on instead of adding compatibility magic to turn it into an implicit copy-on-read filter internally.

Quoted 22 lines
Quoted 11 lines
Quoted 8 lines

That sounds like a horrible ordeal to implement, so it hasn’t been done yet. Maybe it never will. It isn’t that bad for the job filters, because they generally freeze the block graph, so there is no problem with potential modifications.

All in all I do think having implicit nodes makes sense. Maybe not so much now, but in the future (if someone implements converting -drive COR and throttle options to implicit nodes...).

But do we have at least strong definition of how implicit nodes should behave on any graph-modifying operations around them?

No. Depends on the node.

Quoted 2 lines

Should new implicit/explicit
filters be created above or under them?

That was always the most difficult question we had when we introduced filters.

The problem is that we never answered it in our code base.

Quoted 8 lines

One day, we just added filters (“let’s see what happens”), and in the beginning, they were all explicit, so we still didn’t have to answer this question (in code). Then job filters were added, and we still didn’t have to, because they set blockers so the graph couldn’t be reorganized with them in place anyway.

If we converted copy-on-read=on to a COR node, we would have to answer that question.

That's why we shouldn't do that, but just remove copy-on-read=on. :-)

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

Depends, the target could have the source as its backing file (like fleecing, but without exporting it right now), and then you could always have a consistent view on the target. Well, almost.

Almost because to guarantee this, we'd have to flush between each CBW and the corresponding write to the same block, to make sure that the old data is backed up before it is overwritten.

Of course, this would perform about as badly as internal COW in qcow2... So probably not a guarantee we should be making by default. But it might make sense as an option.

Kevin

#17 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 118 lines · Kevin Wolf

Am 17.06.2019 um 16:56 hat Max Reitz geschrieben:

Quoted 77 lines · Vladimir Sementsov-Ogievskiy
Quoted 76 lines
Quoted 75 lines · Max Reitz
Quoted 63 lines · Vladimir Sementsov-Ogievskiy
Quoted 62 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

       +-------+
       | Guest |
       +-------+
           |r,w
           v
       +------------+  target   +---------------+
       | backup_top |---------->| target(qcow2) |
       +------------+   CBW     +---------------+
           |
backing |r,w
           v
       +-------------+
       | Active disk |
       +-------------+

The driver will be used in backup instead of write-notifiers.

---
    block/backup-top.h  |  64 +++++++++
    block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
    block/Makefile.objs |   2 +
    3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

Quoted 8 lines
Quoted 3 lines

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

Well, we try, at least. At least we hide them from query-block.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

The theory is that qemu may decide to add nodes at any point, but at least when managing chains etc., they may not be visible to the user. I don’t think we can get rid of them so easily.

I think the important point to understand here is that implicit filters are a concept to maintain compatibility with legacy (pre-blockdev) management tools which simply don't manage stuff on the node level. So changing the structure of the graph is not a problem for them and we just need to make sure that when they do something with a BlockBackend, we perform the action on the node that they actually mean.

Modern management tools are expected to manage the graph on a node level and to assign node names to everything.

Note that libvirt is close to becoming a modern management tool, so it's probably a good idea to now make all block jobs add filters where we'll need them in the long run.

Quoted 9 lines
Quoted 8 lines
Quoted 5 lines

One example that isn’t implemented yet is copy-on-read. In theory, specifying copy-on-read=on for -drive should create an implicit COR node on top. The user shouldn’t see that node when inspecting the drive or when issuing block jobs on it, etc. And this node has to stay there when the user does e.g. an active commit somewhere down the chain.

Why instead not to just write in doc that yes, filter is created when you enable copy-on-read?

Quoted 8 lines

Because the problem is that existing users are not aware of that.

If they were, they could simply create a COR filter already.

I suppose we could interpret the deprecation policy in a way that we could change the behavior of -drive copy-on-read=on, but as I already said, what’s the point of supporting -drive copy-on-read=on, when you can simply explicitly create a COR filter on top?

Quoted 6 lines

I actually changed my opinion on how to do this since we introduced implicit filters. I know think that the right way to move forward is to make sure that the copy-on-read filter can do everything it needs to do, and then just completely deprecate -drive copy-on-read=on instead of adding compatibility magic to turn it into an implicit copy-on-read filter internally.

Sure, so your answer to “what’s the point of supporting -drive copy-on-read=on” is “there is none”. Fair enough. :-)

Quoted 28 lines
Quoted 28 lines
Quoted 11 lines
Quoted 8 lines

That sounds like a horrible ordeal to implement, so it hasn’t been done yet. Maybe it never will. It isn’t that bad for the job filters, because they generally freeze the block graph, so there is no problem with potential modifications.

All in all I do think having implicit nodes makes sense. Maybe not so much now, but in the future (if someone implements converting -drive COR and throttle options to implicit nodes...).

But do we have at least strong definition of how implicit nodes should behave on any graph-modifying operations around them?

No. Depends on the node.

Quoted 2 lines

Should new implicit/explicit
filters be created above or under them?

That was always the most difficult question we had when we introduced filters.

The problem is that we never answered it in our code base.

One day, we just added filters (“let’s see what happens”), and in the beginning, they were all explicit, so we still didn’t have to answer this question (in code). Then job filters were added, and we still didn’t have to, because they set blockers so the graph couldn’t be reorganized with them in place anyway.

Quoted 4 lines
Quoted 2 lines

If we converted copy-on-read=on to a COR node, we would have to answer that question.

That's why we shouldn't do that, but just remove copy-on-read=on. :-)

And BB-level throttling, fair enough.

(Although the first step would be probably to make throttle groups non-experimental? Like, drop the x- prefix for all their parameters.)

Quoted 12 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

Quoted 7 lines

Depends, the target could have the source as its backing file (like fleecing, but without exporting it right now), and then you could always have a consistent view on the target. Well, almost.

Almost because to guarantee this, we'd have to flush between each CBW and the corresponding write to the same block, to make sure that the old data is backed up before it is overwritten.

Yes, that’s what I meant by “enforce on the host that the target is always flushed before the source”. Well, I meant to say there is no good way of doing that, and I kind of don’t consider this a good way.

Quoted 3 lines

Of course, this would perform about as badly as internal COW in qcow2... So probably not a guarantee we should be making by default. But it might make sense as an option.

I don’t know. “Use this option so your data isn’t lost in case of a power failure, but it makes everything pretty slow”? Who is going to do explicitly enable that (before their first data loss)?

Max

#18 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Am 17.06.2019 um 18:01 hat Max Reitz geschrieben:

Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 2 lines

Should new implicit/explicit
filters be created above or under them?

That was always the most difficult question we had when we introduced filters.

The problem is that we never answered it in our code base.

One day, we just added filters (“let’s see what happens”), and in the beginning, they were all explicit, so we still didn’t have to answer this question (in code). Then job filters were added, and we still didn’t have to, because they set blockers so the graph couldn’t be reorganized with them in place anyway.

If we converted copy-on-read=on to a COR node, we would have to answer that question.

Quoted 6 lines
Quoted 1 line

That's why we shouldn't do that, but just remove copy-on-read=on. :-)

And BB-level throttling, fair enough.

(Although the first step would be probably to make throttle groups non-experimental? Like, drop the x- prefix for all their parameters.)

The first step would be making the block drivers full replacements of the old things, which unfortunately isn't true today.

After your "deal with filters" series, we should be a lot closer for the core infrastructure at least.

Not sure about copy-on-read, but I know that throttle still doesn't have feature parity with -drive throttling. At least I'm pretty sure that something about changing the group or group options at runtime (and not just dropping the x-) was missing.

Quoted 24 lines
Quoted 12 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

Quoted 7 lines

Depends, the target could have the source as its backing file (like fleecing, but without exporting it right now), and then you could always have a consistent view on the target. Well, almost.

Almost because to guarantee this, we'd have to flush between each CBW and the corresponding write to the same block, to make sure that the old data is backed up before it is overwritten.

Yes, that’s what I meant by “enforce on the host that the target is always flushed before the source”. Well, I meant to say there is no good way of doing that, and I kind of don’t consider this a good way.

Quoted 7 lines
Quoted 3 lines

Of course, this would perform about as badly as internal COW in qcow2... So probably not a guarantee we should be making by default. But it might make sense as an option.

I don’t know. “Use this option so your data isn’t lost in case of a power failure, but it makes everything pretty slow”? Who is going to do explicitly enable that (before their first data loss)?

Maybe the backup job wasn't that clever after all. At least if you care about keeping the point-in-time snapshot at the start of the backup job instead of just retrying and getting a snapshot of a different point in time that is just as good.

If you do care about the specific point in time, the safe way to do it is to take a snapshot and copy that away, and then delete the snapshot again.

The only problem is that merging external snapshots is slow and you end up writing the new data twice. If only we had a COW image format... :-)

Kevin

#19 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 28 lines · Kevin Wolf

Am 17.06.2019 um 18:01 hat Max Reitz geschrieben:

Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 2 lines

Should new implicit/explicit
filters be created above or under them?

That was always the most difficult question we had when we introduced filters.

The problem is that we never answered it in our code base.

One day, we just added filters (“let’s see what happens”), and in the beginning, they were all explicit, so we still didn’t have to answer this question (in code). Then job filters were added, and we still didn’t have to, because they set blockers so the graph couldn’t be reorganized with them in place anyway.

If we converted copy-on-read=on to a COR node, we would have to answer that question.

Quoted 6 lines
Quoted 1 line

That's why we shouldn't do that, but just remove copy-on-read=on. :-)

And BB-level throttling, fair enough.

(Although the first step would be probably to make throttle groups non-experimental? Like, drop the x- prefix for all their parameters.)

The first step would be making the block drivers full replacements of the old things, which unfortunately isn't true today.

Quoted 7 lines

After your "deal with filters" series, we should be a lot closer for the core infrastructure at least.

Not sure about copy-on-read, but I know that throttle still doesn't have feature parity with -drive throttling. At least I'm pretty sure that something about changing the group or group options at runtime (and not just dropping the x-) was missing.

OK, thanks, I understand now that implicit filters are not just a feature but compatibility mechanism.

So, can we at some point deprecate "optionality" of filter-node-name parameters of jobs, in addition to deprecation of things like old copy-on-read option?
And actually deprecate implicit filters by this?

Quoted 38 lines
Quoted 25 lines
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

Quoted 7 lines

Depends, the target could have the source as its backing file (like fleecing, but without exporting it right now), and then you could always have a consistent view on the target. Well, almost.

Almost because to guarantee this, we'd have to flush between each CBW and the corresponding write to the same block, to make sure that the old data is backed up before it is overwritten.

Yes, that’s what I meant by “enforce on the host that the target is always flushed before the source”. Well, I meant to say there is no good way of doing that, and I kind of don’t consider this a good way.

Quoted 7 lines
Quoted 3 lines

Of course, this would perform about as badly as internal COW in qcow2... So probably not a guarantee we should be making by default. But it might make sense as an option.

I don’t know. “Use this option so your data isn’t lost in case of a power failure, but it makes everything pretty slow”? Who is going to do explicitly enable that (before their first data loss)?

Maybe the backup job wasn't that clever after all. At least if you care about keeping the point-in-time snapshot at the start of the backup job instead of just retrying and getting a snapshot of a different point in time that is just as good.

Quoted 7 lines

If you do care about the specific point in time, the safe way to do it is to take a snapshot and copy that away, and then delete the snapshot again.

The only problem is that merging external snapshots is slow and you end up writing the new data twice. If only we had a COW image format... :-)

So, I still don't understand the reason of flushing backup target in a meantime. Backup target remains invalid until the successful end of the job, at which point we definitely flush target, but only once.

If node crashes during backup, backup would be invalid independently of were there flushes after each write (or better just enable O_DIRECT) or not.

Signature · 2 lines
Best regards,
Vladimir
#20 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Am 18.06.2019 um 09:19 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 12 lines
Quoted 10 lines · Kevin Wolf

The first step would be making the block drivers full replacements of the old things, which unfortunately isn't true today.

After your "deal with filters" series, we should be a lot closer for the core infrastructure at least.

Not sure about copy-on-read, but I know that throttle still doesn't have feature parity with -drive throttling. At least I'm pretty sure that something about changing the group or group options at runtime (and not just dropping the x-) was missing.

Quoted 6 lines

OK, thanks, I understand now that implicit filters are not just a feature but compatibility mechanism.

So, can we at some point deprecate "optionality" of filter-node-name parameters of jobs, in addition to deprecation of things like old copy-on-read option? And actually deprecate implicit filters by this?

Hm, I'm not sure if we have ever moved an optional feature to required, and how to communicate this to libvirt, but this would be ideal, yes.

Quoted 44 lines
Quoted 37 lines
Quoted 24 lines
Quoted 12 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

Quoted 7 lines

Depends, the target could have the source as its backing file (like fleecing, but without exporting it right now), and then you could always have a consistent view on the target. Well, almost.

Almost because to guarantee this, we'd have to flush between each CBW and the corresponding write to the same block, to make sure that the old data is backed up before it is overwritten.

Yes, that’s what I meant by “enforce on the host that the target is always flushed before the source”. Well, I meant to say there is no good way of doing that, and I kind of don’t consider this a good way.

Quoted 7 lines
Quoted 3 lines

Of course, this would perform about as badly as internal COW in qcow2... So probably not a guarantee we should be making by default. But it might make sense as an option.

I don’t know. “Use this option so your data isn’t lost in case of a power failure, but it makes everything pretty slow”? Who is going to do explicitly enable that (before their first data loss)?

Maybe the backup job wasn't that clever after all. At least if you care about keeping the point-in-time snapshot at the start of the backup job instead of just retrying and getting a snapshot of a different point in time that is just as good.

Quoted 6 lines

If you do care about the specific point in time, the safe way to do it is to take a snapshot and copy that away, and then delete the snapshot again.

The only problem is that merging external snapshots is slow and you end up writing the new data twice. If only we had a COW image format... :-)

Quoted 7 lines

So, I still don't understand the reason of flushing backup target in a meantime. Backup target remains invalid until the successful end of the job, at which point we definitely flush target, but only once.

If node crashes during backup, backup would be invalid independently of were there flushes after each write (or better just enable O_DIRECT) or not.

Say your VM is running on disk.qcow2 and you use a backup job to copy data to backup.qcow2. At some point, the host crashes. If we made sure that every CBW to backup.qcow2 has completed before we write new data to disk.qcow2, backup.qcow2 contains valid data that represents the state at the start of the backup job as long as you use disk.qcow2 as its backing file.

The only way to ensure the right order is flushing between the two operations. If you don't do that, then yes, your backing is invalid before it has completed.

O_DIRECT doesn't guarantee that the data is on disk, only a flush does that. Maybe your chances that things actually make it to the disk in case of a host crash are higher if it sits in the disk cache rather than in the host's RAM, but there is no guarantee without a flush.

Now flushing the target when the guest flushes its disk doesn't give you new guarantees. Maybe it increases your chances that you're lucky and your data is correct, but you won't be able to tell. So... not really a reason not to do it, but it's probably kind of useless.

Kevin

#21 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

Quoted 91 lines · Kevin Wolf

Am 18.06.2019 um 09:19 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 12 lines
Quoted 10 lines · Kevin Wolf

The first step would be making the block drivers full replacements of the old things, which unfortunately isn't true today.

After your "deal with filters" series, we should be a lot closer for the core infrastructure at least.

Not sure about copy-on-read, but I know that throttle still doesn't have feature parity with -drive throttling. At least I'm pretty sure that something about changing the group or group options at runtime (and not just dropping the x-) was missing.

Quoted 6 lines

OK, thanks, I understand now that implicit filters are not just a feature but compatibility mechanism.

So, can we at some point deprecate "optionality" of filter-node-name parameters of jobs, in addition to deprecation of things like old copy-on-read option? And actually deprecate implicit filters by this?

Hm, I'm not sure if we have ever moved an optional feature to required, and how to communicate this to libvirt, but this would be ideal, yes.

Quoted 44 lines
Quoted 37 lines
Quoted 24 lines
Quoted 12 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 6 lines

But should really filter do that job, or it is here only to do CBW? Maybe target must have another parent which will care about flushing.

Ok, I think I'll flush target here too for safety, and leave a comment, that something smarter would be better.
(or, if we decide to flush all children by default in generic code, I'll drop this handler)

[1] Thinking more about this, for normal backups the target file does not reflect a valid disk state until the backup is done; just like for qemu-img convert. Just like qemu-img convert, there is therefore no reason to flush the target until the job is done.

Quoted 7 lines

Depends, the target could have the source as its backing file (like fleecing, but without exporting it right now), and then you could always have a consistent view on the target. Well, almost.

Almost because to guarantee this, we'd have to flush between each CBW and the corresponding write to the same block, to make sure that the old data is backed up before it is overwritten.

Yes, that’s what I meant by “enforce on the host that the target is always flushed before the source”. Well, I meant to say there is no good way of doing that, and I kind of don’t consider this a good way.

Quoted 7 lines
Quoted 3 lines

Of course, this would perform about as badly as internal COW in qcow2... So probably not a guarantee we should be making by default. But it might make sense as an option.

I don’t know. “Use this option so your data isn’t lost in case of a power failure, but it makes everything pretty slow”? Who is going to do explicitly enable that (before their first data loss)?

Maybe the backup job wasn't that clever after all. At least if you care about keeping the point-in-time snapshot at the start of the backup job instead of just retrying and getting a snapshot of a different point in time that is just as good.

Quoted 6 lines

If you do care about the specific point in time, the safe way to do it is to take a snapshot and copy that away, and then delete the snapshot again.

The only problem is that merging external snapshots is slow and you end up writing the new data twice. If only we had a COW image format... :-)

Quoted 7 lines

So, I still don't understand the reason of flushing backup target in a meantime. Backup target remains invalid until the successful end of the job, at which point we definitely flush target, but only once.

If node crashes during backup, backup would be invalid independently of were there flushes after each write (or better just enable O_DIRECT) or not.

Say your VM is running on disk.qcow2 and you use a backup job to copy data to backup.qcow2. At some point, the host crashes. If we made sure that every CBW to backup.qcow2 has completed before we write new data to disk.qcow2, backup.qcow2 contains valid data that represents the state at the start of the backup job as long as you use disk.qcow2 as its backing file.

The only way to ensure the right order is flushing between the two operations. If you don't do that, then yes, your backing is invalid before it has completed.

O_DIRECT doesn't guarantee that the data is on disk, only a flush does that. Maybe your chances that things actually make it to the disk in case of a host crash are higher if it sits in the disk cache rather than in the host's RAM, but there is no guarantee without a flush.

Quoted 5 lines

Now flushing the target when the guest flushes its disk doesn't give you new guarantees. Maybe it increases your chances that you're lucky and your data is correct, but you won't be able to tell. So... not really a reason not to do it, but it's probably kind of useless.

All clear now, thanks!

Signature · 2 lines
Best regards,
Vladimir
#22 lore raw

Re: [Qemu-devel] [PATCH v8 4/7] block: introduce backup-top filter driver

In reply to Max Reitz
Quoted 217 lines · Max Reitz
Quoted 84 lines · Vladimir Sementsov-Ogievskiy
Quoted 83 lines · Max Reitz
Quoted 63 lines · Vladimir Sementsov-Ogievskiy
Quoted 62 lines · Max Reitz
Quoted 37 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 35 lines · Vladimir Sementsov-Ogievskiy

Backup-top filter does copy-before-write operation. It should be inserted above active disk and has a target node for CBW, like the following:

        +-------+
        | Guest |
        +-------+
            |r,w
            v
        +------------+  target   +---------------+
        | backup_top |---------->| target(qcow2) |
        +------------+   CBW     +---------------+
            |
backing |r,w
            v
        +-------------+
        | Active disk |
        +-------------+

The driver will be used in backup instead of write-notifiers.

---
     block/backup-top.h  |  64 +++++++++
     block/backup-top.c  | 322 ++++++++++++++++++++++++++++++++++++++++++++
     block/Makefile.objs |   2 +
     3 files changed, 388 insertions(+)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

block/backup-top.h new file +0 -0
new file mode 100644
index 0000000000..788e18c358

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

+/*
+ * bdrv_backup_top_append
+ *
+ * Append backup_top filter node above @source node. @target node will receive
+ * the data backed up during CBE operations. New filter together with @target
+ * node are attached to @source aio context.
+ *
+ * The resulting filter node is implicit.

Quoted 6 lines
Quoted 2 lines

Why? It’s just as easy for the caller to just make it implicit if it should be. (And usually the caller should decide that.)

Personally, I don't know what are the reasons for filters to bi implicit or not, I just made it like other job-filters.. I can move making-implicit to the caller or drop it at all (if it will work).

Nodes are implicit if they haven’t been added consciously by the user. A node added by a block job can be non-implicit, too, as mirror shows; If the user specifies the filter-node-name option, they will know about the node, thus it is no longer implicit.

Quoted 8 lines
Quoted 3 lines

If the user doesn’t know about the node (they didn’t give the filter-node-name option), the node is implicit.

Ok, I understand it. But it doesn't show, why it should be implicit? Isn't it less buggy to make all filters explicit? We don't hide implicit nodes from query-named-block-nodes (the only interface to explore the whole graph for now) anyway. And we can't absolutely hide side effects of additional node in the graph.

Well, we try, at least. At least we hide them from query-block.

Quoted 3 lines

So, is there any real benefit of supporting separation into implicit and explicit filters? It seems for me that it only complicates things...
In other words, what will break if we make all filters explicit?

The theory is that qemu may decide to add nodes at any point, but at least when managing chains etc., they may not be visible to the user. I don’t think we can get rid of them so easily.

Quoted 8 lines
Quoted 5 lines

One example that isn’t implemented yet is copy-on-read. In theory, specifying copy-on-read=on for -drive should create an implicit COR node on top. The user shouldn’t see that node when inspecting the drive or when issuing block jobs on it, etc. And this node has to stay there when the user does e.g. an active commit somewhere down the chain.

Why instead not to just write in doc that yes, filter is created when you enable copy-on-read?

Because the problem is that existing users are not aware of that.

If they were, they could simply create a COR filter already.

I suppose we could interpret the deprecation policy in a way that we could change the behavior of -drive copy-on-read=on, but as I already said, what’s the point of supporting -drive copy-on-read=on, when you can simply explicitly create a COR filter on top?

Quoted 3 lines

And do same for all operations which may create filter,
we don't have a lot of them? And add optional parameter to set node-name for created filter.

That optional parameter exists, and if it is given, the user shows that they are aware of the node. Hence the node becomes explicit then.

The problem is with legacy users that use an existing interface and suddenly the externally visible interface of qemu changes in a way that could break them. I suppose additional entries in query-named-block-nodes is not breakage. Maybe it is, that would be a bug then.

(If nobody has noticed so far, that may be because these legacy applications didn’t use query-named-block-nodes. But now maybe libvirt does, and so if we converted copy-on-read=on to a COR node, it would notice now and thus break. So just because our handling of implicit nodes is broken right now does not mean we can abandon the concept altogether.)

Quoted 11 lines
Quoted 8 lines

That sounds like a horrible ordeal to implement, so it hasn’t been done yet. Maybe it never will. It isn’t that bad for the job filters, because they generally freeze the block graph, so there is no problem with potential modifications.

All in all I do think having implicit nodes makes sense. Maybe not so much now, but in the future (if someone implements converting -drive COR and throttle options to implicit nodes...).

But do we have at least strong definition of how implicit nodes should behave on any graph-modifying operations around them?

No. Depends on the node.

Quoted 2 lines

Should new implicit/explicit
filters be created above or under them?

That was always the most difficult question we had when we introduced filters.

The problem is that we never answered it in our code base.

One day, we just added filters (“let’s see what happens”), and in the beginning, they were all explicit, so we still didn’t have to answer this question (in code). Then job filters were added, and we still didn’t have to, because they set blockers so the graph couldn’t be reorganized with them in place anyway.

If we converted copy-on-read=on to a COR node, we would have to answer that question.

Quoted 4 lines

We should specify it in doc about all
such operations, otherwise effect of implicit nodes may change unpredictably for the user. I'm afraid, that implicit filters will just continue my list of features which should not be used if we want to be able to maintain all this mess..

Yeah, well, that is nothing new at all. For example, note the “Split I/O throttling off into own BDS – Requires some care with snapshots etc.” here:
https://wiki.qemu.org/ToDo/Block#Dynamic_graph_reconfiguration_.28e.g._adding_block_filters.2C_taking_snapshots.2C_etc..29

This was always a problem. It was always the biggest problem with filters. We never answered it because it never become an acute problem, as I described above.

We just said “For explicit filters, they just stay where they are, and that’s OK because the user can take care of them.” We never did add the implicit filters that were actually going to become a problem (COR and throttle).

So as I said above, the behavior of implicit nodes needs to be examined on a case-by-case basis. Well, actually, this is only about COR and throttle, really, and both would behave the same: Always stay at the BlockBackend, because that is the behavior when you currently use BB-level throttling or COR.

The jobs that now use implicit nodes freeze the graph, so there is no problem there.

(Also, by the way, we do not need the option to give COR/throttle nodes created through BB options a node name. As I said above, at that point you can just create the node yourself.)

Quoted 2 lines

1. If you something around filenames don't work, use node-names, and better don't use filename-based APIs

I think we agree that filename-based APIs were a mistake. (Though only in hindsight. They were added when there where no node names, so it did make sense then.)

So now you should not use them, correct.

Quoted 2 lines

2. If you qemu-img can't do what you need, use paused qemu process, and better, don't use qemu-img

There was always a plan to give qemu-img feature parity to qemu proper.
But often the interface is just a problem. Node names really don’t
make much sense with qemu-img, I think.

(I mean, feel free to add something that automatically names the backing nodes e.g. backing.0, backing.1, etc., and then add an interface to qemu-img commit to use node names, but I find that weird.)

So I don’t quite see the problem. Of course qemu has more functionality than qemu-img. qemu-img is fine as long as you have a simple set-up, but once it gets more complicated, you need management software; and that software shouldn’t care much whether it uses qemu-img or qemu itself.

Quoted 5 lines

and a new one:

3. If something is broken around jobs and filters and any block operations, set filter-node-name in parameters to make filter node explicit. And better, always set it..

Quoted 5 lines

Uh, yeah? I don’t see the problem here.

This is like saying “If I want to use new features, I should use -blockdev and not -hda”. Yes, that is correct. You should set filter-node-name if you care about the exact state of the graph.

OK, yes, understand now. Sorry for me listing quite unrelated things, but the picture is clearer for me now, thanks.

Quoted 7 lines
Quoted 3 lines

So at least, I think we should make it possible for all filters to be explicit if user wants, by setting its name like in mirror.

For all filters that the user has no other way of creating them, yes. (That is, the filters created by the block jobs.)

Quoted 8 lines
Quoted 6 lines

(and then, I will not really care of implicit-filters related logic, like I don't really care about filename-api related logic)

Also, now, implict filters are actually available for direct manipulation by user, as their node-names are exported in query-named-block-nodes and nothing prevents using these names in differet APIs.

Hm, yes. Is that a problem?

Not. I tried to prove that implicit filters are "bad feature", but now when I see that it's for compatibility, these reasons become weak.

[..]

Signature · 2 lines
Best regards,
Vladimir
#23 lore raw

PATCH v8 5/7 block/io: refactor wait_serialising_requests

In reply to Vladimir Sementsov-Ogievskiy

Split out do_wait_serialising_requests with additional possibility to not actually wait but just check, that there is something to wait for.

---
 block/io.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)
block/io.c +16 -8
index 3134a60a48..c347f90722 100644
@@ -720,12 +720,13 @@ void bdrv_dec_in_flight(BlockDriverState *bs)
     bdrv_wakeup(bs);
 }
 
-static bool coroutine_fn wait_serialising_requests(BdrvTrackedRequest *self)
+static bool coroutine_fn do_wait_serialising_requests(BdrvTrackedRequest *self,
+                                                      bool wait)
 {
     BlockDriverState *bs = self->bs;
     BdrvTrackedRequest *req;
     bool retry;
-    bool waited = false;
+    bool found = false;
 
     if (!atomic_read(&bs->serialising_in_flight)) {
         return false;
@@ -751,11 +752,13 @@ static bool coroutine_fn wait_serialising_requests(BdrvTrackedRequest *self)
                  * will wait for us as soon as it wakes up, then just go on
                  * (instead of producing a deadlock in the former case). */
                 if (!req->waiting_for) {
-                    self->waiting_for = req;
-                    qemu_co_queue_wait(&req->wait_queue, &bs->reqs_lock);
-                    self->waiting_for = NULL;
-                    retry = true;
-                    waited = true;
+                    found = true;
+                    if (wait) {
+                        self->waiting_for = req;
+                        qemu_co_queue_wait(&req->wait_queue, &bs->reqs_lock);
+                        self->waiting_for = NULL;
+                        retry = true;
+                    }
                     break;
                 }
             }
@@ -763,7 +766,12 @@ static bool coroutine_fn wait_serialising_requests(BdrvTrackedRequest *self)
         qemu_co_mutex_unlock(&bs->reqs_lock);
     } while (retry);
 
-    return waited;
+    return found;
+}
+
+static bool coroutine_fn wait_serialising_requests(BdrvTrackedRequest *self)
+{
+    return do_wait_serialising_requests(self, true);
 }
 
 static int bdrv_check_byte_request(BlockDriverState *bs, int64_t offset,
Signature · 1 line
2.18.0
#24 lore raw

PATCH v8 6/7 block: add lock/unlock range functions

In reply to Vladimir Sementsov-Ogievskiy

From: Vladimir Sementsov-Ogievskiy <[email protected]>

Introduce lock/unlock range functionality, based on serialized requests. This is needed to refactor backup, dropping local tracked-request-like synchronization.

---
 include/block/block_int.h |  4 ++++
 block/io.c                | 44 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 47 insertions(+), 1 deletion(-)
include/block/block_int.h +4 -0
index 1eebc7c8f3..9d9d4346e9 100644
@@ -66,6 +66,7 @@ enum BdrvTrackedRequestType {
     BDRV_TRACKED_WRITE,
     BDRV_TRACKED_DISCARD,
     BDRV_TRACKED_TRUNCATE,
+    BDRV_TRACKED_LOCK,
 };
 
 typedef struct BdrvTrackedRequest {
@@ -928,6 +929,9 @@ int coroutine_fn bdrv_co_preadv(BdrvChild *child,
 int coroutine_fn bdrv_co_pwritev(BdrvChild *child,
     int64_t offset, unsigned int bytes, QEMUIOVector *qiov,
     BdrvRequestFlags flags);
+void *coroutine_fn bdrv_co_try_lock(BlockDriverState *bs,
+                                    int64_t offset, unsigned int bytes);
+void coroutine_fn bdrv_co_unlock(void *opaque);
 
 static inline int coroutine_fn bdrv_co_pread(BdrvChild *child,
     int64_t offset, unsigned int bytes, void *buf, BdrvRequestFlags flags)
block/io.c +43 -1

65 lines not on this page · raw message

Signature · 1 line
2.18.0
Reviewed-by on v8 6/7
#25 lore raw

Re: [Qemu-devel] [PATCH v8 6/7] block: add lock/unlock range functions

Quoted 11 lines · Vladimir Sementsov-Ogievskiy

From: Vladimir Sementsov-Ogievskiy <[email protected]>

Introduce lock/unlock range functionality, based on serialized requests. This is needed to refactor backup, dropping local tracked-request-like synchronization.

---
 include/block/block_int.h |  4 ++++
 block/io.c                | 44 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 47 insertions(+), 1 deletion(-)
#26 lore raw

PATCH v8 7/7 block/backup: use backup-top instead of write notifiers

In reply to Vladimir Sementsov-Ogievskiy

Drop write notifiers and use filter node instead. Changes:

1. copy-before-writes now handled by filter node, so, drop all is_write_notifier arguments.

2. we don't have intersecting requests, so their handling is dropped. Instead, synchronization works as follows:
when backup or backup-top starts copying of some area it firstly clears copy-bitmap bits, and nobody touches areas, not marked with dirty bits in copy-bitmap, so there is no intersection. Also, backup job copy operations are surrounded by bdrv region lock, which is actually serializing request, to not interfere with guest writes and not read changed data from source (before reading we clear corresponding bit in copy-bitmap, so, this area is not more handled by backup-top).

3. To sync with in-flight requests at job finish we now have drained removing of the filter, we don't need rw-lock.

== RFC part ==

iotests changed:
56: op-blocker doesn't shot now, as we set it on source, but then check on filter, when trying to start second backup... Should I workaround it somehow?

129: Hmm, now it is not busy at this moment.. But it's illegal to check busy, as job has pause-points and set busy to false in these points. Why we assert it in this test?

141: Obvious, as drv0 is not root node now, but backing of the filter, when we try to remove it.

---
 block/backup.c             | 171 ++++++++++++++-----------------------
 tests/qemu-iotests/056     |   2 +-
 tests/qemu-iotests/129     |   1 -
 tests/qemu-iotests/141.out |   2 +-
 4 files changed, 68 insertions(+), 108 deletions(-)
block/backup.c +66 -105
index 00f4f8af53..a5b8e04c9c 100644
@@ -2,6 +2,7 @@
  * QEMU backup
  *
  * Copyright (C) 2013 Proxmox Server Solutions
+ * Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved.
  *
  * Authors:
  *  Dietmar Maurer ([email protected])
@@ -26,14 +27,9 @@
 #include "qemu/bitmap.h"
 #include "qemu/error-report.h"
 
-#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
+#include "block/backup-top.h"
 
-typedef struct CowRequest {
-    int64_t start_byte;
-    int64_t end_byte;
-    QLIST_ENTRY(CowRequest) list;
-    CoQueue wait_queue; /* coroutines blocked on this request */
-} CowRequest;
+#define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
 
 typedef struct BackupBlockJob {
     BlockJob common;
@@ -43,13 +39,10 @@ typedef struct BackupBlockJob {
     MirrorSyncMode sync_mode;
     BlockdevOnError on_source_error;
     BlockdevOnError on_target_error;
-    CoRwlock flush_rwlock;
     uint64_t len;
     uint64_t bytes_read;
     int64_t cluster_size;
     bool compress;
-    NotifierWithReturn before_write;
-    QLIST_HEAD(, CowRequest) inflight_reqs;
 
     HBitmap *copy_bitmap;
     bool use_copy_range;
@@ -60,56 +53,17 @@ typedef struct BackupBlockJob {
 
 static const BlockJobDriver backup_job_driver;
 
-/* See if in-flight requests overlap and wait for them to complete */
-static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job,
-                                                       int64_t start,
-                                                       int64_t end)
-{
-    CowRequest *req;
-    bool retry;
-
-    do {
-        retry = false;
-        QLIST_FOREACH(req, &job->inflight_reqs, list) {
-            if (end > req->start_byte && start < req->end_byte) {
-                qemu_co_queue_wait(&req->wait_queue, NULL);
-                retry = true;
-                break;
-            }
-        }
-    } while (retry);
-}
-
-/* Keep track of an in-flight request */
-static void cow_request_begin(CowRequest *req, BackupBlockJob *job,
-                              int64_t start, int64_t end)
-{
-    req->start_byte = start;
-    req->end_byte = end;
-    qemu_co_queue_init(&req->wait_queue);
-    QLIST_INSERT_HEAD(&job->inflight_reqs, req, list);
-}
-
-/* Forget about a completed request */
-static void cow_request_end(CowRequest *req)
-{
-    QLIST_REMOVE(req, list);
-    qemu_co_queue_restart_all(&req->wait_queue);
-}
-
 /* Copy range to target with a bounce buffer and return the bytes copied. If
  * error occurred, return a negative error number */
 static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job,
                                                       int64_t start,
                                                       int64_t end,
-                                                      bool is_write_notifier,
                                                       bool *error_is_read,
                                                       void **bounce_buffer)
 {
     int ret;
     BlockBackend *blk = job->common.blk;
     int nbytes;
-    int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0;
     int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
     assert(QEMU_IS_ALIGNED(start, job->cluster_size));
@@ -119,7 +73,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job,
         *bounce_buffer = blk_blockalign(blk, job->cluster_size);
     }
 
-    ret = blk_co_pread(blk, start, nbytes, *bounce_buffer, read_flags);
+    ret = blk_co_pread(blk, start, nbytes, *bounce_buffer, 0);
     if (ret < 0) {
         trace_backup_do_cow_read_fail(job, start, ret);
         if (error_is_read) {
@@ -154,15 +108,12 @@ fail:
 /* Copy range to target and return the bytes copied. If error occurred, return a
  * negative error number. */
 static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job,
-                                                int64_t start,
-                                                int64_t end,
-                                                bool is_write_notifier)
+                                                int64_t start, int64_t end)
 {
     int ret;
     int nr_clusters;
     BlockBackend *blk = job->common.blk;
     int nbytes;
-    int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0;
     int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
     assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size));
@@ -171,7 +122,7 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job,
     nr_clusters = DIV_ROUND_UP(nbytes, job->cluster_size);
     hbitmap_reset(job->copy_bitmap, start, job->cluster_size * nr_clusters);
     ret = blk_co_copy_range(blk, start, job->target, start, nbytes,
-                            read_flags, write_flags);
+                            0, write_flags);
     if (ret < 0) {
         trace_backup_do_cow_copy_range_fail(job, start, ret);
         hbitmap_set(job->copy_bitmap, start, job->cluster_size * nr_clusters);
@@ -183,24 +134,17 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job,
 
 static int coroutine_fn backup_do_cow(BackupBlockJob *job,
                                       int64_t offset, uint64_t bytes,
-                                      bool *error_is_read,
-                                      bool is_write_notifier)
+                                      bool *error_is_read)
 {
-    CowRequest cow_request;
     int ret = 0;
     int64_t start, end; /* bytes */
     void *bounce_buffer = NULL;
 
-    qemu_co_rwlock_rdlock(&job->flush_rwlock);
-
     start = QEMU_ALIGN_DOWN(offset, job->cluster_size);
     end = QEMU_ALIGN_UP(bytes + offset, job->cluster_size);
 
     trace_backup_do_cow_enter(job, start, offset, bytes);
 
-    wait_for_overlapping_requests(job, start, end);
-    cow_request_begin(&cow_request, job, start, end);
-
     while (start < end) {
         if (!hbitmap_get(job->copy_bitmap, start)) {
             trace_backup_do_cow_skip(job, start);
@@ -211,13 +155,13 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
         trace_backup_do_cow_process(job, start);
 
         if (job->use_copy_range) {
-            ret = backup_cow_with_offload(job, start, end, is_write_notifier);
+            ret = backup_cow_with_offload(job, start, end);
             if (ret < 0) {
                 job->use_copy_range = false;
             }
         }
         if (!job->use_copy_range) {
-            ret = backup_cow_with_bounce_buffer(job, start, end, is_write_notifier,
+            ret = backup_cow_with_bounce_buffer(job, start, end,
                                                 error_is_read, &bounce_buffer);
         }
         if (ret < 0) {
@@ -237,29 +181,11 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
         qemu_vfree(bounce_buffer);
     }
 
-    cow_request_end(&cow_request);
-
     trace_backup_do_cow_return(job, offset, bytes, ret);
 
-    qemu_co_rwlock_unlock(&job->flush_rwlock);
-
     return ret;
 }
 
-static int coroutine_fn backup_before_write_notify(
-        NotifierWithReturn *notifier,
-        void *opaque)
-{
-    BackupBlockJob *job = container_of(notifier, BackupBlockJob, before_write);
-    BdrvTrackedRequest *req = opaque;
-
-    assert(req->bs == blk_bs(job->common.blk));
-    assert(QEMU_IS_ALIGNED(req->offset, BDRV_SECTOR_SIZE));
-    assert(QEMU_IS_ALIGNED(req->bytes, BDRV_SECTOR_SIZE));
-
-    return backup_do_cow(job, req->offset, req->bytes, NULL, true);
-}
-
 static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
 {
     BdrvDirtyBitmap *bm;
@@ -295,14 +221,30 @@ static void backup_abort(Job *job)
 static void backup_clean(Job *job)
 {
     BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
+    BlockJob *bjob = &s->common;
+    BlockDriverState *backup_top = blk_bs(s->common.blk);
+    BlockDriverState *src = backup_top->backing->bs;
+
     assert(s->target);
     blk_unref(s->target);
     s->target = NULL;
 
+    bdrv_ref(backup_top);
+    bdrv_ref(src);
+
     if (s->copy_bitmap) {
         hbitmap_free(s->copy_bitmap);
         s->copy_bitmap = NULL;
     }
+
+    bdrv_backup_top_drop(backup_top);
+
+    blk_remove_bs(bjob->blk);
+    blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort);
+    blk_insert_bs(bjob->blk, backup_top, &error_abort);
+
+    bdrv_unref(src);
+    bdrv_unref(backup_top);
 }
 
 void backup_do_checkpoint(BlockJob *job, Error **errp)
@@ -391,28 +333,41 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
     int64_t offset;
     HBitmapIter hbi;
     BlockDriverState *bs = blk_bs(job->common.blk);
+    void *lock;
 
     hbitmap_iter_init(&hbi, job->copy_bitmap, 0);
     while ((offset = hbitmap_iter_next(&hbi)) != -1) {
+        lock = bdrv_co_try_lock(backing_bs(blk_bs(job->common.blk)), offset,
+                                job->cluster_size);
+        /*
+         * Dirty bit is set, which means that there are no in-flight
+         * write requests on this area. We must succeed.
+         */
+        assert(lock);
+
         if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
             bdrv_is_unallocated_range(bs, offset, job->cluster_size))
         {
             hbitmap_reset(job->copy_bitmap, offset, job->cluster_size);
+            bdrv_co_unlock(lock);
             continue;
         }
 
         do {
             if (yield_and_check(job)) {
+                bdrv_co_unlock(lock);
                 return 0;
             }
-            ret = backup_do_cow(job, offset,
-                                job->cluster_size, &error_is_read, false);
+            ret = backup_do_cow(job, offset, job->cluster_size, &error_is_read);
             if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
                            BLOCK_ERROR_ACTION_REPORT)
             {
+                bdrv_co_unlock(lock);
                 return ret;
             }
         } while (ret < 0);
+
+        bdrv_co_unlock(lock);
     }
 
     return 0;
@@ -444,12 +399,8 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job)
 static int coroutine_fn backup_run(Job *job, Error **errp)
 {
     BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
-    BlockDriverState *bs = blk_bs(s->common.blk);
     int ret = 0;
 
-    QLIST_INIT(&s->inflight_reqs);
-    qemu_co_rwlock_init(&s->flush_rwlock);
-
     job_progress_set_remaining(job, s->len);
 
     if (s->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) {
@@ -458,27 +409,20 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
         hbitmap_set(s->copy_bitmap, 0, s->len);
     }
 
-    s->before_write.notify = backup_before_write_notify;
-    bdrv_add_before_write_notifier(bs, &s->before_write);
-
     if (s->sync_mode == MIRROR_SYNC_MODE_NONE) {
         /* All bits are set in copy_bitmap to allow any cluster to be copied.
          * This does not actually require them to be copied. */
         while (!job_is_cancelled(job)) {
-            /* Yield until the job is cancelled.  We just let our before_write
-             * notify callback service CoW requests. */
+            /*
+             * Yield until the job is cancelled.  We just let our backup-top
+             * filter driver service CbW requests.
+             */
             job_yield(job);
         }
     } else {
         ret = backup_loop(s);
     }
 
-    notifier_with_return_remove(&s->before_write);
-
-    /* wait until pending backup_do_cow() calls have completed */
-    qemu_co_rwlock_wrlock(&s->flush_rwlock);
-    qemu_co_rwlock_unlock(&s->flush_rwlock);
-
     return ret;
 }
 
@@ -533,6 +477,11 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
     return MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size);
 }
 
+static void backup_top_progress(uint64_t done, void *opaque)
+{
+    job_progress_update((Job *)opaque, done);
+}
+
 BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   BlockDriverState *target, int64_t speed,
                   MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
@@ -548,6 +497,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     int ret;
     int64_t cluster_size;
     HBitmap *copy_bitmap = NULL;
+    BlockDriverState *backup_top = NULL;
 
     assert(bs);
     assert(target);
@@ -616,8 +566,13 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
 
     copy_bitmap = hbitmap_alloc(len, ctz32(cluster_size));
 
+    backup_top = bdrv_backup_top_append(bs, target, copy_bitmap, errp);
+    if (!backup_top) {
+        goto error;
+    }
+
     /* job->len is fixed, so we can't allow resize */
-    job = block_job_create(job_id, &backup_job_driver, txn, bs,
+    job = block_job_create(job_id, &backup_job_driver, txn, backup_top,
                            BLK_PERM_CONSISTENT_READ,
                            BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE |
                            BLK_PERM_WRITE_UNCHANGED | BLK_PERM_GRAPH_MOD,
@@ -626,6 +581,9 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         goto error;
     }
 
+    bdrv_backup_top_set_progress_callback(backup_top, backup_top_progress,
+                                          &job->common.job);
+
     /* The target must match the source in size, so no resize here either */
     job->target = blk_new(BLK_PERM_WRITE,
                           BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE |
@@ -662,10 +620,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     return &job->common;
 
  error:
-    if (copy_bitmap) {
-        assert(!job || !job->copy_bitmap);
-        hbitmap_free(copy_bitmap);
-    }
     if (sync_bitmap) {
         bdrv_reclaim_dirty_bitmap(bs, sync_bitmap, NULL);
     }
@@ -673,6 +627,13 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         backup_clean(&job->common.job);
         job_early_fail(&job->common.job);
     }
+    if (backup_top) {
+        bdrv_backup_top_drop(backup_top);
+    }
+    if (copy_bitmap) {
+        assert(!job || !job->copy_bitmap);
+        hbitmap_free(copy_bitmap);
+    }
 
     return NULL;
 }
tests/qemu-iotests/056 +1 -1
index f40fc11a09..d3e6fe4b11 100755
@@ -214,7 +214,7 @@ class BackupTest(iotests.QMPTestCase):
         res = self.vm.qmp('query-block-jobs')
         self.assert_qmp(res, 'return[0]/status', 'concluded')
         # Leave zombie job un-dismissed, observe a failure:
-        res = self.qmp_backup_and_wait(serror="Node 'drive0' is busy: block device is in use by block job: backup",
+        res = self.qmp_backup_and_wait(serror='Failed to get "write" lock',
                                        device='drive0', format=iotests.imgfmt,
                                        sync='full', target=self.dest_img,
                                        auto_dismiss=False)
tests/qemu-iotests/129 +0 -1
index 9e87e1c8d9..d719492deb 100755
@@ -66,7 +66,6 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
         result = self.vm.qmp("stop")
         self.assert_qmp(result, 'return', {})
         result = self.vm.qmp("query-block-jobs")
-        self.assert_qmp(result, 'return[0]/busy', True)
         self.assert_qmp(result, 'return[0]/ready', False)
 
     def test_drive_mirror(self):
tests/qemu-iotests/141.out +1 -1
index 41c7291258..a2645bea1a 100644
@@ -11,7 +11,7 @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}}
 {"return": {}}
-{"error": {"class": "GenericError", "desc": "Node drv0 is in use"}}
+{"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is used as backing hd of 'NODE_NAME'"}}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 0, "speed": 0, "type": "backup"}}
Signature · 1 line
2.18.0
#27 lore raw

Re: [Qemu-devel] [PATCH v8 7/7] block/backup: use backup-top instead of write notifiers

Quoted 19 lines · Vladimir Sementsov-Ogievskiy

Drop write notifiers and use filter node instead. Changes:

1. copy-before-writes now handled by filter node, so, drop all is_write_notifier arguments.

2. we don't have intersecting requests, so their handling is dropped. Instead, synchronization works as follows:
when backup or backup-top starts copying of some area it firstly clears copy-bitmap bits, and nobody touches areas, not marked with dirty bits in copy-bitmap, so there is no intersection. Also, backup job copy operations are surrounded by bdrv region lock, which is actually serializing request, to not interfere with guest writes and not read changed data from source (before reading we clear corresponding bit in copy-bitmap, so, this area is not more handled by backup-top).

3. To sync with in-flight requests at job finish we now have drained removing of the filter, we don't need rw-lock.

Quoted 6 lines

== RFC part ==

iotests changed:
56: op-blocker doesn't shot now, as we set it on source, but then check on filter, when trying to start second backup... Should I workaround it somehow?

Hm. Where does that error message even come from? The fact that the target image is in use already (Due to file locks)?

It appears that way indeed.

It seems reasonable to me that you can now run a backup on top of another backup. Well, I mean, it is a stupid thing to do, but I don’t see why the block layer would forbid doing so.

So the test seems superfluous to me. If we want to keep it (why not), it should test the opposite, namely that a backup to a different image (with a different job ID) works. (It seems simple enough to modify the job that way, so why not.)

Quoted 3 lines

129: Hmm, now it is not busy at this moment.. But it's illegal to check busy, as job has pause-points and set busy to false in these points. Why we assert it in this test?

Nobody knows, it’s probably wrong. All I know is that 129 is just broken anyway.

Quoted 2 lines

141: Obvious, as drv0 is not root node now, but backing of the filter, when we try to remove it.

I get a failed assertion in 256. That is probably because the bdrv_set_aio_context() calls weren’t as unnecessary as I deemed them to be.

Quoted 7 lines
---
 block/backup.c             | 171 ++++++++++++++-----------------------
 tests/qemu-iotests/056     |   2 +-
 tests/qemu-iotests/129     |   1 -
 tests/qemu-iotests/141.out |   2 +-
 4 files changed, 68 insertions(+), 108 deletions(-)

For some reason, my gcc starts to complain that backup_loop() may not initialize error_is_read after this patch. I don’t know why that is. Perhaps it inlines backup_do_cow() now? (So before it just saw that a pointer to error_is_read was passed to backup_do_cow() and took it as an opaque function, so it surely would set this value somewhere. Now it inlines it and it can’t find whether that will definitely happen, so it complains.)

I don’t think it is strictly necessary to initialize error_is_read, but, well, it won’t hurt.

Quoted 4 lines
block/backup.c +0 -0
index 00f4f8af53..a5b8e04c9c 100644

[...]

Quoted 39 lines
block/backup.c +0 -35
@@ -60,56 +53,17 @@ typedef struct BackupBlockJob {
 
 static const BlockJobDriver backup_job_driver;
 
-/* See if in-flight requests overlap and wait for them to complete */
-static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job,
-                                                       int64_t start,
-                                                       int64_t end)
-{
-    CowRequest *req;
-    bool retry;
-
-    do {
-        retry = false;
-        QLIST_FOREACH(req, &job->inflight_reqs, list) {
-            if (end > req->start_byte && start < req->end_byte) {
-                qemu_co_queue_wait(&req->wait_queue, NULL);
-                retry = true;
-                break;
-            }
-        }
-    } while (retry);
-}
-
-/* Keep track of an in-flight request */
-static void cow_request_begin(CowRequest *req, BackupBlockJob *job,
-                              int64_t start, int64_t end)
-{
-    req->start_byte = start;
-    req->end_byte = end;
-    qemu_co_queue_init(&req->wait_queue);
-    QLIST_INSERT_HEAD(&job->inflight_reqs, req, list);
-}
-
-/* Forget about a completed request */
-static void cow_request_end(CowRequest *req)
-{
-    QLIST_REMOVE(req, list);
-    qemu_co_queue_restart_all(&req->wait_queue);

block/backup.c typedef struct BackupBlockJob { · line 62

Quoted 10 lines
block/backup.c +0 -3
-}
-
 /* Copy range to target with a bounce buffer and return the bytes copied. If
  * error occurred, return a negative error number */
 static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job,
                                                       int64_t start,
                                                       int64_t end,
-                                                      bool is_write_notifier,
                                                       bool *error_is_read,
                                                       void **bounce_buffer)

Future feature: Somehow get this functionality done with backup-top, I suppose. (This is effectively just backup_top_cbw() with some bells and whistles, isn’t it?)

Quoted 8 lines
block/backup.c +0 -1
 {
     int ret;
     BlockBackend *blk = job->common.blk;
     int nbytes;
-    int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0;
     int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
     assert(QEMU_IS_ALIGNED(start, job->cluster_size));

[...]

block/backup.c fail: · line 111

Quoted 8 lines
block/backup.c +1 -3
@@ -154,15 +108,12 @@ fail:
 /* Copy range to target and return the bytes copied. If error occurred, return a
  * negative error number. */
 static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job,
-                                                int64_t start,
-                                                int64_t end,
-                                                bool is_write_notifier)
+                                                int64_t start, int64_t end)

And I suppose this is something backup-top maybe should support, too.

Quoted 9 lines
block/backup.c +0 -1
 {
     int ret;
     int nr_clusters;
     BlockBackend *blk = job->common.blk;
     int nbytes;
-    int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0;
     int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
     assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size));

[...]

Quoted 6 lines
block/backup.c +1 -0
@@ -391,28 +333,41 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
     int64_t offset;
     HBitmapIter hbi;
     BlockDriverState *bs = blk_bs(job->common.blk);
+    void *lock;
 

block/backup.c static int coroutine_fn backup_loop(BackupBlockJob *job) · line 347

Quoted 10 lines
block/backup.c +8 -0
     hbitmap_iter_init(&hbi, job->copy_bitmap, 0);
     while ((offset = hbitmap_iter_next(&hbi)) != -1) {
+        lock = bdrv_co_try_lock(backing_bs(blk_bs(job->common.blk)), offset,
+                                job->cluster_size);
+        /*
+         * Dirty bit is set, which means that there are no in-flight
+         * write requests on this area. We must succeed.
+         */
+        assert(lock);
+

Hm. It makes me uneasy but I suppose you’re right.

block/backup.c static int coroutine_fn backup_loop(BackupBlockJob *job) · line 349

Quoted 2 lines
block/backup.c +0 -0
         if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
             bdrv_is_unallocated_range(bs, offset, job->cluster_size))

This can yield, right? If it does, the bitmap is still set. backup-top will see this, unset the bitmap and try to start its CBW operation. That is halted by the lock just taken, but the progress will still be published after completion, so the job can go beyond 100 %, I think.

Even if it doesn’t, copying the data twice is weird. It may even get weirder if one of both requests fails.

Can we lock the backup-top node instead? I don’t know whether locking would always succeed there, though...

Max

Quoted 26 lines
block/backup.c +6 -2
         {
             hbitmap_reset(job->copy_bitmap, offset, job->cluster_size);
+            bdrv_co_unlock(lock);
             continue;
         }
 
         do {
             if (yield_and_check(job)) {
+                bdrv_co_unlock(lock);
                 return 0;
             }
-            ret = backup_do_cow(job, offset,
-                                job->cluster_size, &error_is_read, false);
+            ret = backup_do_cow(job, offset, job->cluster_size, &error_is_read);
             if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
                            BLOCK_ERROR_ACTION_REPORT)
             {
+                bdrv_co_unlock(lock);
                 return ret;
             }
         } while (ret < 0);
+
+        bdrv_co_unlock(lock);
     }
 
     return 0;
#28 lore raw

Re: [Qemu-devel] [PATCH v8 7/7] block/backup: use backup-top instead of write notifiers

Quoted 45 lines · Max Reitz
Quoted 19 lines · Vladimir Sementsov-Ogievskiy

Drop write notifiers and use filter node instead. Changes:

1. copy-before-writes now handled by filter node, so, drop all is_write_notifier arguments.

2. we don't have intersecting requests, so their handling is dropped. Instead, synchronization works as follows:
when backup or backup-top starts copying of some area it firstly clears copy-bitmap bits, and nobody touches areas, not marked with dirty bits in copy-bitmap, so there is no intersection. Also, backup job copy operations are surrounded by bdrv region lock, which is actually serializing request, to not interfere with guest writes and not read changed data from source (before reading we clear corresponding bit in copy-bitmap, so, this area is not more handled by backup-top).

3. To sync with in-flight requests at job finish we now have drained removing of the filter, we don't need rw-lock.

Quoted 6 lines

== RFC part ==

iotests changed:
56: op-blocker doesn't shot now, as we set it on source, but then check on filter, when trying to start second backup... Should I workaround it somehow?

Hm. Where does that error message even come from? The fact that the target image is in use already (Due to file locks)?

It appears that way indeed.

It seems reasonable to me that you can now run a backup on top of another backup. Well, I mean, it is a stupid thing to do, but I don’t see why the block layer would forbid doing so.

So the test seems superfluous to me. If we want to keep it (why not), it should test the opposite, namely that a backup to a different image (with a different job ID) works. (It seems simple enough to modify the job that way, so why not.)

Quoted 3 lines

129: Hmm, now it is not busy at this moment.. But it's illegal to check busy, as job has pause-points and set busy to false in these points. Why we assert it in this test?

Quoted 8 lines

Nobody knows, it’s probably wrong. All I know is that 129 is just broken anyway.

Quoted 2 lines

141: Obvious, as drv0 is not root node now, but backing of the filter, when we try to remove it.

I get a failed assertion in 256. That is probably because the bdrv_set_aio_context() calls weren’t as unnecessary as I deemed them to be.

hmm, will check.

Quoted 72 lines
Quoted 8 lines
---
  block/backup.c             | 171 ++++++++++++++-----------------------
  tests/qemu-iotests/056     |   2 +-
  tests/qemu-iotests/129     |   1 -
  tests/qemu-iotests/141.out |   2 +-
  4 files changed, 68 insertions(+), 108 deletions(-)

For some reason, my gcc starts to complain that backup_loop() may not initialize error_is_read after this patch. I don’t know why that is. Perhaps it inlines backup_do_cow() now? (So before it just saw that a pointer to error_is_read was passed to backup_do_cow() and took it as an opaque function, so it surely would set this value somewhere. Now it inlines it and it can’t find whether that will definitely happen, so it complains.)

I don’t think it is strictly necessary to initialize error_is_read, but, well, it won’t hurt.

Quoted 4 lines
block/backup.c +0 -0
index 00f4f8af53..a5b8e04c9c 100644

[...]

Quoted 45 lines
block/backup.c +0 -37
@@ -60,56 +53,17 @@ typedef struct BackupBlockJob {
 
  static const BlockJobDriver backup_job_driver;
 
-/* See if in-flight requests overlap and wait for them to complete */
-static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job,
-                                                       int64_t start,
-                                                       int64_t end)
-{
-    CowRequest *req;
-    bool retry;
-
-    do {
-        retry = false;
-        QLIST_FOREACH(req, &job->inflight_reqs, list) {
-            if (end > req->start_byte && start < req->end_byte) {
-                qemu_co_queue_wait(&req->wait_queue, NULL);
-                retry = true;
-                break;
-            }
-        }
-    } while (retry);
-}
-
-/* Keep track of an in-flight request */
-static void cow_request_begin(CowRequest *req, BackupBlockJob *job,
-                              int64_t start, int64_t end)
-{
-    req->start_byte = start;
-    req->end_byte = end;
-    qemu_co_queue_init(&req->wait_queue);
-    QLIST_INSERT_HEAD(&job->inflight_reqs, req, list);
-}
-
-/* Forget about a completed request */
-static void cow_request_end(CowRequest *req)
-{
-    QLIST_REMOVE(req, list);
-    qemu_co_queue_restart_all(&req->wait_queue);
-}
-
  /* Copy range to target with a bounce buffer and return the bytes copied. If
   * error occurred, return a negative error number */
  static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job,
                                                        int64_t start,
Quoted 8 lines

block/backup.c typedef struct BackupBlockJob { · line 62

Quoted 4 lines
block/backup.c +0 -1
                                                        int64_t end,
-                                                      bool is_write_notifier,
                                                        bool *error_is_read,
                                                        void **bounce_buffer)

Future feature: Somehow get this functionality done with backup-top, I suppose. (This is effectively just backup_top_cbw() with some bells and whistles, isn’t it?)

or may be separate it as bdrv_co_pcopy or something like this.

Quoted 62 lines
Quoted 9 lines
block/backup.c +0 -1
  {
      int ret;
      BlockBackend *blk = job->common.blk;
      int nbytes;
-    int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0;
      int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
      assert(QEMU_IS_ALIGNED(start, job->cluster_size));

[...]

block/backup.c fail: · line 111

Quoted 8 lines
block/backup.c +1 -3
@@ -154,15 +108,12 @@ fail:
  /* Copy range to target and return the bytes copied. If error occurred, return a
   * negative error number. */
  static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job,
-                                                int64_t start,
-                                                int64_t end,
-                                                bool is_write_notifier)
+                                                int64_t start, int64_t end)

And I suppose this is something backup-top maybe should support, too.

Quoted 9 lines
block/backup.c +0 -1
  {
      int ret;
      int nr_clusters;
      BlockBackend *blk = job->common.blk;
      int nbytes;
-    int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0;
      int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0;
 
      assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size));

[...]

Quoted 6 lines
block/backup.c +1 -0
@@ -391,28 +333,41 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
      int64_t offset;
      HBitmapIter hbi;
      BlockDriverState *bs = blk_bs(job->common.blk);
+    void *lock;
 

block/backup.c static int coroutine_fn backup_loop(BackupBlockJob *job) · line 347

Quoted 10 lines
block/backup.c +8 -0
      hbitmap_iter_init(&hbi, job->copy_bitmap, 0);
      while ((offset = hbitmap_iter_next(&hbi)) != -1) {
+        lock = bdrv_co_try_lock(backing_bs(blk_bs(job->common.blk)), offset,
+                                job->cluster_size);
+        /*
+         * Dirty bit is set, which means that there are no in-flight
+         * write requests on this area. We must succeed.
+         */
+        assert(lock);
+

Hm. It makes me uneasy but I suppose you’re right.

block/backup.c static int coroutine_fn backup_loop(BackupBlockJob *job) · line 349

Quoted 2 lines
block/backup.c +0 -0
          if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
              bdrv_is_unallocated_range(bs, offset, job->cluster_size))

This can yield, right? If it does, the bitmap is still set. backup-top will see this, unset the bitmap and try to start its CBW operation. That is halted by the lock just taken, but the progress will still be published after completion, so the job can go beyond 100 %, I think.

Quoted 6 lines

Even if it doesn’t, copying the data twice is weird. It may even get weirder if one of both requests fails.

Can we lock the backup-top node instead? I don’t know whether locking would always succeed there, though...

Hmm, I'll look closely at the code, but seems that we'd better reset bit before yield.

Quoted 28 lines
Quoted 27 lines
block/backup.c +6 -2
          {
              hbitmap_reset(job->copy_bitmap, offset, job->cluster_size);
+            bdrv_co_unlock(lock);
              continue;
          }
 
          do {
              if (yield_and_check(job)) {
+                bdrv_co_unlock(lock);
                  return 0;
              }
-            ret = backup_do_cow(job, offset,
-                                job->cluster_size, &error_is_read, false);
+            ret = backup_do_cow(job, offset, job->cluster_size, &error_is_read);
              if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
                             BLOCK_ERROR_ACTION_REPORT)
              {
+                bdrv_co_unlock(lock);
                  return ret;
              }
          } while (ret < 0);
+
+        bdrv_co_unlock(lock);
      }
 
      return 0;
Signature · 2 lines
Best regards,
Vladimir
#29 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Hi all!

These series introduce backup-top driver. It's a filter-node, which do copy-before-write operation. Mirror uses filter-node for handling guest writes, let's move to filter-node (from write-notifiers) for backup too

Quoted 8 lines

v8:
01-03: new
05: add Max's r-b
others changed, change description in each patch mail in Notes section.

v6-v7 was preparing refactoring, which now is in Max's pull request, and these series based on it:
Based-on: https://lists.gnu.org/archive/html/qemu-devel/2019-05/msg06526.html

It merged, so, now the series based on master.

Quoted 25 lines

Vladimir Sementsov-Ogievskiy (7):
block: teach bdrv_debug_breakpoint skip filters with backing
block: swap operation order in bdrv_append
block: allow not one child for implicit node
block: introduce backup-top filter driver block/io: refactor wait_serialising_requests
block: add lock/unlock range functions block/backup: use backup-top instead of write notifiers

  block/backup-top.h         |  64 ++++++++
  include/block/block_int.h  |   4 +
  block.c                    |  60 +++++--
  block/backup-top.c         | 322 +++++++++++++++++++++++++++++++++++++
  block/backup.c             | 171 ++++++++------------
  block/io.c                 |  68 ++++++--
  block/Makefile.objs        |   2 +
  tests/qemu-iotests/056     |   2 +-
  tests/qemu-iotests/085.out |   2 +-
  tests/qemu-iotests/129     |   1 -
  tests/qemu-iotests/141.out |   2 +-
  11 files changed, 567 insertions(+), 131 deletions(-)

create mode 100644 block/backup-top.h
create mode 100644 block/backup-top.c

Signature · 2 lines
Best regards,
Vladimir
#30 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

  CC      crypto/cipher.o
  CC      crypto/tlscreds.o
  CC      crypto/tlscredsanon.o
/tmp/qemu-test/src/block/backup-top.c:268:5: error: implicit declaration of function 'bdrv_set_aio_context' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    bdrv_set_aio_context(top, bdrv_get_aio_context(source));
    ^
/tmp/qemu-test/src/block/backup-top.c:268:5: note: did you mean 'bdrv_get_aio_context'?
/tmp/qemu-test/src/include/block/block.h:579:13: note: 'bdrv_get_aio_context' declared here
AioContext *bdrv_get_aio_context(BlockDriverState *bs);
            ^
/tmp/qemu-test/src/block/backup-top.c:268:5: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes]
    bdrv_set_aio_context(top, bdrv_get_aio_context(source));
    ^

2 errors generated.

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#31 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed build test on s390x host. Please find the details below.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with # HEAD pointing to a commit that has the patches applied on top of "base"

# branch
set -e
CC=$HOME/bin/cc
INSTALL=$PWD/install
BUILD=$PWD/build
mkdir -p $BUILD $INSTALL
SRC=$PWD
cd $BUILD
$SRC/configure --cc=$CC --prefix=$INSTALL
make -j4
# XXX: we need reliable clean up
# make check -j4 V=1
make install

echo
echo "=== ENV ==="
env

echo
echo "=== PACKAGES ==="
rpm -qa
=== TEST SCRIPT END ===

CC block/replication.o
CC block/throttle.o
/var/tmp/patchew-tester-tmp-eg3dw7w5/src/block/backup.c: In function ‘backup_run’:
/var/tmp/patchew-tester-tmp-eg3dw7w5/src/block/backup.c:283:8: error: ‘error_is_read’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
283 | if (read) {
| ^
/var/tmp/patchew-tester-tmp-eg3dw7w5/src/block/backup.c:332:10: note: ‘error_is_read’ was declared here

The full log is available at
http://patchew.org/logs/[email protected]/testing.s390x/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]