Skip to contentSkip to conversation

block: publish backup-top filter

PATCH Superseded Vladimir Sementsov-Ogievskiy 2 participants 66 messages May 17, 2021 – May 19, 2021

Subsystems (2)

v1 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

cover block: publish backup-top filter

Hi all!

We have image fleecing scheme to export point-in-time state of active disk (iotest 222):

backup(sync=none)
┌───────────────────────────────────────┐
▼ │

┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                                                             ▲
┌────────────┐                                               │
│ guest blk  │ ──────────────────────────────────────────────┘
└────────────┘                        

Actually, backup job inserts a backup-top filter, so in detail it looks like:

backup(sync=none)
┌───────────────────────────────────────┐
▼ │

┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                     ▲                                       ▲
                     │ target                                │
                     │                                       │
┌────────────┐     ┌────────────────┐  backing               │
│ guest blk  │ ──▶ │   backup-top   │ ───────────────────────┘
└────────────┘     └────────────────┘

And job does nothing here. In a new blockdev world user is intended to operate on node level, and insert/remove filters by hand. Let's get rid of job in the scheme:

┌────────────┐     ┌────────────────┐  backing             ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘     └────────────────┘                      └─────────────┘
                     ▲                                       ▲
                     │ target                                │
                     │                                       │
┌────────────┐     ┌────────────────┐  backing               │
│ guest blk  │ ──▶ │   backup-top   │ ───────────────────────┘
└────────────┘     └────────────────┘

The series prepares qom-set to make possible inserting filters above root node (patches 01-04), rename backup-top to copy-before-write, do other preparations for publishing the filter, and finally publish it, add qapi interface and test new fleecing scheme in 222.

Vladimir Sementsov-Ogievskiy (21):
block: introduce bdrv_replace_child_bs()
block: introduce blk_replace_bs
qdev-properties: PropertyInfo: add realized_set_allowed field
qdev: allow setting drive property for realized device
block: rename backup-top to copy-before-write block/backup: drop support for copy_range
block-copy: always set BDRV_REQ_SERIALISING flag block/backup: stricter backup_calculate_cluster_size() block/backup: move cluster size calculation to block-copy block/copy-before-write: relax permission requirements when no parents block/copy-before-write: use file child instead of backing block/copy-before-write: bdrv_cbw_append(): replace child at last block/copy-before-write: introduce cbw_init() block/copy-before-write: cbw_init(): rename variables block/copy-before-write: cbw_init(): use file child after attaching block/copy-before-write: cbw_init(): use options block/block-copy: switch to fully set bitmap by default block/block-copy: make setting progress optional block/copy-before-write: make public block driver
qapi: publish copy-before-write filter
itotests/222: add test-case for copy-before-write filter

 qapi/block-core.json                        |  22 +-
 block/{backup-top.h => copy-before-write.h} |  26 +-
 include/block/block-copy.h                  |   4 +-
 include/block/block.h                       |   2 +
 include/hw/qdev-properties.h                |   1 +
 include/sysemu/block-backend.h              |   1 +
 block.c                                     |  36 +++
 block/backup-top.c                          | 253 -------------------
 block/backup.c                              | 115 ++-------
 block/block-backend.c                       |   8 +
 block/block-copy.c                          |  95 +++++++-
 block/copy-before-write.c                   | 257 ++++++++++++++++++++
 hw/core/qdev-properties-system.c            |  30 ++-
 hw/core/qdev-properties.c                   |   6 +-
 MAINTAINERS                                 |   4 +-
 block/meson.build                           |   2 +-
 tests/qemu-iotests/222                      |  56 ++++-
 tests/qemu-iotests/222.out                  |  72 ++++++
 tests/qemu-iotests/283                      |  35 ++-
 tests/qemu-iotests/283.out                  |   4 +-
 20 files changed, 606 insertions(+), 423 deletions(-)
 rename block/{backup-top.h => copy-before-write.h} (56%)
 delete mode 100644 block/backup-top.c
 create mode 100644 block/copy-before-write.c
Signature · 1 line
2.29.2
#1 lore raw

PATCH 1/1 block/copy-on-read: use bdrv_drop_filter() and drop s->active

Now, after huge update of block graph permission update algorithm, we don't need this workaround with active state of the filter. Drop it and use new smart bdrv_drop_filter() function.

---
 block/copy-on-read.c | 33 +--------------------------------
 1 file changed, 1 insertion(+), 32 deletions(-)
block/copy-on-read.c +1 -32
index 9cad9e1b8c..c428682272 100644
@@ -29,7 +29,6 @@
 
 
 typedef struct BDRVStateCOR {
-    bool active;
     BlockDriverState *bottom_bs;
     bool chain_frozen;
 } BDRVStateCOR;
@@ -89,7 +88,6 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags,
          */
         bdrv_ref(bottom_bs);
     }
-    state->active = true;
     state->bottom_bs = bottom_bs;
 
     /*
@@ -112,17 +110,6 @@ static void cor_child_perm(BlockDriverState *bs, BdrvChild *c,
                            uint64_t perm, uint64_t shared,
                            uint64_t *nperm, uint64_t *nshared)
 {
-    BDRVStateCOR *s = bs->opaque;
-
-    if (!s->active) {
-        /*
-         * While the filter is being removed
-         */
-        *nperm = 0;
-        *nshared = BLK_PERM_ALL;
-        return;
-    }
-
     *nperm = perm & PERM_PASSTHROUGH;
     *nshared = (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED;
 
@@ -280,32 +267,14 @@ static BlockDriver bdrv_copy_on_read = {
 
 void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs)
 {
-    BdrvChild *child;
-    BlockDriverState *bs;
     BDRVStateCOR *s = cor_filter_bs->opaque;
 
-    child = bdrv_filter_child(cor_filter_bs);
-    if (!child) {
-        return;
-    }
-    bs = child->bs;
-
-    /* Retain the BDS until we complete the graph change. */
-    bdrv_ref(bs);
-    /* Hold a guest back from writing while permissions are being reset. */
-    bdrv_drained_begin(bs);
-    /* Drop permissions before the graph change. */
-    s->active = false;
     /* unfreeze, as otherwise bdrv_replace_node() will fail */
     if (s->chain_frozen) {
         s->chain_frozen = false;
         bdrv_unfreeze_backing_chain(cor_filter_bs, s->bottom_bs);
     }
-    bdrv_child_refresh_perms(cor_filter_bs, child, &error_abort);
-    bdrv_replace_node(cor_filter_bs, bs, &error_abort);
-
-    bdrv_drained_end(bs);
-    bdrv_unref(bs);
+    bdrv_drop_filter(cor_filter_bs, &error_abort);
     bdrv_unref(cor_filter_bs);
 }
 
Signature · 1 line
2.29.2
#2 lore raw

PATCH 1/21 block: introduce bdrv_replace_child_bs()

In reply to Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
 include/block/block.h |  2 ++
 block.c               | 36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
                 Error **errp);
 int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                       Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
 BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                    int flags, Error **errp);
 int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
block.c +36 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
     return ret;
 }
 
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {
+        bdrv_ref(old_bs);
+        bdrv_drained_begin(old_bs);
+    }
+    bdrv_drained_begin(new_bs);
+
+    bdrv_replace_child(child, new_bs, tran);
+
+    found = g_hash_table_new(NULL, NULL);
+    if (old_bs) {
+        refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
+    }
+    refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+    ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);
+
+    tran_finalize(tran, ret);
+
+    if (old_bs) {
+        bdrv_drained_end(old_bs);
+        bdrv_unref(old_bs);
+    }
+    bdrv_drained_end(new_bs);
+
+    return ret;
+}
+
 static void bdrv_delete(BlockDriverState *bs)
 {
     assert(bdrv_op_blocker_is_empty(bs));
Signature · 1 line
2.29.2
#3 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++
  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 21 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
                  Error **errp);
  int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                        Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
  BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                     int flags, Error **errp);
  int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
block.c +0 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
      return ret;
  }
 

block.c out: · line 4973

Quoted 10 lines
block.c +10 -0
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm. Can child->bs be ever NULL?

block.c out: · line 4977

Quoted 4 lines
block.c +4 -0
+        bdrv_ref(old_bs);
+        bdrv_drained_begin(old_bs);
+    }
+    bdrv_drained_begin(new_bs);

(I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL. Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.

Hence me wondering whether we could assume child->bs not to be NULL.)

block.c out: · line 4987

Quoted 10 lines
block.c +10 -0
+
+    bdrv_replace_child(child, new_bs, tran);
+
+    found = g_hash_table_new(NULL, NULL);
+    if (old_bs) {
+        refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
+    }
+    refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+    ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);

Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?

I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general?
Would there be a problem if we refreshed permissions there? Or is not
refreshing permissions just part of the function’s interface?

Max

Quoted 16 lines
block.c +12 -0
+
+    tran_finalize(tran, ret);
+
+    if (old_bs) {
+        bdrv_drained_end(old_bs);
+        bdrv_unref(old_bs);
+    }
+    bdrv_drained_end(new_bs);
+
+    return ret;
+}
+
  static void bdrv_delete(BlockDriverState *bs)
  {
      assert(bdrv_op_blocker_is_empty(bs));
#4 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

Quoted 35 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++

  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me.  However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 24 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
                   Error **errp);
   int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                         Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
   BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                      int flags, Error **errp);
   int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
block.c +4 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
       return ret;
   }
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
Quoted 8 lines

block.c out: · line 4973

Quoted 6 lines
block.c +6 -0
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm.  Can child->bs be ever NULL?

Hmm. Most probably not :)

In some intermediate states we don't have bs in child, but it shouldn't be the place where bdrv_replace_child_bs is called.

Quoted 9 lines

block.c out: · line 4977

Quoted 5 lines
block.c +4 -0
+        bdrv_ref(old_bs);
+        bdrv_drained_begin(old_bs);
+    }
+    bdrv_drained_begin(new_bs);

(I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL.  Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.

Hence me wondering whether we could assume child->bs not to be NULL.)

bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't do drained section nor permission update. And new bdrv_replace_child_bs() is public function, which cares about these things.

Quoted 14 lines

block.c out: · line 4987

Quoted 11 lines
block.c +10 -0
+
+    bdrv_replace_child(child, new_bs, tran);
+
+    found = g_hash_table_new(NULL, NULL);
+    if (old_bs) {
+        refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
+    }
+    refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+    ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);

Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions.  I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?

Quoted 2 lines

I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions.  And that assumption should hold true right now, given its callers.  But is that a safe assumption in general?  Would there be a problem if we refreshed permissions there?  Or is not refreshing permissions just part of the function’s interface?

Caller of bdrv_remove_filter_or_cow_child() should care about permissions: bdrv_replace_node_common() do this, and bdrv_set_backing_noperm() has "_noperm" in the name..

The main impact of previous big rework of permission is new scheme of working with permission update:

- first do all graph modifications, not thinking about permissions
- refresh permissions for the whole updated subgraph
- if refresh failed, rollback all the modifications (main sense if transactions here and there is possibility to do this rollback)

So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.

Quoted 18 lines
Quoted 17 lines
block.c +12 -0
+
+    tran_finalize(tran, ret);
+
+    if (old_bs) {
+        bdrv_drained_end(old_bs);
+        bdrv_unref(old_bs);
+    }
+    bdrv_drained_end(new_bs);
+
+    return ret;
+}
+
   static void bdrv_delete(BlockDriverState *bs)
   {
       assert(bdrv_op_blocker_is_empty(bs));
 
Signature · 2 lines
Best regards,
Vladimir
#5 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

Quoted 77 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++

  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me.  However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 25 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, 
 BlockDriverState *bs_top,
                   Error **errp);
   int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                         Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
   BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                      int flags, Error **errp);

  int bdrv_drop_filter(BlockDriverState *bs, Error **errp);

block.c +4 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
       return ret;
   }
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
Quoted 8 lines

block.c out: · line 4973

Quoted 6 lines
block.c +6 -0
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm.  Can child->bs be ever NULL?

Hmm. Most probably not :)

In some intermediate states we don't have bs in child, but it shouldn't be the place where bdrv_replace_child_bs is called.

Quoted 9 lines

block.c out: · line 4977

Quoted 5 lines
block.c +4 -0
+        bdrv_ref(old_bs);
+        bdrv_drained_begin(old_bs);
+    }
+    bdrv_drained_begin(new_bs);

(I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL.  Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.

Hence me wondering whether we could assume child->bs not to be NULL.)

bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't do drained section nor permission update. And new bdrv_replace_child_bs() is public function, which cares about these things.

Quoted 15 lines

block.c out: · line 4987

Quoted 12 lines
block.c +10 -0
+
+    bdrv_replace_child(child, new_bs, tran);
+
+    found = g_hash_table_new(NULL, NULL);
+    if (old_bs) {
+        refresh_list = bdrv_topological_dfs(refresh_list, found, 
 old_bs);
+    }
+    refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+    ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);

Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions.  I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?

Quoted 4 lines
Quoted 2 lines

I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions.  And that assumption should hold true right now, given its callers.  But is that a safe assumption in general?  Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?

Caller of bdrv_remove_filter_or_cow_child() should care about permissions:  bdrv_replace_node_common() do this, and bdrv_set_backing_noperm() has "_noperm" in the name..

OK. Makes me wonder why bdrv_remove_filter_or_cow_child() then doesn’t have _noperm in its name, or why its comment doesn’t explain this interface contract, but, well. :)

Quoted 8 lines

The main impact of previous big rework of permission is new scheme of working with permission update:

 - first do all graph modifications, not thinking about permissions
 - refresh permissions for the whole updated subgraph
 - if refresh failed, rollback all the modifications (main sense if
transactions here and there is possibility to do this rollback)

So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.

Ah, OK. Makes sense, thanks.

Max

#6 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

Quoted 76 lines · Max Reitz
Quoted 75 lines · Vladimir Sementsov-Ogievskiy
Quoted 35 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++

  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me.  However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 24 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
                   Error **errp);
   int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                         Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
   BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                      int flags, Error **errp);
   int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
block.c +4 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
       return ret;
   }
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
Quoted 8 lines

block.c out: · line 4973

Quoted 6 lines
block.c +6 -0
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm.  Can child->bs be ever NULL?

Hmm. Most probably not :)

In some intermediate states we don't have bs in child, but it shouldn't be the place where bdrv_replace_child_bs is called.

Quoted 9 lines

block.c out: · line 4977

Quoted 5 lines
block.c +4 -0
+        bdrv_ref(old_bs);
+        bdrv_drained_begin(old_bs);
+    }
+    bdrv_drained_begin(new_bs);

(I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL.  Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.

Hence me wondering whether we could assume child->bs not to be NULL.)

bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't do drained section nor permission update. And new bdrv_replace_child_bs() is public function, which cares about these things.

Quoted 14 lines

block.c out: · line 4987

Quoted 11 lines
block.c +10 -0
+
+    bdrv_replace_child(child, new_bs, tran);
+
+    found = g_hash_table_new(NULL, NULL);
+    if (old_bs) {
+        refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
+    }
+    refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+    ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);

Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions.  I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions.  And that assumption should hold true right now, given its callers.  But is that a safe assumption in general?  Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?

Caller of bdrv_remove_filter_or_cow_child() should care about permissions:  bdrv_replace_node_common() do this, and bdrv_set_backing_noperm() has "_noperm" in the name..

OK.  Makes me wonder why bdrv_remove_filter_or_cow_child() then doesn’t have _noperm in its name, or why its comment doesn’t explain this interface contract, but, well. :)

You are right that's unclear. I'll make the patch that cleans that up.

Quoted 13 lines
Quoted 8 lines

The main impact of previous big rework of permission is new scheme of working with permission update:

  - first do all graph modifications, not thinking about permissions
  - refresh permissions for the whole updated subgraph
  - if refresh failed, rollback all the modifications (main sense if transactions here and there is possibility to do this rollback)

So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.

Ah, OK.  Makes sense, thanks.

Max

Signature · 2 lines
Best regards,
Vladimir
#7 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

In reply to Max Reitz
Quoted 35 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++

  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me.  However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 24 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
                   Error **errp);
   int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                         Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
   BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                      int flags, Error **errp);
   int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
block.c +4 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
       return ret;
   }
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
Quoted 8 lines

block.c out: · line 4973

Quoted 6 lines
block.c +6 -0
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm.  Can child->bs be ever NULL?

Seems it can. For example we have hmp_drive_del command, that removes bs from blk :(

qmp eject and blockdev-remove-medium seems do it too.

Quoted 44 lines

block.c out: · line 4977

Quoted 5 lines
block.c +4 -0
+        bdrv_ref(old_bs);
+        bdrv_drained_begin(old_bs);
+    }
+    bdrv_drained_begin(new_bs);

(I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL.  Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.

Hence me wondering whether we could assume child->bs not to be NULL.)

block.c out: · line 4987

Quoted 10 lines
block.c +10 -0
+
+    bdrv_replace_child(child, new_bs, tran);
+
+    found = g_hash_table_new(NULL, NULL);
+    if (old_bs) {
+        refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs);
+    }
+    refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs);
+
+    ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);

Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions.  I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?

I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions.  And that assumption should hold true right now, given its callers.  But is that a safe assumption in general?  Would there be a problem if we refreshed permissions there?  Or is not refreshing permissions just part of the function’s interface?

Max

Quoted 16 lines
block.c +12 -0
+
+    tran_finalize(tran, ret);
+
+    if (old_bs) {
+        bdrv_drained_end(old_bs);
+        bdrv_unref(old_bs);
+    }
+    bdrv_drained_end(new_bs);
+
+    return ret;
+}
+
   static void bdrv_delete(BlockDriverState *bs)
   {
       assert(bdrv_op_blocker_is_empty(bs));
 
Signature · 2 lines
Best regards,
Vladimir
#8 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

Quoted 46 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++

  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me.  However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 25 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, 
 BlockDriverState *bs_top,
                   Error **errp);
   int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                         Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
   BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                      int flags, Error **errp);

  int bdrv_drop_filter(BlockDriverState *bs, Error **errp);

block.c +4 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
       return ret;
   }
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
Quoted 8 lines

block.c out: · line 4973

Quoted 6 lines
block.c +6 -0
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm.  Can child->bs be ever NULL?

Quoted 3 lines

Seems it can. For example we have hmp_drive_del command, that removes bs from blk :(

qmp eject and blockdev-remove-medium seems do it too.

blk_remove_bs() doesn’t eject the BDS from the BdrvChild blk->root, though, it drops blk->root altogether. Doesn’t it?

Max

#9 lore raw

Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()

Quoted 46 lines · Max Reitz
Quoted 45 lines · Vladimir Sementsov-Ogievskiy
Quoted 43 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add function to transactionally replace bs inside BdrvChild.

---
  include/block/block.h |  2 ++
  block.c               | 36 ++++++++++++++++++++++++++++++++++++

  2 files changed, 38 insertions(+)

As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me.  However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.

Quoted 20 lines
include/block/block.h +2 -0
index 82185965ff..f9d5fcb108 100644
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
                   Error **errp);
   int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
                         Error **errp);
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp);
   BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
                                      int flags, Error **errp);
   int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
block.c +0 -0
index 9ad725d205..755fa53d85 100644
@@ -4961,6 +4961,42 @@ out:
       return ret;
   }

block.c out: · line 4972

Quoted 10 lines
block.c +10 -0
+int bdrv_replace_child_bs(BdrvChild *child, BlockDriverState *new_bs,
+                          Error **errp)
+{
+    int ret;
+    Transaction *tran = tran_new();
+    g_autoptr(GHashTable) found = NULL;
+    g_autoptr(GSList) refresh_list = NULL;
+    BlockDriverState *old_bs = child->bs;
+
+    if (old_bs) {

Hm.  Can child->bs be ever NULL?

Quoted 6 lines
Quoted 3 lines

Seems it can. For example we have hmp_drive_del command, that removes bs from blk :(

qmp eject and blockdev-remove-medium seems do it too.

blk_remove_bs() doesn’t eject the BDS from the BdrvChild blk->root, though, it drops blk->root altogether.  Doesn’t it?

A hm, yes. What I say is that we can have empty blk. But probably we should never have BdrvChild with NULL bs. I'll check it

Signature · 2 lines
Best regards,
Vladimir
#10 lore raw

PATCH 2/21 block: introduce blk_replace_bs

In reply to Vladimir Sementsov-Ogievskiy

Add function to change bs inside blk.

---
 include/sysemu/block-backend.h | 1 +
 block/block-backend.c          | 8 ++++++++
 2 files changed, 9 insertions(+)
include/sysemu/block-backend.h +1 -0
index 880e903293..aec05ef0a0 100644
@@ -98,6 +98,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public);
 BlockDriverState *blk_bs(BlockBackend *blk);
 void blk_remove_bs(BlockBackend *blk);
 int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp);
+int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp);
 bool bdrv_has_blk(BlockDriverState *bs);
 bool bdrv_is_root_node(BlockDriverState *bs);
 int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,
block/block-backend.c +8 -0
index de5496af66..b1abc6f3e6 100644
@@ -870,6 +870,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp)
     return 0;
 }
 
+/*
+ * Change BlockDriverState associated with @blk.
+ */
+int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp)
+{
+    return bdrv_replace_child_bs(blk->root, new_bs, errp);
+}
+
 /*
  * Sets the permission bitmasks that the user of the BlockBackend needs.
  */
Signature · 1 line
2.29.2
Reviewed-by on v1 2/21
#11 lore raw

Re: [PATCH 02/21] block: introduce blk_replace_bs

Quoted 25 lines · Vladimir Sementsov-Ogievskiy

Add function to change bs inside blk.

---
  include/sysemu/block-backend.h | 1 +
  block/block-backend.c          | 8 ++++++++
  2 files changed, 9 insertions(+)
include/sysemu/block-backend.h +1 -0
index 880e903293..aec05ef0a0 100644
@@ -98,6 +98,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public);
  BlockDriverState *blk_bs(BlockBackend *blk);
  void blk_remove_bs(BlockBackend *blk);
  int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp);
+int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp);
  bool bdrv_has_blk(BlockDriverState *bs);
  bool bdrv_is_root_node(BlockDriverState *bs);
  int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,
block/block-backend.c +0 -0
index de5496af66..b1abc6f3e6 100644

block/block-backend.c int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp) · line 879

Quoted 10 lines
block/block-backend.c +7 -0
@@ -870,6 +870,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp)
      return 0;
  }
 
+/*
+ * Change BlockDriverState associated with @blk.
+ */
+int blk_replace_bs(BlockBackend *blk, BlockDriverState *new_bs, Error **errp)
+{
+    return bdrv_replace_child_bs(blk->root, new_bs, errp);
+}

(Looks indeed like we don’t need to do any of the things that blk_insert_bs() and blk_remove_bs() do besides inserting and removing the node.)

#12 lore raw

PATCH 3/21 qdev-properties: PropertyInfo: add realized_set_allowed field

In reply to Vladimir Sementsov-Ogievskiy

Add field, so property can declare support for setting the property when device is realized. To be used in the following commit.

---
 include/hw/qdev-properties.h | 1 +
 hw/core/qdev-properties.c    | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)
include/hw/qdev-properties.h +1 -0
index 0ef97d60ce..007e1f69f4 100644
@@ -32,6 +32,7 @@ struct PropertyInfo {
     const char *name;
     const char *description;
     const QEnumLookup *enum_table;
+    bool realized_set_allowed;
     int (*print)(Object *obj, Property *prop, char *dest, size_t len);
     void (*set_default_value)(ObjectProperty *op, const Property *prop);
     ObjectProperty *(*create)(ObjectClass *oc, const char *name,
hw/core/qdev-properties.c +3 -3
index 50f40949f5..c34aac6ebc 100644
@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,
 
 /* returns: true if property is allowed to be set, false otherwise */
 static bool qdev_prop_allow_set(Object *obj, const char *name,
-                                Error **errp)
+                                const PropertyInfo *info, Error **errp)
 {
     DeviceState *dev = DEVICE(obj);
 
-    if (dev->realized) {
+    if (dev->realized && !info->realized_set_allowed) {
         qdev_prop_set_after_realize(dev, name, errp);
         return false;
     }
@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name,
 {
     Property *prop = opaque;
 
-    if (!qdev_prop_allow_set(obj, name, errp)) {
+    if (!qdev_prop_allow_set(obj, name, prop->info, errp)) {
         return;
     }
 
Signature · 1 line
2.29.2
#13 lore raw

Re: [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Add field, so property can declare support for setting the property when device is realized. To be used in the following commit.

---
  include/hw/qdev-properties.h | 1 +
  hw/core/qdev-properties.c    | 6 +++---
  2 files changed, 4 insertions(+), 3 deletions(-)

Looks OK to me, although qdev isn’t my specialty.

include/hw/qdev-properties.h struct PropertyInfo { · line 35

Quoted 9 lines
include/hw/qdev-properties.h +1 -0
index 0ef97d60ce..007e1f69f4 100644
@@ -32,6 +32,7 @@ struct PropertyInfo {
      const char *name;
      const char *description;
      const QEnumLookup *enum_table;
+    bool realized_set_allowed;

I think a comment would be nice, though.

Max

Quoted 31 lines
include/hw/qdev-properties.h +0 -0
      int (*print)(Object *obj, Property *prop, char *dest, size_t len);
      void (*set_default_value)(ObjectProperty *op, const Property *prop);
      ObjectProperty *(*create)(ObjectClass *oc, const char *name,
hw/core/qdev-properties.c +3 -3
index 50f40949f5..c34aac6ebc 100644
@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,
 
  /* returns: true if property is allowed to be set, false otherwise */
  static bool qdev_prop_allow_set(Object *obj, const char *name,
-                                Error **errp)
+                                const PropertyInfo *info, Error **errp)
  {
      DeviceState *dev = DEVICE(obj);
 
-    if (dev->realized) {
+    if (dev->realized && !info->realized_set_allowed) {
          qdev_prop_set_after_realize(dev, name, errp);
          return false;
      }
@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name,
  {
      Property *prop = opaque;
 
-    if (!qdev_prop_allow_set(obj, name, errp)) {
+    if (!qdev_prop_allow_set(obj, name, prop->info, errp)) {
          return;
      }
 
#14 lore raw

Re: [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field

Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Add field, so property can declare support for setting the property when device is realized. To be used in the following commit.

---
  include/hw/qdev-properties.h | 1 +
  hw/core/qdev-properties.c    | 6 +++---

  2 files changed, 4 insertions(+), 3 deletions(-)

Looks OK to me, although qdev isn’t my specialty.

Neither my :) Thanks for looking anyway!

Quoted 13 lines

include/hw/qdev-properties.h struct PropertyInfo { · line 35

Quoted 10 lines
include/hw/qdev-properties.h +1 -0
index 0ef97d60ce..007e1f69f4 100644
@@ -32,6 +32,7 @@ struct PropertyInfo {
       const char *name;
       const char *description;
       const QEnumLookup *enum_table;
+    bool realized_set_allowed;

I think a comment would be nice, though.

Agree, will add.

Quoted 29 lines
Quoted 28 lines
include/hw/qdev-properties.h +0 -0
       int (*print)(Object *obj, Property *prop, char *dest, size_t len);
       void (*set_default_value)(ObjectProperty *op, const Property *prop);
       ObjectProperty *(*create)(ObjectClass *oc, const char *name,
hw/core/qdev-properties.c +3 -3
index 50f40949f5..c34aac6ebc 100644
@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,
   /* returns: true if property is allowed to be set, false otherwise */
   static bool qdev_prop_allow_set(Object *obj, const char *name,
-                                Error **errp)
+                                const PropertyInfo *info, Error **errp)
   {
       DeviceState *dev = DEVICE(obj);
-    if (dev->realized) {
+    if (dev->realized && !info->realized_set_allowed) {
           qdev_prop_set_after_realize(dev, name, errp);
           return false;
       }
@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name,
   {
       Property *prop = opaque;
-    if (!qdev_prop_allow_set(obj, name, errp)) {
+    if (!qdev_prop_allow_set(obj, name, prop->info, errp)) {
           return;
       }
 
Signature · 2 lines
Best regards,
Vladimir
#15 lore raw

PATCH 4/21 qdev: allow setting drive property for realized device

In reply to Vladimir Sementsov-Ogievskiy

We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.

Intended usage:

1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.

2. qom-set, to change bs attached to root blk from original node to newly create filter.

---
 hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
 1 file changed, 22 insertions(+), 8 deletions(-)
hw/core/qdev-properties-system.c +22 -8
index 2760c21f11..7d97562654 100644
@@ -93,16 +93,30 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,
     BlockBackend *blk;
     bool blk_created = false;
     int ret;
+    BlockDriverState *bs;
+    AioContext *ctx;
 
     if (!visit_type_str(v, name, &str, errp)) {
         return;
     }
 
-    /*
-     * TODO Should this really be an error?  If no, the old value
-     * needs to be released before we store the new one.
-     */
-    if (!check_prop_still_unset(obj, name, *ptr, str, errp)) {
+    if (*ptr) {
+        /* BlockBackend alread exists. So, we want to change attached node */
+        blk = *ptr;
+        ctx = blk_get_aio_context(blk);
+        bs = bdrv_lookup_bs(NULL, str, errp);
+        if (!bs) {
+            return;
+        }
+
+        if (ctx != bdrv_get_aio_context(bs)) {
+            error_setg(errp, "Different aio context is not supported for new "
+                       "node");
+        }
+
+        aio_context_acquire(ctx);
+        blk_replace_bs(blk, bs, errp);
+        aio_context_release(ctx);
         return;
     }
 
@@ -114,7 +128,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,
 
     blk = blk_by_name(str);
     if (!blk) {
-        BlockDriverState *bs = bdrv_lookup_bs(NULL, str, NULL);
+        bs = bdrv_lookup_bs(NULL, str, NULL);
         if (bs) {
             /*
              * If the device supports iothreads, it will make sure to move the
@@ -123,8 +137,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,
              * aware of iothreads require their BlockBackends to be in the main
              * AioContext.
              */
-            AioContext *ctx = iothread ? bdrv_get_aio_context(bs) :
-                                         qemu_get_aio_context();
+            ctx = iothread ? bdrv_get_aio_context(bs) : qemu_get_aio_context();
             blk = blk_new(ctx, 0, BLK_PERM_ALL);
             blk_created = true;
 
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
 const PropertyInfo qdev_prop_drive = {
     .name  = "str",
     .description = "Node name or ID of a block device to use as a backend",
+    .realized_set_allowed = true,
     .get   = get_drive,
     .set   = set_drive,
     .release = release_drive,
Signature · 1 line
2.29.2
#16 lore raw

Re: [PATCH 04/21] qdev: allow setting drive property for realized device

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.

Intended usage:

1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.

Is a “not” missing here, i.e. “not attached to any guest block device”?
I would have thought one would create a filtered tree that is not in
use by any frontend, so that the filter need not take any permissions.

Quoted 7 lines

2. qom-set, to change bs attached to root blk from original node to newly create filter.

---
  hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------
  1 file changed, 22 insertions(+), 8 deletions(-)

Looks good, just one question: (well, two, one was above)

Quoted 4 lines
hw/core/qdev-properties-system.c +0 -0
index 2760c21f11..7d97562654 100644

[...]

hw/core/qdev-properties-system.c static void release_drive(Object *obj, const char *name, void *opaque) · line 215

Quoted 8 lines
hw/core/qdev-properties-system.c +1 -0
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
  const PropertyInfo qdev_prop_drive = {
      .name  = "str",
      .description = "Node name or ID of a block device to use as a backend",
+    .realized_set_allowed = true,
      .get   = get_drive,
      .set   = set_drive,
      .release = release_drive,

Why not for qdev_prop_drive_iothread?

Max

#17 lore raw

Re: [PATCH 04/21] qdev: allow setting drive property for realized device

Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy

We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.

Intended usage:

1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.

Is a “not” missing here, i.e. “not attached to any guest block device”?  I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.

node A is attached.

So, we have [blk] --root-> [A}

And want to insert a filter between blk and A.

We do

1.

[filter] --file--\
v
[blk] --root--> [A]

2.

[blk] --root--> [filer] --file--> [A]

Quoted 18 lines
Quoted 8 lines

2. qom-set, to change bs attached to root blk from original node to newly create filter.

---
  hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------

  1 file changed, 22 insertions(+), 8 deletions(-)

Looks good, just one question: (well, two, one was above)

Quoted 4 lines
hw/core/qdev-properties-system.c +0 -0
index 2760c21f11..7d97562654 100644

[...]

Quoted 11 lines

hw/core/qdev-properties-system.c static void release_drive(Object *obj, const char *name, void *opaque) · line 215

Quoted 8 lines
hw/core/qdev-properties-system.c +1 -0
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
   const PropertyInfo qdev_prop_drive = {
       .name  = "str",
       .description = "Node name or ID of a block device to use as a backend",
+    .realized_set_allowed = true,
       .get   = get_drive,
       .set   = set_drive,
       .release = release_drive,

Why not for qdev_prop_drive_iothread?

Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version

Signature · 2 lines
Best regards,
Vladimir
#18 lore raw

Re: [PATCH 04/21] qdev: allow setting drive property for realized device

Quoted 19 lines · Vladimir Sementsov-Ogievskiy
Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy

We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.

Intended usage:

1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.

Is a “not” missing here, i.e. “not attached to any guest block device”?  I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.

node A is attached.

So, we have [blk] --root->  [A}

And want to insert a filter between blk and A.

Quoted 7 lines

We do

1.

[filter] --file--\
                 v
[blk] --root-->  [A]

Oh, so you mean node A is attached to a guest device. The sentence sounded to me like the newly created filter tree were attached to it.

Yes, that’s how I expected it to be. I just find the sentence not quite clear, because I found it ambiguous which node the “attached to some guest block device” refers to.

Perhaps:
“Intended usage:

Assume there is a node A that is attached to some guest device.
1. blockdev-add to create a filter node B that has A as its child.
2. qom-set to change the node attached to the guest device’s BlockBackend from A to B.”

?

Quoted 36 lines

2.

[blk] --root--> [filer] --file--> [A]

Quoted 18 lines
Quoted 8 lines

2. qom-set, to change bs attached to root blk from original node to newly create filter.

---
  hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------

  1 file changed, 22 insertions(+), 8 deletions(-)

Looks good, just one question: (well, two, one was above)

Quoted 4 lines
hw/core/qdev-properties-system.c +0 -0
index 2760c21f11..7d97562654 100644

[...]

Quoted 12 lines
Quoted 9 lines
hw/core/qdev-properties-system.c +1 -0
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char 
 *name, void *opaque)
   const PropertyInfo qdev_prop_drive = {
       .name  = "str",
       .description = "Node name or ID of a block device to use as a backend",
+    .realized_set_allowed = true,
       .get   = get_drive,
       .set   = set_drive,

      .release = release_drive,

Why not for qdev_prop_drive_iothread?

Quoted 1 line

Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version

OK. (I just saw it because it was right below this structure, too, it isn’t like I actually know what I’m saying.)

Max

#19 lore raw

Re: [PATCH 04/21] qdev: allow setting drive property for realized device

Quoted 32 lines · Max Reitz
Quoted 19 lines · Vladimir Sementsov-Ogievskiy
Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy

We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.

Intended usage:

1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.

Is a “not” missing here, i.e. “not attached to any guest block device”?  I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.

node A is attached.

So, we have [blk] --root->  [A}

And want to insert a filter between blk and A.

Quoted 7 lines

We do

1.

[filter] --file--\
                  v
[blk] --root-->  [A]

Oh, so you mean node A is attached to a guest device.  The sentence sounded to me like the newly created filter tree were attached to it.

Yes, that’s how I expected it to be.  I just find the sentence not quite clear, because I found it ambiguous which node the “attached to some guest block device” refers to.

Quoted 8 lines

Perhaps:
“Intended usage:

Assume there is a node A that is attached to some guest device.
1. blockdev-add to create a filter node B that has A as its child.
2. qom-set to change the node attached to the guest device’s BlockBackend from A to B.”

?

Yes, sounds good, thanks

Quoted 42 lines
Quoted 36 lines

2.

[blk] --root--> [filer] --file--> [A]

Quoted 18 lines
Quoted 8 lines

2. qom-set, to change bs attached to root blk from original node to newly create filter.

---
  hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++--------

  1 file changed, 22 insertions(+), 8 deletions(-)

Looks good, just one question: (well, two, one was above)

Quoted 4 lines
hw/core/qdev-properties-system.c +0 -0
index 2760c21f11..7d97562654 100644

[...]

Quoted 11 lines

hw/core/qdev-properties-system.c static void release_drive(Object *obj, const char *name, void *opaque) · line 215

Quoted 8 lines
hw/core/qdev-properties-system.c +1 -0
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)
   const PropertyInfo qdev_prop_drive = {
       .name  = "str",
       .description = "Node name or ID of a block device to use as a backend",
+    .realized_set_allowed = true,
       .get   = get_drive,
       .set   = set_drive,
       .release = release_drive,

Why not for qdev_prop_drive_iothread?

Quoted 1 line

Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version

OK.  (I just saw it because it was right below this structure, too, it isn’t like I actually know what I’m saying.)

Max

Signature · 2 lines
Best regards,
Vladimir
#20 lore raw

PATCH 5/21 block: rename backup-top to copy-before-write

In reply to Vladimir Sementsov-Ogievskiy

We are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".

While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.

---
 block/{backup-top.h => copy-before-write.h} |  28 +++---
 block/backup.c                              |  22 ++---
 block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
 MAINTAINERS                                 |   4 +-
 block/meson.build                           |   2 +-
 tests/qemu-iotests/283                      |  35 +++----
 tests/qemu-iotests/283.out                  |   4 +-
 7 files changed, 95 insertions(+), 100 deletions(-)
 rename block/{backup-top.h => copy-before-write.h} (56%)
 rename block/{backup-top.c => copy-before-write.c} (62%)
block/copy-before-write.h renamed from block/backup-top.h +14 -14
similarity index 56%
rename from block/backup-top.h
rename to block/copy-before-write.h
index b28b0031c4..5977b7aa31 100644
@@ -1,10 +1,10 @@
 /*
- * backup-top filter driver
+ * copy-before-write filter driver
  *
  * The driver performs Copy-Before-Write (CBW) operation: it is injected above
  * some node, and before each write it copies _old_ data to the target node.
  *
- * Copyright (c) 2018-2019 Virtuozzo International GmbH.
+ * Copyright (c) 2018-2021 Virtuozzo International GmbH.
  *
  * Author:
  *  Sementsov-Ogievskiy Vladimir <[email protected]>
@@ -23,20 +23,20 @@
  * along with this program. If not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef BACKUP_TOP_H
-#define BACKUP_TOP_H
+#ifndef COPY_BEFORE_WRITE_H
+#define COPY_BEFORE_WRITE_H
 
 #include "block/block_int.h"
 #include "block/block-copy.h"
 
-BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
-                                         BlockDriverState *target,
-                                         const char *filter_node_name,
-                                         uint64_t cluster_size,
-                                         BackupPerf *perf,
-                                         BdrvRequestFlags write_flags,
-                                         BlockCopyState **bcs,
-                                         Error **errp);
-void bdrv_backup_top_drop(BlockDriverState *bs);
+BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
+                                  BlockDriverState *target,
+                                  const char *filter_node_name,
+                                  uint64_t cluster_size,
+                                  BackupPerf *perf,
+                                  BdrvRequestFlags write_flags,
+                                  BlockCopyState **bcs,
+                                  Error **errp);
+void bdrv_cbw_drop(BlockDriverState *bs);
 
-#endif /* BACKUP_TOP_H */
+#endif /* COPY_BEFORE_WRITE_H */
block/backup.c +11 -11
index 6cf2f974aa..0b5887269c 100644
@@ -27,13 +27,13 @@
 #include "qemu/bitmap.h"
 #include "qemu/error-report.h"
 
-#include "block/backup-top.h"
+#include "block/copy-before-write.h"
 
 #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16)
 
 typedef struct BackupBlockJob {
     BlockJob common;
-    BlockDriverState *backup_top;
+    BlockDriverState *cbw;
     BlockDriverState *source_bs;
     BlockDriverState *target_bs;
 
@@ -104,7 +104,7 @@ static void backup_clean(Job *job)
 {
     BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
     block_job_remove_all_bdrv(&s->common);
-    bdrv_backup_top_drop(s->backup_top);
+    bdrv_cbw_drop(s->cbw);
 }
 
 void backup_do_checkpoint(BlockJob *job, Error **errp)
@@ -408,7 +408,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     BackupBlockJob *job = NULL;
     int64_t cluster_size;
     BdrvRequestFlags write_flags;
-    BlockDriverState *backup_top = NULL;
+    BlockDriverState *cbw = NULL;
     BlockCopyState *bcs = NULL;
 
     assert(bs);
@@ -521,22 +521,22 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) |
                   (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
 
-    backup_top = bdrv_backup_top_append(bs, target, filter_node_name,
+    cbw = bdrv_cbw_append(bs, target, filter_node_name,
                                         cluster_size, perf,
                                         write_flags, &bcs, errp);
-    if (!backup_top) {
+    if (!cbw) {
         goto error;
     }
 
     /* job->len is fixed, so we can't allow resize */
-    job = block_job_create(job_id, &backup_job_driver, txn, backup_top,
+    job = block_job_create(job_id, &backup_job_driver, txn, cbw,
                            0, BLK_PERM_ALL,
                            speed, creation_flags, cb, opaque, errp);
     if (!job) {
         goto error;
     }
 
-    job->backup_top = backup_top;
+    job->cbw = cbw;
     job->source_bs = bs;
     job->target_bs = target;
     job->on_source_error = on_source_error;
@@ -552,7 +552,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     block_copy_set_progress_meter(bcs, &job->common.job.progress);
     block_copy_set_speed(bcs, speed);
 
-    /* Required permissions are already taken by backup-top target */
+    /* Required permissions are taken by copy-before-write filter target */
     block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
                        &error_abort);
 
@@ -562,8 +562,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     if (sync_bitmap) {
         bdrv_reclaim_dirty_bitmap(sync_bitmap, NULL);
     }
-    if (backup_top) {
-        bdrv_backup_top_drop(backup_top);
+    if (cbw) {
+        bdrv_cbw_drop(cbw);
     }
 
     return NULL;
block/copy-before-write.c renamed from block/backup-top.c +50 -50

206 lines not on this page · raw message

MAINTAINERS +2 -2
index 78561a223f..4321f67fef 100644
@@ -2281,8 +2281,8 @@ F: block/mirror.c
 F: qapi/job.json
 F: block/block-copy.c
 F: include/block/block-copy.c
-F: block/backup-top.h
-F: block/backup-top.c
+F: block/copy-before-write.h
+F: block/copy-before-write.c
 F: include/block/aio_task.h
 F: block/aio_task.c
 F: util/qemu-co-shared-resource.c
block/meson.build +1 -1
index e687c54dbc..6e27aa9624 100644
@@ -4,7 +4,7 @@ block_ss.add(files(
   'aio_task.c',
   'amend.c',
   'backup.c',
-  'backup-top.c',
+  'copy-before-write.c',
   'blkdebug.c',
   'blklogwrites.c',
   'blkverify.c',
tests/qemu-iotests/283 +15 -20
index 010c22f0a2..a09e0183ae 100755
@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 # group: auto quick
 #
-# Test for backup-top filter permission activation failure
+# Test for copy-before-write filter permission conflict
 #
 # Copyright (c) 2019 Virtuozzo International GmbH.
 #
@@ -31,13 +31,13 @@ size = 1024 * 1024
 """ Test description
 
 When performing a backup, all writes on the source subtree must go through the
-backup-top filter so it can copy all data to the target before it is changed.
-backup-top filter is appended above source node, to achieve this thing, so all
-parents of source node are handled. A configuration with side parents of source
-sub-tree with write permission is unsupported (we'd have append several
-backup-top filter like nodes to handle such parents). The test create an
-example of such configuration and checks that a backup is then not allowed
-(blockdev-backup command should fail).
+copy-before-write filter so it can copy all data to the target before it is
+changed.  copy-before-write filter is appended above source node, to achieve
+this thing, so all parents of source node are handled. A configuration with
+side parents of source sub-tree with write permission is unsupported (we'd have
+append several copy-before-write filter like nodes to handle such parents). The
+test create an example of such configuration and checks that a backup is then
+not allowed (blockdev-backup command should fail).
 
 The configuration:
 
@@ -57,11 +57,10 @@ The configuration:
                         │    base     │ ◀──────────── │ other │
                         └─────────────┘               └───────┘
 
-On activation (see .active field of backup-top state in block/backup-top.c),
-backup-top is going to unshare write permission on its source child. Write
-unsharing will be propagated to the "source->base" link and will conflict with
-other node write permission. So permission update will fail and backup job will
-not be started.
+copy-before-write filter wants to unshare write permission on its source child.
+Write unsharing will be propagated to the "source->base" link and will conflict
+with other node write permission. So permission update will fail and backup job
+will not be started.
 
 Note, that the only thing which prevents backup of running on such
 configuration is default permission propagation scheme. It may be altered by
@@ -99,13 +98,9 @@ vm.qmp_log('blockdev-backup', sync='full', device='source', target='target')
 vm.shutdown()
 
 
-print('\n=== backup-top should be gone after job-finalize ===\n')
+print('\n=== copy-before-write filter should be gone after job-finalize ===\n')
 
-# Check that the backup-top node is gone after job-finalize.
-#
-# During finalization, the node becomes inactive and can no longer
-# function.  If it is still present, new parents might be attached, and
-# there would be no meaningful way to handle their I/O requests.
+# Check that the copy-before-write node is gone after job-finalize.
 
 vm = iotests.VM()
 vm.launch()
@@ -131,7 +126,7 @@ vm.qmp_log('blockdev-backup',
 
 vm.event_wait('BLOCK_JOB_PENDING', 5.0)
 
-# The backup-top filter should still be present prior to finalization
+# The copy-before-write filter should still be present prior to finalization
 assert vm.node_info('backup-filter') is not None
 
 vm.qmp_log('job-finalize', id='backup')
tests/qemu-iotests/283.out +2 -2
index 97e62a4c94..e08f807dab 100644
@@ -5,9 +5,9 @@
 {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
 {"return": {}}
 {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot append backup-top filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
 
-=== backup-top should be gone after job-finalize ===
+=== copy-before-write filter should be gone after job-finalize ===
 
 {"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name": "source"}}
 {"return": {}}
Signature · 1 line
2.29.2
#21 lore raw

Re: [PATCH 05/21] block: rename backup-top to copy-before-write

Quoted 3 lines · Vladimir Sementsov-Ogievskiy

We are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".

Is this safe? The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)

(Regardless, renaming files and so on is fine, of course.)

Quoted 16 lines

While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.

---
  block/{backup-top.h => copy-before-write.h} |  28 +++---
  block/backup.c                              |  22 ++---
  block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
  MAINTAINERS                                 |   4 +-
  block/meson.build                           |   2 +-
  tests/qemu-iotests/283                      |  35 +++----
  tests/qemu-iotests/283.out                  |   4 +-
  7 files changed, 95 insertions(+), 100 deletions(-)

rename block/{backup-top.h => copy-before-write.h} (56%) rename block/{backup-top.c => copy-before-write.c} (62%)

[...]

Quoted 7 lines
block/copy-before-write.c renamed from block/backup-top.c +0 -0
similarity index 62%
rename from block/backup-top.c
rename to block/copy-before-write.c
index 425e3778be..40e91832d7 100644

[...]

Quoted 1 line
block/copy-before-write.c renamed from block/backup-top.c +0 -0
@@ -32,25 +32,25 @@

[...]

block/copy-before-write.c line 51

Quoted 4 lines
block/copy-before-write.c renamed from block/backup-top.c +2 -2
-static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
-                                       uint64_t bytes, BdrvRequestFlags flags)
+static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
+                                uint64_t bytes, BdrvRequestFlags flags)

I’m sure you noticed it, too, but cbw_cbw() is weird. Perhaps cbw_do_cbw() at least?

Max

#22 lore raw

Re: [PATCH 05/21] block: rename backup-top to copy-before-write

Quoted 8 lines · Max Reitz
Quoted 3 lines · Vladimir Sementsov-Ogievskiy

We are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".

Is this safe?  The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)

(Regardless, renaming files and so on is fine, of course.)

Hmmm. I don't know.

It was visible yes.. But we've never documented it. And if someone depends on name of the format of the filter automatically inserted during backup job, it's a kind of "undocumented feature" use..

Another change I is changing child from backing to file in 11, from this point of view it's unsafe too. But ше even more reasonable than good name: having all public filters behave similar is a very good thing.

So, may be it a bit risky, but I think good interface worth that risk. And we always can say "sorry guys, but that was not documented, we didn't promise anything".

But I'm OK to go on with "backup-top" and "backing", is someone has strict opinion about this.

Quoted 34 lines
Quoted 17 lines

While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.

---
  block/{backup-top.h => copy-before-write.h} |  28 +++--- block/backup.c                              |  22 ++--- block/{backup-top.c => copy-before-write.c} | 100 ++++++++++---------- MAINTAINERS                                 |   4 +- block/meson.build                           |   2 +- tests/qemu-iotests/283                      |  35 +++---- tests/qemu-iotests/283.out                  |   4 +-

  7 files changed, 95 insertions(+), 100 deletions(-)

  rename block/{backup-top.h => copy-before-write.h} (56%) rename block/{backup-top.c => copy-before-write.c} (62%)

[...]

Quoted 7 lines
block/copy-before-write.c renamed from block/backup-top.c +0 -0
similarity index 62%
rename from block/backup-top.c
rename to block/copy-before-write.c
index 425e3778be..40e91832d7 100644

[...]

Quoted 1 line
block/copy-before-write.c renamed from block/backup-top.c +0 -0
@@ -32,25 +32,25 @@

[...]

Quoted 7 lines

block/copy-before-write.c line 51

Quoted 4 lines
block/copy-before-write.c renamed from block/backup-top.c +2 -2
-static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
-                                       uint64_t bytes, BdrvRequestFlags flags)
+static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
+                                uint64_t bytes, BdrvRequestFlags flags)

I’m sure you noticed it, too, but cbw_cbw() is weird.  Perhaps cbw_do_cbw() at least?

OK. Maybe even cbw_do_copy_before_write()

Signature · 2 lines
Best regards,
Vladimir
#23 lore raw

Re: [PATCH 05/21] block: rename backup-top to copy-before-write

Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 8 lines · Max Reitz
Quoted 3 lines · Vladimir Sementsov-Ogievskiy

We are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".

Is this safe?  The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)

(Regardless, renaming files and so on is fine, of course.)

Quoted 7 lines

Hmmm. I don't know.

It was visible yes.. But we've never documented it. And if someone depends on name of the format of the filter automatically inserted during backup job, it's a kind of "undocumented feature" use..

Another change I is changing child from backing to file in 11, from this point of view it's unsafe too. But ше even more reasonable than good name: having all public filters behave similar is a very good thing.

So, may be it a bit risky, but I think good interface worth that risk. And we always can say "sorry guys, but that was not documented, we didn't promise anything".

OK.

Max

Quoted 49 lines

But I'm OK to go on with "backup-top" and "backing", is someone has strict opinion about this.

Quoted 34 lines
Quoted 17 lines

While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.

---
  block/{backup-top.h => copy-before-write.h} |  28 +++--- block/backup.c                              |  22 ++--- block/{backup-top.c => copy-before-write.c} | 100 ++++++++++---------- MAINTAINERS                                 |   4 +- block/meson.build                           |   2 +- tests/qemu-iotests/283                      |  35 +++---- tests/qemu-iotests/283.out                  |   4 +-

  7 files changed, 95 insertions(+), 100 deletions(-)

  rename block/{backup-top.h => copy-before-write.h} (56%) rename block/{backup-top.c => copy-before-write.c} (62%)

[...]

Quoted 7 lines
block/copy-before-write.c renamed from block/backup-top.c +0 -0
similarity index 62%
rename from block/backup-top.c
rename to block/copy-before-write.c
index 425e3778be..40e91832d7 100644

[...]

Quoted 1 line
block/copy-before-write.c renamed from block/backup-top.c +0 -0
@@ -32,25 +32,25 @@

[...]

Quoted 9 lines
Quoted 6 lines

-static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
-                                       uint64_t bytes, BdrvRequestFlags flags)
+static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset, +                                uint64_t bytes, BdrvRequestFlags flags)

I’m sure you noticed it, too, but cbw_cbw() is weird.  Perhaps cbw_do_cbw() at least?

OK. Maybe even cbw_do_copy_before_write()

#24 lore raw

PATCH 6/21 block/backup: drop support for copy_range

In reply to Vladimir Sementsov-Ogievskiy

copy_range is not a default behavior since 6a30f663d4c0b3c, and it's now available only though x-perf experimantal argument, so it's OK to drop it.

Even when backup is used to copy disk to same filesystem, and filesystem supports zero-copy copy_range, copy_range is probably not what we want for backup: backup has good property of making a copy of active disk, with no impact to active disk itself (unlike creating a snapshot). And if copy_range instead of copying data adds fs-level references, and on next guest write COW operation occurs, it's seems most possible, that new block will be allocated for original vm disk, not for backup disk. Thus, fragmentation of original disk will increase.

We can simply add support back on demand. Now we want to publish copy-before-write filter, and instead of thinking how to pass use-copy-range argument to block-copy (create x-block-copy parameter for new public filter driver, or may be set it by hand after filter node creation?), instead of this let's just drop copy-range support in backup for now.

After this patch copy-range support in block-copy becomes unused. Let's keep it for a while, it won't hurt:

1. If there would be request for supporting copy_range in backup (and/or in a new public copy-before-write filter), it will be easy to satisfy it.

2. Probably, qemu-img convert will reuse block-copy, and qemu-img has option to enable copy-range. qemu-img convert is not a backup, and copy_range may be more reasonable for some cases in context of qemu-img convert.

---
 block/copy-before-write.h | 1 -
 block/backup.c            | 3 +--
 block/copy-before-write.c | 4 +---
 3 files changed, 2 insertions(+), 6 deletions(-)
block/copy-before-write.h +0 -1
index 5977b7aa31..e284dfb6a7 100644
@@ -33,7 +33,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
                                   uint64_t cluster_size,
-                                  BackupPerf *perf,
                                   BdrvRequestFlags write_flags,
                                   BlockCopyState **bcs,
                                   Error **errp);
block/backup.c +1 -2
index 0b5887269c..7c7a88a0f7 100644
@@ -522,8 +522,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
 
     cbw = bdrv_cbw_append(bs, target, filter_node_name,
-                                        cluster_size, perf,
-                                        write_flags, &bcs, errp);
+                          cluster_size, write_flags, &bcs, errp);
     if (!cbw) {
         goto error;
     }
block/copy-before-write.c +1 -3
index 40e91832d7..fe087a7347 100644
@@ -170,7 +170,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
                                   uint64_t cluster_size,
-                                  BackupPerf *perf,
                                   BdrvRequestFlags write_flags,
                                   BlockCopyState **bcs,
                                   Error **errp)
@@ -217,8 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
 
     state->cluster_size = cluster_size;
     state->bcs = block_copy_state_new(top->backing, state->target,
-                                      cluster_size, perf->use_copy_range,
-                                      write_flags, errp);
+                                      cluster_size, false, write_flags, errp);
     if (!state->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
Signature · 1 line
2.29.2
Reviewed-by on v1 6/21
#25 lore raw

Re: [PATCH 06/21] block/backup: drop support for copy_range

Quoted 4 lines · Vladimir Sementsov-Ogievskiy

copy_range is not a default behavior since 6a30f663d4c0b3c, and it's now available only though x-perf experimantal argument, so it's OK to drop it.

Quoted 9 lines

Even when backup is used to copy disk to same filesystem, and filesystem supports zero-copy copy_range, copy_range is probably not what we want for backup: backup has good property of making a copy of active disk, with no impact to active disk itself (unlike creating a snapshot). And if copy_range instead of copying data adds fs-level references, and on next guest write COW operation occurs, it's seems most possible, that new block will be allocated for original vm disk, not for backup disk. Thus, fragmentation of original disk will increase.

Good point.

Quoted 19 lines

We can simply add support back on demand. Now we want to publish copy-before-write filter, and instead of thinking how to pass use-copy-range argument to block-copy (create x-block-copy parameter for new public filter driver, or may be set it by hand after filter node creation?), instead of this let's just drop copy-range support in backup for now.

After this patch copy-range support in block-copy becomes unused. Let's keep it for a while, it won't hurt:

1. If there would be request for supporting copy_range in backup (and/or in a new public copy-before-write filter), it will be easy to satisfy it.

2. Probably, qemu-img convert will reuse block-copy, and qemu-img has option to enable copy-range. qemu-img convert is not a backup, and copy_range may be more reasonable for some cases in context of qemu-img convert.

Quoted 6 lines
---
  block/copy-before-write.h | 1 -
  block/backup.c            | 3 +--
  block/copy-before-write.c | 4 +---
  3 files changed, 2 insertions(+), 6 deletions(-)
#26 lore raw

PATCH 7/21 block-copy: always set BDRV_REQ_SERIALISING flag

In reply to Vladimir Sementsov-Ogievskiy

It won't hurt in common case, so let's not bother with detecting image fleecing.

Also, we want to simplify initialization interface of copy-before-write filter as we are going to make it public.

---
 block/copy-before-write.h  |  2 +-
 include/block/block-copy.h |  3 +--
 block/backup.c             | 20 +-------------------
 block/block-copy.c         | 29 ++++++++++++++++++++++++++---
 block/copy-before-write.c  |  4 ++--
 5 files changed, 31 insertions(+), 27 deletions(-)
block/copy-before-write.h +1 -1
index e284dfb6a7..538aab8bdb 100644
@@ -33,7 +33,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
                                   uint64_t cluster_size,
-                                  BdrvRequestFlags write_flags,
+                                  bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp);
 void bdrv_cbw_drop(BlockDriverState *bs);
include/block/block-copy.h +1 -2
index 338f2ea7fd..c013a20e1e 100644
@@ -24,8 +24,7 @@ typedef struct BlockCopyCallState BlockCopyCallState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
                                      int64_t cluster_size, bool use_copy_range,
-                                     BdrvRequestFlags write_flags,
-                                     Error **errp);
+                                     bool compress, Error **errp);
 
 void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
 
block/backup.c +1 -19
index 7c7a88a0f7..fe685e411b 100644
@@ -407,7 +407,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     int64_t len, target_len;
     BackupBlockJob *job = NULL;
     int64_t cluster_size;
-    BdrvRequestFlags write_flags;
     BlockDriverState *cbw = NULL;
     BlockCopyState *bcs = NULL;
 
@@ -504,25 +503,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         goto error;
     }
 
-    /*
-     * If source is in backing chain of target assume that target is going to be
-     * used for "image fleecing", i.e. it should represent a kind of snapshot of
-     * source at backup-start point in time. And target is going to be read by
-     * somebody (for example, used as NBD export) during backup job.
-     *
-     * In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid
-     * intersection of backup writes and third party reads from target,
-     * otherwise reading from target we may occasionally read already updated by
-     * guest data.
-     *
-     * For more information see commit f8d59dfb40bb and test
-     * tests/qemu-iotests/222
-     */
-    write_flags = (bdrv_chain_contains(target, bs) ? BDRV_REQ_SERIALISING : 0) |
-                  (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
-
     cbw = bdrv_cbw_append(bs, target, filter_node_name,
-                          cluster_size, write_flags, &bcs, errp);
+                          cluster_size, compress, &bcs, errp);
     if (!cbw) {
         goto error;
     }
block/block-copy.c +26 -3
index 9b4af00614..daa1a2bf9f 100644
@@ -245,7 +245,7 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target)
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
                                      int64_t cluster_size, bool use_copy_range,
-                                     BdrvRequestFlags write_flags, Error **errp)
+                                     bool compress, Error **errp)
 {
     BlockCopyState *s;
     BdrvDirtyBitmap *copy_bitmap;
@@ -257,6 +257,28 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
     }
     bdrv_disable_dirty_bitmap(copy_bitmap);
 
+    /*
+     * Why we always set BDRV_REQ_SERIALISING write flag:
+     *
+     * Assume source is in backing chain of target assume that target is going
+     * to be used for "image fleecing", i.e. it should represent a kind of
+     * snapshot of source at backup-start point in time. And target is going to
+     * be read by somebody (for example, used as NBD export) during backup job.
+     *
+     * In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid
+     * intersection of backup writes and third party reads from target,
+     * otherwise reading from target we may occasionally read already updated by
+     * guest data.
+     *
+     * For more information see commit f8d59dfb40bb and test
+     * tests/qemu-iotests/222
+     *
+     * Other cases? The only known reasonable case is "just copy to target, and
+     * target is not used for something else". In this case BDRV_REQ_SERIALISING
+     * change nothing, so let's not bother with detecting the "image fleecing"
+     * case and enabling BDRV_REQ_SERIALISING only for it.
+     */
+
     s = g_new(BlockCopyState, 1);
     *s = (BlockCopyState) {
         .source = source,
@@ -264,7 +286,8 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
         .copy_bitmap = copy_bitmap,
         .cluster_size = cluster_size,
         .len = bdrv_dirty_bitmap_size(copy_bitmap),
-        .write_flags = write_flags,
+        .write_flags = BDRV_REQ_SERIALISING |
+            (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
         .mem = shres_create(BLOCK_COPY_MAX_MEM),
     };
 
@@ -277,7 +300,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
          */
         s->use_copy_range = false;
         s->copy_size = cluster_size;
-    } else if (write_flags & BDRV_REQ_WRITE_COMPRESSED) {
+    } else if (compress) {
         /* Compression supports only cluster-size writes and no copy-range. */
         s->use_copy_range = false;
         s->copy_size = cluster_size;
block/copy-before-write.c +2 -2
index fe087a7347..19472912d1 100644
@@ -170,7 +170,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
                                   uint64_t cluster_size,
-                                  BdrvRequestFlags write_flags,
+                                  bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp)
 {
@@ -216,7 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
 
     state->cluster_size = cluster_size;
     state->bcs = block_copy_state_new(top->backing, state->target,
-                                      cluster_size, false, write_flags, errp);
+                                      cluster_size, false, compress, errp);
     if (!state->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
Signature · 1 line
2.29.2
Reviewed-by on v1 7/21
#27 lore raw

Re: [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag

Quoted 6 lines · Vladimir Sementsov-Ogievskiy

It won't hurt in common case, so let's not bother with detecting image fleecing.

Also, we want to simplify initialization interface of copy-before-write filter as we are going to make it public.

Quoted 8 lines
---
  block/copy-before-write.h  |  2 +-
  include/block/block-copy.h |  3 +--
  block/backup.c             | 20 +-------------------
  block/block-copy.c         | 29 ++++++++++++++++++++++++++---
  block/copy-before-write.c  |  4 ++--
  5 files changed, 31 insertions(+), 27 deletions(-)
#28 lore raw

PATCH 8/21 block/backup: stricter backup_calculate_cluster_size()

In reply to Vladimir Sementsov-Ogievskiy

No reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's just error-out, it's simpler and safer.

---
 block/backup.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)
block/backup.c +5 -9
index fe685e411b..fe7a1f1e37 100644
@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
      * targets with a backing file, try to avoid COW if possible.
      */
     ret = bdrv_get_info(target, &bdi);
-    if (ret == -ENOTSUP && !target_does_cow) {
+    if (ret < 0 && ret != -ENOTSUP) {
+        error_setg_errno(errp, -ret, "Failed to get target info");
+        return ret;
+    } else if (ret == -ENOTSUP && !target_does_cow) {
         /* Cluster size is not defined */
         warn_report("The target block device doesn't provide "
                     "information about the block size and it doesn't have a "
@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
                     "this default, the backup may be unusable",
                     BACKUP_CLUSTER_SIZE_DEFAULT);
         return BACKUP_CLUSTER_SIZE_DEFAULT;
-    } else if (ret < 0 && !target_does_cow) {
-        error_setg_errno(errp, -ret,
-            "Couldn't determine the cluster size of the target image, "
-            "which has no backing file");
-        error_append_hint(errp,
-            "Aborting, since this may create an unusable destination image\n");
-        return ret;
-    } else if (ret < 0 && target_does_cow) {
+    } else if (ret == -ENOTSUP && target_does_cow) {
         /* Not fatal; just trudge on ahead. */
         return BACKUP_CLUSTER_SIZE_DEFAULT;
     }
Signature · 1 line
2.29.2
#29 lore raw

Re: [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()

Quoted 2 lines · Vladimir Sementsov-Ogievskiy

No reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's just error-out, it's simpler and safer.

Hm, doesn’t look that much simpler to me. Not sure how much safer it is, because the point was that in the target_does_cow case, we would like a cluster size hint, but it isn’t necessary. So if we don’t get one, regardless of the reason, we use the default cluster size. I don’t know why ENOTSUP should be treated in a special way there.

So I don’t know.

Max

Quoted 38 lines
---
  block/backup.c | 14 +++++---------
  1 file changed, 5 insertions(+), 9 deletions(-)
block/backup.c +5 -9
index fe685e411b..fe7a1f1e37 100644
@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
       * targets with a backing file, try to avoid COW if possible.
       */
      ret = bdrv_get_info(target, &bdi);
-    if (ret == -ENOTSUP && !target_does_cow) {
+    if (ret < 0 && ret != -ENOTSUP) {
+        error_setg_errno(errp, -ret, "Failed to get target info");
+        return ret;
+    } else if (ret == -ENOTSUP && !target_does_cow) {
          /* Cluster size is not defined */
          warn_report("The target block device doesn't provide "
                      "information about the block size and it doesn't have a "
@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
                      "this default, the backup may be unusable",
                      BACKUP_CLUSTER_SIZE_DEFAULT);
          return BACKUP_CLUSTER_SIZE_DEFAULT;
-    } else if (ret < 0 && !target_does_cow) {
-        error_setg_errno(errp, -ret,
-            "Couldn't determine the cluster size of the target image, "
-            "which has no backing file");
-        error_append_hint(errp,
-            "Aborting, since this may create an unusable destination image\n");
-        return ret;
-    } else if (ret < 0 && target_does_cow) {
+    } else if (ret == -ENOTSUP && target_does_cow) {
          /* Not fatal; just trudge on ahead. */
          return BACKUP_CLUSTER_SIZE_DEFAULT;
      }
#30 lore raw

Re: [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()

Quoted 8 lines · Max Reitz
Quoted 2 lines · Vladimir Sementsov-Ogievskiy

No reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's just error-out, it's simpler and safer.

Hm, doesn’t look that much simpler to me.  Not sure how much safer it is, because the point was that in the target_does_cow case, we would like a cluster size hint, but it isn’t necessary.  So if we don’t get one, regardless of the reason, we use the default cluster size.  I don’t know why ENOTSUP should be treated in a special way there.

So I don’t know.

I'm probably OK to drop this for now and don't care. Still, I can share what brings me to this:

First I thought that cluster size should be easily available for any driver:

protocol drivers and not-backing-supporting format drivers can set it to 1 or to request_alignment, if they don't have a "cluster" in mind.

backing-supporting format drivers should of course provide actual cluster size

And I decided to just add bs->cluster_size variable, set on driver open, to simplify the whole thing and make it clean. Then, most this detect-cluster-size function would be just dropped.

But it occurs, that there is one driver, that has a good and rather tricky reason for ENOTSUP: vmdk can have several extents with different cluster size..

So I give up refactored, and finished with this one patch. It can be simply dropped, I am not really a fan of it..

Quoted 40 lines
Quoted 39 lines

---
  block/backup.c | 14 +++++---------

  1 file changed, 5 insertions(+), 9 deletions(-)
block/backup.c +4 -1
index fe685e411b..fe7a1f1e37 100644
@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
        * targets with a backing file, try to avoid COW if possible.
        */
       ret = bdrv_get_info(target, &bdi);
-    if (ret == -ENOTSUP && !target_does_cow) {
+    if (ret < 0 && ret != -ENOTSUP) {
+        error_setg_errno(errp, -ret, "Failed to get target info");
+        return ret;
+    } else if (ret == -ENOTSUP && !target_does_cow) {
           /* Cluster size is not defined */

          warn_report("The target block device doesn't provide " "information about the block size and it doesn't have a "

block/backup.c +1 -8
@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,
                       "this default, the backup may be unusable",
                       BACKUP_CLUSTER_SIZE_DEFAULT);
           return BACKUP_CLUSTER_SIZE_DEFAULT;
-    } else if (ret < 0 && !target_does_cow) {
-        error_setg_errno(errp, -ret,
-            "Couldn't determine the cluster size of the target image, "
-            "which has no backing file");
-        error_append_hint(errp,
-            "Aborting, since this may create an unusable destination image\n");
-        return ret;
-    } else if (ret < 0 && target_does_cow) {
+    } else if (ret == -ENOTSUP && target_does_cow) {
           /* Not fatal; just trudge on ahead. */
           return BACKUP_CLUSTER_SIZE_DEFAULT;
       }
Signature · 2 lines
Best regards,
Vladimir
#31 lore raw

PATCH 9/21 block/backup: move cluster size calculation to block-copy

In reply to Vladimir Sementsov-Ogievskiy

The main consumer of cluster-size is block-copy. Let's calculate it here instead of passing through backup-top.

We are going to publish copy-before-write filter soon, so it will be created through options. But we don't want for now to make explicit option for cluster-size, let's continue to calculate it automatically. So, now is the time to get rid of cluster_size argument for bdrv_cbw_append().

---
 block/copy-before-write.h  |  1 -
 include/block/block-copy.h |  5 ++--
 block/backup.c             | 58 ++++++--------------------------------
 block/block-copy.c         | 47 +++++++++++++++++++++++++++++-
 block/copy-before-write.c  | 10 +++----
 5 files changed, 62 insertions(+), 59 deletions(-)
block/copy-before-write.h +0 -1
index 538aab8bdb..b386fd8f01 100644
@@ -32,7 +32,6 @@
 BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
-                                  uint64_t cluster_size,
                                   bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp);
include/block/block-copy.h +3 -2
index c013a20e1e..8138686eb4 100644
@@ -23,8 +23,8 @@ typedef struct BlockCopyState BlockCopyState;
 typedef struct BlockCopyCallState BlockCopyCallState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size, bool use_copy_range,
-                                     bool compress, Error **errp);
+                                     bool use_copy_range, bool compress,
+                                     Error **errp);
 
 void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
 
@@ -86,6 +86,7 @@ void block_copy_kick(BlockCopyCallState *call_state);
 void block_copy_call_cancel(BlockCopyCallState *call_state);
 
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
+int64_t block_copy_cluster_size(BlockCopyState *s);
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
 
 #endif /* BLOCK_COPY_H */
block/backup.c +9 -49

92 lines not on this page · raw message

block/block-copy.c +46 -1

71 lines not on this page · raw message

block/copy-before-write.c +4 -6
index 19472912d1..4776172f77 100644
@@ -37,7 +37,6 @@
 typedef struct BDRVCopyBeforeWriteState {
     BlockCopyState *bcs;
     BdrvChild *target;
-    int64_t cluster_size;
 } BDRVCopyBeforeWriteState;
 
 static coroutine_fn int cbw_co_preadv(
@@ -52,13 +51,14 @@ static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
     uint64_t off, end;
+    int64_t cluster_size = block_copy_cluster_size(s->bcs);
 
     if (flags & BDRV_REQ_WRITE_UNCHANGED) {
         return 0;
     }
 
-    off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
-    end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
+    off = QEMU_ALIGN_DOWN(offset, cluster_size);
+    end = QEMU_ALIGN_UP(offset + bytes, cluster_size);
 
     return block_copy(s->bcs, off, end - off, true);
 }
@@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = {
 BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   BlockDriverState *target,
                                   const char *filter_node_name,
-                                  uint64_t cluster_size,
                                   bool compress,
                                   BlockCopyState **bcs,
                                   Error **errp)
@@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     }
     appended = true;
 
-    state->cluster_size = cluster_size;
     state->bcs = block_copy_state_new(top->backing, state->target,
-                                      cluster_size, false, compress, errp);
+                                      false, compress, errp);
     if (!state->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
Signature · 1 line
2.29.2
Reviewed-by on v1 9/21
#32 lore raw

Re: [PATCH 09/21] block/backup: move cluster size calculation to block-copy

Quoted 9 lines · Vladimir Sementsov-Ogievskiy

The main consumer of cluster-size is block-copy. Let's calculate it here instead of passing through backup-top.

We are going to publish copy-before-write filter soon, so it will be created through options. But we don't want for now to make explicit option for cluster-size, let's continue to calculate it automatically. So, now is the time to get rid of cluster_size argument for bdrv_cbw_append().

Quoted 8 lines
---
  block/copy-before-write.h  |  1 -
  include/block/block-copy.h |  5 ++--
  block/backup.c             | 58 ++++++--------------------------------
  block/block-copy.c         | 47 +++++++++++++++++++++++++++++-
  block/copy-before-write.c  | 10 +++----
  5 files changed, 62 insertions(+), 59 deletions(-)
#33 lore raw

PATCH 10/21 block/copy-before-write: relax permission requirements when no parents

In reply to Vladimir Sementsov-Ogievskiy

We are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed.

Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent.

Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node.

---
 block/copy-before-write.c  | 8 +++++---
 tests/qemu-iotests/283.out | 2 +-
 2 files changed, 6 insertions(+), 4 deletions(-)
block/copy-before-write.c +5 -3
index 4776172f77..af2bb97a30 100644
@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
         bdrv_default_perms(bs, c, role, reopen_queue,
                            perm, shared, nperm, nshared);
 
-        if (perm & BLK_PERM_WRITE) {
-            *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+        if (!QLIST_EMPTY(&bs->parents)) {
+            if (perm & BLK_PERM_WRITE) {
+                *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+            }
+            *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
         }
-        *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
     }
 }
 
tests/qemu-iotests/283.out +1 -1
index e08f807dab..d5350ce7a7 100644
@@ -5,7 +5,7 @@
 {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
 {"return": {}}
 {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}
 
 === copy-before-write filter should be gone after job-finalize ===
 
Signature · 1 line
2.29.2
Reviewed-by on v1 10/21
#34 lore raw

Re: [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents

Quoted 23 lines · Vladimir Sementsov-Ogievskiy

We are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed.

Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent.

Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node.

---
  block/copy-before-write.c  | 8 +++++---
  tests/qemu-iotests/283.out | 2 +-
  2 files changed, 6 insertions(+), 4 deletions(-)
block/copy-before-write.c +0 -0
index 4776172f77..af2bb97a30 100644

block/copy-before-write.c static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, · line 145

Quoted 6 lines
block/copy-before-write.c +1 -2
@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
          bdrv_default_perms(bs, c, role, reopen_queue,
                             perm, shared, nperm, nshared);
 
-        if (perm & BLK_PERM_WRITE) {
-            *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+        if (!QLIST_EMPTY(&bs->parents)) {

I understand this works because with the transactional system, at the time the permissions are checked, the graph has already been changed, yes?

I was wondering still whether there was any way to express this through permissions alone. I guess we could check
“perm & BLK_PERM_CONSISTENT_READ”, but that would actually be just a crutch to see whether there are any parents. I suppose this really is about whether there are parents or not. So:

Quoted 23 lines
block/copy-before-write.c +4 -1
+            if (perm & BLK_PERM_WRITE) {
+                *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+            }
+            *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
          }
-        *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
      }
  }
 
tests/qemu-iotests/283.out +1 -1
index e08f807dab..d5350ce7a7 100644
@@ -5,7 +5,7 @@
  {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
  {"return": {}}
  {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}
 
  === copy-before-write filter should be gone after job-finalize ===
 
#35 lore raw

Re: [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents

Quoted 19 lines · Max Reitz
Quoted 18 lines · Vladimir Sementsov-Ogievskiy

We are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed.

Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent.

Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node.

---
  block/copy-before-write.c  | 8 +++++---
  tests/qemu-iotests/283.out | 2 +-

  2 files changed, 6 insertions(+), 4 deletions(-)
Quoted 12 lines

block/copy-before-write.c static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, · line 144

Quoted 10 lines
block/copy-before-write.c +1 -2
index 4776172f77..af2bb97a30 100644
@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
           bdrv_default_perms(bs, c, role, reopen_queue,
                              perm, shared, nperm, nshared);
-        if (perm & BLK_PERM_WRITE) {
-            *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+        if (!QLIST_EMPTY(&bs->parents)) {

I understand this works because with the transactional system, at the time the permissions are checked, the graph has already been changed, yes?

Right

Quoted 3 lines

I was wondering still whether there was any way to express this through permissions alone.  I guess we could check
“perm & BLK_PERM_CONSISTENT_READ”, but that would actually be just a crutch to see whether there are any parents.  I suppose this really is about whether there are parents or not.  So:

It's a workaround for blockdev-add to work. It's a workaround, because it would be better for the filter to never share WRITE. But this way we'll have to make an interface to transactionally create and insert filter. Probably we will at some point, than this workaround could be dropped. So, if we have to special-case a situation when there no parents, let's check exactly this, to be more strict.

Quoted 24 lines
Quoted 20 lines
block/copy-before-write.c +4 -1
+            if (perm & BLK_PERM_WRITE) {
+                *nperm = *nperm | BLK_PERM_CONSISTENT_READ;
+            }
+            *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
           }
-        *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE);
       }
   }
tests/qemu-iotests/283.out +1 -1
index e08f807dab..d5350ce7a7 100644
@@ -5,7 +5,7 @@
   {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}}
   {"return": {}}
   {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}

  === copy-before-write filter should be gone after job-finalize ===

Signature · 2 lines
Best regards,
Vladimir
#36 lore raw

PATCH 11/21 block/copy-before-write: use file child instead of backing

In reply to Vladimir Sementsov-Ogievskiy

We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead.

---
 block/copy-before-write.c | 40 ++++++++++++++++++++++-----------------
 1 file changed, 23 insertions(+), 17 deletions(-)
block/copy-before-write.c +23 -17
index af2bb97a30..2f06a524b8 100644
@@ -43,7 +43,7 @@ static coroutine_fn int cbw_co_preadv(
         BlockDriverState *bs, uint64_t offset, uint64_t bytes,
         QEMUIOVector *qiov, int flags)
 {
-    return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags);
+    return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
 }
 
 static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
@@ -71,7 +71,7 @@ static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,
         return ret;
     }
 
-    return bdrv_co_pdiscard(bs->backing, offset, bytes);
+    return bdrv_co_pdiscard(bs->file, offset, bytes);
 }
 
 static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,
@@ -82,7 +82,7 @@ static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,
         return ret;
     }
 
-    return bdrv_co_pwrite_zeroes(bs->backing, offset, bytes, flags);
+    return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags);
 }
 
 static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,
@@ -95,29 +95,22 @@ static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,
         return ret;
     }
 
-    return bdrv_co_pwritev(bs->backing, offset, bytes, qiov, flags);
+    return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags);
 }
 
 static int coroutine_fn cbw_co_flush(BlockDriverState *bs)
 {
-    if (!bs->backing) {
+    if (!bs->file) {
         return 0;
     }
 
-    return bdrv_co_flush(bs->backing->bs);
+    return bdrv_co_flush(bs->file->bs);
 }
 
 static void cbw_refresh_filename(BlockDriverState *bs)
 {
-    if (bs->backing == NULL) {
-        /*
-         * we can be here after failed bdrv_attach_child in
-         * bdrv_set_backing_hd
-         */
-        return;
-    }
     pstrcpy(bs->exact_filename, sizeof(bs->exact_filename),
-            bs->backing->bs->filename);
+            bs->file->bs->filename);
 }
 
 static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
@@ -186,6 +179,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name,
                                BDRV_O_RDWR, errp);
     if (!top) {
+        error_prepend(errp, "Cannot open driver: ");
         return NULL;
     }
 
@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
                                       BDRV_CHILD_DATA, errp);
     if (!state->target) {
+        error_prepend(errp, "Cannot attach target child: ");
         bdrv_unref(target);
         bdrv_unref(top);
         return NULL;
     }
 
+    bdrv_ref(source);
+    top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
+                                  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                                  errp);
+    if (!top->file) {
+        error_prepend(errp, "Cannot attach file child: ");
+        bdrv_unref(source);
+        bdrv_unref(top);
+        return NULL;
+    }
+
     bdrv_drained_begin(source);
 
-    ret = bdrv_append(top, source, errp);
+    ret = bdrv_replace_node(source, top, errp);
     if (ret < 0) {
         error_prepend(errp, "Cannot append copy-before-write filter: ");
         goto fail;
     }
     appended = true;
 
-    state->bcs = block_copy_state_new(top->backing, state->target,
-                                      false, compress, errp);
+    state->bcs = block_copy_state_new(top->file, state->target, false, compress,
+                                      errp);
     if (!state->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
Signature · 1 line
2.29.2
#37 lore raw

Re: [PATCH 11/21] block/copy-before-write: use file child instead of backing

Quoted 13 lines · Vladimir Sementsov-Ogievskiy

We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead.

---
  block/copy-before-write.c | 40 ++++++++++++++++++++++-----------------
  1 file changed, 23 insertions(+), 17 deletions(-)
block/copy-before-write.c +0 -0
index af2bb97a30..2f06a524b8 100644

[...]

Quoted 7 lines
block/copy-before-write.c +1 -0
@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
      state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
                                        BDRV_CHILD_DATA, errp);
      if (!state->target) {
+        error_prepend(errp, "Cannot attach target child: ");
          bdrv_unref(target);
          bdrv_unref(top);

block/copy-before-write.c BlockDriverState *bdrv_cbw_append(BlockDriverState *source, · line 210

Quoted 10 lines
block/copy-before-write.c +7 -0
          return NULL;
      }
 
+    bdrv_ref(source);
+    top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
+                                  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                                  errp);
+    if (!top->file) {
+        error_prepend(errp, "Cannot attach file child: ");
+        bdrv_unref(source);

Already existing in the code above, but why is the reference to @source dropped here? Shouldn’t bdrv_attach_child()’s error path have done that already?

Max

Quoted 3 lines
block/copy-before-write.c +3 -0
+        bdrv_unref(top);
+        return NULL;
+    }
#38 lore raw

Re: [PATCH 11/21] block/copy-before-write: use file child instead of backing

Quoted 27 lines · Max Reitz
Quoted 13 lines · Vladimir Sementsov-Ogievskiy

We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead.

---
  block/copy-before-write.c | 40 ++++++++++++++++++++++-----------------

  1 file changed, 23 insertions(+), 17 deletions(-)
block/copy-before-write.c +0 -0
index af2bb97a30..2f06a524b8 100644

[...]

Quoted 10 lines
block/copy-before-write.c +2 -0
@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
       state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
                                         BDRV_CHILD_DATA, errp);
       if (!state->target) {
+        error_prepend(errp, "Cannot attach target child: ");
           bdrv_unref(target);
           bdrv_unref(top);
           return NULL;
       }
+    bdrv_ref(source);
Quoted 9 lines

block/copy-before-write.c BlockDriverState *bdrv_cbw_append(BlockDriverState *source, · line 210

Quoted 6 lines
block/copy-before-write.c +6 -0
+    top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
+                                  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                                  errp);
+    if (!top->file) {
+        error_prepend(errp, "Cannot attach file child: ");
+        bdrv_unref(source);

Already existing in the code above, but why is the reference to @source dropped here?  Shouldn’t bdrv_attach_child()’s error path have done that already?

Hmm, yes. Looks like a preexisting bug. will fix and update this patch.

Signature · 2 lines
Best regards,
Vladimir
#39 lore raw

PATCH 12/21 block/copy-before-write: bdrv_cbw_append(): replace child at last

In reply to Vladimir Sementsov-Ogievskiy

Refactor the function to replace child at last. Thus we don't need to revert it and code is simplified.

block-copy state initilization being done before replacing the child doesn't need any drained section.

---
 block/copy-before-write.c | 33 +++++++++++----------------------
 1 file changed, 11 insertions(+), 22 deletions(-)
block/copy-before-write.c +11 -22
index 2f06a524b8..35a9aa163d 100644
@@ -172,7 +172,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     int ret;
     BDRVCopyBeforeWriteState *state;
     BlockDriverState *top;
-    bool appended = false;
 
     assert(source->total_sectors == target->total_sectors);
 
@@ -197,8 +196,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     if (!state->target) {
         error_prepend(errp, "Cannot attach target child: ");
         bdrv_unref(target);
-        bdrv_unref(top);
-        return NULL;
+        goto fail;
     }
 
     bdrv_ref(source);
@@ -208,18 +206,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     if (!top->file) {
         error_prepend(errp, "Cannot attach file child: ");
         bdrv_unref(source);
-        bdrv_unref(top);
-        return NULL;
-    }
-
-    bdrv_drained_begin(source);
-
-    ret = bdrv_replace_node(source, top, errp);
-    if (ret < 0) {
-        error_prepend(errp, "Cannot append copy-before-write filter: ");
         goto fail;
     }
-    appended = true;
 
     state->bcs = block_copy_state_new(top->file, state->target, false, compress,
                                       errp);
@@ -227,21 +215,22 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
         error_prepend(errp, "Cannot create block-copy-state: ");
         goto fail;
     }
-    *bcs = state->bcs;
 
+    bdrv_drained_begin(source);
+    ret = bdrv_replace_node(source, top, errp);
     bdrv_drained_end(source);
+    if (ret < 0) {
+        error_prepend(errp, "Cannot append copy-before-write filter: ");
+        goto fail;
+    }
+
+    *bcs = state->bcs;
 
     return top;
 
 fail:
-    if (appended) {
-        bdrv_cbw_drop(top);
-    } else {
-        bdrv_unref(top);
-    }
-
-    bdrv_drained_end(source);
-
+    block_copy_state_free(state->bcs);
+    bdrv_unref(top);
     return NULL;
 }
 
Signature · 1 line
2.29.2
Reviewed-by on v1 12/21
#40 lore raw

Re: [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last

Quoted 4 lines · Vladimir Sementsov-Ogievskiy

Refactor the function to replace child at last. Thus we don't need to revert it and code is simplified.

block-copy state initilization being done before replacing the child

*initialization

Quoted 6 lines

doesn't need any drained section.

---
  block/copy-before-write.c | 33 +++++++++++----------------------
  1 file changed, 11 insertions(+), 22 deletions(-)
#41 lore raw

PATCH 13/21 block/copy-before-write: introduce cbw_init()

In reply to Vladimir Sementsov-Ogievskiy

Move part of bdrv_cbw_append() to new function cbw_open(). It's an intermediate step for adding noramal .bdrv_open() handler to the filter. With this commit no logic is changed, but we have a function which will be turned into .bdrv_open() handler in future commit.

---
 block/copy-before-write.c | 73 +++++++++++++++++++++++----------------
 1 file changed, 43 insertions(+), 30 deletions(-)
block/copy-before-write.c +43 -30
index 35a9aa163d..94b1357fd8 100644
@@ -144,6 +144,47 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
     }
 }
 
+static int cbw_init(BlockDriverState *top, BlockDriverState *source,
+                    BlockDriverState *target, bool compress, Error **errp)
+{
+    BDRVCopyBeforeWriteState *state = top->opaque;
+
+    top->total_sectors = source->total_sectors;
+    top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+            (BDRV_REQ_FUA & source->supported_write_flags);
+    top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+            ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
+             source->supported_zero_flags);
+
+    bdrv_ref(target);
+    state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
+                                      BDRV_CHILD_DATA, errp);
+    if (!state->target) {
+        error_prepend(errp, "Cannot attach target child: ");
+        bdrv_unref(target);
+        return -EINVAL;
+    }
+
+    bdrv_ref(source);
+    top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
+                                  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                                  errp);
+    if (!top->file) {
+        error_prepend(errp, "Cannot attach file child: ");
+        bdrv_unref(source);
+        return -EINVAL;
+    }
+
+    state->bcs = block_copy_state_new(top->file, state->target, false, compress,
+                                      errp);
+    if (!state->bcs) {
+        error_prepend(errp, "Cannot create block-copy-state: ");
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
 BlockDriver bdrv_cbw_filter = {
     .format_name = "copy-before-write",
     .instance_size = sizeof(BDRVCopyBeforeWriteState),
@@ -181,38 +222,10 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
         error_prepend(errp, "Cannot open driver: ");
         return NULL;
     }
-
     state = top->opaque;
-    top->total_sectors = source->total_sectors;
-    top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
-            (BDRV_REQ_FUA & source->supported_write_flags);
-    top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
-            ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
-             source->supported_zero_flags);
-
-    bdrv_ref(target);
-    state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
-                                      BDRV_CHILD_DATA, errp);
-    if (!state->target) {
-        error_prepend(errp, "Cannot attach target child: ");
-        bdrv_unref(target);
-        goto fail;
-    }
-
-    bdrv_ref(source);
-    top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
-                                  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-                                  errp);
-    if (!top->file) {
-        error_prepend(errp, "Cannot attach file child: ");
-        bdrv_unref(source);
-        goto fail;
-    }
 
-    state->bcs = block_copy_state_new(top->file, state->target, false, compress,
-                                      errp);
-    if (!state->bcs) {
-        error_prepend(errp, "Cannot create block-copy-state: ");
+    ret = cbw_init(top, source, target, compress, errp);
+    if (ret < 0) {
         goto fail;
     }
 
Signature · 1 line
2.29.2
Reviewed-by on v1 13/21
#42 lore raw

Re: [PATCH 13/21] block/copy-before-write: introduce cbw_init()

Quoted 9 lines · Vladimir Sementsov-Ogievskiy

Move part of bdrv_cbw_append() to new function cbw_open(). It's an intermediate step for adding noramal .bdrv_open() handler to the filter. With this commit no logic is changed, but we have a function which will be turned into .bdrv_open() handler in future commit.

---
  block/copy-before-write.c | 73 +++++++++++++++++++++++----------------
  1 file changed, 43 insertions(+), 30 deletions(-)
#43 lore raw

PATCH 14/21 block/copy-before-write: cbw_init(): rename variables

In reply to Vladimir Sementsov-Ogievskiy

One more step closer to real .bdrv_open() handler: use more usual names for bs being initialized and its state.

---
 block/copy-before-write.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)
block/copy-before-write.c +14 -15
index 94b1357fd8..78dc421db2 100644
@@ -144,40 +144,39 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
     }
 }
 
-static int cbw_init(BlockDriverState *top, BlockDriverState *source,
+static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
                     BlockDriverState *target, bool compress, Error **errp)
 {
-    BDRVCopyBeforeWriteState *state = top->opaque;
+    BDRVCopyBeforeWriteState *s = bs->opaque;
 
-    top->total_sectors = source->total_sectors;
-    top->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+    bs->total_sectors = source->total_sectors;
+    bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
             (BDRV_REQ_FUA & source->supported_write_flags);
-    top->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+    bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
             ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
              source->supported_zero_flags);
 
     bdrv_ref(target);
-    state->target = bdrv_attach_child(top, target, "target", &child_of_bds,
-                                      BDRV_CHILD_DATA, errp);
-    if (!state->target) {
+    s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
+                                  BDRV_CHILD_DATA, errp);
+    if (!s->target) {
         error_prepend(errp, "Cannot attach target child: ");
         bdrv_unref(target);
         return -EINVAL;
     }
 
     bdrv_ref(source);
-    top->file = bdrv_attach_child(top, source, "file", &child_of_bds,
-                                  BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-                                  errp);
-    if (!top->file) {
+    bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
+                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                                 errp);
+    if (!bs->file) {
         error_prepend(errp, "Cannot attach file child: ");
         bdrv_unref(source);
         return -EINVAL;
     }
 
-    state->bcs = block_copy_state_new(top->file, state->target, false, compress,
-                                      errp);
-    if (!state->bcs) {
+    s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
+    if (!s->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         return -EINVAL;
     }
Signature · 1 line
2.29.2
Reviewed-by on v1 14/21
#44 lore raw

Re: [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables

Quoted 7 lines · Vladimir Sementsov-Ogievskiy

One more step closer to real .bdrv_open() handler: use more usual names for bs being initialized and its state.

---
  block/copy-before-write.c | 29 ++++++++++++++---------------
  1 file changed, 14 insertions(+), 15 deletions(-)
#45 lore raw

PATCH 15/21 block/copy-before-write: cbw_init(): use file child after attaching

In reply to Vladimir Sementsov-Ogievskiy

In the next commit we'll get rid of source argument of cbw_init(). Prepare to it now, to make next commit simpler: move the code block that uses source below attaching the child and use bs->file->bs instead of source variable.

---
 block/copy-before-write.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
block/copy-before-write.c +7 -7
index 78dc421db2..ddd79b3686 100644
@@ -149,13 +149,6 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
 
-    bs->total_sectors = source->total_sectors;
-    bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
-            (BDRV_REQ_FUA & source->supported_write_flags);
-    bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
-            ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
-             source->supported_zero_flags);
-
     bdrv_ref(target);
     s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
                                   BDRV_CHILD_DATA, errp);
@@ -175,6 +168,13 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
         return -EINVAL;
     }
 
+    bs->total_sectors = bs->file->bs->total_sectors;
+    bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+            (BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
+    bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+            ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
+             bs->file->bs->supported_zero_flags);
+
     s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
     if (!s->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
Signature · 1 line
2.29.2
Reviewed-by on v1 15/21
#46 lore raw

Re: [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching

Quoted 9 lines · Vladimir Sementsov-Ogievskiy

In the next commit we'll get rid of source argument of cbw_init(). Prepare to it now, to make next commit simpler: move the code block that uses source below attaching the child and use bs->file->bs instead of source variable.

---
  block/copy-before-write.c | 14 +++++++-------
  1 file changed, 7 insertions(+), 7 deletions(-)
#47 lore raw

PATCH 16/21 block/copy-before-write: cbw_init(): use options

In reply to Vladimir Sementsov-Ogievskiy

One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers.

---
 block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
 1 file changed, 20 insertions(+), 17 deletions(-)
block/copy-before-write.c +20 -17
index ddd79b3686..9ff1bf676c 100644
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
     }
 }
 
-static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
-                    BlockDriverState *target, bool compress, Error **errp)
+static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
 
-    bdrv_ref(target);
-    s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
-                                  BDRV_CHILD_DATA, errp);
-    if (!s->target) {
-        error_prepend(errp, "Cannot attach target child: ");
-        bdrv_unref(target);
+    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+                               BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                               false, errp);
+    if (!bs->file) {
         return -EINVAL;
     }
 
-    bdrv_ref(source);
-    bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
-                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-                                 errp);
-    if (!bs->file) {
-        error_prepend(errp, "Cannot attach file child: ");
-        bdrv_unref(source);
+    s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
+                                BDRV_CHILD_DATA, false, errp);
+    if (!s->target) {
         return -EINVAL;
     }
 
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
             ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
              bs->file->bs->supported_zero_flags);
 
-    s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
+    qdict_del(options, "cluster-size");
+    s->bcs = block_copy_state_new(bs->file, s->target, false,
+            qdict_get_try_bool(options, "x-deprecated-compress", false), errp);
+    qdict_del(options, "x-deprecated-compress");
     if (!s->bcs) {
         error_prepend(errp, "Cannot create block-copy-state: ");
         return -EINVAL;
@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     int ret;
     BDRVCopyBeforeWriteState *state;
     BlockDriverState *top;
+    QDict *opts;
 
     assert(source->total_sectors == target->total_sectors);
 
@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
     }
     state = top->opaque;
 
-    ret = cbw_init(top, source, target, compress, errp);
+    opts = qdict_new();
+    qdict_put_str(opts, "file", bdrv_get_node_name(source));
+    qdict_put_str(opts, "target", bdrv_get_node_name(target));
+    qdict_put_bool(opts, "x-deprecated-compress", compress);
+
+    ret = cbw_init(top, opts, errp);
+    qobject_unref(opts);
     if (ret < 0) {
         goto fail;
     }
Signature · 1 line
2.29.2
#48 lore raw

Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options

Quoted 50 lines · Vladimir Sementsov-Ogievskiy

One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers.

---
  block/copy-before-write.c | 37 ++++++++++++++++++++-----------------
  1 file changed, 20 insertions(+), 17 deletions(-)
block/copy-before-write.c +8 -15
index ddd79b3686..9ff1bf676c 100644
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
      }
  }
 
-static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
-                    BlockDriverState *target, bool compress, Error **errp)
+static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
  {
      BDRVCopyBeforeWriteState *s = bs->opaque;
 
-    bdrv_ref(target);
-    s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
-                                  BDRV_CHILD_DATA, errp);
-    if (!s->target) {
-        error_prepend(errp, "Cannot attach target child: ");
-        bdrv_unref(target);
+    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+                               BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                               false, errp);
+    if (!bs->file) {
          return -EINVAL;
      }
 
-    bdrv_ref(source);
-    bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
-                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-                                 errp);
-    if (!bs->file) {
-        error_prepend(errp, "Cannot attach file child: ");
-        bdrv_unref(source);
+    s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
+                                BDRV_CHILD_DATA, false, errp);
+    if (!s->target) {
          return -EINVAL;
      }
 

block/copy-before-write.c static int cbw_init(BlockDriverState *bs, BlockDriverState *source, · line 171

Quoted 5 lines
block/copy-before-write.c +1 -1
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
              ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
               bs->file->bs->supported_zero_flags);
 
-    s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
+    qdict_del(options, "cluster-size");

What is this about?

block/copy-before-write.c static int cbw_init(BlockDriverState *bs, BlockDriverState *source, · line 173

Quoted 2 lines
block/copy-before-write.c +2 -0
+    s->bcs = block_copy_state_new(bs->file, s->target, false,
+            qdict_get_try_bool(options, "x-deprecated-compress", false), errp);

First, I’d keep the compress variable and use it to store the value, because this doesn’t look very nice.

Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.

Max

Quoted 28 lines
block/copy-before-write.c +9 -1
+    qdict_del(options, "x-deprecated-compress");
      if (!s->bcs) {
          error_prepend(errp, "Cannot create block-copy-state: ");
          return -EINVAL;
@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
      int ret;
      BDRVCopyBeforeWriteState *state;
      BlockDriverState *top;
+    QDict *opts;
 
      assert(source->total_sectors == target->total_sectors);
 
@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
      }
      state = top->opaque;
 
-    ret = cbw_init(top, source, target, compress, errp);
+    opts = qdict_new();
+    qdict_put_str(opts, "file", bdrv_get_node_name(source));
+    qdict_put_str(opts, "target", bdrv_get_node_name(target));
+    qdict_put_bool(opts, "x-deprecated-compress", compress);
+
+    ret = cbw_init(top, opts, errp);
+    qobject_unref(opts);
      if (ret < 0) {
          goto fail;
      }
#49 lore raw

Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options

Quoted 45 lines · Max Reitz
Quoted 44 lines · Vladimir Sementsov-Ogievskiy

One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers.

---
  block/copy-before-write.c | 37 ++++++++++++++++++++-----------------

  1 file changed, 20 insertions(+), 17 deletions(-)
block/copy-before-write.c +8 -15
index ddd79b3686..9ff1bf676c 100644
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
       }
   }
-static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
-                    BlockDriverState *target, bool compress, Error **errp)
+static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
   {
       BDRVCopyBeforeWriteState *s = bs->opaque;
-    bdrv_ref(target);
-    s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
-                                  BDRV_CHILD_DATA, errp);
-    if (!s->target) {
-        error_prepend(errp, "Cannot attach target child: ");
-        bdrv_unref(target);
+    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+                               BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                               false, errp);
+    if (!bs->file) {
           return -EINVAL;
       }
-    bdrv_ref(source);
-    bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
-                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-                                 errp);
-    if (!bs->file) {
-        error_prepend(errp, "Cannot attach file child: ");
-        bdrv_unref(source);
+    s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
+                                BDRV_CHILD_DATA, false, errp);
+    if (!s->target) {
           return -EINVAL;
Quoted 8 lines
Quoted 6 lines
block/copy-before-write.c +0 -0
       }
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,

              ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags);
-    s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp); +    qdict_del(options, "cluster-size");

What is this about?

accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to block_copy)

Quoted 5 lines
Quoted 3 lines

+    s->bcs = block_copy_state_new(bs->file, s->target, false,
+            qdict_get_try_bool(options, "x-deprecated-compress", false), errp);

First, I’d keep the compress variable and use it to store the value, because this doesn’t look very nice.

OK

Quoted 3 lines

Second, what’s the story here?  “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.

I should have described it in commit message.

We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.

Quoted 27 lines
Quoted 26 lines

+    qdict_del(options, "x-deprecated-compress");
      if (!s->bcs) {
          error_prepend(errp, "Cannot create block-copy-state: "); return -EINVAL;

block/copy-before-write.c +8 -1
@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
       int ret;
       BDRVCopyBeforeWriteState *state;
       BlockDriverState *top;
+    QDict *opts;
       assert(source->total_sectors == target->total_sectors);
@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
       }
       state = top->opaque;
-    ret = cbw_init(top, source, target, compress, errp);
+    opts = qdict_new();
+    qdict_put_str(opts, "file", bdrv_get_node_name(source));
+    qdict_put_str(opts, "target", bdrv_get_node_name(target));
+    qdict_put_bool(opts, "x-deprecated-compress", compress);
+
+    ret = cbw_init(top, opts, errp);
+    qobject_unref(opts);
       if (ret < 0) {
           goto fail;
       }
 
Signature · 2 lines
Best regards,
Vladimir
#50 lore raw

Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options

Quoted 75 lines · Vladimir Sementsov-Ogievskiy
Quoted 53 lines · Max Reitz
Quoted 52 lines · Vladimir Sementsov-Ogievskiy

One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers.

---
  block/copy-before-write.c | 37 ++++++++++++++++++++-----------------

  1 file changed, 20 insertions(+), 17 deletions(-)
block/copy-before-write.c +8 -15
index ddd79b3686..9ff1bf676c 100644
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState 
 *bs, BdrvChild *c,
       }
   }
-static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
-                    BlockDriverState *target, bool compress, Error 
 **errp)
+static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
   {
       BDRVCopyBeforeWriteState *s = bs->opaque;
-    bdrv_ref(target);
-    s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
-                                  BDRV_CHILD_DATA, errp);
-    if (!s->target) {
-        error_prepend(errp, "Cannot attach target child: ");
-        bdrv_unref(target);
+    bs->file = bdrv_open_child(NULL, options, "file", bs, 
 &child_of_bds,
+                               BDRV_CHILD_FILTERED | 
 BDRV_CHILD_PRIMARY,
+                               false, errp);
+    if (!bs->file) {
           return -EINVAL;
       }
-    bdrv_ref(source);
-    bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
-                                 BDRV_CHILD_FILTERED | 
 BDRV_CHILD_PRIMARY,
-                                 errp);
-    if (!bs->file) {
-        error_prepend(errp, "Cannot attach file child: ");
-        bdrv_unref(source);
+    s->target = bdrv_open_child(NULL, options, "target", bs, 
 &child_of_bds,
+                                BDRV_CHILD_DATA, false, errp);
+    if (!s->target) {

          return -EINVAL;
      }

block/copy-before-write.c +0 -0
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, 
Quoted 8 lines
Quoted 6 lines
block/copy-before-write.c +0 -0
 BlockDriverState *source,

              ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags);
-    s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp);
+    qdict_del(options, "cluster-size");

What is this about?

accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to block_copy)

Quoted 6 lines
Quoted 4 lines

+    s->bcs = block_copy_state_new(bs->file, s->target, false, +            qdict_get_try_bool(options, "x-deprecated-compress", false), errp);

First, I’d keep the compress variable and use it to store the value, because this doesn’t look very nice.

OK

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

Second, what’s the story here?  “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.

I should have described it in commit message.

We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.

Oh, that’s true. Until backup’s option is deprecated, I think a comment would be more useful than a note in the commit message, though.

Max

#51 lore raw

Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options

Quoted 67 lines · Max Reitz
Quoted 66 lines · Vladimir Sementsov-Ogievskiy
Quoted 45 lines · Max Reitz
Quoted 44 lines · Vladimir Sementsov-Ogievskiy

One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers.

---
  block/copy-before-write.c | 37 ++++++++++++++++++++-----------------

  1 file changed, 20 insertions(+), 17 deletions(-)
block/copy-before-write.c +8 -15
index ddd79b3686..9ff1bf676c 100644
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
       }
   }
-static int cbw_init(BlockDriverState *bs, BlockDriverState *source,
-                    BlockDriverState *target, bool compress, Error **errp)
+static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
   {
       BDRVCopyBeforeWriteState *s = bs->opaque;
-    bdrv_ref(target);
-    s->target = bdrv_attach_child(bs, target, "target", &child_of_bds,
-                                  BDRV_CHILD_DATA, errp);
-    if (!s->target) {
-        error_prepend(errp, "Cannot attach target child: ");
-        bdrv_unref(target);
+    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
+                               BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
+                               false, errp);
+    if (!bs->file) {
           return -EINVAL;
       }
-    bdrv_ref(source);
-    bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds,
-                                 BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
-                                 errp);
-    if (!bs->file) {
-        error_prepend(errp, "Cannot attach file child: ");
-        bdrv_unref(source);
+    s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds,
+                                BDRV_CHILD_DATA, false, errp);
+    if (!s->target) {
           return -EINVAL;
Quoted 8 lines
Quoted 6 lines
block/copy-before-write.c +0 -0
       }
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,

              ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) & bs->file->bs->supported_zero_flags);
-    s->bcs = block_copy_state_new(bs->file, s->target, false, compress, errp); +    qdict_del(options, "cluster-size");

What is this about?

accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to block_copy)

Quoted 5 lines
Quoted 3 lines

+    s->bcs = block_copy_state_new(bs->file, s->target, false,
+            qdict_get_try_bool(options, "x-deprecated-compress", false), errp);

First, I’d keep the compress variable and use it to store the value, because this doesn’t look very nice.

OK

Quoted 1 line
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines

Second, what’s the story here?  “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.

I should have described it in commit message.

We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.

Oh, that’s true.  Until backup’s option is deprecated, I think a comment would be more useful than a note in the commit message, though.

Agree, will add

Signature · 2 lines
Best regards,
Vladimir
#52 lore raw

PATCH 17/21 block/block-copy: switch to fully set bitmap by default

In reply to Vladimir Sementsov-Ogievskiy

block-copy has a bit inconvenient interface around dirty bitmap: user should get pointer to it and than set by hand. We do need a possibility to share the bitmap with backup job.

But default of empty bitmap is strange. Switch to full-set bitmap by default. This way we will not care about setting dirty bitmap in copy-before-write filter when publish it so that it can be used in separate of backup job.

---
 block/backup.c     | 16 +++++++---------
 block/block-copy.c |  1 +
 2 files changed, 8 insertions(+), 9 deletions(-)
block/backup.c +7 -9
index 90cca1ca30..706c54fea1 100644
@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
     BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
 
     if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+        bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
         ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
                                                NULL, true);
         assert(ret);
-    } else {
-        if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
-            /*
-             * We can't hog the coroutine to initialize this thoroughly.
-             * Set a flag and resume work when we are able to yield safely.
-             */
-            block_copy_set_skip_unallocated(job->bcs, true);
-        }
-        bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
+    } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
+        /*
+         * We can't hog the coroutine to initialize this thoroughly.
+         * Set a flag and resume work when we are able to yield safely.
+         */
+        block_copy_set_skip_unallocated(job->bcs, true);
     }
 
     estimate = bdrv_get_dirty_count(bcs_bitmap);
block/block-copy.c +1 -0
index 9020234c6e..4126f7e8cc 100644
@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
         return NULL;
     }
     bdrv_disable_dirty_bitmap(copy_bitmap);
+    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
 
     /*
      * Why we always set BDRV_REQ_SERIALISING write flag:
Signature · 1 line
2.29.2
Reviewed-by on v1 17/21
#53 lore raw

Re: [PATCH 17/21] block/block-copy: switch to fully set bitmap by default

Quoted 5 lines · Vladimir Sementsov-Ogievskiy

block-copy has a bit inconvenient interface around dirty bitmap: user should get pointer to it and than set by hand. We do need a possibility to share the bitmap with backup job.

But default of empty bitmap is strange.

I don’t know, I don’t find it strange. It expects its user to specify what data to copy, so clearly it gives said user a blank slate.

Quoted 4 lines

Switch to full-set bitmap by
default. This way we will not care about setting dirty bitmap in copy-before-write filter when publish it so that it can be used in separate of backup job.

That’s a valid reason, though, so:

Still, I find it stranger this way, because I’m more used to “initialization to 0 by default”.

Quoted 49 lines
---
  block/backup.c     | 16 +++++++---------
  block/block-copy.c |  1 +
  2 files changed, 8 insertions(+), 9 deletions(-)
block/backup.c +7 -9
index 90cca1ca30..706c54fea1 100644
@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
      BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
 
      if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+        bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
          ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
                                                 NULL, true);
          assert(ret);
-    } else {
-        if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
-            /*
-             * We can't hog the coroutine to initialize this thoroughly.
-             * Set a flag and resume work when we are able to yield safely.
-             */
-            block_copy_set_skip_unallocated(job->bcs, true);
-        }
-        bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
+    } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
+        /*
+         * We can't hog the coroutine to initialize this thoroughly.
+         * Set a flag and resume work when we are able to yield safely.
+         */
+        block_copy_set_skip_unallocated(job->bcs, true);
      }
 
      estimate = bdrv_get_dirty_count(bcs_bitmap);
block/block-copy.c +1 -0
index 9020234c6e..4126f7e8cc 100644
@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
          return NULL;
      }
      bdrv_disable_dirty_bitmap(copy_bitmap);
+    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
 
      /*
       * Why we always set BDRV_REQ_SERIALISING write flag:
#54 lore raw

Re: [PATCH 17/21] block/block-copy: switch to fully set bitmap by default

Quoted 14 lines · Max Reitz
Quoted 5 lines · Vladimir Sementsov-Ogievskiy

block-copy has a bit inconvenient interface around dirty bitmap: user should get pointer to it and than set by hand. We do need a possibility to share the bitmap with backup job.

But default of empty bitmap is strange.

I don’t know, I don’t find it strange.  It expects its user to specify what data to copy, so clearly it gives said user a blank slate.

Quoted 4 lines

Switch to full-set bitmap by
default. This way we will not care about setting dirty bitmap in copy-before-write filter when publish it so that it can be used in separate of backup job.

Quoted 5 lines

That’s a valid reason, though, so:

Still, I find it stranger this way, because I’m more used to “initialization to 0 by default”.

When I started to test this series it didn't work, because bitmap was not initialized. I fixed it and call original behavior strange :)

Probably better to keep block-copy as is and set bitmap in cbw_open. Backup code should be modified anyway, as it will get the bitmap after initialization in cbw_open.

Next step would be support for bitmap in copy-before-write filter anyway, and this place will be changed again.. So, I don't know, but it's not important thing.

Quoted 48 lines
Quoted 47 lines

---
  block/backup.c     | 16 +++++++---------
  block/block-copy.c |  1 +

  2 files changed, 8 insertions(+), 9 deletions(-)
block/backup.c +7 -9
index 90cca1ca30..706c54fea1 100644
@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
       BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs);
       if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+        bdrv_clear_dirty_bitmap(bcs_bitmap, NULL);
           ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap,
                                                  NULL, true);
           assert(ret);
-    } else {
-        if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
-            /*
-             * We can't hog the coroutine to initialize this thoroughly.
-             * Set a flag and resume work when we are able to yield safely.
-             */
-            block_copy_set_skip_unallocated(job->bcs, true);
-        }
-        bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len);
+    } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) {
+        /*
+         * We can't hog the coroutine to initialize this thoroughly.
+         * Set a flag and resume work when we are able to yield safely.
+         */
+        block_copy_set_skip_unallocated(job->bcs, true);
       }
       estimate = bdrv_get_dirty_count(bcs_bitmap);
block/block-copy.c +1 -0
index 9020234c6e..4126f7e8cc 100644
@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
           return NULL;
       }
       bdrv_disable_dirty_bitmap(copy_bitmap);
+    bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap));
       /*
        * Why we always set BDRV_REQ_SERIALISING write flag:
 
Signature · 2 lines
Best regards,
Vladimir
#55 lore raw

PATCH 18/21 block/block-copy: make setting progress optional

In reply to Vladimir Sementsov-Ogievskiy

Now block-copy will crash if user don't set progress meter by block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it.

---
 block/block-copy.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
block/block-copy.c +11 -7
index 4126f7e8cc..63ec5c41a2 100644
@@ -506,7 +506,7 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
     if (ret < 0 && !t->call_state->ret) {
         t->call_state->ret = ret;
         t->call_state->error_is_read = error_is_read;
-    } else {
+    } else if (t->s->progress) {
         progress_work_done(t->s->progress, t->bytes);
     }
     co_put_to_shres(t->s->mem, t->bytes);
@@ -610,9 +610,11 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
 
     if (!ret) {
         bdrv_reset_dirty_bitmap(s->copy_bitmap, offset, bytes);
-        progress_set_remaining(s->progress,
-                               bdrv_get_dirty_count(s->copy_bitmap) +
-                               s->in_flight_bytes);
+        if (s->progress) {
+            progress_set_remaining(s->progress,
+                                   bdrv_get_dirty_count(s->copy_bitmap) +
+                                   s->in_flight_bytes);
+        }
     }
 
     *count = bytes;
@@ -672,9 +674,11 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
         }
         if (s->skip_unallocated && !(ret & BDRV_BLOCK_ALLOCATED)) {
             block_copy_task_end(task, 0);
-            progress_set_remaining(s->progress,
-                                   bdrv_get_dirty_count(s->copy_bitmap) +
-                                   s->in_flight_bytes);
+            if (s->progress) {
+                progress_set_remaining(s->progress,
+                                       bdrv_get_dirty_count(s->copy_bitmap) +
+                                       s->in_flight_bytes);
+            }
             trace_block_copy_skip_range(s, task->offset, task->bytes);
             offset = task_end(task);
             bytes = end - offset;
Signature · 1 line
2.29.2
Reviewed-by on v1 18/21
#56 lore raw

Re: [PATCH 18/21] block/block-copy: make setting progress optional

Quoted 9 lines · Vladimir Sementsov-Ogievskiy

Now block-copy will crash if user don't set progress meter by block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it.

---
  block/block-copy.c | 18 +++++++++++-------
  1 file changed, 11 insertions(+), 7 deletions(-)

Makes me wonder whether an “if (not NULL)” check in progress_meter.h wouldn’t be a nice addition, but:

#57 lore raw

Re: [PATCH 18/21] block/block-copy: make setting progress optional

Quoted 6 lines · Max Reitz
Quoted 5 lines · Vladimir Sementsov-Ogievskiy

Now block-copy will crash if user don't set progress meter by block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it.

Quoted 6 lines
Quoted 4 lines

---
  block/block-copy.c | 18 +++++++++++-------

  1 file changed, 11 insertions(+), 7 deletions(-)

Makes me wonder whether an “if (not NULL)” check in progress_meter.h wouldn’t be a nice addition, but:

IMHO it isn't. It's unusual to check if (obj is not NULL) in all obj API functions.. The only place where it is usual is obj_free() like functions.

Quoted 3 lines

Thanks!

Signature · 2 lines
Best regards,
Vladimir
#58 lore raw

PATCH 19/21 block/copy-before-write: make public block driver

In reply to Vladimir Sementsov-Ogievskiy

Finally, copy-before-write gets own .bdrv_open and .bdrv_close handlers, block_init() call and becomes available through bdrv_open().

To achieve this:

- cbw_init gets unused flags argument and becomes cbw_open
- block_copy_state_free() call moved to new cbw_close()
- in bdrv_cbw_append:
- options are completed with driver and node-name, and we can simply use bdrv_insert_node() to do both open and drained replacing
- in bdrv_cbw_drop:
- cbw_close() is now responsible for freeing s->bcs, so don't do it here

---
 block/copy-before-write.c | 60 ++++++++++++++++++---------------------
 1 file changed, 28 insertions(+), 32 deletions(-)
block/copy-before-write.c +28 -32
index 9ff1bf676c..20c933bfb0 100644
@@ -144,7 +144,8 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
     }
 }
 
-static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
+static int cbw_open(BlockDriverState *bs, QDict *options, int flags,
+                    Error **errp)
 {
     BDRVCopyBeforeWriteState *s = bs->opaque;
 
@@ -180,10 +181,21 @@ static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)
     return 0;
 }
 
+static void cbw_close(BlockDriverState *bs)
+{
+    BDRVCopyBeforeWriteState *s = bs->opaque;
+
+    block_copy_state_free(s->bcs);
+    s->bcs = NULL;
+}
+
 BlockDriver bdrv_cbw_filter = {
     .format_name = "copy-before-write",
     .instance_size = sizeof(BDRVCopyBeforeWriteState),
 
+    .bdrv_open                  = cbw_open,
+    .bdrv_close                 = cbw_close,
+
     .bdrv_co_preadv             = cbw_co_preadv,
     .bdrv_co_pwritev            = cbw_co_pwritev,
     .bdrv_co_pwrite_zeroes      = cbw_co_pwrite_zeroes,
@@ -205,57 +217,41 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,
                                   Error **errp)
 {
     ERRP_GUARD();
-    int ret;
     BDRVCopyBeforeWriteState *state;
     BlockDriverState *top;
     QDict *opts;
 
     assert(source->total_sectors == target->total_sectors);
 
-    top = bdrv_new_open_driver(&bdrv_cbw_filter, filter_node_name,
-                               BDRV_O_RDWR, errp);
-    if (!top) {
-        error_prepend(errp, "Cannot open driver: ");
-        return NULL;
-    }
-    state = top->opaque;
-
     opts = qdict_new();
+    qdict_put_str(opts, "driver", "copy-before-write");
+    if (filter_node_name) {
+        qdict_put_str(opts, "node-name", filter_node_name);
+    }
     qdict_put_str(opts, "file", bdrv_get_node_name(source));
     qdict_put_str(opts, "target", bdrv_get_node_name(target));
     qdict_put_bool(opts, "x-deprecated-compress", compress);
 
-    ret = cbw_init(top, opts, errp);
-    qobject_unref(opts);
-    if (ret < 0) {
-        goto fail;
-    }
-
-    bdrv_drained_begin(source);
-    ret = bdrv_replace_node(source, top, errp);
-    bdrv_drained_end(source);
-    if (ret < 0) {
-        error_prepend(errp, "Cannot append copy-before-write filter: ");
-        goto fail;
+    top = bdrv_insert_node(source, opts, BDRV_O_RDWR, errp);
+    if (!top) {
+        return NULL;
     }
 
+    state = top->opaque;
     *bcs = state->bcs;
 
     return top;
-
-fail:
-    block_copy_state_free(state->bcs);
-    bdrv_unref(top);
-    return NULL;
 }
 
 void bdrv_cbw_drop(BlockDriverState *bs)
 {
-    BDRVCopyBeforeWriteState *s = bs->opaque;
-
     bdrv_drop_filter(bs, &error_abort);
-
-    block_copy_state_free(s->bcs);
-
     bdrv_unref(bs);
 }
+
+static void cbw_init(void)
+{
+    bdrv_register(&bdrv_cbw_filter);
+}
+
+block_init(cbw_init);
Signature · 1 line
2.29.2
Reviewed-by on v1 19/21
#59 lore raw

Re: [PATCH 19/21] block/copy-before-write: make public block driver

Quoted 14 lines · Vladimir Sementsov-Ogievskiy

Finally, copy-before-write gets own .bdrv_open and .bdrv_close handlers, block_init() call and becomes available through bdrv_open().

To achieve this:

- cbw_init gets unused flags argument and becomes cbw_open
- block_copy_state_free() call moved to new cbw_close()
- in bdrv_cbw_append:
- options are completed with driver and node-name, and we can simply use bdrv_insert_node() to do both open and drained replacing
- in bdrv_cbw_drop:
- cbw_close() is now responsible for freeing s->bcs, so don't do it here

Quoted 4 lines
---
  block/copy-before-write.c | 60 ++++++++++++++++++---------------------
  1 file changed, 28 insertions(+), 32 deletions(-)
#60 lore raw

PATCH 20/21 qapi: publish copy-before-write filter

In reply to Vladimir Sementsov-Ogievskiy
---
 qapi/block-core.json | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)
qapi/block-core.json +18 -4
index 2ea294129e..f8fbcb6416 100644
@@ -2808,15 +2808,17 @@
 # @blklogwrites: Since 3.0
 # @blkreplay: Since 4.2
 # @compress: Since 5.0
+# @copy-before-write: Since 6.1
 #
 # Since: 2.9
 ##
 { 'enum': 'BlockdevDriver',
   'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
-            'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
-            'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
-            'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
-            'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+            'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
+            'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device',
+            'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio',
+            'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2',
+            'qed', 'quorum', 'raw', 'rbd',
             { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
             'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
 
@@ -3937,6 +3939,17 @@
   'base': 'BlockdevOptionsGenericFormat',
   'data': { '*bottom': 'str' } }
 
+##
+# @BlockdevOptionsCbw:
+#
+# Driver specific block device options for the copy-before-write driver.
+#
+# Since: 6.1
+##
+{ 'struct': 'BlockdevOptionsCbw',
+  'base': 'BlockdevOptionsGenericFormat',
+  'data': { 'target': 'BlockdevRef' } }
+
 ##
 # @BlockdevOptions:
 #
@@ -3989,6 +4002,7 @@
       'bochs':      'BlockdevOptionsGenericFormat',
       'cloop':      'BlockdevOptionsGenericFormat',
       'compress':   'BlockdevOptionsGenericFormat',
+      'copy-before-write':'BlockdevOptionsCbw',
       'copy-on-read':'BlockdevOptionsCor',
       'dmg':        'BlockdevOptionsGenericFormat',
       'file':       'BlockdevOptionsFile',
Signature · 1 line
2.29.2
#61 lore raw

Re: [PATCH 20/21] qapi: publish copy-before-write filter

Quoted 32 lines · Vladimir Sementsov-Ogievskiy
---
  qapi/block-core.json | 22 ++++++++++++++++++----
  1 file changed, 18 insertions(+), 4 deletions(-)
qapi/block-core.json +6 -4
index 2ea294129e..f8fbcb6416 100644
@@ -2808,15 +2808,17 @@
  # @blklogwrites: Since 3.0
  # @blkreplay: Since 4.2
  # @compress: Since 5.0
+# @copy-before-write: Since 6.1
  #
  # Since: 2.9
  ##
  { 'enum': 'BlockdevDriver',
    'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
-            'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
-            'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
-            'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
-            'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+            'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
+            'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device',
+            'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio',
+            'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2',
+            'qed', 'quorum', 'raw', 'rbd',
              { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
              'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
 

qapi/block-core.json line 3946

Quoted 8 lines
qapi/block-core.json +5 -0
@@ -3937,6 +3939,17 @@
    'base': 'BlockdevOptionsGenericFormat',
    'data': { '*bottom': 'str' } }
 
+##
+# @BlockdevOptionsCbw:
+#
+# Driver specific block device options for the copy-before-write driver.
+#

I think there should be a description of @target here.

(Looks good otherwise, though.)

Max

Quoted 18 lines
qapi/block-core.json +7 -0
+# Since: 6.1
+##
+{ 'struct': 'BlockdevOptionsCbw',
+  'base': 'BlockdevOptionsGenericFormat',
+  'data': { 'target': 'BlockdevRef' } }
+
  ##
  # @BlockdevOptions:
  #
@@ -3989,6 +4002,7 @@
        'bochs':      'BlockdevOptionsGenericFormat',
        'cloop':      'BlockdevOptionsGenericFormat',
        'compress':   'BlockdevOptionsGenericFormat',
+      'copy-before-write':'BlockdevOptionsCbw',
        'copy-on-read':'BlockdevOptionsCor',
        'dmg':        'BlockdevOptionsGenericFormat',
        'file':       'BlockdevOptionsFile',
#62 lore raw

Re: [PATCH 20/21] qapi: publish copy-before-write filter

Quoted 30 lines · Max Reitz
Quoted 29 lines · Vladimir Sementsov-Ogievskiy

---
  qapi/block-core.json | 22 ++++++++++++++++++----

  1 file changed, 18 insertions(+), 4 deletions(-)
qapi/block-core.json +6 -4
index 2ea294129e..f8fbcb6416 100644
@@ -2808,15 +2808,17 @@
   # @blklogwrites: Since 3.0
   # @blkreplay: Since 4.2
   # @compress: Since 5.0
+# @copy-before-write: Since 6.1
   #
   # Since: 2.9
   ##
   { 'enum': 'BlockdevDriver',
     'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
-            'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
-            'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
-            'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
-            'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+            'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg',
+            'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device',
+            'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio',
+            'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2',
+            'qed', 'quorum', 'raw', 'rbd',
               { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
Quoted 8 lines

qapi/block-core.json line 3944

Quoted 8 lines
qapi/block-core.json +4 -0
               'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
@@ -3937,6 +3939,17 @@
     'base': 'BlockdevOptionsGenericFormat',
     'data': { '*bottom': 'str' } }
+##
+# @BlockdevOptionsCbw:
+#
+# Driver specific block device options for the copy-before-write driver,

which does so called copy-before-write operations: when data is written to the filter, it firstly reads corresponding blocks from its file child and copies them to @target child. After successful copying the write request is propagated to file child. If copying filed, the original write request is failed too and no data is written to file child.

Quoted 3 lines

qapi/block-core.json line 3945

Quoted 1 line
qapi/block-core.json +1 -0
+#

I think there should be a description of @target here.

Right, I forget. Something like:

@target: The target for copy-before-write operations.

Quoted 24 lines

(Looks good otherwise, though.)

Max

Quoted 18 lines
qapi/block-core.json +7 -0
+# Since: 6.1
+##
+{ 'struct': 'BlockdevOptionsCbw',
+  'base': 'BlockdevOptionsGenericFormat',
+  'data': { 'target': 'BlockdevRef' } }
+
   ##
   # @BlockdevOptions:
   #
@@ -3989,6 +4002,7 @@
         'bochs':      'BlockdevOptionsGenericFormat',
         'cloop':      'BlockdevOptionsGenericFormat',
         'compress':   'BlockdevOptionsGenericFormat',
+      'copy-before-write':'BlockdevOptionsCbw',
         'copy-on-read':'BlockdevOptionsCor',
         'dmg':        'BlockdevOptionsGenericFormat',
         'file':       'BlockdevOptionsFile',
Signature · 2 lines
Best regards,
Vladimir
#63 lore raw

PATCH 21/21 itotests/222: add test-case for copy-before-write filter

In reply to Vladimir Sementsov-Ogievskiy

New fleecing method becomes available: copy-before-write filter.

Actually we don't need backup job to setup image fleecing. Add test fore new recommended way of image fleecing.

---
 tests/qemu-iotests/222     | 56 ++++++++++++++++++++++-------
 tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 115 insertions(+), 13 deletions(-)
tests/qemu-iotests/222 +43 -13
index b48afe623e..8f8e1e4d7f 100755
@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000",  "32k"), # Right-end of partial-left [1]
              ("0xdc", "32M",       "32k"), # Left-end of partial-right [2]
              ("0xcd", "0x3ff0000", "64k")] # patterns[3]
 
-with iotests.FilePath('base.img') as base_img_path, \
-     iotests.FilePath('fleece.img') as fleece_img_path, \
-     iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \
-     iotests.VM() as vm:
+def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
+    if use_cbw:
+        log('=== Test filter based fleecing ===')
+    else:
+        log('=== Test backup(sync=none) based fleecing ===')
 
+    log('')
     log('--- Setting up images ---')
     log('')
 
@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \
     log('--- Launching VM ---')
     log('')
 
-    vm.add_drive(base_img_path)
+    vm.add_drive(base_img_path, "node-name=node0")
     vm.launch()
     log('Done')
 
@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \
         "backing": src_node,
     }))
 
-    # Establish COW from source to fleecing node
-    log(vm.qmp("blockdev-backup",
-               device=src_node,
-               target=tgt_node,
-               sync="none"))
+    # Establish CBW from source to fleecing node
+    if use_cbw:
+        log(vm.qmp("blockdev-add", **{
+            "driver": "copy-before-write",
+            "node-name": "fl-cbw",
+            "file": src_node,
+            "target": tgt_node
+        }))
+
+        log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+                   property="drive", value="fl-cbw"))
+    else:
+        log(vm.qmp("blockdev-backup",
+                   device=src_node,
+                   target=tgt_node,
+                   sync="none"))
 
     log('')
     log('--- Setting up NBD Export ---')
@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \
     log('--- Cleanup ---')
     log('')
 
-    log(vm.qmp('block-job-cancel', device=src_node))
-    log(vm.event_wait('BLOCK_JOB_CANCELLED'),
-        filters=[iotests.filter_qmp_event])
+    if use_cbw:
+        log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+                   property="drive", value="node0"))
+        log(vm.qmp("blockdev-del", node_name="fl-cbw"))
+    else:
+        log(vm.qmp('block-job-cancel', device=src_node))
+        log(vm.event_wait('BLOCK_JOB_CANCELLED'),
+            filters=[iotests.filter_qmp_event])
+
     log(vm.qmp('nbd-server-stop'))
     log(vm.qmp('blockdev-del', node_name=tgt_node))
     vm.shutdown()
@@ -157,3 +176,14 @@ with iotests.FilePath('base.img') as base_img_path, \
 
     log('')
     log('Done')
+
+def test(use_cbw):
+    with iotests.FilePath('base.img') as base_img_path, \
+         iotests.FilePath('fleece.img') as fleece_img_path, \
+         iotests.FilePath('nbd.sock',
+                          base_dir=iotests.sock_dir) as nbd_sock_path, \
+         iotests.VM() as vm:
+        do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm)
+
+test(False)
+test(True)
tests/qemu-iotests/222.out +72 -0

78 lines not on this page · raw message

Signature · 1 line
2.29.2
#64 lore raw

Re: [PATCH 21/21] itotests/222: add test-case for copy-before-write filter

Quoted 4 lines · Vladimir Sementsov-Ogievskiy

New fleecing method becomes available: copy-before-write filter.

Actually we don't need backup job to setup image fleecing. Add test fore new recommended way of image fleecing.

*for

Quoted 6 lines
---
  tests/qemu-iotests/222     | 56 ++++++++++++++++++++++-------
  tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++
  2 files changed, 115 insertions(+), 13 deletions(-)

Looks good, just wondering about some variable usage (why is src_node a BB name?) and whether the virtio-blk qdev path couldn’t be expressed in some nicer way.

Quoted 23 lines
tests/qemu-iotests/222 +6 -4
index b48afe623e..8f8e1e4d7f 100755
@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000",  "32k"), # Right-end of partial-left [1]
               ("0xdc", "32M",       "32k"), # Left-end of partial-right [2]
               ("0xcd", "0x3ff0000", "64k")] # patterns[3]
 
-with iotests.FilePath('base.img') as base_img_path, \
-     iotests.FilePath('fleece.img') as fleece_img_path, \
-     iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \
-     iotests.VM() as vm:
+def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
+    if use_cbw:
+        log('=== Test filter based fleecing ===')
+    else:
+        log('=== Test backup(sync=none) based fleecing ===')
 
+    log('')
      log('--- Setting up images ---')
      log('')
 

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 77

Quoted 8 lines
tests/qemu-iotests/222 +1 -1
@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \
      log('--- Launching VM ---')
      log('')
 
-    vm.add_drive(base_img_path)
+    vm.add_drive(base_img_path, "node-name=node0")
      vm.launch()
      log('Done')
 

Afterwards, src_node is set to 'drive0'. So src_node is actually a BB name...

Quoted 6 lines
tests/qemu-iotests/222 +0 -2
@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \
          "backing": src_node,
      }))
 
-    # Establish COW from source to fleecing node
-    log(vm.qmp("blockdev-backup",

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 102

Quoted 10 lines
tests/qemu-iotests/222 +7 -3
-               device=src_node,
-               target=tgt_node,
-               sync="none"))
+    # Establish CBW from source to fleecing node
+    if use_cbw:
+        log(vm.qmp("blockdev-add", **{
+            "driver": "copy-before-write",
+            "node-name": "fl-cbw",
+            "file": src_node,
+            "target": tgt_node

Which makes this look strange, and also TIL (or maybe “today I was reminded”?) that you can use BB names as node references.

(It’s also already weird in the "backing": src_node line right at the beginning of this hunk...)

So I guess it works, but I’d find it would be nicer to distinguish between the node name and the BB name.

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 106

Quoted 4 lines
tests/qemu-iotests/222 +4 -0
+        }))
+
+        log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+                   property="drive", value="fl-cbw"))

Is that path portable across targets? Should we maybe instead manually add a virtio-scsi device that we can give a proper name?

Or I suppose we could do

path = next(dev for dev in vm.qmp('query-block')['return']

            if dev['device'] == 'drive0')['qdev']
Quoted 9 lines
tests/qemu-iotests/222 +5 -0
+    else:
+        log(vm.qmp("blockdev-backup",
+                   device=src_node,
+                   target=tgt_node,
+                   sync="none"))
 
      log('')
      log('--- Setting up NBD Export ---')

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 157

Quoted 9 lines
tests/qemu-iotests/222 +3 -3
@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \
      log('--- Cleanup ---')
      log('')
 
-    log(vm.qmp('block-job-cancel', device=src_node))
-    log(vm.event_wait('BLOCK_JOB_CANCELLED'),
-        filters=[iotests.filter_qmp_event])
+    if use_cbw:
+        log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+                   property="drive", value="node0"))

If src_node were 'node0', we wouldn’t need to use the 'node0' literal here and could use src_node instead. (Because it kind of seems like this test wants to use those variables instead of literals. I mean, we could also just call the node 'src-node', but, well.)

Max

Quoted 9 lines
tests/qemu-iotests/222 +6 -0
+        log(vm.qmp("blockdev-del", node_name="fl-cbw"))
+    else:
+        log(vm.qmp('block-job-cancel', device=src_node))
+        log(vm.event_wait('BLOCK_JOB_CANCELLED'),
+            filters=[iotests.filter_qmp_event])
+
      log(vm.qmp('nbd-server-stop'))
      log(vm.qmp('blockdev-del', node_name=tgt_node))
      vm.shutdown()
#65 lore raw

Re: [PATCH 21/21] itotests/222: add test-case for copy-before-write filter

Quoted 65 lines · Max Reitz
Quoted 4 lines · Vladimir Sementsov-Ogievskiy

New fleecing method becomes available: copy-before-write filter.

Actually we don't need backup job to setup image fleecing. Add test fore new recommended way of image fleecing.

*for

Quoted 6 lines

---
  tests/qemu-iotests/222     | 56 ++++++++++++++++++++++------- tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++

  2 files changed, 115 insertions(+), 13 deletions(-)

Looks good, just wondering about some variable usage (why is src_node a BB name?) and whether the virtio-blk qdev path couldn’t be expressed in some nicer way.

Quoted 20 lines
tests/qemu-iotests/222 +0 -0
index b48afe623e..8f8e1e4d7f 100755
@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000",  "32k"), # Right-end of partial-left [1]

               ("0xdc", "32M",       "32k"), # Left-end of partial-right [2] ("0xcd", "0x3ff0000", "64k")] # patterns[3]

tests/qemu-iotests/222 +6 -4
-with iotests.FilePath('base.img') as base_img_path, \
-     iotests.FilePath('fleece.img') as fleece_img_path, \
-     iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \
-     iotests.VM() as vm:
+def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm):
+    if use_cbw:
+        log('=== Test filter based fleecing ===')
+    else:
+        log('=== Test backup(sync=none) based fleecing ===')
+    log('')

      log('--- Setting up images ---')
      log('')

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 76

Quoted 7 lines
tests/qemu-iotests/222 +1 -1
@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \
       log('--- Launching VM ---')
       log('')
-    vm.add_drive(base_img_path)
+    vm.add_drive(base_img_path, "node-name=node0")
       vm.launch()
       log('Done')
 

Afterwards, src_node is set to 'drive0'.  So src_node is actually a BB name...

Quoted 5 lines
tests/qemu-iotests/222 +0 -2
@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \
           "backing": src_node,
       }))
-    # Establish COW from source to fleecing node
-    log(vm.qmp("blockdev-backup",

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 101

Quoted 10 lines
tests/qemu-iotests/222 +7 -3
-               device=src_node,
-               target=tgt_node,
-               sync="none"))
+    # Establish CBW from source to fleecing node
+    if use_cbw:
+        log(vm.qmp("blockdev-add", **{
+            "driver": "copy-before-write",
+            "node-name": "fl-cbw",
+            "file": src_node,
+            "target": tgt_node

Which makes this look strange, and also TIL (or maybe “today I was reminded”?) that you can use BB names as node references.

Quoted 3 lines

(It’s also already weird in the "backing": src_node line right at the beginning of this hunk...)

So I guess it works, but I’d find it would be nicer to distinguish between the node name and the BB name.

Oh yes, that's all the mess, I'll improve.

Quoted 6 lines

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 106

Quoted 5 lines
tests/qemu-iotests/222 +4 -0
+        }))
+
+        log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+                   property="drive", value="fl-cbw"))
Quoted 6 lines

Is that path portable across targets?  Should we maybe instead manually add a virtio-scsi device that we can give a proper name?

Or I suppose we could do

path = next(dev for dev in vm.qmp('query-block')['return']
            if dev['device'] == 'drive0')['qdev']

OK

Quoted 20 lines
Quoted 9 lines
tests/qemu-iotests/222 +5 -0
+    else:
+        log(vm.qmp("blockdev-backup",
+                   device=src_node,
+                   target=tgt_node,
+                   sync="none"))
       log('')
       log('--- Setting up NBD Export ---')

tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 156

Quoted 8 lines
tests/qemu-iotests/222 +3 -3
@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \
       log('--- Cleanup ---')
       log('')
-    log(vm.qmp('block-job-cancel', device=src_node))
-    log(vm.event_wait('BLOCK_JOB_CANCELLED'),
-        filters=[iotests.filter_qmp_event])
+    if use_cbw:
+        log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]",
+                   property="drive", value="node0"))

If src_node were 'node0', we wouldn’t need to use the 'node0' literal here and could use src_node instead.  (Because it kind of seems like this test wants to use those variables instead of literals.  I mean, we could also just call the node 'src-node', but, well.)

Quoted 12 lines

Max

Quoted 9 lines
tests/qemu-iotests/222 +6 -0
+        log(vm.qmp("blockdev-del", node_name="fl-cbw"))
+    else:
+        log(vm.qmp('block-job-cancel', device=src_node))
+        log(vm.event_wait('BLOCK_JOB_CANCELLED'),
+            filters=[iotests.filter_qmp_event])
+
       log(vm.qmp('nbd-server-stop'))
       log(vm.qmp('blockdev-del', node_name=tgt_node))
       vm.shutdown()

Great thanks for reviewing!

Signature · 2 lines
Best regards,
Vladimir