block: publish backup-top filter
Subsystems (2)
cover block: publish backup-top filter
Hi all!
We have image fleecing scheme to export point-in-time state of active disk (iotest 222):
backup(sync=none)
┌───────────────────────────────────────┐
▼ │
┌────────────┐ ┌────────────────┐ backing ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘ └────────────────┘ └─────────────┘
▲
┌────────────┐ │
│ guest blk │ ──────────────────────────────────────────────┘
└────────────┘ Actually, backup job inserts a backup-top filter, so in detail it looks like:
backup(sync=none)
┌───────────────────────────────────────┐
▼ │
┌────────────┐ ┌────────────────┐ backing ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘ └────────────────┘ └─────────────┘
▲ ▲
│ target │
│ │
┌────────────┐ ┌────────────────┐ backing │
│ guest blk │ ──▶ │ backup-top │ ───────────────────────┘
└────────────┘ └────────────────┘And job does nothing here. In a new blockdev world user is intended to operate on node level, and insert/remove filters by hand. Let's get rid of job in the scheme:
┌────────────┐ ┌────────────────┐ backing ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘ └────────────────┘ └─────────────┘
▲ ▲
│ target │
│ │
┌────────────┐ ┌────────────────┐ backing │
│ guest blk │ ──▶ │ backup-top │ ───────────────────────┘
└────────────┘ └────────────────┘The series prepares qom-set to make possible inserting filters above root node (patches 01-04), rename backup-top to copy-before-write, do other preparations for publishing the filter, and finally publish it, add qapi interface and test new fleecing scheme in 222.
Vladimir Sementsov-Ogievskiy (21):
block: introduce bdrv_replace_child_bs()
block: introduce blk_replace_bs
qdev-properties: PropertyInfo: add realized_set_allowed field
qdev: allow setting drive property for realized device
block: rename backup-top to copy-before-write block/backup: drop support for copy_range
block-copy: always set BDRV_REQ_SERIALISING flag block/backup: stricter backup_calculate_cluster_size() block/backup: move cluster size calculation to block-copy block/copy-before-write: relax permission requirements when no parents block/copy-before-write: use file child instead of backing block/copy-before-write: bdrv_cbw_append(): replace child at last block/copy-before-write: introduce cbw_init() block/copy-before-write: cbw_init(): rename variables block/copy-before-write: cbw_init(): use file child after attaching block/copy-before-write: cbw_init(): use options block/block-copy: switch to fully set bitmap by default block/block-copy: make setting progress optional block/copy-before-write: make public block driver
qapi: publish copy-before-write filter
itotests/222: add test-case for copy-before-write filter
qapi/block-core.json | 22 +-
block/{backup-top.h => copy-before-write.h} | 26 +-
include/block/block-copy.h | 4 +-
include/block/block.h | 2 +
include/hw/qdev-properties.h | 1 +
include/sysemu/block-backend.h | 1 +
block.c | 36 +++
block/backup-top.c | 253 -------------------
block/backup.c | 115 ++-------
block/block-backend.c | 8 +
block/block-copy.c | 95 +++++++-
block/copy-before-write.c | 257 ++++++++++++++++++++
hw/core/qdev-properties-system.c | 30 ++-
hw/core/qdev-properties.c | 6 +-
MAINTAINERS | 4 +-
block/meson.build | 2 +-
tests/qemu-iotests/222 | 56 ++++-
tests/qemu-iotests/222.out | 72 ++++++
tests/qemu-iotests/283 | 35 ++-
tests/qemu-iotests/283.out | 4 +-
20 files changed, 606 insertions(+), 423 deletions(-)
rename block/{backup-top.h => copy-before-write.h} (56%)
delete mode 100644 block/backup-top.c
create mode 100644 block/copy-before-write.cSignature · 1 line
2.29.2
PATCH 1/1 block/copy-on-read: use bdrv_drop_filter() and drop s->active
Now, after huge update of block graph permission update algorithm, we don't need this workaround with active state of the filter. Drop it and use new smart bdrv_drop_filter() function.
--- block/copy-on-read.c | 33 +-------------------------------- 1 file changed, 1 insertion(+), 32 deletions(-)
block/copy-on-read.c +1 -32
@@ -29,7 +29,6 @@typedef struct BDRVStateCOR { -bool active;BlockDriverState *bottom_bs;bool chain_frozen; }BDRVStateCOR ;
@@ -89,7 +88,6 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags,*/ bdrv_ref (bottom_bs); } - state->active = true; state->bottom_bs = bottom_bs;/*
@@ -112,17 +110,6 @@ static void cor_child_perm(BlockDriverState *bs, BdrvChild *c, uint64_t perm, uint64_t shared, uint64_t *nperm, uint64_t *nshared) { - BDRVStateCOR *s = bs->opaque; - -if (!s->active) { -/* -* While the filter is being removed -*/ - *nperm =0 ; - *nshared =BLK_PERM_ALL ; -return ; - } - *nperm = perm &PERM_PASSTHROUGH ; *nshared = (shared &PERM_PASSTHROUGH ) |PERM_UNCHANGED ;
@@ -280,32 +267,14 @@ static BlockDriver bdrv_copy_on_read = {void bdrv_cor_filter_drop (BlockDriverState *cor_filter_bs) { -BdrvChild *child; -BlockDriverState *bs;BDRVStateCOR *s = cor_filter_bs->opaque; - child =bdrv_filter_child (cor_filter_bs); -if (!child) { -return ; - } - bs = child->bs; - -/* Retain the BDS until we complete the graph change. */ -bdrv_ref (bs); -/* Hold a guest back from writing while permissions are being reset. */ -bdrv_drained_begin (bs); -/* Drop permissions before the graph change. */ - s->active = false;/* unfreeze, as otherwise bdrv_replace_node() will fail */ if (s->chain_frozen) { s->chain_frozen = false;bdrv_unfreeze_backing_chain (cor_filter_bs, s->bottom_bs); } -bdrv_child_refresh_perms (cor_filter_bs, child, &error_abort); -bdrv_replace_node (cor_filter_bs, bs, &error_abort); - -bdrv_drained_end (bs); -bdrv_unref (bs); +bdrv_drop_filter (cor_filter_bs, &error_abort);bdrv_unref (cor_filter_bs); }
Signature · 1 line
2.29.2
PATCH 1/21 block: introduce bdrv_replace_child_bs()
In reply to Vladimir Sementsov-OgievskiyAdd function to transactionally replace bs inside BdrvChild.
--- include/block/block.h | 2 ++ block.c | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+)
include/block/block.h +2 -0
@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp);int bdrv_replace_node (BlockDriverState *from,BlockDriverState *to,Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, +Error **errp);BlockDriverState *bdrv_insert_node (BlockDriverState *bs,QDict *node_options,int flags,Error **errp);int bdrv_drop_filter (BlockDriverState *bs,Error **errp);
block.c +36 -0
@@ -4961,6 +4961,42 @@ out:return ret; } +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, +Error **errp) +{ +int ret; +Transaction *tran =tran_new (); +g_autoptr (GHashTable) found =NULL ; +g_autoptr (GSList) refresh_list =NULL ; +BlockDriverState *old_bs = child->bs; + +if (old_bs) { +bdrv_ref (old_bs); +bdrv_drained_begin (old_bs); + } +bdrv_drained_begin (new_bs); + +bdrv_replace_child (child, new_bs, tran); + + found =g_hash_table_new (NULL ,NULL ); +if (old_bs) { + refresh_list =bdrv_topological_dfs (refresh_list, found, old_bs); + } + refresh_list =bdrv_topological_dfs (refresh_list, found, new_bs); + + ret =bdrv_list_refresh_perms (refresh_list,NULL , tran, errp); + +tran_finalize (tran, ret); + +if (old_bs) { +bdrv_drained_end (old_bs); +bdrv_unref (old_bs); + } +bdrv_drained_end (new_bs); + +return ret; +} +static void bdrv_delete (BlockDriverState *bs) {assert (bdrv_op_blocker_is_empty (bs));
Signature · 1 line
2.29.2
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/block.h | 2 ++ block.c | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+)
As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to bdrv_replace_node_common() to make me happy.
Quoted 21 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp);int bdrv_replace_node (BlockDriverState *from,BlockDriverState *to,Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, +Error **errp);BlockDriverState *bdrv_insert_node (BlockDriverState *bs,QDict *node_options,int flags,Error **errp);int bdrv_drop_filter (BlockDriverState *bs,Error **errp);block.c +0
-0@@ -4961,6 +4961,42 @@ out:return ret; }
block.c out: · line 4973
Quoted 10 lines
block.c +10
-0+int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, +Error **errp) +{ +int ret; +Transaction *tran =tran_new (); +g_autoptr (GHashTable) found =NULL ; +g_autoptr (GSList) refresh_list =NULL ; +BlockDriverState *old_bs = child->bs; + +if (old_bs) {
Hm. Can child->bs be ever NULL?
block.c out: · line 4977
Quoted 4 lines
block.c +4
-0+bdrv_ref (old_bs); +bdrv_drained_begin (old_bs); + } +bdrv_drained_begin (new_bs);
(I was wondering why we couldn’t handle the new_bs == NULL case here to replace bdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely because child->bs at least should generally be non-NULL. Which is why bdrv_remove_filter_or_cow_child() needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.
Hence me wondering whether we could assume child->bs not to be NULL.)
block.c out: · line 4987
Quoted 10 lines
block.c +10
-0+ +bdrv_replace_child (child, new_bs, tran); + + found =g_hash_table_new (NULL ,NULL ); +if (old_bs) { + refresh_list =bdrv_topological_dfs (refresh_list, found, old_bs); + } + refresh_list =bdrv_topological_dfs (refresh_list, found, new_bs); + + ret =bdrv_list_refresh_perms (refresh_list,NULL , tran, errp);
Speaking of bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general?
Would there be a problem if we refreshed permissions there? Or is not
refreshing permissions just part of the function’s interface?
Max
Quoted 16 lines
block.c +12
-0+ +tran_finalize (tran, ret); + +if (old_bs) { +bdrv_drained_end (old_bs); +bdrv_unref (old_bs); + } +bdrv_drained_end (new_bs); + +return ret; +} +static void bdrv_delete (BlockDriverState *bs) {assert (bdrv_op_blocker_is_empty (bs));
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
Quoted 35 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/block/block.h| 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++2 files changed, 38 insertions(+)As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to
bdrv_replace_node_common()to make me happy.Quoted 24 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp); int bdrv_drop_filter(BlockDriverState *bs, Error **errp);block.c +4
-0@@ -4961,6 +4961,42 @@ out: return ret; } +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp) +{ + int ret;
Quoted 8 lines
block.c out: · line 4973
Quoted 6 lines
block.c +6
-0+ Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + if (old_bs) {Hm. Can
child->bsbe ever NULL?
Hmm. Most probably not :)
In some intermediate states we don't have bs in child, but it shouldn't be the place where bdrv_replace_child_bs is called.
Quoted 9 lines
block.c out: · line 4977
Quoted 5 lines
block.c +4
-0+ bdrv_ref(old_bs); + bdrv_drained_begin(old_bs); + } + bdrv_drained_begin(new_bs);(I was wondering why we couldn’t handle the
new_bs== NULL case here to replacebdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely becausechild->bsat least should generally be non-NULL. Which is whybdrv_remove_filter_or_cow_child()needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.Hence me wondering whether we could assume
child->bsnot to be NULL.)
bdrv_remove_filter_or_cow_child() is "lower leve" function: it doesn't do drained section nor permission update. And new bdrv_replace_child_bs() is public function, which cares about these things.
Quoted 14 lines
block.c out: · line 4987
Quoted 11 lines
block.c +10
-0+ + bdrv_replace_child(child, new_bs, tran); + + found = g_hash_table_new(NULL, NULL); + if (old_bs) { + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs); + } + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs); + + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);Speaking of
bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
Quoted 2 lines
I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
Caller of bdrv_remove_filter_or_cow_child() should care about permissions: bdrv_replace_node_common() do this, and bdrv_set_backing_noperm() has "_noperm" in the name..
The main impact of previous big rework of permission is new scheme of working with permission update:
- first do all graph modifications, not thinking about permissions
- refresh permissions for the whole updated subgraph
- if refresh failed, rollback all the modifications (main sense if transactions here and there is possibility to do this rollback)
So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.
Quoted 18 lines
Quoted 17 lines
block.c +12
-0+ + tran_finalize(tran, ret); + + if (old_bs) { + bdrv_drained_end(old_bs); + bdrv_unref(old_bs); + } + bdrv_drained_end(new_bs); + + return ret; +} + static void bdrv_delete(BlockDriverState *bs) { assert(bdrv_op_blocker_is_empty(bs));
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
Quoted 77 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/block/block.h| 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++2 files changed, 38 insertions(+)As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to
bdrv_replace_node_common()to make me happy.Quoted 25 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new,BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp);int
bdrv_drop_filter(BlockDriverState *bs, Error **errp);block.c +4
-0@@ -4961,6 +4961,42 @@ out: return ret; } +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp) +{ + int ret;Quoted 8 lines
block.c out: · line 4973
Quoted 6 lines
block.c +6
-0+ Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + if (old_bs) {Hm. Can
child->bsbe ever NULL?Hmm. Most probably not :)
In some intermediate states we don't have bs in child, but it shouldn't be the place where
bdrv_replace_child_bsis called.Quoted 9 lines
block.c out: · line 4977
Quoted 5 lines
block.c +4
-0+ bdrv_ref(old_bs); + bdrv_drained_begin(old_bs); + } + bdrv_drained_begin(new_bs);(I was wondering why we couldn’t handle the
new_bs== NULL case here to replacebdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely becausechild->bsat least should generally be non-NULL. Which is whybdrv_remove_filter_or_cow_child()needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.Hence me wondering whether we could assume
child->bsnot to be NULL.)
bdrv_remove_filter_or_cow_child()is "lower leve" function: it doesn't do drained section nor permission update. And newbdrv_replace_child_bs()is public function, which cares about these things.Quoted 15 lines
block.c out: · line 4987
Quoted 12 lines
block.c +10
-0+ + bdrv_replace_child(child, new_bs, tran); + + found = g_hash_table_new(NULL, NULL); + if (old_bs) { + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs); + } + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs); + + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);Speaking of
bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
Quoted 4 lines
Quoted 2 lines
I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
Caller of
bdrv_remove_filter_or_cow_child()should care about permissions:bdrv_replace_node_common()do this, andbdrv_set_backing_noperm()has "_noperm" in the name..
OK. Makes me wonder why bdrv_remove_filter_or_cow_child() then doesn’t have _noperm in its name, or why its comment doesn’t explain this interface contract, but, well. :)
Quoted 8 lines
The main impact of previous big rework of permission is new scheme of working with permission update:
- first do all graph modifications, not thinking about permissions
- refresh permissions for the whole updated subgraph
- if refresh failed, rollback all the modifications (main sense if
transactions here and there is possibility to do this rollback)So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.
Ah, OK. Makes sense, thanks.
Max
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
Quoted 76 lines · Max Reitz
Quoted 75 lines · Vladimir Sementsov-Ogievskiy
Quoted 35 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/block/block.h| 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++2 files changed, 38 insertions(+)As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to
bdrv_replace_node_common()to make me happy.Quoted 24 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp); int bdrv_drop_filter(BlockDriverState *bs, Error **errp);block.c +4
-0@@ -4961,6 +4961,42 @@ out: return ret; } +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp) +{ + int ret;Quoted 8 lines
block.c out: · line 4973
Quoted 6 lines
block.c +6
-0+ Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + if (old_bs) {Hm. Can
child->bsbe ever NULL?Hmm. Most probably not :)
In some intermediate states we don't have bs in child, but it shouldn't be the place where
bdrv_replace_child_bsis called.Quoted 9 lines
block.c out: · line 4977
Quoted 5 lines
block.c +4
-0+ bdrv_ref(old_bs); + bdrv_drained_begin(old_bs); + } + bdrv_drained_begin(new_bs);(I was wondering why we couldn’t handle the
new_bs== NULL case here to replacebdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely becausechild->bsat least should generally be non-NULL. Which is whybdrv_remove_filter_or_cow_child()needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.Hence me wondering whether we could assume
child->bsnot to be NULL.)
bdrv_remove_filter_or_cow_child()is "lower leve" function: it doesn't do drained section nor permission update. And newbdrv_replace_child_bs()is public function, which cares about these things.Quoted 14 lines
block.c out: · line 4987
Quoted 11 lines
block.c +10
-0+ + bdrv_replace_child(child, new_bs, tran); + + found = g_hash_table_new(NULL, NULL); + if (old_bs) { + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs); + } + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs); + + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);Speaking of
bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
Caller of
bdrv_remove_filter_or_cow_child()should care about permissions:bdrv_replace_node_common()do this, andbdrv_set_backing_noperm()has "_noperm" in the name..OK. Makes me wonder why
bdrv_remove_filter_or_cow_child()then doesn’t have_nopermin its name, or why its comment doesn’t explain this interface contract, but, well. :)
You are right that's unclear. I'll make the patch that cleans that up.
Quoted 13 lines
Quoted 8 lines
The main impact of previous big rework of permission is new scheme of working with permission update:
- first do all graph modifications, not thinking about permissions
- refresh permissions for the whole updated subgraph
- if refresh failed, rollback all the modifications (main sense if transactions here and there is possibility to do this rollback)So a lot of internal functions with @tran argument don't update permissions. But of course, we should care to update permissions after any graph modification.
Ah, OK. Makes sense, thanks.
Max
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
In reply to Max ReitzQuoted 35 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/block/block.h| 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++2 files changed, 38 insertions(+)As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to
bdrv_replace_node_common()to make me happy.Quoted 24 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp); int bdrv_drop_filter(BlockDriverState *bs, Error **errp);block.c +4
-0@@ -4961,6 +4961,42 @@ out: return ret; } +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp) +{ + int ret;
Quoted 8 lines
block.c out: · line 4973
Quoted 6 lines
block.c +6
-0+ Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + if (old_bs) {Hm. Can
child->bsbe ever NULL?
Seems it can. For example we have hmp_drive_del command, that removes bs from blk :(
qmp eject and blockdev-remove-medium seems do it too.
Quoted 44 lines
block.c out: · line 4977
Quoted 5 lines
block.c +4
-0+ bdrv_ref(old_bs); + bdrv_drained_begin(old_bs); + } + bdrv_drained_begin(new_bs);(I was wondering why we couldn’t handle the
new_bs== NULL case here to replacebdrv_remove_filter_or_cow_child(), but then I realized it’s probably because that’s kind of difficult, precisely becausechild->bsat least should generally be non-NULL. Which is whybdrv_remove_filter_or_cow_child()needs to add its own transaction entry to handle the BdrvChild object and the pointer to it.Hence me wondering whether we could assume
child->bsnot to be NULL.)block.c out: · line 4987
Quoted 10 lines
block.c +10
-0+ + bdrv_replace_child(child, new_bs, tran); + + found = g_hash_table_new(NULL, NULL); + if (old_bs) { + refresh_list = bdrv_topological_dfs(refresh_list, found, old_bs); + } + refresh_list = bdrv_topological_dfs(refresh_list, found, new_bs); + + ret = bdrv_list_refresh_perms(refresh_list, NULL, tran, errp);Speaking of
bdrv_remove_filter_or_cow_child(): That function doesn’t refresh permissions. I think it’s correct to do it here, so the following question doesn’t really concern this patch, but: Why don’t we do it there?I guess it’s because we expect the node to go away anyway, so we don’t need to refresh the permissions. And that assumption should hold true right now, given its callers. But is that a safe assumption in general? Would there be a problem if we refreshed permissions there? Or is not refreshing permissions just part of the function’s interface?
Max
Quoted 16 lines
block.c +12
-0+ + tran_finalize(tran, ret); + + if (old_bs) { + bdrv_drained_end(old_bs); + bdrv_unref(old_bs); + } + bdrv_drained_end(new_bs); + + return ret; +} + static void bdrv_delete(BlockDriverState *bs) { assert(bdrv_op_blocker_is_empty(bs));
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
Quoted 46 lines · Vladimir Sementsov-Ogievskiy
Quoted 36 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/block/block.h| 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++2 files changed, 38 insertions(+)As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to
bdrv_replace_node_common()to make me happy.Quoted 25 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new,BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp);int
bdrv_drop_filter(BlockDriverState *bs, Error **errp);block.c +4
-0@@ -4961,6 +4961,42 @@ out: return ret; } +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp) +{ + int ret;Quoted 8 lines
block.c out: · line 4973
Quoted 6 lines
block.c +6
-0+ Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + if (old_bs) {Hm. Can
child->bsbe ever NULL?
Quoted 3 lines
Seems it can. For example we have
hmp_drive_delcommand, that removes bs from blk :(qmp eject and blockdev-remove-medium seems do it too.
blk_remove_bs() doesn’t eject the BDS from the BdrvChild blk->root, though, it drops blk->root altogether. Doesn’t it?
Max
Re: [PATCH 01/21] block: introduce bdrv_replace_child_bs()
Quoted 46 lines · Max Reitz
Quoted 45 lines · Vladimir Sementsov-Ogievskiy
Quoted 43 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/block/block.h| 2 ++
block.c | 36 ++++++++++++++++++++++++++++++++++++2 files changed, 38 insertions(+)As you may guess, I know little about the rewritten replacing functions, so this is kind of difficult to review for me. However, nothing looks out of place, and the function looks sufficiently similar to
bdrv_replace_node_common()to make me happy.Quoted 20 lines
include/block/block.h +2
-0@@ -361,6 +361,8 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top, Error **errp); int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to, Error **errp); +int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp); BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options, int flags, Error **errp); int bdrv_drop_filter(BlockDriverState *bs, Error **errp);block.c +0
-0@@ -4961,6 +4961,42 @@ out: return ret; }block.c out: · line 4972
Quoted 10 lines
block.c +10
-0+int bdrv_replace_child_bs (BdrvChild *child,BlockDriverState *new_bs, + Error **errp) +{ + int ret; + Transaction *tran = tran_new(); + g_autoptr(GHashTable) found = NULL; + g_autoptr(GSList) refresh_list = NULL; + BlockDriverState *old_bs = child->bs; + + if (old_bs) {Hm. Can
child->bsbe ever NULL?
Quoted 6 lines
Quoted 3 lines
Seems it can. For example we have
hmp_drive_delcommand, that removes bs from blk :(qmp eject and blockdev-remove-medium seems do it too.
blk_remove_bs()doesn’t eject the BDS from the BdrvChildblk->root, though, it dropsblk->rootaltogether. Doesn’t it?
A hm, yes. What I say is that we can have empty blk. But probably we should never have BdrvChild with NULL bs. I'll check it
Signature · 2 lines
Best regards, Vladimir
PATCH 2/21 block: introduce blk_replace_bs
In reply to Vladimir Sementsov-OgievskiyAdd function to change bs inside blk.
--- include/sysemu/block-backend.h | 1 + block/block-backend.c | 8 ++++++++ 2 files changed, 9 insertions(+)
include/sysemu/block-backend.h +1 -0
@@ -98,6 +98,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public);BlockDriverState *blk_bs (BlockBackend *blk);void blk_remove_bs (BlockBackend *blk);int blk_insert_bs (BlockBackend *blk,BlockDriverState *bs,Error **errp); +int blk_replace_bs (BlockBackend *blk,BlockDriverState *new_bs,Error **errp);bool bdrv_has_blk (BlockDriverState *bs);bool bdrv_is_root_node (BlockDriverState *bs); int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,
block/block-backend.c +8 -0
@@ -870,6 +870,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp)return 0 ; } +/* +* Change BlockDriverState associated with @blk. +*/ +int blk_replace_bs (BlockBackend *blk,BlockDriverState *new_bs,Error **errp) +{ +return bdrv_replace_child_bs (blk->root, new_bs, errp); +} +/* * Sets the permission bitmasks that the user of the BlockBackend needs. */
Signature · 1 line
2.29.2
Re: [PATCH 02/21] block: introduce blk_replace_bs
Quoted 25 lines · Vladimir Sementsov-Ogievskiy
Add function to change bs inside blk.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/sysemu/block-backend.h | 1 + block/block-backend.c | 8 ++++++++ 2 files changed, 9 insertions(+)include/sysemu/block-backend.h +1
-0@@ -98,6 +98,7 @@ BlockBackend *blk_by_public(BlockBackendPublic *public);BlockDriverState *blk_bs (BlockBackend *blk);void blk_remove_bs (BlockBackend *blk);int blk_insert_bs (BlockBackend *blk,BlockDriverState *bs,Error **errp); +int blk_replace_bs (BlockBackend *blk,BlockDriverState *new_bs,Error **errp);bool bdrv_has_blk (BlockDriverState *bs);bool bdrv_is_root_node (BlockDriverState *bs); int blk_set_perm(BlockBackend *blk, uint64_t perm, uint64_t shared_perm,block/block-backend.c +0
-0
block/block-backend.c int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp) · line 879
Quoted 10 lines
block/block-backend.c +7
-0@@ -870,6 +870,14 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, Error **errp)return 0 ; } +/* +* Change BlockDriverState associated with @blk. +*/ +int blk_replace_bs (BlockBackend *blk,BlockDriverState *new_bs,Error **errp) +{ +return bdrv_replace_child_bs (blk->root, new_bs, errp); +}
(Looks indeed like we don’t need to do any of the things that blk_insert_bs() and blk_remove_bs() do besides inserting and removing the node.)
PATCH 3/21 qdev-properties: PropertyInfo: add realized_set_allowed field
In reply to Vladimir Sementsov-OgievskiyAdd field, so property can declare support for setting the property when device is realized. To be used in the following commit.
--- include/hw/qdev-properties.h | 1 + hw/core/qdev-properties.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-)
include/hw/qdev-properties.h +1 -0
@@ -32,6 +32,7 @@ struct PropertyInfo {const char *name;const char *description;const QEnumLookup *enum_table; +bool realized_set_allowed;int (*print)(Object *obj,Property *prop,char *dest,size_t len);void (*set_default_value)(ObjectProperty *op,const Property *prop); ObjectProperty *(*create)(ObjectClass *oc, const char *name,
hw/core/qdev-properties.c +3 -3
@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,/* returns: true if property is allowed to be set, false otherwise */ static bool qdev_prop_allow_set (Object *obj,const char *name, -Error **errp) +const PropertyInfo *info,Error **errp) {DeviceState *dev =DEVICE (obj); -if (dev->realized) { +if (dev->realized && !info->realized_set_allowed) {qdev_prop_set_after_realize (dev, name, errp);return false; }
@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name, {Property *prop = opaque; -if (!qdev_prop_allow_set (obj, name, errp)) { +if (!qdev_prop_allow_set (obj, name, prop->info, errp)) {return ; }
Signature · 1 line
2.29.2
Re: [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Add field, so property can declare support for setting the property when device is realized. To be used in the following commit.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/hw/qdev-properties.h | 1 + hw/core/qdev-properties.c | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-)
Looks OK to me, although qdev isn’t my specialty.
include/hw/qdev-properties.h struct PropertyInfo { · line 35
Quoted 9 lines
include/hw/qdev-properties.h +1
-0@@ -32,6 +32,7 @@ struct PropertyInfo {const char *name;const char *description;const QEnumLookup *enum_table; +bool realized_set_allowed;
I think a comment would be nice, though.
Max
Quoted 31 lines
include/hw/qdev-properties.h +0
-0int (*print)(Object *obj,Property *prop,char *dest,size_t len);void (*set_default_value)(ObjectProperty *op,const Property *prop); ObjectProperty *(*create)(ObjectClass *oc, const char *name,hw/core/qdev-properties.c +3
-3@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name,/* returns: true if property is allowed to be set, false otherwise */ static bool qdev_prop_allow_set (Object *obj,const char *name, -Error **errp) +const PropertyInfo *info,Error **errp) {DeviceState *dev =DEVICE (obj); -if (dev->realized) { +if (dev->realized && !info->realized_set_allowed) {qdev_prop_set_after_realize (dev, name, errp);return false; }@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name, {Property *prop = opaque; -if (!qdev_prop_allow_set (obj, name, errp)) { +if (!qdev_prop_allow_set (obj, name, prop->info, errp)) {return ; }
Re: [PATCH 03/21] qdev-properties: PropertyInfo: add realized_set_allowed field
Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Add field, so property can declare support for setting the property when device is realized. To be used in the following commit.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
include/hw/qdev-properties.h| 1 +
hw/core/qdev-properties.c | 6 +++---2 files changed, 4 insertions(+), 3 deletions(-)Looks OK to me, although qdev isn’t my specialty.
Neither my :) Thanks for looking anyway!
Quoted 13 lines
include/hw/qdev-properties.h struct PropertyInfo { · line 35
Quoted 10 lines
include/hw/qdev-properties.h +1
-0@@ -32,6 +32,7 @@ struct PropertyInfo { const char *name; const char *description; const QEnumLookup *enum_table; + bool realized_set_allowed;I think a comment would be nice, though.
Agree, will add.
Quoted 29 lines
Quoted 28 lines
include/hw/qdev-properties.h +0
-0int (*print)(Object *obj, Property *prop, char *dest, size_t len); void (*set_default_value)(ObjectProperty *op, const Property *prop); ObjectProperty *(*create)(ObjectClass *oc, const char *name,hw/core/qdev-properties.c +3
-3@@ -26,11 +26,11 @@ void qdev_prop_set_after_realize(DeviceState *dev, const char *name, /* returns: true if property is allowed to be set, false otherwise */ static bool qdev_prop_allow_set(Object *obj, const char *name, - Error **errp) + const PropertyInfo *info, Error **errp) { DeviceState *dev = DEVICE(obj); - if (dev->realized) { + if (dev->realized && !info->realized_set_allowed) { qdev_prop_set_after_realize(dev, name, errp); return false; }@@ -79,7 +79,7 @@ static void field_prop_set(Object *obj, Visitor *v, const char *name, { Property *prop = opaque; - if (!qdev_prop_allow_set(obj, name, errp)) { + if (!qdev_prop_allow_set(obj, name, prop->info, errp)) { return; }
Signature · 2 lines
Best regards, Vladimir
PATCH 4/21 qdev: allow setting drive property for realized device
In reply to Vladimir Sementsov-OgievskiyWe need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.
Intended usage:
1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.
2. qom-set, to change bs attached to root blk from original node to newly create filter.
--- hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-)
hw/core/qdev-properties-system.c +22 -8
@@ -93,16 +93,30 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,BlockBackend *blk;bool blk_created = false;int ret; +BlockDriverState *bs; +AioContext *ctx;if (!visit_type_str (v, name, &str, errp)) {return ; } -/* -* TODO Should this really be an error? If no, the old value -* needs to be released before we store the new one. -*/ -if (!check_prop_still_unset (obj, name, *ptr, str, errp)) { +if (*ptr) { +/* BlockBackend alread exists. So, we want to change attached node */ + blk = *ptr; + ctx =blk_get_aio_context (blk); + bs =bdrv_lookup_bs (NULL , str, errp); +if (!bs) { +return ; + } + +if (ctx !=bdrv_get_aio_context (bs)) { +error_setg (errp,"Different aio context is not supported for new " +"node" ); + } + +aio_context_acquire (ctx); +blk_replace_bs (blk, bs, errp); +aio_context_release (ctx);return ; }
@@ -114,7 +128,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name, blk =blk_by_name (str);if (!blk) { -BlockDriverState *bs =bdrv_lookup_bs (NULL , str,NULL ); + bs =bdrv_lookup_bs (NULL , str,NULL );if (bs) {/* * If the device supports iothreads, it will make sure to move the
@@ -123,8 +137,7 @@ static void set_drive_helper(Object *obj, Visitor *v, const char *name,* aware of iothreads require their BlockBackends to be in the main * AioContext. */ -AioContext *ctx = iothread ?bdrv_get_aio_context (bs) : -qemu_get_aio_context (); + ctx = iothread ?bdrv_get_aio_context (bs) :qemu_get_aio_context (); blk =blk_new (ctx,0 ,BLK_PERM_ALL ); blk_created = true;
@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)const PropertyInfo qdev_prop_drive = { .name ="str" , .description ="Node name or ID of a block device to use as a backend" , + .realized_set_allowed = true, .get = get_drive, .set = set_drive, .release = release_drive,
Signature · 1 line
2.29.2
Re: [PATCH 04/21] qdev: allow setting drive property for realized device
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.
Intended usage:
1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.
Is a “not” missing here, i.e. “not attached to any guest block device”?
I would have thought one would create a filtered tree that is not in
use by any frontend, so that the filter need not take any permissions.
Quoted 7 lines
2. qom-set, to change bs attached to root blk from original node to newly create filter.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- hw/core/qdev-properties-system.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-)
Looks good, just one question: (well, two, one was above)
Quoted 4 lines
hw/core/qdev-properties-system.c +0
-0
[...]
hw/core/qdev-properties-system.c static void release_drive(Object *obj, const char *name, void *opaque) · line 215
Quoted 8 lines
hw/core/qdev-properties-system.c +1
-0@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque)const PropertyInfo qdev_prop_drive = { .name ="str" , .description ="Node name or ID of a block device to use as a backend" , + .realized_set_allowed = true, .get = get_drive, .set = set_drive, .release = release_drive,
Why not for qdev_prop_drive_iothread?
Max
Re: [PATCH 04/21] qdev: allow setting drive property for realized device
Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.
Intended usage:
1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.
Is a “not” missing here, i.e. “not attached to any guest block device”? I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.
node A is attached.
So, we have [blk] --root-> [A}
And want to insert a filter between blk and A.
We do
1.
[filter] --file--\
v
[blk] --root--> [A]
2.
[blk] --root--> [filer] --file--> [A]
Quoted 18 lines
Quoted 8 lines
2. qom-set, to change bs attached to root blk from original node to newly create filter.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
hw/core/qdev-properties-system.c| 30 ++++++++++++++++++++++--------1 file changed, 22 insertions(+), 8 deletions(-)Looks good, just one question: (well, two, one was above)
Quoted 4 lines
hw/core/qdev-properties-system.c +0
-0[...]
Quoted 11 lines
hw/core/qdev-properties-system.c static void release_drive(Object *obj, const char *name, void *opaque) · line 215
Quoted 8 lines
hw/core/qdev-properties-system.c +1
-0@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque) const PropertyInfo qdev_prop_drive = { .name = "str", .description = "Node name or ID of a block device to use as a backend", + .realized_set_allowed = true, .get = get_drive, .set = set_drive, .release = release_drive,Why not for
qdev_prop_drive_iothread?
Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 04/21] qdev: allow setting drive property for realized device
Quoted 19 lines · Vladimir Sementsov-Ogievskiy
Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.
Intended usage:
1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.
Is a “not” missing here, i.e. “not attached to any guest block device”? I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.
node A is attached.
So, we have [blk] --root-> [A}
And want to insert a filter between blk and A.
Quoted 7 lines
We do
1.
[filter] --file--\
v
[blk] --root--> [A]
Oh, so you mean node A is attached to a guest device. The sentence sounded to me like the newly created filter tree were attached to it.
Yes, that’s how I expected it to be. I just find the sentence not quite clear, because I found it ambiguous which node the “attached to some guest block device” refers to.
Perhaps:
“Intended usage:
Assume there is a node A that is attached to some guest device.
1. blockdev-add to create a filter node B that has A as its child.
2. qom-set to change the node attached to the guest device’s BlockBackend from A to B.”
?
Quoted 36 lines
2.
[blk] --root--> [filer] --file--> [A]
Quoted 18 lines
Quoted 8 lines
2. qom-set, to change bs attached to root blk from original node to newly create filter.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
hw/core/qdev-properties-system.c| 30 ++++++++++++++++++++++--------1 file changed, 22 insertions(+), 8 deletions(-)Looks good, just one question: (well, two, one was above)
Quoted 4 lines
hw/core/qdev-properties-system.c +0
-0[...]
Quoted 12 lines
Quoted 9 lines
hw/core/qdev-properties-system.c +1
-0@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque) const PropertyInfo qdev_prop_drive = { .name = "str", .description = "Node name or ID of a block device to use as a backend", + .realized_set_allowed = true, .get = get_drive, .set = set_drive,.release =
release_drive,Why not for
qdev_prop_drive_iothread?
Quoted 1 line
Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version
OK. (I just saw it because it was right below this structure, too, it isn’t like I actually know what I’m saying.)
Max
Re: [PATCH 04/21] qdev: allow setting drive property for realized device
Quoted 32 lines · Max Reitz
Quoted 19 lines · Vladimir Sementsov-Ogievskiy
Quoted 11 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set.
Intended usage:
1. blockdev-add, creating the filter, which child is at top node A, attached to some guest block device.
Is a “not” missing here, i.e. “not attached to any guest block device”? I would have thought one would create a filtered tree that is not in use by any frontend, so that the filter need not take any permissions.
node A is attached.
So, we have [blk] --root-> [A}
And want to insert a filter between blk and A.
Quoted 7 lines
We do
1.
[filter] --file--\
v
[blk] --root--> [A]Oh, so you mean node A is attached to a guest device. The sentence sounded to me like the newly created filter tree were attached to it.
Yes, that’s how I expected it to be. I just find the sentence not quite clear, because I found it ambiguous which node the “attached to some guest block device” refers to.
Quoted 8 lines
Perhaps:
“Intended usage:Assume there is a node A that is attached to some guest device.
1. blockdev-add to create a filter node B that has A as its child.
2. qom-set to change the node attached to the guest device’s BlockBackend from A to B.”?
Yes, sounds good, thanks
Quoted 42 lines
Quoted 36 lines
2.
[blk] --root--> [filer] --file--> [A]
Quoted 18 lines
Quoted 8 lines
2. qom-set, to change bs attached to root blk from original node to newly create filter.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
hw/core/qdev-properties-system.c| 30 ++++++++++++++++++++++--------1 file changed, 22 insertions(+), 8 deletions(-)Looks good, just one question: (well, two, one was above)
Quoted 4 lines
hw/core/qdev-properties-system.c +0
-0[...]
Quoted 11 lines
hw/core/qdev-properties-system.c static void release_drive(Object *obj, const char *name, void *opaque) · line 215
Quoted 8 lines
hw/core/qdev-properties-system.c +1
-0@@ -196,6 +209,7 @@ static void release_drive(Object *obj, const char *name, void *opaque) const PropertyInfo qdev_prop_drive = { .name = "str", .description = "Node name or ID of a block device to use as a backend", + .realized_set_allowed = true, .get = get_drive, .set = set_drive, .release = release_drive,Why not for
qdev_prop_drive_iothread?Quoted 1 line
Hmm, the only reason is that I missed that part of architecture around here, I'm new to qdev code. Will add with next version
OK. (I just saw it because it was right below this structure, too, it isn’t like I actually know what I’m saying.)
Max
Signature · 2 lines
Best regards, Vladimir
PATCH 5/21 block: rename backup-top to copy-before-write
In reply to Vladimir Sementsov-OgievskiyWe are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".
While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.
---
block/{backup-top.h => copy-before-write.h} | 28 +++---
block/backup.c | 22 ++---
block/{backup-top.c => copy-before-write.c} | 100 ++++++++++----------
MAINTAINERS | 4 +-
block/meson.build | 2 +-
tests/qemu-iotests/283 | 35 +++----
tests/qemu-iotests/283.out | 4 +-
7 files changed, 95 insertions(+), 100 deletions(-)
rename block/{backup-top.h => copy-before-write.h} (56%)
rename block/{backup-top.c => copy-before-write.c} (62%)block/copy-before-write.h renamed from block/backup-top.h +14 -14
@@ -1,10 +1,10 @@/* -* backup-top filter driver +* copy-before-write filter driver * * The driver performs Copy-Before-Write (CBW) operation: it is injected above * some node, and before each write it copies _old_ data to the target node. * -* Copyright (c) 2018-2019 Virtuozzo International GmbH. +* Copyright (c) 2018-2021 Virtuozzo International GmbH. * * Author: * Sementsov-Ogievskiy Vladimir <[email protected]>
@@ -23,20 +23,20 @@* along with this program. If not, see <http://www.gnu.org/licenses/>. */ -#ifndef BACKUP_TOP_H -#define BACKUP_TOP_H +#ifndef COPY_BEFORE_WRITE_H +#define COPY_BEFORE_WRITE_H #include "block/block_int.h" #include "block/block-copy.h" -BlockDriverState *bdrv_backup_top_append (BlockDriverState *source, -BlockDriverState *target, -const char *filter_node_name, -uint64_t cluster_size, -BackupPerf *perf, -BdrvRequestFlags write_flags, -BlockCopyState **bcs, -Error **errp); -void bdrv_backup_top_drop (BlockDriverState *bs); +BlockDriverState *bdrv_cbw_append (BlockDriverState *source, +BlockDriverState *target, +const char *filter_node_name, +uint64_t cluster_size, +BackupPerf *perf, +BdrvRequestFlags write_flags, +BlockCopyState **bcs, +Error **errp); +void bdrv_cbw_drop (BlockDriverState *bs); -#endif /* BACKUP_TOP_H */ +#endif /* COPY_BEFORE_WRITE_H */
block/backup.c +11 -11
@@ -27,13 +27,13 @@#include "qemu/bitmap.h" #include "qemu/error-report.h" -#include "block/backup-top.h" +#include "block/copy-before-write.h" #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) typedef struct BackupBlockJob { BlockJob common; - BlockDriverState *backup_top; + BlockDriverState *cbw; BlockDriverState *source_bs; BlockDriverState *target_bs;
@@ -104,7 +104,7 @@ static void backup_clean(Job *job) {BackupBlockJob *s =container_of (job, BackupBlockJob, common.job);block_job_remove_all_bdrv (&s->common); -bdrv_backup_top_drop (s->backup_top); +bdrv_cbw_drop (s->cbw); }void backup_do_checkpoint (BlockJob *job,Error **errp)
@@ -408,7 +408,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,BackupBlockJob *job =NULL ;int64_t cluster_size;BdrvRequestFlags write_flags; -BlockDriverState *backup_top =NULL ; +BlockDriverState *cbw =NULL ;BlockCopyState *bcs =NULL ;assert (bs);
@@ -521,22 +521,22 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, write_flags = (bdrv_chain_contains (target, bs) ?BDRV_REQ_SERIALISING :0 ) | (compress ?BDRV_REQ_WRITE_COMPRESSED :0 ), - backup_top =bdrv_backup_top_append (bs, target, filter_node_name, + cbw =bdrv_cbw_append (bs, target, filter_node_name, cluster_size, perf, write_flags, &bcs, errp); -if (!backup_top) { +if (!cbw) { goto error; }/* job->len is fixed, so we can't allow resize */ - job =block_job_create (job_id, &backup_job_driver, txn, backup_top, + job =block_job_create (job_id, &backup_job_driver, txn, cbw,0 ,BLK_PERM_ALL , speed, creation_flags, cb, opaque, errp);if (!job) { goto error; } - job->backup_top = backup_top; + job->cbw = cbw; job->source_bs = bs; job->target_bs = target; job->on_source_error = on_source_error;
@@ -552,7 +552,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,block_copy_set_progress_meter (bcs, &job->common.job.progress);block_copy_set_speed (bcs, speed); -/* Required permissions are already taken by backup-top target */ +/* Required permissions are taken by copy-before-write filter target */ block_job_add_bdrv (&job->common,"target" , target,0 ,BLK_PERM_ALL , &error_abort);
@@ -562,8 +562,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,if (sync_bitmap) {bdrv_reclaim_dirty_bitmap (sync_bitmap,NULL ); } -if (backup_top) { -bdrv_backup_top_drop (backup_top); +if (cbw) { +bdrv_cbw_drop (cbw); }return NULL ;
block/copy-before-write.c renamed from block/backup-top.c +50 -50
206 lines not on this page · raw message
MAINTAINERS +2 -2
@@ -2281,8 +2281,8 @@ F: block/mirror.c F: qapi/job.json F: block/block-copy.c F: include/block/block-copy.c -F: block/backup-top.h -F: block/backup-top.c +F: block/copy-before-write.h +F: block/copy-before-write.c F: include/block/aio_task.h F: block/aio_task.c F: util/qemu-co-shared-resource.c
block/meson.build +1 -1
@@ -4,7 +4,7 @@ block_ss.add(files( 'aio_task.c', 'amend.c', 'backup.c', - 'backup-top.c', + 'copy-before-write.c', 'blkdebug.c', 'blklogwrites.c', 'blkverify.c',
tests/qemu-iotests/283 +15 -20
@@ -1,7 +1,7 @@ #!/usr/bin/env python3 # group: auto quick # -# Test for backup-top filter permission activation failure +# Test for copy-before-write filter permission conflict # # Copyright (c) 2019 Virtuozzo International GmbH. #
@@ -31,13 +31,13 @@ size = 1024 * 1024 """ Test description When performing a backup, all writes on the source subtree must go through the -backup-top filter so it can copy all data to the target before it is changed. -backup-top filter is appended above source node, to achieve this thing, so all -parents of source node are handled. A configuration with side parents of source -sub-tree with write permission is unsupported (we'd have append several -backup-top filter like nodes to handle such parents). The test create an -example of such configuration and checks that a backup is then not allowed -(blockdev-backup command should fail). +copy-before-write filter so it can copy all data to the target before it is +changed. copy-before-write filter is appended above source node, to achieve +this thing, so all parents of source node are handled. A configuration with +side parents of source sub-tree with write permission is unsupported (we'd have +append several copy-before-write filter like nodes to handle such parents). The +test create an example of such configuration and checks that a backup is then +not allowed (blockdev-backup command should fail). The configuration:
@@ -57,11 +57,10 @@ The configuration: │ base │ ◀──────────── │ other │ └─────────────┘ └───────┘ -On activation (see .active field of backup-top state in block/backup-top.c), -backup-top is going to unshare write permission on its source child. Write -unsharing will be propagated to the "source->base" link and will conflict with -other node write permission. So permission update will fail and backup job will -not be started. +copy-before-write filter wants to unshare write permission on its source child. +Write unsharing will be propagated to the "source->base" link and will conflict +with other node write permission. So permission update will fail and backup job +will not be started. Note, that the only thing which prevents backup of running on such configuration is default permission propagation scheme. It may be altered by
@@ -99,13 +98,9 @@ vm.qmp_log('blockdev-backup', sync='full', device='source', target='target') vm.shutdown() -print('\n=== backup-top should be gone after job-finalize ===\n') +print('\n=== copy-before-write filter should be gone after job-finalize ===\n') -# Check that the backup-top node is gone after job-finalize. -# -# During finalization, the node becomes inactive and can no longer -# function. If it is still present, new parents might be attached, and -# there would be no meaningful way to handle their I/O requests. +# Check that the copy-before-write node is gone after job-finalize. vm = iotests.VM() vm.launch()
@@ -131,7 +126,7 @@ vm.qmp_log('blockdev-backup', vm.event_wait('BLOCK_JOB_PENDING', 5.0) -# The backup-top filter should still be present prior to finalization +# The copy-before-write filter should still be present prior to finalization assert vm.node_info('backup-filter') is not None vm.qmp_log('job-finalize', id='backup')
tests/qemu-iotests/283.out +2 -2
@@ -5,9 +5,9 @@ {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}} {"return": {}} {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}} -{"error": {"class": "GenericError", "desc": "Cannot append backup-top filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}} +{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}} -=== backup-top should be gone after job-finalize === +=== copy-before-write filter should be gone after job-finalize === {"execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name": "source"}} {"return": {}}
Signature · 1 line
2.29.2
Re: [PATCH 05/21] block: rename backup-top to copy-before-write
Quoted 3 lines · Vladimir Sementsov-Ogievskiy
We are going to convert
backup_topto full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".
Is this safe? The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)
(Regardless, renaming files and so on is fine, of course.)
Quoted 16 lines
While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/{backup-top.h => copy-before-write.h} | 28 +++--- block/backup.c | 22 ++--- block/{backup-top.c => copy-before-write.c} | 100 ++++++++++---------- MAINTAINERS | 4 +- block/meson.build | 2 +- tests/qemu-iotests/283 | 35 +++---- tests/qemu-iotests/283.out | 4 +- 7 files changed, 95 insertions(+), 100 deletions(-)rename
block/{backup-top.h=>copy-before-write.h} (56%) renameblock/{backup-top.c=>copy-before-write.c} (62%)
[...]
Quoted 7 lines
block/copy-before-write.c renamed from block/backup-top.c +0
-0
[...]
Quoted 1 line
block/copy-before-write.c renamed from block/backup-top.c +0
-0@@ -32,25 +32,25 @@
[...]
block/copy-before-write.c line 51
Quoted 4 lines
block/copy-before-write.c renamed from block/backup-top.c +2
-2-static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset, -uint64_t bytes,BdrvRequestFlags flags) +static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset, +uint64_t bytes,BdrvRequestFlags flags)
I’m sure you noticed it, too, but cbw_cbw() is weird. Perhaps cbw_do_cbw() at least?
Max
Re: [PATCH 05/21] block: rename backup-top to copy-before-write
Quoted 8 lines · Max Reitz
Quoted 3 lines · Vladimir Sementsov-Ogievskiy
We are going to convert
backup_topto full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".Is this safe? The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)
(Regardless, renaming files and so on is fine, of course.)
Hmmm. I don't know.
It was visible yes.. But we've never documented it. And if someone depends on name of the format of the filter automatically inserted during backup job, it's a kind of "undocumented feature" use..
Another change I is changing child from backing to file in 11, from this point of view it's unsafe too. But ше even more reasonable than good name: having all public filters behave similar is a very good thing.
So, may be it a bit risky, but I think good interface worth that risk. And we always can say "sorry guys, but that was not documented, we didn't promise anything".
But I'm OK to go on with "backup-top" and "backing", is someone has strict opinion about this.
Quoted 34 lines
Quoted 17 lines
While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/{backup-top.h=>copy-before-write.h} | 28 +++--- block/backup.c | 22 ++---block/{backup-top.c=>copy-before-write.c} | 100 ++++++++++---------- MAINTAINERS | 4 +- block/meson.build | 2 +- tests/qemu-iotests/283 | 35 +++---- tests/qemu-iotests/283.out | 4 +-7 files changed, 95 insertions(+), 100 deletions(-)rename
block/{backup-top.h=>copy-before-write.h} (56%) renameblock/{backup-top.c=>copy-before-write.c} (62%)[...]
Quoted 7 lines
block/copy-before-write.c renamed from block/backup-top.c +0
-0[...]
Quoted 1 line
block/copy-before-write.c renamed from block/backup-top.c +0
-0@@ -32,25 +32,25 @@[...]
Quoted 7 lines
block/copy-before-write.c line 51
Quoted 4 lines
block/copy-before-write.c renamed from block/backup-top.c +2
-2-static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset, - uint64_t bytes, BdrvRequestFlags flags) +static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, BdrvRequestFlags flags)I’m sure you noticed it, too, but
cbw_cbw()is weird. Perhapscbw_do_cbw()at least?
OK. Maybe even cbw_do_copy_before_write()
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 05/21] block: rename backup-top to copy-before-write
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 8 lines · Max Reitz
Quoted 3 lines · Vladimir Sementsov-Ogievskiy
We are going to convert
backup_topto full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does".Is this safe? The name was externally visible in queries after all. (I’m not saying it is unsafe, I just don’t know and would like to know whether you’ve considered this already.)
(Regardless, renaming files and so on is fine, of course.)
Quoted 7 lines
Hmmm. I don't know.
It was visible yes.. But we've never documented it. And if someone depends on name of the format of the filter automatically inserted during backup job, it's a kind of "undocumented feature" use..
Another change I is changing child from backing to file in 11, from this point of view it's unsafe too. But ше even more reasonable than good name: having all public filters behave similar is a very good thing.
So, may be it a bit risky, but I think good interface worth that risk. And we always can say "sorry guys, but that was not documented, we didn't promise anything".
OK.
Max
Quoted 49 lines
But I'm OK to go on with "backup-top" and "backing", is someone has strict opinion about this.
Quoted 34 lines
Quoted 17 lines
While updating comments in 283 iotest, drop and rephrase also things about ".active", as this field is now dropped, and filter doesn't have "inactive" mode.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/{backup-top.h=>copy-before-write.h} | 28 +++--- block/backup.c | 22 ++---block/{backup-top.c=>copy-before-write.c} | 100 ++++++++++---------- MAINTAINERS | 4 +- block/meson.build | 2 +- tests/qemu-iotests/283 | 35 +++---- tests/qemu-iotests/283.out | 4 +-7 files changed, 95 insertions(+), 100 deletions(-)rename
block/{backup-top.h=>copy-before-write.h} (56%) renameblock/{backup-top.c=>copy-before-write.c} (62%)[...]
Quoted 7 lines
block/copy-before-write.c renamed from block/backup-top.c +0
-0[...]
Quoted 1 line
block/copy-before-write.c renamed from block/backup-top.c +0
-0@@ -32,25 +32,25 @@[...]
Quoted 9 lines
Quoted 6 lines
-static
coroutine_fnint backup_top_cbw(BlockDriverState *bs,uint64_toffset,
-uint64_tbytes, BdrvRequestFlags flags)
+staticcoroutine_fnint cbw_cbw(BlockDriverState *bs,uint64_toffset, +uint64_tbytes, BdrvRequestFlags flags)I’m sure you noticed it, too, but
cbw_cbw()is weird. Perhapscbw_do_cbw()at least?OK. Maybe even
cbw_do_copy_before_write()
PATCH 6/21 block/backup: drop support for copy_range
In reply to Vladimir Sementsov-Ogievskiycopy_range is not a default behavior since 6a30f663d4c0b3c, and it's now available only though x-perf experimantal argument, so it's OK to drop it.
Even when backup is used to copy disk to same filesystem, and filesystem supports zero-copy copy_range, copy_range is probably not what we want for backup: backup has good property of making a copy of active disk, with no impact to active disk itself (unlike creating a snapshot). And if copy_range instead of copying data adds fs-level references, and on next guest write COW operation occurs, it's seems most possible, that new block will be allocated for original vm disk, not for backup disk. Thus, fragmentation of original disk will increase.
We can simply add support back on demand. Now we want to publish copy-before-write filter, and instead of thinking how to pass use-copy-range argument to block-copy (create x-block-copy parameter for new public filter driver, or may be set it by hand after filter node creation?), instead of this let's just drop copy-range support in backup for now.
After this patch copy-range support in block-copy becomes unused. Let's keep it for a while, it won't hurt:
1. If there would be request for supporting copy_range in backup (and/or in a new public copy-before-write filter), it will be easy to satisfy it.
2. Probably, qemu-img convert will reuse block-copy, and qemu-img has option to enable copy-range. qemu-img convert is not a backup, and copy_range may be more reasonable for some cases in context of qemu-img convert.
--- block/copy-before-write.h | 1 - block/backup.c | 3 +-- block/copy-before-write.c | 4 +--- 3 files changed, 2 insertions(+), 6 deletions(-)
block/copy-before-write.h +0 -1
@@ -33,7 +33,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, - BackupPerf *perf, BdrvRequestFlags write_flags, BlockCopyState **bcs, Error **errp);
block/backup.c +1 -2
@@ -522,8 +522,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, (compress ?BDRV_REQ_WRITE_COMPRESSED :0 ), cbw =bdrv_cbw_append (bs, target, filter_node_name, - cluster_size, perf, - write_flags, &bcs, errp); + cluster_size, write_flags, &bcs, errp);if (!cbw) { goto error; }
block/copy-before-write.c +1 -3
@@ -170,7 +170,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, - BackupPerf *perf, BdrvRequestFlags write_flags, BlockCopyState **bcs, Error **errp)
@@ -217,8 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->cluster_size = cluster_size; state->bcs =block_copy_state_new (top->backing, state->target, - cluster_size, perf->use_copy_range, - write_flags, errp); + cluster_size, false, write_flags, errp);if (!state->bcs) {error_prepend (errp,"Cannot create block-copy-state: " ); goto fail;
Signature · 1 line
2.29.2
Re: [PATCH 06/21] block/backup: drop support for copy_range
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
copy_rangeis not a default behavior since6a30f663d4c0b3c, and it's now available only though x-perf experimantal argument, so it's OK to drop it.
Quoted 9 lines
Even when backup is used to copy disk to same filesystem, and filesystem supports zero-copy
copy_range,copy_rangeis probably not what we want for backup: backup has good property of making a copy of active disk, with no impact to active disk itself (unlike creating a snapshot). And ifcopy_rangeinstead of copying data adds fs-level references, and on next guest write COW operation occurs, it's seems most possible, that new block will be allocated for original vm disk, not for backup disk. Thus, fragmentation of original disk will increase.
Good point.
Quoted 19 lines
We can simply add support back on demand. Now we want to publish copy-before-write filter, and instead of thinking how to pass use-copy-range argument to block-copy (create x-block-copy parameter for new public filter driver, or may be set it by hand after filter node creation?), instead of this let's just drop copy-range support in backup for now.
After this patch copy-range support in block-copy becomes unused. Let's keep it for a while, it won't hurt:
1. If there would be request for supporting
copy_rangein backup (and/or in a new public copy-before-write filter), it will be easy to satisfy it.2. Probably, qemu-img convert will reuse block-copy, and qemu-img has option to enable copy-range. qemu-img convert is not a backup, and
copy_rangemay be more reasonable for some cases in context of qemu-img convert.
Quoted 6 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.h | 1 - block/backup.c | 3 +-- block/copy-before-write.c | 4 +--- 3 files changed, 2 insertions(+), 6 deletions(-)
PATCH 7/21 block-copy: always set BDRV_REQ_SERIALISING flag
In reply to Vladimir Sementsov-OgievskiyIt won't hurt in common case, so let's not bother with detecting image fleecing.
Also, we want to simplify initialization interface of copy-before-write filter as we are going to make it public.
--- block/copy-before-write.h | 2 +- include/block/block-copy.h | 3 +-- block/backup.c | 20 +------------------- block/block-copy.c | 29 ++++++++++++++++++++++++++--- block/copy-before-write.c | 4 ++-- 5 files changed, 31 insertions(+), 27 deletions(-)
block/copy-before-write.h +1 -1
@@ -33,7 +33,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, - BdrvRequestFlags write_flags, + bool compress, BlockCopyState **bcs, Error **errp);void bdrv_cbw_drop (BlockDriverState *bs);
include/block/block-copy.h +1 -2
@@ -24,8 +24,7 @@ typedef struct BlockCopyCallState BlockCopyCallState;BlockCopyState *block_copy_state_new (BdrvChild *source,BdrvChild *target,int64_t cluster_size,bool use_copy_range, -BdrvRequestFlags write_flags, -Error **errp); +bool compress,Error **errp);void block_copy_set_progress_meter (BlockCopyState *s,ProgressMeter *pm);
block/backup.c +1 -19
@@ -407,7 +407,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,int64_t len, target_len;BackupBlockJob *job =NULL ;int64_t cluster_size; -BdrvRequestFlags write_flags;BlockDriverState *cbw =NULL ;BlockCopyState *bcs =NULL ;
@@ -504,25 +503,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } -/* -* If source is in backing chain of target assume that target is going to be -* used for "image fleecing", i.e. it should represent a kind of snapshot of -* source at backup-start point in time. And target is going to be read by -* somebody (for example, used as NBD export) during backup job. -* -* In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid -* intersection of backup writes and third party reads from target, -* otherwise reading from target we may occasionally read already updated by -* guest data. -* -* For more information see commit f8d59dfb40bb and test -* tests/qemu-iotests/222 -*/ - write_flags = (bdrv_chain_contains (target, bs) ?BDRV_REQ_SERIALISING :0 ) | - (compress ?BDRV_REQ_WRITE_COMPRESSED :0 ), - cbw =bdrv_cbw_append (bs, target, filter_node_name, - cluster_size, write_flags, &bcs, errp); + cluster_size, compress, &bcs, errp);if (!cbw) { goto error; }
block/block-copy.c +26 -3
@@ -245,7 +245,7 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target)BlockCopyState *block_copy_state_new (BdrvChild *source,BdrvChild *target,int64_t cluster_size,bool use_copy_range, -BdrvRequestFlags write_flags,Error **errp) +bool compress,Error **errp) {BlockCopyState *s;BdrvDirtyBitmap *copy_bitmap;
@@ -257,6 +257,28 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, }bdrv_disable_dirty_bitmap (copy_bitmap); +/* +* Why we always set BDRV_REQ_SERIALISING write flag: +* +* Assume source is in backing chain of target assume that target is going +* to be used for "image fleecing", i.e. it should represent a kind of +* snapshot of source at backup-start point in time. And target is going to +* be read by somebody (for example, used as NBD export) during backup job. +* +* In this case, we need to add BDRV_REQ_SERIALISING write flag to avoid +* intersection of backup writes and third party reads from target, +* otherwise reading from target we may occasionally read already updated by +* guest data. +* +* For more information see commit f8d59dfb40bb and test +* tests/qemu-iotests/222 +* +* Other cases? The only known reasonable case is "just copy to target, and +* target is not used for something else". In this case BDRV_REQ_SERIALISING +* change nothing, so let's not bother with detecting the "image fleecing" +* case and enabling BDRV_REQ_SERIALISING only for it. +*/ + s =g_new (BlockCopyState,1 ); *s = (BlockCopyState ) { .source = source,
@@ -264,7 +286,8 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, .copy_bitmap = copy_bitmap, .cluster_size = cluster_size, .len = bdrv_dirty_bitmap_size(copy_bitmap), - .write_flags = write_flags, + .write_flags = BDRV_REQ_SERIALISING | + (compress ? BDRV_REQ_WRITE_COMPRESSED : 0), .mem = shres_create(BLOCK_COPY_MAX_MEM), };
@@ -277,7 +300,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,*/ s->use_copy_range = false; s->copy_size = cluster_size; - }else if (write_flags &BDRV_REQ_WRITE_COMPRESSED ) { + }else if (compress) {/* Compression supports only cluster-size writes and no copy-range. */ s->use_copy_range = false; s->copy_size = cluster_size;
block/copy-before-write.c +2 -2
@@ -170,7 +170,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, - BdrvRequestFlags write_flags, + bool compress, BlockCopyState **bcs, Error **errp) {
@@ -216,7 +216,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->cluster_size = cluster_size; state->bcs =block_copy_state_new (top->backing, state->target, - cluster_size, false, write_flags, errp); + cluster_size, false, compress, errp);if (!state->bcs) {error_prepend (errp,"Cannot create block-copy-state: " ); goto fail;
Signature · 1 line
2.29.2
Re: [PATCH 07/21] block-copy: always set BDRV_REQ_SERIALISING flag
Quoted 6 lines · Vladimir Sementsov-Ogievskiy
It won't hurt in common case, so let's not bother with detecting image fleecing.
Also, we want to simplify initialization interface of copy-before-write filter as we are going to make it public.
Quoted 8 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.h | 2 +- include/block/block-copy.h | 3 +-- block/backup.c | 20 +------------------- block/block-copy.c | 29 ++++++++++++++++++++++++++--- block/copy-before-write.c | 4 ++-- 5 files changed, 31 insertions(+), 27 deletions(-)
PATCH 8/21 block/backup: stricter backup_calculate_cluster_size()
In reply to Vladimir Sementsov-OgievskiyNo reason to tolerate bdrv_get_info() errors except for ENOTSUP. Let's just error-out, it's simpler and safer.
--- block/backup.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
block/backup.c +5 -9
@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,* targets with a backing file, try to avoid COW if possible. */ ret =bdrv_get_info (target, &bdi); -if (ret == -ENOTSUP && !target_does_cow) { +if (ret <0 && ret != -ENOTSUP ) { +error_setg_errno (errp, -ret,"Failed to get target info" ); +return ret; + }else if (ret == -ENOTSUP && !target_does_cow) {/* Cluster size is not defined */ warn_report ("The target block device doesn't provide " "information about the block size and it doesn't have a "
@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,"this default, the backup may be unusable" , BACKUP_CLUSTER_SIZE_DEFAULT);return BACKUP_CLUSTER_SIZE_DEFAULT ; - }else if (ret <0 && !target_does_cow) { -error_setg_errno (errp, -ret, -"Couldn't determine the cluster size of the target image, " -"which has no backing file" ); -error_append_hint (errp, -"Aborting, since this may create an unusable destination image\n" ); -return ret; - }else if (ret <0 && target_does_cow) { + }else if (ret == -ENOTSUP && target_does_cow) {/* Not fatal; just trudge on ahead. */ return BACKUP_CLUSTER_SIZE_DEFAULT ; }
Signature · 1 line
2.29.2
Re: [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()
Quoted 2 lines · Vladimir Sementsov-Ogievskiy
No reason to tolerate
bdrv_get_info()errors except for ENOTSUP. Let's just error-out, it's simpler and safer.
Hm, doesn’t look that much simpler to me. Not sure how much safer it is, because the point was that in the target_does_cow case, we would like a cluster size hint, but it isn’t necessary. So if we don’t get one, regardless of the reason, we use the default cluster size. I don’t know why ENOTSUP should be treated in a special way there.
So I don’t know.
Max
Quoted 38 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/backup.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)block/backup.c +5
-9@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,* targets with a backing file, try to avoid COW if possible. */ ret =bdrv_get_info (target, &bdi); -if (ret == -ENOTSUP && !target_does_cow) { +if (ret <0 && ret != -ENOTSUP ) { +error_setg_errno (errp, -ret,"Failed to get target info" ); +return ret; + }else if (ret == -ENOTSUP && !target_does_cow) {/* Cluster size is not defined */ warn_report ("The target block device doesn't provide " "information about the block size and it doesn't have a "@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,"this default, the backup may be unusable" , BACKUP_CLUSTER_SIZE_DEFAULT);return BACKUP_CLUSTER_SIZE_DEFAULT ; - }else if (ret <0 && !target_does_cow) { -error_setg_errno (errp, -ret, -"Couldn't determine the cluster size of the target image, " -"which has no backing file" ); -error_append_hint (errp, -"Aborting, since this may create an unusable destination image\n" ); -return ret; - }else if (ret <0 && target_does_cow) { + }else if (ret == -ENOTSUP && target_does_cow) {/* Not fatal; just trudge on ahead. */ return BACKUP_CLUSTER_SIZE_DEFAULT ; }
Re: [PATCH 08/21] block/backup: stricter backup_calculate_cluster_size()
Quoted 8 lines · Max Reitz
Quoted 2 lines · Vladimir Sementsov-Ogievskiy
No reason to tolerate
bdrv_get_info()errors except for ENOTSUP. Let's just error-out, it's simpler and safer.Hm, doesn’t look that much simpler to me. Not sure how much safer it is, because the point was that in the
target_does_cowcase, we would like a cluster size hint, but it isn’t necessary. So if we don’t get one, regardless of the reason, we use the default cluster size. I don’t know why ENOTSUP should be treated in a special way there.So I don’t know.
I'm probably OK to drop this for now and don't care. Still, I can share what brings me to this:
First I thought that cluster size should be easily available for any driver:
protocol drivers and not-backing-supporting format drivers can set it to 1 or to request_alignment, if they don't have a "cluster" in mind.
backing-supporting format drivers should of course provide actual cluster size
And I decided to just add bs->cluster_size variable, set on driver open, to simplify the whole thing and make it clean. Then, most this detect-cluster-size function would be just dropped.
But it occurs, that there is one driver, that has a good and rather tricky reason for ENOTSUP: vmdk can have several extents with different cluster size..
So I give up refactored, and finished with this one patch. It can be simply dropped, I am not really a fan of it..
Quoted 40 lines
Quoted 39 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/backup.c| 14 +++++---------1 file changed, 5 insertions(+), 9 deletions(-)block/backup.c +4
-1@@ -367,7 +367,10 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target,* targets with a backing file, try to avoid COW if possible. */ ret = bdrv_get_info(target, &bdi); - if (ret == -ENOTSUP && !target_does_cow) { + if (ret < 0 && ret != -ENOTSUP) { + error_setg_errno(errp, -ret, "Failed to get target info"); + return ret; + } else if (ret == -ENOTSUP && !target_does_cow) { /* Cluster size is not defined */warn_report("The target block device doesn't provide " "information about the block size and it doesn't have a "
block/backup.c +1
-8@@ -376,14 +379,7 @@ static int64_t backup_calculate_cluster_size(BlockDriverState *target, "this default, the backup may be unusable", BACKUP_CLUSTER_SIZE_DEFAULT); return BACKUP_CLUSTER_SIZE_DEFAULT; - } else if (ret < 0 && !target_does_cow) { - error_setg_errno(errp, -ret, - "Couldn't determine the cluster size of the target image, " - "which has no backing file"); - error_append_hint(errp, - "Aborting, since this may create an unusable destination image\n"); - return ret; - } else if (ret < 0 && target_does_cow) { + } else if (ret == -ENOTSUP && target_does_cow) { /* Not fatal; just trudge on ahead. */ return BACKUP_CLUSTER_SIZE_DEFAULT; }
Signature · 2 lines
Best regards, Vladimir
PATCH 9/21 block/backup: move cluster size calculation to block-copy
In reply to Vladimir Sementsov-OgievskiyThe main consumer of cluster-size is block-copy. Let's calculate it here instead of passing through backup-top.
We are going to publish copy-before-write filter soon, so it will be created through options. But we don't want for now to make explicit option for cluster-size, let's continue to calculate it automatically. So, now is the time to get rid of cluster_size argument for bdrv_cbw_append().
--- block/copy-before-write.h | 1 - include/block/block-copy.h | 5 ++-- block/backup.c | 58 ++++++-------------------------------- block/block-copy.c | 47 +++++++++++++++++++++++++++++- block/copy-before-write.c | 10 +++---- 5 files changed, 62 insertions(+), 59 deletions(-)
block/copy-before-write.h +0 -1
@@ -32,7 +32,6 @@BlockDriverState *bdrv_cbw_append (BlockDriverState *source,BlockDriverState *target,const char *filter_node_name, -uint64_t cluster_size,bool compress,BlockCopyState **bcs,Error **errp);
include/block/block-copy.h +3 -2
@@ -23,8 +23,8 @@ typedef struct BlockCopyState BlockCopyState;typedef struct BlockCopyCallState BlockCopyCallState ;BlockCopyState *block_copy_state_new (BdrvChild *source,BdrvChild *target, -int64_t cluster_size,bool use_copy_range, -bool compress,Error **errp); +bool use_copy_range,bool compress, +Error **errp);void block_copy_set_progress_meter (BlockCopyState *s,ProgressMeter *pm);
@@ -86,6 +86,7 @@ void block_copy_kick(BlockCopyCallState *call_state);void block_copy_call_cancel (BlockCopyCallState *call_state);BdrvDirtyBitmap *block_copy_dirty_bitmap (BlockCopyState *s); +int64_t block_copy_cluster_size (BlockCopyState *s);void block_copy_set_skip_unallocated (BlockCopyState *s,bool skip);#endif /* BLOCK_COPY_H */
block/backup.c +9 -49
92 lines not on this page · raw message
block/block-copy.c +46 -1
71 lines not on this page · raw message
block/copy-before-write.c +4 -6
@@ -37,7 +37,6 @@typedef struct BDRVCopyBeforeWriteState {BlockCopyState *bcs;BdrvChild *target; -int64_t cluster_size; }BDRVCopyBeforeWriteState ; static coroutine_fn int cbw_co_preadv(
@@ -52,13 +51,14 @@ static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset, {BDRVCopyBeforeWriteState *s = bs->opaque;uint64_t off, end; +int64_t cluster_size =block_copy_cluster_size (s->bcs);if (flags &BDRV_REQ_WRITE_UNCHANGED ) {return 0 ; } - off =QEMU_ALIGN_DOWN (offset, s->cluster_size); - end =QEMU_ALIGN_UP (offset + bytes, s->cluster_size); + off =QEMU_ALIGN_DOWN (offset, cluster_size); + end =QEMU_ALIGN_UP (offset + bytes, cluster_size);return block_copy (s->bcs, off, end - off, true); }
@@ -169,7 +169,6 @@ BlockDriver bdrv_cbw_filter = {BlockDriverState *bdrv_cbw_append (BlockDriverState *source,BlockDriverState *target,const char *filter_node_name, -uint64_t cluster_size,bool compress,BlockCopyState **bcs,Error **errp)
@@ -214,9 +213,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } appended = true; - state->cluster_size = cluster_size; state->bcs =block_copy_state_new (top->backing, state->target, - cluster_size, false, compress, errp); + false, compress, errp);if (!state->bcs) {error_prepend (errp,"Cannot create block-copy-state: " ); goto fail;
Signature · 1 line
2.29.2
Re: [PATCH 09/21] block/backup: move cluster size calculation to block-copy
Quoted 9 lines · Vladimir Sementsov-Ogievskiy
The main consumer of cluster-size is block-copy. Let's calculate it here instead of passing through backup-top.
We are going to publish copy-before-write filter soon, so it will be created through options. But we don't want for now to make explicit option for cluster-size, let's continue to calculate it automatically. So, now is the time to get rid of
cluster_sizeargument forbdrv_cbw_append().
Quoted 8 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.h | 1 - include/block/block-copy.h | 5 ++-- block/backup.c | 58 ++++++-------------------------------- block/block-copy.c | 47 +++++++++++++++++++++++++++++- block/copy-before-write.c | 10 +++---- 5 files changed, 62 insertions(+), 59 deletions(-)
PATCH 10/21 block/copy-before-write: relax permission requirements when no parents
In reply to Vladimir Sementsov-OgievskiyWe are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed.
Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent.
Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node.
--- block/copy-before-write.c | 8 +++++--- tests/qemu-iotests/283.out | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-)
block/copy-before-write.c +5 -3
@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,bdrv_default_perms (bs, c, role, reopen_queue, perm, shared, nperm, nshared); -if (perm &BLK_PERM_WRITE ) { - *nperm = *nperm |BLK_PERM_CONSISTENT_READ ; +if (!QLIST_EMPTY (&bs->parents)) { +if (perm &BLK_PERM_WRITE ) { + *nperm = *nperm |BLK_PERM_CONSISTENT_READ ; + } + *nshared &= ~(BLK_PERM_WRITE |BLK_PERM_RESIZE ); } - *nshared &= ~(BLK_PERM_WRITE |BLK_PERM_RESIZE ); } }
tests/qemu-iotests/283.out +1 -1
@@ -5,7 +5,7 @@ {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}} {"return": {}} {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}} -{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}} +{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}} === copy-before-write filter should be gone after job-finalize ===
Signature · 1 line
2.29.2
Re: [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents
Quoted 23 lines · Vladimir Sementsov-Ogievskiy
We are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed.
Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent.
Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 8 +++++--- tests/qemu-iotests/283.out | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-)block/copy-before-write.c +0
-0
block/copy-before-write.c static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, · line 145
Quoted 6 lines
block/copy-before-write.c +1
-2@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,bdrv_default_perms (bs, c, role, reopen_queue, perm, shared, nperm, nshared); -if (perm &BLK_PERM_WRITE ) { - *nperm = *nperm |BLK_PERM_CONSISTENT_READ ; +if (!QLIST_EMPTY (&bs->parents)) {
I understand this works because with the transactional system, at the time the permissions are checked, the graph has already been changed, yes?
I was wondering still whether there was any way to express this through permissions alone. I guess we could check
“perm & BLK_PERM_CONSISTENT_READ”, but that would actually be just a crutch to see whether there are any parents. I suppose this really is about whether there are parents or not. So:
Quoted 23 lines
block/copy-before-write.c +4
-1+if (perm &BLK_PERM_WRITE ) { + *nperm = *nperm |BLK_PERM_CONSISTENT_READ ; + } + *nshared &= ~(BLK_PERM_WRITE |BLK_PERM_RESIZE ); } - *nshared &= ~(BLK_PERM_WRITE |BLK_PERM_RESIZE ); } }tests/qemu-iotests/283.out +1
-1@@ -5,7 +5,7 @@ {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}} {"return": {}} {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}} -{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}} +{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}} === copy-before-write filter should be gone after job-finalize ===
Re: [PATCH 10/21] block/copy-before-write: relax permission requirements when no parents
Quoted 19 lines · Max Reitz
Quoted 18 lines · Vladimir Sementsov-Ogievskiy
We are going to publish copy-before-write filter. So, user should be able to create it with blockdev-add first, specifying both filtered and target children. And then do blockdev-reopen, to actually insert the filter where needed.
Currently, filter unshares write permission unconditionally on source node. It's good, but it will not allow to do blockdev-add. So, let's relax restrictions when filter doesn't have any parent.
Test output is modified, as now permission conflict happens only when job creates a blk parent for filter node.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/copy-before-write.c | 8 +++++---
tests/qemu-iotests/283.out | 2 +-2 files changed, 6 insertions(+), 4 deletions(-)
Quoted 12 lines
block/copy-before-write.c static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, · line 144
Quoted 10 lines
block/copy-before-write.c +1
-2@@ -142,10 +142,12 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, bdrv_default_perms(bs, c, role, reopen_queue, perm, shared, nperm, nshared); - if (perm & BLK_PERM_WRITE) { - *nperm = *nperm | BLK_PERM_CONSISTENT_READ; + if (!QLIST_EMPTY(&bs->parents)) {I understand this works because with the transactional system, at the time the permissions are checked, the graph has already been changed, yes?
Right
Quoted 3 lines
I was wondering still whether there was any way to express this through permissions alone. I guess we could check
“perm & BLK_PERM_CONSISTENT_READ”, but that would actually be just a crutch to see whether there are any parents. I suppose this really is about whether there are parents or not. So:
It's a workaround for blockdev-add to work. It's a workaround, because it would be better for the filter to never share WRITE. But this way we'll have to make an interface to transactionally create and insert filter. Probably we will at some point, than this workaround could be dropped. So, if we have to special-case a situation when there no parents, let's check exactly this, to be more strict.
Quoted 24 lines
Reviewed-by Max Reitz <[email protected]>Quoted 20 lines
block/copy-before-write.c +4
-1+ if (perm & BLK_PERM_WRITE) { + *nperm = *nperm | BLK_PERM_CONSISTENT_READ; + } + *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } - *nshared &= ~(BLK_PERM_WRITE | BLK_PERM_RESIZE); } }tests/qemu-iotests/283.out +1
-1@@ -5,7 +5,7 @@ {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": "base", "node-name": "other", "take-child-perms": ["write"]}} {"return": {}} {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": "full", "target": "target"}} -{"error": {"class": "GenericError", "desc": "Cannot append copy-before-write filter: Conflicts with use by source as 'image', which does not allow 'write' on base"}} +{"error": {"class": "GenericError", "desc": "Conflicts with use by source as 'image', which does not allow 'write' on base"}}=== copy-before-write filter should be gone after job-finalize ===
Signature · 2 lines
Best regards, Vladimir
PATCH 11/21 block/copy-before-write: use file child instead of backing
In reply to Vladimir Sementsov-OgievskiyWe are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead.
--- block/copy-before-write.c | 40 ++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-)
block/copy-before-write.c +23 -17
@@ -43,7 +43,7 @@ static coroutine_fn int cbw_co_preadv( BlockDriverState *bs, uint64_t offset, uint64_t bytes, QEMUIOVector *qiov, int flags) { - return bdrv_co_preadv(bs->backing, offset, bytes, qiov, flags); + return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); } static coroutine_fn int cbw_cbw(BlockDriverState *bs, uint64_t offset,
@@ -71,7 +71,7 @@ static int coroutine_fn cbw_co_pdiscard(BlockDriverState *bs,return ret; } -return bdrv_co_pdiscard (bs->backing, offset, bytes); +return bdrv_co_pdiscard (bs->file, offset, bytes); } static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,
@@ -82,7 +82,7 @@ static int coroutine_fn cbw_co_pwrite_zeroes(BlockDriverState *bs,return ret; } -return bdrv_co_pwrite_zeroes (bs->backing, offset, bytes, flags); +return bdrv_co_pwrite_zeroes (bs->file, offset, bytes, flags); } static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,
@@ -95,29 +95,22 @@ static coroutine_fn int cbw_co_pwritev(BlockDriverState *bs,return ret; } -return bdrv_co_pwritev (bs->backing, offset, bytes, qiov, flags); +return bdrv_co_pwritev (bs->file, offset, bytes, qiov, flags); } static int coroutine_fn cbw_co_flush(BlockDriverState *bs) { -if (!bs->backing) { +if (!bs->file) {return 0 ; } -return bdrv_co_flush (bs->backing->bs); +return bdrv_co_flush (bs->file->bs); }static void cbw_refresh_filename (BlockDriverState *bs) { -if (bs->backing ==NULL ) { -/* -* we can be here after failed bdrv_attach_child in -* bdrv_set_backing_hd -*/ -return ; - }pstrcpy (bs->exact_filename,sizeof (bs->exact_filename), - bs->backing->bs->filename); + bs->file->bs->filename); } static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c,
@@ -186,6 +179,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, top =bdrv_new_open_driver (&bdrv_cbw_filter, filter_node_name,BDRV_O_RDWR , errp);if (!top) { +error_prepend (errp,"Cannot open driver: " );return NULL ; }
@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->target =bdrv_attach_child (top, target,"target" , &child_of_bds,BDRV_CHILD_DATA , errp);if (!state->target) { +error_prepend (errp,"Cannot attach target child: " );bdrv_unref (target);bdrv_unref (top);return NULL ; } +bdrv_ref (source); + top->file =bdrv_attach_child (top, source,"file" , &child_of_bds, +BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + errp); +if (!top->file) { +error_prepend (errp,"Cannot attach file child: " ); +bdrv_unref (source); +bdrv_unref (top); +return NULL ; + } +bdrv_drained_begin (source); - ret =bdrv_append (top, source, errp); + ret =bdrv_replace_node (source, top, errp);if (ret <0 ) {error_prepend (errp,"Cannot append copy-before-write filter: " ); goto fail; } appended = true; - state->bcs =block_copy_state_new (top->backing, state->target, - false, compress, errp); + state->bcs =block_copy_state_new (top->file, state->target, false, compress, + errp);if (!state->bcs) {error_prepend (errp,"Cannot create block-copy-state: " ); goto fail;
Signature · 1 line
2.29.2
Re: [PATCH 11/21] block/copy-before-write: use file child instead of backing
Quoted 13 lines · Vladimir Sementsov-Ogievskiy
We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 40 ++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-)block/copy-before-write.c +0
-0
[...]
Quoted 7 lines
block/copy-before-write.c +1
-0@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->target =bdrv_attach_child (top, target,"target" , &child_of_bds,BDRV_CHILD_DATA , errp);if (!state->target) { +error_prepend (errp,"Cannot attach target child: " );bdrv_unref (target);bdrv_unref (top);
block/copy-before-write.c BlockDriverState *bdrv_cbw_append(BlockDriverState *source, · line 210
Quoted 10 lines
block/copy-before-write.c +7
-0return NULL ; } +bdrv_ref (source); + top->file =bdrv_attach_child (top, source,"file" , &child_of_bds, +BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + errp); +if (!top->file) { +error_prepend (errp,"Cannot attach file child: " ); +bdrv_unref (source);
Already existing in the code above, but why is the reference to @source dropped here? Shouldn’t bdrv_attach_child()’s error path have done that already?
Max
Quoted 3 lines
block/copy-before-write.c +3
-0+bdrv_unref (top); +return NULL ; + }
Re: [PATCH 11/21] block/copy-before-write: use file child instead of backing
Quoted 27 lines · Max Reitz
Quoted 13 lines · Vladimir Sementsov-Ogievskiy
We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/copy-before-write.c| 40 ++++++++++++++++++++++-----------------1 file changed, 23 insertions(+), 17 deletions(-)block/copy-before-write.c +0
-0[...]
Quoted 10 lines
block/copy-before-write.c +2
-0@@ -201,22 +195,34 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, state->target = bdrv_attach_child(top, target, "target", &child_of_bds, BDRV_CHILD_DATA, errp); if (!state->target) { + error_prepend(errp, "Cannot attach target child: "); bdrv_unref(target); bdrv_unref(top); return NULL; } + bdrv_ref(source);
Quoted 9 lines
block/copy-before-write.c BlockDriverState *bdrv_cbw_append(BlockDriverState *source, · line 210
Quoted 6 lines
block/copy-before-write.c +6
-0+ top->file = bdrv_attach_child(top, source, "file", &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + errp); + if (!top->file) { + error_prepend(errp, "Cannot attach file child: "); + bdrv_unref(source);Already existing in the code above, but why is the reference to @source dropped here? Shouldn’t bdrv_attach_child()’s error path have done that already?
Hmm, yes. Looks like a preexisting bug. will fix and update this patch.
Signature · 2 lines
Best regards, Vladimir
PATCH 12/21 block/copy-before-write: bdrv_cbw_append(): replace child at last
In reply to Vladimir Sementsov-OgievskiyRefactor the function to replace child at last. Thus we don't need to revert it and code is simplified.
block-copy state initilization being done before replacing the child doesn't need any drained section.
--- block/copy-before-write.c | 33 +++++++++++---------------------- 1 file changed, 11 insertions(+), 22 deletions(-)
block/copy-before-write.c +11 -22
@@ -172,7 +172,6 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,int ret;BDRVCopyBeforeWriteState *state;BlockDriverState *top; -bool appended = false;assert (source->total_sectors == target->total_sectors);
@@ -197,8 +196,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,if (!state->target) {error_prepend (errp,"Cannot attach target child: " );bdrv_unref (target); -bdrv_unref (top); -return NULL ; + goto fail; }bdrv_ref (source);
@@ -208,18 +206,8 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,if (!top->file) {error_prepend (errp,"Cannot attach file child: " );bdrv_unref (source); -bdrv_unref (top); -return NULL ; - } - -bdrv_drained_begin (source); - - ret =bdrv_replace_node (source, top, errp); -if (ret <0 ) { -error_prepend (errp,"Cannot append copy-before-write filter: " ); goto fail; } - appended = true; state->bcs =block_copy_state_new (top->file, state->target, false, compress, errp);
@@ -227,21 +215,22 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,error_prepend (errp,"Cannot create block-copy-state: " ); goto fail; } - *bcs = state->bcs; +bdrv_drained_begin (source); + ret =bdrv_replace_node (source, top, errp);bdrv_drained_end (source); +if (ret <0 ) { +error_prepend (errp,"Cannot append copy-before-write filter: " ); + goto fail; + } + + *bcs = state->bcs;return top; fail: -if (appended) { -bdrv_cbw_drop (top); - }else { -bdrv_unref (top); - } - -bdrv_drained_end (source); - +block_copy_state_free (state->bcs); +bdrv_unref (top);return NULL ; }
Signature · 1 line
2.29.2
Re: [PATCH 12/21] block/copy-before-write: bdrv_cbw_append(): replace child at last
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
Refactor the function to replace child at last. Thus we don't need to revert it and code is simplified.
block-copy state initilization being done before replacing the child
*initialization
Quoted 6 lines
doesn't need any drained section.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 33 +++++++++++---------------------- 1 file changed, 11 insertions(+), 22 deletions(-)
PATCH 13/21 block/copy-before-write: introduce cbw_init()
In reply to Vladimir Sementsov-OgievskiyMove part of bdrv_cbw_append() to new function cbw_open(). It's an intermediate step for adding noramal .bdrv_open() handler to the filter. With this commit no logic is changed, but we have a function which will be turned into .bdrv_open() handler in future commit.
--- block/copy-before-write.c | 73 +++++++++++++++++++++++---------------- 1 file changed, 43 insertions(+), 30 deletions(-)
block/copy-before-write.c +43 -30
@@ -144,6 +144,47 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } +static int cbw_init (BlockDriverState *top,BlockDriverState *source, +BlockDriverState *target,bool compress,Error **errp) +{ +BDRVCopyBeforeWriteState *state = top->opaque; + + top->total_sectors = source->total_sectors; + top->supported_write_flags =BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & source->supported_write_flags); + top->supported_zero_flags =BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & + source->supported_zero_flags); + +bdrv_ref (target); + state->target =bdrv_attach_child (top, target,"target" , &child_of_bds, +BDRV_CHILD_DATA , errp); +if (!state->target) { +error_prepend (errp,"Cannot attach target child: " ); +bdrv_unref (target); +return -EINVAL ; + } + +bdrv_ref (source); + top->file =bdrv_attach_child (top, source,"file" , &child_of_bds, +BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + errp); +if (!top->file) { +error_prepend (errp,"Cannot attach file child: " ); +bdrv_unref (source); +return -EINVAL ; + } + + state->bcs =block_copy_state_new (top->file, state->target, false, compress, + errp); +if (!state->bcs) { +error_prepend (errp,"Cannot create block-copy-state: " ); +return -EINVAL ; + } + +return 0 ; +} +BlockDriver bdrv_cbw_filter = { .format_name ="copy-before-write" , .instance_size =sizeof (BDRVCopyBeforeWriteState),
@@ -181,38 +222,10 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,error_prepend (errp,"Cannot open driver: " );return NULL ; } - state = top->opaque; - top->total_sectors = source->total_sectors; - top->supported_write_flags =BDRV_REQ_WRITE_UNCHANGED | - (BDRV_REQ_FUA & source->supported_write_flags); - top->supported_zero_flags =BDRV_REQ_WRITE_UNCHANGED | - ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & - source->supported_zero_flags); - -bdrv_ref (target); - state->target =bdrv_attach_child (top, target,"target" , &child_of_bds, -BDRV_CHILD_DATA , errp); -if (!state->target) { -error_prepend (errp,"Cannot attach target child: " ); -bdrv_unref (target); - goto fail; - } - -bdrv_ref (source); - top->file =bdrv_attach_child (top, source,"file" , &child_of_bds, -BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , - errp); -if (!top->file) { -error_prepend (errp,"Cannot attach file child: " ); -bdrv_unref (source); - goto fail; - } - state->bcs =block_copy_state_new (top->file, state->target, false, compress, - errp); -if (!state->bcs) { -error_prepend (errp,"Cannot create block-copy-state: " ); + ret =cbw_init (top, source, target, compress, errp); +if (ret <0 ) { goto fail; }
Signature · 1 line
2.29.2
Re: [PATCH 13/21] block/copy-before-write: introduce cbw_init()
Quoted 9 lines · Vladimir Sementsov-Ogievskiy
Move part of
bdrv_cbw_append()to new functioncbw_open(). It's an intermediate step for adding noramal .bdrv_open() handler to the filter. With this commit no logic is changed, but we have a function which will be turned into .bdrv_open() handler in future commit.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 73 +++++++++++++++++++++++---------------- 1 file changed, 43 insertions(+), 30 deletions(-)
PATCH 14/21 block/copy-before-write: cbw_init(): rename variables
In reply to Vladimir Sementsov-OgievskiyOne more step closer to real .bdrv_open() handler: use more usual names for bs being initialized and its state.
--- block/copy-before-write.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)
block/copy-before-write.c +14 -15
@@ -144,40 +144,39 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *top,BlockDriverState *source, +static int cbw_init (BlockDriverState *bs,BlockDriverState *source,BlockDriverState *target,bool compress,Error **errp) { -BDRVCopyBeforeWriteState *state = top->opaque; +BDRVCopyBeforeWriteState *s = bs->opaque; - top->total_sectors = source->total_sectors; - top->supported_write_flags =BDRV_REQ_WRITE_UNCHANGED | + bs->total_sectors = source->total_sectors; + bs->supported_write_flags =BDRV_REQ_WRITE_UNCHANGED | (BDRV_REQ_FUA & source->supported_write_flags); - top->supported_zero_flags =BDRV_REQ_WRITE_UNCHANGED | + bs->supported_zero_flags =BDRV_REQ_WRITE_UNCHANGED | ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & source->supported_zero_flags);bdrv_ref (target); - state->target =bdrv_attach_child (top, target,"target" , &child_of_bds, -BDRV_CHILD_DATA , errp); -if (!state->target) { + s->target =bdrv_attach_child (bs, target,"target" , &child_of_bds, +BDRV_CHILD_DATA , errp); +if (!s->target) {error_prepend (errp,"Cannot attach target child: " );bdrv_unref (target);return -EINVAL ; }bdrv_ref (source); - top->file =bdrv_attach_child (top, source,"file" , &child_of_bds, -BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , - errp); -if (!top->file) { + bs->file =bdrv_attach_child (bs, source,"file" , &child_of_bds, +BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + errp); +if (!bs->file) {error_prepend (errp,"Cannot attach file child: " );bdrv_unref (source);return -EINVAL ; } - state->bcs =block_copy_state_new (top->file, state->target, false, compress, - errp); -if (!state->bcs) { + s->bcs =block_copy_state_new (bs->file, s->target, false, compress, errp); +if (!s->bcs) {error_prepend (errp,"Cannot create block-copy-state: " );return -EINVAL ; }
Signature · 1 line
2.29.2
Re: [PATCH 14/21] block/copy-before-write: cbw_init(): rename variables
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
One more step closer to real .bdrv_open() handler: use more usual names for bs being initialized and its state.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)
PATCH 15/21 block/copy-before-write: cbw_init(): use file child after attaching
In reply to Vladimir Sementsov-OgievskiyIn the next commit we'll get rid of source argument of cbw_init(). Prepare to it now, to make next commit simpler: move the code block that uses source below attaching the child and use bs->file->bs instead of source variable.
--- block/copy-before-write.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
block/copy-before-write.c +7 -7
@@ -149,13 +149,6 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source, {BDRVCopyBeforeWriteState *s = bs->opaque; - bs->total_sectors = source->total_sectors; - bs->supported_write_flags =BDRV_REQ_WRITE_UNCHANGED | - (BDRV_REQ_FUA & source->supported_write_flags); - bs->supported_zero_flags =BDRV_REQ_WRITE_UNCHANGED | - ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & - source->supported_zero_flags); -bdrv_ref (target); s->target =bdrv_attach_child (bs, target,"target" , &child_of_bds,BDRV_CHILD_DATA , errp);
@@ -175,6 +168,13 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,return -EINVAL ; } + bs->total_sectors = bs->file->bs->total_sectors; + bs->supported_write_flags =BDRV_REQ_WRITE_UNCHANGED | + (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); + bs->supported_zero_flags =BDRV_REQ_WRITE_UNCHANGED | + ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & + bs->file->bs->supported_zero_flags); + s->bcs =block_copy_state_new (bs->file, s->target, false, compress, errp);if (!s->bcs) {error_prepend (errp,"Cannot create block-copy-state: " );
Signature · 1 line
2.29.2
Re: [PATCH 15/21] block/copy-before-write: cbw_init(): use file child after attaching
Quoted 9 lines · Vladimir Sementsov-Ogievskiy
In the next commit we'll get rid of source argument of
cbw_init(). Prepare to it now, to make next commit simpler: move the code block that uses source below attaching the child and usebs->file->bsinstead of source variable.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
PATCH 16/21 block/copy-before-write: cbw_init(): use options
In reply to Vladimir Sementsov-OgievskiyOne more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers.
--- block/copy-before-write.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-)
block/copy-before-write.c +20 -17
@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *bs,BlockDriverState *source, -BlockDriverState *target,bool compress,Error **errp) +static int cbw_init (BlockDriverState *bs,QDict *options,Error **errp) {BDRVCopyBeforeWriteState *s = bs->opaque; -bdrv_ref (target); - s->target =bdrv_attach_child (bs, target,"target" , &child_of_bds, -BDRV_CHILD_DATA , errp); -if (!s->target) { -error_prepend (errp,"Cannot attach target child: " ); -bdrv_unref (target); + bs->file =bdrv_open_child (NULL , options,"file" , bs, &child_of_bds, +BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + false, errp); +if (!bs->file) {return -EINVAL ; } -bdrv_ref (source); - bs->file =bdrv_attach_child (bs, source,"file" , &child_of_bds, -BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , - errp); -if (!bs->file) { -error_prepend (errp,"Cannot attach file child: " ); -bdrv_unref (source); + s->target =bdrv_open_child (NULL , options,"target" , bs, &child_of_bds, +BDRV_CHILD_DATA , false, errp); +if (!s->target) {return -EINVAL ; }
@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source, ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & bs->file->bs->supported_zero_flags); - s->bcs =block_copy_state_new (bs->file, s->target, false, compress, errp); +qdict_del (options,"cluster-size" ); + s->bcs =block_copy_state_new (bs->file, s->target, false, +qdict_get_try_bool (options,"x-deprecated-compress" , false), errp); +qdict_del (options,"x-deprecated-compress" );if (!s->bcs) {error_prepend (errp,"Cannot create block-copy-state: " );return -EINVAL ;
@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,int ret;BDRVCopyBeforeWriteState *state;BlockDriverState *top; +QDict *opts;assert (source->total_sectors == target->total_sectors);
@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } state = top->opaque; - ret =cbw_init (top, source, target, compress, errp); + opts =qdict_new (); +qdict_put_str (opts,"file" ,bdrv_get_node_name (source)); +qdict_put_str (opts,"target" ,bdrv_get_node_name (target)); +qdict_put_bool (opts,"x-deprecated-compress" , compress); + + ret =cbw_init (top, opts, errp); +qobject_unref (opts);if (ret <0 ) { goto fail; }
Signature · 1 line
2.29.2
Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
Quoted 50 lines · Vladimir Sementsov-Ogievskiy
One more step closer to .bdrv_open(): use options instead of plain arguments. Move to
bdrv_open_child()calls, native for drive open handlers.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-)block/copy-before-write.c +8
-15@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *bs,BlockDriverState *source, -BlockDriverState *target,bool compress,Error **errp) +static int cbw_init (BlockDriverState *bs,QDict *options,Error **errp) {BDRVCopyBeforeWriteState *s = bs->opaque; -bdrv_ref (target); - s->target =bdrv_attach_child (bs, target,"target" , &child_of_bds, -BDRV_CHILD_DATA , errp); -if (!s->target) { -error_prepend (errp,"Cannot attach target child: " ); -bdrv_unref (target); + bs->file =bdrv_open_child (NULL , options,"file" , bs, &child_of_bds, +BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + false, errp); +if (!bs->file) {return -EINVAL ; } -bdrv_ref (source); - bs->file =bdrv_attach_child (bs, source,"file" , &child_of_bds, -BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , - errp); -if (!bs->file) { -error_prepend (errp,"Cannot attach file child: " ); -bdrv_unref (source); + s->target =bdrv_open_child (NULL , options,"target" , bs, &child_of_bds, +BDRV_CHILD_DATA , false, errp); +if (!s->target) {return -EINVAL ; }
block/copy-before-write.c static int cbw_init(BlockDriverState *bs, BlockDriverState *source, · line 171
Quoted 5 lines
block/copy-before-write.c +1
-1@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source, ((BDRV_REQ_FUA |BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ) & bs->file->bs->supported_zero_flags); - s->bcs =block_copy_state_new (bs->file, s->target, false, compress, errp); +qdict_del (options,"cluster-size" );
What is this about?
block/copy-before-write.c static int cbw_init(BlockDriverState *bs, BlockDriverState *source, · line 173
Quoted 2 lines
block/copy-before-write.c +2
-0+ s->bcs =block_copy_state_new (bs->file, s->target, false, +qdict_get_try_bool (options,"x-deprecated-compress" , false), errp);
First, I’d keep the compress variable and use it to store the value, because this doesn’t look very nice.
Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.
Max
Quoted 28 lines
block/copy-before-write.c +9
-1+qdict_del (options,"x-deprecated-compress" );if (!s->bcs) {error_prepend (errp,"Cannot create block-copy-state: " );return -EINVAL ;@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source,int ret;BDRVCopyBeforeWriteState *state;BlockDriverState *top; +QDict *opts;assert (source->total_sectors == target->total_sectors);@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } state = top->opaque; - ret =cbw_init (top, source, target, compress, errp); + opts =qdict_new (); +qdict_put_str (opts,"file" ,bdrv_get_node_name (source)); +qdict_put_str (opts,"target" ,bdrv_get_node_name (target)); +qdict_put_bool (opts,"x-deprecated-compress" , compress); + + ret =cbw_init (top, opts, errp); +qobject_unref (opts);if (ret <0 ) { goto fail; }
Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
Quoted 45 lines · Max Reitz
Quoted 44 lines · Vladimir Sementsov-Ogievskiy
One more step closer to .bdrv_open(): use options instead of plain arguments. Move to
bdrv_open_child()calls, native for drive open handlers.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/copy-before-write.c| 37 ++++++++++++++++++++-----------------1 file changed, 20 insertions(+), 17 deletions(-)block/copy-before-write.c +8
-15@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *bs,BlockDriverState *source, - BlockDriverState *target, bool compress, Error **errp) +static int cbw_init (BlockDriverState *bs,QDict *options,Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; - bdrv_ref(target); - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds, - BDRV_CHILD_DATA, errp); - if (!s->target) { - error_prepend(errp, "Cannot attach target child: "); - bdrv_unref(target); + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + false, errp); + if (!bs->file) { return -EINVAL; } - bdrv_ref(source); - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds, - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, - errp); - if (!bs->file) { - error_prepend(errp, "Cannot attach file child: "); - bdrv_unref(source); + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds, + BDRV_CHILD_DATA, false, errp); + if (!s->target) { return -EINVAL;
Quoted 8 lines
Quoted 6 lines
block/copy-before-write.c +0
-0}@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,((
BDRV_REQ_FUA|BDRV_REQ_MAY_UNMAP|BDRV_REQ_NO_FALLBACK) &bs->file->bs->supported_zero_flags);
-s->bcs=block_copy_state_new(bs->file, s->target, false, compress, errp); +qdict_del(options, "cluster-size");What is this about?
accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to block_copy)
Quoted 5 lines
Quoted 3 lines
+
s->bcs= block_copy_state_new(bs->file,s->target, false,
+qdict_get_try_bool(options, "x-deprecated-compress", false), errp);First, I’d keep the
compressvariable and use it to store the value, because this doesn’t look very nice.
OK
Quoted 3 lines
Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.
I should have described it in commit message.
We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.
Quoted 27 lines
Quoted 26 lines
+
qdict_del(options, "x-deprecated-compress");
if (!s->bcs) {
error_prepend(errp, "Cannot create block-copy-state: "); return-EINVAL;block/copy-before-write.c +8
-1@@ -212,6 +208,7 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, int ret; BDRVCopyBeforeWriteState *state; BlockDriverState *top; + QDict *opts; assert(source->total_sectors == target->total_sectors);@@ -223,7 +220,13 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, } state = top->opaque; - ret = cbw_init(top, source, target, compress, errp); + opts = qdict_new(); + qdict_put_str(opts, "file", bdrv_get_node_name(source)); + qdict_put_str(opts, "target", bdrv_get_node_name(target)); + qdict_put_bool(opts, "x-deprecated-compress", compress); + + ret = cbw_init(top, opts, errp); + qobject_unref(opts); if (ret < 0) { goto fail; }
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
Quoted 75 lines · Vladimir Sementsov-Ogievskiy
Quoted 53 lines · Max Reitz
Quoted 52 lines · Vladimir Sementsov-Ogievskiy
One more step closer to .bdrv_open(): use options instead of plain arguments. Move to
bdrv_open_child()calls, native for drive open handlers.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/copy-before-write.c| 37 ++++++++++++++++++++-----------------1 file changed, 20 insertions(+), 17 deletions(-)block/copy-before-write.c +8
-15@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *bs,BlockDriverState *source, - BlockDriverState *target, bool compress, Error **errp) +static int cbw_init (BlockDriverState *bs,QDict *options,Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; - bdrv_ref(target); - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds, - BDRV_CHILD_DATA, errp); - if (!s->target) { - error_prepend(errp, "Cannot attach target child: "); - bdrv_unref(target); + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds, + BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , + false, errp); + if (!bs->file) { return -EINVAL; } - bdrv_ref(source); - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds, - BDRV_CHILD_FILTERED |BDRV_CHILD_PRIMARY , - errp); - if (!bs->file) { - error_prepend(errp, "Cannot attach file child: "); - bdrv_unref(source); + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds, + BDRV_CHILD_DATA, false, errp); + if (!s->target) {return
-EINVAL;
}block/copy-before-write.c +0
-0@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs,Quoted 8 lines
Quoted 6 lines
block/copy-before-write.c +0
-0BlockDriverState *source,((
BDRV_REQ_FUA|BDRV_REQ_MAY_UNMAP|BDRV_REQ_NO_FALLBACK) &bs->file->bs->supported_zero_flags);
-s->bcs=block_copy_state_new(bs->file, s->target, false, compress, errp);
+qdict_del(options, "cluster-size");What is this about?
accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to
block_copy)Quoted 6 lines
Quoted 4 lines
+
s->bcs= block_copy_state_new(bs->file,s->target, false, +qdict_get_try_bool(options, "x-deprecated-compress", false), errp);First, I’d keep the
compressvariable and use it to store the value, because this doesn’t look very nice.OK
Quoted 1 line
Quoted 6 lines
Quoted 2 lines
Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.
I should have described it in commit message.
We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.
Oh, that’s true. Until backup’s option is deprecated, I think a comment would be more useful than a note in the commit message, though.
Max
Re: [PATCH 16/21] block/copy-before-write: cbw_init(): use options
Quoted 67 lines · Max Reitz
Quoted 66 lines · Vladimir Sementsov-Ogievskiy
Quoted 45 lines · Max Reitz
Quoted 44 lines · Vladimir Sementsov-Ogievskiy
One more step closer to .bdrv_open(): use options instead of plain arguments. Move to
bdrv_open_child()calls, native for drive open handlers.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/copy-before-write.c| 37 ++++++++++++++++++++-----------------1 file changed, 20 insertions(+), 17 deletions(-)block/copy-before-write.c +8
-15@@ -144,27 +144,20 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *bs,BlockDriverState *source, - BlockDriverState *target, bool compress, Error **errp) +static int cbw_init (BlockDriverState *bs,QDict *options,Error **errp) { BDRVCopyBeforeWriteState *s = bs->opaque; - bdrv_ref(target); - s->target = bdrv_attach_child(bs, target, "target", &child_of_bds, - BDRV_CHILD_DATA, errp); - if (!s->target) { - error_prepend(errp, "Cannot attach target child: "); - bdrv_unref(target); + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds, + BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, + false, errp); + if (!bs->file) { return -EINVAL; } - bdrv_ref(source); - bs->file = bdrv_attach_child(bs, source, "file", &child_of_bds, - BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY, - errp); - if (!bs->file) { - error_prepend(errp, "Cannot attach file child: "); - bdrv_unref(source); + s->target = bdrv_open_child(NULL, options, "target", bs, &child_of_bds, + BDRV_CHILD_DATA, false, errp); + if (!s->target) { return -EINVAL;Quoted 8 lines
Quoted 6 lines
block/copy-before-write.c +0
-0}@@ -175,7 +168,10 @@ static int cbw_init(BlockDriverState *bs, BlockDriverState *source,((
BDRV_REQ_FUA|BDRV_REQ_MAY_UNMAP|BDRV_REQ_NO_FALLBACK) &bs->file->bs->supported_zero_flags);
-s->bcs=block_copy_state_new(bs->file, s->target, false, compress, errp); +qdict_del(options, "cluster-size");What is this about?
accidental, will drop. (it's a remaining of my first solution where I tried to pass cluster-size, then I decided that's better move cluster-size detection to
block_copy)Quoted 5 lines
Quoted 3 lines
+
s->bcs= block_copy_state_new(bs->file,s->target, false,
+qdict_get_try_bool(options, "x-deprecated-compress", false), errp);First, I’d keep the
compressvariable and use it to store the value, because this doesn’t look very nice.OK
Quoted 1 line
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
Second, what’s the story here? “deprecated” sounds to me like you’re planning to use a different interface eventually, but looking ahead for a bit I didn’t find anything yet.
I should have described it in commit message.
We have "compress" filter driver. So instead adding "compress" option to every block job or filter, user should use "compress" filter. That's why I don't want to publish compress option for copy-before-write filter. Still we need it to maintain "compress" option of backup job. I also want to deprecate "compress" option in backup, then everything will be clear.
Oh, that’s true. Until backup’s option is deprecated, I think a comment would be more useful than a note in the commit message, though.
Agree, will add
Signature · 2 lines
Best regards, Vladimir
PATCH 17/21 block/block-copy: switch to fully set bitmap by default
In reply to Vladimir Sementsov-Ogievskiyblock-copy has a bit inconvenient interface around dirty bitmap: user should get pointer to it and than set by hand. We do need a possibility to share the bitmap with backup job.
But default of empty bitmap is strange. Switch to full-set bitmap by default. This way we will not care about setting dirty bitmap in copy-before-write filter when publish it so that it can be used in separate of backup job.
--- block/backup.c | 16 +++++++--------- block/block-copy.c | 1 + 2 files changed, 8 insertions(+), 9 deletions(-)
block/backup.c +7 -9
@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)BdrvDirtyBitmap *bcs_bitmap =block_copy_dirty_bitmap (job->bcs);if (job->sync_mode ==MIRROR_SYNC_MODE_BITMAP ) { +bdrv_clear_dirty_bitmap (bcs_bitmap,NULL ); ret =bdrv_dirty_bitmap_merge_internal (bcs_bitmap, job->sync_bitmap,NULL , true);assert (ret); - }else { -if (job->sync_mode ==MIRROR_SYNC_MODE_TOP ) { -/* -* We can't hog the coroutine to initialize this thoroughly. -* Set a flag and resume work when we are able to yield safely. -*/ -block_copy_set_skip_unallocated (job->bcs, true); - } -bdrv_set_dirty_bitmap (bcs_bitmap,0 , job->len); + }else if (job->sync_mode ==MIRROR_SYNC_MODE_TOP ) { +/* +* We can't hog the coroutine to initialize this thoroughly. +* Set a flag and resume work when we are able to yield safely. +*/ +block_copy_set_skip_unallocated (job->bcs, true); } estimate =bdrv_get_dirty_count (bcs_bitmap);
block/block-copy.c +1 -0
@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,return NULL ; }bdrv_disable_dirty_bitmap (copy_bitmap); +bdrv_set_dirty_bitmap (copy_bitmap,0 ,bdrv_dirty_bitmap_size (copy_bitmap));/* * Why we always set BDRV_REQ_SERIALISING write flag:
Signature · 1 line
2.29.2
Re: [PATCH 17/21] block/block-copy: switch to fully set bitmap by default
Quoted 5 lines · Vladimir Sementsov-Ogievskiy
block-copy has a bit inconvenient interface around dirty bitmap: user should get pointer to it and than set by hand. We do need a possibility to share the bitmap with backup job.
But default of empty bitmap is strange.
I don’t know, I don’t find it strange. It expects its user to specify what data to copy, so clearly it gives said user a blank slate.
Quoted 4 lines
Switch to full-set bitmap by
default. This way we will not care about setting dirty bitmap in copy-before-write filter when publish it so that it can be used in separate of backup job.
That’s a valid reason, though, so:
Still, I find it stranger this way, because I’m more used to “initialization to 0 by default”.
Quoted 49 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/backup.c | 16 +++++++--------- block/block-copy.c | 1 + 2 files changed, 8 insertions(+), 9 deletions(-)block/backup.c +7
-9@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)BdrvDirtyBitmap *bcs_bitmap =block_copy_dirty_bitmap (job->bcs);if (job->sync_mode ==MIRROR_SYNC_MODE_BITMAP ) { +bdrv_clear_dirty_bitmap (bcs_bitmap,NULL ); ret =bdrv_dirty_bitmap_merge_internal (bcs_bitmap, job->sync_bitmap,NULL , true);assert (ret); - }else { -if (job->sync_mode ==MIRROR_SYNC_MODE_TOP ) { -/* -* We can't hog the coroutine to initialize this thoroughly. -* Set a flag and resume work when we are able to yield safely. -*/ -block_copy_set_skip_unallocated (job->bcs, true); - } -bdrv_set_dirty_bitmap (bcs_bitmap,0 , job->len); + }else if (job->sync_mode ==MIRROR_SYNC_MODE_TOP ) { +/* +* We can't hog the coroutine to initialize this thoroughly. +* Set a flag and resume work when we are able to yield safely. +*/ +block_copy_set_skip_unallocated (job->bcs, true); } estimate =bdrv_get_dirty_count (bcs_bitmap);block/block-copy.c +1
-0@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,return NULL ; }bdrv_disable_dirty_bitmap (copy_bitmap); +bdrv_set_dirty_bitmap (copy_bitmap,0 ,bdrv_dirty_bitmap_size (copy_bitmap));/* * Why we always set BDRV_REQ_SERIALISING write flag:
Re: [PATCH 17/21] block/block-copy: switch to fully set bitmap by default
Quoted 14 lines · Max Reitz
Quoted 5 lines · Vladimir Sementsov-Ogievskiy
block-copy has a bit inconvenient interface around dirty bitmap: user should get pointer to it and than set by hand. We do need a possibility to share the bitmap with backup job.
But default of empty bitmap is strange.
I don’t know, I don’t find it strange. It expects its user to specify what data to copy, so clearly it gives said user a blank slate.
Quoted 4 lines
Switch to full-set bitmap by
default. This way we will not care about setting dirty bitmap in copy-before-write filter when publish it so that it can be used in separate of backup job.
Quoted 5 lines
That’s a valid reason, though, so:
Reviewed-by Max Reitz <[email protected]>Still, I find it stranger this way, because I’m more used to “initialization to 0 by default”.
When I started to test this series it didn't work, because bitmap was not initialized. I fixed it and call original behavior strange :)
Probably better to keep block-copy as is and set bitmap in cbw_open. Backup code should be modified anyway, as it will get the bitmap after initialization in cbw_open.
Next step would be support for bitmap in copy-before-write filter anyway, and this place will be changed again.. So, I don't know, but it's not important thing.
Quoted 48 lines
Quoted 47 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/backup.c | 16 +++++++---------
block/block-copy.c| 1 +2 files changed, 8 insertions(+), 9 deletions(-)block/backup.c +7
-9@@ -233,18 +233,16 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job) BdrvDirtyBitmap *bcs_bitmap = block_copy_dirty_bitmap(job->bcs); if (job->sync_mode == MIRROR_SYNC_MODE_BITMAP) { + bdrv_clear_dirty_bitmap(bcs_bitmap, NULL); ret = bdrv_dirty_bitmap_merge_internal(bcs_bitmap, job->sync_bitmap, NULL, true); assert(ret); - } else { - if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { - /* - * We can't hog the coroutine to initialize this thoroughly. - * Set a flag and resume work when we are able to yield safely. - */ - block_copy_set_skip_unallocated(job->bcs, true); - } - bdrv_set_dirty_bitmap(bcs_bitmap, 0, job->len); + } else if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { + /* + * We can't hog the coroutine to initialize this thoroughly. + * Set a flag and resume work when we are able to yield safely. + */ + block_copy_set_skip_unallocated(job->bcs, true); } estimate = bdrv_get_dirty_count(bcs_bitmap);block/block-copy.c +1
-0@@ -296,6 +296,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, return NULL; } bdrv_disable_dirty_bitmap(copy_bitmap); + bdrv_set_dirty_bitmap(copy_bitmap, 0, bdrv_dirty_bitmap_size(copy_bitmap)); /** Why we always set BDRV_REQ_SERIALISING write flag:
Signature · 2 lines
Best regards, Vladimir
PATCH 18/21 block/block-copy: make setting progress optional
In reply to Vladimir Sementsov-OgievskiyNow block-copy will crash if user don't set progress meter by block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it.
--- block/block-copy.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
block/block-copy.c +11 -7
@@ -506,7 +506,7 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)if (ret <0 && !t->call_state->ret) { t->call_state->ret = ret; t->call_state->error_is_read = error_is_read; - }else { + }else if (t->s->progress) {progress_work_done (t->s->progress, t->bytes); }co_put_to_shres (t->s->mem, t->bytes);
@@ -610,9 +610,11 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,if (!ret) {bdrv_reset_dirty_bitmap (s->copy_bitmap, offset, bytes); -progress_set_remaining (s->progress, -bdrv_get_dirty_count (s->copy_bitmap) + - s->in_flight_bytes); +if (s->progress) { +progress_set_remaining (s->progress, +bdrv_get_dirty_count (s->copy_bitmap) + + s->in_flight_bytes); + } } *count = bytes;
@@ -672,9 +674,11 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state) }if (s->skip_unallocated && !(ret &BDRV_BLOCK_ALLOCATED )) {block_copy_task_end (task,0 ); -progress_set_remaining (s->progress, -bdrv_get_dirty_count (s->copy_bitmap) + - s->in_flight_bytes); +if (s->progress) { +progress_set_remaining (s->progress, +bdrv_get_dirty_count (s->copy_bitmap) + + s->in_flight_bytes); + }trace_block_copy_skip_range (s, task->offset, task->bytes); offset =task_end (task); bytes = end - offset;
Signature · 1 line
2.29.2
Re: [PATCH 18/21] block/block-copy: make setting progress optional
Quoted 9 lines · Vladimir Sementsov-Ogievskiy
Now block-copy will crash if user don't set progress meter by
block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/block-copy.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-)
Makes me wonder whether an “if (not NULL)” check in progress_meter.h wouldn’t be a nice addition, but:
Re: [PATCH 18/21] block/block-copy: make setting progress optional
Quoted 6 lines · Max Reitz
Quoted 5 lines · Vladimir Sementsov-Ogievskiy
Now block-copy will crash if user don't set progress meter by
block_copy_set_progress_meter(). copy-before-write filter will be used in separate of backup job, and it doesn't want any progress meter (for now). So, allow not setting it.
Quoted 6 lines
Quoted 4 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/block-copy.c| 18 +++++++++++-------1 file changed, 11 insertions(+), 7 deletions(-)Makes me wonder whether an “if (not NULL)” check in
progress_meter.hwouldn’t be a nice addition, but:
IMHO it isn't. It's unusual to check if (obj is not NULL) in all obj API functions.. The only place where it is usual is obj_free() like functions.
Quoted 3 lines
Reviewed-by Max Reitz <[email protected]>
Thanks!
Signature · 2 lines
Best regards, Vladimir
PATCH 19/21 block/copy-before-write: make public block driver
In reply to Vladimir Sementsov-OgievskiyFinally, copy-before-write gets own .bdrv_open and .bdrv_close handlers, block_init() call and becomes available through bdrv_open().
To achieve this:
- cbw_init gets unused flags argument and becomes cbw_open
- block_copy_state_free() call moved to new cbw_close()
- in bdrv_cbw_append:
- options are completed with driver and node-name, and we can simply use bdrv_insert_node() to do both open and drained replacing
- in bdrv_cbw_drop:
- cbw_close() is now responsible for freeing s->bcs, so don't do it here
--- block/copy-before-write.c | 60 ++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 32 deletions(-)
block/copy-before-write.c +28 -32
@@ -144,7 +144,8 @@ static void cbw_child_perm(BlockDriverState *bs, BdrvChild *c, } } -static int cbw_init (BlockDriverState *bs,QDict *options,Error **errp) +static int cbw_open (BlockDriverState *bs,QDict *options,int flags, +Error **errp) {BDRVCopyBeforeWriteState *s = bs->opaque;
@@ -180,10 +181,21 @@ static int cbw_init(BlockDriverState *bs, QDict *options, Error **errp)return 0 ; } +static void cbw_close (BlockDriverState *bs) +{ +BDRVCopyBeforeWriteState *s = bs->opaque; + +block_copy_state_free (s->bcs); + s->bcs =NULL ; +} +BlockDriver bdrv_cbw_filter = { .format_name ="copy-before-write" , .instance_size =sizeof (BDRVCopyBeforeWriteState), + .bdrv_open = cbw_open, + .bdrv_close = cbw_close, + .bdrv_co_preadv = cbw_co_preadv, .bdrv_co_pwritev = cbw_co_pwritev, .bdrv_co_pwrite_zeroes = cbw_co_pwrite_zeroes,
@@ -205,57 +217,41 @@ BlockDriverState *bdrv_cbw_append(BlockDriverState *source, Error **errp) {ERRP_GUARD (); -int ret;BDRVCopyBeforeWriteState *state;BlockDriverState *top;QDict *opts;assert (source->total_sectors == target->total_sectors); - top =bdrv_new_open_driver (&bdrv_cbw_filter, filter_node_name, -BDRV_O_RDWR , errp); -if (!top) { -error_prepend (errp,"Cannot open driver: " ); -return NULL ; - } - state = top->opaque; - opts =qdict_new (); +qdict_put_str (opts,"driver" ,"copy-before-write" ); +if (filter_node_name) { +qdict_put_str (opts,"node-name" , filter_node_name); + }qdict_put_str (opts,"file" ,bdrv_get_node_name (source));qdict_put_str (opts,"target" ,bdrv_get_node_name (target));qdict_put_bool (opts,"x-deprecated-compress" , compress); - ret =cbw_init (top, opts, errp); -qobject_unref (opts); -if (ret <0 ) { - goto fail; - } - -bdrv_drained_begin (source); - ret =bdrv_replace_node (source, top, errp); -bdrv_drained_end (source); -if (ret <0 ) { -error_prepend (errp,"Cannot append copy-before-write filter: " ); - goto fail; + top =bdrv_insert_node (source, opts,BDRV_O_RDWR , errp); +if (!top) { +return NULL ; } + state = top->opaque; *bcs = state->bcs;return top; - -fail: -block_copy_state_free (state->bcs); -bdrv_unref (top); -return NULL ; }void bdrv_cbw_drop (BlockDriverState *bs) { -BDRVCopyBeforeWriteState *s = bs->opaque; -bdrv_drop_filter (bs, &error_abort); - -block_copy_state_free (s->bcs); -bdrv_unref (bs); } + +static void cbw_init (void ) +{ +bdrv_register (&bdrv_cbw_filter); +} + +block_init (cbw_init);
Signature · 1 line
2.29.2
Re: [PATCH 19/21] block/copy-before-write: make public block driver
Quoted 14 lines · Vladimir Sementsov-Ogievskiy
Finally, copy-before-write gets own .bdrv_open and .bdrv_close handlers,
block_init()call and becomes available throughbdrv_open().To achieve this:
-
cbw_initgets unused flags argument and becomescbw_open
-block_copy_state_free()call moved to newcbw_close()
- inbdrv_cbw_append:
- options are completed with driver and node-name, and we can simply usebdrv_insert_node()to do both open and drained replacing
- inbdrv_cbw_drop:
-cbw_close()is now responsible for freeings->bcs, so don't do it here
Quoted 4 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/copy-before-write.c | 60 ++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 32 deletions(-)
PATCH 20/21 qapi: publish copy-before-write filter
In reply to Vladimir Sementsov-Ogievskiy--- qapi/block-core.json | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
qapi/block-core.json +18 -4
@@ -2808,15 +2808,17 @@ # @blklogwrites: Since 3.0 # @blkreplay: Since 4.2 # @compress: Since 5.0 +# @copy-before-write: Since 6.1 # # Since: 2.9 ## { 'enum': 'BlockdevDriver', 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs', - 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', - 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', - 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', - 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', + 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg', + 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device', + 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', + 'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2', + 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' }, 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
@@ -3937,6 +3939,17 @@ 'base': 'BlockdevOptionsGenericFormat', 'data': { '*bottom': 'str' } } +## +# @BlockdevOptionsCbw: +# +# Driver specific block device options for the copy-before-write driver. +# +# Since: 6.1 +## +{ 'struct': 'BlockdevOptionsCbw', + 'base': 'BlockdevOptionsGenericFormat', + 'data': { 'target': 'BlockdevRef' } } + ## # @BlockdevOptions: #
@@ -3989,6 +4002,7 @@ 'bochs': 'BlockdevOptionsGenericFormat', 'cloop': 'BlockdevOptionsGenericFormat', 'compress': 'BlockdevOptionsGenericFormat', + 'copy-before-write':'BlockdevOptionsCbw', 'copy-on-read':'BlockdevOptionsCor', 'dmg': 'BlockdevOptionsGenericFormat', 'file': 'BlockdevOptionsFile',
Signature · 1 line
2.29.2
Re: [PATCH 20/21] qapi: publish copy-before-write filter
Quoted 32 lines · Vladimir Sementsov-Ogievskiy
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- qapi/block-core.json | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)qapi/block-core.json +6
-4@@ -2808,15 +2808,17 @@ # @blklogwrites: Since 3.0 # @blkreplay: Since 4.2 # @compress: Since 5.0 +# @copy-before-write: Since 6.1 # # Since: 2.9 ## { 'enum': 'BlockdevDriver', 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs', - 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', - 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', - 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', - 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', + 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg', + 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device', + 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', + 'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2', + 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' }, 'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
qapi/block-core.json line 3946
Quoted 8 lines
qapi/block-core.json +5
-0@@ -3937,6 +3939,17 @@ 'base': 'BlockdevOptionsGenericFormat', 'data': { '*bottom': 'str' } } +## +# @BlockdevOptionsCbw: +# +# Driver specific block device options for the copy-before-write driver. +#
I think there should be a description of @target here.
(Looks good otherwise, though.)
Max
Quoted 18 lines
qapi/block-core.json +7
-0+# Since: 6.1 +## +{ 'struct': 'BlockdevOptionsCbw', + 'base': 'BlockdevOptionsGenericFormat', + 'data': { 'target': 'BlockdevRef' } } + ## # @BlockdevOptions: #@@ -3989,6 +4002,7 @@ 'bochs': 'BlockdevOptionsGenericFormat', 'cloop': 'BlockdevOptionsGenericFormat', 'compress': 'BlockdevOptionsGenericFormat', + 'copy-before-write':'BlockdevOptionsCbw', 'copy-on-read':'BlockdevOptionsCor', 'dmg': 'BlockdevOptionsGenericFormat', 'file': 'BlockdevOptionsFile',
Re: [PATCH 20/21] qapi: publish copy-before-write filter
Quoted 30 lines · Max Reitz
Quoted 29 lines · Vladimir Sementsov-Ogievskiy
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
qapi/block-core.json| 22 ++++++++++++++++++----1 file changed, 18 insertions(+), 4 deletions(-)qapi/block-core.json +6
-4@@ -2808,15 +2808,17 @@ # @blklogwrites: Since 3.0 # @blkreplay: Since 4.2 # @compress: Since 5.0 +# @copy-before-write: Since 6.1 # # Since: 2.9 ## { 'enum': 'BlockdevDriver', 'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs', - 'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', - 'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi', - 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', - 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', + 'cloop', 'compress', 'copy-before-write', 'copy-on-read', 'dmg', + 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', 'host_device', + 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', 'null-aio', + 'null-co', 'nvme', 'parallels', 'preallocate', 'qcow', 'qcow2', + 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
Quoted 8 lines
qapi/block-core.json line 3944
Quoted 8 lines
qapi/block-core.json +4
-0'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }@@ -3937,6 +3939,17 @@ 'base': 'BlockdevOptionsGenericFormat', 'data': { '*bottom': 'str' } } +## +# @BlockdevOptionsCbw: +# +# Driver specific block device options for the copy-before-write driver,
which does so called copy-before-write operations: when data is written to the filter, it firstly reads corresponding blocks from its file child and copies them to @target child. After successful copying the write request is propagated to file child. If copying filed, the original write request is failed too and no data is written to file child.
Quoted 3 lines
qapi/block-core.json line 3945
Quoted 1 line
qapi/block-core.json +1
-0+#I think there should be a description of @target here.
Right, I forget. Something like:
@target: The target for copy-before-write operations.
Quoted 24 lines
(Looks good otherwise, though.)
Max
Quoted 18 lines
qapi/block-core.json +7
-0+# Since: 6.1 +## +{ 'struct': 'BlockdevOptionsCbw', + 'base': 'BlockdevOptionsGenericFormat', + 'data': { 'target': 'BlockdevRef' } } + ## # @BlockdevOptions: #@@ -3989,6 +4002,7 @@ 'bochs': 'BlockdevOptionsGenericFormat', 'cloop': 'BlockdevOptionsGenericFormat', 'compress': 'BlockdevOptionsGenericFormat', + 'copy-before-write':'BlockdevOptionsCbw', 'copy-on-read':'BlockdevOptionsCor', 'dmg': 'BlockdevOptionsGenericFormat', 'file': 'BlockdevOptionsFile',
Signature · 2 lines
Best regards, Vladimir
PATCH 21/21 itotests/222: add test-case for copy-before-write filter
In reply to Vladimir Sementsov-OgievskiyNew fleecing method becomes available: copy-before-write filter.
Actually we don't need backup job to setup image fleecing. Add test fore new recommended way of image fleecing.
--- tests/qemu-iotests/222 | 56 ++++++++++++++++++++++------- tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+), 13 deletions(-)
tests/qemu-iotests/222 +43 -13
@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] ("0xdc", "32M", "32k"), # Left-end of partial-right [2] ("0xcd", "0x3ff0000", "64k")] # patterns[3] -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \ - iotests.VM() as vm: +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm): + if use_cbw: + log('=== Test filter based fleecing ===') + else: + log('=== Test backup(sync=none) based fleecing ===') + log('') log('--- Setting up images ---') log('')
@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Launching VM ---') log('') - vm.add_drive(base_img_path) + vm.add_drive(base_img_path, "node-name=node0") vm.launch() log('Done')
@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \ "backing": src_node, })) - # Establish COW from source to fleecing node - log(vm.qmp("blockdev-backup", - device=src_node, - target=tgt_node, - sync="none")) + # Establish CBW from source to fleecing node + if use_cbw: + log(vm.qmp("blockdev-add", **{ + "driver": "copy-before-write", + "node-name": "fl-cbw", + "file": src_node, + "target": tgt_node + })) + + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]", + property="drive", value="fl-cbw")) + else: + log(vm.qmp("blockdev-backup", + device=src_node, + target=tgt_node, + sync="none")) log('') log('--- Setting up NBD Export ---')
@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Cleanup ---') log('') - log(vm.qmp('block-job-cancel', device=src_node)) - log(vm.event_wait('BLOCK_JOB_CANCELLED'), - filters=[iotests.filter_qmp_event]) + if use_cbw: + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]", + property="drive", value="node0")) + log(vm.qmp("blockdev-del", node_name="fl-cbw")) + else: + log(vm.qmp('block-job-cancel', device=src_node)) + log(vm.event_wait('BLOCK_JOB_CANCELLED'), + filters=[iotests.filter_qmp_event]) + log(vm.qmp('nbd-server-stop')) log(vm.qmp('blockdev-del', node_name=tgt_node)) vm.shutdown()
@@ -157,3 +176,14 @@ with iotests.FilePath('base.img') as base_img_path, \ log('') log('Done') + +def test(use_cbw): + with iotests.FilePath('base.img') as base_img_path, \ + iotests.FilePath('fleece.img') as fleece_img_path, \ + iotests.FilePath('nbd.sock', + base_dir=iotests.sock_dir) as nbd_sock_path, \ + iotests.VM() as vm: + do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm) + +test(False) +test(True)
tests/qemu-iotests/222.out +72 -0
78 lines not on this page · raw message
Signature · 1 line
2.29.2
Re: [PATCH 21/21] itotests/222: add test-case for copy-before-write filter
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
New fleecing method becomes available: copy-before-write filter.
Actually we don't need backup job to setup image fleecing. Add test fore new recommended way of image fleecing.
*for
Quoted 6 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- tests/qemu-iotests/222 | 56 ++++++++++++++++++++++------- tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+), 13 deletions(-)
Looks good, just wondering about some variable usage (why is src_node a BB name?) and whether the virtio-blk qdev path couldn’t be expressed in some nicer way.
Quoted 23 lines
tests/qemu-iotests/222 +6
-4@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] ("0xdc", "32M", "32k"), # Left-end of partial-right [2] ("0xcd", "0x3ff0000", "64k")] # patterns[3] -with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \ - iotests.VM() as vm: +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm): + if use_cbw: + log('=== Test filter based fleecing ===') + else: + log('=== Test backup(sync=none) based fleecing ===') + log('') log('--- Setting up images ---') log('')
tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 77
Quoted 8 lines
tests/qemu-iotests/222 +1
-1@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Launching VM ---') log('') - vm.add_drive(base_img_path) + vm.add_drive(base_img_path, "node-name=node0") vm.launch() log('Done')
Afterwards, src_node is set to 'drive0'. So src_node is actually a BB name...
Quoted 6 lines
tests/qemu-iotests/222 +0
-2@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \ "backing": src_node, })) - # Establish COW from source to fleecing node - log(vm.qmp("blockdev-backup",
tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 102
Quoted 10 lines
tests/qemu-iotests/222 +7
-3- device=src_node, - target=tgt_node, - sync="none")) + # Establish CBW from source to fleecing node + if use_cbw: + log(vm.qmp("blockdev-add", **{ + "driver": "copy-before-write", + "node-name": "fl-cbw", + "file": src_node, + "target": tgt_node
Which makes this look strange, and also TIL (or maybe “today I was reminded”?) that you can use BB names as node references.
(It’s also already weird in the "backing": src_node line right at the beginning of this hunk...)
So I guess it works, but I’d find it would be nicer to distinguish between the node name and the BB name.
tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 106
Quoted 4 lines
tests/qemu-iotests/222 +4
-0+ })) + + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]", + property="drive", value="fl-cbw"))
Is that path portable across targets? Should we maybe instead manually add a virtio-scsi device that we can give a proper name?
Or I suppose we could do
path = next(dev for dev in vm.qmp('query-block')['return']
if dev['device'] == 'drive0')['qdev']
Quoted 9 lines
tests/qemu-iotests/222 +5
-0+ else: + log(vm.qmp("blockdev-backup", + device=src_node, + target=tgt_node, + sync="none")) log('') log('--- Setting up NBD Export ---')
tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 157
Quoted 9 lines
tests/qemu-iotests/222 +3
-3@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Cleanup ---') log('') - log(vm.qmp('block-job-cancel', device=src_node)) - log(vm.event_wait('BLOCK_JOB_CANCELLED'), - filters=[iotests.filter_qmp_event]) + if use_cbw: + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]", + property="drive", value="node0"))
If src_node were 'node0', we wouldn’t need to use the 'node0' literal here and could use src_node instead. (Because it kind of seems like this test wants to use those variables instead of literals. I mean, we could also just call the node 'src-node', but, well.)
Max
Quoted 9 lines
tests/qemu-iotests/222 +6
-0+ log(vm.qmp("blockdev-del", node_name="fl-cbw")) + else: + log(vm.qmp('block-job-cancel', device=src_node)) + log(vm.event_wait('BLOCK_JOB_CANCELLED'), + filters=[iotests.filter_qmp_event]) + log(vm.qmp('nbd-server-stop')) log(vm.qmp('blockdev-del', node_name=tgt_node)) vm.shutdown()
Re: [PATCH 21/21] itotests/222: add test-case for copy-before-write filter
Quoted 65 lines · Max Reitz
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
New fleecing method becomes available: copy-before-write filter.
Actually we don't need backup job to setup image fleecing. Add test fore new recommended way of image fleecing.
*for
Quoted 6 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
tests/qemu-iotests/222 | 56 ++++++++++++++++++++++------- tests/qemu-iotests/222.out | 72 ++++++++++++++++++++++++++++++++++++++2 files changed, 115 insertions(+), 13 deletions(-)Looks good, just wondering about some variable usage (why is
src_nodea BB name?) and whether the virtio-blk qdev path couldn’t be expressed in some nicer way.Quoted 20 lines
tests/qemu-iotests/222 +0
-0@@ -48,11 +48,13 @@ remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1]("
0xdc", "32M", "32k"), # Left-end of partial-right [2] ("0xcd", "0x3ff0000", "64k")] # patterns[3]tests/qemu-iotests/222 +6
-4-with iotests.FilePath('base.img') as base_img_path, \ - iotests.FilePath('fleece.img') as fleece_img_path, \ - iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \ - iotests.VM() as vm: +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm): + if use_cbw: + log('=== Test filter based fleecing ===') + else: + log('=== Test backup(sync=none) based fleecing ===') + log('')log('--- Setting up images ---')
log('')tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 76
Quoted 7 lines
tests/qemu-iotests/222 +1
-1@@ -69,7 +71,7 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Launching VM ---') log('') - vm.add_drive(base_img_path) + vm.add_drive(base_img_path, "node-name=node0") vm.launch() log('Done')Afterwards,
src_nodeis set to 'drive0'. Sosrc_nodeis actually a BB name...Quoted 5 lines
tests/qemu-iotests/222 +0
-2@@ -91,11 +93,22 @@ with iotests.FilePath('base.img') as base_img_path, \ "backing": src_node, })) - # Establish COW from source to fleecing node - log(vm.qmp("blockdev-backup",tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 101
Quoted 10 lines
tests/qemu-iotests/222 +7
-3- device=src_node, - target=tgt_node, - sync="none")) + # Establish CBW from source to fleecing node + if use_cbw: + log(vm.qmp("blockdev-add", **{ + "driver": "copy-before-write", + "node-name": "fl-cbw", + "file": src_node, + "target": tgt_nodeWhich makes this look strange, and also TIL (or maybe “today I was reminded”?) that you can use BB names as node references.
Quoted 3 lines
(It’s also already weird in the
"backing": src_nodeline right at the beginning of this hunk...)So I guess it works, but I’d find it would be nicer to distinguish between the node name and the BB name.
Oh yes, that's all the mess, I'll improve.
Quoted 6 lines
tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 106
Quoted 5 lines
tests/qemu-iotests/222 +4
-0+ })) + + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]", + property="drive", value="fl-cbw"))
Quoted 6 lines
Is that path portable across targets? Should we maybe instead manually add a virtio-scsi device that we can give a proper name?
Or I suppose we could do
path = next(dev for dev in vm.qmp('query-block')['return']
if dev['device'] == 'drive0')['qdev']
OK
Quoted 20 lines
Quoted 9 lines
tests/qemu-iotests/222 +5
-0+ else: + log(vm.qmp("blockdev-backup", + device=src_node, + target=tgt_node, + sync="none")) log('') log('--- Setting up NBD Export ---')tests/qemu-iotests/222 with iotests.FilePath('base.img') as base_img_path, \ · line 156
Quoted 8 lines
tests/qemu-iotests/222 +3
-3@@ -139,9 +152,15 @@ with iotests.FilePath('base.img') as base_img_path, \ log('--- Cleanup ---') log('') - log(vm.qmp('block-job-cancel', device=src_node)) - log(vm.event_wait('BLOCK_JOB_CANCELLED'), - filters=[iotests.filter_qmp_event]) + if use_cbw: + log(vm.qmp("qom-set", path="/machine/peripheral-anon/device[0]", + property="drive", value="node0"))If
src_nodewere 'node0', we wouldn’t need to use the 'node0' literal here and could usesrc_nodeinstead. (Because it kind of seems like this test wants to use those variables instead of literals. I mean, we could also just call the node 'src-node', but, well.)
Quoted 12 lines
Max
Quoted 9 lines
tests/qemu-iotests/222 +6
-0+ log(vm.qmp("blockdev-del", node_name="fl-cbw")) + else: + log(vm.qmp('block-job-cancel', device=src_node)) + log(vm.event_wait('BLOCK_JOB_CANCELLED'), + filters=[iotests.filter_qmp_event]) + log(vm.qmp('nbd-server-stop')) log(vm.qmp('blockdev-del', node_name=tgt_node)) vm.shutdown()
Great thanks for reviewing!
Signature · 2 lines
Best regards, Vladimir