bitmaps: add inconsistent bit
cover bitmaps: add inconsistent bit
Allow QEMU to read in bitmaps that have the in-use bit set, for the purposes of allowing users to clear or reset these bitmaps.
This is chosen in preference to a hard error on load to minimize impact for a non-critical error, but to force the user or management utility to acknowledge that the bitmap is no longer viable.
Requires: [PATCH v2 0/6] dirty-bitmaps: deprecate @status field
(Which in turn requires my bitmaps staging branch.)
RFC: This is just for general approach, naming, and API.
I chose NOT to overload the busy predicate so that it would be distinct, which unfortunately means many more manual checks across blockdev.
- I chose "inconsistent" over "corrupt" to be more literal to the meaning of the "in-use" bit, but maybe this is overcautious.
- I chose to make the field optional so that it disappears in normative cases, as the information is only really relevant when inconsistent=true.
I also have NOT tested this and I didn't verify the saving logic for what happens if you don't delete or clear the bitmap, but I'm on PTO the next two days and I wanted this to see daylight.
John Snow (2):
block/dirty-bitmaps: add inconsistent bit
block/dirty-bitmap: implement inconsistent bit
block/dirty-bitmap.c | 34 ++++++++++++++++++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 3 +++ qapi/block-core.json | 9 ++++++-- 5 files changed, 109 insertions(+), 22 deletions(-)
Signature · 1 line
2.17.2
RFC PATCH 1/2 block/dirty-bitmaps: add inconsistent bit
Add an inconsistent bit to dirty-bitmaps that allows us to report a bitmap as persistent but potentially inconsistent, i.e. if we find bitmaps on a qcow2 that have been marked as "in use".
--- block/dirty-bitmap.c | 19 +++++++++++++++++++ include/block/dirty-bitmap.h | 2 ++ qapi/block-core.json | 9 +++++++-- 3 files changed, 28 insertions(+), 2 deletions(-)
block/dirty-bitmap.c +19 -0
@@ -47,6 +47,9 @@ struct BdrvDirtyBitmap {and this bitmap must remain unchanged while this flag is set. */ bool persistent;/* bitmap must be saved to owner disk image */ +bool inconsistent;/* bitmap is persistent, but not owned by QEMU. +* It cannot be used at all in any way, except +* a QMP user can remove or clear it. */ bool migration;/* Bitmap is selected for migration, it should not be stored on the next inactivation (persistent flag doesn't matter until next
@@ -461,6 +464,8 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs) info->recording =bdrv_dirty_bitmap_recording (bm); info->busy =bdrv_dirty_bitmap_busy (bm); info->persistent = bm->persistent; + info->has_inconsistent = bm->inconsistent; + info->inconsistent = bm->inconsistent; entry->value = info; *plist = entry; plist = &entry->next;
@@ -708,6 +713,15 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent)qemu_mutex_unlock (bitmap->mutex); } +/* Called with BQL taken. */ +void bdrv_dirty_bitmap_set_inconsistent (BdrvDirtyBitmap *bitmap,bool value) +{ +qemu_mutex_lock (bitmap->mutex); + bitmap->inconsistent = value; + bitmap->disabled = value; +qemu_mutex_unlock (bitmap->mutex); +} +/* Called with BQL taken. */ void bdrv_dirty_bitmap_set_migration (BdrvDirtyBitmap *bitmap,bool migration) {
@@ -721,6 +735,11 @@ bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap)return bitmap->persistent && !bitmap->migration; } +bool bdrv_dirty_bitmap_inconsistent (BdrvDirtyBitmap *bitmap) +{ +return bitmap->inconsistent; +} +bool bdrv_has_changed_persistent_bitmaps (BlockDriverState *bs) {BdrvDirtyBitmap *bm;
include/block/dirty-bitmap.h +2 -0
@@ -68,6 +68,7 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);void bdrv_dirty_bitmap_set_readonly (BdrvDirtyBitmap *bitmap,bool value);void bdrv_dirty_bitmap_set_persistance (BdrvDirtyBitmap *bitmap,bool persistent); +void bdrv_dirty_bitmap_set_inconsistent (BdrvDirtyBitmap *bitmap,bool value);void bdrv_dirty_bitmap_set_busy (BdrvDirtyBitmap *bitmap,bool busy);void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp);
@@ -91,6 +92,7 @@ bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap);bool bdrv_has_readonly_bitmaps (BlockDriverState *bs);bool bdrv_dirty_bitmap_get_autoload (const BdrvDirtyBitmap *bitmap);bool bdrv_dirty_bitmap_get_persistance (BdrvDirtyBitmap *bitmap); +bool bdrv_dirty_bitmap_inconsistent (BdrvDirtyBitmap *bitmap);bool bdrv_dirty_bitmap_busy (BdrvDirtyBitmap *bitmap);bool bdrv_has_changed_persistent_bitmaps (BlockDriverState *bs); BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
qapi/block-core.json +7 -2
@@ -470,12 +470,17 @@ # @persistent: true if the bitmap will eventually be flushed to persistent # storage (since 4.0) # +# @inconsistent: true if this is a persistent bitmap that QEMU does not own. +# Implies @recording to be false and @busy to be true. To +# reclaim ownership, see @block-dirty-bitmap-remove or +# @block-dirty-bitmap-clear (since 4.0) +# # Since: 1.3 ## { 'struct': 'BlockDirtyInfo', 'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32', - 'recording': 'bool', 'busy': 'bool', - 'status': 'DirtyBitmapStatus', 'persistent': 'bool' } } + 'recording': 'bool', 'busy': 'bool', 'status': 'DirtyBitmapStatus', + 'persistent': 'bool', '*inconsistent': 'bool' } } ## # @Qcow2BitmapInfoFlags:
Signature · 1 line
2.17.2
Re: [Qemu-devel] [RFC PATCH 1/2] block/dirty-bitmaps: add inconsistent bit
Quoted 35 lines · John Snow
Add an inconsistent bit to dirty-bitmaps that allows us to report a bitmap as persistent but potentially inconsistent, i.e. if we find bitmaps on a qcow2 that have been marked as "in use".
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 19 +++++++++++++++++++ include/block/dirty-bitmap.h | 2 ++ qapi/block-core.json | 9 +++++++-- 3 files changed, 28 insertions(+), 2 deletions(-)block/dirty-bitmap.c +5
-0@@ -47,6 +47,9 @@ struct BdrvDirtyBitmap {and this bitmap must remain unchanged while this flag is set. */ bool persistent;/* bitmap must be saved to owner disk image */ +bool inconsistent;/* bitmap is persistent, but not owned by QEMU. +* It cannot be used at all in any way, except +* a QMP user can remove or clear it. */ bool migration;/* Bitmap is selected for migration, it should not be stored on the next inactivation (persistent flag doesn't matter until next @@ -461,6 +464,8 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs) info->recording =bdrv_dirty_bitmap_recording (bm); info->busy =bdrv_dirty_bitmap_busy (bm); info->persistent = bm->persistent; + info->has_inconsistent = bm->inconsistent; + info->inconsistent = bm->inconsistent; entry->value = info; *plist = entry; plist = &entry->next;
block/dirty-bitmap.c void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent) · line 716
Quoted 4 lines
block/dirty-bitmap.c +1
-0@@ -708,6 +713,15 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent)qemu_mutex_unlock (bitmap->mutex); } +/* Called with BQL taken. */
Do we need BQL if we use mutex explicitly?
block/dirty-bitmap.c void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent) · line 721
Quoted 5 lines
block/dirty-bitmap.c +5
-0+void bdrv_dirty_bitmap_set_inconsistent (BdrvDirtyBitmap *bitmap,bool value) +{ +qemu_mutex_lock (bitmap->mutex); + bitmap->inconsistent = value; + bitmap->disabled = value;
So, set inconsistent to false will enable the bitmap? Seems tricky.
Quoted 63 lines
block/dirty-bitmap.c +8
-0+qemu_mutex_unlock (bitmap->mutex); +} +/* Called with BQL taken. */ void bdrv_dirty_bitmap_set_migration (BdrvDirtyBitmap *bitmap,bool migration) {@@ -721,6 +735,11 @@ bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap)return bitmap->persistent && !bitmap->migration; } +bool bdrv_dirty_bitmap_inconsistent (BdrvDirtyBitmap *bitmap) +{ +return bitmap->inconsistent; +} +bool bdrv_has_changed_persistent_bitmaps (BlockDriverState *bs) {BdrvDirtyBitmap *bm;include/block/dirty-bitmap.h +2
-0@@ -68,6 +68,7 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);void bdrv_dirty_bitmap_set_readonly (BdrvDirtyBitmap *bitmap,bool value);void bdrv_dirty_bitmap_set_persistance (BdrvDirtyBitmap *bitmap,bool persistent); +void bdrv_dirty_bitmap_set_inconsistent (BdrvDirtyBitmap *bitmap,bool value);void bdrv_dirty_bitmap_set_busy (BdrvDirtyBitmap *bitmap,bool busy);void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp);@@ -91,6 +92,7 @@ bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap);bool bdrv_has_readonly_bitmaps (BlockDriverState *bs);bool bdrv_dirty_bitmap_get_autoload (const BdrvDirtyBitmap *bitmap);bool bdrv_dirty_bitmap_get_persistance (BdrvDirtyBitmap *bitmap); +bool bdrv_dirty_bitmap_inconsistent (BdrvDirtyBitmap *bitmap);bool bdrv_dirty_bitmap_busy (BdrvDirtyBitmap *bitmap);bool bdrv_has_changed_persistent_bitmaps (BlockDriverState *bs); BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,qapi/block-core.json +7
-2@@ -470,12 +470,17 @@ # @persistent: true if the bitmap will eventually be flushed to persistent # storage (since 4.0) # +# @inconsistent: true if this is a persistent bitmap that QEMU does not own. +# Implies @recording to be false and @busy to be true. To +# reclaim ownership, see @block-dirty-bitmap-remove or +# @block-dirty-bitmap-clear (since 4.0) +# # Since: 1.3 ## { 'struct': 'BlockDirtyInfo', 'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32', - 'recording': 'bool', 'busy': 'bool', - 'status': 'DirtyBitmapStatus', 'persistent': 'bool' } } + 'recording': 'bool', 'busy': 'bool', 'status': 'DirtyBitmapStatus', + 'persistent': 'bool', '*inconsistent': 'bool' } } ## # @Qcow2BitmapInfoFlags:
Signature · 2 lines
Best regards, Vladimir
Re: [Qemu-devel] [RFC PATCH 1/2] block/dirty-bitmaps: add inconsistent bit
Quoted 43 lines · Vladimir Sementsov-Ogievskiy
Quoted 35 lines · John Snow
Add an inconsistent bit to dirty-bitmaps that allows us to report a bitmap as persistent but potentially inconsistent, i.e. if we find bitmaps on a qcow2 that have been marked as "in use".
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 19 +++++++++++++++++++ include/block/dirty-bitmap.h | 2 ++ qapi/block-core.json | 9 +++++++-- 3 files changed, 28 insertions(+), 2 deletions(-)block/dirty-bitmap.c +5
-0@@ -47,6 +47,9 @@ struct BdrvDirtyBitmap {and this bitmap must remain unchanged while this flag is set. */ bool persistent;/* bitmap must be saved to owner disk image */ +bool inconsistent;/* bitmap is persistent, but not owned by QEMU. +* It cannot be used at all in any way, except +* a QMP user can remove or clear it. */ bool migration;/* Bitmap is selected for migration, it should not be stored on the next inactivation (persistent flag doesn't matter until next @@ -461,6 +464,8 @@ BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs) info->recording =bdrv_dirty_bitmap_recording (bm); info->busy =bdrv_dirty_bitmap_busy (bm); info->persistent = bm->persistent; + info->has_inconsistent = bm->inconsistent; + info->inconsistent = bm->inconsistent; entry->value = info; *plist = entry; plist = &entry->next;block/dirty-bitmap.c void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent) · line 716
Quoted 4 lines
block/dirty-bitmap.c +1
-0@@ -708,6 +713,15 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent)qemu_mutex_unlock (bitmap->mutex); } +/* Called with BQL taken. */ Do we need BQL if we use mutex explicitly?
Quoted 8 lines
block/dirty-bitmap.c void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent) · line 721
Quoted 5 lines
block/dirty-bitmap.c +5
-0+void bdrv_dirty_bitmap_set_inconsistent (BdrvDirtyBitmap *bitmap,bool value) +{ +qemu_mutex_lock (bitmap->mutex); + bitmap->inconsistent = value; + bitmap->disabled = value;So, set inconsistent to false will enable the bitmap? Seems tricky.
It was really meant to be inconsistent = disabled = true. It falls apart in the reverse. However, if we don't allow clear, this can just be a constant.
Quoted 65 lines
Quoted 63 lines
block/dirty-bitmap.c +8
-0+qemu_mutex_unlock (bitmap->mutex); +} +/* Called with BQL taken. */ void bdrv_dirty_bitmap_set_migration (BdrvDirtyBitmap *bitmap,bool migration) {@@ -721,6 +735,11 @@ bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap)return bitmap->persistent && !bitmap->migration; } +bool bdrv_dirty_bitmap_inconsistent (BdrvDirtyBitmap *bitmap) +{ +return bitmap->inconsistent; +} +bool bdrv_has_changed_persistent_bitmaps (BlockDriverState *bs) {BdrvDirtyBitmap *bm;include/block/dirty-bitmap.h +2
-0@@ -68,6 +68,7 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);void bdrv_dirty_bitmap_set_readonly (BdrvDirtyBitmap *bitmap,bool value);void bdrv_dirty_bitmap_set_persistance (BdrvDirtyBitmap *bitmap,bool persistent); +void bdrv_dirty_bitmap_set_inconsistent (BdrvDirtyBitmap *bitmap,bool value);void bdrv_dirty_bitmap_set_busy (BdrvDirtyBitmap *bitmap,bool busy);void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp);@@ -91,6 +92,7 @@ bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap);bool bdrv_has_readonly_bitmaps (BlockDriverState *bs);bool bdrv_dirty_bitmap_get_autoload (const BdrvDirtyBitmap *bitmap);bool bdrv_dirty_bitmap_get_persistance (BdrvDirtyBitmap *bitmap); +bool bdrv_dirty_bitmap_inconsistent (BdrvDirtyBitmap *bitmap);bool bdrv_dirty_bitmap_busy (BdrvDirtyBitmap *bitmap);bool bdrv_has_changed_persistent_bitmaps (BlockDriverState *bs); BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,qapi/block-core.json +7
-2@@ -470,12 +470,17 @@ # @persistent: true if the bitmap will eventually be flushed to persistent # storage (since 4.0) # +# @inconsistent: true if this is a persistent bitmap that QEMU does not own. +# Implies @recording to be false and @busy to be true. To +# reclaim ownership, see @block-dirty-bitmap-remove or +# @block-dirty-bitmap-clear (since 4.0) +# # Since: 1.3 ## { 'struct': 'BlockDirtyInfo', 'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32', - 'recording': 'bool', 'busy': 'bool', - 'status': 'DirtyBitmapStatus', 'persistent': 'bool' } } + 'recording': 'bool', 'busy': 'bool', 'status': 'DirtyBitmapStatus', + 'persistent': 'bool', '*inconsistent': 'bool' } } ## # @Qcow2BitmapInfoFlags:
RFC PATCH 2/2 block/dirty-bitmap: implement inconsistent bit
In reply to John Snow--- block/dirty-bitmap.c | 15 +++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + 4 files changed, 81 insertions(+), 20 deletions(-)
block/dirty-bitmap.c +15 -0
@@ -589,6 +589,7 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) *out = backup; }bdrv_dirty_bitmap_unlock (bitmap); +bdrv_dirty_bitmap_set_inconsistent (bitmap, false); }void bdrv_restore_dirty_bitmap (BdrvDirtyBitmap *bitmap,HBitmap *backup)
@@ -776,6 +777,13 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,return hbitmap_next_dirty_area (bitmap->bitmap, offset, bytes); } +void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp) +{ +error_append_hint (errp,"Try block-dirty-bitmap-clear to mark this " +"bitmap consistent again, or block-dirty-bitmap-remove " +"to delete it." ); +} +void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp) {
@@ -798,6 +806,13 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, goto out; } +if (bdrv_dirty_bitmap_inconsistent (dest)) { +error_setg (errp,"Bitmap '%s' is inconsistent and cannot be used as" +" a merge target" , dest->name); +bdrv_dirty_bitmap_add_inconsistent_hint (errp); + goto out; + } +if (!hbitmap_can_merge (dest->bitmap, src->bitmap)) {error_setg (errp,"Bitmaps are incompatible and can't be merged" ); goto out;
block/qcow2-bitmap.c +22 -20
63 lines not on this page · raw message
blockdev.c +43 -0
97 lines not on this page · raw message
include/block/dirty-bitmap.h +1 -0
@@ -105,5 +105,6 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked (BlockDriverState *bs,BdrvDirtyBitmap *bitmap,Error **errp); +void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp);#endif
Signature · 1 line
2.17.2
Re: [Qemu-devel] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit
Quoted 21 lines · John Snow
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 15 +++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + 4 files changed, 81 insertions(+), 20 deletions(-)block/dirty-bitmap.c +1
-0@@ -589,6 +589,7 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) *out = backup; }bdrv_dirty_bitmap_unlock (bitmap); +bdrv_dirty_bitmap_set_inconsistent (bitmap, false); }void bdrv_restore_dirty_bitmap (BdrvDirtyBitmap *bitmap,HBitmap *backup)
block/dirty-bitmap.c bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap, · line 784
Quoted 8 lines
block/dirty-bitmap.c +5
-0@@ -776,6 +777,13 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,return hbitmap_next_dirty_area (bitmap->bitmap, offset, bytes); } +void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp) +{ +error_append_hint (errp,"Try block-dirty-bitmap-clear to mark this " +"bitmap consistent again, or block-dirty-bitmap-remove " +"to delete it." );
bitmaps created by libvirt (or somebody) are related to some checkpoint. And their name is probably (Eric?) related to this checkpoint too. So, clear will never make them consistent.. Only clear :)
So, I don't like idea of clearing in-use bitmaps.
Quoted 6 lines
block/dirty-bitmap.c +2
-0+} +void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp) {
block/dirty-bitmap.c void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, · line 814
Quoted 9 lines
block/dirty-bitmap.c +6
-0@@ -798,6 +806,13 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, goto out; } +if (bdrv_dirty_bitmap_inconsistent (dest)) { +error_setg (errp,"Bitmap '%s' is inconsistent and cannot be used as" +" a merge target" , dest->name); +bdrv_dirty_bitmap_add_inconsistent_hint (errp); + goto out; + }
I think, we need common predicate which will combine busy and inconsistent, as almost in all cases we need both to be false to do any operation.
bdrv_dirty_bitmap_usable() ? :)
and pass errp to this helper.
Actually, we already need it, to fill errp, which we almost always fill in the same manner.
block/qcow2-bitmap.c
Quoted 8 lines
block/dirty-bitmap.c +1
-0+if (!hbitmap_can_merge (dest->bitmap, src->bitmap)) {error_setg (errp,"Bitmaps are incompatible and can't be merged" ); goto out;block/qcow2-bitmap.c +0
-0
hmm, I also think we should report our deprecated status as locked for inconsistent bitmaps..
Signature · 2 lines
Best regards, Vladimir
Re: [Qemu-devel] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit
Quoted 19 lines · Vladimir Sementsov-Ogievskiy
block/dirty-bitmap.c bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap, · line 784
Quoted 13 lines · John Snow
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 15 +++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + 4 files changed, 81 insertions(+), 20 deletions(-)block/dirty-bitmap.c +5
-0+void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp) +{ +error_append_hint (errp,"Try block-dirty-bitmap-clear to mark this " +"bitmap consistent again, or block-dirty-bitmap-remove " +"to delete it." );bitmaps created by libvirt (or somebody) are related to some checkpoint. And their name is probably (Eric?) related to this checkpoint too. So, clear will never make them consistent.. Only clear :)
Quoted 1 line
So, I don't like idea of clearing in-use bitmaps.
It's always possible to delete a bitmap and then create a new one by the same name, to get the same effect of clearing an in-use bitmap. So let's start simple and declare that the only valid operation on an inconsistent bitmap is deletion.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit
Quoted 20 lines · Eric Blake
Quoted 19 lines · Vladimir Sementsov-Ogievskiy
block/dirty-bitmap.c bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap, · line 784
Quoted 13 lines · John Snow
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 15 +++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + 4 files changed, 81 insertions(+), 20 deletions(-)block/dirty-bitmap.c +5
-0+void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp) +{ +error_append_hint (errp,"Try block-dirty-bitmap-clear to mark this " +"bitmap consistent again, or block-dirty-bitmap-remove " +"to delete it." );bitmaps created by libvirt (or somebody) are related to some checkpoint. And their name is probably (Eric?) related to this checkpoint too. So, clear will never make them consistent.. Only clear :)
Quoted 7 lines
Quoted 1 line
So, I don't like idea of clearing in-use bitmaps.
It's always possible to delete a bitmap and then create a new one by the same name, to get the same effect of clearing an in-use bitmap. So let's start simple and declare that the only valid operation on an inconsistent bitmap is deletion.
OK, from the viewpoint of checkpoints specifically, that's a convincing argument against it for now.
I'll tighten this.
Re: [Qemu-devel] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit
In reply to Vladimir Sementsov-OgievskiyQuoted 62 lines · Vladimir Sementsov-Ogievskiy
Quoted 21 lines · John Snow
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 15 +++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + 4 files changed, 81 insertions(+), 20 deletions(-)block/dirty-bitmap.c +1
-0@@ -589,6 +589,7 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) *out = backup; }bdrv_dirty_bitmap_unlock (bitmap); +bdrv_dirty_bitmap_set_inconsistent (bitmap, false); }void bdrv_restore_dirty_bitmap (BdrvDirtyBitmap *bitmap,HBitmap *backup)block/dirty-bitmap.c bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap, · line 784
Quoted 8 lines
block/dirty-bitmap.c +5
-0@@ -776,6 +777,13 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,return hbitmap_next_dirty_area (bitmap->bitmap, offset, bytes); } +void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp) +{ +error_append_hint (errp,"Try block-dirty-bitmap-clear to mark this " +"bitmap consistent again, or block-dirty-bitmap-remove " +"to delete it." );bitmaps created by libvirt (or somebody) are related to some checkpoint. And their name is probably (Eric?) related to this checkpoint too. So, clear will never make them consistent.. Only clear :)
So, I don't like idea of clearing in-use bitmaps.
Quoted 6 lines
block/dirty-bitmap.c +2
-0+} +void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp) {block/dirty-bitmap.c void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, · line 814
Quoted 9 lines
block/dirty-bitmap.c +6
-0@@ -798,6 +806,13 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, goto out; } +if (bdrv_dirty_bitmap_inconsistent (dest)) { +error_setg (errp,"Bitmap '%s' is inconsistent and cannot be used as" +" a merge target" , dest->name); +bdrv_dirty_bitmap_add_inconsistent_hint (errp); + goto out; + }I think, we need common predicate which will combine busy and inconsistent, as almost in all cases we need both to be false to do any operation.
bdrv_dirty_bitmap_usable()? :)and pass errp to this helper.
Actually, we already need it, to fill errp, which we almost always fill in the same manner.
Quoted 12 lines
block/qcow2-bitmap.c
Quoted 8 lines
block/dirty-bitmap.c +1
-0+if (!hbitmap_can_merge (dest->bitmap, src->bitmap)) {error_setg (errp,"Bitmaps are incompatible and can't be merged" ); goto out;block/qcow2-bitmap.c +0
-0hmm, I also think we should report our deprecated status as locked for inconsistent bitmaps..
Though we're trying to deprecate the field altogether, I *could* add a special status flag that makes it unambiguous. This will catch the attention of anyone using the old API.
--js
Re: [Qemu-devel] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit
Quoted 78 lines · John Snow
Quoted 62 lines · Vladimir Sementsov-Ogievskiy
Quoted 21 lines · John Snow
Signed-off-by John Snow <[email protected]>--- block/dirty-bitmap.c | 15 +++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 1 + 4 files changed, 81 insertions(+), 20 deletions(-)block/dirty-bitmap.c +1
-0@@ -589,6 +589,7 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) *out = backup; }bdrv_dirty_bitmap_unlock (bitmap); +bdrv_dirty_bitmap_set_inconsistent (bitmap, false); }void bdrv_restore_dirty_bitmap (BdrvDirtyBitmap *bitmap,HBitmap *backup)block/dirty-bitmap.c bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap, · line 784
Quoted 8 lines
block/dirty-bitmap.c +5
-0@@ -776,6 +777,13 @@ bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,return hbitmap_next_dirty_area (bitmap->bitmap, offset, bytes); } +void bdrv_dirty_bitmap_add_inconsistent_hint (Error **errp) +{ +error_append_hint (errp,"Try block-dirty-bitmap-clear to mark this " +"bitmap consistent again, or block-dirty-bitmap-remove " +"to delete it." );bitmaps created by libvirt (or somebody) are related to some checkpoint. And their name is probably (Eric?) related to this checkpoint too. So, clear will never make them consistent.. Only clear :)
So, I don't like idea of clearing in-use bitmaps.
Quoted 6 lines
block/dirty-bitmap.c +2
-0+} +void bdrv_merge_dirty_bitmap (BdrvDirtyBitmap *dest,const BdrvDirtyBitmap *src,HBitmap **backup,Error **errp) {block/dirty-bitmap.c void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, · line 814
Quoted 9 lines
block/dirty-bitmap.c +6
-0@@ -798,6 +806,13 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, goto out; } +if (bdrv_dirty_bitmap_inconsistent (dest)) { +error_setg (errp,"Bitmap '%s' is inconsistent and cannot be used as" +" a merge target" , dest->name); +bdrv_dirty_bitmap_add_inconsistent_hint (errp); + goto out; + }I think, we need common predicate which will combine busy and inconsistent, as almost in all cases we need both to be false to do any operation.
bdrv_dirty_bitmap_usable()? :)and pass errp to this helper.
Actually, we already need it, to fill errp, which we almost always fill in the same manner.
Quoted 12 lines
block/qcow2-bitmap.c
Quoted 8 lines
block/dirty-bitmap.c +1
-0+if (!hbitmap_can_merge (dest->bitmap, src->bitmap)) {error_setg (errp,"Bitmaps are incompatible and can't be merged" ); goto out;block/qcow2-bitmap.c +0
-0hmm, I also think we should report our deprecated status as locked for inconsistent bitmaps..
Quoted 3 lines
Though we're trying to deprecate the field altogether, I *could* add a special status flag that makes it unambiguous. This will catch the attention of anyone using the old API.
I agree.
Signature · 2 lines
Best regards, Vladimir
Re: [Qemu-devel] [RFC PATCH 2/2] block/dirty-bitmap: implement inconsistent bit
In reply to John SnowQuoted 8 lines · John Snow
Quoted 4 lines
hmm, I also think we should report our deprecated status as locked for inconsistent bitmaps..
Though we're trying to deprecate the field altogether, I *could* add a special status flag that makes it unambiguous. This will catch the attention of anyone using the old API.
Adding to the enum, even though it is going away in the future, is still helpful in the present, so I can live with that approach.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Quoted 7 lines · John Snow
Allow QEMU to read in bitmaps that have the in-use bit set, for the purposes of allowing users to clear or reset these bitmaps.
This is chosen in preference to a hard error on load to minimize impact for a non-critical error, but to force the user or management utility to acknowledge that the bitmap is no longer viable.
Quoted 6 lines
Requires: [PATCH v2 0/6] dirty-bitmaps: deprecate @status field
(Which in turn requires my bitmaps staging branch.)RFC: This is just for general approach, naming, and API.
I chose NOT to overload the busy predicate so that it would be distinct, which unfortunately means many more manual checks across blockdev.
I think we just need a helper predicate with errp parameter, which will call both _busy and _inconsistent.
Quoted 3 lines
- I chose "inconsistent" over "corrupt" to be more literal to the meaning of the "in-use" bit, but maybe this is overcautious.
I'm ok with both
Quoted 4 lines
- I chose to make the field optional so that it disappears in normative cases, as the information is only really relevant when inconsistent=true.
Agree
Quoted 4 lines
I also have NOT tested this and I didn't verify the saving logic for what happens if you don't delete or clear the bitmap, but I'm on PTO the next two days and I wanted this to see daylight.
I think, we should not change clear() logic to make bitmaps consistent. Is there real usage of it?
1. bitmap (and its name) is related to some checkpoint, and clearing don't make it consistent
2. it is extra difficulty
3. we can add this logic separately, later, on demand, if needed
4. without it, we can use the following simple interface and easily save RAM:
no bdrv_dirty_bitmap_set_inconsistent(), but just bdrv_create_inconsistent_dirty_bitmap() (ahaha cool function name), and call it in qcow2 code, when found in-use bitmap. And, in this bdrv_create_inconsitent_dirty_bitmap() do not allocate HBitmap, which may save a lot of memory.
Quoted 12 lines
John Snow (2):
block/dirty-bitmaps: add inconsistent bit
block/dirty-bitmap: implement inconsistent bitblock/dirty-bitmap.c | 34 ++++++++++++++++++++++++++++ block/qcow2-bitmap.c | 42 ++++++++++++++++++----------------- blockdev.c | 43 ++++++++++++++++++++++++++++++++++++ include/block/dirty-bitmap.h | 3 +++ qapi/block-core.json | 9 ++++++-- 5 files changed, 109 insertions(+), 22 deletions(-)
Signature · 2 lines
Best regards, Vladimir