Skip to contentSkip to conversation

block/qcow2-bitmap: Enable resize with persistent bitmaps

PATCH In review Vladimir Sementsov-Ogievskiy 2 participants 8 messages Mar 11, 2019 – Mar 12, 2019 Block layer

lore
Share & tools
lore
#0 lore raw

v2 cover block/qcow2-bitmap: Enable resize with persistent bitmaps

Hi!

This is my counter-propasal on
"[PATCH 0/5] block/qcow2-bitmap: Enable resize with persistent bitmaps" by John.

It does bitmap resize, but don't flush bitmaps during resize, postponing this controversial thing.

Based-on: https://github.com/jnsnow/qemu bitmaps

John Snow (2):
block/qcow2-bitmap: Allow resizes with persistent bitmaps tests/qemu-iotests: add bitmap resize test 246

Vladimir Sementsov-Ogievskiy (2):
docs/interop/qcow2: Improve bitmap flag in_use specification block/qcow2-bitmap: Don't check size for IN_USE bitmap

 docs/interop/qcow2.txt     |  18 ++-
 block/qcow2.h              |   1 +
 block/qcow2-bitmap.c       |  67 ++++++++-
 block/qcow2.c              |   4 +-
 tests/qemu-iotests/246     | 114 ++++++++++++++
 tests/qemu-iotests/246.out | 295 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 7 files changed, 491 insertions(+), 9 deletions(-)
 create mode 100755 tests/qemu-iotests/246
 create mode 100644 tests/qemu-iotests/246.out
Signature · 1 line
2.18.0
#1 lore raw

PATCH v2 1/4 docs/interop/qcow2: Improve bitmap flag in_use specification

We already use (we didn't notice it) IN_USE flag for marking bitmap metadata outdated, such as AUTO flag, which mirrors enabled/disabled bitmaps. No we are going to support bitmap resize, so it's good to write IN_USE meaning with more details.

---
 docs/interop/qcow2.txt | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
docs/interop/qcow2.txt +15 -3
index fb5cb47245..575a5f25e2 100644
@@ -589,7 +589,19 @@ Structure of a bitmap directory entry:
                     Bit
                       0: in_use
                          The bitmap was not saved correctly and may be
-                         inconsistent.
+                         inconsistent. This inconsitency may touch both bitmap
+                         data and metadata, and this mean that bitmap state
+                         (its data and metadata) was changed but not stored
+                         back to the image. This flag doesn't relate to format
+                         corruption, all fields are still correct from qcow2
+                         point of view, they just may be outdated.
+
+                         Note: Currently, Qemu may change (additionally to
+                         bitmap data) @auto flag and size of the bitmap during
+                         image resize. This mean, that not only bitmap data
+                         may be outdated if @in_use flag set, but also value of
+                         @auto flag and bitmap size (which is indirectly
+                         referenced by @bitmap_table_size).
 
                       1: auto
                          The bitmap must reflect all changes of the virtual
@@ -717,8 +729,8 @@ corresponding range of the virtual disk (see above) was written to while the
 bitmap was 'enabled'. An unset bit means that this range was not written to.
 
 The software doesn't have to sync the bitmap in the image file with its
-representation in RAM after each write. Flag 'in_use' should be set while the
-bitmap is not synced.
+representation in RAM after each write or metadata change. Flag 'in_use'
+should be set while the bitmap is not synced.
 
 In the image file the 'enabled' state is reflected by the 'auto' flag. If this
 flag is set, the software must consider the bitmap as 'enabled' and start
Signature · 1 line
2.18.0
#2 lore raw

Re: [Qemu-devel] [PATCH v2 1/4] docs/interop/qcow2: Improve bitmap flag in_use specification

Quoted 3 lines · Vladimir Sementsov-Ogievskiy

We already use (we didn't notice it) IN_USE flag for marking bitmap metadata outdated, such as AUTO flag, which mirrors enabled/disabled bitmaps. No we are going to support bitmap resize, so it's good to

s/No/Now/

Quoted 12 lines

write IN_USE meaning with more details.

---
 docs/interop/qcow2.txt | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
docs/interop/qcow2.txt +0 -0
index fb5cb47245..575a5f25e2 100644

docs/interop/qcow2.txt Structure of a bitmap directory entry: · line 592

Quoted 5 lines
docs/interop/qcow2.txt +1 -1
@@ -589,7 +589,19 @@ Structure of a bitmap directory entry:
                     Bit
                       0: in_use
                          The bitmap was not saved correctly and may be
-                         inconsistent.
+                         inconsistent. This inconsitency may touch both bitmap

inconsistency

docs/interop/qcow2.txt Structure of a bitmap directory entry: · line 604

Quoted 12 lines
docs/interop/qcow2.txt +12 -0
+                         data and metadata, and this mean that bitmap state
+                         (its data and metadata) was changed but not stored
+                         back to the image. This flag doesn't relate to format
+                         corruption, all fields are still correct from qcow2
+                         point of view, they just may be outdated.
+
+                         Note: Currently, Qemu may change (additionally to
+                         bitmap data) @auto flag and size of the bitmap during
+                         image resize. This mean, that not only bitmap data
+                         may be outdated if @in_use flag set, but also value of
+                         @auto flag and bitmap size (which is indirectly
+                         referenced by @bitmap_table_size).

Feels wordy. Maybe drop the second paragraph starting with Note, and merely use this for the first paragraph:

The bitmap was not saved correctly and may be inconsistent. Although the bitmap metadata is still well-formed from a qcow2 perspective, the metadata (such as the auto flag or bitmap size) or data contents may be outdated.

Quoted 15 lines
docs/interop/qcow2.txt +2 -2
                       1: auto
                          The bitmap must reflect all changes of the virtual
@@ -717,8 +729,8 @@ corresponding range of the virtual disk (see above) was written to while the
 bitmap was 'enabled'. An unset bit means that this range was not written to.
 
 The software doesn't have to sync the bitmap in the image file with its
-representation in RAM after each write. Flag 'in_use' should be set while the
-bitmap is not synced.
+representation in RAM after each write or metadata change. Flag 'in_use'
+should be set while the bitmap is not synced.
 
 In the image file the 'enabled' state is reflected by the 'auto' flag. If this
 flag is set, the software must consider the bitmap as 'enabled' and start
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#3 lore raw

Re: [Qemu-devel] [PATCH v2 1/4] docs/interop/qcow2: Improve bitmap flag in_use specification

Quoted 40 lines · Eric Blake
Quoted 3 lines · Vladimir Sementsov-Ogievskiy

We already use (we didn't notice it) IN_USE flag for marking bitmap metadata outdated, such as AUTO flag, which mirrors enabled/disabled bitmaps. No we are going to support bitmap resize, so it's good to

s/No/Now/

Quoted 12 lines

write IN_USE meaning with more details.

---
  docs/interop/qcow2.txt | 18 +++++++++++++++---
  1 file changed, 15 insertions(+), 3 deletions(-)
docs/interop/qcow2.txt +0 -0
index fb5cb47245..575a5f25e2 100644

docs/interop/qcow2.txt Structure of a bitmap directory entry: · line 592

Quoted 5 lines
docs/interop/qcow2.txt +1 -1
@@ -589,7 +589,19 @@ Structure of a bitmap directory entry:
                      Bit
                        0: in_use
                           The bitmap was not saved correctly and may be
-                         inconsistent.
+                         inconsistent. This inconsitency may touch both bitmap

inconsistency

docs/interop/qcow2.txt Structure of a bitmap directory entry: · line 604

Quoted 12 lines
docs/interop/qcow2.txt +12 -0
+                         data and metadata, and this mean that bitmap state
+                         (its data and metadata) was changed but not stored
+                         back to the image. This flag doesn't relate to format
+                         corruption, all fields are still correct from qcow2
+                         point of view, they just may be outdated.
+
+                         Note: Currently, Qemu may change (additionally to
+                         bitmap data) @auto flag and size of the bitmap during
+                         image resize. This mean, that not only bitmap data
+                         may be outdated if @in_use flag set, but also value of
+                         @auto flag and bitmap size (which is indirectly
+                         referenced by @bitmap_table_size).
Quoted 7 lines

Feels wordy. Maybe drop the second paragraph starting with Note, and merely use this for the first paragraph:

The bitmap was not saved correctly and may be inconsistent. Although the bitmap metadata is still well-formed from a qcow2 perspective, the metadata (such as the auto flag or bitmap size) or data contents may be outdated.

Sounds good (as always), thanks)

Quoted 17 lines
Quoted 16 lines
docs/interop/qcow2.txt +2 -2
                        1: auto
                           The bitmap must reflect all changes of the virtual
@@ -717,8 +729,8 @@ corresponding range of the virtual disk (see above) was written to while the
  bitmap was 'enabled'. An unset bit means that this range was not written to.
 
  The software doesn't have to sync the bitmap in the image file with its
-representation in RAM after each write. Flag 'in_use' should be set while the
-bitmap is not synced.
+representation in RAM after each write or metadata change. Flag 'in_use'
+should be set while the bitmap is not synced.
 
  In the image file the 'enabled' state is reflected by the 'auto' flag. If this
  flag is set, the software must consider the bitmap as 'enabled' and start
#4 lore raw

PATCH v2 2/4 block/qcow2-bitmap: Don't check size for IN_USE bitmap

In reply to Vladimir Sementsov-Ogievskiy

We are going to allow image resize when there are persistent bitmaps. It may lead to appearing of inconsistent bitmaps (IN_USE=1) with inconsistent size. But we still want to load them as inconsistent.

---
 block/qcow2-bitmap.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)
block/qcow2-bitmap.c +18 -3
index 6adbe06b4d..141bc1e52c 100644
@@ -462,10 +462,25 @@ static int check_dir_entry(BlockDriverState *bs, Qcow2BitmapDirEntry *entry)
         return len;
     }
 
-    fail = (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) ||
-           (len > ((phys_bitmap_bytes * 8) << entry->granularity_bits));
+    if (phys_bitmap_bytes > BME_MAX_PHYS_SIZE) {
+        return -EINVAL;
+    }
 
-    return fail ? -EINVAL : 0;
+    if (!(entry->flags & BME_FLAG_IN_USE) &&
+        (len > ((phys_bitmap_bytes * 8) << entry->granularity_bits)))
+    {
+        /*
+         * We've loaded valid bitmap (IN_USE not set) or we are going to store
+         * valid bitmap. But allocated bitmap table size is not enough to store
+         * such bitmap.
+         *
+         * Note, that it's OK to have invalid bitmap with invalid size during to
+         * bitmap was not correctly saved after image resize.
+         */
+        return -EINVAL;
+    }
+
+    return 0;
 }
 
 static inline void bitmap_directory_to_be(uint8_t *dir, size_t size)
Signature · 1 line
2.18.0
#5 lore raw

Re: [Qemu-devel] [PATCH v2 2/4] block/qcow2-bitmap: Don't check size for IN_USE bitmap

Quoted 9 lines · Vladimir Sementsov-Ogievskiy

We are going to allow image resize when there are persistent bitmaps. It may lead to appearing of inconsistent bitmaps (IN_USE=1) with inconsistent size. But we still want to load them as inconsistent.

---
 block/qcow2-bitmap.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

block/qcow2-bitmap.c static int check_dir_entry(BlockDriverState *bs, Qcow2BitmapDirEntry *entry) · line 477

Quoted 7 lines
block/qcow2-bitmap.c +7 -0
+    {
+        /*
+         * We've loaded valid bitmap (IN_USE not set) or we are going to store
+         * valid bitmap. But allocated bitmap table size is not enough to store
+         * such bitmap.
+         *
+         * Note, that it's OK to have invalid bitmap with invalid size during to

s/during/due/

Quoted 1 line

+ * bitmap was not correctly saved after image resize.

s/bitmap/a bitmap that/

Quoted 9 lines
block/qcow2-bitmap.c +5 -0
+         */
+        return -EINVAL;
+    }
+
+    return 0;
 }
 
 static inline void bitmap_directory_to_be(uint8_t *dir, size_t size)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#6 lore raw

PATCH v2 3/4 block/qcow2-bitmap: Allow resizes with persistent bitmaps

In reply to Vladimir Sementsov-Ogievskiy

From: John Snow <[email protected]>

Since we now load all bitmaps into memory anyway, we can just truncate them in-memory and then flush them back to disk. Just in case, we will still check and enforce that this shortcut is valid -- i.e. that any bitmap described on-disk is indeed in-memory and can be modified.

If there are any inconsistent bitmaps, we refuse to allow the truncate as we do not actually load these bitmaps into memory, and it isn't safe or reasonable to attempt to truncate corrupted data.

[vsementsov: drop bitmap flushing, fix block comments style]
---
 block/qcow2.h        |  1 +
 block/qcow2-bitmap.c | 46 ++++++++++++++++++++++++++++++++++++++++++++
 block/qcow2.c        |  4 +---
 3 files changed, 48 insertions(+), 3 deletions(-)
block/qcow2.h +1 -0
index 9dd02df831..bbac694a84 100644
@@ -689,6 +689,7 @@ Qcow2BitmapInfoList *qcow2_get_bitmap_info_list(BlockDriverState *bs,
 int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
                                  Error **errp);
 int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
+int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
 void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
 int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
 bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
block/qcow2-bitmap.c +46 -0

52 lines not on this page · raw message

block/qcow2.c +1 -3
index 7fb2730f09..da2697f912 100644
@@ -3499,9 +3499,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset,
     }
 
     /* cannot proceed if image has bitmaps */
-    if (s->nb_bitmaps) {
-        /* TODO: resize bitmaps in the image */
-        error_setg(errp, "Can't resize an image which has bitmaps");
+    if (qcow2_truncate_bitmaps_check(bs, errp)) {
         ret = -ENOTSUP;
         goto fail;
     }
Signature · 1 line
2.18.0
#7 lore raw

PATCH v2 4/4 tests/qemu-iotests: add bitmap resize test 246

In reply to Vladimir Sementsov-Ogievskiy

From: John Snow <[email protected]>

Test that we can actually resize qcow2 images with persistent bitmaps correctly. Throw some other goofy stuff at the test while we're at it, like adding bitmaps of different granularities and at different times.

[vsmentsov: drop \n from the end of test output,
test output changed a bit: some bitmaps goes in other order int the output]

---
 tests/qemu-iotests/246     | 114 ++++++++++++++
 tests/qemu-iotests/246.out | 295 +++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |   1 +
 3 files changed, 410 insertions(+)
 create mode 100755 tests/qemu-iotests/246
 create mode 100644 tests/qemu-iotests/246.out
tests/qemu-iotests/246 new file +114 -0
new file mode 100755
index 0000000000..b0997a392f
@@ -0,0 +1,114 @@
+#!/usr/bin/env python
+#
+# Test persistent bitmap resizing.
+#
+# Copyright (c) 2019 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# [email protected]
+
+import iotests
+from iotests import log
+
+iotests.verify_image_format(supported_fmts=['qcow2'])
+size = 64 * 1024 * 1024 * 1024
+gran_small = 32 * 1024
+gran_large = 128 * 1024
+
+def query_bitmaps(vm):
+    res = vm.qmp("query-block")
+    return { "bitmaps": { device['device']: device.get('dirty-bitmaps', []) for
+                          device in res['return'] } }
+
+with iotests.FilePath('img') as img_path, \
+     iotests.VM() as vm:
+
+    log('--- Preparing image & VM ---\n')
+    iotests.qemu_img_create('-f', iotests.imgfmt, img_path, str(size))
+    vm.add_drive(img_path)
+
+
+    log('--- 1st Boot (Establish Baseline Image) ---\n')
+    vm.launch()
+
+    log('\n--- Adding bitmaps Small, Medium, Large, and Transient ---\n')
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="Small", granularity=gran_small, persistent=True)
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="Medium", persistent=True)
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="Large", granularity=gran_large, persistent=True)
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="Transient", persistent=False)
+
+    log('--- Forcing flush of bitmaps to disk ---\n')
+    log(query_bitmaps(vm), indent=2)
+    vm.shutdown()
+
+
+    log('--- 2nd Boot (Grow Image) ---\n')
+    vm.launch()
+    log(query_bitmaps(vm), indent=2)
+
+    log('--- Adding new bitmap, growing image, and adding 2nd new bitmap ---')
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="New", persistent=True)
+    vm.qmp_log("human-monitor-command",
+               command_line="block_resize drive0 70G")
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="Newtwo", persistent=True)
+    log(query_bitmaps(vm), indent=2)
+
+    log('--- Forcing flush of bitmaps to disk ---\n')
+    vm.shutdown()
+
+
+    log('--- 3rd Boot (Shrink Image) ---\n')
+    vm.launch()
+    log(query_bitmaps(vm), indent=2)
+
+    log('--- Adding "NewB" bitmap, removing "New" bitmap ---')
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="NewB", persistent=True)
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0",
+               name="New")
+
+    log('--- Truncating image ---\n')
+    vm.qmp_log("human-monitor-command",
+               command_line="block_resize drive0 50G")
+
+    log('--- Adding "NewC" bitmap, removing "NewTwo" bitmap ---')
+    vm.qmp_log("block-dirty-bitmap-add", node="drive0",
+               name="NewC", persistent=True)
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0", name="Newtwo")
+
+    log('--- Forcing flush of bitmaps to disk ---\n')
+    vm.shutdown()
+
+
+    log('--- 4th Boot (Verification and Cleanup) ---\n')
+    vm.launch()
+    log(query_bitmaps(vm), indent=2)
+
+    log('--- Removing all Bitmaps ---\n')
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0", name="Small")
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0", name="Medium")
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0", name="Large")
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0", name="NewB")
+    vm.qmp_log("block-dirty-bitmap-remove", node="drive0", name="NewC")
+    log(query_bitmaps(vm), indent=2)
+
+    log('\n--- Done ---')
+    vm.shutdown()
tests/qemu-iotests/246.out new file +295 -0

295 lines not on this page · raw message

tests/qemu-iotests/group +1 -0
index b5ca63cf72..0683f6e10d 100644
@@ -241,3 +241,4 @@
 239 rw auto quick
 240 auto quick
 242 rw auto quick
+246 rw auto quick
Signature · 1 line
2.18.0