qemu-img: Add convert --bitmaps
cover qemu-img: Add convert --bitmaps
Without this series, the process for copying one qcow2 image to another including all of its bitmaps involves running qemu and doing the copying by hand with a series of QMP commands. This makes the process a bit more convenient.
I still think that someday we will need a 'qemu-img bitmap' with various subcommands for manipulating bitmaps within an offline image, but in the meantime, this seems like a useful addition on its own.
Series can also be downloaded at:
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/qemu-img-bitmaps-v1
Eric Blake (3):
blockdev: Split off basic bitmap operations for qemu-img
qemu-img: Add convert --bitmaps option
iotests: Add test 291 to for qemu-img convert --bitmaps
docs/tools/qemu-img.rst | 6 +- Makefile.objs | 2 +- include/sysemu/blockdev.h | 10 ++ blockbitmaps.c | 217 +++++++++++++++++++++++++++++++++++++ blockdev.c | 184 ------------------------------- qemu-img.c | 81 +++++++++++++- MAINTAINERS | 1 + qemu-img-cmds.hx | 4 +- tests/qemu-iotests/291 | 143 ++++++++++++++++++++++++ tests/qemu-iotests/291.out | 56 ++++++++++ tests/qemu-iotests/group | 1 + 11 files changed, 514 insertions(+), 191 deletions(-) create mode 100644 blockbitmaps.c create mode 100755 tests/qemu-iotests/291 create mode 100644 tests/qemu-iotests/291.out
Signature · 1 line
2.26.0
PATCH 1/3 blockdev: Split off basic bitmap operations for qemu-img
The next patch wants to teach qemu how to copy a bitmap from one qcow2 file to another. But blockdev.o is too heavyweight to link into qemu-img, so it's time to split off the bare bones of what we will need into a new file blockbitmaps.o. Transactions are not needed in qemu-img (if things fail while creating the new image, the fix is to delete the botched copy, rather than worrying about atomic rollback).
For now, I stuck to just the minimum code motion (add and merge); we could instead decide to move everything bitmap-related that does not also pull in transactions (delete, enable, disable).
--- Makefile.objs | 2 +- include/sysemu/blockdev.h | 10 ++ blockbitmaps.c | 217 ++++++++++++++++++++++++++++++++++++++ blockdev.c | 184 -------------------------------- MAINTAINERS | 1 + 5 files changed, 229 insertions(+), 185 deletions(-) create mode 100644 blockbitmaps.c
Makefile.objs +1 -1
@@ -14,7 +14,7 @@ chardev-obj-y = chardev/ authz-obj-y = authz/ block-obj-y = nbd/ -block-obj-y += block.o blockjob.o job.o +block-obj-y += block.o blockbitmaps.o blockjob.o job.o block-obj-y += block/ scsi/ block-obj-y += qemu-io-cmds.o block-obj-$(CONFIG_REPLICATION) += replication.o
include/sysemu/blockdev.h +10 -0
@@ -57,4 +57,14 @@ QemuOpts *drive_add(BlockInterfaceType type, int index, const char *file,DriveInfo *drive_new (QemuOpts *arg,BlockInterfaceType block_default_type,Error **errp); +BdrvDirtyBitmap *block_dirty_bitmap_lookup (const char *node, +const char *name, +BlockDriverState **pbs, +Error **errp); +BdrvDirtyBitmap *do_block_dirty_bitmap_merge (const char *node, +const char *target, +BlockDirtyBitmapMergeSourceList *bitmaps, +HBitmap **backup,Error **errp); + +#endif
blockbitmaps.c new file +217 -0
217 lines not on this page · raw message
blockdev.c +0 -184
208 lines not on this page · raw message
MAINTAINERS +1 -0
@@ -1989,6 +1989,7 @@ T: git https://github.com/jnsnow/qemu.git jobs Block QAPI, monitor, command line M: Markus Armbruster <[email protected]> S: Supported +F: blockbitmaps.c F: blockdev.c F: blockdev-hmp-cmds.c F: block/qapi.c
Signature · 1 line
2.26.0
PATCH 2/3 qemu-img: Add convert --bitmaps option
In reply to Eric BlakeMake it easier to copy all the persistent bitmaps of a source image along with the contents, by adding a boolean flag for use with qemu-img convert.
See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893
--- docs/tools/qemu-img.rst | 6 ++- qemu-img.c | 81 +++++++++++++++++++++++++++++++++++++++-- qemu-img-cmds.hx | 4 +- 3 files changed, 85 insertions(+), 6 deletions(-)
docs/tools/qemu-img.rst +5 -1
@@ -186,6 +186,10 @@ Parameters to convert subcommand: .. program:: qemu-img-convert +.. option:: --bitmaps + + Additionally copy all bitmaps + .. option:: -n Skip the creation of the target volume
@@ -373,7 +377,7 @@ Command description: 4 Error on reading data -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM* to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can
qemu-img.c +78 -3
148 lines not on this page · raw message
qemu-img-cmds.hx +2 -2
@@ -39,9 +39,9 @@ SRST ERST DEF("convert", img_convert, - "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") + "convert [--object objectdef] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] [--salvage] filename [filename2 [...]] output_filename") SRST -.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME +.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] [--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-m NUM_COROUTINES] [-W] [--salvage] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME ERST DEF("create", img_create,
Signature · 1 line
2.26.0
PATCH 3/3 iotests: Add test 291 to for qemu-img convert --bitmaps
In reply to Eric BlakeAdd a new test covering the feature added in the previous patch.
--- tests/qemu-iotests/291 | 143 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/291.out | 56 +++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 200 insertions(+) create mode 100755 tests/qemu-iotests/291 create mode 100644 tests/qemu-iotests/291.out
tests/qemu-iotests/291 new file +143 -0
@@ -0,0 +1,143 @@ +#!/usr/bin/env bash +# +# Test qemu-img convert --bitmaps +# +# Copyright (C) 2018-2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +seq="$(basename $0)" +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + nbd_server_stop +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.nbd + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux +_require_command QEMU_NBD + +do_run_qemu() +{ + echo Testing: "$@" + $QEMU -nographic -qmp stdio -serial none "$@" + echo +} + +run_qemu() +{ + do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ + | _filter_qemu | _filter_imgfmt \ + | _filter_actual_image_size | _filter_qemu_io +} + +# Create initial image and populate two bitmaps: one active, one inactive +_make_test_img 10M +run_qemu <<EOF +{ "execute": "qmp_capabilities" } +{ "execute": "blockdev-add", + "arguments": { + "driver": "$IMGFMT", + "node-name": "n", + "file": { + "driver": "file", + "filename": "$TEST_IMG" + } + } +} +{ "execute": "human-monitor-command", + "arguments": { + "command-line": 'qemu-io n "write 0 1M"' + } +} +{ "execute": "block-dirty-bitmap-add", + "arguments": { + "node": "n", + "name": "b1", + "persistent": true, + "granularity": 65536 + } +} +{ "execute": "human-monitor-command", + "arguments": { + "command-line": 'qemu-io n "write 1M 1M"' + } +} +{ "execute": "transaction", + "arguments": { + "actions": [ + { "type": "block-dirty-bitmap-disable", + "data": { + "node": "n", + "name": "b1" + } }, + { "type": "block-dirty-bitmap-add", + "data": { + "node": "n", + "name": "b2", + "persistent": true + } } + ] + } +} +{ "execute": "human-monitor-command", + "arguments": { + "command-line": 'qemu-io n "write 2M 1M"' + } +} +{ "execute": "quit" } +EOF + +echo +echo "=== Bitmap preservation not possible to non-qcow2 ===" +echo + +mv "$TEST_IMG" "$TEST_IMG.orig" +$QEMU_IMG convert --bitmaps -O raw "$TEST_IMG.orig" "$TEST_IMG" + +echo +echo "=== convert with bitmap preservation ===" +echo + +$QEMU_IMG convert --bitmaps -O qcow2 "$TEST_IMG.orig" "$TEST_IMG" +$QEMU_IMG info "$TEST_IMG" | _filter_img_info --format-specific + +echo +echo "=== check bitmap contents ===" +echo + +IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket" +nbd_server_start_unix_socket -r -f qcow2 -B b1 "$TEST_IMG" +$QEMU_IMG map --output=json --image-opts \ + "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b1" | _filter_qemu_img_map +nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG" +$QEMU_IMG map --output=json --image-opts \ + "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map + +# success, all done +echo '*** done' +rm -f $seq.full +status=0
tests/qemu-iotests/291.out new file +56 -0
56 lines not on this page · raw message
tests/qemu-iotests/group +1 -0
@@ -297,3 +297,4 @@ 288 quick 289 rw quick 290 rw auto quick +291 rw quick
Signature · 1 line
2.26.0
Quoted 5 lines · Eric Blake
Without this series, the process for copying one qcow2 image to another including all of its bitmaps involves running qemu and doing the copying by hand with a series of QMP commands. This makes the process a bit more convenient.
This seems good for copying an image chain from one storage to another, but I think we need a similar --bitmaps option to qemu-img measure to make this really useful.
Here is example use case showing how qemu-img measure is related:
Source chain:/dev/vg1/base/dev/vg1/top
Destination chain:/dev/vg2/base/dev/vg2/top
We create empty lvs with the same name on destination storage (/dev/vg2).
We measure the base lv using qemu-img measure for creating the target lv:
qemu-img measure -f qcow2 -O qcow2 /dev/vg1/base lvcreate -L required_size /dev/vg2/base qemu-img create -f qcow2 /dev/vg2/base 10g
For the top lv we use the current size of the source lv - I think we should measure it instead but
I'm not sure if qemu-img measure supports measuring a single image in a chain (maybe -o backing_file?).
lvcreate -L current_size /dev/vg2/top
qemu-img create -f qcow2 -b /dev/vg2/base -F qcow2 /dev/vg2/top 10g
And then convert the lvs one by one:
qemu-img convert -f qcow2 -O qcow2 -n --bitmaps /dev/vg1/base /dev/vg2/base
qemu-img convert -f qcow2 -O qcow2 -n --bitmaps -B /dev/vg2/base/dev/vg1/top /dev/vg2/top
The first copy may fail with ENOSPC since qemu-img measure of the base does not consider the
bitmaps in the required size.
So I think we need to add a similar --bitmaps option to qemu-img measure, hopefully reusing the
same code to find and estimate the size of the bitmaps.
Maybe we can estimate the size using qemu-img info --bitmaps, but I think the right way to
do this is in qemu-img measure.
We have also another use case when we collapsed an image chain to single image:
Source chain:/dev/vg1/base/dev/vg1/top
Destination:/dev/vg2/collapsed
In this case we measure the size of the entire chain (/dev/vg1/base <- /dev/vg1/top) and create/dev/vg2/collapsed in the correct size, and then we convert the chain using:
qemu-img convert /dev/vg1/top /dev/vg2/collapsed
Currently we use this for exporting images, for example when creating templates, or as a simple
backup. In this case we don't need to copy the bitmaps in the target image - this is a new image
not used by any VM. Copying the bitmaps may also be non-trivial since we may have the bitmaps
with the same names in several layers (e.g. result of live snapshot).
So I think using --bitmaps should be disabled when doing this kind of convert. We can handle this
on our side easily, but I think this should fail or log a warning on qemu-img, or require merging of
bitmaps with same names during the copy. I did not check if you already handle this.
Finally we also have a use case when we copy the chain as is to new or same storage, but
we create a new vm. In this case I don't think the backup history makes sense for the new
vm, so we don't need to copy the bitmaps.
I will review the rest of the patches next week and can maybe give this some testing.
Nir
Quoted 31 lines
I still think that someday we will need a 'qemu-img bitmap' with various subcommands for manipulating bitmaps within an offline image, but in the meantime, this seems like a useful addition on its own.
Series can also be downloaded at:
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/qemu-img-bitmaps-v1Eric Blake (3):
blockdev: Split off basic bitmap operations for qemu-img
qemu-img: Add convert --bitmaps option
iotests: Add test 291 to for qemu-img convert --bitmapsdocs/tools/qemu-img.rst | 6 +- Makefile.objs | 2 +- include/sysemu/blockdev.h | 10 ++ blockbitmaps.c | 217 +++++++++++++++++++++++++++++++++++++ blockdev.c | 184 ------------------------------- qemu-img.c | 81 +++++++++++++- MAINTAINERS | 1 + qemu-img-cmds.hx | 4 +- tests/qemu-iotests/291 | 143 ++++++++++++++++++++++++ tests/qemu-iotests/291.out | 56 ++++++++++ tests/qemu-iotests/group | 1 + 11 files changed, 514 insertions(+), 191 deletions(-) create mode 100644 blockbitmaps.c create mode 100755 tests/qemu-iotests/291 create mode 100644 tests/qemu-iotests/291.out--
2.26.0
(adding Markus for a CLI question, look for [*])
Quoted 25 lines · Nir Soffer
Quoted 5 lines · Eric Blake
Without this series, the process for copying one qcow2 image to another including all of its bitmaps involves running qemu and doing the copying by hand with a series of QMP commands. This makes the process a bit more convenient.
This seems good for copying an image chain from one storage to another, but I think we need a similar --bitmaps option to qemu-img measure to make this really useful.
Here is example use case showing how qemu-img measure is related:
Source chain:
/dev/vg1/base/dev/vg1/topDestination chain:
/dev/vg2/base/dev/vg2/topWe create empty lvs with the same name on destination storage (
/dev/vg2).We measure the base lv using qemu-img measure for creating the target lv:
Quoted 8 lines
qemu-img measure -f qcow2 -O qcow2
/dev/vg1/baselvcreate -Lrequired_size/dev/vg2/baseqemu-img create -f qcow2/dev/vg2/base10gFor the top lv we use the current size of the source lv - I think we should measure it instead but
I'm not sure if qemu-img measure supports measuring a single image in a chain (maybe -obacking_file?).
qemu-measure --image-opts should be able to measure a single image by specifying image opts that purposefully treat the image as standalone rather than with its normal backing file included. Let's see if I can whip up an example:
$ qemu-img create -f qcow2 img.base 100M Formatting 'img.base', fmt=qcow2 size=104857600 cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ qemu-io -f qcow2 -c 'w 0 25m' img.base wrote 26214400/26214400 bytes at offset 0 25 MiB, 1 ops; 00.24 sec (103.405 MiB/sec and 4.1362 ops/sec) $ qemu-img create -f qcow2 -F qcow2 -b img.base img.top Formatting 'img.top', fmt=qcow2 size=104857600 backing_file=img.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16 $ qemu-io -f qcow2 -c 'w 25m 25m' img.top wrote 26214400/26214400 bytes at offset 26214400 25 MiB, 1 ops; 00.24 sec (103.116 MiB/sec and 4.1247 ops/sec) $ qemu-img measure -f qcow2 -O qcow2 img.base required size: 26542080 fully allocated size: 105185280 required size: 52756480 fully allocated size: 105185280
Okay, I can reproduce what you are seeing - measuring the top image defaults to measuring the full allocation of the entire chain, rather than the allocation of just the top image. And now with --image-opts to the rescue:
$ qemu-img measure --image-opts -O qcow2 driver=qcow2,backing=,file.driver=file,file.filename=img.top qemu-img: warning: Use of "backing": "" is deprecated; use "backing": null instead required size: 26542080 fully allocated size: 105185280
There you go - by forcing qemu to treat the overlay image as though it had no backing, you can then measure that image in isolation.
(*) Hmm - that warning about backing="" being deprecated is annoying, but I don't know any other way to use dotted command line syntax and still express that we want a QMP null. I tried to see if I could inject an alternative backing driver, such as null-co, but was met with errors:
$ ./qemu-img measure --image-opts -O qcow2 driver=qcow2,backing.driver=null-co,file.driver=file,file.filename=img.top qemu-img: Could not open 'driver=qcow2,backing.driver=null-co,file.driver=file,file.filename=img.top': Could not open backing file: The only allowed filename for this driver is 'null-co://' $ ./qemu-img measure --image-opts -O qcow2 driver=qcow2,backing.driver=null-co,backing.file=null-co://,file.driver=file,file.filename=img.top qemu-img: Could not open 'driver=qcow2,backing.driver=null-co,backing.file=null-co://,file.driver=file,file.filename=img.top': Could not open backing file: The only allowed filename for this driver is 'null-co://' $ ./qemu-img measure --image-opts -O qcow2 driver=qcow2,backing.driver=null-co,backing.file.filename=null-co://,file.driver=file,file.filename=img.top qemu-img: Could not open 'driver=qcow2,backing.driver=null-co,backing.file.filename=null-co://,file.driver=file,file.filename=img.top': Could not open backing file: Block protocol 'null-co' doesn't support the option 'file.filename'
We don't want to support "" in the QMP syntax forever, but if the CLI syntax has to handle the empty string specially in order to get null passed to the QMP code, then so be it.
I also tried, but failed, to use JSON syntax. I don't know why we haven't wired up --image-opts to use JSON syntax yet.
$ qemu-img measure --image-opts -O qcow2 '{"driver":"qcow2", "backing":null,
"file":{"driver":"file", "filename":"img.top"}}'
qemu-img: Could not open '{"driver":"qcow2", "backing":null,
"file":{"driver":"file", "filename":"img.top"}}': Cannot find
device={"driver":"qcow2" nor node_name={"driver":"qcow2"I guess there's always the pseudo-json protocol:
$ qemu-img measure -O qcow2 'json:{"driver":"qcow2", "backing":null,
"file":{"driver":"file", "filename":"img.top"}}'
required size: 26542080
fully allocated size: 105185280Quoted 6 lines
lvcreate -L
current_size/dev/vg2/top
qemu-img create -f qcow2 -b/dev/vg2/base-F qcow2/dev/vg2/top10gAnd then convert the lvs one by one:
Quoted 7 lines
qemu-img convert -f qcow2 -O qcow2 -n --bitmaps /dev/vg1/base /dev/vg2/base qemu-img convert -f qcow2 -O qcow2 -n --bitmaps -B /dev/vg2/base
/dev/vg1/top/dev/vg2/topThe first copy may fail with ENOSPC since qemu-img measure of the base does not consider the
bitmaps in the required size.
Yes, that's a good argument for adding 'qemu-img measure --bitmaps'. In the meantime...
Quoted 6 lines
So I think we need to add a similar --bitmaps option to qemu-img measure, hopefully reusing the
same code to find and estimate the size of the bitmaps.Maybe we can estimate the size using qemu-img info --bitmaps,
...you are correct, this works today. Well, 'qemu-img info --bitmaps' doesn't exist, but 'qemu-img info' does output the number of bitmaps, as well as their granularity, and if you also assume that each bitmap is sized to match the virtual image size, you can compute the estimated space occupied by those bitmaps.
Quoted 3 lines
but I
think the right way to
do this is in qemu-img measure.
Yes, even though we have existing multi-step code, being able to do it in a single step is justification for the improvement (although you may still end up having to code the multi-step mode yourself if you can't guarantee new-enough qemu-img - the addition of 'qemu-img measure --bitmaps' won't land until at least 4 months from now with qemu 5.1). Not to mention that the estimation computation (image size / granularity rounded up to the next cluster size, summed over all bitmaps) is hairy enough that it shouldn't have to be reimplemented by multiple layers of software.
Quoted 7 lines
We have also another use case when we collapsed an image chain to single image:
Source chain:
/dev/vg1/base/dev/vg1/top
Quoted 8 lines
Destination:
/dev/vg2/collapsedIn this case we measure the size of the entire chain (
/dev/vg1/base<-/dev/vg1/top) and create/dev/vg2/collapsedin the correct size, and then we convert the chain using:qemu-img convert
/dev/vg1/top/dev/vg2/collapsed
When I submitted my v1 patch, I had only tested with a single source image to a single destination. But now that you mention flattening, it's easy enough for me to test of what happens (tl;dr: my patch only looks at bitmaps in the active layer of the source):
$ qemu-img create -f qcow2 img.base 100M
Formatting 'img.base', fmt=qcow2 size=104857600 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-img create -f qcow2 -F qcow2 -b img.base img.top
Formatting 'img.top', fmt=qcow2 size=104857600 backing_file=img.base backing_fmt=qcow2 cluster_size=65536 lazy_refcounts=off refcount_bits=16
$ qemu-kvm --nographic --nodefaults --qmp stdio
{'execute':'qmp_capabilities'}
{'execute':'blockdev-add','arguments':{'driver':'qcow2',
'node-name':'base','file':{'driver':'file','filename':'img.base'}}}
{'execute':'blockdev-add','arguments':{'driver':'qcow2',
'node-name':'top','file':{'driver':'file','filename':'img.top'},
'backing':'base'}}
{'execute':'block-dirty-bitmap-add','arguments':{'node':'base',
'name':'b1','persistent':true}}
{'execute':'block-dirty-bitmap-add','arguments':{'node':'base',
'name':'b2','persistent':true}}
{'execute':'block-dirty-bitmap-add','arguments':{'node':'top',
'name':'b2','persistent':true}}
{'execute':'block-dirty-bitmap-add','arguments':{'node':'top',
'name':'b3','persistent':true}}
{'execute':'quit'}
$ qemu-img convert -f qcow2 -O qcow2 --bitmaps img.top img.flat
$ qemu-img info img.flat
image: img.flat
file format: qcow2
virtual size: 100 MiB (104857600 bytes)
disk size: 208 KiB
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
bitmaps:
[0]:
flags:
[0]: auto
name: b3
granularity: 65536
[1]:
flags:
[0]: auto
name: b2
granularity: 65536
refcount bits: 16
corrupt: falseIt only copied the bitmaps from the active layer, not all bitmaps from all layers. If we want more complicated handling (such as whether to pull in bitmaps deeper in the backing chain, rename bitmaps, policies on merging duplicated bitmap names across nodes vs. failure, etc), we really should be designing 'qemu-img bitmap' that gives full command-line control over all sorts of bitmap operations, rather than trying to further overload 'qemu-img convert'.
Quoted 9 lines
Currently we use this for exporting images, for example when creating templates, or as a simple
backup. In this case we don't need to copy the bitmaps in the target image - this is a new image
not used by any VM. Copying the bitmaps may also be non-trivial since we may have the bitmaps
with the same names in several layers (e.g. result of live snapshot).
Quoted 6 lines
So I think using --bitmaps should be disabled when doing this kind of convert. We can handle this
on our side easily, but I think this should fail or log a warning on qemu-img, or require merging of
bitmaps with same names during the copy. I did not check if you already handle this.
My patch only copies top-most bitmaps. If you are using qemu-img convert to flatten a chain, we'll need something else to control what to do with bitmaps across that chain.
Quoted 10 lines
Finally we also have a use case when we copy the chain as is to new or same storage, but
we create a new vm. In this case I don't think the backup history makes sense for the new
vm, so we don't need to copy the bitmaps.I will review the rest of the patches next week and can maybe give this some testing.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org