block: Fix multiple blockdev-snapshot calls
cover block: Fix multiple blockdev-snapshot calls
Kevin Wolf (2):
block: Remove 'backing': null from bs->{explicit_,}options
iotests: Test multiple blockdev-snapshot calls
block.c | 2 + tests/qemu-iotests/273 | 76 +++++++++ tests/qemu-iotests/273.out | 337 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 4 files changed, 416 insertions(+) create mode 100755 tests/qemu-iotests/273 create mode 100644 tests/qemu-iotests/273.out
Signature · 1 line
2.20.1
PATCH 1/2 block: Remove 'backing': null from bs->{explicit_, }options
bs->options and bs->explicit_options shouldn't contain any options for child nodes. bdrv_open_inherited() takes care to remove any options that match a child name after opening the image and the same is done when reopening.
However, we miss the case of 'backing': null, which is a child option, but results in no child being created. This means that a 'backing': null remains in bs->options and bs->explicit_options.
A typical use for 'backing': null is in live snapshots: blockdev-add for the qcow2 overlay makes sure not to open the backing file (because it is already opened and blockdev-snapshot will attach it). After doing a blockdev-snapshot, bs->options and bs->explicit_options become inconsistent with the actual state (bs has a backing file now, but the options still say null). On the next occasion that the image is reopened, e.g. switching it from read-write to read-only when another snapshot is taken, the option will take effect again and the node incorrectly loses its backing file.
Fix bdrv_open_inherited() to remove the 'backing' option from bs->options and bs->explicit_options even for the case where it specifies that no backing file is wanted.
--- block.c | 2 ++ 1 file changed, 2 insertions(+)
block.c +2 -0
@@ -3019,6 +3019,8 @@ static BlockDriverState *bdrv_open_inherit(const char *filename, "use \"backing\": null instead"); } flags |=BDRV_O_NO_BACKING ; +qdict_del (bs->explicit_options,"backing" ); +qdict_del (bs->options,"backing" );qdict_del (options,"backing" ); }
Signature · 1 line
2.20.1
Re: [PATCH 1/2] block: Remove 'backing': null from bs->{explicit_, }options
Quoted 19 lines · Kevin Wolf
bs->optionsandbs->explicit_optionsshouldn't contain any options for child nodes.bdrv_open_inherited()takes care to remove any options that match a child name after opening the image and the same is done when reopening.However, we miss the case of 'backing': null, which is a child option, but results in no child being created. This means that a 'backing': null remains in
bs->optionsandbs->explicit_options.A typical use for 'backing': null is in live snapshots: blockdev-add for the qcow2 overlay makes sure not to open the backing file (because it is already opened and blockdev-snapshot will attach it). After doing a blockdev-snapshot,
bs->optionsandbs->explicit_optionsbecome inconsistent with the actual state (bs has a backing file now, but the options still say null). On the next occasion that the image is reopened, e.g. switching it from read-write to read-only when another snapshot is taken, the option will take effect again and the node incorrectly loses its backing file.
Quoted 6 lines
Fix
bdrv_open_inherited()to remove the 'backing' option frombs->optionsandbs->explicit_optionseven for the case where it specifies that no backing file is wanted.Reported-by Peter Krempa <[email protected]>Signed-off-by Kevin Wolf <[email protected]>
Berto
Re: [PATCH 1/2] block: Remove 'backing': null from bs->{explicit_,}options
In reply to Kevin WolfQuoted 11 lines · Kevin Wolf
bs->optionsandbs->explicit_optionsshouldn't contain any options for child nodes.bdrv_open_inherited()takes care to remove any options that match a child name after opening the image and the same is done when reopening.However, we miss the case of 'backing': null, which is a child option, but results in no child being created. This means that a 'backing': null remains in
bs->optionsandbs->explicit_options.A typical use for 'backing': null is in live snapshots: blockdev-add for the qcow2 overlay makes sure not to open the backing file (because it is
Note that we also use '"backing": null' as a terminator for the last image in the chain if the user configures the chain manually.
This is kind-of a protection from opening the backing file from the header if it was misconfigured somehow. I think this functionality should be kept despite probably not making practical sense.
In my testing this scenario worked properly.
Quoted 8 lines
already opened and blockdev-snapshot will attach it). After doing a blockdev-snapshot,
bs->optionsandbs->explicit_optionsbecome inconsistent with the actual state (bs has a backing file now, but the options still say null). On the next occasion that the image is reopened, e.g. switching it from read-write to read-only when another snapshot is taken, the option will take effect again and the node incorrectly loses its backing file.
Quoted 6 lines
Fix
bdrv_open_inherited()to remove the 'backing' option frombs->optionsandbs->explicit_optionseven for the case where it specifies that no backing file is wanted.Reported-by Peter Krempa <[email protected]>Signed-off-by Kevin Wolf <[email protected]>
The fix looks sane-enough to me and works as expected, but since I'm not familiar enough with this code I'm comfortable only with a:
Quoted 3 lines
--- block.c | 2 ++ 1 file changed, 2 insertions(+)
PATCH 2/2 iotests: Test multiple blockdev-snapshot calls
In reply to Kevin WolfTest that doing a second blockdev-snapshot doesn't make the first overlay's backing file go away.
--- tests/qemu-iotests/273 | 76 +++++++++ tests/qemu-iotests/273.out | 337 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 414 insertions(+) create mode 100755 tests/qemu-iotests/273 create mode 100644 tests/qemu-iotests/273.out
tests/qemu-iotests/273 new file +76 -0
@@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# +# Test large write to a qcow2 image +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +seq=$(basename "$0") +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# This is a qcow2 regression test +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +do_run_qemu() +{ + echo Testing: "$@" + $QEMU -nographic -qmp-pretty stdio -nodefaults "$@" + echo +} + +run_qemu() +{ + do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qemu | _filter_qmp | + _filter_generated_node_ids | _filter_imgfmt | _filter_actual_image_size +} + +TEST_IMG="$TEST_IMG.base" _make_test_img 64M +TEST_IMG="$TEST_IMG.mid" _make_test_img -b "$TEST_IMG.base" +_make_test_img -b "$TEST_IMG.mid" + +run_qemu \ + -blockdev file,node-name=base,filename="$TEST_IMG.base" \ + -blockdev file,node-name=midf,filename="$TEST_IMG.mid" \ + -blockdev '{"driver":"qcow2","node-name":"mid","file":"midf","backing":null}' \ + -blockdev file,node-name=topf,filename="$TEST_IMG" \ + -blockdev '{"driver":"qcow2","file":"topf","node-name":"top","backing":null}' \ +<<EOF +{"execute":"qmp_capabilities"} +{"execute":"blockdev-snapshot","arguments":{"node":"base","overlay":"mid"}} +{"execute":"blockdev-snapshot","arguments":{"node":"mid","overlay":"top"}} +{"execute":"query-named-block-nodes"} +{"execute":"x-debug-query-block-graph"} +{"execute":"quit"} +EOF + +# success, all done +echo "*** done" +rm -f $seq.full +status=0
tests/qemu-iotests/273.out new file +337 -0
337 lines not on this page · raw message
tests/qemu-iotests/group +1 -0
@@ -282,3 +282,4 @@ 267 rw auto quick snapshot 268 rw auto quick 270 rw backing quick +273 backing quick
Signature · 1 line
2.20.1
Re: [PATCH 2/2] iotests: Test multiple blockdev-snapshot calls
Quoted 1 line · Kevin Wolf
+# Test large write to a qcow2 image
This doesn't belong here I guess :)
I wonder if this test could go in 245 instead.
Berto
Re: [PATCH 2/2] iotests: Test multiple blockdev-snapshot calls
Am 08.11.2019 um 15:32 hat Alberto Garcia geschrieben:
Quoted 4 lines · Kevin Wolf
Quoted 1 line
+# Test large write to a qcow2 image
This doesn't belong here I guess :)
Yes, fixed.
Quoted 1 line
I wonder if this test could go in 245 instead.
The headline for 245 is "Test cases for the QMP 'x-blockdev-reopen' command", so I don't think so?
We end up testing the reopen code, but only as a side effect of snapshotting, not through x-blockdev-reopen.
Kevin
Re: [PATCH 2/2] iotests: Test multiple blockdev-snapshot calls
In reply to Kevin WolfQuoted 11 lines · Kevin Wolf
Test that doing a second blockdev-snapshot doesn't make the first overlay's backing file go away.
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/273 | 76 +++++++++ tests/qemu-iotests/273.out | 337 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 414 insertions(+) create mode 100755 tests/qemu-iotests/273 create mode 100644 tests/qemu-iotests/273.out
Didn't apply cleanly for me.
tests/qemu-iotests/273 line 3
Quoted 10 lines
tests/qemu-iotests/273 new file +3
-0@@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# +# Test large write to a qcow2 image
Cut&paste?
Rest looks good
Quoted 31 lines
tests/qemu-iotests/273 new file +31
-0+# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +seq=$(basename "$0") +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter
tests/qemu-iotests/273 line 44
Quoted 10 lines
tests/qemu-iotests/273 new file +10
-0+ +# This is a qcow2 regression test +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +do_run_qemu() +{ + echo Testing: "$@" + $QEMU -nographic -qmp-pretty stdio -nodefaults "$@"
-qmp-pretty, that's useful
Quoted 15 lines
tests/qemu-iotests/273 new file +15
-0+ echo +} + +run_qemu() +{ + do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qemu | _filter_qmp | + _filter_generated_node_ids | _filter_imgfmt | _filter_actual_image_size +} + +TEST_IMG="$TEST_IMG.base" _make_test_img 64M +TEST_IMG="$TEST_IMG.mid" _make_test_img -b "$TEST_IMG.base" +_make_test_img -b "$TEST_IMG.mid" + +run_qemu \ + -blockdev file,node-name=base,filename="$TEST_IMG.base" \
tests/qemu-iotests/273 line 69
Quoted 10 lines
tests/qemu-iotests/273 new file +10
-0+ -blockdev file,node-name=midf,filename="$TEST_IMG.mid" \ + -blockdev '{"driver":"qcow2","node-name":"mid","file":"midf","backing":null}' \ + -blockdev file,node-name=topf,filename="$TEST_IMG" \ + -blockdev '{"driver":"qcow2","file":"topf","node-name":"top","backing":null}' \ +<<EOF +{"execute":"qmp_capabilities"} +{"execute":"blockdev-snapshot","arguments":{"node":"base","overlay":"mid"}} +{"execute":"blockdev-snapshot","arguments":{"node":"mid","overlay":"top"}} +{"execute":"query-named-block-nodes"} +{"execute":"x-debug-query-block-graph"}
Oh, this too!
Quoted 7 lines
tests/qemu-iotests/273 new file +7
-0+{"execute":"quit"} +EOF + +# success, all done +echo "*** done" +rm -f $seq.full +status=0
Re: [PATCH 2/2] iotests: Test multiple blockdev-snapshot calls
Am 12.11.2019 um 17:07 hat Peter Krempa geschrieben:
Quoted 16 lines · Kevin Wolf
Quoted 11 lines
Test that doing a second blockdev-snapshot doesn't make the first overlay's backing file go away.
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/273 | 76 +++++++++ tests/qemu-iotests/273.out | 337 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 414 insertions(+) create mode 100755 tests/qemu-iotests/273 create mode 100644 tests/qemu-iotests/273.outDidn't apply cleanly for me.
Quoted 1 line
Quoted 11 lines
tests/qemu-iotests/273 line 3
Quoted 9 lines
tests/qemu-iotests/273 new file +3
-0@@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# +# Test large write to a qcow2 imageCut&paste?
Indeed. I'll change it like this:
# Test multiple blockdev-snapshot calls with 'backing': null
Quoted 3 lines
Rest looks good
Reviewed-by Peter Krempa <[email protected]>
Thanks!
Kevin
Re: [PATCH 2/2] iotests: Test multiple blockdev-snapshot calls
In reply to Kevin WolfQuoted 11 lines · Kevin Wolf
Test that doing a second blockdev-snapshot doesn't make the first overlay's backing file go away.
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/273 | 76 +++++++++ tests/qemu-iotests/273.out | 337 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 414 insertions(+) create mode 100755 tests/qemu-iotests/273 create mode 100644 tests/qemu-iotests/273.out
[...]
tests/qemu-iotests/273.out line 48
Quoted 8 lines
tests/qemu-iotests/273.out new file +8
-0+ "format-specific": { + "type": "IMGFMT", + "data": { + "compat": "1.1", + "lazy-refcounts": false, + "refcount-bits": 16, + "corrupt": false + }
I get a mismatch here with -o compat=0.10 or -o refcount_bits=1.
Max
Quoted 3 lines · Kevin Wolf
Kevin Wolf (2):
block: Remove 'backing': null from bs->{explicit_,}options
iotests: Test multiple blockdev-snapshot calls
Hi,
I'm not sure how the freeze rules for qemu are at this point thus:
Will this patchset make it into 4.2. I argue it's a pretty important fix as it ends up in image corruption.
If it won't make into we will need to add a QMP feature flag to detect presence of the fix so that I can gate libvirt's support of blockdev with it. (This can also be done if we make it into 4.2 but I'll be okay infering it from other fixes present in 4.2).
Thanks
Peter
Am 18.11.2019 um 17:26 hat Peter Krempa geschrieben:
Quoted 9 lines · Kevin Wolf
Quoted 3 lines
Kevin Wolf (2):
block: Remove 'backing': null from bs->{explicit_,}options
iotests: Test multiple blockdev-snapshot callsHi,
I'm not sure how the freeze rules for qemu are at this point thus:
Quoted 7 lines
Will this patchset make it into 4.2. I argue it's a pretty important fix as it ends up in image corruption.
If it won't make into we will need to add a QMP feature flag to detect presence of the fix so that I can gate libvirt's support of blockdev with it. (This can also be done if we make it into 4.2 but I'll be okay infering it from other fixes present in 4.2).
Yes, this is planned for 4.2. I'm in the process of preparing a pull request right now.
Kevin