block/monitor: force read-only on live-added persistent bitmap
PATCH 1/1 block/monitor: force read-only on live-added persistent bitmap
qmp_block_dirty_bitmap_add() marks a new bitmap persistent without checking write access to its node. bdrv_create_dirty_bitmap() always creates bitmaps writable, so a persistent bitmap added live to an already read-only node stays writable in memory on a node that can never store it. The next global inactivation (migration completion, save, or any other bdrv_inactivate_all() caller) then fails trying to flush it back:
Lost persistent bitmaps during inactivation of node '<node>': No write access migration_block_inactivate: bdrv_inactivate_all() failed: -22
qcow2_load_dirty_bitmaps() and qcow2_reopen_bitmaps_ro() already force a persistent bitmap read-only when its node is read-only; do the same right after the bitmap is marked persistent, whether it arrived through a plain block-dirty-bitmap-add or the transaction action that calls the same function. qcow2_reopen_bitmaps_rw() already promotes any persistent, read-only bitmap back to writable once its node is reopened read-write, so the reverse direction needs no changes.
Also check BDRV_O_INACTIVE, matching qcow2's private can_write() helper, to cover adding a bitmap to a node that is already inactive. Document the new behavior on BlockDirtyBitmapAdd.persistent.
Extend remove-bitmap-from-backing to cover this: add a persistent bitmap live to the already read-only "base" node, then inactivate all nodes the way migration completion, virsh save and qemu-update do, and check the bitmap ends up as protected as one already on disk.
--- block/monitor/bitmap-qmp-cmds.c | 9 +++++++++ qapi/block-core.json | 4 +++- .../tests/remove-bitmap-from-backing | 16 ++++++++++++++++ .../tests/remove-bitmap-from-backing.out | 11 +++++++++++ 4 files changed, 39 insertions(+), 1 deletion(-)
block/monitor/bitmap-qmp-cmds.c +9 -0
@@ -141,6 +141,15 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, }bdrv_dirty_bitmap_set_persistence (bitmap, persistent); + +/* +* A persistent bitmap on a node that can't be written must not be +* writable either, same as one loaded from or reopened onto it. +*/ +if (persistent && + (bdrv_is_read_only (bs) || (bdrv_get_flags (bs) &BDRV_O_INACTIVE ))) { +bdrv_dirty_bitmap_set_readonly (bitmap, true); + } } BdrvDirtyBitmap *block_dirty_bitmap_remove(const char *node, const char *name,
qapi/block-core.json +3 -1
@@ -2353,7 +2353,9 @@ # @persistent: the bitmap is persistent, i.e. it will be saved to the # corresponding block device image file on its close. For now # only Qcow2 disks support persistent bitmaps. Default is false -# for `block-dirty-bitmap-add`. (Since: 2.10) +# for `block-dirty-bitmap-add`. If the node is read-only, the +# bitmap is created read-only instead, the same as a persistent +# bitmap already stored on such a node. (Since: 2.10) # # @disabled: the bitmap is created in the disabled state, which means # that it will not track drive changes. The bitmap may be enabled
tests/qemu-iotests/tests/remove-bitmap-from-backing +16 -0
@@ -66,6 +66,22 @@ result = vm.qmp('blockdev-reopen', **new_base_opts) if result != {'return': {}}: log('Failed to reopen: ' + str(result)) +# A bitmap added live (not loaded from disk) must be protected the same +# way as bitmap0 above once its node is read-only. +log('Adding a persistent bitmap live to the r-o base node:') +vm.qmp_log('block-dirty-bitmap-add', node='base', name='bitmap1', + persistent=True) + +# Inactivate globally, like migration completion/save/qemu-update do; +# inactivating "base" directly would hit an unrelated guard, since it is +# a backing child of the still-active "top". +log('Inactivating all nodes must not fail flushing bitmap1:') +vm.qmp_log('blockdev-set-active', active=False) +vm.qmp_log('blockdev-set-active', active=True) + +log('Trying to remove the live-added bitmap, should fail same as bitmap0:') +vm.qmp_log('block-dirty-bitmap-remove', node='base', name='bitmap1') + vm.shutdown() if 'bitmaps' in qemu_img_info(base)['format-specific']['data']:
tests/qemu-iotests/tests/remove-bitmap-from-backing.out +11 -0
@@ -4,3 +4,14 @@ Trying to remove persistent bitmap from r-o base node, should fail: Remove persistent bitmap from base node reopened to RW: {"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "base"}} {"return": {}} +Adding a persistent bitmap live to the r-o base node: +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap1", "node": "base", "persistent": true}} +{"return": {}} +Inactivating all nodes must not fail flushing bitmap1: +{"execute": "blockdev-set-active", "arguments": {"active": false}} +{"return": {}} +{"execute": "blockdev-set-active", "arguments": {"active": true}} +{"return": {}} +Trying to remove the live-added bitmap, should fail same as bitmap0: +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap1", "node": "base"}} +{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap1' is readonly and cannot be modified"}}
Signature · 1 line
2.53.0
Quoted 8 lines · Denis V. Lunev
qmp_block_dirty_bitmap_add()marks a new bitmap persistent without checking write access to its node.bdrv_create_dirty_bitmap()always creates bitmaps writable, so a persistent bitmap added live to an already read-only node stays writable in memory on a node that can never store it. The next global inactivation (migration completion, save, or any otherbdrv_inactivate_all()caller) then fails trying to flush it back:
Quoted 4 lines
Lost persistent bitmaps during inactivation of node '<node>': No write access
migration_block_inactivate:bdrv_inactivate_all()failed: -22
qcow2_load_dirty_bitmaps()
Hm, but qcow2_load_dirty_bitmaps() and qmp_block_dirty_bitmap_add() are different:
In qcow2_load_dirty_bitmaps(), we load bitmap from the image, so we are sure, that the bitmap is already stored. And if image is readonly, we mark bitmap readonly as well. So, on exit, we assume, that bitmap is already stored, and don't produce "Lost bitmaps" error.
In qmp_block_dirty_bitmap_add(), we create a new bitmap, that not yet stored. So, even if we don't modify it, the error "Lost bitmap" would be correct on exit, as bitmap is not stored at all.
---
Also, this patch breaks the scenario of create and than merge some bitmap into created one. That's a breaking change, and would need a deprecation period.
---
Still. Maybe better be more strict, and simply forbid creating persistent bitmaps in read-only images at all? (this will need a deprecation period anyway).
What is your scenario?
Quoted 109 lines
and
qcow2_reopen_bitmaps_ro()already force
a persistent bitmap read-only when its node is read-only; do the same right after the bitmap is marked persistent, whether it arrived through a plain block-dirty-bitmap-add or the transaction action that calls the same function.qcow2_reopen_bitmaps_rw()already promotes any persistent, read-only bitmap back to writable once its node is reopened read-write, so the reverse direction needs no changes.Also check
BDRV_O_INACTIVE, matching qcow2's privatecan_write()helper, to cover adding a bitmap to a node that is already inactive. Document the new behavior on BlockDirtyBitmapAdd.persistent.Extend remove-bitmap-from-backing to cover this: add a persistent bitmap live to the already read-only "base" node, then inactivate all nodes the way migration completion, virsh save and qemu-update do, and check the bitmap ends up as protected as one already on disk.
Signed-off-by Denis V. Lunev <[email protected]>CC Eric Blake <[email protected]>CC Vladimir Sementsov-Ogievskiy <[email protected]>CC John Snow <[email protected]>--- block/monitor/bitmap-qmp-cmds.c | 9 +++++++++ qapi/block-core.json | 4 +++- .../tests/remove-bitmap-from-backing | 16 ++++++++++++++++ .../tests/remove-bitmap-from-backing.out | 11 +++++++++++ 4 files changed, 39 insertions(+), 1 deletion(-)block/monitor/bitmap-qmp-cmds.c +9
-0@@ -141,6 +141,15 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, }bdrv_dirty_bitmap_set_persistence (bitmap, persistent); + +/* +* A persistent bitmap on a node that can't be written must not be +* writable either, same as one loaded from or reopened onto it. +*/ +if (persistent && + (bdrv_is_read_only (bs) || (bdrv_get_flags (bs) &BDRV_O_INACTIVE ))) { +bdrv_dirty_bitmap_set_readonly (bitmap, true); + } } BdrvDirtyBitmap *block_dirty_bitmap_remove(const char *node, const char *name,qapi/block-core.json +3
-1@@ -2353,7 +2353,9 @@ # @persistent: the bitmap is persistent, i.e. it will be saved to the # corresponding block device image file on its close. For now # only Qcow2 disks support persistent bitmaps. Default is false -# for `block-dirty-bitmap-add`. (Since: 2.10) +# for `block-dirty-bitmap-add`. If the node is read-only, the +# bitmap is created read-only instead, the same as a persistent +# bitmap already stored on such a node. (Since: 2.10) # # @disabled: the bitmap is created in the disabled state, which means # that it will not track drive changes. The bitmap may be enabledtests/qemu-iotests/tests/remove-bitmap-from-backing +16
-0@@ -66,6 +66,22 @@ result = vm.qmp('blockdev-reopen', **new_base_opts) if result != {'return': {}}: log('Failed to reopen: ' + str(result)) +# A bitmap added live (not loaded from disk) must be protected the same +# way as bitmap0 above once its node is read-only. +log('Adding a persistent bitmap live to the r-o base node:') +vm.qmp_log('block-dirty-bitmap-add', node='base', name='bitmap1', + persistent=True) + +# Inactivate globally, like migration completion/save/qemu-update do; +# inactivating "base" directly would hit an unrelated guard, since it is +# a backing child of the still-active "top". +log('Inactivating all nodes must not fail flushing bitmap1:') +vm.qmp_log('blockdev-set-active', active=False) +vm.qmp_log('blockdev-set-active', active=True) + +log('Trying to remove the live-added bitmap, should fail same as bitmap0:') +vm.qmp_log('block-dirty-bitmap-remove', node='base', name='bitmap1') + vm.shutdown() if 'bitmaps' in qemu_img_info(base)['format-specific']['data']:tests/qemu-iotests/tests/remove-bitmap-from-backing.out +11
-0@@ -4,3 +4,14 @@ Trying to remove persistent bitmap from r-o base node, should fail: Remove persistent bitmap from base node reopened to RW: {"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "base"}} {"return": {}} +Adding a persistent bitmap live to the r-o base node: +{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap1", "node": "base", "persistent": true}} +{"return": {}} +Inactivating all nodes must not fail flushing bitmap1: +{"execute": "blockdev-set-active", "arguments": {"active": false}} +{"return": {}} +{"execute": "blockdev-set-active", "arguments": {"active": true}} +{"return": {}} +Trying to remove the live-added bitmap, should fail same as bitmap0: +{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap1", "node": "base"}} +{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap1' is readonly and cannot be modified"}}
Signature · 2 lines
Best regards, Vladimir
Quoted 31 lines · Vladimir Sementsov-Ogievskiy
Quoted 8 lines · Denis V. Lunev
qmp_block_dirty_bitmap_add()marks a new bitmap persistent without checking write access to its node.bdrv_create_dirty_bitmap()always creates bitmaps writable, so a persistent bitmap added live to an already read-only node stays writable in memory on a node that can never store it. The next global inactivation (migration completion, save, or any otherbdrv_inactivate_all()caller) then fails trying to flush it back:Quoted 4 lines
Lost persistent bitmaps during inactivation of node '<node>': No write access
migration_block_inactivate:bdrv_inactivate_all()failed: -22
qcow2_load_dirty_bitmaps()Hm, but
qcow2_load_dirty_bitmaps()andqmp_block_dirty_bitmap_add()are different:In
qcow2_load_dirty_bitmaps(), we load bitmap from the image, so we are sure, that the bitmap is already stored. And if image is readonly, we mark bitmap readonly as well. So, on exit, we assume, that bitmap is already stored, and don't produce "Lost bitmaps" error.In
qmp_block_dirty_bitmap_add(), we create a new bitmap, that not yet stored. So, even if we don't modify it, the error "Lost bitmap" would be correct on exit, as bitmap is not stored at all.---
Also, this patch breaks the scenario of create and than merge some bitmap into created one. That's a breaking change, and would need a deprecation period.
Quoted 5 lines
---
Still. Maybe better be more strict, and simply forbid creating persistent bitmaps in read-only images at all? (this will need a deprecation period anyway).
this was also an option. We will need a couple of additional tweaks but the approach is ok.
I'll followup with new series.
Den
Quoted 11 lines · Denis V. Lunev
qmp_block_dirty_bitmap_add()marks a new bitmap persistent without checking write access to its node.bdrv_create_dirty_bitmap()always creates bitmaps writable, so a persistent bitmap added live to an already read-only node stays writable in memory on a node that can never store it. The next global inactivation (migration completion, save, or any otherbdrv_inactivate_all()caller) then fails trying to flush it back:Lost persistent bitmaps during inactivation of node '<node>': No write access
migration_block_inactivate:bdrv_inactivate_all()failed: -22
Quoted 8 lines
qcow2_load_dirty_bitmaps()andqcow2_reopen_bitmaps_ro()already force a persistent bitmap read-only when its node is read-only; do the same right after the bitmap is marked persistent, whether it arrived through a plain block-dirty-bitmap-add or the transaction action that calls the same function.qcow2_reopen_bitmaps_rw()already promotes any persistent, read-only bitmap back to writable once its node is reopened read-write, so the reverse direction needs no changes.
As Vladimir noticed, we don't really get our non-persistent bitmap automatically converted to RW on reopen, as qcow2_reopen_bitmaps_rw() -> bitmap_list_load() only loads bitmaps actually stored in the image. Before that patch, after RW reopen we have both the node and the bitmap in RW. But with this patch the bitmap stays RO, and we can trigger an assert crash by issuing a backup. Here's a script which reproduces it:
Quoted 28 lines
#!/bin/bash
set -uSRCDIR=$(dirname "$(readlink -f "$0")")
BUILD=$SRCDIR/build
QSD=$BUILD/storage-daemon/qemu-storage-daemon
TEST=$(mktemp -d)
QMPSHELL="$BUILD/run qmp-shell"
QMPSOCK=$TEST/qmp.sock
QSD_PID=cleanup() { [ -n "$QSD_PID" ] && kill "$QSD_PID" 2>/dev/null rm -rf "$TEST" }trap cleanup EXIT
qemu-img create -f qcow2 "$TEST/tgt.qcow2" 1M >/dev/null qemu-img create -f qcow2 "$TEST/src.qcow2" 1M >/dev/null qemu-io -f qcow2 -c 'write 0 1M' "$TEST/src.qcow2" >/dev/null
"$QSD" \ --blockdev file,node-name=tgtfile,filename="$TEST/tgt.qcow2" \ --blockdev qcow2,node-name=tgt,file=tgtfile,read-only=on \ --blockdev file,node-name=srcfile,filename="$TEST/src.qcow2" \ --blockdev qcow2,node-name=src,file=srcfile \ --chardev socket,id=qmp0,path=$QMPSOCK,server=on,wait=off \ --monitor chardev=qmp0 &
Quoted 7 lines
QSD_PID=$!
$QMPSHELL -p $QMPSOCK <<EOF
block-dirty-bitmap-add node=tgt name=bmp0 persistent=true blockdev-reopen options=[{"driver":"qcow2","node-name":"tgt","file":"tgtfile","read-only":false}] blockdev-backup job-id=job0 device=src target=tgt sync=full x-perf={"use-copy-range":true}EOF
qemu-storage-daemon: ../block/dirty-bitmap.c:670: void bdrv_set_dirty(BlockDriverState *, int64_t, int64_t): Assertion `!bdrv_dirty_bitmap_readonly(bitmap)' failed.
Quoted 15 lines
Also check
BDRV_O_INACTIVE, matching qcow2's privatecan_write()helper, to cover adding a bitmap to a node that is already inactive. Document the new behavior on BlockDirtyBitmapAdd.persistent.Extend remove-bitmap-from-backing to cover this: add a persistent bitmap live to the already read-only "base" node, then inactivate all nodes the way migration completion, virsh save and qemu-update do, and check the bitmap ends up as protected as one already on disk.
Signed-off-by Denis V. Lunev <[email protected]>CC Eric Blake <[email protected]>CC Vladimir Sementsov-Ogievskiy <[email protected]>CC John Snow <[email protected]>---
[...]