Skip to contentSkip to conversation

qemu-img: Support bitmap --merge into backing image

PATCH Superseded Eric Blake 2 participants 4 messages Sep 9, 2020 – Sep 14, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Eric Blake

author
#0 lore raw

PATCH 1/1 qemu-img: Support bitmap --merge into backing image

If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a bitmap from top into base, qemu-img was failing with:

qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock Is another process using the image [base.qcow2]?

The easiest fix is to not open the entire backing chain of the source image, so that we aren't worrying about competing BDS visiting the backing image as both a read-only backing of the source and the writeable destination.

---
 qemu-img.c                 |  3 +-
 tests/qemu-iotests/291     | 12 ++++++++
 tests/qemu-iotests/291.out | 56 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 70 insertions(+), 1 deletion(-)
qemu-img.c +2 -1
index eb2fc1f86243..b15098a2f9b3 100644
@@ -4755,7 +4755,8 @@ static int img_bitmap(int argc, char **argv)
     }
     bs = blk_bs(blk);
     if (src_filename) {
-        src = img_open(false, src_filename, src_fmt, 0, false, false, false);
+        src = img_open(false, src_filename, src_fmt, BDRV_O_NO_BACKING,
+                       false, false, false);
         if (!src) {
             goto out;
         }
tests/qemu-iotests/291 +12 -0
index 1e0bb76959bb..4f837b205655 100755
@@ -91,6 +91,15 @@ $QEMU_IMG bitmap --remove --image-opts \
     driver=$IMGFMT,file.driver=file,file.filename="$TEST_IMG" tmp
 _img_info --format-specific
 
+echo
+echo "=== Merge from top layer into backing image ==="
+echo
+
+$QEMU_IMG rebase -u -F qcow2 -b "$TEST_IMG.base" "$TEST_IMG"
+$QEMU_IMG bitmap --add --merge b2 -b "$TEST_IMG" -F $IMGFMT \
+     -f $IMGFMT "$TEST_IMG.base" b3
+_img_info --format-specific --backing-chain
+
 echo
 echo "=== Check bitmap contents ==="
 echo
@@ -107,6 +116,9 @@ $QEMU_IMG map --output=json --image-opts \
 nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG"
 $QEMU_IMG map --output=json --image-opts \
     "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map
+nbd_server_start_unix_socket -r -f qcow2 -B b3 "$TEST_IMG"
+$QEMU_IMG map --output=json --image-opts \
+    "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b3" | _filter_qemu_img_map
 
 # success, all done
 echo '*** done'
tests/qemu-iotests/291.out +56 -0

66 lines not on this page · raw message

Signature · 1 line
2.28.0
#1 lore raw
Quoted 25 lines · Eric Blake

If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a bitmap from top into base, qemu-img was failing with:

qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock Is another process using the image [base.qcow2]?

The easiest fix is to not open the entire backing chain of the source image, so that we aren't worrying about competing BDS visiting the backing image as both a read-only backing of the source and the writeable destination.

---
 qemu-img.c                 |  3 +-
 tests/qemu-iotests/291     | 12 ++++++++
 tests/qemu-iotests/291.out | 56 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 70 insertions(+), 1 deletion(-)
qemu-img.c +0 -0
index eb2fc1f86243..b15098a2f9b3 100644

qemu-img.c static int img_bitmap(int argc, char **argv) · line 4759

Quoted 6 lines
qemu-img.c +2 -1
@@ -4755,7 +4755,8 @@ static int img_bitmap(int argc, char **argv)
     }
     bs = blk_bs(blk);
     if (src_filename) {
-        src = img_open(false, src_filename, src_fmt, 0, false, false, false);
+        src = img_open(false, src_filename, src_fmt, BDRV_O_NO_BACKING,
+                       false, false, false);

Why not do the same for the destination BB?

Quoted 7 lines
qemu-img.c +0 -0
         if (!src) {
             goto out;
         }
tests/qemu-iotests/291 +0 -0
index 1e0bb76959bb..4f837b205655 100755

[...]

tests/qemu-iotests/291 $QEMU_IMG map --output=json --image-opts \ · line 119

Quoted 5 lines
tests/qemu-iotests/291 +1 -0
@@ -107,6 +116,9 @@ $QEMU_IMG map --output=json --image-opts \
 nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG"
 $QEMU_IMG map --output=json --image-opts \
     "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map
+nbd_server_start_unix_socket -r -f qcow2 -B b3 "$TEST_IMG"

Why not look into $TEST_IMG.base to see specifically whether the bitmap is there?

(I also am quite surprised that it’s even possible to export bitmaps from backing nodes, but, well.)

Max

author
#2 lore raw
Quoted 21 lines · Max Reitz
Quoted 20 lines · Eric Blake

If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a bitmap from top into base, qemu-img was failing with:

qemu-img: Could not open 'top.qcow2': Could not open backing file: Failed to get shared "write" lock Is another process using the image [base.qcow2]?

The easiest fix is to not open the entire backing chain of the source image, so that we aren't worrying about competing BDS visiting the backing image as both a read-only backing of the source and the writeable destination.

---
  qemu-img.c                 |  3 +-
  tests/qemu-iotests/291     | 12 ++++++++
  tests/qemu-iotests/291.out | 56 ++++++++++++++++++++++++++++++++++++++
  3 files changed, 70 insertions(+), 1 deletion(-)
Quoted 13 lines

qemu-img.c static int img_bitmap(int argc, char **argv) · line 4759

Quoted 11 lines
qemu-img.c +2 -1
index eb2fc1f86243..b15098a2f9b3 100644
@@ -4755,7 +4755,8 @@ static int img_bitmap(int argc, char **argv)
      }
      bs = blk_bs(blk);
      if (src_filename) {
-        src = img_open(false, src_filename, src_fmt, 0, false, false, false);
+        src = img_open(false, src_filename, src_fmt, BDRV_O_NO_BACKING,
+                       false, false, false);

Why not do the same for the destination BB?

Yeah, that should work, too.

Quoted 11 lines
Quoted 8 lines
qemu-img.c +0 -0
          if (!src) {
              goto out;
          }
tests/qemu-iotests/291 +0 -0
index 1e0bb76959bb..4f837b205655 100755

[...]

Quoted 8 lines

tests/qemu-iotests/291 $QEMU_IMG map --output=json --image-opts \ · line 119

Quoted 5 lines
tests/qemu-iotests/291 +1 -0
@@ -107,6 +116,9 @@ $QEMU_IMG map --output=json --image-opts \
  nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG"
  $QEMU_IMG map --output=json --image-opts \
      "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map
+nbd_server_start_unix_socket -r -f qcow2 -B b3 "$TEST_IMG"

Why not look into $TEST_IMG.base to see specifically whether the bitmap is there?

We did just that, several lines earlier, with the qemu-img info --backing-chain.

Quoted 3 lines

(I also am quite surprised that it’s even possible to export bitmaps from backing nodes, but, well.)

I actually ought to call that out in the commit message. It used to be that we were inconsistent on what we could see from the backing chain (a filter would make it so we can't), but as soon as your filter patches land, then we _do_ want to always be able to find a bitmap from the backing chain (incremental backup depends on that: we create an overlay disk to run the block-copy job as a filter, and _want_ to expose that overlay image with the bitmap it inherits from the original image). So being able to export bitmaps from a backing node is normally a feature; and it is only in 'qemu-img bitmap' where we don't want accidental inheritance to get in the way from what we are actually merging.

I'll send a v2.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#3 lore raw
Quoted 60 lines · Eric Blake
Quoted 22 lines · Max Reitz
Quoted 21 lines · Eric Blake

If you have the chain 'base.qcow2 <- top.qcow2' and want to merge a bitmap from top into base, qemu-img was failing with:

qemu-img: Could not open 'top.qcow2': Could not open backing file:
Failed to get shared "write" lock
Is another process using the image [base.qcow2]?

The easiest fix is to not open the entire backing chain of the source image, so that we aren't worrying about competing BDS visiting the backing image as both a read-only backing of the source and the writeable destination.

---
  qemu-img.c                 |  3 +-
  tests/qemu-iotests/291     | 12 ++++++++ tests/qemu-iotests/291.out | 56 ++++++++++++++++++++++++++++++++++++++

  3 files changed, 70 insertions(+), 1 deletion(-)
Quoted 14 lines

qemu-img.c static int img_bitmap(int argc, char **argv) · line 4760

Quoted 12 lines
qemu-img.c +2 -1
index eb2fc1f86243..b15098a2f9b3 100644
@@ -4755,7 +4755,8 @@ static int img_bitmap(int argc, char **argv)
       }
       bs = blk_bs(blk);
       if (src_filename) {
-        src = img_open(false, src_filename, src_fmt, 0, false,
 false, false);
+        src = img_open(false, src_filename, src_fmt, BDRV_O_NO_BACKING,
+                       false, false, false);

Why not do the same for the destination BB?

Yeah, that should work, too.

Quoted 11 lines
Quoted 8 lines
qemu-img.c +0 -0
           if (!src) {
               goto out;

          }

tests/qemu-iotests/291 +0 -0
index 1e0bb76959bb..4f837b205655 100755

[...]

Quoted 8 lines
Quoted 5 lines
tests/qemu-iotests/291 +0 -0
@@ -107,6 +116,9 @@ $QEMU_IMG map --output=json --image-opts \

  nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG" $QEMU_IMG map --output=json --image-opts \
      "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map +nbd_server_start_unix_socket -r -f qcow2 -B b3 "$TEST_IMG"

Why not look into $TEST_IMG.base to see specifically whether the bitmap is there?

Quoted 2 lines

We did just that, several lines earlier, with the qemu-img info --backing-chain.

OK, perfect.

Quoted 4 lines
Quoted 3 lines

(I also am quite surprised that it’s even possible to export bitmaps from backing nodes, but, well.)

Quoted 10 lines

I actually ought to call that out in the commit message.  It used to be that we were inconsistent on what we could see from the backing chain (a filter would make it so we can't), but as soon as your filter patches land, then we _do_ want to always be able to find a bitmap from the backing chain (incremental backup depends on that: we create an overlay disk to run the block-copy job as a filter, and _want_ to expose that overlay image with the bitmap it inherits from the original image).  So being able to export bitmaps from a backing node is normally a feature; and it is only in 'qemu-img bitmap' where we don't want accidental inheritance to get in the way from what we are actually merging.

Understood.

Max