Skip to contentSkip to conversation

backup: Make sure that source and target size match

PATCH Superseded Kevin Wolf 3 participants 10 messages Apr 29, 2020 – Apr 30, 2020 Block layer

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

cover backup: Make sure that source and target size match

Kevin Wolf (3):
backup: Improve error for bdrv_getlength() failure
backup: Make sure that source and target size match
iotests: Backup with different source/target size

 block/backup-top.c         | 12 +++++---
 block/backup.c             | 18 ++++++++++--
 tests/qemu-iotests/055     | 60 ++++++++++++++++++++++++++++++++++++--
 tests/qemu-iotests/055.out |  4 +--
 4 files changed, 83 insertions(+), 11 deletions(-)
Signature · 1 line
2.25.3
author
#1 lore raw

PATCH 1/3 backup: Improve error for bdrv_getlength() failure

bdrv_get_device_name() will be an empty string with modern management tools that don't use -drive. Use bdrv_get_device_or_node_name() instead so that the node name is used if the BlockBackend is anonymous.

While at it, start with upper case to make the message consistent with the rest of the function.

---
 block/backup.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
block/backup.c +2 -2
index a7a7dcaf4c..c4c3b8cd46 100644
@@ -400,8 +400,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
 
     len = bdrv_getlength(bs);
     if (len < 0) {
-        error_setg_errno(errp, -len, "unable to get length for '%s'",
-                         bdrv_get_device_name(bs));
+        error_setg_errno(errp, -len, "Unable to get length for '%s'",
+                         bdrv_get_device_or_node_name(bs));
         goto error;
     }
 
Signature · 1 line
2.25.3
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] backup: Improve error for bdrv_getlength() failure

Quoted 8 lines · Kevin Wolf

bdrv_get_device_name() will be an empty string with modern management tools that don't use -drive. Use bdrv_get_device_or_node_name() instead so that the node name is used if the BlockBackend is anonymous.

While at it, start with upper case to make the message consistent with the rest of the function.

Signature · 2 lines
Best regards,
Vladimir
Reviewed-by on v1 1/3
#3 lore raw

Re: [PATCH 1/3] backup: Improve error for bdrv_getlength() failure

In reply to Kevin Wolf
Quoted 8 lines · Kevin Wolf

bdrv_get_device_name() will be an empty string with modern management tools that don't use -drive. Use bdrv_get_device_or_node_name() instead so that the node name is used if the BlockBackend is anonymous.

While at it, start with upper case to make the message consistent with the rest of the function.

Berto

author
#4 lore raw

PATCH 2/3 backup: Make sure that source and target size match

In reply to Kevin Wolf

Since the introduction of a backup filter node, the backup block job crashes when the target image is smaller than the source image because it will try to write after the end of the target node without having BLK_PERM_RESIZE. (Previously, the BlockBackend layer would have caught this and errored out gracefully.)

We can fix this and even do better than the old behaviour: Check that source and target have the same image size at the start of the block job and unshare BLK_PERM_RESIZE. This will immediately error out when starting the job instead of only when writing to a block that doesn't exist in the target.

Longer target than source would technically work because we would never write to blocks that don't exist, but semantically these are invalid, too, because a backup is supposed to create a copy, not just an image that starts with a copy.

The bugs were introduced in commits 2c8074c45 (BLK_PERM_RESIZE is shared since this commit) and 00e30f05d (BdrvChild instead of BlockBackend turns I/O errors into assertion failures).

Fixes 2c8074c453ff13a94bd08ec26061917670ec03be
Fixes 00e30f05de1d19586345ec373970ef4c192c6270
---
 block/backup-top.c | 12 ++++++++----
 block/backup.c     | 14 +++++++++++++-
 2 files changed, 21 insertions(+), 5 deletions(-)
block/backup-top.c +8 -4
index 3b50c06e2c..0e515a7705 100644
@@ -148,8 +148,10 @@ static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c,
          *
          * Share write to target (child_file), to not interfere
          * with guest writes to its disk which may be in target backing chain.
+         * Can't resize during a backup block job because we check the size
+         * only upfront.
          */
-        *nshared = BLK_PERM_ALL;
+        *nshared = BLK_PERM_ALL & ~BLK_PERM_RESIZE;
         *nperm = BLK_PERM_WRITE;
     } else {
         /* Source child */
@@ -192,11 +194,13 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
 {
     Error *local_err = NULL;
     BDRVBackupTopState *state;
-    BlockDriverState *top = bdrv_new_open_driver(&bdrv_backup_top_filter,
-                                                 filter_node_name,
-                                                 BDRV_O_RDWR, errp);
+    BlockDriverState *top;
     bool appended = false;
 
+    assert(source->total_sectors == target->total_sectors);
+
+    top = bdrv_new_open_driver(&bdrv_backup_top_filter, filter_node_name,
+                              BDRV_O_RDWR, errp);
     if (!top) {
         return NULL;
     }
block/backup.c +13 -1
index c4c3b8cd46..4f13bb20a5 100644
@@ -340,7 +340,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   BlockCompletionFunc *cb, void *opaque,
                   JobTxn *txn, Error **errp)
 {
-    int64_t len;
+    int64_t len, target_len;
     BackupBlockJob *job = NULL;
     int64_t cluster_size;
     BdrvRequestFlags write_flags;
@@ -405,6 +405,18 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         goto error;
     }
 
+    target_len = bdrv_getlength(target);
+    if (target_len < 0) {
+        error_setg_errno(errp, -target_len, "Unable to get length for '%s'",
+                         bdrv_get_device_or_node_name(bs));
+        goto error;
+    }
+
+    if (target_len != len) {
+        error_setg(errp, "Source and target image have different sizes");
+        goto error;
+    }
+
     cluster_size = backup_calculate_cluster_size(target, errp);
     if (cluster_size < 0) {
         goto error;
Signature · 1 line
2.25.3
Reviewed-by on v1 2/3
#5 lore raw
Quoted 12 lines · Kevin Wolf

Since the introduction of a backup filter node, the backup block job crashes when the target image is smaller than the source image because it will try to write after the end of the target node without having BLK_PERM_RESIZE. (Previously, the BlockBackend layer would have caught this and errored out gracefully.)

We can fix this and even do better than the old behaviour: Check that source and target have the same image size at the start of the block job and unshare BLK_PERM_RESIZE. This will immediately error out when starting the job instead of only when writing to a block that doesn't exist in the target.

Quoted 6 lines

Longer target than source would technically work because we would never write to blocks that don't exist, but semantically these are invalid, too, because a backup is supposed to create a copy, not just an image that starts with a copy.

The bugs were introduced in commits 2c8074c45 (BLK_PERM_RESIZE is shared

no, it was unshared by blks in block-copy

Quoted 2 lines

since this commit) and 00e30f05d (BdrvChild instead of BlockBackend turns I/O errors into assertion failures).

and here becomes shared.

So, seems only 00e30f05d is broken and introduces both problems

Quoted 28 lines
Fixes 2c8074c453ff13a94bd08ec26061917670ec03be
Fixes 00e30f05de1d19586345ec373970ef4c192c6270
---
  block/backup-top.c | 12 ++++++++----
  block/backup.c     | 14 +++++++++++++-
  2 files changed, 21 insertions(+), 5 deletions(-)
block/backup-top.c +3 -1
index 3b50c06e2c..0e515a7705 100644
@@ -148,8 +148,10 @@ static void backup_top_child_perm(BlockDriverState *bs, BdrvChild *c,
           *
           * Share write to target (child_file), to not interfere
           * with guest writes to its disk which may be in target backing chain.
+         * Can't resize during a backup block job because we check the size
+         * only upfront.
           */
-        *nshared = BLK_PERM_ALL;
+        *nshared = BLK_PERM_ALL & ~BLK_PERM_RESIZE;
          *nperm = BLK_PERM_WRITE;
      } else {
          /* Source child */

block/backup-top.c BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, · line 200

Quoted 10 lines
block/backup-top.c +2 -3
@@ -192,11 +194,13 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
  {
      Error *local_err = NULL;
      BDRVBackupTopState *state;
-    BlockDriverState *top = bdrv_new_open_driver(&bdrv_backup_top_filter,
-                                                 filter_node_name,
-                                                 BDRV_O_RDWR, errp);
+    BlockDriverState *top;
      bool appended = false;
 
+    assert(source->total_sectors == target->total_sectors);

Is it correct to use directly total_sectors and not bdrv_getlenght()? Anyway, using bdrv_getlength() seems safer, and will help us if we move to byte-accurate block-layer at some moment in future.

Hmm but total_sectors used directly anyway in this function, so OK

block/backup-top.c BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, · line 203

Quoted 3 lines
block/backup-top.c +3 -0
+
+    top = bdrv_new_open_driver(&bdrv_backup_top_filter, filter_node_name,
+                              BDRV_O_RDWR, errp);

alignment broken. With it fixed:

Quoted 36 lines
block/backup-top.c +0 -0
      if (!top) {
          return NULL;
      }
block/backup.c +13 -1
index c4c3b8cd46..4f13bb20a5 100644
@@ -340,7 +340,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                    BlockCompletionFunc *cb, void *opaque,
                    JobTxn *txn, Error **errp)
  {
-    int64_t len;
+    int64_t len, target_len;
      BackupBlockJob *job = NULL;
      int64_t cluster_size;
      BdrvRequestFlags write_flags;
@@ -405,6 +405,18 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
          goto error;
      }
 
+    target_len = bdrv_getlength(target);
+    if (target_len < 0) {
+        error_setg_errno(errp, -target_len, "Unable to get length for '%s'",
+                         bdrv_get_device_or_node_name(bs));
+        goto error;
+    }
+
+    if (target_len != len) {
+        error_setg(errp, "Source and target image have different sizes");
+        goto error;
+    }
+
      cluster_size = backup_calculate_cluster_size(target, errp);
      if (cluster_size < 0) {
          goto error;
Signature · 2 lines
Best regards,
Vladimir
author
#6 lore raw

PATCH 3/3 iotests: Backup with different source/target size

In reply to Kevin Wolf

This tests that the backup jobs catches situations where the target node has a different size than the source node. It must also forbid resize operations when the job is already running.

---
 tests/qemu-iotests/055     | 60 ++++++++++++++++++++++++++++++++++++--
 tests/qemu-iotests/055.out |  4 +--
 2 files changed, 60 insertions(+), 4 deletions(-)
tests/qemu-iotests/055 +58 -2
index 82b9f5f47d..243d66a62e 100755
@@ -48,8 +48,10 @@ class TestSingleDrive(iotests.QMPTestCase):
     def setUp(self):
         qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
 
-        self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
-        self.vm.add_drive(blockdev_target_img, interface="none")
+        self.vm = iotests.VM()
+        self.vm.add_drive('blkdebug::' + test_img, 'node-name=source')
+        self.vm.add_drive(blockdev_target_img, 'node-name=target',
+                          interface="none")
         if iotests.qemu_default_machine == 'pc':
             self.vm.add_drive(None, 'media=cdrom', 'ide')
         self.vm.launch()
@@ -112,6 +114,60 @@ class TestSingleDrive(iotests.QMPTestCase):
     def test_pause_blockdev_backup(self):
         self.do_test_pause('blockdev-backup', 'drive1', blockdev_target_img)
 
+    def test_source_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive0', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='source', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)
+
+    def test_target_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive1', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='target', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)
+
+    def test_small_target(self):
+        short_len = image_len // 2
+        result = self.vm.qmp('block_resize', device='drive1', size=short_len)
+        self.assert_qmp(result, 'return', {})
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full')
+        self.assert_qmp(result, 'error/class', 'GenericError')
+
+    def test_large_target(self):
+        short_len = image_len * 2
+        result = self.vm.qmp('block_resize', device='drive1', size=short_len)
+        self.assert_qmp(result, 'return', {})
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full')
+        self.assert_qmp(result, 'error/class', 'GenericError')
+
     def test_medium_not_found(self):
         if iotests.qemu_default_machine != 'pc':
             return
tests/qemu-iotests/055.out +2 -2
index 5ce2f9a2ed..88bf7fa73a 100644
@@ -1,5 +1,5 @@
-..............................
+..................................
 ----------------------------------------------------------------------
-Ran 30 tests
+Ran 34 tests
 
 OK
Signature · 1 line
2.25.3
#7 lore raw

Re: [PATCH 3/3] iotests: Backup with different source/target size

Quoted 28 lines · Kevin Wolf

This tests that the backup jobs catches situations where the target node has a different size than the source node. It must also forbid resize operations when the job is already running.

---
  tests/qemu-iotests/055     | 60 ++++++++++++++++++++++++++++++++++++--
  tests/qemu-iotests/055.out |  4 +--
  2 files changed, 60 insertions(+), 4 deletions(-)
tests/qemu-iotests/055 +4 -2
index 82b9f5f47d..243d66a62e 100755
@@ -48,8 +48,10 @@ class TestSingleDrive(iotests.QMPTestCase):
      def setUp(self):
          qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
 
-        self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
-        self.vm.add_drive(blockdev_target_img, interface="none")
+        self.vm = iotests.VM()
+        self.vm.add_drive('blkdebug::' + test_img, 'node-name=source')
+        self.vm.add_drive(blockdev_target_img, 'node-name=target',
+                          interface="none")
          if iotests.qemu_default_machine == 'pc':
              self.vm.add_drive(None, 'media=cdrom', 'ide')
          self.vm.launch()

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 118

Quoted 5 lines
tests/qemu-iotests/055 +2 -0
@@ -112,6 +114,60 @@ class TestSingleDrive(iotests.QMPTestCase):
      def test_pause_blockdev_backup(self):
          self.do_test_pause('blockdev-backup', 'drive1', blockdev_target_img)
 
+    def test_source_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()

this will never fire, as vm is created a moment before, I'd drop it.

Quoted 23 lines
tests/qemu-iotests/055 +23 -0
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive0', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='source', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)
+
+    def test_target_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive1', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 151

Quoted 10 lines
tests/qemu-iotests/055 +10 -0
+            result = self.vm.qmp('block_resize', node_name='target', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)

these two functions are almost identical.. worth refactoring to be use common helper?

Quoted 8 lines
tests/qemu-iotests/055 +8 -0
+
+    def test_small_target(self):
+        short_len = image_len // 2
+        result = self.vm.qmp('block_resize', device='drive1', size=short_len)
+        self.assert_qmp(result, 'return', {})
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full')

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 169

Quoted 10 lines
tests/qemu-iotests/055 +10 -0
+        self.assert_qmp(result, 'error/class', 'GenericError')
+
+    def test_large_target(self):
+        short_len = image_len * 2
+        result = self.vm.qmp('block_resize', device='drive1', size=short_len)
+        self.assert_qmp(result, 'return', {})
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full')
+        self.assert_qmp(result, 'error/class', 'GenericError')

these are identical too, still they are smaller and hardly needs refactoring

Quoted 17 lines
tests/qemu-iotests/055 +1 -0
+
      def test_medium_not_found(self):
          if iotests.qemu_default_machine != 'pc':
              return
tests/qemu-iotests/055.out +2 -2
index 5ce2f9a2ed..88bf7fa73a 100644
@@ -1,5 +1,5 @@
-..............................
+..................................
  ----------------------------------------------------------------------
-Ran 30 tests
+Ran 34 tests
 
  OK
Signature · 2 lines
Best regards,
Vladimir
author
#8 lore raw

Re: [PATCH 3/3] iotests: Backup with different source/target size

Am 29.04.2020 um 14:28 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 9 lines
Quoted 8 lines · Kevin Wolf

This tests that the backup jobs catches situations where the target node has a different size than the source node. It must also forbid resize operations when the job is already running.

---
  tests/qemu-iotests/055     | 60 ++++++++++++++++++++++++++++++++++++--
  tests/qemu-iotests/055.out |  4 +--

One general remark and question that came up while I was running 055 a lot and really got annonyed by the long time it takes:

TestDriveCompression is quite unconventional in that 055 is raw/qcow2 only per se, but some of the test cases always test qcow2 and vmdk. The slow one is vmdk.

I found out that zero writes in vmdk are completely broken (I'll send patches), but even after fixing this, it's still slow. I think this is the combination of VMDK not having writeback metadata caching and backup serving lots of tiny 64k zero writes.

Has anyone ever looked into making backup use more reasonable request sizes for the background copy like mirror does?

Quoted 17 lines
Quoted 17 lines
  2 files changed, 60 insertions(+), 4 deletions(-)
tests/qemu-iotests/055 +4 -2
index 82b9f5f47d..243d66a62e 100755
@@ -48,8 +48,10 @@ class TestSingleDrive(iotests.QMPTestCase):
      def setUp(self):
          qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
-        self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
-        self.vm.add_drive(blockdev_target_img, interface="none")
+        self.vm = iotests.VM()
+        self.vm.add_drive('blkdebug::' + test_img, 'node-name=source')
+        self.vm.add_drive(blockdev_target_img, 'node-name=target',
+                          interface="none")
          if iotests.qemu_default_machine == 'pc':
              self.vm.add_drive(None, 'media=cdrom', 'ide')
Quoted 8 lines

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 117

Quoted 6 lines
tests/qemu-iotests/055 +2 -0
          self.vm.launch()
@@ -112,6 +114,60 @@ class TestSingleDrive(iotests.QMPTestCase):
      def test_pause_blockdev_backup(self):
          self.do_test_pause('blockdev-backup', 'drive1', blockdev_target_img)
+    def test_source_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()

this will never fire, as vm is created a moment before, I'd drop it.

This pattern exists all over the place in 055, but you're right, it's kind of pointless.

Quoted 27 lines
Quoted 27 lines
tests/qemu-iotests/055 +27 -0
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive0', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='source', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)
+
+    def test_target_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive1', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='target', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
Quoted 8 lines

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 133

Quoted 6 lines
tests/qemu-iotests/055 +6 -0
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)

these two functions are almost identical.. worth refactoring to be use common helper?

Ok, I'll see what I can do.

Kevin

#9 lore raw

Re: [PATCH 3/3] iotests: Backup with different source/target size

Quoted 11 lines · Kevin Wolf

Am 29.04.2020 um 14:28 hat Vladimir Sementsov-Ogievskiy geschrieben:

Quoted 9 lines
Quoted 8 lines · Kevin Wolf

This tests that the backup jobs catches situations where the target node has a different size than the source node. It must also forbid resize operations when the job is already running.

---
   tests/qemu-iotests/055     | 60 ++++++++++++++++++++++++++++++++++++--
   tests/qemu-iotests/055.out |  4 +--
Quoted 5 lines

One general remark and question that came up while I was running 055 a lot and really got annonyed by the long time it takes:

TestDriveCompression is quite unconventional in that 055 is raw/qcow2 only per se, but some of the test cases always test qcow2 and vmdk.

Yes, that's bad. Oh, seems I have a patch for it not still sent. Will do soon.

Quoted 10 lines

The
slow one is vmdk.

I found out that zero writes in vmdk are completely broken (I'll send patches), but even after fixing this, it's still slow. I think this is the combination of VMDK not having writeback metadata caching and backup serving lots of tiny 64k zero writes.

Has anyone ever looked into making backup use more reasonable request sizes for the background copy like mirror does?

I am :)

The whole series improving backup is
"[RFC 00/24] backup performance: block_status + async"
First part is already merged, current chunk close to be finished:
"[PATCH v4 0/5] block-copy: use aio-task-pool"

Quoted 70 lines
Quoted 18 lines
Quoted 18 lines
   2 files changed, 60 insertions(+), 4 deletions(-)
tests/qemu-iotests/055 +4 -2
index 82b9f5f47d..243d66a62e 100755
@@ -48,8 +48,10 @@ class TestSingleDrive(iotests.QMPTestCase):
       def setUp(self):
           qemu_img('create', '-f', iotests.imgfmt, blockdev_target_img, str(image_len))
-        self.vm = iotests.VM().add_drive('blkdebug::' + test_img)
-        self.vm.add_drive(blockdev_target_img, interface="none")
+        self.vm = iotests.VM()
+        self.vm.add_drive('blkdebug::' + test_img, 'node-name=source')
+        self.vm.add_drive(blockdev_target_img, 'node-name=target',
+                          interface="none")
           if iotests.qemu_default_machine == 'pc':
               self.vm.add_drive(None, 'media=cdrom', 'ide')
Quoted 8 lines

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 117

Quoted 6 lines
tests/qemu-iotests/055 +2 -0
           self.vm.launch()
@@ -112,6 +114,60 @@ class TestSingleDrive(iotests.QMPTestCase):
       def test_pause_blockdev_backup(self):
           self.do_test_pause('blockdev-backup', 'drive1', blockdev_target_img)
+    def test_source_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()

this will never fire, as vm is created a moment before, I'd drop it.

This pattern exists all over the place in 055, but you're right, it's kind of pointless.

Quoted 27 lines
Quoted 27 lines
tests/qemu-iotests/055 +27 -0
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive0', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='source', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)
+
+    def test_target_resize_blockdev_backup(self):
+        self.assert_no_active_block_jobs()
+
+        def pre_finalize():
+            result = self.vm.qmp('block_resize', device='drive1', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+            result = self.vm.qmp('block_resize', node_name='target', size=65536)
+            self.assert_qmp(result, 'error/class', 'GenericError')
+
+        result = self.vm.qmp('blockdev-backup', job_id='job0', device='drive0',
Quoted 8 lines

tests/qemu-iotests/055 class TestSingleDrive(iotests.QMPTestCase): · line 133

Quoted 6 lines
tests/qemu-iotests/055 +6 -0
+                             target='drive1', sync='full', auto_finalize=False,
+                             auto_dismiss=False)
+        self.assert_qmp(result, 'return', {})
+
+        self.vm.run_job('job0', auto_finalize=False, pre_finalize=pre_finalize,
+                        use_log=False)

these two functions are almost identical.. worth refactoring to be use common helper?

Ok, I'll see what I can do.

Kevin

Signature · 2 lines
Best regards,
Vladimir