Skip to contentSkip to conversation

backup performance: block_status + async

PATCH Superseded Vladimir Sementsov-Ogievskiy 3 participants 53 messages Oct 26, 2020 – Jan 20, 2021 Block layer

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover backup performance: block_status + async

Hi all!

The series turn backup into series of block_copy_async calls, covering the whole disk, so we get block-status based paralallel async requests out of the box, which gives performance gain:

All results are in seconds

-----------------  -----------  -------------  --------------  ---------------------  --------------------------------  ------------------------------------
                   A            B              C               D                      E                                 F
                   mirror(old)  backup(old)    backup(old)     backup(new)            backup(new)                       backup(new)
                                copy-range=on  copy-range=off                         copy-range=on                     copy-range=on
                                                                                                                        max-workers=1
hdd-ext4:hdd-ext4  19           20             21 ± 14%        19                     51 ± 12%                          22 ± 24%
                                  A+5%           A+12%  B+6%     A+3%  B-2%  C-8%       A+174%  B+161%  C+145%  D+165%    A+18%  B+12%  C+5%  D+14%  E-57%
hdd-ext4:ssd-ext4  8.7          9.4 ± 3%       9.6 ± 2%        8.8                    24 ± 2%                           8.9
                                  A+8%           A+10%  B+2%     A+1%  B-7%  C-9%       A+174%  B+155%  C+149%  D+173%    A+2%  B-5%  C-8%  D+1%  E-63%
ssd-ext4:hdd-ext4  9            12 ± 9%        11 ± 7%         9.7 ± 7%               11 ± 2%                           10 ± 3%
                                  A+36%          A+28%  B-6%     A+7%  B-21%  C-16%     A+21%  B-11%  C-5%  D+13%         A+16%  B-14%  C-9%  D+8%  E-4%
ssd-ext4:ssd-ext4  4.4          11 ± 4%        10 ± 3%         4.7                    5.7                               10 ± 5%
                                  A+143%         A+134%  B-4%    A+6%  B-56%  C-55%     A+30%  B-46%  C-45%  D+22%        A+133%  B-4%  C-1%  D+119%  E+79%
hdd-xfs:hdd-xfs    19           20 ± 3%        20              20                     45 ± 4%                           19
                                  A+3%           A+4%  B+1%      A+3%  B+0%  C-1%       A+131%  B+125%  C+122%  D+125%    A-1%  B-4%  C-4%  D-3%  E-57%
hdd-xfs:ssd-xfs    9.1          9.9 ± 4%       9.5             9.1 ± 3%               23 ± 2%                           9.2
                                  A+8%           A+4%  B-4%      A+0%  B-8%  C-4%       A+151%  B+132%  C+142%  D+151%    A+1%  B-7%  C-3%  D+1%  E-60%
ssd-xfs:hdd-xfs    9.1          11 ± 9%        11              9.5 ± 4%               12 ± 22%                          11 ± 3%
                                  A+16%          A+22%  B+6%     A+4%  B-10%  C-15%     A+32%  B+14%  C+8%  D+26%         A+18%  B+2%  C-4%  D+13%  E-10%
ssd-xfs:ssd-xfs    4.1          8.7 ± 7%       9.2 ± 5%        4.5 ± 2%               5.7 ± 3%                          9.7 ± 5%
                                  A+113%         A+126%  B+6%    A+11%  B-48%  C-51%    A+40%  B-34%  C-38%  D+27%        A+138%  B+12%  C+5%  D+115%  E+70%
ssd-ext4:nbd       9.1 ± 2%     37             37 ± 2%         11                     11 ± 3%                           19 ± 2%
                                  A+302%         A+304%  B+1%    A+18%  B-71%  C-71%    A+18%  B-71%  C-71%  D+0%         A+106%  B-49%  C-49%  D+74%  E+75%
nbd:ssd-ext4       9            30 ± 3%        31              9                      9                                 17
                                  A+237%         A+245%  B+2%    A+0%  B-70%  C-71%     A+0%  B-70%  C-71%  D+0%          A+93%  B-43%  C-44%  D+93%  E+93%

----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------

Here column B is current backup and column D is new backup with default parameters.

Mirror is still faster, but we are very close to it.

v3:
01: add Max's r-b
02: change to perf.use-copy-range
03: add Max's r-b
04: - more explicit finish status of async block_copy
- block_copy_async always return non-NULL
- personal opaque for new cb
05: - new arguments added in this patch
- no default value for arguments in block_copy_async()
06: new
07: - caller does _kick() by hand
- grammar in commit msg
- add new parameter in _this_ patch
- switch to opposite ignore_ratelimit
08: cancel now is async
09,10: add Max's r-b
11: changed a lot
12: add timeout
14: rebase on x-perf, keep r-b
15: rebase on x-perf
16: rebase on x-perf, keep r-b
17,18: new
19: now only backup.c is changed in this patch, changed a lot 20,21: new
22: rebased, keep r-b
23: new, split from 24
24: drop unrelated change (now patch23), keep r-b
25: changed a lot, explicitly specify options for each env (test table column)

To run benchmark do the following:

prepare images:
In a directories, where you want to place source and target images, prepare images by:

for img in test-source test-target; do
./qemu-img create -f raw $img 1000M;
./qemu-img bench -c 1000 -d 1 -f raw -s 1M -w --pattern=0xff $img
done

prepare similar image for nbd server, and start it somewhere by

qemu-nbd --persistent --nocache -f raw IMAGE

Then, run benchmark, like this:
./bench-backup.py --env old:/work/src/qemu/up-backup-block-copy-master/build/qemu-system-x86_64,mirror old,copy-range=on old,copy-range=off new:../../build/qemu-system-x86_64 new,copy-range=on new,copy-range=on,max-workers=1 --dir hdd-ext4:/test-a hdd-xfs:/test-b ssd-ext4:/ssd ssd-xfs:/ssd-xfs --test $(for fs in ext4 xfs; do echo hdd-$fs:hdd-$fs hdd-$fs:ssd-$fs ssd-$fs:hdd-$fs ssd-$fs:ssd-$fs; done) --nbd 192.168.100.5 --test ssd-ext4:nbd nbd:ssd-ext4

(you may simply reduce number of directories/test-cases, use --help for
help)

Vladimir Sementsov-Ogievskiy (25):
iotests: 129 don't check backup "busy"
qapi: backup: add perf.use-copy-range parameter block/block-copy: More explicit call_state block/block-copy: implement block_copy_async block/block-copy: add max_chunk and max_workers parameters block/block-copy: add list of all call-states block/block-copy: add ratelimit to block-copy block/block-copy: add block_copy_cancel
blockjob: add set_speed to BlockJobDriver
job: call job_enter from job_user_pause
qapi: backup: add max-chunk and max-workers to x-perf struct
iotests: 56: prepare for backup over block-copy
iotests: 129: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 257: prepare for backup over block-copy block/block-copy: make progress_bytes_callback optional block/backup: drop extra gotos from backup_run()
backup: move to block-copy
qapi: backup: disable copy_range by default block/block-copy: drop unused block_copy_set_progress_callback() block/block-copy: drop unused argument of block_copy() simplebench/bench_block_job: use correct shebang line with python3
simplebench: bench_block_job: add cmd_options argument
simplebench: add bench-backup.py

 qapi/block-core.json                   |  26 ++-
 block/backup-top.h                     |   1 +
 include/block/block-copy.h             |  58 ++++-
 include/block/block_int.h              |   3 +
 include/block/blockjob_int.h           |   2 +
 block/backup-top.c                     |   6 +-
 block/backup.c                         | 233 ++++++++++++-------
 block/block-copy.c                     | 227 +++++++++++++++---
 block/replication.c                    |   2 +
 blockdev.c                             |  14 ++
 blockjob.c                             |   6 +
 job.c                                  |   1 +
 scripts/simplebench/bench-backup.py    | 165 +++++++++++++
 scripts/simplebench/bench-example.py   |   2 +-
 scripts/simplebench/bench_block_job.py |  13 +-
 tests/qemu-iotests/056                 |   9 +-
 tests/qemu-iotests/129                 |   3 +-
 tests/qemu-iotests/185                 |   3 +-
 tests/qemu-iotests/185.out             |   2 +-
 tests/qemu-iotests/219                 |  13 +-
 tests/qemu-iotests/257                 |   1 +
 tests/qemu-iotests/257.out             | 306 ++++++++++++-------------
 22 files changed, 798 insertions(+), 298 deletions(-)
 create mode 100755 scripts/simplebench/bench-backup.py
Signature · 1 line
2.21.3
#1 lore raw

PATCH v3 1/25 iotests: 129 don't check backup "busy"

Busy is racy, job has it's "pause-points" when it's not busy. Drop this check.

---
 tests/qemu-iotests/129 | 1 -
 1 file changed, 1 deletion(-)
tests/qemu-iotests/129 +0 -1
index 0e13244d85..3c22f6448c 100755
@@ -67,7 +67,6 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
         result = self.vm.qmp("stop")
         self.assert_qmp(result, 'return', {})
         result = self.vm.qmp("query-block-jobs")
-        self.assert_qmp(result, 'return[0]/busy', True)
         self.assert_qmp(result, 'return[0]/ready', False)
 
     def test_drive_mirror(self):
Signature · 1 line
2.21.3
#2 lore raw

PATCH v3 2/25 qapi: backup: add perf.use-copy-range parameter

In reply to Vladimir Sementsov-Ogievskiy

Experiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.

For now, add new backup qmp parameter with x- prefix for the following reasons:

- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.

So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.

In do_backup_common() new x-perf parameter handled in a way to make further options addition simpler.

We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.

---
 qapi/block-core.json       | 17 ++++++++++++++++-
 block/backup-top.h         |  1 +
 include/block/block-copy.h |  2 +-
 include/block/block_int.h  |  3 +++
 block/backup-top.c         |  4 +++-
 block/backup.c             |  6 +++++-
 block/block-copy.c         |  4 ++--
 block/replication.c        |  2 ++
 blockdev.c                 |  8 ++++++++
 9 files changed, 41 insertions(+), 6 deletions(-)
qapi/block-core.json +16 -1
index e74669159c..71e6faa15c 100644
@@ -1370,6 +1370,19 @@
 { 'struct': 'BlockdevSnapshot',
   'data': { 'node': 'str', 'overlay': 'str' } }
 
+##
+# @BackupPerf:
+#
+# Optional parameters for backup. These parameters don't affect
+# functionality, but may significantly affect performance.
+#
+# @use-copy-range: Use copy offloading. Default true.
+#
+# Since: 5.2
+##
+{ 'struct': 'BackupPerf',
+  'data': { '*use-copy-range': 'bool' }}
+
 ##
 # @BackupCommon:
 #
@@ -1425,6 +1438,8 @@
 #                    above node specified by @drive. If this option is not given,
 #                    a node name is autogenerated. (Since: 4.2)
 #
+# @x-perf: Performance options. (Since 5.2)
+#
 # Note: @on-source-error and @on-target-error only affect background
 #       I/O.  If an error occurs during a guest write request, the device's
 #       rerror/werror actions will be used.
@@ -1439,7 +1454,7 @@
             '*on-source-error': 'BlockdevOnError',
             '*on-target-error': 'BlockdevOnError',
             '*auto-finalize': 'bool', '*auto-dismiss': 'bool',
-            '*filter-node-name': 'str' } }
+            '*filter-node-name': 'str', '*x-perf': 'BackupPerf'  } }
 
 ##
 # @DriveBackup:
block/backup-top.h +1 -0
index e5cabfa197..b28b0031c4 100644
@@ -33,6 +33,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
                                          BlockDriverState *target,
                                          const char *filter_node_name,
                                          uint64_t cluster_size,
+                                         BackupPerf *perf,
                                          BdrvRequestFlags write_flags,
                                          BlockCopyState **bcs,
                                          Error **errp);
include/block/block-copy.h +1 -1
index aac85e1488..6397505f30 100644
@@ -22,7 +22,7 @@ typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
 typedef struct BlockCopyState BlockCopyState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size,
+                                     int64_t cluster_size, bool use_copy_range,
                                      BdrvRequestFlags write_flags,
                                      Error **errp);
 
include/block/block_int.h +3 -0
index 887b0668d8..dc1ed349df 100644
@@ -1248,6 +1248,8 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
  * @sync_mode: What parts of the disk image should be copied to the destination.
  * @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental'
  * @bitmap_mode: The bitmap synchronization policy to use.
+ * @perf: Performance options. All actual fields assumed to be present,
+ *        all ".has_*" fields are ignored.
  * @on_source_error: The action to take upon error reading from the source.
  * @on_target_error: The action to take upon error writing to the target.
  * @creation_flags: Flags that control the behavior of the Job lifetime.
@@ -1266,6 +1268,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                             BitmapSyncMode bitmap_mode,
                             bool compress,
                             const char *filter_node_name,
+                            BackupPerf *perf,
                             BlockdevOnError on_source_error,
                             BlockdevOnError on_target_error,
                             int creation_flags,
block/backup-top.c +3 -1
index fe6883cc97..789acf6965 100644
@@ -186,6 +186,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
                                          BlockDriverState *target,
                                          const char *filter_node_name,
                                          uint64_t cluster_size,
+                                         BackupPerf *perf,
                                          BdrvRequestFlags write_flags,
                                          BlockCopyState **bcs,
                                          Error **errp)
@@ -244,7 +245,8 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,
 
     state->cluster_size = cluster_size;
     state->bcs = block_copy_state_new(top->backing, state->target,
-                                      cluster_size, write_flags, &local_err);
+                                      cluster_size, perf->use_copy_range,
+                                      write_flags, &local_err);
     if (local_err) {
         error_prepend(&local_err, "Cannot create block-copy-state: ");
         goto fail;
block/backup.c +5 -1
index 9afa0bf3b4..4b07e9115d 100644
@@ -46,6 +46,7 @@ typedef struct BackupBlockJob {
     uint64_t len;
     uint64_t bytes_read;
     int64_t cluster_size;
+    BackupPerf perf;
 
     BlockCopyState *bcs;
 } BackupBlockJob;
@@ -335,6 +336,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   BitmapSyncMode bitmap_mode,
                   bool compress,
                   const char *filter_node_name,
+                  BackupPerf *perf,
                   BlockdevOnError on_source_error,
                   BlockdevOnError on_target_error,
                   int creation_flags,
@@ -441,7 +443,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                   (compress ? BDRV_REQ_WRITE_COMPRESSED : 0),
 
     backup_top = bdrv_backup_top_append(bs, target, filter_node_name,
-                                        cluster_size, write_flags, &bcs, errp);
+                                        cluster_size, perf,
+                                        write_flags, &bcs, errp);
     if (!backup_top) {
         goto error;
     }
@@ -464,6 +467,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     job->bcs = bcs;
     job->cluster_size = cluster_size;
     job->len = len;
+    job->perf = *perf;
 
     block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, job);
     block_copy_set_progress_meter(bcs, &job->common.job.progress);
block/block-copy.c +2 -2
index cd9bc47c8f..63398a171c 100644
@@ -218,7 +218,7 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target)
 }
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
-                                     int64_t cluster_size,
+                                     int64_t cluster_size, bool use_copy_range,
                                      BdrvRequestFlags write_flags, Error **errp)
 {
     BlockCopyState *s;
@@ -260,7 +260,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
          * We enable copy-range, but keep small copy_size, until first
          * successful copy_range (look at block_copy_do_copy).
          */
-        s->use_copy_range = true;
+        s->use_copy_range = use_copy_range;
         s->copy_size = MAX(s->cluster_size, BLOCK_COPY_MAX_BUFFER);
     }
 
block/replication.c +2 -0
index 0c70215784..22ffc811ee 100644
@@ -454,6 +454,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
     int64_t active_length, hidden_length, disk_length;
     AioContext *aio_context;
     Error *local_err = NULL;
+    BackupPerf perf = { .use_copy_range = true };
 
     aio_context = bdrv_get_aio_context(bs);
     aio_context_acquire(aio_context);
@@ -558,6 +559,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
         s->backup_job = backup_job_create(
                                 NULL, s->secondary_disk->bs, s->hidden_disk->bs,
                                 0, MIRROR_SYNC_MODE_NONE, NULL, 0, false, NULL,
+                                &perf,
                                 BLOCKDEV_ON_ERROR_REPORT,
                                 BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL,
                                 backup_job_completed, bs, NULL, &local_err);
blockdev.c +8 -0
index fe6fb5dc1d..b71ed08a3b 100644
@@ -2788,6 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
 {
     BlockJob *job = NULL;
     BdrvDirtyBitmap *bmap = NULL;
+    BackupPerf perf = { .use_copy_range = true };
     int job_flags = JOB_DEFAULT;
 
     if (!backup->has_speed) {
@@ -2812,6 +2813,12 @@ static BlockJob *do_backup_common(BackupCommon *backup,
         backup->compress = false;
     }
 
+    if (backup->x_perf) {
+        if (backup->x_perf->has_use_copy_range) {
+            perf.use_copy_range = backup->x_perf->use_copy_range;
+        }
+    }
+
     if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
         (backup->sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
         /* done before desugaring 'incremental' to print the right message */
@@ -2885,6 +2892,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
                             backup->sync, bmap, backup->bitmap_mode,
                             backup->compress,
                             backup->filter_node_name,
+                            &perf,
                             backup->on_source_error,
                             backup->on_target_error,
                             job_flags, NULL, NULL, txn, errp);
Signature · 1 line
2.21.3
Reviewed-by on v3 2/25
#3 lore raw

Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter

Quoted 25 lines · Vladimir Sementsov-Ogievskiy

Experiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.

For now, add new backup qmp parameter with x- prefix for the following reasons:

- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.

So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.

In do_backup_common() new x-perf parameter handled in a way to make further options addition simpler.

We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.

Quoted 11 lines
---
  qapi/block-core.json       | 17 ++++++++++++++++-
  block/backup-top.h         |  1 +
  include/block/block-copy.h |  2 +-
  include/block/block_int.h  |  3 +++
  block/backup-top.c         |  4 +++-
  block/backup.c             |  6 +++++-
  block/block-copy.c         |  4 ++--
  block/replication.c        |  2 ++
  blockdev.c                 |  8 ++++++++
  9 files changed, 41 insertions(+), 6 deletions(-)
#4 lore raw

Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter

In reply to Vladimir Sementsov-Ogievskiy
Quoted 31 lines · Vladimir Sementsov-Ogievskiy

Experiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.

For now, add new backup qmp parameter with x- prefix for the following reasons:

- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.

So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.

In do_backup_common() new x-perf parameter handled in a way to make further options addition simpler.

We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.

---

+++ b/qapi/block-core.json

qapi/block-core.json +0 -0
@@ -1370,6 +1370,19 @@
 { 'struct': 'BlockdevSnapshot',
   'data': { 'node': 'str', 'overlay': 'str' } }

qapi/block-core.json line 1381

Quoted 10 lines
qapi/block-core.json +9 -0
 
+##
+# @BackupPerf:
+#
+# Optional parameters for backup. These parameters don't affect
+# functionality, but may significantly affect performance.
+#
+# @use-copy-range: Use copy offloading. Default true.
+#
+# Since: 5.2

We've missed 5.2; this will need to be updated to 6.0

qapi/block-core.json line 1441

Quoted 12 lines
qapi/block-core.json +5 -0
+##
+{ 'struct': 'BackupPerf',
+  'data': { '*use-copy-range': 'bool' }}
+
 ##
 # @BackupCommon:
 #
@@ -1425,6 +1438,8 @@
 #                    above node specified by @drive. If this option is not given,
 #                    a node name is autogenerated. (Since: 4.2)
 #
+# @x-perf: Performance options. (Since 5.2)

and here

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#5 lore raw

Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter

Quoted 45 lines · Eric Blake
Quoted 31 lines · Vladimir Sementsov-Ogievskiy

Experiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.

For now, add new backup qmp parameter with x- prefix for the following reasons:

- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.

So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.

In do_backup_common() new x-perf parameter handled in a way to make further options addition simpler.

We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.

---

+++ b/qapi/block-core.json

qapi/block-core.json +0 -0
@@ -1370,6 +1370,19 @@
  { 'struct': 'BlockdevSnapshot',
    'data': { 'node': 'str', 'overlay': 'str' } }

qapi/block-core.json line 1381

Quoted 10 lines
qapi/block-core.json +9 -0
 
+##
+# @BackupPerf:
+#
+# Optional parameters for backup. These parameters don't affect
+# functionality, but may significantly affect performance.
+#
+# @use-copy-range: Use copy offloading. Default true.
+#
+# Since: 5.2

We've missed 5.2; this will need to be updated to 6.0

Quoted 15 lines

qapi/block-core.json line 1441

Quoted 12 lines
qapi/block-core.json +5 -0
+##
+{ 'struct': 'BackupPerf',
+  'data': { '*use-copy-range': 'bool' }}
+
  ##
  # @BackupCommon:
  #
@@ -1425,6 +1438,8 @@
  #                    above node specified by @drive. If this option is not given,
  #                    a node name is autogenerated. (Since: 4.2)
  #
+# @x-perf: Performance options. (Since 5.2)

and here

I wanted to say that it's fixed in v4 (which is already in Max's tree), but... It's not fixed. Max, could you touch this up?

Signature · 2 lines
Best regards,
Vladimir
#6 lore raw

Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter

Quoted 61 lines · Vladimir Sementsov-Ogievskiy
Quoted 44 lines · Eric Blake
Quoted 30 lines · Vladimir Sementsov-Ogievskiy

Experiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.

For now, add new backup qmp parameter with x- prefix for the following reasons:

  - We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
  - On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.

So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.

In do_backup_common() new x-perf parameter handled in a way to make further options addition simpler.

We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.

---

+++ b/qapi/block-core.json

qapi/block-core.json +0 -0
@@ -1370,6 +1370,19 @@
   { 'struct': 'BlockdevSnapshot',

qapi/block-core.json line 1380

Quoted 10 lines
qapi/block-core.json +9 -0
     'data': { 'node': 'str', 'overlay': 'str' } }
+##
+# @BackupPerf:
+#
+# Optional parameters for backup. These parameters don't affect
+# functionality, but may significantly affect performance.
+#
+# @use-copy-range: Use copy offloading. Default true.
+#
+# Since: 5.2

We've missed 5.2; this will need to be updated to 6.0

Quoted 15 lines

qapi/block-core.json line 1441

Quoted 12 lines
qapi/block-core.json +5 -0
+##
+{ 'struct': 'BackupPerf',
+  'data': { '*use-copy-range': 'bool' }}
+
   ##
   # @BackupCommon:
   #
@@ -1425,6 +1438,8 @@
   #                    above node specified by @drive. If this option is not given,
   #                    a node name is autogenerated. (Since: 4.2)
   #
+# @x-perf: Performance options. (Since 5.2)

and here

Quoted 1 line

I wanted to say that it's fixed in v4 (which is already in Max's tree), but... It's not fixed. Max, could you touch this up?

Sure.

Max

#7 lore raw

PATCH v3 3/25 block/block-copy: More explicit call_state

In reply to Vladimir Sementsov-Ogievskiy

Refactor common path to use BlockCopyCallState pointer as parameter, to prepare it for use in asynchronous block-copy (at least, we'll need to run block-copy in a coroutine, passing the whole parameters as one pointer).

---
 block/block-copy.c | 51 ++++++++++++++++++++++++++++++++++------------
 1 file changed, 38 insertions(+), 13 deletions(-)
block/block-copy.c +38 -13
index 63398a171c..6ea55f1f9a 100644
@@ -30,7 +30,15 @@
 static coroutine_fn int block_copy_task_entry(AioTask *task);
 
 typedef struct BlockCopyCallState {
+    /* IN parameters */
+    BlockCopyState *s;
+    int64_t offset;
+    int64_t bytes;
+
+    /* State */
     bool failed;
+
+    /* OUT parameters */
     bool error_is_read;
 } BlockCopyCallState;
 
@@ -544,15 +552,17 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
  * Returns 1 if dirty clusters found and successfully copied, 0 if no dirty
  * clusters found and -errno on failure.
  */
-static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s,
-                                                  int64_t offset, int64_t bytes,
-                                                  bool *error_is_read)
+static int coroutine_fn
+block_copy_dirty_clusters(BlockCopyCallState *call_state)
 {
+    BlockCopyState *s = call_state->s;
+    int64_t offset = call_state->offset;
+    int64_t bytes = call_state->bytes;
+
     int ret = 0;
     bool found_dirty = false;
     int64_t end = offset + bytes;
     AioTaskPool *aio = NULL;
-    BlockCopyCallState call_state = {false, false};
 
     /*
      * block_copy() user is responsible for keeping source and target in same
@@ -568,7 +578,7 @@ static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s,
         BlockCopyTask *task;
         int64_t status_bytes;
 
-        task = block_copy_task_create(s, &call_state, offset, bytes);
+        task = block_copy_task_create(s, call_state, offset, bytes);
         if (!task) {
             /* No more dirty bits in the bitmap */
             trace_block_copy_skip_range(s, offset, bytes);
@@ -633,15 +643,12 @@ out:
 
         aio_task_pool_free(aio);
     }
-    if (error_is_read && ret < 0) {
-        *error_is_read = call_state.error_is_read;
-    }
 
     return ret < 0 ? ret : found_dirty;
 }
 
 /*
- * block_copy
+ * block_copy_common
  *
  * Copy requested region, accordingly to dirty bitmap.
  * Collaborate with parallel block_copy requests: if they succeed it will help
@@ -649,16 +656,16 @@ out:
  * it means that some I/O operation failed in context of _this_ block_copy call,
  * not some parallel operation.
  */
-int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
-                            bool *error_is_read)
+static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
 {
     int ret;
 
     do {
-        ret = block_copy_dirty_clusters(s, offset, bytes, error_is_read);
+        ret = block_copy_dirty_clusters(call_state);
 
         if (ret == 0) {
-            ret = block_copy_wait_one(s, offset, bytes);
+            ret = block_copy_wait_one(call_state->s, call_state->offset,
+                                      call_state->bytes);
         }
 
         /*
@@ -675,6 +682,24 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
     return ret;
 }
 
+int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
+                            bool *error_is_read)
+{
+    BlockCopyCallState call_state = {
+        .s = s,
+        .offset = start,
+        .bytes = bytes,
+    };
+
+    int ret = block_copy_common(&call_state);
+
+    if (error_is_read && ret < 0) {
+        *error_is_read = call_state.error_is_read;
+    }
+
+    return ret;
+}
+
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
 {
     return s->copy_bitmap;
Signature · 1 line
2.21.3
#8 lore raw

PATCH v3 4/25 block/block-copy: implement block_copy_async

In reply to Vladimir Sementsov-Ogievskiy

We'll need async block-copy invocation to use in backup directly.

---
 include/block/block-copy.h | 26 ++++++++++++
 block/block-copy.c         | 81 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 103 insertions(+), 4 deletions(-)
include/block/block-copy.h +26 -0
index 6397505f30..3566827599 100644
@@ -19,7 +19,9 @@
 #include "qemu/co-shared-resource.h"
 
 typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
+typedef void (*BlockCopyAsyncCallbackFunc)(void *opaque);
 typedef struct BlockCopyState BlockCopyState;
+typedef struct BlockCopyCallState BlockCopyCallState;
 
 BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
                                      int64_t cluster_size, bool use_copy_range,
@@ -41,6 +43,30 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
 int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
                             bool *error_is_read);
 
+/*
+ * Run block-copy in a coroutine, create corresponding BlockCopyCallState
+ * object and return pointer to it. Never returns NULL.
+ *
+ * Caller is responsible to call block_copy_call_free() to free
+ * BlockCopyCallState object.
+ */
+BlockCopyCallState *block_copy_async(BlockCopyState *s,
+                                     int64_t offset, int64_t bytes,
+                                     BlockCopyAsyncCallbackFunc cb,
+                                     void *cb_opaque);
+
+/*
+ * Free finished BlockCopyCallState. Trying to free running
+ * block-copy will crash.
+ */
+void block_copy_call_free(BlockCopyCallState *call_state);
+
+/* Note, that block-copy call is marked finished prior to calling the callback. */
+bool block_copy_call_finished(BlockCopyCallState *call_state);
+bool block_copy_call_succeeded(BlockCopyCallState *call_state);
+bool block_copy_call_failed(BlockCopyCallState *call_state);
+int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read);
+
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
 
block/block-copy.c +77 -4

110 lines not on this page · raw message

Signature · 1 line
2.21.3
Reviewed-by on v3 4/25
#9 lore raw

Re: [PATCH v3 04/25] block/block-copy: implement block_copy_async

Quoted 7 lines · Vladimir Sementsov-Ogievskiy

We'll need async block-copy invocation to use in backup directly.

---
  include/block/block-copy.h | 26 ++++++++++++
  block/block-copy.c         | 81 ++++++++++++++++++++++++++++++++++++--
  2 files changed, 103 insertions(+), 4 deletions(-)
#10 lore raw

PATCH v3 5/25 block/block-copy: add max_chunk and max_workers parameters

In reply to Vladimir Sementsov-Ogievskiy

They will be used for backup.

---
 include/block/block-copy.h |  6 ++++++
 block/block-copy.c         | 11 +++++++++--
 2 files changed, 15 insertions(+), 2 deletions(-)
include/block/block-copy.h +6 -0
index 3566827599..57806e8a49 100644
@@ -49,9 +49,15 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
  *
  * Caller is responsible to call block_copy_call_free() to free
  * BlockCopyCallState object.
+ *
+ * @max_workers means maximum of parallel coroutines to execute sub-requests,
+ * must be > 0.
+ *
+ * @max_chunk means maximum length for one IO operation. Zero means unlimited.
  */
 BlockCopyCallState *block_copy_async(BlockCopyState *s,
                                      int64_t offset, int64_t bytes,
+                                     int max_workers, int64_t max_chunk,
                                      BlockCopyAsyncCallbackFunc cb,
                                      void *cb_opaque);
 
block/block-copy.c +9 -2
index 74655b86f8..35213bd832 100644
@@ -34,6 +34,8 @@ typedef struct BlockCopyCallState {
     BlockCopyState *s;
     int64_t offset;
     int64_t bytes;
+    int max_workers;
+    int64_t max_chunk;
     BlockCopyAsyncCallbackFunc cb;
     void *cb_opaque;
 
@@ -148,10 +150,11 @@ static BlockCopyTask *block_copy_task_create(BlockCopyState *s,
                                              int64_t offset, int64_t bytes)
 {
     BlockCopyTask *task;
+    int64_t max_chunk = MIN_NON_ZERO(s->copy_size, call_state->max_chunk);
 
     if (!bdrv_dirty_bitmap_next_dirty_area(s->copy_bitmap,
                                            offset, offset + bytes,
-                                           s->copy_size, &offset, &bytes))
+                                           max_chunk, &offset, &bytes))
     {
         return NULL;
     }
@@ -623,7 +626,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
         bytes = end - offset;
 
         if (!aio && bytes) {
-            aio = aio_task_pool_new(BLOCK_COPY_MAX_WORKERS);
+            aio = aio_task_pool_new(call_state->max_workers);
         }
 
         ret = block_copy_task_run(aio, task);
@@ -701,6 +704,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
         .s = s,
         .offset = start,
         .bytes = bytes,
+        .max_workers = BLOCK_COPY_MAX_WORKERS,
     };
 
     int ret = block_copy_common(&call_state);
@@ -719,6 +723,7 @@ static void coroutine_fn block_copy_async_co_entry(void *opaque)
 
 BlockCopyCallState *block_copy_async(BlockCopyState *s,
                                      int64_t offset, int64_t bytes,
+                                     int max_workers, int64_t max_chunk,
                                      BlockCopyAsyncCallbackFunc cb,
                                      void *cb_opaque)
 {
@@ -728,6 +733,8 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s,
         .s = s,
         .offset = offset,
         .bytes = bytes,
+        .max_workers = max_workers,
+        .max_chunk = max_chunk,
         .cb = cb,
         .cb_opaque = cb_opaque,
 
Signature · 1 line
2.21.3
#12 lore raw

PATCH v3 6/25 block/block-copy: add list of all call-states

In reply to Vladimir Sementsov-Ogievskiy

It simplifies debugging.

---
 block/block-copy.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
block/block-copy.c +10 -1
index 35213bd832..6bf1735b93 100644
@@ -42,6 +42,9 @@ typedef struct BlockCopyCallState {
     /* Coroutine where async block-copy is running */
     Coroutine *co;
 
+    /* To reference all call states from BlockCopyState */
+    QLIST_ENTRY(BlockCopyCallState) list;
+
     /* State */
     int ret;
     bool finished;
@@ -81,7 +84,8 @@ typedef struct BlockCopyState {
     bool use_copy_range;
     int64_t copy_size;
     uint64_t len;
-    QLIST_HEAD(, BlockCopyTask) tasks;
+    QLIST_HEAD(, BlockCopyTask) tasks; /* All tasks from all block-copy calls */
+    QLIST_HEAD(, BlockCopyCallState) calls;
 
     BdrvRequestFlags write_flags;
 
@@ -282,6 +286,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
     }
 
     QLIST_INIT(&s->tasks);
+    QLIST_INIT(&s->calls);
 
     return s;
 }
@@ -669,6 +674,8 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
 {
     int ret;
 
+    QLIST_INSERT_HEAD(&call_state->s->calls, call_state, list);
+
     do {
         ret = block_copy_dirty_clusters(call_state);
 
@@ -694,6 +701,8 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
         call_state->cb(call_state->cb_opaque);
     }
 
+    QLIST_REMOVE(call_state, list);
+
     return ret;
 }
 
Signature · 1 line
2.21.3
#14 lore raw

PATCH v3 7/25 block/block-copy: add ratelimit to block-copy

In reply to Vladimir Sementsov-Ogievskiy

We are going to directly use one async block-copy operation for backup job, so we need rate limiter.

We want to maintain current backup behavior: only background copying is limited and copy-before-write operations only participate in limit calculation. Therefore we need one rate limiter for block-copy state and boolean flag for block-copy call state for actual limitation.

Note, that we can't just calculate each chunk in limiter after successful copying: it will not save us from starting a lot of async sub-requests which will exceed limit too much. Instead let's use the following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one sub-request (which pretty much matches current backup behavior).

Note also, that if there is in-flight block-copy async call, block_copy_kick() should be used after set-speed to apply new setup faster. For that block_copy_kick() published in this patch.

---
 include/block/block-copy.h |  5 ++++-
 block/backup-top.c         |  2 +-
 block/backup.c             |  2 +-
 block/block-copy.c         | 46 +++++++++++++++++++++++++++++++++++++-
 4 files changed, 51 insertions(+), 4 deletions(-)
include/block/block-copy.h +4 -1
index 57806e8a49..0fab64e12a 100644
@@ -41,7 +41,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
                                      int64_t offset, int64_t *count);
 
 int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
-                            bool *error_is_read);
+                            bool ignore_ratelimit, bool *error_is_read);
 
 /*
  * Run block-copy in a coroutine, create corresponding BlockCopyCallState
@@ -73,6 +73,9 @@ bool block_copy_call_succeeded(BlockCopyCallState *call_state);
 bool block_copy_call_failed(BlockCopyCallState *call_state);
 int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read);
 
+void block_copy_set_speed(BlockCopyState *s, uint64_t speed);
+void block_copy_kick(BlockCopyCallState *call_state);
+
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
 
block/backup-top.c +1 -1
index 789acf6965..779956ddc2 100644
@@ -61,7 +61,7 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
     off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
     end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
 
-    return block_copy(s->bcs, off, end - off, NULL);
+    return block_copy(s->bcs, off, end - off, true, NULL);
 }
 
 static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
block/backup.c +1 -1
index 4b07e9115d..09ff5a92ef 100644
@@ -72,7 +72,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
 
     trace_backup_do_cow_enter(job, start, offset, bytes);
 
-    ret = block_copy(job->bcs, start, end - start, error_is_read);
+    ret = block_copy(job->bcs, start, end - start, true, error_is_read);
 
     trace_backup_do_cow_return(job, offset, bytes, ret);
 
block/block-copy.c +45 -1

96 lines not on this page · raw message

Signature · 1 line
2.21.3
Reviewed-by on v3 7/25
#15 lore raw

Re: [PATCH v3 07/25] block/block-copy: add ratelimit to block-copy

Quoted 23 lines · Vladimir Sementsov-Ogievskiy

We are going to directly use one async block-copy operation for backup job, so we need rate limiter.

We want to maintain current backup behavior: only background copying is limited and copy-before-write operations only participate in limit calculation. Therefore we need one rate limiter for block-copy state and boolean flag for block-copy call state for actual limitation.

Note, that we can't just calculate each chunk in limiter after successful copying: it will not save us from starting a lot of async sub-requests which will exceed limit too much. Instead let's use the following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one sub-request (which pretty much matches current backup behavior).

Note also, that if there is in-flight block-copy async call, block_copy_kick() should be used after set-speed to apply new setup faster. For that block_copy_kick() published in this patch.

Quoted 7 lines
---
  include/block/block-copy.h |  5 ++++-
  block/backup-top.c         |  2 +-
  block/backup.c             |  2 +-
  block/block-copy.c         | 46 +++++++++++++++++++++++++++++++++++++-
  4 files changed, 51 insertions(+), 4 deletions(-)
#16 lore raw

PATCH v3 8/25 block/block-copy: add block_copy_cancel

In reply to Vladimir Sementsov-Ogievskiy

Add function to cancel running async block-copy call. It will be used in backup.

---
 include/block/block-copy.h | 13 +++++++++++++
 block/block-copy.c         | 24 +++++++++++++++++++-----
 2 files changed, 32 insertions(+), 5 deletions(-)
include/block/block-copy.h +13 -0
index 0fab64e12a..8f5013d0aa 100644
@@ -71,11 +71,24 @@ void block_copy_call_free(BlockCopyCallState *call_state);
 bool block_copy_call_finished(BlockCopyCallState *call_state);
 bool block_copy_call_succeeded(BlockCopyCallState *call_state);
 bool block_copy_call_failed(BlockCopyCallState *call_state);
+bool block_copy_call_cancelled(BlockCopyCallState *call_state);
 int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read);
 
 void block_copy_set_speed(BlockCopyState *s, uint64_t speed);
 void block_copy_kick(BlockCopyCallState *call_state);
 
+/*
+ * Cancel running block-copy call.
+ *
+ * Cancel leaves block-copy state valid: dirty bits are correct and you may use
+ * cancel + <run block_copy with same parameters> to emulate pause/resume.
+ *
+ * Note also, that the cancel is async: it only marks block-copy call to be
+ * cancelled. So, the call may be cancelled (block_copy_call_cancelled() reports
+ * true) but not yet finished (block_copy_call_finished() reports false).
+ */
+void block_copy_call_cancel(BlockCopyCallState *call_state);
+
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
 void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
 
block/block-copy.c +19 -5
index fa27450b14..82cf945693 100644
@@ -51,6 +51,7 @@ typedef struct BlockCopyCallState {
     int ret;
     bool finished;
     QemuCoSleepState *sleep_state;
+    bool cancelled;
 
     /* OUT parameters */
     bool error_is_read;
@@ -594,7 +595,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
     assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
     assert(QEMU_IS_ALIGNED(bytes, s->cluster_size));
 
-    while (bytes && aio_task_pool_status(aio) == 0) {
+    while (bytes && aio_task_pool_status(aio) == 0 && !call_state->cancelled) {
         BlockCopyTask *task;
         int64_t status_bytes;
 
@@ -707,7 +708,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
     do {
         ret = block_copy_dirty_clusters(call_state);
 
-        if (ret == 0) {
+        if (ret == 0 && !call_state->cancelled) {
             ret = block_copy_wait_one(call_state->s, call_state->offset,
                                       call_state->bytes);
         }
@@ -721,7 +722,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
          * 2. We have waited for some intersecting block-copy request
          *    It may have failed and produced new dirty bits.
          */
-    } while (ret > 0);
+    } while (ret > 0 && !call_state->cancelled);
 
     call_state->finished = true;
 
@@ -801,12 +802,19 @@ bool block_copy_call_finished(BlockCopyCallState *call_state)
 
 bool block_copy_call_succeeded(BlockCopyCallState *call_state)
 {
-    return call_state->finished && call_state->ret == 0;
+    return call_state->finished && !call_state->cancelled &&
+        call_state->ret == 0;
 }
 
 bool block_copy_call_failed(BlockCopyCallState *call_state)
 {
-    return call_state->finished && call_state->ret < 0;
+    return call_state->finished && !call_state->cancelled &&
+        call_state->ret < 0;
+}
+
+bool block_copy_call_cancelled(BlockCopyCallState *call_state)
+{
+    return call_state->cancelled;
 }
 
 int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read)
@@ -818,6 +826,12 @@ int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read)
     return call_state->ret;
 }
 
+void block_copy_call_cancel(BlockCopyCallState *call_state)
+{
+    call_state->cancelled = true;
+    block_copy_kick(call_state);
+}
+
 BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s)
 {
     return s->copy_bitmap;
Signature · 1 line
2.21.3
Reviewed-by on v3 8/25
#17 lore raw

Re: [PATCH v3 08/25] block/block-copy: add block_copy_cancel

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Add function to cancel running async block-copy call. It will be used in backup.

---
  include/block/block-copy.h | 13 +++++++++++++
  block/block-copy.c         | 24 +++++++++++++++++++-----
  2 files changed, 32 insertions(+), 5 deletions(-)
#18 lore raw

PATCH v3 9/25 blockjob: add set_speed to BlockJobDriver

In reply to Vladimir Sementsov-Ogievskiy

We are going to use async block-copy call in backup, so we'll need to passthrough setting backup speed to block-copy call.

---
 include/block/blockjob_int.h | 2 ++
 blockjob.c                   | 6 ++++++
 2 files changed, 8 insertions(+)
include/block/blockjob_int.h +2 -0
index e2824a36a8..6633d83da2 100644
@@ -52,6 +52,8 @@ struct BlockJobDriver {
      * besides job->blk to the new AioContext.
      */
     void (*attached_aio_context)(BlockJob *job, AioContext *new_context);
+
+    void (*set_speed)(BlockJob *job, int64_t speed);
 };
 
 /**
blockjob.c +6 -0
index 98ac8af982..db3a21699c 100644
@@ -256,6 +256,7 @@ static bool job_timer_pending(Job *job)
 
 void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
 {
+    const BlockJobDriver *drv = block_job_driver(job);
     int64_t old_speed = job->speed;
 
     if (job_apply_verb(&job->job, JOB_VERB_SET_SPEED, errp)) {
@@ -270,6 +271,11 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)
     ratelimit_set_speed(&job->limit, speed, BLOCK_JOB_SLICE_TIME);
 
     job->speed = speed;
+
+    if (drv->set_speed) {
+        drv->set_speed(job, speed);
+    }
+
     if (speed && speed <= old_speed) {
         return;
     }
Signature · 1 line
2.21.3
#19 lore raw

PATCH v3 10/25 job: call job_enter from job_user_pause

In reply to Vladimir Sementsov-Ogievskiy

If main job coroutine called job_yield (while some background process is in progress), we should give it a chance to call job_pause_point(). It will be used in backup, when moved on async block-copy.

---
 job.c | 1 +
 1 file changed, 1 insertion(+)
job.c +1 -0
index 8fecf38960..50070ae6b1 100644
@@ -578,6 +578,7 @@ void job_user_pause(Job *job, Error **errp)
     }
     job->user_paused = true;
     job_pause(job);
+    job_enter(job);
 }
 
 bool job_user_paused(Job *job)
Signature · 1 line
2.21.3
#20 lore raw

PATCH v3 11/25 qapi: backup: add max-chunk and max-workers to x-perf struct

In reply to Vladimir Sementsov-Ogievskiy

Add new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior.

---
 qapi/block-core.json | 11 ++++++++++-
 block/backup.c       | 28 +++++++++++++++++++++++-----
 block/replication.c  |  2 +-
 blockdev.c           |  8 +++++++-
 4 files changed, 41 insertions(+), 8 deletions(-)
qapi/block-core.json +10 -1
index 71e6faa15c..5a21c24c1d 100644
@@ -1378,10 +1378,19 @@
 #
 # @use-copy-range: Use copy offloading. Default true.
 #
+# @max-workers: Maximum number of parallel requests for the sustained background
+#               copying process. Doesn't influence copy-before-write operations.
+#               Default 64.
+#
+# @max-chunk: Maximum request length for the sustained background copying
+#             process. Doesn't influence copy-before-write operations.
+#             0 means unlimited. Default 0.
+#
 # Since: 5.2
 ##
 { 'struct': 'BackupPerf',
-  'data': { '*use-copy-range': 'bool' }}
+  'data': { '*use-copy-range': 'bool',
+            '*max-workers': 'int', '*max-chunk': 'int64' } }
 
 ##
 # @BackupCommon:
block/backup.c +23 -5
index 09ff5a92ef..8c67d77504 100644
@@ -388,6 +388,29 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         return NULL;
     }
 
+    cluster_size = backup_calculate_cluster_size(target, errp);
+    if (cluster_size < 0) {
+        goto error;
+    }
+
+    if (perf->max_workers < 1) {
+        error_setg(errp, "max-worker must be greater than zero");
+        return NULL;
+    }
+
+    if (perf->max_chunk < 0) {
+        error_setg(errp, "max-chunk must be zero (which means no limit) or "
+                   "positive");
+        return NULL;
+    }
+
+    if (perf->max_chunk && perf->max_chunk < cluster_size) {
+        error_setg(errp, "Required max-chunk (%" PRIi64 ") is less than backup "
+                   "cluster size (%" PRIi64 ")", perf->max_chunk, cluster_size);
+        return NULL;
+    }
+
+
     if (sync_bitmap) {
         /* If we need to write to this bitmap, check that we can: */
         if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -420,11 +443,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
         goto error;
     }
 
-    cluster_size = backup_calculate_cluster_size(target, errp);
-    if (cluster_size < 0) {
-        goto error;
-    }
-
     /*
      * If source is in backing chain of target assume that target is going to be
      * used for "image fleecing", i.e. it should represent a kind of snapshot of
block/replication.c +1 -1
index 22ffc811ee..97be7ef4de 100644
@@ -454,7 +454,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,
     int64_t active_length, hidden_length, disk_length;
     AioContext *aio_context;
     Error *local_err = NULL;
-    BackupPerf perf = { .use_copy_range = true };
+    BackupPerf perf = { .use_copy_range = true, .max_workers = 1 };
 
     aio_context = bdrv_get_aio_context(bs);
     aio_context_acquire(aio_context);
blockdev.c +7 -1
index b71ed08a3b..0ed390abe0 100644
@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
 {
     BlockJob *job = NULL;
     BdrvDirtyBitmap *bmap = NULL;
-    BackupPerf perf = { .use_copy_range = true };
+    BackupPerf perf = { .use_copy_range = true, .max_workers = 64 };
     int job_flags = JOB_DEFAULT;
 
     if (!backup->has_speed) {
@@ -2817,6 +2817,12 @@ static BlockJob *do_backup_common(BackupCommon *backup,
         if (backup->x_perf->has_use_copy_range) {
             perf.use_copy_range = backup->x_perf->use_copy_range;
         }
+        if (backup->x_perf->has_max_workers) {
+            perf.max_workers = backup->x_perf->max_workers;
+        }
+        if (backup->x_perf->has_max_chunk) {
+            perf.max_chunk = backup->x_perf->max_chunk;
+        }
     }
 
     if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
Signature · 1 line
2.21.3
#21 lore raw

Re: [PATCH v3 11/25] qapi: backup: add max-chunk and max-workers to x-perf struct

Quoted 29 lines · Vladimir Sementsov-Ogievskiy

Add new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior.

---
  qapi/block-core.json | 11 ++++++++++-
  block/backup.c       | 28 +++++++++++++++++++++++-----
  block/replication.c  |  2 +-
  blockdev.c           |  8 +++++++-
  4 files changed, 41 insertions(+), 8 deletions(-)
qapi/block-core.json +7 -0
index 71e6faa15c..5a21c24c1d 100644
@@ -1378,10 +1378,19 @@
  #
  # @use-copy-range: Use copy offloading. Default true.
  #
+# @max-workers: Maximum number of parallel requests for the sustained background
+#               copying process. Doesn't influence copy-before-write operations.
+#               Default 64.
+#
+# @max-chunk: Maximum request length for the sustained background copying
+#             process. Doesn't influence copy-before-write operations.
+#             0 means unlimited. Default 0.

qapi/block-core.json line 1396

Quoted 10 lines
qapi/block-core.json +3 -1
+#
  # Since: 5.2
  ##
  { 'struct': 'BackupPerf',
-  'data': { '*use-copy-range': 'bool' }}
+  'data': { '*use-copy-range': 'bool',
+            '*max-workers': 'int', '*max-chunk': 'int64' } }
 
  ##
  # @BackupCommon:

squash-in:

qapi/block-core.json +3 -1
index 5591189b75..276ad5e6a8 100644
@@ -1406,7 +1406,9 @@
 #
 # @max-chunk: Maximum request length for the sustained background copying
 #             process. Doesn't influence copy-before-write operations.
-#             0 means unlimited. Default 0.
+#             0 means unlimited. If max-chunk is non-zero then it should not be
+#             less than job cluster size which is calculated as maximum of
+#             target image cluster size and 64k. Default 0.
 #
 # Since: 5.2
 ##
Signature · 2 lines
Best regards,
Vladimir
Reviewed-by on v3 11/25
#22 lore raw

Re: [PATCH v3 11/25] qapi: backup: add max-chunk and max-workers to x-perf struct

In reply to Vladimir Sementsov-Ogievskiy
Quoted 13 lines · Vladimir Sementsov-Ogievskiy

Add new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior.

---
  qapi/block-core.json | 11 ++++++++++-
  block/backup.c       | 28 +++++++++++++++++++++++-----
  block/replication.c  |  2 +-
  blockdev.c           |  8 +++++++-
  4 files changed, 41 insertions(+), 8 deletions(-)

[...]

Quoted 5 lines
block/backup.c +0 -0
index 09ff5a92ef..8c67d77504 100644

block/backup.c BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, · line 397

Quoted 10 lines
block/backup.c +7 -0
@@ -388,6 +388,29 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
          return NULL;
      }
 
+    cluster_size = backup_calculate_cluster_size(target, errp);
+    if (cluster_size < 0) {
+        goto error;
+    }
+
+    if (perf->max_workers < 1) {
+        error_setg(errp, "max-worker must be greater than zero");

*max-workers

With that fixed:

Quoted 2 lines
block/backup.c +2 -0
+        return NULL;
+    }
#23 lore raw

PATCH v3 12/25 iotests: 56: prepare for backup over block-copy

In reply to Vladimir Sementsov-Ogievskiy

After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, we'll have to wait for paused status, as we need to wait for several parallel request. So, let's gently wait instead of just asserting that job already paused.

---
 tests/qemu-iotests/056 | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
tests/qemu-iotests/056 +7 -2
index 052456aa00..e2978ba019 100755
@@ -307,8 +307,13 @@ class BackupTest(iotests.QMPTestCase):
         event = self.vm.event_wait(name="BLOCK_JOB_ERROR",
                                    match={'data': {'device': 'drive0'}})
         self.assertNotEqual(event, None)
-        # OK, job should be wedged
-        res = self.vm.qmp('query-block-jobs')
+        # OK, job should pause, but it can't do it immediately, as it can't
+        # cancel other parallel requests (which didn't fail)
+        with iotests.Timeout(60, "Timeout waiting for backup actually paused"):
+            while True:
+                res = self.vm.qmp('query-block-jobs')
+                if res['return'][0]['status'] == 'paused':
+                    break
         self.assert_qmp(res, 'return[0]/status', 'paused')
         res = self.vm.qmp('block-job-dismiss', id='drive0')
         self.assert_qmp(res, 'error/desc',
Signature · 1 line
2.21.3
Reviewed-by on v3 12/25
#24 lore raw

Re: [PATCH v3 12/25] iotests: 56: prepare for backup over block-copy

Quoted 9 lines · Vladimir Sementsov-Ogievskiy

After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, we'll have to wait for paused status, as we need to wait for several parallel request. So, let's gently wait instead of just asserting that job already paused.

---
  tests/qemu-iotests/056 | 9 +++++++--
  1 file changed, 7 insertions(+), 2 deletions(-)
#25 lore raw

PATCH v3 13/25 iotests: 129: prepare for backup over block-copy

In reply to Vladimir Sementsov-Ogievskiy

After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in do_test_stop. Let's require slow backup explicitly by specifying speed parameter.

---
 tests/qemu-iotests/129 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/129 +1 -1
index 3c22f6448c..e9da0208c4 100755
@@ -77,7 +77,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
     def test_drive_backup(self):
         self.do_test_stop("drive-backup", device="drive0",
                           target=self.target_img,
-                          sync="full")
+                          sync="full", speed=1024)
 
     def test_block_commit(self):
         self.do_test_stop("block-commit", device="drive0")
Signature · 1 line
2.21.3
Acked-by on v3 13/25
#26 lore raw

Re: [PATCH v3 13/25] iotests: 129: prepare for backup over block-copy

Quoted 15 lines · Vladimir Sementsov-Ogievskiy

After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in do_test_stop. Let's require slow backup explicitly by specifying speed parameter.

---
  tests/qemu-iotests/129 | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/129 +0 -0
index 3c22f6448c..e9da0208c4 100755

tests/qemu-iotests/129 class TestStopWithBlockJob(iotests.QMPTestCase): · line 83

Quoted 8 lines
tests/qemu-iotests/129 +1 -1
@@ -77,7 +77,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
      def test_drive_backup(self):
          self.do_test_stop("drive-backup", device="drive0",
                            target=self.target_img,
-                          sync="full")
+                          sync="full", speed=1024)
 
      def test_block_commit(self):
          self.do_test_stop("block-commit", device="drive0")

I still don’t like this very much. We write 128 MB of data, do_test_stop throttles the source to 1 kB/s, so even “a lot of parallel requests” shouldn’t finish immediately.

I feel like there’s a bigger problem here.

Then again, I have 129 disabled all the time anyway because its throttling simply doesn’t work, so I guess

#27 lore raw

Re: [PATCH v3 13/25] iotests: 129: prepare for backup over block-copy

Quoted 26 lines · Max Reitz
Quoted 15 lines · Vladimir Sementsov-Ogievskiy

After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in do_test_stop. Let's require slow backup explicitly by specifying speed parameter.

---
  tests/qemu-iotests/129 | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/129 +0 -0
index 3c22f6448c..e9da0208c4 100755
@@ -77,7 +77,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
Quoted 7 lines

      def test_drive_backup(self):
          self.do_test_stop("drive-backup", device="drive0", target=self.target_img, -                          sync="full")
+                          sync="full", speed=1024) def test_block_commit(self):
          self.do_test_stop("block-commit", device="drive0")

I still don’t like this very much.  We write 128 MB of data, do_test_stop throttles the source to 1 kB/s, so even “a lot of parallel requests” shouldn’t finish immediately.

Quoted 1 line

I feel like there’s a bigger problem here.

You were right!:) And finally I found it. Will fix in v4.

Quoted 5 lines

Then again, I have 129 disabled all the time anyway because its throttling simply doesn’t work, so I guess

Signature · 2 lines
Best regards,
Vladimir
#28 lore raw

PATCH v3 14/25 iotests: 185: prepare for backup over block-copy

In reply to Vladimir Sementsov-Ogievskiy

The further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. 185 test however assumes, that with speed limited to 64K, one iteration would result in offset=64K. It will change, as first iteration would result in offset=1M independently of speed.

So, let's explicitly specify, what test wants: set max-chunk to 64K, so that one iteration is 64K. Note, that we don't need to limit max-workers, as block-copy rate limiter will handle the situation and wouldn't start new workers when speed limit is obviously reached.

---
 tests/qemu-iotests/185     | 3 ++-
 tests/qemu-iotests/185.out | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/185 +2 -1
index fd5e6ebe11..0efadcbf62 100755
@@ -182,7 +182,8 @@ _send_qemu_cmd $h \
                       'target': '$TEST_IMG.copy',
                       'format': '$IMGFMT',
                       'sync': 'full',
-                      'speed': 65536 } }" \
+                      'speed': 65536,
+                      'x-perf': {'max-chunk': 65536} } }" \
     "return"
 
 # If we don't sleep here 'quit' command races with disk I/O
tests/qemu-iotests/185.out +1 -1
index 339438ac68..dd31d69cea 100644
@@ -61,7 +61,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off
 
 { 'execute': 'qmp_capabilities' }
 {"return": {}}
-{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } }
+{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536, 'x-perf': {'max-chunk': 65536} } }
 Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
Signature · 1 line
2.21.3
#29 lore raw

PATCH v3 15/25 iotests: 219: prepare for backup over block-copy

In reply to Vladimir Sementsov-Ogievskiy

The further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. Test 219 depends on specified chunk-size. Update it for explicit chunk-size for backup as for mirror.

---
 tests/qemu-iotests/219 | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
tests/qemu-iotests/219 +7 -6
index db272c5249..d7b177bf09 100755
@@ -203,13 +203,13 @@ with iotests.FilePath('disk.img') as disk_path, \
     # but related to this also automatic state transitions like job
     # completion), but still get pause points often enough to avoid making this
     # test very slow, it's important to have the right ratio between speed and
-    # buf_size.
+    # copy-chunk-size.
     #
-    # For backup, buf_size is hard-coded to the source image cluster size (64k),
-    # so we'll pick the same for mirror. The slice time, i.e. the granularity
-    # of the rate limiting is 100ms. With a speed of 256k per second, we can
-    # get four pause points per second. This gives us 250ms per iteration,
-    # which should be enough to stay deterministic.
+    # Chose 64k copy-chunk-size both for mirror (by buf_size) and backup (by
+    # x-max-chunk). The slice time, i.e. the granularity of the rate limiting
+    # is 100ms. With a speed of 256k per second, we can get four pause points
+    # per second. This gives us 250ms per iteration, which should be enough to
+    # stay deterministic.
 
     test_job_lifecycle(vm, 'drive-mirror', has_ready=True, job_args={
         'device': 'drive0-node',
@@ -226,6 +226,7 @@ with iotests.FilePath('disk.img') as disk_path, \
                 'target': copy_path,
                 'sync': 'full',
                 'speed': 262144,
+                'x-perf': {'max-chunk': 65536},
                 'auto-finalize': auto_finalize,
                 'auto-dismiss': auto_dismiss,
             })
Signature · 1 line
2.21.3
Reviewed-by on v3 15/25
#30 lore raw

Re: [PATCH v3 15/25] iotests: 219: prepare for backup over block-copy

Quoted 10 lines · Vladimir Sementsov-Ogievskiy

The further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. Test 219 depends on specified chunk-size. Update it for explicit chunk-size for backup as for mirror.

---
  tests/qemu-iotests/219 | 13 +++++++------
  1 file changed, 7 insertions(+), 6 deletions(-)
#31 lore raw

PATCH v3 16/25 iotests: 257: prepare for backup over block-copy

In reply to Vladimir Sementsov-Ogievskiy

Iotest 257 dumps a lot of in-progress information of backup job, such as offset and bitmap dirtiness. Further commit will move backup to be one block-copy call, which will introduce async parallel requests instead of plain cluster-by-cluster copying. To keep things deterministic, allow only one worker (only one copy request at a time) for this test.

---
 tests/qemu-iotests/257     |   1 +
 tests/qemu-iotests/257.out | 306 ++++++++++++++++++-------------------
 2 files changed, 154 insertions(+), 153 deletions(-)
tests/qemu-iotests/257 +1 -0
index c80e06ae28..0b37ed7708 100755
@@ -191,6 +191,7 @@ def blockdev_backup(vm, device, target, sync, **kwargs):
                         target=target,
                         sync=sync,
                         filter_node_name='backup-top',
+                        x_perf={'max-workers': 1},
                         **kwargs)
     return result
 
tests/qemu-iotests/257.out +153 -153

1,043 lines not on this page · raw message

Signature · 1 line
2.21.3
#32 lore raw

PATCH v3 17/25 block/block-copy: make progress_bytes_callback optional

In reply to Vladimir Sementsov-Ogievskiy

We are going to stop use of this callback in the following commit. Still the callback handling code will be dropped in a separate commit. So, for now let's make it optional.

---
 block/block-copy.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
block/block-copy.c +3 -1
index 82cf945693..61d82d9a1c 100644
@@ -454,7 +454,9 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
         t->call_state->error_is_read = error_is_read;
     } else {
         progress_work_done(t->s->progress, t->bytes);
-        t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque);
+        if (t->s->progress_bytes_callback) {
+            t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque);
+        }
     }
     co_put_to_shres(t->s->mem, t->bytes);
     block_copy_task_end(t, ret);
Signature · 1 line
2.21.3
Reviewed-by on v3 17/25
#33 lore raw

Re: [PATCH v3 17/25] block/block-copy: make progress_bytes_callback optional

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

We are going to stop use of this callback in the following commit. Still the callback handling code will be dropped in a separate commit. So, for now let's make it optional.

---
  block/block-copy.c | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)
#34 lore raw

PATCH v3 18/25 block/backup: drop extra gotos from backup_run()

In reply to Vladimir Sementsov-Ogievskiy
---
 block/backup.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)
block/backup.c +5 -7
index 8c67d77504..449b763ce4 100644
@@ -236,7 +236,7 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job)
 static int coroutine_fn backup_run(Job *job, Error **errp)
 {
     BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
-    int ret = 0;
+    int ret;
 
     backup_init_bcs_bitmap(s);
 
@@ -246,13 +246,12 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
 
         for (offset = 0; offset < s->len; ) {
             if (yield_and_check(s)) {
-                ret = -ECANCELED;
-                goto out;
+                return -ECANCELED;
             }
 
             ret = block_copy_reset_unallocated(s->bcs, offset, &count);
             if (ret < 0) {
-                goto out;
+                return ret;
             }
 
             offset += count;
@@ -273,11 +272,10 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
             job_yield(job);
         }
     } else {
-        ret = backup_loop(s);
+        return backup_loop(s);
     }
 
- out:
-    return ret;
+    return 0;
 }
 
 static const BlockJobDriver backup_job_driver = {
Signature · 1 line
2.21.3
#36 lore raw

PATCH v3 19/25 backup: move to block-copy

In reply to Vladimir Sementsov-Ogievskiy

This brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance.

---
 block/backup.c | 187 ++++++++++++++++++++++++++++++++-----------------
 1 file changed, 121 insertions(+), 66 deletions(-)
block/backup.c +121 -66
index 449b763ce4..0d1cd64eab 100644
@@ -44,41 +44,17 @@ typedef struct BackupBlockJob {
     BlockdevOnError on_source_error;
     BlockdevOnError on_target_error;
     uint64_t len;
-    uint64_t bytes_read;
     int64_t cluster_size;
     BackupPerf perf;
 
     BlockCopyState *bcs;
+
+    bool wait;
+    BlockCopyCallState *bg_bcs_call;
 } BackupBlockJob;
 
 static const BlockJobDriver backup_job_driver;
 
-static void backup_progress_bytes_callback(int64_t bytes, void *opaque)
-{
-    BackupBlockJob *s = opaque;
-
-    s->bytes_read += bytes;
-}
-
-static int coroutine_fn backup_do_cow(BackupBlockJob *job,
-                                      int64_t offset, uint64_t bytes,
-                                      bool *error_is_read)
-{
-    int ret = 0;
-    int64_t start, end; /* bytes */
-
-    start = QEMU_ALIGN_DOWN(offset, job->cluster_size);
-    end = QEMU_ALIGN_UP(bytes + offset, job->cluster_size);
-
-    trace_backup_do_cow_enter(job, start, offset, bytes);
-
-    ret = block_copy(job->bcs, start, end - start, true, error_is_read);
-
-    trace_backup_do_cow_return(job, offset, bytes, ret);
-
-    return ret;
-}
-
 static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
 {
     BdrvDirtyBitmap *bm;
@@ -158,53 +134,97 @@ static BlockErrorAction backup_error_action(BackupBlockJob *job,
     }
 }
 
-static bool coroutine_fn yield_and_check(BackupBlockJob *job)
+static void coroutine_fn backup_block_copy_callback(void *opaque)
 {
-    uint64_t delay_ns;
-
-    if (job_is_cancelled(&job->common.job)) {
-        return true;
-    }
-
-    /*
-     * We need to yield even for delay_ns = 0 so that bdrv_drain_all() can
-     * return. Without a yield, the VM would not reboot.
-     */
-    delay_ns = block_job_ratelimit_get_delay(&job->common, job->bytes_read);
-    job->bytes_read = 0;
-    job_sleep_ns(&job->common.job, delay_ns);
+    BackupBlockJob *s = opaque;
 
-    if (job_is_cancelled(&job->common.job)) {
-        return true;
+    if (s->wait) {
+        s->wait = false;
+        aio_co_wake(s->common.job.co);
+    } else {
+        job_enter(&s->common.job);
     }
-
-    return false;
 }
 
 static int coroutine_fn backup_loop(BackupBlockJob *job)
 {
-    bool error_is_read;
-    int64_t offset;
-    BdrvDirtyBitmapIter *bdbi;
+    BlockCopyCallState *s = NULL;
     int ret = 0;
+    bool error_is_read;
+    BlockErrorAction act;
+
+    while (true) { /* retry loop */
+        job->bg_bcs_call = s = block_copy_async(job->bcs, 0,
+                QEMU_ALIGN_UP(job->len, job->cluster_size),
+                job->perf.max_workers, job->perf.max_chunk,
+                backup_block_copy_callback, job);
+
+        while (!block_copy_call_finished(s) &&
+               !job_is_cancelled(&job->common.job))
+        {
+            job_yield(&job->common.job);
+        }
 
-    bdbi = bdrv_dirty_iter_new(block_copy_dirty_bitmap(job->bcs));
-    while ((offset = bdrv_dirty_iter_next(bdbi)) != -1) {
-        do {
-            if (yield_and_check(job)) {
-                goto out;
-            }
-            ret = backup_do_cow(job, offset, job->cluster_size, &error_is_read);
-            if (ret < 0 && backup_error_action(job, error_is_read, -ret) ==
-                           BLOCK_ERROR_ACTION_REPORT)
-            {
-                goto out;
-            }
-        } while (ret < 0);
+        if (!block_copy_call_finished(s) &&
+            job_is_cancelled(&job->common.job))
+        {
+            /*
+             * Note that we can't use job_yield() here, as it doesn't work for
+             * cancelled job.
+             */
+            block_copy_call_cancel(s);
+            job->wait = true;
+            qemu_coroutine_yield();
+            assert(block_copy_call_finished(s));
+            ret = 0;
+            goto out;
+        }
+
+        if (job_is_cancelled(&job->common.job) ||
+            block_copy_call_succeeded(s))
+        {
+            ret = 0;
+            goto out;
+        }
+
+        if (block_copy_call_cancelled(s)) {
+            /*
+             * Job is not cancelled but only block-copy call. This is possible
+             * after job pause. Now the pause is finished, start new block-copy
+             * iteration.
+             */
+            block_copy_call_free(s);
+            continue;
+        }
+
+        /* The only remaining case is failed block-copy call. */
+        assert(block_copy_call_failed(s));
+
+        ret = block_copy_call_status(s, &error_is_read);
+        act = backup_error_action(job, error_is_read, -ret);
+        switch (act) {
+        case BLOCK_ERROR_ACTION_REPORT:
+            goto out;
+        case BLOCK_ERROR_ACTION_STOP:
+            /*
+             * Go to pause prior to starting new block-copy call on the next
+             * iteration.
+             */
+            job_pause_point(&job->common.job);
+            break;
+        case BLOCK_ERROR_ACTION_IGNORE:
+            /* Proceed to new block-copy call to retry. */
+            break;
+        default:
+            abort();
+        }
+
+        block_copy_call_free(s);
     }
 
- out:
-    bdrv_dirty_iter_free(bdbi);
+out:
+    block_copy_call_free(s);
+    job->bg_bcs_call = NULL;
     return ret;
 }
 
@@ -245,7 +265,13 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
         int64_t count;
 
         for (offset = 0; offset < s->len; ) {
-            if (yield_and_check(s)) {
+            if (job_is_cancelled(job)) {
+                return -ECANCELED;
+            }
+
+            job_pause_point(job);
+
+            if (job_is_cancelled(job)) {
                 return -ECANCELED;
             }
 
@@ -278,6 +304,33 @@ static int coroutine_fn backup_run(Job *job, Error **errp)
     return 0;
 }
 
+static void coroutine_fn backup_pause(Job *job)
+{
+    BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
+
+    if (s->bg_bcs_call && !block_copy_call_finished(s->bg_bcs_call)) {
+        block_copy_call_cancel(s->bg_bcs_call);
+        s->wait = true;
+        qemu_coroutine_yield();
+    }
+}
+
+static void coroutine_fn backup_set_speed(BlockJob *job, int64_t speed)
+{
+    BackupBlockJob *s = container_of(job, BackupBlockJob, common);
+
+    /*
+     * block_job_set_speed() is called first from block_job_create(), when we
+     * don't yet have s->bcs.
+     */
+    if (s->bcs) {
+        block_copy_set_speed(s->bcs, speed);
+        if (s->bg_bcs_call) {
+            block_copy_kick(s->bg_bcs_call);
+        }
+    }
+}
+
 static const BlockJobDriver backup_job_driver = {
     .job_driver = {
         .instance_size          = sizeof(BackupBlockJob),
@@ -288,7 +341,9 @@ static const BlockJobDriver backup_job_driver = {
         .commit                 = backup_commit,
         .abort                  = backup_abort,
         .clean                  = backup_clean,
-    }
+        .pause                  = backup_pause,
+    },
+    .set_speed = backup_set_speed,
 };
 
 static int64_t backup_calculate_cluster_size(BlockDriverState *target,
@@ -485,8 +540,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
     job->len = len;
     job->perf = *perf;
 
-    block_copy_set_progress_callback(bcs, backup_progress_bytes_callback, job);
     block_copy_set_progress_meter(bcs, &job->common.job.progress);
+    block_copy_set_speed(bcs, speed);
 
     /* Required permissions are already taken by backup-top target */
     block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
Signature · 1 line
2.21.3
Reviewed-by on v3 19/25
#37 lore raw

Re: [PATCH v3 19/25] backup: move to block-copy

Quoted 7 lines · Vladimir Sementsov-Ogievskiy

This brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance.

---
  block/backup.c | 187 ++++++++++++++++++++++++++++++++-----------------
  1 file changed, 121 insertions(+), 66 deletions(-)

Irrelevant notes below.

block/backup.c

Quoted 4 lines
block/backup.c +0 -0
index 449b763ce4..0d1cd64eab 100644

(Just something I noted when looking for remaining instances of “ratelimit”: backup.c includes ratelimit.h, which I don’t think it needs to do any longer)

[...]

Quoted 2 lines
  static int coroutine_fn backup_loop(BackupBlockJob *job)
  {

[...]

block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 170

Quoted 3 lines
block/backup.c +3 -0
+        if (!block_copy_call_finished(s) &&
+            job_is_cancelled(&job->common.job))
+        {

Just for the sake of clarity: If !block_copy_call_finished(), then job_is_cancelled() must be true, right?

(I.e., could be an assertion instead of the second part of the condition. I don’t mind how it is, but then again, it did made me ask.)

Quoted 17 lines
block/backup.c +17 -0
+            /*
+             * Note that we can't use job_yield() here, as it doesn't work for
+             * cancelled job.
+             */
+            block_copy_call_cancel(s);
+            job->wait = true;
+            qemu_coroutine_yield();
+            assert(block_copy_call_finished(s));
+            ret = 0;
+            goto out;
+        }
+
+        if (job_is_cancelled(&job->common.job) ||
+            block_copy_call_succeeded(s))
+        {
+            ret = 0;
+            goto out;

block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 197

Quoted 10 lines
block/backup.c +10 -0
+        }
+
+        if (block_copy_call_cancelled(s)) {
+            /*
+             * Job is not cancelled but only block-copy call. This is possible
+             * after job pause. Now the pause is finished, start new block-copy
+             * iteration.
+             */
+            block_copy_call_free(s);
+            continue;

If one were to avoid all continues, we could alternatively put the whole error handling into a block_copy_call_failed() condition, and have the block_copy_call_free() common for both the cancel and the fail case.

*shrug*

Quoted 21 lines
block/backup.c +21 -0
+        }
+
+        /* The only remaining case is failed block-copy call. */
+        assert(block_copy_call_failed(s));
+
+        ret = block_copy_call_status(s, &error_is_read);
+        act = backup_error_action(job, error_is_read, -ret);
+        switch (act) {
+        case BLOCK_ERROR_ACTION_REPORT:
+            goto out;
+        case BLOCK_ERROR_ACTION_STOP:
+            /*
+             * Go to pause prior to starting new block-copy call on the next
+             * iteration.
+             */
+            job_pause_point(&job->common.job);
+            break;
+        case BLOCK_ERROR_ACTION_IGNORE:
+            /* Proceed to new block-copy call to retry. */
+            break;
+        default:

block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 226

Quoted 10 lines
block/backup.c +6 -2
+            abort();
+        }
+
+        block_copy_call_free(s);
      }
 
- out:
-    bdrv_dirty_iter_free(bdbi);
+out:
+    block_copy_call_free(s);

Reads a bit weird after the block_copy_call_free(s) at the end of the while (true) loop, but is entirely correct, of course. (And I can’t offer any better alternative.)

Max

Quoted 3 lines
+    job->bg_bcs_call = NULL;
      return ret;

}

#38 lore raw

Re: [PATCH v3 19/25] backup: move to block-copy

Quoted 10 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

This brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance.

---
  block/backup.c | 187 ++++++++++++++++++++++++++++++++-----------------

  1 file changed, 121 insertions(+), 66 deletions(-)

Thanks a lot!

My way to this point was started in far 2016 summer.. And final concept is far-far away from my first attempt:)

Quoted 9 lines

Irrelevant notes below.

block/backup.c

Quoted 4 lines
block/backup.c +0 -0
index 449b763ce4..0d1cd64eab 100644

(Just something I noted when looking for remaining instances of “ratelimit”: backup.c includes ratelimit.h, which I don’t think it needs to do any longer)

agree

Quoted 8 lines

[...]

Quoted 2 lines

  static int coroutine_fn backup_loop(BackupBlockJob *job) {

[...]

Quoted 7 lines
Quoted 3 lines

+        if (!block_copy_call_finished(s) &&
+            job_is_cancelled(&job->common.job))
+        {

Just for the sake of clarity: If !block_copy_call_finished(), then job_is_cancelled() must be true, right?

(I.e., could be an assertion instead of the second part of the condition.  I don’t mind how it is, but then again, it did made me ask.)

Hmm, keeping in mind previous loop "while (!block_copy_call_finished(s) && !job_is_cancelled(&job->common.job)) {}", yes, you are right.

Quoted 31 lines
Quoted 18 lines

+            /*
+             * Note that we can't use job_yield() here, as it doesn't work for +             * cancelled job.
+             */
+            block_copy_call_cancel(s);
+            job->wait = true;
+            qemu_coroutine_yield();
+            assert(block_copy_call_finished(s));
+            ret = 0;
+            goto out;
+        }

block/backup.c +6 -0
+
+        if (job_is_cancelled(&job->common.job) ||
+            block_copy_call_succeeded(s))
+        {
+            ret = 0;
+            goto out;

block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 197

Quoted 10 lines
block/backup.c +10 -0
+        }
+
+        if (block_copy_call_cancelled(s)) {
+            /*
+             * Job is not cancelled but only block-copy call. This is possible
+             * after job pause. Now the pause is finished, start new block-copy
+             * iteration.
+             */
+            block_copy_call_free(s);
+            continue;

If one were to avoid all continues, we could alternatively put the whole error handling into a block_copy_call_failed() condition, and have the block_copy_call_free() common for both the cancel and the fail case.

Quoted 1 line

*shrug*

Hmm. Same emotion, *shrug*:). I like my comment, describing exactly block_copy_call_cancelled() in corresponding if.. Being combined with failure case it would be less demonstrative. But I'm OK with your suggestion too.

Quoted 25 lines
Quoted 25 lines

+        }

block/backup.c +23 -0
+
+        /* The only remaining case is failed block-copy call. */
+        assert(block_copy_call_failed(s));
+
+        ret = block_copy_call_status(s, &error_is_read);
+        act = backup_error_action(job, error_is_read, -ret);
+        switch (act) {
+        case BLOCK_ERROR_ACTION_REPORT:
+            goto out;
+        case BLOCK_ERROR_ACTION_STOP:
+            /*
+             * Go to pause prior to starting new block-copy call on the next
+             * iteration.
+             */
+            job_pause_point(&job->common.job);
+            break;
+        case BLOCK_ERROR_ACTION_IGNORE:
+            /* Proceed to new block-copy call to retry. */
+            break;
+        default:
+            abort();
+        }
+
Quoted 9 lines
Quoted 6 lines

+        block_copy_call_free(s);
      }
- out:
-    bdrv_dirty_iter_free(bdbi);
+out:
+    block_copy_call_free(s);

Reads a bit weird after the block_copy_call_free(s) at the end of the while (true) loop, but is entirely correct, of course.  (And I can’t offer any better alternative.)

Yes, looks weird.. Probably we'll invent some good refactoring later. If the function doesn't fit into one screen, there must exist some good refactoring)

Signature · 2 lines
Best regards,
Vladimir
#39 lore raw

PATCH v3 20/25 qapi: backup: disable copy_range by default

In reply to Vladimir Sementsov-Ogievskiy

Further commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabled copy_range. So, let's disable copy_range by default.

---
 qapi/block-core.json | 2 +-
 blockdev.c           | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
qapi/block-core.json +1 -1
index 5a21c24c1d..58eb2bcb86 100644
@@ -1376,7 +1376,7 @@
 # Optional parameters for backup. These parameters don't affect
 # functionality, but may significantly affect performance.
 #
-# @use-copy-range: Use copy offloading. Default true.
+# @use-copy-range: Use copy offloading. Default false.
 #
 # @max-workers: Maximum number of parallel requests for the sustained background
 #               copying process. Doesn't influence copy-before-write operations.
blockdev.c +1 -1
index 0ed390abe0..1ac64d8ee2 100644
@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
 {
     BlockJob *job = NULL;
     BdrvDirtyBitmap *bmap = NULL;
-    BackupPerf perf = { .use_copy_range = true, .max_workers = 64 };
+    BackupPerf perf = { .max_workers = 64 };
     int job_flags = JOB_DEFAULT;
 
     if (!backup->has_speed) {
Signature · 1 line
2.21.3
#40 lore raw

Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default

Quoted 10 lines · Vladimir Sementsov-Ogievskiy

Further commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabled copy_range. So, let's disable copy_range by default.

---
  qapi/block-core.json | 2 +-
  blockdev.c           | 2 +-
  2 files changed, 2 insertions(+), 2 deletions(-)

Uh, well, er, then why did you add it as true in patch 2?

Do you mean this patch as the basis for a discussion whether it should be true or false by default? I don’t have anything to contribute, though, ergo I don’t mind either way.

Do you have an idea why copy offloading isn’t better?

Max

Quoted 27 lines
qapi/block-core.json +1 -1
index 5a21c24c1d..58eb2bcb86 100644
@@ -1376,7 +1376,7 @@
  # Optional parameters for backup. These parameters don't affect
  # functionality, but may significantly affect performance.
  #
-# @use-copy-range: Use copy offloading. Default true.
+# @use-copy-range: Use copy offloading. Default false.
  #
  # @max-workers: Maximum number of parallel requests for the sustained background
  #               copying process. Doesn't influence copy-before-write operations.
blockdev.c +1 -1
index 0ed390abe0..1ac64d8ee2 100644
@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
  {
      BlockJob *job = NULL;
      BdrvDirtyBitmap *bmap = NULL;
-    BackupPerf perf = { .use_copy_range = true, .max_workers = 64 };
+    BackupPerf perf = { .max_workers = 64 };
      int job_flags = JOB_DEFAULT;
 
      if (!backup->has_speed) {
#41 lore raw

Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default

Quoted 12 lines · Max Reitz
Quoted 10 lines · Vladimir Sementsov-Ogievskiy

Further commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabled copy_range. So, let's disable copy_range by default.

---
  qapi/block-core.json | 2 +-
  blockdev.c           | 2 +-

  2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 3 lines

Uh, well, er, then why did you add it as true in patch 2?

Do you mean this patch as the basis for a discussion whether it should be true or false by default?

Yes, I just kept the change in separate, to make it simpler to discuss.. Hmm, or seems I've tried to keep it close to the point when we are going to increase performance anyway (and making default false is needed for good final performance), when if we do the change earlier it would like a degradation during previous 20 commits.

Quoted 4 lines

I don’t have anything to contribute, though, ergo I don’t mind either way.

Do you have an idea why copy offloading isn’t better?

As I understand, it may be really fast for filesystem which will avoid data copying, like btrfs. Why it works slower for others when we enable async + block-status things, I don't know. It should depend on kernel implementation.

I have two additional thoughts:

1. for backup we usually want to do a copy on another node, or at least another hard drive, so we really going to copy the data, not just create some metadata links in fs.

2. even if fs can just create some links to existing clusters instead of copying data, IMHO it is not good for backup:

Main difference of backup from snapshot is that we don't touch active disk: it works in the same way like without backup job running. But if we do create fs-metadata links in backup image to original clusters, it means that on next guest write fs will have to do COW operation, so backup file will reference old cluster from active disk, and new cluster will be allocated for active disk. This means, that backup job influence on active vm disk, increasing its fragmentation.. Possibly I'm wrong, I'm not really good in how filesystems works.

Quoted 28 lines
Quoted 27 lines
qapi/block-core.json +1 -1
index 5a21c24c1d..58eb2bcb86 100644
@@ -1376,7 +1376,7 @@
   # Optional parameters for backup. These parameters don't affect
   # functionality, but may significantly affect performance.
   #
-# @use-copy-range: Use copy offloading. Default true.
+# @use-copy-range: Use copy offloading. Default false.
   #
   # @max-workers: Maximum number of parallel requests for the sustained background
   #               copying process. Doesn't influence copy-before-write operations.
blockdev.c +1 -1
index 0ed390abe0..1ac64d8ee2 100644
@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
   {
       BlockJob *job = NULL;
       BdrvDirtyBitmap *bmap = NULL;
-    BackupPerf perf = { .use_copy_range = true, .max_workers = 64 };
+    BackupPerf perf = { .max_workers = 64 };
       int job_flags = JOB_DEFAULT;
       if (!backup->has_speed) {
 
Signature · 2 lines
Best regards,
Vladimir
Reviewed-by
#42 lore raw

Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default

Quoted 13 lines · Vladimir Sementsov-Ogievskiy
Quoted 12 lines · Max Reitz
Quoted 10 lines · Vladimir Sementsov-Ogievskiy

Further commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabled copy_range. So, let's disable copy_range by default.

---
  qapi/block-core.json | 2 +-
  blockdev.c           | 2 +-

  2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 3 lines

Uh, well, er, then why did you add it as true in patch 2?

Do you mean this patch as the basis for a discussion whether it should be true or false by default?

Yes, I just kept the change in separate, to make it simpler to discuss.. Hmm, or seems I've tried to keep it close to the point when we are going to increase performance anyway (and making default false is needed for good final performance), when if we do the change earlier it would like a degradation during previous 20 commits.

Hm, yes, before this series, copy_range is effectively the default, right.

Quoted 7 lines
Quoted 4 lines

I don’t have anything to contribute, though, ergo I don’t mind either way.

Do you have an idea why copy offloading isn’t better?

As I understand, it may be really fast for filesystem which will avoid data copying, like btrfs. Why it works slower for others when we enable async + block-status things, I don't know. It should depend on kernel implementation.

Quoted 7 lines

I have two additional thoughts:

1. for backup we usually want to do a copy on another node, or at least another hard drive, so we really going to copy the data, not just create some metadata links in fs.

2. even if fs can just create some links to existing clusters instead of copying data, IMHO it is not good for backup:

Main difference of backup from snapshot is that we don't touch active disk: it works in the same way like without backup job running. But if we do create fs-metadata links in backup image to original clusters, it means that on next guest write fs will have to do COW operation, so backup file will reference old cluster from active disk, and new cluster will be allocated for active disk. This means, that backup job influence on active vm disk, increasing its fragmentation.. Possibly I'm wrong, I'm not really good in how filesystems works.

Both sound like good points to me. So as far as I’m concerned, disabling copy offloading sounds good to me.

So now this separate patch here makes sense to me:

Perhaps it deserves some explanation in a commit message, though. I.e., that originally, this defaulted to true, as it was before, but now that async copying is implemented, we should let it default to false.

Max

#43 lore raw

PATCH v3 21/25 block/block-copy: drop unused block_copy_set_progress_callback()

In reply to Vladimir Sementsov-Ogievskiy

Drop unused code.

---
 include/block/block-copy.h |  6 ------
 block/block-copy.c         | 15 ---------------
 2 files changed, 21 deletions(-)
include/block/block-copy.h +0 -6
index 8f5013d0aa..7b77abf70b 100644
@@ -18,7 +18,6 @@
 #include "block/block.h"
 #include "qemu/co-shared-resource.h"
 
-typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);
 typedef void (*BlockCopyAsyncCallbackFunc)(void *opaque);
 typedef struct BlockCopyState BlockCopyState;
 typedef struct BlockCopyCallState BlockCopyCallState;
@@ -28,11 +27,6 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
                                      BdrvRequestFlags write_flags,
                                      Error **errp);
 
-void block_copy_set_progress_callback(
-        BlockCopyState *s,
-        ProgressBytesCallbackFunc progress_bytes_callback,
-        void *progress_opaque);
-
 void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm);
 
 void block_copy_state_free(BlockCopyState *s);
block/block-copy.c +0 -15
index 61d82d9a1c..2ea8b28684 100644
@@ -110,9 +110,6 @@ typedef struct BlockCopyState {
     bool skip_unallocated;
 
     ProgressMeter *progress;
-    /* progress_bytes_callback: called when some copying progress is done. */
-    ProgressBytesCallbackFunc progress_bytes_callback;
-    void *progress_opaque;
 
     SharedResource *mem;
 
@@ -298,15 +295,6 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,
     return s;
 }
 
-void block_copy_set_progress_callback(
-        BlockCopyState *s,
-        ProgressBytesCallbackFunc progress_bytes_callback,
-        void *progress_opaque)
-{
-    s->progress_bytes_callback = progress_bytes_callback;
-    s->progress_opaque = progress_opaque;
-}
-
 void block_copy_set_progress_meter(BlockCopyState *s, ProgressMeter *pm)
 {
     s->progress = pm;
@@ -454,9 +442,6 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
         t->call_state->error_is_read = error_is_read;
     } else {
         progress_work_done(t->s->progress, t->bytes);
-        if (t->s->progress_bytes_callback) {
-            t->s->progress_bytes_callback(t->bytes, t->s->progress_opaque);
-        }
     }
     co_put_to_shres(t->s->mem, t->bytes);
     block_copy_task_end(t, ret);
Signature · 1 line
2.21.3
#45 lore raw

PATCH v3 22/25 block/block-copy: drop unused argument of block_copy()

In reply to Vladimir Sementsov-Ogievskiy
---
 include/block/block-copy.h |  2 +-
 block/backup-top.c         |  2 +-
 block/block-copy.c         | 10 ++--------
 3 files changed, 4 insertions(+), 10 deletions(-)
include/block/block-copy.h +1 -1
index 7b77abf70b..f731dbd573 100644
@@ -35,7 +35,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,
                                      int64_t offset, int64_t *count);
 
 int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,
-                            bool ignore_ratelimit, bool *error_is_read);
+                            bool ignore_ratelimit);
 
 /*
  * Run block-copy in a coroutine, create corresponding BlockCopyCallState
block/backup-top.c +1 -1
index 779956ddc2..6e7e7bf340 100644
@@ -61,7 +61,7 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset,
     off = QEMU_ALIGN_DOWN(offset, s->cluster_size);
     end = QEMU_ALIGN_UP(offset + bytes, s->cluster_size);
 
-    return block_copy(s->bcs, off, end - off, true, NULL);
+    return block_copy(s->bcs, off, end - off, true);
 }
 
 static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
block/block-copy.c +2 -8
index 2ea8b28684..39ae481c8b 100644
@@ -723,7 +723,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)
 }
 
 int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
-                            bool ignore_ratelimit, bool *error_is_read)
+                            bool ignore_ratelimit)
 {
     BlockCopyCallState call_state = {
         .s = s,
@@ -733,13 +733,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes,
         .max_workers = BLOCK_COPY_MAX_WORKERS,
     };
 
-    int ret = block_copy_common(&call_state);
-
-    if (error_is_read && ret < 0) {
-        *error_is_read = call_state.error_is_read;
-    }
-
-    return ret;
+    return block_copy_common(&call_state);
 }
 
 static void coroutine_fn block_copy_async_co_entry(void *opaque)
Signature · 1 line
2.21.3
#46 lore raw

PATCH v3 23/25 simplebench/bench_block_job: use correct shebang line with python3

In reply to Vladimir Sementsov-Ogievskiy
---
 scripts/simplebench/bench_block_job.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/simplebench/bench_block_job.py +1 -1
index 9808d696cf..a0dda1dc4e 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 #
 # Benchmark block jobs
 #
Signature · 1 line
2.21.3
#48 lore raw

PATCH v3 24/25 simplebench: bench_block_job: add cmd_options argument

In reply to Vladimir Sementsov-Ogievskiy

Add argument to allow additional block-job options.

---
 scripts/simplebench/bench-example.py   |  2 +-
 scripts/simplebench/bench_block_job.py | 11 +++++++----
 2 files changed, 8 insertions(+), 5 deletions(-)
scripts/simplebench/bench-example.py +1 -1
index d9c7f7bc17..4864435f39 100644
@@ -25,7 +25,7 @@ from bench_block_job import bench_block_copy, drv_file, drv_nbd
 
 def bench_func(env, case):
     """ Handle one "cell" of benchmarking table. """
-    return bench_block_copy(env['qemu_binary'], env['cmd'],
+    return bench_block_copy(env['qemu_binary'], env['cmd'], {}
                             case['source'], case['target'])
 
 
scripts/simplebench/bench_block_job.py +7 -4
index a0dda1dc4e..7332845c1c 100755
@@ -78,16 +78,19 @@ def bench_block_job(cmd, cmd_args, qemu_args):
 
 
 # Bench backup or mirror
-def bench_block_copy(qemu_binary, cmd, source, target):
+def bench_block_copy(qemu_binary, cmd, cmd_options, source, target):
     """Helper to run bench_block_job() for mirror or backup"""
     assert cmd in ('blockdev-backup', 'blockdev-mirror')
 
     source['node-name'] = 'source'
     target['node-name'] = 'target'
 
-    return bench_block_job(cmd,
-                           {'job-id': 'job0', 'device': 'source',
-                            'target': 'target', 'sync': 'full'},
+    cmd_options['job-id'] = 'job0'
+    cmd_options['device'] = 'source'
+    cmd_options['target'] = 'target'
+    cmd_options['sync'] = 'full'
+
+    return bench_block_job(cmd, cmd_options,
                            [qemu_binary,
                             '-blockdev', json.dumps(source),
                             '-blockdev', json.dumps(target)])
Signature · 1 line
2.21.3
#49 lore raw

PATCH v3 25/25 simplebench: add bench-backup.py

In reply to Vladimir Sementsov-Ogievskiy

Add script to benchmark new backup architecture.

---
 scripts/simplebench/bench-backup.py | 165 ++++++++++++++++++++++++++++
 1 file changed, 165 insertions(+)
 create mode 100755 scripts/simplebench/bench-backup.py
scripts/simplebench/bench-backup.py new file +165 -0
new file mode 100755
index 0000000000..5c62b1a7ed
@@ -0,0 +1,165 @@
+#!/usr/bin/env python3
+#
+# Bench backup block-job
+#
+# Copyright (c) 2020 Virtuozzo International GmbH.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+import argparse
+import json
+
+import simplebench
+from results_to_text import results_to_text
+from bench_block_job import bench_block_copy, drv_file, drv_nbd
+
+
+def bench_func(env, case):
+    """ Handle one "cell" of benchmarking table. """
+    cmd_options = env['cmd-options'] if 'cmd-options' in env else {}
+    return bench_block_copy(env['qemu-binary'], env['cmd'],
+                            cmd_options,
+                            case['source'], case['target'])
+
+
+def bench(args):
+    test_cases = []
+
+    sources = {}
+    targets = {}
+    for d in args.dir:
+        label, path = d.split(':')
+        sources[label] = drv_file(path + '/test-source')
+        targets[label] = drv_file(path + '/test-target')
+
+    if args.nbd:
+        nbd = args.nbd.split(':')
+        host = nbd[0]
+        port = '10809' if len(nbd) == 1 else nbd[1]
+        drv = drv_nbd(host, port)
+        sources['nbd'] = drv
+        targets['nbd'] = drv
+
+    for t in args.test:
+        src, dst = t.split(':')
+
+        test_cases.append({
+            'id': t,
+            'source': sources[src],
+            'target': targets[dst]
+        })
+
+    binaries = []  # list of (<label>, <path>, [<options>])
+    for i, q in enumerate(args.env):
+        name_path = q.split(':')
+        if len(name_path) == 1:
+            label = f'q{i}'
+            path_opts = name_path[0].split(',')
+        else:
+            label = name_path[0]
+            path_opts = name_path[1].split(',')
+
+        binaries.append((label, path_opts[0], path_opts[1:]))
+
+    test_envs = []
+
+    bin_paths = {}
+    for i, q in enumerate(args.env):
+        opts = q.split(',')
+        label_path = opts[0]
+        opts = opts[1:]
+
+        if ':' in label_path:
+            label, path = label_path.split(':')
+            bin_paths[label] = path
+        elif label_path in bin_paths:
+            label = label_path
+            path = bin_paths[label]
+        else:
+            path = label_path
+            label = f'q{i}'
+            bin_paths[label] = path
+
+        x_perf = {}
+        is_mirror = False
+        for opt in opts:
+            if opt == 'mirror':
+                is_mirror = True
+            elif opt == 'copy-range=on':
+                x_perf['use-copy-range'] = True
+            elif opt == 'copy-range=off':
+                x_perf['use-copy-range'] = False
+            elif opt.startswith('max-workers='):
+                x_perf['max-workers'] = int(opt.split('=')[1])
+
+        if is_mirror:
+            assert not x_perf
+            test_envs.append({
+                    'id': f'mirror({label})',
+                    'cmd': 'blockdev-mirror',
+                    'qemu-binary': path
+                })
+        else:
+            test_envs.append({
+                'id': f'backup({label})\n' + '\n'.join(opts),
+                'cmd': 'blockdev-backup',
+                'cmd-options': {'x-perf': x_perf} if x_perf else {},
+                'qemu-binary': path
+            })
+
+    result = simplebench.bench(bench_func, test_envs, test_cases, count=3)
+    with open('results.json', 'w') as f:
+        json.dump(result, f, indent=4)
+    print(results_to_text(result))
+
+
+class ExtendAction(argparse.Action):
+    def __call__(self, parser, namespace, values, option_string=None):
+        items = getattr(namespace, self.dest) or []
+        items.extend(values)
+        setattr(namespace, self.dest, items)
+
+
+if __name__ == '__main__':
+    p = argparse.ArgumentParser('Backup benchmark', epilog='''
+ENV format
+
+    (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror]
+
+    LABEL                short name for the binary
+    PATH                 path to the binary
+    max-workers          set x-perf.max-workers of backup job
+    use-copy-range       set x-perf.disable-copy-range of backup job
+    mirror               use mirror job instead of backup''',
+                                formatter_class=argparse.RawTextHelpFormatter)
+    p.add_argument('--env', nargs='+', help='''\
+Qemu binaries with labels and options, see below
+"ENV format" section''',
+                   action=ExtendAction)
+    p.add_argument('--dir', nargs='+', help='''\
+Directories, each containing "test-source" and/or
+"test-target" files, raw images to used in
+benchmarking. File path with label, like
+label:/path/to/directory''',
+                   action=ExtendAction)
+    p.add_argument('--nbd', help='''\
+host:port for remote NBD image, (or just host, for
+default port 10809). Use it in tests, label is "nbd"
+(but you cannot create test nbd:nbd).''')
+    p.add_argument('--test', nargs='+', help='''\
+Tests, in form source-dir-label:target-dir-label''',
+                   action=ExtendAction)
+
+    bench(p.parse_args())
Signature · 1 line
2.21.3
#50 lore raw

Re: [PATCH v3 25/25] simplebench: add bench-backup.py

Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add script to benchmark new backup architecture.

---
  scripts/simplebench/bench-backup.py | 165 ++++++++++++++++++++++++++++
  1 file changed, 165 insertions(+)

create mode 100755 scripts/simplebench/bench-backup.py

Loose review, because I’m not too involved, and this is “just” a performance test, too.

Result: Looks good to me, two minor things below.

Quoted 5 lines
scripts/simplebench/bench-backup.py new file +0 -0
new file mode 100755
index 0000000000..5c62b1a7ed

[...]

scripts/simplebench/bench-backup.py line 43

Quoted 7 lines
scripts/simplebench/bench-backup.py new file +7 -0
+def bench(args):
+    test_cases = []
+
+    sources = {}
+    targets = {}
+    for d in args.dir:
+        label, path = d.split(':')

Should this be d.split(':', 1) to accommodate path names with colons in them?

Quoted 13 lines
scripts/simplebench/bench-backup.py new file +13 -0
+        sources[label] = drv_file(path + '/test-source')
+        targets[label] = drv_file(path + '/test-target')
+
+    if args.nbd:
+        nbd = args.nbd.split(':')
+        host = nbd[0]
+        port = '10809' if len(nbd) == 1 else nbd[1]
+        drv = drv_nbd(host, port)
+        sources['nbd'] = drv
+        targets['nbd'] = drv
+
+    for t in args.test:
+        src, dst = t.split(':')

scripts/simplebench/bench-backup.py line 66

Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10 -0
+
+        test_cases.append({
+            'id': t,
+            'source': sources[src],
+            'target': targets[dst]
+        })
+
+    binaries = []  # list of (<label>, <path>, [<options>])
+    for i, q in enumerate(args.env):
+        name_path = q.split(':')

(Same here)

Quoted 9 lines
scripts/simplebench/bench-backup.py new file +9 -0
+        if len(name_path) == 1:
+            label = f'q{i}'
+            path_opts = name_path[0].split(',')
+        else:
+            label = name_path[0]
+            path_opts = name_path[1].split(',')
+
+        binaries.append((label, path_opts[0], path_opts[1:]))
+

scripts/simplebench/bench-backup.py line 85

Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10 -0
+    test_envs = []
+
+    bin_paths = {}
+    for i, q in enumerate(args.env):
+        opts = q.split(',')
+        label_path = opts[0]
+        opts = opts[1:]
+
+        if ':' in label_path:
+            label, path = label_path.split(':')

(And here)

Quoted 8 lines
scripts/simplebench/bench-backup.py new file +8 -0
+            bin_paths[label] = path
+        elif label_path in bin_paths:
+            label = label_path
+            path = bin_paths[label]
+        else:
+            path = label_path
+            label = f'q{i}'
+            bin_paths[label] = path

[...]

scripts/simplebench/bench-backup.py line 144

Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10 -0
+if __name__ == '__main__':
+    p = argparse.ArgumentParser('Backup benchmark', epilog='''
+ENV format
+
+    (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror]
+
+    LABEL                short name for the binary
+    PATH                 path to the binary
+    max-workers          set x-perf.max-workers of backup job
+    use-copy-range       set x-perf.disable-copy-range of backup job

s/disable/use/?

Max

#51 lore raw

Re: [PATCH v3 25/25] simplebench: add bench-backup.py

Quoted 20 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy

Add script to benchmark new backup architecture.

---
  scripts/simplebench/bench-backup.py | 165 ++++++++++++++++++++++++++++

  1 file changed, 165 insertions(+)

  create mode 100755 scripts/simplebench/bench-backup.py

Loose review, because I’m not too involved, and this is “just” a performance test, too.

Result: Looks good to me, two minor things below.

Quoted 5 lines
scripts/simplebench/bench-backup.py new file +0 -0
new file mode 100755
index 0000000000..5c62b1a7ed

[...]

Quoted 9 lines

scripts/simplebench/bench-backup.py line 43

Quoted 7 lines
scripts/simplebench/bench-backup.py new file +7 -0
+def bench(args):
+    test_cases = []
+
+    sources = {}
+    targets = {}
+    for d in args.dir:
+        label, path = d.split(':')

Should this be d.split(':', 1) to accommodate path names with colons in them?

Yes, it makes sense.

Quoted 18 lines
Quoted 18 lines

+        sources[label] = drv_file(path + '/test-source') +        targets[label] = drv_file(path + '/test-target')

scripts/simplebench/bench-backup.py new file +15 -0
+
+    if args.nbd:
+        nbd = args.nbd.split(':')
+        host = nbd[0]
+        port = '10809' if len(nbd) == 1 else nbd[1]
+        drv = drv_nbd(host, port)
+        sources['nbd'] = drv
+        targets['nbd'] = drv
+
+    for t in args.test:
+        src, dst = t.split(':')
+
+        test_cases.append({
+            'id': t,
+            'source': sources[src],
Quoted 8 lines
Quoted 6 lines

+            'target': targets[dst]
+        })
+
+    binaries = []  # list of (<label>, <path>, [<options>]) +    for i, q in enumerate(args.env):
+        name_path = q.split(':')

(Same here)

hmm here we can't distinguish path with ':' inside without a tag from tag:path.. but anyway, using ", 1" constantly is good. Possible failures will be more clear.

Quoted 34 lines
Quoted 10 lines

+        if len(name_path) == 1:
+            label = f'q{i}'
+            path_opts = name_path[0].split(',')
+        else:
+            label = name_path[0]
+            path_opts = name_path[1].split(',')

scripts/simplebench/bench-backup.py new file +3 -0
+
+        binaries.append((label, path_opts[0], path_opts[1:]))
+

scripts/simplebench/bench-backup.py line 85

Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10 -0
+    test_envs = []
+
+    bin_paths = {}
+    for i, q in enumerate(args.env):
+        opts = q.split(',')
+        label_path = opts[0]
+        opts = opts[1:]
+
+        if ':' in label_path:
+            label, path = label_path.split(':')

(And here)

Quoted 8 lines

+            bin_paths[label] = path
+        elif label_path in bin_paths:
+            label = label_path
+            path = bin_paths[label]
+        else:
+            path = label_path
+            label = f'q{i}'
+            bin_paths[label] = path

[...]

Quoted 13 lines

scripts/simplebench/bench-backup.py line 144

Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10 -0
+if __name__ == '__main__':
+    p = argparse.ArgumentParser('Backup benchmark', epilog='''
+ENV format
+
+    (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror]
+
+    LABEL                short name for the binary
+    PATH                 path to the binary
+    max-workers          set x-perf.max-workers of backup job
+    use-copy-range       set x-perf.disable-copy-range of backup job

s/disable/use/?

yes

Signature · 2 lines
Best regards,
Vladimir
#52 lore raw
In reply to Vladimir Sementsov-Ogievskiy

ping

Quoted 143 lines · Vladimir Sementsov-Ogievskiy

Hi all!

The series turn backup into series of block_copy_async calls, covering the whole disk, so we get block-status based paralallel async requests out of the box, which gives performance gain:

All results are in seconds

----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------

                    A            B              C               D                      E                                 F
                    mirror(old)  backup(old)    backup(old)     backup(new)            backup(new)                       backup(new)
                                 copy-range=on  copy-range=off                         copy-range=on                     copy-range=on
                                                                                                                         max-workers=1
hdd-ext4:hdd-ext4  19           20             21 ± 14%        19                     51 ± 12%                          22 ± 24%
                                   A+5%           A+12%  B+6%     A+3%  B-2%  C-8%       A+174%  B+161%  C+145%  D+165%    A+18%  B+12%  C+5%  D+14%  E-57%
hdd-ext4:ssd-ext4  8.7          9.4 ± 3%       9.6 ± 2%        8.8                    24 ± 2%                           8.9
                                   A+8%           A+10%  B+2%     A+1%  B-7%  C-9%       A+174%  B+155%  C+149%  D+173%    A+2%  B-5%  C-8%  D+1%  E-63%
ssd-ext4:hdd-ext4  9            12 ± 9%        11 ± 7%         9.7 ± 7%               11 ± 2%                           10 ± 3%
                                   A+36%          A+28%  B-6%     A+7%  B-21%  C-16%     A+21%  B-11%  C-5%  D+13%         A+16%  B-14%  C-9%  D+8%  E-4%
ssd-ext4:ssd-ext4  4.4          11 ± 4%        10 ± 3%         4.7                    5.7                               10 ± 5%
                                   A+143%         A+134%  B-4%    A+6%  B-56%  C-55%     A+30%  B-46%  C-45%  D+22%        A+133%  B-4%  C-1%  D+119%  E+79%
hdd-xfs:hdd-xfs    19           20 ± 3%        20              20                     45 ± 4%                           19
                                   A+3%           A+4%  B+1%      A+3%  B+0%  C-1%       A+131%  B+125%  C+122%  D+125%    A-1%  B-4%  C-4%  D-3%  E-57%
hdd-xfs:ssd-xfs    9.1          9.9 ± 4%       9.5             9.1 ± 3%               23 ± 2%                           9.2
                                   A+8%           A+4%  B-4%      A+0%  B-8%  C-4%       A+151%  B+132%  C+142%  D+151%    A+1%  B-7%  C-3%  D+1%  E-60%
ssd-xfs:hdd-xfs    9.1          11 ± 9%        11              9.5 ± 4%               12 ± 22%                          11 ± 3%
                                   A+16%          A+22%  B+6%     A+4%  B-10%  C-15%     A+32%  B+14%  C+8%  D+26%         A+18%  B+2%  C-4%  D+13%  E-10%
ssd-xfs:ssd-xfs    4.1          8.7 ± 7%       9.2 ± 5%        4.5 ± 2%               5.7 ± 3%                          9.7 ± 5%
                                   A+113%         A+126%  B+6%    A+11%  B-48%  C-51%    A+40%  B-34%  C-38%  D+27%        A+138%  B+12%  C+5%  D+115%  E+70%
ssd-ext4:nbd       9.1 ± 2%     37             37 ± 2%         11                     11 ± 3%                           19 ± 2%
                                   A+302%         A+304%  B+1%    A+18%  B-71%  C-71%    A+18%  B-71%  C-71%  D+0%         A+106%  B-49%  C-49%  D+74%  E+75%
nbd:ssd-ext4       9            30 ± 3%        31              9                      9                                 17
                                   A+237%         A+245%  B+2%    A+0%  B-70%  C-71%     A+0%  B-70%  C-71%  D+0%          A+93%  B-43%  C-44%  D+93%  E+93%

----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------

Here column B is current backup and column D is new backup with default parameters.

Mirror is still faster, but we are very close to it.

v3:
01: add Max's r-b
02: change to perf.use-copy-range
03: add Max's r-b
04: - more explicit finish status of async block_copy
- block_copy_async always return non-NULL
- personal opaque for new cb
05: - new arguments added in this patch
- no default value for arguments in block_copy_async()
06: new
07: - caller does _kick() by hand
- grammar in commit msg
- add new parameter in _this_ patch
- switch to opposite ignore_ratelimit
08: cancel now is async
09,10: add Max's r-b
11: changed a lot
12: add timeout
14: rebase on x-perf, keep r-b
15: rebase on x-perf
16: rebase on x-perf, keep r-b
17,18: new
19: now only backup.c is changed in this patch, changed a lot 20,21: new
22: rebased, keep r-b
23: new, split from 24
24: drop unrelated change (now patch23), keep r-b
25: changed a lot, explicitly specify options for each env (test table column)

To run benchmark do the following:

prepare images:
In a directories, where you want to place source and target images, prepare images by:

for img in test-source test-target; do
./qemu-img create -f raw $img 1000M;
./qemu-img bench -c 1000 -d 1 -f raw -s 1M -w --pattern=0xff $img
done

prepare similar image for nbd server, and start it somewhere by

  qemu-nbd --persistent --nocache -f raw IMAGE

Then, run benchmark, like this:
./bench-backup.py --env old:/work/src/qemu/up-backup-block-copy-master/build/qemu-system-x86_64,mirror old,copy-range=on old,copy-range=off new:../../build/qemu-system-x86_64 new,copy-range=on new,copy-range=on,max-workers=1 --dir hdd-ext4:/test-a hdd-xfs:/test-b ssd-ext4:/ssd ssd-xfs:/ssd-xfs --test $(for fs in ext4 xfs; do echo hdd-$fs:hdd-$fs hdd-$fs:ssd-$fs ssd-$fs:hdd-$fs ssd-$fs:ssd-$fs; done) --nbd 192.168.100.5 --test ssd-ext4:nbd nbd:ssd-ext4

(you may simply reduce number of directories/test-cases, use --help for
help)

Vladimir Sementsov-Ogievskiy (25):
iotests: 129 don't check backup "busy"
qapi: backup: add perf.use-copy-range parameter block/block-copy: More explicit call_state block/block-copy: implement block_copy_async block/block-copy: add max_chunk and max_workers parameters block/block-copy: add list of all call-states block/block-copy: add ratelimit to block-copy block/block-copy: add block_copy_cancel
blockjob: add set_speed to BlockJobDriver
job: call job_enter from job_user_pause
qapi: backup: add max-chunk and max-workers to x-perf struct
iotests: 56: prepare for backup over block-copy
iotests: 129: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 257: prepare for backup over block-copy block/block-copy: make progress_bytes_callback optional block/backup: drop extra gotos from backup_run()
backup: move to block-copy
qapi: backup: disable copy_range by default block/block-copy: drop unused block_copy_set_progress_callback() block/block-copy: drop unused argument of block_copy() simplebench/bench_block_job: use correct shebang line with python3
simplebench: bench_block_job: add cmd_options argument
simplebench: add bench-backup.py

  qapi/block-core.json                   |  26 ++-
  block/backup-top.h                     |   1 +
  include/block/block-copy.h             |  58 ++++-
  include/block/block_int.h              |   3 +
  include/block/blockjob_int.h           |   2 +
  block/backup-top.c                     |   6 +-
  block/backup.c                         | 233 ++++++++++++-------
  block/block-copy.c                     | 227 +++++++++++++++---
  block/replication.c                    |   2 +
  blockdev.c                             |  14 ++
  blockjob.c                             |   6 +
  job.c                                  |   1 +
  scripts/simplebench/bench-backup.py    | 165 +++++++++++++
  scripts/simplebench/bench-example.py   |   2 +-
  scripts/simplebench/bench_block_job.py |  13 +-
  tests/qemu-iotests/056                 |   9 +-
  tests/qemu-iotests/129                 |   3 +-
  tests/qemu-iotests/185                 |   3 +-
  tests/qemu-iotests/185.out             |   2 +-
  tests/qemu-iotests/219                 |  13 +-
  tests/qemu-iotests/257                 |   1 +
  tests/qemu-iotests/257.out             | 306 ++++++++++++-------------
  22 files changed, 798 insertions(+), 298 deletions(-)

create mode 100755 scripts/simplebench/bench-backup.py

Signature · 2 lines
Best regards,
Vladimir