backup performance: block_status + async
v3 cover backup performance: block_status + async
Hi all!
The series turn backup into series of block_copy_async calls, covering the whole disk, so we get block-status based paralallel async requests out of the box, which gives performance gain:
All results are in seconds
----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------
A B C D E F
mirror(old) backup(old) backup(old) backup(new) backup(new) backup(new)
copy-range=on copy-range=off copy-range=on copy-range=on
max-workers=1
hdd-ext4:hdd-ext4 19 20 21 ± 14% 19 51 ± 12% 22 ± 24%
A+5% A+12% B+6% A+3% B-2% C-8% A+174% B+161% C+145% D+165% A+18% B+12% C+5% D+14% E-57%
hdd-ext4:ssd-ext4 8.7 9.4 ± 3% 9.6 ± 2% 8.8 24 ± 2% 8.9
A+8% A+10% B+2% A+1% B-7% C-9% A+174% B+155% C+149% D+173% A+2% B-5% C-8% D+1% E-63%
ssd-ext4:hdd-ext4 9 12 ± 9% 11 ± 7% 9.7 ± 7% 11 ± 2% 10 ± 3%
A+36% A+28% B-6% A+7% B-21% C-16% A+21% B-11% C-5% D+13% A+16% B-14% C-9% D+8% E-4%
ssd-ext4:ssd-ext4 4.4 11 ± 4% 10 ± 3% 4.7 5.7 10 ± 5%
A+143% A+134% B-4% A+6% B-56% C-55% A+30% B-46% C-45% D+22% A+133% B-4% C-1% D+119% E+79%
hdd-xfs:hdd-xfs 19 20 ± 3% 20 20 45 ± 4% 19
A+3% A+4% B+1% A+3% B+0% C-1% A+131% B+125% C+122% D+125% A-1% B-4% C-4% D-3% E-57%
hdd-xfs:ssd-xfs 9.1 9.9 ± 4% 9.5 9.1 ± 3% 23 ± 2% 9.2
A+8% A+4% B-4% A+0% B-8% C-4% A+151% B+132% C+142% D+151% A+1% B-7% C-3% D+1% E-60%
ssd-xfs:hdd-xfs 9.1 11 ± 9% 11 9.5 ± 4% 12 ± 22% 11 ± 3%
A+16% A+22% B+6% A+4% B-10% C-15% A+32% B+14% C+8% D+26% A+18% B+2% C-4% D+13% E-10%
ssd-xfs:ssd-xfs 4.1 8.7 ± 7% 9.2 ± 5% 4.5 ± 2% 5.7 ± 3% 9.7 ± 5%
A+113% A+126% B+6% A+11% B-48% C-51% A+40% B-34% C-38% D+27% A+138% B+12% C+5% D+115% E+70%
ssd-ext4:nbd 9.1 ± 2% 37 37 ± 2% 11 11 ± 3% 19 ± 2%
A+302% A+304% B+1% A+18% B-71% C-71% A+18% B-71% C-71% D+0% A+106% B-49% C-49% D+74% E+75%
nbd:ssd-ext4 9 30 ± 3% 31 9 9 17
A+237% A+245% B+2% A+0% B-70% C-71% A+0% B-70% C-71% D+0% A+93% B-43% C-44% D+93% E+93%----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------
Here column B is current backup and column D is new backup with default parameters.
Mirror is still faster, but we are very close to it.
v3:
01: add Max's r-b
02: change to perf.use-copy-range
03: add Max's r-b
04: - more explicit finish status of async block_copy
- block_copy_async always return non-NULL
- personal opaque for new cb
05: - new arguments added in this patch
- no default value for arguments in block_copy_async()
06: new
07: - caller does _kick() by hand
- grammar in commit msg
- add new parameter in _this_ patch
- switch to opposite ignore_ratelimit
08: cancel now is async
09,10: add Max's r-b
11: changed a lot
12: add timeout
14: rebase on x-perf, keep r-b
15: rebase on x-perf
16: rebase on x-perf, keep r-b
17,18: new
19: now only backup.c is changed in this patch, changed a lot 20,21: new
22: rebased, keep r-b
23: new, split from 24
24: drop unrelated change (now patch23), keep r-b
25: changed a lot, explicitly specify options for each env (test table column)
To run benchmark do the following:
prepare images:
In a directories, where you want to place source and target images, prepare images by:
for img in test-source test-target; do
./qemu-img create -f raw $img 1000M;
./qemu-img bench -c 1000 -d 1 -f raw -s 1M -w --pattern=0xff $img
done
prepare similar image for nbd server, and start it somewhere by
qemu-nbd --persistent --nocache -f raw IMAGE
Then, run benchmark, like this:
./bench-backup.py --env old:/work/src/qemu/up-backup-block-copy-master/build/qemu-system-x86_64,mirror old,copy-range=on old,copy-range=off new:../../build/qemu-system-x86_64 new,copy-range=on new,copy-range=on,max-workers=1 --dir hdd-ext4:/test-a hdd-xfs:/test-b ssd-ext4:/ssd ssd-xfs:/ssd-xfs --test $(for fs in ext4 xfs; do echo hdd-$fs:hdd-$fs hdd-$fs:ssd-$fs ssd-$fs:hdd-$fs ssd-$fs:ssd-$fs; done) --nbd 192.168.100.5 --test ssd-ext4:nbd nbd:ssd-ext4
(you may simply reduce number of directories/test-cases, use --help for
help)
Vladimir Sementsov-Ogievskiy (25):
iotests: 129 don't check backup "busy"
qapi: backup: add perf.use-copy-range parameter block/block-copy: More explicit call_state block/block-copy: implement block_copy_async block/block-copy: add max_chunk and max_workers parameters block/block-copy: add list of all call-states block/block-copy: add ratelimit to block-copy block/block-copy: add block_copy_cancel
blockjob: add set_speed to BlockJobDriver
job: call job_enter from job_user_pause
qapi: backup: add max-chunk and max-workers to x-perf struct
iotests: 56: prepare for backup over block-copy
iotests: 129: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 257: prepare for backup over block-copy block/block-copy: make progress_bytes_callback optional block/backup: drop extra gotos from backup_run()
backup: move to block-copy
qapi: backup: disable copy_range by default block/block-copy: drop unused block_copy_set_progress_callback() block/block-copy: drop unused argument of block_copy() simplebench/bench_block_job: use correct shebang line with python3
simplebench: bench_block_job: add cmd_options argument
simplebench: add bench-backup.py
qapi/block-core.json | 26 ++- block/backup-top.h | 1 + include/block/block-copy.h | 58 ++++- include/block/block_int.h | 3 + include/block/blockjob_int.h | 2 + block/backup-top.c | 6 +- block/backup.c | 233 ++++++++++++------- block/block-copy.c | 227 +++++++++++++++--- block/replication.c | 2 + blockdev.c | 14 ++ blockjob.c | 6 + job.c | 1 + scripts/simplebench/bench-backup.py | 165 +++++++++++++ scripts/simplebench/bench-example.py | 2 +- scripts/simplebench/bench_block_job.py | 13 +- tests/qemu-iotests/056 | 9 +- tests/qemu-iotests/129 | 3 +- tests/qemu-iotests/185 | 3 +- tests/qemu-iotests/185.out | 2 +- tests/qemu-iotests/219 | 13 +- tests/qemu-iotests/257 | 1 + tests/qemu-iotests/257.out | 306 ++++++++++++------------- 22 files changed, 798 insertions(+), 298 deletions(-) create mode 100755 scripts/simplebench/bench-backup.py
Signature · 1 line
2.21.3
PATCH v3 1/25 iotests: 129 don't check backup "busy"
Busy is racy, job has it's "pause-points" when it's not busy. Drop this check.
--- tests/qemu-iotests/129 | 1 - 1 file changed, 1 deletion(-)
tests/qemu-iotests/129 +0 -1
@@ -67,7 +67,6 @@ class TestStopWithBlockJob(iotests.QMPTestCase): result = self.vm.qmp("stop") self.assert_qmp(result, 'return', {}) result = self.vm.qmp("query-block-jobs") - self.assert_qmp(result, 'return[0]/busy', True) self.assert_qmp(result, 'return[0]/ready', False) def test_drive_mirror(self):
Signature · 1 line
2.21.3
PATCH v3 2/25 qapi: backup: add perf.use-copy-range parameter
In reply to Vladimir Sementsov-OgievskiyExperiments show, that copy_range is not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.
For now, add new backup qmp parameter with x- prefix for the following reasons:
- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.
So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.
In do_backup_common() new x-perf parameter handled in a way to make further options addition simpler.
We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.
--- qapi/block-core.json | 17 ++++++++++++++++- block/backup-top.h | 1 + include/block/block-copy.h | 2 +- include/block/block_int.h | 3 +++ block/backup-top.c | 4 +++- block/backup.c | 6 +++++- block/block-copy.c | 4 ++-- block/replication.c | 2 ++ blockdev.c | 8 ++++++++ 9 files changed, 41 insertions(+), 6 deletions(-)
qapi/block-core.json +16 -1
@@ -1370,6 +1370,19 @@ { 'struct': 'BlockdevSnapshot', 'data': { 'node': 'str', 'overlay': 'str' } } +## +# @BackupPerf: +# +# Optional parameters for backup. These parameters don't affect +# functionality, but may significantly affect performance. +# +# @use-copy-range: Use copy offloading. Default true. +# +# Since: 5.2 +## +{ 'struct': 'BackupPerf', + 'data': { '*use-copy-range': 'bool' }} + ## # @BackupCommon: #
@@ -1425,6 +1438,8 @@ # above node specified by @drive. If this option is not given, # a node name is autogenerated. (Since: 4.2) # +# @x-perf: Performance options. (Since 5.2) +# # Note: @on-source-error and @on-target-error only affect background # I/O. If an error occurs during a guest write request, the device's # rerror/werror actions will be used.
@@ -1439,7 +1454,7 @@ '*on-source-error': 'BlockdevOnError', '*on-target-error': 'BlockdevOnError', '*auto-finalize': 'bool', '*auto-dismiss': 'bool', - '*filter-node-name': 'str' } } + '*filter-node-name': 'str', '*x-perf': 'BackupPerf' } } ## # @DriveBackup:
block/backup-top.h +1 -0
@@ -33,6 +33,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, + BackupPerf *perf, BdrvRequestFlags write_flags, BlockCopyState **bcs, Error **errp);
include/block/block-copy.h +1 -1
@@ -22,7 +22,7 @@ typedef void (*ProgressBytesCallbackFunc)(int64_t bytes, void *opaque);typedef struct BlockCopyState BlockCopyState ;BlockCopyState *block_copy_state_new (BdrvChild *source,BdrvChild *target, -int64_t cluster_size, +int64_t cluster_size,bool use_copy_range,BdrvRequestFlags write_flags,Error **errp);
include/block/block_int.h +3 -0
@@ -1248,6 +1248,8 @@ void mirror_start(const char *job_id, BlockDriverState *bs,* @sync_mode: What parts of the disk image should be copied to the destination. * @sync_bitmap: The dirty bitmap if sync_mode is 'bitmap' or 'incremental' * @bitmap_mode: The bitmap synchronization policy to use. +* @perf: Performance options. All actual fields assumed to be present, +* all ".has_*" fields are ignored. * @on_source_error: The action to take upon error reading from the source. * @on_target_error: The action to take upon error writing to the target. * @creation_flags: Flags that control the behavior of the Job lifetime.
@@ -1266,6 +1268,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,BitmapSyncMode bitmap_mode, bool compress, const char *filter_node_name, + BackupPerf *perf, BlockdevOnError on_source_error, BlockdevOnError on_target_error, int creation_flags,
block/backup-top.c +3 -1
@@ -186,6 +186,7 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source,BlockDriverState *target, const char *filter_node_name, uint64_t cluster_size, + BackupPerf *perf, BdrvRequestFlags write_flags, BlockCopyState **bcs, Error **errp)
@@ -244,7 +245,8 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState *source, state->cluster_size = cluster_size; state->bcs =block_copy_state_new (top->backing, state->target, - cluster_size, write_flags, &local_err); + cluster_size, perf->use_copy_range, + write_flags, &local_err);if (local_err) {error_prepend (&local_err,"Cannot create block-copy-state: " ); goto fail;
block/backup.c +5 -1
@@ -46,6 +46,7 @@ typedef struct BackupBlockJob {uint64_t len;uint64_t bytes_read;int64_t cluster_size; +BackupPerf perf;BlockCopyState *bcs; } BackupBlockJob;
@@ -335,6 +336,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,BitmapSyncMode bitmap_mode, bool compress, const char *filter_node_name, + BackupPerf *perf, BlockdevOnError on_source_error, BlockdevOnError on_target_error, int creation_flags,
@@ -441,7 +443,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, (compress ?BDRV_REQ_WRITE_COMPRESSED :0 ), backup_top =bdrv_backup_top_append (bs, target, filter_node_name, - cluster_size, write_flags, &bcs, errp); + cluster_size, perf, + write_flags, &bcs, errp);if (!backup_top) { goto error; }
@@ -464,6 +467,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, job->bcs = bcs; job->cluster_size = cluster_size; job->len = len; + job->perf = *perf;block_copy_set_progress_callback (bcs, backup_progress_bytes_callback, job);block_copy_set_progress_meter (bcs, &job->common.job.progress);
block/block-copy.c +2 -2
@@ -218,7 +218,7 @@ static uint32_t block_copy_max_transfer(BdrvChild *source, BdrvChild *target) }BlockCopyState *block_copy_state_new (BdrvChild *source,BdrvChild *target, -int64_t cluster_size, +int64_t cluster_size,bool use_copy_range,BdrvRequestFlags write_flags,Error **errp) {BlockCopyState *s;
@@ -260,7 +260,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,* We enable copy-range, but keep small copy_size, until first * successful copy_range (look at block_copy_do_copy). */ - s->use_copy_range = true; + s->use_copy_range = use_copy_range; s->copy_size =MAX (s->cluster_size,BLOCK_COPY_MAX_BUFFER ); }
block/replication.c +2 -0
@@ -454,6 +454,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,int64_t active_length, hidden_length, disk_length;AioContext *aio_context;Error *local_err =NULL ; +BackupPerf perf = { .use_copy_range = true }; aio_context =bdrv_get_aio_context (bs);aio_context_acquire (aio_context);
@@ -558,6 +559,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, s->backup_job =backup_job_create (NULL , s->secondary_disk->bs, s->hidden_disk->bs,0 ,MIRROR_SYNC_MODE_NONE ,NULL ,0 , false,NULL , + &perf,BLOCKDEV_ON_ERROR_REPORT ,BLOCKDEV_ON_ERROR_REPORT ,JOB_INTERNAL , backup_job_completed, bs,NULL , &local_err);
blockdev.c +8 -0
@@ -2788,6 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, {BlockJob *job =NULL ;BdrvDirtyBitmap *bmap =NULL ; +BackupPerf perf = { .use_copy_range = true };int job_flags =JOB_DEFAULT ;if (!backup->has_speed) {
@@ -2812,6 +2813,12 @@ static BlockJob *do_backup_common(BackupCommon *backup, backup->compress = false; } +if (backup->x_perf) { +if (backup->x_perf->has_use_copy_range) { + perf.use_copy_range = backup->x_perf->use_copy_range; + } + } +if ((backup->sync ==MIRROR_SYNC_MODE_BITMAP ) || (backup->sync ==MIRROR_SYNC_MODE_INCREMENTAL )) {/* done before desugaring 'incremental' to print the right message */
@@ -2885,6 +2892,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, backup->sync, bmap, backup->bitmap_mode, backup->compress, backup->filter_node_name, + &perf, backup->on_source_error, backup->on_target_error, job_flags, NULL, NULL, txn, errp);
Signature · 1 line
2.21.3
Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter
Quoted 25 lines · Vladimir Sementsov-Ogievskiy
Experiments show, that
copy_rangeis not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.For now, add new backup qmp parameter with x- prefix for the following reasons:
- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.
In
do_backup_common()new x-perf parameter handled in a way to make further options addition simpler.We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>
Quoted 11 lines
--- qapi/block-core.json | 17 ++++++++++++++++- block/backup-top.h | 1 + include/block/block-copy.h | 2 +- include/block/block_int.h | 3 +++ block/backup-top.c | 4 +++- block/backup.c | 6 +++++- block/block-copy.c | 4 ++-- block/replication.c | 2 ++ blockdev.c | 8 ++++++++ 9 files changed, 41 insertions(+), 6 deletions(-)
Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter
In reply to Vladimir Sementsov-OgievskiyQuoted 31 lines · Vladimir Sementsov-Ogievskiy
Experiments show, that
copy_rangeis not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.For now, add new backup qmp parameter with x- prefix for the following reasons:
- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.
In
do_backup_common()new x-perf parameter handled in a way to make further options addition simpler.We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
+++
b/qapi/block-core.jsonqapi/block-core.json +0
-0@@ -1370,6 +1370,19 @@ { 'struct': 'BlockdevSnapshot', 'data': { 'node': 'str', 'overlay': 'str' } }
qapi/block-core.json line 1381
Quoted 10 lines
qapi/block-core.json +9
-0+## +# @BackupPerf: +# +# Optional parameters for backup. These parameters don't affect +# functionality, but may significantly affect performance. +# +# @use-copy-range: Use copy offloading. Default true. +# +# Since: 5.2
We've missed 5.2; this will need to be updated to 6.0
qapi/block-core.json line 1441
Quoted 12 lines
qapi/block-core.json +5
-0+## +{ 'struct': 'BackupPerf', + 'data': { '*use-copy-range': 'bool' }} + ## # @BackupCommon: #@@ -1425,6 +1438,8 @@ # above node specified by @drive. If this option is not given, # a node name is autogenerated. (Since: 4.2) # +# @x-perf: Performance options. (Since 5.2)
and here
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter
Quoted 45 lines · Eric Blake
Quoted 31 lines · Vladimir Sementsov-Ogievskiy
Experiments show, that
copy_rangeis not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.For now, add new backup qmp parameter with x- prefix for the following reasons:
- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.
In
do_backup_common()new x-perf parameter handled in a way to make further options addition simpler.We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
+++
b/qapi/block-core.jsonqapi/block-core.json +0
-0@@ -1370,6 +1370,19 @@ { 'struct': 'BlockdevSnapshot', 'data': { 'node': 'str', 'overlay': 'str' } }qapi/block-core.json line 1381
Quoted 10 lines
qapi/block-core.json +9
-0+## +# @BackupPerf: +# +# Optional parameters for backup. These parameters don't affect +# functionality, but may significantly affect performance. +# +# @use-copy-range: Use copy offloading. Default true. +# +# Since: 5.2We've missed 5.2; this will need to be updated to 6.0
Quoted 15 lines
qapi/block-core.json line 1441
Quoted 12 lines
qapi/block-core.json +5
-0+## +{ 'struct': 'BackupPerf', + 'data': { '*use-copy-range': 'bool' }} + ## # @BackupCommon: #@@ -1425,6 +1438,8 @@ # above node specified by @drive. If this option is not given, # a node name is autogenerated. (Since: 4.2) # +# @x-perf: Performance options. (Since 5.2)and here
I wanted to say that it's fixed in v4 (which is already in Max's tree), but... It's not fixed. Max, could you touch this up?
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH v3 02/25] qapi: backup: add perf.use-copy-range parameter
Quoted 61 lines · Vladimir Sementsov-Ogievskiy
Quoted 44 lines · Eric Blake
Quoted 30 lines · Vladimir Sementsov-Ogievskiy
Experiments show, that
copy_rangeis not always making things faster. So, to make experimentation simpler, let's add a parameter. Some more perf parameters will be added soon, so here is a new struct.For now, add new backup qmp parameter with x- prefix for the following reasons:
- We are going to add more performance parameters, some will be related to the whole block-copy process, some only to background copying in backup (ignored for copy-before-write operations).
- On the other hand, we are going to use block-copy interface in other block jobs, which will need performance options as well.. And it should be the same structure or at least somehow related.So, there are too much unclean things about how the interface and now we need the new options mostly for testing. Let's keep them experimental for a while.
In
do_backup_common()new x-perf parameter handled in a way to make further options addition simpler.We add use-copy-range with default=true, and we'll change the default in further patch, after moving backup to use block-copy.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
+++
b/qapi/block-core.jsonqapi/block-core.json +0
-0@@ -1370,6 +1370,19 @@ { 'struct': 'BlockdevSnapshot',qapi/block-core.json line 1380
Quoted 10 lines
qapi/block-core.json +9
-0'data': { 'node': 'str', 'overlay': 'str' } } +## +# @BackupPerf: +# +# Optional parameters for backup. These parameters don't affect +# functionality, but may significantly affect performance. +# +# @use-copy-range: Use copy offloading. Default true. +# +# Since: 5.2We've missed 5.2; this will need to be updated to 6.0
Quoted 15 lines
qapi/block-core.json line 1441
Quoted 12 lines
qapi/block-core.json +5
-0+## +{ 'struct': 'BackupPerf', + 'data': { '*use-copy-range': 'bool' }} + ## # @BackupCommon: #@@ -1425,6 +1438,8 @@ # above node specified by @drive. If this option is not given, # a node name is autogenerated. (Since: 4.2) # +# @x-perf: Performance options. (Since 5.2)and here
Quoted 1 line
I wanted to say that it's fixed in v4 (which is already in Max's tree), but... It's not fixed. Max, could you touch this up?
Sure.
Max
PATCH v3 3/25 block/block-copy: More explicit call_state
In reply to Vladimir Sementsov-OgievskiyRefactor common path to use BlockCopyCallState pointer as parameter, to prepare it for use in asynchronous block-copy (at least, we'll need to run block-copy in a coroutine, passing the whole parameters as one pointer).
--- block/block-copy.c | 51 ++++++++++++++++++++++++++++++++++------------ 1 file changed, 38 insertions(+), 13 deletions(-)
block/block-copy.c +38 -13
@@ -30,7 +30,15 @@ static coroutine_fn int block_copy_task_entry(AioTask *task);typedef struct BlockCopyCallState { +/* IN parameters */ +BlockCopyState *s; +int64_t offset; +int64_t bytes; + +/* State */ bool failed; + +/* OUT parameters */ bool error_is_read; }BlockCopyCallState ;
@@ -544,15 +552,17 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s,* Returns 1 if dirty clusters found and successfully copied, 0 if no dirty * clusters found and -errno on failure. */ -static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s, -int64_t offset,int64_t bytes, -bool *error_is_read) +static int coroutine_fn +block_copy_dirty_clusters (BlockCopyCallState *call_state) { +BlockCopyState *s = call_state->s; +int64_t offset = call_state->offset; +int64_t bytes = call_state->bytes; +int ret =0 ;bool found_dirty = false;int64_t end = offset + bytes;AioTaskPool *aio =NULL ; -BlockCopyCallState call_state = {false, false};/* * block_copy() user is responsible for keeping source and target in same
@@ -568,7 +578,7 @@ static int coroutine_fn block_copy_dirty_clusters(BlockCopyState *s,BlockCopyTask *task;int64_t status_bytes; - task =block_copy_task_create (s, &call_state, offset, bytes); + task =block_copy_task_create (s, call_state, offset, bytes);if (!task) {/* No more dirty bits in the bitmap */ trace_block_copy_skip_range (s, offset, bytes);
@@ -633,15 +643,12 @@ out:aio_task_pool_free (aio); } -if (error_is_read && ret <0 ) { - *error_is_read = call_state.error_is_read; - }return ret <0 ? ret : found_dirty; }/* -* block_copy +* block_copy_common * * Copy requested region, accordingly to dirty bitmap. * Collaborate with parallel block_copy requests: if they succeed it will help
@@ -649,16 +656,16 @@ out:* it means that some I/O operation failed in context of _this_ block_copy call, * not some parallel operation. */ -int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes, -bool *error_is_read) +static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) {int ret; do { - ret = block_copy_dirty_clusters(s, offset, bytes, error_is_read); + ret = block_copy_dirty_clusters(call_state); if (ret == 0) { - ret = block_copy_wait_one(s, offset, bytes); + ret = block_copy_wait_one(call_state->s, call_state->offset, + call_state->bytes); } /*
@@ -675,6 +682,24 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,return ret; } +int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes, +bool *error_is_read) +{ +BlockCopyCallState call_state = { + .s = s, + .offset = start, + .bytes = bytes, + }; + +int ret =block_copy_common (&call_state); + +if (error_is_read && ret <0 ) { + *error_is_read = call_state.error_is_read; + } + +return ret; +} +BdrvDirtyBitmap *block_copy_dirty_bitmap (BlockCopyState *s) {return s->copy_bitmap;
Signature · 1 line
2.21.3
PATCH v3 4/25 block/block-copy: implement block_copy_async
In reply to Vladimir Sementsov-OgievskiyWe'll need async block-copy invocation to use in backup directly.
--- include/block/block-copy.h | 26 ++++++++++++ block/block-copy.c | 81 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 103 insertions(+), 4 deletions(-)
include/block/block-copy.h +26 -0
@@ -19,7 +19,9 @@#include "qemu/co-shared-resource.h" typedef void (*ProgressBytesCallbackFunc )(int64_t bytes,void *opaque); +typedef void (*BlockCopyAsyncCallbackFunc )(void *opaque);typedef struct BlockCopyState BlockCopyState ; +typedef struct BlockCopyCallState BlockCopyCallState ; BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, int64_t cluster_size, bool use_copy_range,
@@ -41,6 +43,30 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,bool *error_is_read); +/* +* Run block-copy in a coroutine, create corresponding BlockCopyCallState +* object and return pointer to it. Never returns NULL. +* +* Caller is responsible to call block_copy_call_free() to free +* BlockCopyCallState object. +*/ +BlockCopyCallState *block_copy_async (BlockCopyState *s, +int64_t offset,int64_t bytes, +BlockCopyAsyncCallbackFunc cb, +void *cb_opaque); + +/* +* Free finished BlockCopyCallState. Trying to free running +* block-copy will crash. +*/ +void block_copy_call_free (BlockCopyCallState *call_state); + +/* Note, that block-copy call is marked finished prior to calling the callback. */ +bool block_copy_call_finished (BlockCopyCallState *call_state); +bool block_copy_call_succeeded (BlockCopyCallState *call_state); +bool block_copy_call_failed (BlockCopyCallState *call_state); +int block_copy_call_status (BlockCopyCallState *call_state,bool *error_is_read); +BdrvDirtyBitmap *block_copy_dirty_bitmap (BlockCopyState *s);void block_copy_set_skip_unallocated (BlockCopyState *s,bool skip);
block/block-copy.c +77 -4
110 lines not on this page · raw message
Signature · 1 line
2.21.3
Re: [PATCH v3 04/25] block/block-copy: implement block_copy_async
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
We'll need async block-copy invocation to use in backup directly.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/block-copy.h | 26 ++++++++++++ block/block-copy.c | 81 ++++++++++++++++++++++++++++++++++++-- 2 files changed, 103 insertions(+), 4 deletions(-)
PATCH v3 5/25 block/block-copy: add max_chunk and max_workers parameters
In reply to Vladimir Sementsov-OgievskiyThey will be used for backup.
--- include/block/block-copy.h | 6 ++++++ block/block-copy.c | 11 +++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-)
include/block/block-copy.h +6 -0
@@ -49,9 +49,15 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes,* * Caller is responsible to call block_copy_call_free() to free * BlockCopyCallState object. +* +* @max_workers means maximum of parallel coroutines to execute sub-requests, +* must be > 0. +* +* @max_chunk means maximum length for one IO operation. Zero means unlimited. */ BlockCopyCallState *block_copy_async (BlockCopyState *s,int64_t offset,int64_t bytes, +int max_workers,int64_t max_chunk,BlockCopyAsyncCallbackFunc cb,void *cb_opaque);
block/block-copy.c +9 -2
@@ -34,6 +34,8 @@ typedef struct BlockCopyCallState {BlockCopyState *s;int64_t offset;int64_t bytes; +int max_workers; +int64_t max_chunk;BlockCopyAsyncCallbackFunc cb;void *cb_opaque;
@@ -148,10 +150,11 @@ static BlockCopyTask *block_copy_task_create(BlockCopyState *s, int64_t offset, int64_t bytes) {BlockCopyTask *task; +int64_t max_chunk =MIN_NON_ZERO (s->copy_size, call_state->max_chunk);if (!bdrv_dirty_bitmap_next_dirty_area (s->copy_bitmap, offset, offset + bytes, - s->copy_size, &offset, &bytes)) + max_chunk, &offset, &bytes)) {return NULL ; }
@@ -623,7 +626,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state) bytes = end - offset;if (!aio && bytes) { - aio =aio_task_pool_new (BLOCK_COPY_MAX_WORKERS ); + aio =aio_task_pool_new (call_state->max_workers); } ret =block_copy_task_run (aio, task);
@@ -701,6 +704,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes, .s = s, .offset = start, .bytes = bytes, + .max_workers = BLOCK_COPY_MAX_WORKERS, };int ret =block_copy_common (&call_state);
@@ -719,6 +723,7 @@ static void coroutine_fn block_copy_async_co_entry(void *opaque)BlockCopyCallState *block_copy_async (BlockCopyState *s,int64_t offset,int64_t bytes, +int max_workers,int64_t max_chunk,BlockCopyAsyncCallbackFunc cb,void *cb_opaque) {
@@ -728,6 +733,8 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s, .s = s, .offset = offset, .bytes = bytes, + .max_workers = max_workers, + .max_chunk = max_chunk, .cb = cb, .cb_opaque = cb_opaque,
Signature · 1 line
2.21.3
Re: [PATCH v3 05/25] block/block-copy: add max_chunk and max_workers parameters
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
They will be used for backup.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/block-copy.h | 6 ++++++ block/block-copy.c | 11 +++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-)
PATCH v3 6/25 block/block-copy: add list of all call-states
In reply to Vladimir Sementsov-OgievskiyIt simplifies debugging.
--- block/block-copy.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
block/block-copy.c +10 -1
@@ -42,6 +42,9 @@ typedef struct BlockCopyCallState {/* Coroutine where async block-copy is running */ Coroutine *co; +/* To reference all call states from BlockCopyState */ +QLIST_ENTRY (BlockCopyCallState) list; +/* State */ int ret;bool finished;
@@ -81,7 +84,8 @@ typedef struct BlockCopyState {bool use_copy_range;int64_t copy_size;uint64_t len; - QLIST_HEAD(, BlockCopyTask) tasks; + QLIST_HEAD(, BlockCopyTask) tasks; /* All tasks from all block-copy calls */ + QLIST_HEAD(, BlockCopyCallState) calls;BdrvRequestFlags write_flags;
@@ -282,6 +286,7 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target, }QLIST_INIT (&s->tasks); +QLIST_INIT (&s->calls);return s; }
@@ -669,6 +674,8 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) {int ret; +QLIST_INSERT_HEAD (&call_state->s->calls, call_state, list); + do { ret = block_copy_dirty_clusters(call_state);
@@ -694,6 +701,8 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) call_state->cb (call_state->cb_opaque); } +QLIST_REMOVE (call_state, list); +return ret; }
Signature · 1 line
2.21.3
Re: [PATCH v3 06/25] block/block-copy: add list of all call-states
Quoted 6 lines · Vladimir Sementsov-Ogievskiy
It simplifies debugging.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/block-copy.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
PATCH v3 7/25 block/block-copy: add ratelimit to block-copy
In reply to Vladimir Sementsov-OgievskiyWe are going to directly use one async block-copy operation for backup job, so we need rate limiter.
We want to maintain current backup behavior: only background copying is limited and copy-before-write operations only participate in limit calculation. Therefore we need one rate limiter for block-copy state and boolean flag for block-copy call state for actual limitation.
Note, that we can't just calculate each chunk in limiter after successful copying: it will not save us from starting a lot of async sub-requests which will exceed limit too much. Instead let's use the following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one sub-request (which pretty much matches current backup behavior).
Note also, that if there is in-flight block-copy async call, block_copy_kick() should be used after set-speed to apply new setup faster. For that block_copy_kick() published in this patch.
--- include/block/block-copy.h | 5 ++++- block/backup-top.c | 2 +- block/backup.c | 2 +- block/block-copy.c | 46 +++++++++++++++++++++++++++++++++++++- 4 files changed, 51 insertions(+), 4 deletions(-)
include/block/block-copy.h +4 -1
@@ -41,7 +41,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, int64_t offset, int64_t *count); int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes, -bool *error_is_read); +bool ignore_ratelimit,bool *error_is_read);/* * Run block-copy in a coroutine, create corresponding BlockCopyCallState
@@ -73,6 +73,9 @@ bool block_copy_call_succeeded(BlockCopyCallState *call_state);bool block_copy_call_failed (BlockCopyCallState *call_state);int block_copy_call_status (BlockCopyCallState *call_state,bool *error_is_read); +void block_copy_set_speed (BlockCopyState *s,uint64_t speed); +void block_copy_kick (BlockCopyCallState *call_state); +BdrvDirtyBitmap *block_copy_dirty_bitmap (BlockCopyState *s);void block_copy_set_skip_unallocated (BlockCopyState *s,bool skip);
block/backup-top.c +1 -1
@@ -61,7 +61,7 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset, off =QEMU_ALIGN_DOWN (offset, s->cluster_size); end =QEMU_ALIGN_UP (offset + bytes, s->cluster_size); -return block_copy (s->bcs, off, end - off,NULL ); +return block_copy (s->bcs, off, end - off, true,NULL ); } static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
block/backup.c +1 -1
@@ -72,7 +72,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,trace_backup_do_cow_enter (job, start, offset, bytes); - ret =block_copy (job->bcs, start, end - start, error_is_read); + ret =block_copy (job->bcs, start, end - start, true, error_is_read);trace_backup_do_cow_return (job, offset, bytes, ret);
block/block-copy.c +45 -1
96 lines not on this page · raw message
Signature · 1 line
2.21.3
Re: [PATCH v3 07/25] block/block-copy: add ratelimit to block-copy
Quoted 23 lines · Vladimir Sementsov-Ogievskiy
We are going to directly use one async block-copy operation for backup job, so we need rate limiter.
We want to maintain current backup behavior: only background copying is limited and copy-before-write operations only participate in limit calculation. Therefore we need one rate limiter for block-copy state and boolean flag for block-copy call state for actual limitation.
Note, that we can't just calculate each chunk in limiter after successful copying: it will not save us from starting a lot of async sub-requests which will exceed limit too much. Instead let's use the following scheme on sub-request creation:
1. If at the moment limit is not exceeded, create the request and account it immediately.
2. If at the moment limit is already exceeded, drop create sub-request and handle limit instead (by sleep).
With this approach we'll never exceed the limit more than by one sub-request (which pretty much matches current backup behavior).Note also, that if there is in-flight block-copy async call,
block_copy_kick()should be used after set-speed to apply new setup faster. For thatblock_copy_kick()published in this patch.
Quoted 7 lines
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/block-copy.h | 5 ++++- block/backup-top.c | 2 +- block/backup.c | 2 +- block/block-copy.c | 46 +++++++++++++++++++++++++++++++++++++- 4 files changed, 51 insertions(+), 4 deletions(-)
PATCH v3 8/25 block/block-copy: add block_copy_cancel
In reply to Vladimir Sementsov-OgievskiyAdd function to cancel running async block-copy call. It will be used in backup.
--- include/block/block-copy.h | 13 +++++++++++++ block/block-copy.c | 24 +++++++++++++++++++----- 2 files changed, 32 insertions(+), 5 deletions(-)
include/block/block-copy.h +13 -0
@@ -71,11 +71,24 @@ void block_copy_call_free(BlockCopyCallState *call_state);bool block_copy_call_finished (BlockCopyCallState *call_state);bool block_copy_call_succeeded (BlockCopyCallState *call_state);bool block_copy_call_failed (BlockCopyCallState *call_state); +bool block_copy_call_cancelled (BlockCopyCallState *call_state);int block_copy_call_status (BlockCopyCallState *call_state,bool *error_is_read);void block_copy_set_speed (BlockCopyState *s,uint64_t speed);void block_copy_kick (BlockCopyCallState *call_state); +/* +* Cancel running block-copy call. +* +* Cancel leaves block-copy state valid: dirty bits are correct and you may use +* cancel + <run block_copy with same parameters> to emulate pause/resume. +* +* Note also, that the cancel is async: it only marks block-copy call to be +* cancelled. So, the call may be cancelled (block_copy_call_cancelled() reports +* true) but not yet finished (block_copy_call_finished() reports false). +*/ +void block_copy_call_cancel (BlockCopyCallState *call_state); +BdrvDirtyBitmap *block_copy_dirty_bitmap (BlockCopyState *s);void block_copy_set_skip_unallocated (BlockCopyState *s,bool skip);
block/block-copy.c +19 -5
@@ -51,6 +51,7 @@ typedef struct BlockCopyCallState {int ret;bool finished;QemuCoSleepState *sleep_state; +bool cancelled;/* OUT parameters */ bool error_is_read;
@@ -594,7 +595,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)assert (QEMU_IS_ALIGNED (offset, s->cluster_size));assert (QEMU_IS_ALIGNED (bytes, s->cluster_size)); -while (bytes &&aio_task_pool_status (aio) ==0 ) { +while (bytes &&aio_task_pool_status (aio) ==0 && !call_state->cancelled) {BlockCopyTask *task;int64_t status_bytes;
@@ -707,7 +708,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) do { ret = block_copy_dirty_clusters(call_state); - if (ret == 0) { + if (ret == 0 && !call_state->cancelled) { ret = block_copy_wait_one(call_state->s, call_state->offset, call_state->bytes); }
@@ -721,7 +722,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state)* 2. We have waited for some intersecting block-copy request * It may have failed and produced new dirty bits. */ - }while (ret >0 ); + }while (ret >0 && !call_state->cancelled); call_state->finished = true;
@@ -801,12 +802,19 @@ bool block_copy_call_finished(BlockCopyCallState *call_state)bool block_copy_call_succeeded (BlockCopyCallState *call_state) { -return call_state->finished && call_state->ret ==0 ; +return call_state->finished && !call_state->cancelled && + call_state->ret ==0 ; }bool block_copy_call_failed (BlockCopyCallState *call_state) { -return call_state->finished && call_state->ret <0 ; +return call_state->finished && !call_state->cancelled && + call_state->ret <0 ; +} + +bool block_copy_call_cancelled (BlockCopyCallState *call_state) +{ +return call_state->cancelled; }int block_copy_call_status (BlockCopyCallState *call_state,bool *error_is_read)
@@ -818,6 +826,12 @@ int block_copy_call_status(BlockCopyCallState *call_state, bool *error_is_read)return call_state->ret; } +void block_copy_call_cancel (BlockCopyCallState *call_state) +{ + call_state->cancelled = true; +block_copy_kick (call_state); +} +BdrvDirtyBitmap *block_copy_dirty_bitmap (BlockCopyState *s) {return s->copy_bitmap;
Signature · 1 line
2.21.3
Re: [PATCH v3 08/25] block/block-copy: add block_copy_cancel
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Add function to cancel running async block-copy call. It will be used in backup.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/block-copy.h | 13 +++++++++++++ block/block-copy.c | 24 +++++++++++++++++++----- 2 files changed, 32 insertions(+), 5 deletions(-)
PATCH v3 9/25 blockjob: add set_speed to BlockJobDriver
In reply to Vladimir Sementsov-OgievskiyWe are going to use async block-copy call in backup, so we'll need to passthrough setting backup speed to block-copy call.
--- include/block/blockjob_int.h | 2 ++ blockjob.c | 6 ++++++ 2 files changed, 8 insertions(+)
include/block/blockjob_int.h +2 -0
@@ -52,6 +52,8 @@ struct BlockJobDriver {* besides job->blk to the new AioContext. */ void (*attached_aio_context)(BlockJob *job,AioContext *new_context); + +void (*set_speed)(BlockJob *job,int64_t speed); };/**
blockjob.c +6 -0
@@ -256,6 +256,7 @@ static bool job_timer_pending(Job *job)void block_job_set_speed (BlockJob *job,int64_t speed,Error **errp) { +const BlockJobDriver *drv =block_job_driver (job);int64_t old_speed = job->speed;if (job_apply_verb (&job->job,JOB_VERB_SET_SPEED , errp)) {
@@ -270,6 +271,11 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp)ratelimit_set_speed (&job->limit, speed,BLOCK_JOB_SLICE_TIME ); job->speed = speed; + +if (drv->set_speed) { + drv->set_speed (job, speed); + } +if (speed && speed <= old_speed) {return ; }
Signature · 1 line
2.21.3
PATCH v3 10/25 job: call job_enter from job_user_pause
In reply to Vladimir Sementsov-OgievskiyIf main job coroutine called job_yield (while some background process is in progress), we should give it a chance to call job_pause_point(). It will be used in backup, when moved on async block-copy.
--- job.c | 1 + 1 file changed, 1 insertion(+)
job.c +1 -0
@@ -578,6 +578,7 @@ void job_user_pause(Job *job, Error **errp) } job->user_paused = true;job_pause (job); +job_enter (job); }bool job_user_paused (Job *job)
Signature · 1 line
2.21.3
PATCH v3 11/25 qapi: backup: add max-chunk and max-workers to x-perf struct
In reply to Vladimir Sementsov-OgievskiyAdd new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior.
--- qapi/block-core.json | 11 ++++++++++- block/backup.c | 28 +++++++++++++++++++++++----- block/replication.c | 2 +- blockdev.c | 8 +++++++- 4 files changed, 41 insertions(+), 8 deletions(-)
qapi/block-core.json +10 -1
@@ -1378,10 +1378,19 @@ # # @use-copy-range: Use copy offloading. Default true. # +# @max-workers: Maximum number of parallel requests for the sustained background +# copying process. Doesn't influence copy-before-write operations. +# Default 64. +# +# @max-chunk: Maximum request length for the sustained background copying +# process. Doesn't influence copy-before-write operations. +# 0 means unlimited. Default 0. +# # Since: 5.2 ## { 'struct': 'BackupPerf', - 'data': { '*use-copy-range': 'bool' }} + 'data': { '*use-copy-range': 'bool', + '*max-workers': 'int', '*max-chunk': 'int64' } } ## # @BackupCommon:
block/backup.c +23 -5
@@ -388,6 +388,29 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,return NULL ; } + cluster_size =backup_calculate_cluster_size (target, errp); +if (cluster_size <0 ) { + goto error; + } + +if (perf->max_workers <1 ) { +error_setg (errp,"max-worker must be greater than zero" ); +return NULL ; + } + +if (perf->max_chunk <0 ) { +error_setg (errp,"max-chunk must be zero (which means no limit) or " +"positive" ); +return NULL ; + } + +if (perf->max_chunk && perf->max_chunk < cluster_size) { +error_setg (errp,"Required max-chunk (%" PRIi64") is less than backup " +"cluster size (%" PRIi64")" , perf->max_chunk, cluster_size); +return NULL ; + } + +if (sync_bitmap) {/* If we need to write to this bitmap, check that we can: */ if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
@@ -420,11 +443,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, goto error; } - cluster_size =backup_calculate_cluster_size (target, errp); -if (cluster_size <0 ) { - goto error; - } -/* * If source is in backing chain of target assume that target is going to be * used for "image fleecing", i.e. it should represent a kind of snapshot of
block/replication.c +1 -1
@@ -454,7 +454,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode,int64_t active_length, hidden_length, disk_length;AioContext *aio_context;Error *local_err =NULL ; -BackupPerf perf = { .use_copy_range = true }; +BackupPerf perf = { .use_copy_range = true, .max_workers =1 }; aio_context =bdrv_get_aio_context (bs);aio_context_acquire (aio_context);
blockdev.c +7 -1
@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, {BlockJob *job =NULL ;BdrvDirtyBitmap *bmap =NULL ; -BackupPerf perf = { .use_copy_range = true }; +BackupPerf perf = { .use_copy_range = true, .max_workers =64 };int job_flags =JOB_DEFAULT ;if (!backup->has_speed) {
@@ -2817,6 +2817,12 @@ static BlockJob *do_backup_common(BackupCommon *backup,if (backup->x_perf->has_use_copy_range) { perf.use_copy_range = backup->x_perf->use_copy_range; } +if (backup->x_perf->has_max_workers) { + perf.max_workers = backup->x_perf->max_workers; + } +if (backup->x_perf->has_max_chunk) { + perf.max_chunk = backup->x_perf->max_chunk; + } } if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
Signature · 1 line
2.21.3
Re: [PATCH v3 11/25] qapi: backup: add max-chunk and max-workers to x-perf struct
Quoted 29 lines · Vladimir Sementsov-Ogievskiy
Add new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- qapi/block-core.json | 11 ++++++++++- block/backup.c | 28 +++++++++++++++++++++++----- block/replication.c | 2 +- blockdev.c | 8 +++++++- 4 files changed, 41 insertions(+), 8 deletions(-)qapi/block-core.json +7
-0@@ -1378,10 +1378,19 @@ # # @use-copy-range: Use copy offloading. Default true. # +# @max-workers: Maximum number of parallel requests for the sustained background +# copying process. Doesn't influence copy-before-write operations. +# Default 64. +# +# @max-chunk: Maximum request length for the sustained background copying +# process. Doesn't influence copy-before-write operations. +# 0 means unlimited. Default 0.
qapi/block-core.json line 1396
Quoted 10 lines
qapi/block-core.json +3
-1+# # Since: 5.2 ## { 'struct': 'BackupPerf', - 'data': { '*use-copy-range': 'bool' }} + 'data': { '*use-copy-range': 'bool', + '*max-workers': 'int', '*max-chunk': 'int64' } } ## # @BackupCommon:
squash-in:
qapi/block-core.json +3 -1
@@ -1406,7 +1406,9 @@ # # @max-chunk: Maximum request length for the sustained background copying # process. Doesn't influence copy-before-write operations. -# 0 means unlimited. Default 0. +# 0 means unlimited. If max-chunk is non-zero then it should not be +# less than job cluster size which is calculated as maximum of +# target image cluster size and 64k. Default 0. # # Since: 5.2 ##
Signature · 2 lines
Best regards, Vladimir
Re: [PATCH v3 11/25] qapi: backup: add max-chunk and max-workers to x-perf struct
In reply to Vladimir Sementsov-OgievskiyQuoted 13 lines · Vladimir Sementsov-Ogievskiy
Add new parameters to configure future backup features. The patch doesn't introduce aio backup requests (so we actually have only one worker) neither requests larger than one cluster. Still, formally we satisfy these maximums anyway, so add the parameters now, to facilitate further patch which will really change backup job behavior.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- qapi/block-core.json | 11 ++++++++++- block/backup.c | 28 +++++++++++++++++++++++----- block/replication.c | 2 +- blockdev.c | 8 +++++++- 4 files changed, 41 insertions(+), 8 deletions(-)
[...]
Quoted 5 lines
block/backup.c +0
-0
block/backup.c BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, · line 397
Quoted 10 lines
block/backup.c +7
-0@@ -388,6 +388,29 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,return NULL ; } + cluster_size =backup_calculate_cluster_size (target, errp); +if (cluster_size <0 ) { + goto error; + } + +if (perf->max_workers <1 ) { +error_setg (errp,"max-worker must be greater than zero" );
*max-workers
With that fixed:
Quoted 2 lines
block/backup.c +2
-0+return NULL ; + }
PATCH v3 12/25 iotests: 56: prepare for backup over block-copy
In reply to Vladimir Sementsov-OgievskiyAfter introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, we'll have to wait for paused status, as we need to wait for several parallel request. So, let's gently wait instead of just asserting that job already paused.
--- tests/qemu-iotests/056 | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
tests/qemu-iotests/056 +7 -2
@@ -307,8 +307,13 @@ class BackupTest(iotests.QMPTestCase): event = self.vm.event_wait(name="BLOCK_JOB_ERROR", match={'data': {'device': 'drive0'}}) self.assertNotEqual(event, None) - # OK, job should be wedged - res = self.vm.qmp('query-block-jobs') + # OK, job should pause, but it can't do it immediately, as it can't + # cancel other parallel requests (which didn't fail) + with iotests.Timeout(60, "Timeout waiting for backup actually paused"): + while True: + res = self.vm.qmp('query-block-jobs') + if res['return'][0]['status'] == 'paused': + break self.assert_qmp(res, 'return[0]/status', 'paused') res = self.vm.qmp('block-job-dismiss', id='drive0') self.assert_qmp(res, 'error/desc',
Signature · 1 line
2.21.3
Re: [PATCH v3 12/25] iotests: 56: prepare for backup over block-copy
Quoted 9 lines · Vladimir Sementsov-Ogievskiy
After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, we'll have to wait for paused status, as we need to wait for several parallel request. So, let's gently wait instead of just asserting that job already paused.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- tests/qemu-iotests/056 | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
PATCH v3 13/25 iotests: 129: prepare for backup over block-copy
In reply to Vladimir Sementsov-OgievskiyAfter introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in do_test_stop. Let's require slow backup explicitly by specifying speed parameter.
--- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qemu-iotests/129 +1 -1
@@ -77,7 +77,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", target=self.target_img, - sync="full") + sync="full", speed=1024) def test_block_commit(self): self.do_test_stop("block-commit", device="drive0")
Signature · 1 line
2.21.3
Re: [PATCH v3 13/25] iotests: 129: prepare for backup over block-copy
Quoted 15 lines · Vladimir Sementsov-Ogievskiy
After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in
do_test_stop. Let's require slow backup explicitly by specifying speed parameter.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- tests/qemu-iotests/129 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/qemu-iotests/129 +0
-0
tests/qemu-iotests/129 class TestStopWithBlockJob(iotests.QMPTestCase): · line 83
Quoted 8 lines
tests/qemu-iotests/129 +1
-1@@ -77,7 +77,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase): def test_drive_backup(self): self.do_test_stop("drive-backup", device="drive0", target=self.target_img, - sync="full") + sync="full", speed=1024) def test_block_commit(self): self.do_test_stop("block-commit", device="drive0")
I still don’t like this very much. We write 128 MB of data, do_test_stop throttles the source to 1 kB/s, so even “a lot of parallel requests” shouldn’t finish immediately.
I feel like there’s a bigger problem here.
Then again, I have 129 disabled all the time anyway because its throttling simply doesn’t work, so I guess
Re: [PATCH v3 13/25] iotests: 129: prepare for backup over block-copy
Quoted 26 lines · Max Reitz
Quoted 15 lines · Vladimir Sementsov-Ogievskiy
After introducing parallel async copy requests instead of plain cluster-by-cluster copying loop, backup job may finish earlier than final assertion in
do_test_stop. Let's require slow backup explicitly by specifying speed parameter.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
tests/qemu-iotests/129 | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)tests/qemu-iotests/129 +0
-0@@ -77,7 +77,7 @@ class TestStopWithBlockJob(iotests.QMPTestCase):Quoted 7 lines
def
test_drive_backup(self):
self.do_test_stop("drive-backup", device="drive0", target=self.target_img, - sync="full")
+ sync="full", speed=1024) deftest_block_commit(self):
self.do_test_stop("block-commit", device="drive0")I still don’t like this very much. We write 128 MB of data,
do_test_stopthrottles the source to 1 kB/s, so even “a lot of parallel requests” shouldn’t finish immediately.
Quoted 1 line
I feel like there’s a bigger problem here.
You were right!:) And finally I found it. Will fix in v4.
Quoted 5 lines
Then again, I have 129 disabled all the time anyway because its throttling simply doesn’t work, so I guess
Acked-by Max Reitz <[email protected]>
Signature · 2 lines
Best regards, Vladimir
PATCH v3 14/25 iotests: 185: prepare for backup over block-copy
In reply to Vladimir Sementsov-OgievskiyThe further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. 185 test however assumes, that with speed limited to 64K, one iteration would result in offset=64K. It will change, as first iteration would result in offset=1M independently of speed.
So, let's explicitly specify, what test wants: set max-chunk to 64K, so that one iteration is 64K. Note, that we don't need to limit max-workers, as block-copy rate limiter will handle the situation and wouldn't start new workers when speed limit is obviously reached.
--- tests/qemu-iotests/185 | 3 ++- tests/qemu-iotests/185.out | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-)
tests/qemu-iotests/185 +2 -1
@@ -182,7 +182,8 @@ _send_qemu_cmd $h \ 'target': '$TEST_IMG.copy', 'format': '$IMGFMT', 'sync': 'full', - 'speed': 65536 } }" \ + 'speed': 65536, + 'x-perf': {'max-chunk': 65536} } }" \ "return" # If we don't sleep here 'quit' command races with disk I/O
tests/qemu-iotests/185.out +1 -1
@@ -61,7 +61,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off { 'execute': 'qmp_capabilities' } {"return": {}} -{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536 } } +{ 'execute': 'drive-backup', 'arguments': { 'device': 'disk', 'target': 'TEST_DIR/t.IMGFMT.copy', 'format': 'IMGFMT', 'sync': 'full', 'speed': 65536, 'x-perf': {'max-chunk': 65536} } } Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=67108864 lazy_refcounts=off refcount_bits=16 {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}}
Signature · 1 line
2.21.3
PATCH v3 15/25 iotests: 219: prepare for backup over block-copy
In reply to Vladimir Sementsov-OgievskiyThe further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. Test 219 depends on specified chunk-size. Update it for explicit chunk-size for backup as for mirror.
--- tests/qemu-iotests/219 | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
tests/qemu-iotests/219 +7 -6
@@ -203,13 +203,13 @@ with iotests.FilePath('disk.img') as disk_path, \ # but related to this also automatic state transitions like job # completion), but still get pause points often enough to avoid making this # test very slow, it's important to have the right ratio between speed and - # buf_size. + # copy-chunk-size. # - # For backup, buf_size is hard-coded to the source image cluster size (64k), - # so we'll pick the same for mirror. The slice time, i.e. the granularity - # of the rate limiting is 100ms. With a speed of 256k per second, we can - # get four pause points per second. This gives us 250ms per iteration, - # which should be enough to stay deterministic. + # Chose 64k copy-chunk-size both for mirror (by buf_size) and backup (by + # x-max-chunk). The slice time, i.e. the granularity of the rate limiting + # is 100ms. With a speed of 256k per second, we can get four pause points + # per second. This gives us 250ms per iteration, which should be enough to + # stay deterministic. test_job_lifecycle(vm, 'drive-mirror', has_ready=True, job_args={ 'device': 'drive0-node',
@@ -226,6 +226,7 @@ with iotests.FilePath('disk.img') as disk_path, \ 'target': copy_path, 'sync': 'full', 'speed': 262144, + 'x-perf': {'max-chunk': 65536}, 'auto-finalize': auto_finalize, 'auto-dismiss': auto_dismiss, })
Signature · 1 line
2.21.3
Re: [PATCH v3 15/25] iotests: 219: prepare for backup over block-copy
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
The further change of moving backup to be a one block-copy call will make copying chunk-size and cluster-size two separate things. So, even with 64k cluster sized qcow2 image, default chunk would be 1M. Test 219 depends on specified chunk-size. Update it for explicit chunk-size for backup as for mirror.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- tests/qemu-iotests/219 | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
PATCH v3 16/25 iotests: 257: prepare for backup over block-copy
In reply to Vladimir Sementsov-OgievskiyIotest 257 dumps a lot of in-progress information of backup job, such as offset and bitmap dirtiness. Further commit will move backup to be one block-copy call, which will introduce async parallel requests instead of plain cluster-by-cluster copying. To keep things deterministic, allow only one worker (only one copy request at a time) for this test.
--- tests/qemu-iotests/257 | 1 + tests/qemu-iotests/257.out | 306 ++++++++++++++++++------------------- 2 files changed, 154 insertions(+), 153 deletions(-)
tests/qemu-iotests/257 +1 -0
@@ -191,6 +191,7 @@ def blockdev_backup(vm, device, target, sync, **kwargs): target=target, sync=sync, filter_node_name='backup-top', + x_perf={'max-workers': 1}, **kwargs) return result
tests/qemu-iotests/257.out +153 -153
1,043 lines not on this page · raw message
Signature · 1 line
2.21.3
PATCH v3 17/25 block/block-copy: make progress_bytes_callback optional
In reply to Vladimir Sementsov-OgievskiyWe are going to stop use of this callback in the following commit. Still the callback handling code will be dropped in a separate commit. So, for now let's make it optional.
--- block/block-copy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
block/block-copy.c +3 -1
@@ -454,7 +454,9 @@ static coroutine_fn int block_copy_task_entry(AioTask *task) t->call_state->error_is_read = error_is_read; }else {progress_work_done (t->s->progress, t->bytes); - t->s->progress_bytes_callback (t->bytes, t->s->progress_opaque); +if (t->s->progress_bytes_callback) { + t->s->progress_bytes_callback (t->bytes, t->s->progress_opaque); + } }co_put_to_shres (t->s->mem, t->bytes);block_copy_task_end (t, ret);
Signature · 1 line
2.21.3
Re: [PATCH v3 17/25] block/block-copy: make progress_bytes_callback optional
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
We are going to stop use of this callback in the following commit. Still the callback handling code will be dropped in a separate commit. So, for now let's make it optional.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/block-copy.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
PATCH v3 18/25 block/backup: drop extra gotos from backup_run()
In reply to Vladimir Sementsov-Ogievskiy--- block/backup.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
block/backup.c +5 -7
@@ -236,7 +236,7 @@ static void backup_init_bcs_bitmap(BackupBlockJob *job) static int coroutine_fn backup_run(Job *job, Error **errp) {BackupBlockJob *s =container_of (job, BackupBlockJob, common.job); -int ret =0 ; +int ret;backup_init_bcs_bitmap (s);
@@ -246,13 +246,12 @@ static int coroutine_fn backup_run(Job *job, Error **errp)for (offset =0 ; offset < s->len; ) {if (yield_and_check (s)) { - ret = -ECANCELED ; - goto out; +return -ECANCELED ; } ret =block_copy_reset_unallocated (s->bcs, offset, &count);if (ret <0 ) { - goto out; +return ret; } offset += count;
@@ -273,11 +272,10 @@ static int coroutine_fn backup_run(Job *job, Error **errp)job_yield (job); } }else { - ret =backup_loop (s); +return backup_loop (s); } - out: -return ret; +return 0 ; }static const BlockJobDriver backup_job_driver = {
Signature · 1 line
2.21.3
Re: [PATCH v3 18/25] block/backup: drop extra gotos from backup_run()
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/backup.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
PATCH v3 19/25 backup: move to block-copy
In reply to Vladimir Sementsov-OgievskiyThis brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance.
--- block/backup.c | 187 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 121 insertions(+), 66 deletions(-)
block/backup.c +121 -66
@@ -44,41 +44,17 @@ typedef struct BackupBlockJob {BlockdevOnError on_source_error;BlockdevOnError on_target_error;uint64_t len; -uint64_t bytes_read;int64_t cluster_size;BackupPerf perf;BlockCopyState *bcs; + +bool wait; +BlockCopyCallState *bg_bcs_call; } BackupBlockJob;static const BlockJobDriver backup_job_driver; -static void backup_progress_bytes_callback (int64_t bytes,void *opaque) -{ -BackupBlockJob *s = opaque; - - s->bytes_read += bytes; -} - -static int coroutine_fn backup_do_cow(BackupBlockJob *job, -int64_t offset,uint64_t bytes, -bool *error_is_read) -{ -int ret =0 ; -int64_t start, end;/* bytes */ - - start =QEMU_ALIGN_DOWN (offset, job->cluster_size); - end =QEMU_ALIGN_UP (bytes + offset, job->cluster_size); - -trace_backup_do_cow_enter (job, start, offset, bytes); - - ret =block_copy (job->bcs, start, end - start, true, error_is_read); - -trace_backup_do_cow_return (job, offset, bytes, ret); - -return ret; -} -static void backup_cleanup_sync_bitmap (BackupBlockJob *job,int ret) {BdrvDirtyBitmap *bm;
@@ -158,53 +134,97 @@ static BlockErrorAction backup_error_action(BackupBlockJob *job, } } -static bool coroutine_fn yield_and_check(BackupBlockJob *job) +static void coroutine_fn backup_block_copy_callback(void *opaque) { -uint64_t delay_ns; - -if (job_is_cancelled (&job->common.job)) { -return true; - } - -/* -* We need to yield even for delay_ns = 0 so that bdrv_drain_all() can -* return. Without a yield, the VM would not reboot. -*/ - delay_ns =block_job_ratelimit_get_delay (&job->common, job->bytes_read); - job->bytes_read =0 ; -job_sleep_ns (&job->common.job, delay_ns); +BackupBlockJob *s = opaque; -if (job_is_cancelled (&job->common.job)) { -return true; +if (s->wait) { + s->wait = false; +aio_co_wake (s->common.job.co); + }else { +job_enter (&s->common.job); } - -return false; } static int coroutine_fn backup_loop(BackupBlockJob *job) { -bool error_is_read; -int64_t offset; -BdrvDirtyBitmapIter *bdbi; +BlockCopyCallState *s =NULL ;int ret =0 ; +bool error_is_read; +BlockErrorAction act; + +while (true) {/* retry loop */ + job->bg_bcs_call = s =block_copy_async (job->bcs,0 , +QEMU_ALIGN_UP (job->len, job->cluster_size), + job->perf.max_workers, job->perf.max_chunk, + backup_block_copy_callback, job); + +while (!block_copy_call_finished (s) && + !job_is_cancelled (&job->common.job)) + { +job_yield (&job->common.job); + } - bdbi =bdrv_dirty_iter_new (block_copy_dirty_bitmap (job->bcs)); -while ((offset =bdrv_dirty_iter_next (bdbi)) !=-1 ) { -do { -if (yield_and_check (job)) { - goto out; - } - ret =backup_do_cow (job, offset, job->cluster_size, &error_is_read); -if (ret <0 &&backup_error_action (job, error_is_read, -ret) == -BLOCK_ERROR_ACTION_REPORT ) - { - goto out; - } - }while (ret <0 ); +if (!block_copy_call_finished (s) && +job_is_cancelled (&job->common.job)) + { +/* +* Note that we can't use job_yield() here, as it doesn't work for +* cancelled job. +*/ +block_copy_call_cancel (s); + job->wait = true; +qemu_coroutine_yield (); +assert (block_copy_call_finished (s)); + ret =0 ; + goto out; + } + +if (job_is_cancelled (&job->common.job) || +block_copy_call_succeeded (s)) + { + ret =0 ; + goto out; + } + +if (block_copy_call_cancelled (s)) { +/* +* Job is not cancelled but only block-copy call. This is possible +* after job pause. Now the pause is finished, start new block-copy +* iteration. +*/ +block_copy_call_free (s); +continue ; + } + +/* The only remaining case is failed block-copy call. */ +assert (block_copy_call_failed (s)); + + ret =block_copy_call_status (s, &error_is_read); + act =backup_error_action (job, error_is_read, -ret); +switch (act) { +case BLOCK_ERROR_ACTION_REPORT : + goto out; +case BLOCK_ERROR_ACTION_STOP : +/* +* Go to pause prior to starting new block-copy call on the next +* iteration. +*/ +job_pause_point (&job->common.job); +break ; +case BLOCK_ERROR_ACTION_IGNORE : +/* Proceed to new block-copy call to retry. */ +break ; +default : +abort (); + } + +block_copy_call_free (s); } - out: -bdrv_dirty_iter_free (bdbi); +out: +block_copy_call_free (s); + job->bg_bcs_call =NULL ;return ret; }
@@ -245,7 +265,13 @@ static int coroutine_fn backup_run(Job *job, Error **errp)int64_t count;for (offset =0 ; offset < s->len; ) { -if (yield_and_check (s)) { +if (job_is_cancelled (job)) { +return -ECANCELED ; + } + +job_pause_point (job); + +if (job_is_cancelled (job)) {return -ECANCELED ; }
@@ -278,6 +304,33 @@ static int coroutine_fn backup_run(Job *job, Error **errp)return 0 ; } +static void coroutine_fn backup_pause(Job *job) +{ +BackupBlockJob *s =container_of (job, BackupBlockJob, common.job); + +if (s->bg_bcs_call && !block_copy_call_finished (s->bg_bcs_call)) { +block_copy_call_cancel (s->bg_bcs_call); + s->wait = true; +qemu_coroutine_yield (); + } +} + +static void coroutine_fn backup_set_speed(BlockJob *job, int64_t speed) +{ +BackupBlockJob *s =container_of (job, BackupBlockJob, common); + +/* +* block_job_set_speed() is called first from block_job_create(), when we +* don't yet have s->bcs. +*/ +if (s->bcs) { +block_copy_set_speed (s->bcs, speed); +if (s->bg_bcs_call) { +block_copy_kick (s->bg_bcs_call); + } + } +} +static const BlockJobDriver backup_job_driver = { .job_driver = { .instance_size =sizeof (BackupBlockJob),
@@ -288,7 +341,9 @@ static const BlockJobDriver backup_job_driver = { .commit = backup_commit, .abort = backup_abort, .clean = backup_clean, - } + .pause = backup_pause, + }, + .set_speed = backup_set_speed, }; static int64_t backup_calculate_cluster_size(BlockDriverState *target,
@@ -485,8 +540,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, job->len = len; job->perf = *perf; -block_copy_set_progress_callback (bcs, backup_progress_bytes_callback, job);block_copy_set_progress_meter (bcs, &job->common.job.progress); +block_copy_set_speed (bcs, speed);/* Required permissions are already taken by backup-top target */ block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL,
Signature · 1 line
2.21.3
Re: [PATCH v3 19/25] backup: move to block-copy
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
This brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/backup.c | 187 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 121 insertions(+), 66 deletions(-)
Irrelevant notes below.
block/backup.c
Quoted 4 lines
block/backup.c +0
-0
(Just something I noted when looking for remaining instances of “ratelimit”: backup.c includes ratelimit.h, which I don’t think it needs to do any longer)
[...]
Quoted 2 lines
static int coroutine_fn backup_loop(BackupBlockJob *job)
{[...]
block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 170
Quoted 3 lines
block/backup.c +3
-0+if (!block_copy_call_finished (s) && +job_is_cancelled (&job->common.job)) + {
Just for the sake of clarity: If !block_copy_call_finished(), then job_is_cancelled() must be true, right?
(I.e., could be an assertion instead of the second part of the condition. I don’t mind how it is, but then again, it did made me ask.)
Quoted 17 lines
block/backup.c +17
-0+/* +* Note that we can't use job_yield() here, as it doesn't work for +* cancelled job. +*/ +block_copy_call_cancel (s); + job->wait = true; +qemu_coroutine_yield (); +assert (block_copy_call_finished (s)); + ret =0 ; + goto out; + } + +if (job_is_cancelled (&job->common.job) || +block_copy_call_succeeded (s)) + { + ret =0 ; + goto out;
block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 197
Quoted 10 lines
block/backup.c +10
-0+ } + +if (block_copy_call_cancelled (s)) { +/* +* Job is not cancelled but only block-copy call. This is possible +* after job pause. Now the pause is finished, start new block-copy +* iteration. +*/ +block_copy_call_free (s); +continue ;
If one were to avoid all continues, we could alternatively put the whole error handling into a block_copy_call_failed() condition, and have the block_copy_call_free() common for both the cancel and the fail case.
*shrug*
Quoted 21 lines
block/backup.c +21
-0+ } + +/* The only remaining case is failed block-copy call. */ +assert (block_copy_call_failed (s)); + + ret =block_copy_call_status (s, &error_is_read); + act =backup_error_action (job, error_is_read, -ret); +switch (act) { +case BLOCK_ERROR_ACTION_REPORT : + goto out; +case BLOCK_ERROR_ACTION_STOP : +/* +* Go to pause prior to starting new block-copy call on the next +* iteration. +*/ +job_pause_point (&job->common.job); +break ; +case BLOCK_ERROR_ACTION_IGNORE : +/* Proceed to new block-copy call to retry. */ +break ; +default :
block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 226
Quoted 10 lines
block/backup.c +6
-2+abort (); + } + +block_copy_call_free (s); } - out: -bdrv_dirty_iter_free (bdbi); +out: +block_copy_call_free (s);
Reads a bit weird after the block_copy_call_free(s) at the end of the while (true) loop, but is entirely correct, of course. (And I can’t offer any better alternative.)
Max
Quoted 3 lines
+ job->bg_bcs_call = NULL; return ret;}
Re: [PATCH v3 19/25] backup: move to block-copy
Quoted 10 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
This brings async request handling and block-status driven chunk sizes to backup out of the box, which improves backup performance.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
block/backup.c| 187 ++++++++++++++++++++++++++++++++-----------------1 file changed, 121 insertions(+), 66 deletions(-)Reviewed-by Max Reitz <[email protected]>
Thanks a lot!
My way to this point was started in far 2016 summer.. And final concept is far-far away from my first attempt:)
Quoted 9 lines
Irrelevant notes below.
block/backup.c
Quoted 4 lines
block/backup.c +0
-0(Just something I noted when looking for remaining instances of “ratelimit”:
backup.cincludesratelimit.h, which I don’t think it needs to do any longer)
agree
Quoted 8 lines
[...]
Quoted 2 lines
static int
coroutine_fnbackup_loop(BackupBlockJob *job){[...]
Quoted 7 lines
Quoted 3 lines
+ if (!block_copy_call_finished(s) &&
+job_is_cancelled(&job->common.job))
+ {Just for the sake of clarity: If !block_copy_call_finished(), then
job_is_cancelled()must be true, right?(I.e., could be an assertion instead of the second part of the condition. I don’t mind how it is, but then again, it did made me ask.)
Hmm, keeping in mind previous loop "while (!block_copy_call_finished(s) && !job_is_cancelled(&job->common.job)) {}", yes, you are right.
Quoted 31 lines
Quoted 18 lines
+ /*
+ * Note that we can't usejob_yield()here, as it doesn't work for + * cancelled job.
+ */
+block_copy_call_cancel(s);
+job->wait= true;
+qemu_coroutine_yield();
+ assert(block_copy_call_finished(s));
+ ret = 0;
+ goto out;
+ }block/backup.c +6
-0+ + if (job_is_cancelled(&job->common.job) || + block_copy_call_succeeded(s)) + { + ret = 0; + goto out;block/backup.c static BlockErrorAction backup_error_action(BackupBlockJob *job, · line 197
Quoted 10 lines
block/backup.c +10
-0+ } + + if (block_copy_call_cancelled(s)) { + /* + * Job is not cancelled but only block-copy call. This is possible + * after job pause. Now the pause is finished, start new block-copy + * iteration. + */ + block_copy_call_free(s); + continue;If one were to avoid all continues, we could alternatively put the whole error handling into a
block_copy_call_failed()condition, and have theblock_copy_call_free()common for both the cancel and the fail case.
Quoted 1 line
*shrug*
Hmm. Same emotion, *shrug*:). I like my comment, describing exactly block_copy_call_cancelled() in corresponding if.. Being combined with failure case it would be less demonstrative. But I'm OK with your suggestion too.
Quoted 25 lines
Quoted 25 lines
+ }
block/backup.c +23
-0+ + /* The only remaining case is failed block-copy call. */ + assert(block_copy_call_failed(s)); + + ret = block_copy_call_status(s, &error_is_read); + act = backup_error_action(job, error_is_read, -ret); + switch (act) { + case BLOCK_ERROR_ACTION_REPORT: + goto out; + case BLOCK_ERROR_ACTION_STOP: + /* + * Go to pause prior to starting new block-copy call on the next + * iteration. + */ + job_pause_point(&job->common.job); + break; + case BLOCK_ERROR_ACTION_IGNORE: + /* Proceed to new block-copy call to retry. */ + break; + default: + abort(); + } +
Quoted 9 lines
Quoted 6 lines
+
block_copy_call_free(s);
}
- out:
-bdrv_dirty_iter_free(bdbi);
+out:
+block_copy_call_free(s);Reads a bit weird after the
block_copy_call_free(s)at the end of the while (true) loop, but is entirely correct, of course. (And I can’t offer any better alternative.)
Yes, looks weird.. Probably we'll invent some good refactoring later. If the function doesn't fit into one screen, there must exist some good refactoring)
Signature · 2 lines
Best regards, Vladimir
PATCH v3 20/25 qapi: backup: disable copy_range by default
In reply to Vladimir Sementsov-OgievskiyFurther commit will add a benchmark (scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabled copy_range. So, let's disable copy_range by default.
--- qapi/block-core.json | 2 +- blockdev.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
qapi/block-core.json +1 -1
@@ -1376,7 +1376,7 @@ # Optional parameters for backup. These parameters don't affect # functionality, but may significantly affect performance. # -# @use-copy-range: Use copy offloading. Default true. +# @use-copy-range: Use copy offloading. Default false. # # @max-workers: Maximum number of parallel requests for the sustained background # copying process. Doesn't influence copy-before-write operations.
blockdev.c +1 -1
@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, {BlockJob *job =NULL ;BdrvDirtyBitmap *bmap =NULL ; -BackupPerf perf = { .use_copy_range = true, .max_workers =64 }; +BackupPerf perf = { .max_workers =64 };int job_flags =JOB_DEFAULT ;if (!backup->has_speed) {
Signature · 1 line
2.21.3
Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Further commit will add a benchmark (
scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabledcopy_range. So, let's disablecopy_rangeby default.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- qapi/block-core.json | 2 +- blockdev.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
Uh, well, er, then why did you add it as true in patch 2?
Do you mean this patch as the basis for a discussion whether it should be true or false by default? I don’t have anything to contribute, though, ergo I don’t mind either way.
Do you have an idea why copy offloading isn’t better?
Max
Quoted 27 lines
qapi/block-core.json +1
-1@@ -1376,7 +1376,7 @@ # Optional parameters for backup. These parameters don't affect # functionality, but may significantly affect performance. # -# @use-copy-range: Use copy offloading. Default true. +# @use-copy-range: Use copy offloading. Default false. # # @max-workers: Maximum number of parallel requests for the sustained background # copying process. Doesn't influence copy-before-write operations.blockdev.c +1
-1@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, {BlockJob *job =NULL ;BdrvDirtyBitmap *bmap =NULL ; -BackupPerf perf = { .use_copy_range = true, .max_workers =64 }; +BackupPerf perf = { .max_workers =64 };int job_flags =JOB_DEFAULT ;if (!backup->has_speed) {
Re: [PATCH v3 20/25] qapi: backup: disable copy_range by default
Quoted 12 lines · Max Reitz
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Further commit will add a benchmark (
scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabledcopy_range. So, let's disablecopy_rangeby default.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
qapi/block-core.json| 2 +-
blockdev.c | 2 +-2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 3 lines
Uh, well, er, then why did you add it as true in patch 2?
Do you mean this patch as the basis for a discussion whether it should be true or false by default?
Yes, I just kept the change in separate, to make it simpler to discuss.. Hmm, or seems I've tried to keep it close to the point when we are going to increase performance anyway (and making default false is needed for good final performance), when if we do the change earlier it would like a degradation during previous 20 commits.
Quoted 4 lines
I don’t have anything to contribute, though, ergo I don’t mind either way.
Do you have an idea why copy offloading isn’t better?
As I understand, it may be really fast for filesystem which will avoid data copying, like btrfs. Why it works slower for others when we enable async + block-status things, I don't know. It should depend on kernel implementation.
I have two additional thoughts:
1. for backup we usually want to do a copy on another node, or at least another hard drive, so we really going to copy the data, not just create some metadata links in fs.
2. even if fs can just create some links to existing clusters instead of copying data, IMHO it is not good for backup:
Main difference of backup from snapshot is that we don't touch active disk: it works in the same way like without backup job running. But if we do create fs-metadata links in backup image to original clusters, it means that on next guest write fs will have to do COW operation, so backup file will reference old cluster from active disk, and new cluster will be allocated for active disk. This means, that backup job influence on active vm disk, increasing its fragmentation.. Possibly I'm wrong, I'm not really good in how filesystems works.
Quoted 28 lines
Quoted 27 lines
qapi/block-core.json +1
-1@@ -1376,7 +1376,7 @@ # Optional parameters for backup. These parameters don't affect # functionality, but may significantly affect performance. # -# @use-copy-range: Use copy offloading. Default true. +# @use-copy-range: Use copy offloading. Default false. # # @max-workers: Maximum number of parallel requests for the sustained background # copying process. Doesn't influence copy-before-write operations.blockdev.c +1
-1@@ -2788,7 +2788,7 @@ static BlockJob *do_backup_common(BackupCommon *backup, { BlockJob *job = NULL; BdrvDirtyBitmap *bmap = NULL; - BackupPerf perf = { .use_copy_range = true, .max_workers = 64 }; + BackupPerf perf = { .max_workers = 64 }; int job_flags = JOB_DEFAULT; if (!backup->has_speed) {
Signature · 2 lines
Best regards, Vladimir
Max Reitz
Reviewed-byRe: [PATCH v3 20/25] qapi: backup: disable copy_range by default
Quoted 13 lines · Vladimir Sementsov-Ogievskiy
Quoted 12 lines · Max Reitz
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Further commit will add a benchmark (
scripts/simplebench/bench-backup.py), which will show that backup works better with async parallel requests (previous commit) and disabledcopy_range. So, let's disablecopy_rangeby default.Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
qapi/block-core.json| 2 +-
blockdev.c | 2 +-2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 5 lines
Quoted 3 lines
Uh, well, er, then why did you add it as true in patch 2?
Do you mean this patch as the basis for a discussion whether it should be true or false by default?
Yes, I just kept the change in separate, to make it simpler to discuss.. Hmm, or seems I've tried to keep it close to the point when we are going to increase performance anyway (and making default false is needed for good final performance), when if we do the change earlier it would like a degradation during previous 20 commits.
Hm, yes, before this series, copy_range is effectively the default, right.
Quoted 7 lines
Quoted 4 lines
I don’t have anything to contribute, though, ergo I don’t mind either way.
Do you have an idea why copy offloading isn’t better?
As I understand, it may be really fast for filesystem which will avoid data copying, like btrfs. Why it works slower for others when we enable async + block-status things, I don't know. It should depend on kernel implementation.
Quoted 7 lines
I have two additional thoughts:
1. for backup we usually want to do a copy on another node, or at least another hard drive, so we really going to copy the data, not just create some metadata links in fs.
2. even if fs can just create some links to existing clusters instead of copying data, IMHO it is not good for backup:
Main difference of backup from snapshot is that we don't touch active disk: it works in the same way like without backup job running. But if we do create fs-metadata links in backup image to original clusters, it means that on next guest write fs will have to do COW operation, so backup file will reference old cluster from active disk, and new cluster will be allocated for active disk. This means, that backup job influence on active vm disk, increasing its fragmentation.. Possibly I'm wrong, I'm not really good in how filesystems works.
Both sound like good points to me. So as far as I’m concerned, disabling copy offloading sounds good to me.
So now this separate patch here makes sense to me:
Perhaps it deserves some explanation in a commit message, though. I.e., that originally, this defaulted to true, as it was before, but now that async copying is implemented, we should let it default to false.
Max
PATCH v3 21/25 block/block-copy: drop unused block_copy_set_progress_callback()
In reply to Vladimir Sementsov-OgievskiyDrop unused code.
--- include/block/block-copy.h | 6 ------ block/block-copy.c | 15 --------------- 2 files changed, 21 deletions(-)
include/block/block-copy.h +0 -6
@@ -18,7 +18,6 @@#include "block/block.h" #include "qemu/co-shared-resource.h" -typedef void (*ProgressBytesCallbackFunc )(int64_t bytes,void *opaque);typedef void (*BlockCopyAsyncCallbackFunc )(void *opaque);typedef struct BlockCopyState BlockCopyState ;typedef struct BlockCopyCallState BlockCopyCallState ;
@@ -28,11 +27,6 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,BdrvRequestFlags write_flags, Error **errp); -void block_copy_set_progress_callback ( -BlockCopyState *s, -ProgressBytesCallbackFunc progress_bytes_callback, -void *progress_opaque); -void block_copy_set_progress_meter (BlockCopyState *s,ProgressMeter *pm);void block_copy_state_free (BlockCopyState *s);
block/block-copy.c +0 -15
@@ -110,9 +110,6 @@ typedef struct BlockCopyState {bool skip_unallocated;ProgressMeter *progress; -/* progress_bytes_callback: called when some copying progress is done. */ -ProgressBytesCallbackFunc progress_bytes_callback; -void *progress_opaque;SharedResource *mem;
@@ -298,15 +295,6 @@ BlockCopyState *block_copy_state_new(BdrvChild *source, BdrvChild *target,return s; } -void block_copy_set_progress_callback ( -BlockCopyState *s, -ProgressBytesCallbackFunc progress_bytes_callback, -void *progress_opaque) -{ - s->progress_bytes_callback = progress_bytes_callback; - s->progress_opaque = progress_opaque; -} -void block_copy_set_progress_meter (BlockCopyState *s,ProgressMeter *pm) { s->progress = pm;
@@ -454,9 +442,6 @@ static coroutine_fn int block_copy_task_entry(AioTask *task) t->call_state->error_is_read = error_is_read; }else {progress_work_done (t->s->progress, t->bytes); -if (t->s->progress_bytes_callback) { - t->s->progress_bytes_callback (t->bytes, t->s->progress_opaque); - } }co_put_to_shres (t->s->mem, t->bytes);block_copy_task_end (t, ret);
Signature · 1 line
2.21.3
Re: [PATCH v3 21/25] block/block-copy: drop unused block_copy_set_progress_callback()
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Drop unused code.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/block-copy.h | 6 ------ block/block-copy.c | 15 --------------- 2 files changed, 21 deletions(-)
PATCH v3 22/25 block/block-copy: drop unused argument of block_copy()
In reply to Vladimir Sementsov-Ogievskiy--- include/block/block-copy.h | 2 +- block/backup-top.c | 2 +- block/block-copy.c | 10 ++-------- 3 files changed, 4 insertions(+), 10 deletions(-)
include/block/block-copy.h +1 -1
@@ -35,7 +35,7 @@ int64_t block_copy_reset_unallocated(BlockCopyState *s, int64_t offset, int64_t *count); int coroutine_fn block_copy(BlockCopyState *s, int64_t offset, int64_t bytes, -bool ignore_ratelimit,bool *error_is_read); +bool ignore_ratelimit);/* * Run block-copy in a coroutine, create corresponding BlockCopyCallState
block/backup-top.c +1 -1
@@ -61,7 +61,7 @@ static coroutine_fn int backup_top_cbw(BlockDriverState *bs, uint64_t offset, off =QEMU_ALIGN_DOWN (offset, s->cluster_size); end =QEMU_ALIGN_UP (offset + bytes, s->cluster_size); -return block_copy (s->bcs, off, end - off, true,NULL ); +return block_copy (s->bcs, off, end - off, true); } static int coroutine_fn backup_top_co_pdiscard(BlockDriverState *bs,
block/block-copy.c +2 -8
@@ -723,7 +723,7 @@ static int coroutine_fn block_copy_common(BlockCopyCallState *call_state) } int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes, -bool ignore_ratelimit,bool *error_is_read) +bool ignore_ratelimit) {BlockCopyCallState call_state = { .s = s,
@@ -733,13 +733,7 @@ int coroutine_fn block_copy(BlockCopyState *s, int64_t start, int64_t bytes, .max_workers = BLOCK_COPY_MAX_WORKERS, }; -int ret =block_copy_common (&call_state); - -if (error_is_read && ret <0 ) { - *error_is_read = call_state.error_is_read; - } - -return ret; +return block_copy_common (&call_state); } static void coroutine_fn block_copy_async_co_entry(void *opaque)
Signature · 1 line
2.21.3
PATCH v3 23/25 simplebench/bench_block_job: use correct shebang line with python3
In reply to Vladimir Sementsov-Ogievskiy--- scripts/simplebench/bench_block_job.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/simplebench/bench_block_job.py +1 -1
@@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 # # Benchmark block jobs #
Signature · 1 line
2.21.3
Re: [PATCH v3 23/25] simplebench/bench_block_job: use correct shebang line with python3
Quoted 4 lines · Vladimir Sementsov-Ogievskiy
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- scripts/simplebench/bench_block_job.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
PATCH v3 24/25 simplebench: bench_block_job: add cmd_options argument
In reply to Vladimir Sementsov-OgievskiyAdd argument to allow additional block-job options.
--- scripts/simplebench/bench-example.py | 2 +- scripts/simplebench/bench_block_job.py | 11 +++++++---- 2 files changed, 8 insertions(+), 5 deletions(-)
scripts/simplebench/bench-example.py +1 -1
@@ -25,7 +25,7 @@ from bench_block_job import bench_block_copy, drv_file, drv_nbd def bench_func(env, case): """ Handle one "cell" of benchmarking table. """ - return bench_block_copy(env['qemu_binary'], env['cmd'], + return bench_block_copy(env['qemu_binary'], env['cmd'], {} case['source'], case['target'])
scripts/simplebench/bench_block_job.py +7 -4
@@ -78,16 +78,19 @@ def bench_block_job(cmd, cmd_args, qemu_args): # Bench backup or mirror -def bench_block_copy(qemu_binary, cmd, source, target): +def bench_block_copy(qemu_binary, cmd, cmd_options, source, target): """Helper to run bench_block_job() for mirror or backup""" assert cmd in ('blockdev-backup', 'blockdev-mirror') source['node-name'] = 'source' target['node-name'] = 'target' - return bench_block_job(cmd, - {'job-id': 'job0', 'device': 'source', - 'target': 'target', 'sync': 'full'}, + cmd_options['job-id'] = 'job0' + cmd_options['device'] = 'source' + cmd_options['target'] = 'target' + cmd_options['sync'] = 'full' + + return bench_block_job(cmd, cmd_options, [qemu_binary, '-blockdev', json.dumps(source), '-blockdev', json.dumps(target)])
Signature · 1 line
2.21.3
PATCH v3 25/25 simplebench: add bench-backup.py
In reply to Vladimir Sementsov-OgievskiyAdd script to benchmark new backup architecture.
--- scripts/simplebench/bench-backup.py | 165 ++++++++++++++++++++++++++++ 1 file changed, 165 insertions(+) create mode 100755 scripts/simplebench/bench-backup.py
scripts/simplebench/bench-backup.py new file +165 -0
@@ -0,0 +1,165 @@ +#!/usr/bin/env python3 +# +# Bench backup block-job +# +# Copyright (c) 2020 Virtuozzo International GmbH. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +import argparse +import json + +import simplebench +from results_to_text import results_to_text +from bench_block_job import bench_block_copy, drv_file, drv_nbd + + +def bench_func(env, case): + """ Handle one "cell" of benchmarking table. """ + cmd_options = env['cmd-options'] if 'cmd-options' in env else {} + return bench_block_copy(env['qemu-binary'], env['cmd'], + cmd_options, + case['source'], case['target']) + + +def bench(args): + test_cases = [] + + sources = {} + targets = {} + for d in args.dir: + label, path = d.split(':') + sources[label] = drv_file(path + '/test-source') + targets[label] = drv_file(path + '/test-target') + + if args.nbd: + nbd = args.nbd.split(':') + host = nbd[0] + port = '10809' if len(nbd) == 1 else nbd[1] + drv = drv_nbd(host, port) + sources['nbd'] = drv + targets['nbd'] = drv + + for t in args.test: + src, dst = t.split(':') + + test_cases.append({ + 'id': t, + 'source': sources[src], + 'target': targets[dst] + }) + + binaries = [] # list of (<label>, <path>, [<options>]) + for i, q in enumerate(args.env): + name_path = q.split(':') + if len(name_path) == 1: + label = f'q{i}' + path_opts = name_path[0].split(',') + else: + label = name_path[0] + path_opts = name_path[1].split(',') + + binaries.append((label, path_opts[0], path_opts[1:])) + + test_envs = [] + + bin_paths = {} + for i, q in enumerate(args.env): + opts = q.split(',') + label_path = opts[0] + opts = opts[1:] + + if ':' in label_path: + label, path = label_path.split(':') + bin_paths[label] = path + elif label_path in bin_paths: + label = label_path + path = bin_paths[label] + else: + path = label_path + label = f'q{i}' + bin_paths[label] = path + + x_perf = {} + is_mirror = False + for opt in opts: + if opt == 'mirror': + is_mirror = True + elif opt == 'copy-range=on': + x_perf['use-copy-range'] = True + elif opt == 'copy-range=off': + x_perf['use-copy-range'] = False + elif opt.startswith('max-workers='): + x_perf['max-workers'] = int(opt.split('=')[1]) + + if is_mirror: + assert not x_perf + test_envs.append({ + 'id': f'mirror({label})', + 'cmd': 'blockdev-mirror', + 'qemu-binary': path + }) + else: + test_envs.append({ + 'id': f'backup({label})\n' + '\n'.join(opts), + 'cmd': 'blockdev-backup', + 'cmd-options': {'x-perf': x_perf} if x_perf else {}, + 'qemu-binary': path + }) + + result = simplebench.bench(bench_func, test_envs, test_cases, count=3) + with open('results.json', 'w') as f: + json.dump(result, f, indent=4) + print(results_to_text(result)) + + +class ExtendAction(argparse.Action): + def __call__(self, parser, namespace, values, option_string=None): + items = getattr(namespace, self.dest) or [] + items.extend(values) + setattr(namespace, self.dest, items) + + +if __name__ == '__main__': + p = argparse.ArgumentParser('Backup benchmark', epilog=''' +ENV format + + (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror] + + LABEL short name for the binary + PATH path to the binary + max-workers set x-perf.max-workers of backup job + use-copy-range set x-perf.disable-copy-range of backup job + mirror use mirror job instead of backup''', + formatter_class=argparse.RawTextHelpFormatter) + p.add_argument('--env', nargs='+', help='''\ +Qemu binaries with labels and options, see below +"ENV format" section''', + action=ExtendAction) + p.add_argument('--dir', nargs='+', help='''\ +Directories, each containing "test-source" and/or +"test-target" files, raw images to used in +benchmarking. File path with label, like +label:/path/to/directory''', + action=ExtendAction) + p.add_argument('--nbd', help='''\ +host:port for remote NBD image, (or just host, for +default port 10809). Use it in tests, label is "nbd" +(but you cannot create test nbd:nbd).''') + p.add_argument('--test', nargs='+', help='''\ +Tests, in form source-dir-label:target-dir-label''', + action=ExtendAction) + + bench(p.parse_args())
Signature · 1 line
2.21.3
Re: [PATCH v3 25/25] simplebench: add bench-backup.py
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add script to benchmark new backup architecture.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- scripts/simplebench/bench-backup.py | 165 ++++++++++++++++++++++++++++ 1 file changed, 165 insertions(+)create mode 100755
scripts/simplebench/bench-backup.py
Loose review, because I’m not too involved, and this is “just” a performance test, too.
Result: Looks good to me, two minor things below.
Quoted 5 lines
scripts/simplebench/bench-backup.py new file +0
-0
[...]
scripts/simplebench/bench-backup.py line 43
Quoted 7 lines
scripts/simplebench/bench-backup.py new file +7
-0+def bench(args): + test_cases = [] + + sources = {} + targets = {} + for d in args.dir: + label, path = d.split(':')
Should this be d.split(':', 1) to accommodate path names with colons in them?
Quoted 13 lines
scripts/simplebench/bench-backup.py new file +13
-0+ sources[label] = drv_file(path + '/test-source') + targets[label] = drv_file(path + '/test-target') + + if args.nbd: + nbd = args.nbd.split(':') + host = nbd[0] + port = '10809' if len(nbd) == 1 else nbd[1] + drv = drv_nbd(host, port) + sources['nbd'] = drv + targets['nbd'] = drv + + for t in args.test: + src, dst = t.split(':')
scripts/simplebench/bench-backup.py line 66
Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10
-0+ + test_cases.append({ + 'id': t, + 'source': sources[src], + 'target': targets[dst] + }) + + binaries = [] # list of (<label>, <path>, [<options>]) + for i, q in enumerate(args.env): + name_path = q.split(':')
(Same here)
Quoted 9 lines
scripts/simplebench/bench-backup.py new file +9
-0+ if len(name_path) == 1: + label = f'q{i}' + path_opts = name_path[0].split(',') + else: + label = name_path[0] + path_opts = name_path[1].split(',') + + binaries.append((label, path_opts[0], path_opts[1:])) +
scripts/simplebench/bench-backup.py line 85
Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10
-0+ test_envs = [] + + bin_paths = {} + for i, q in enumerate(args.env): + opts = q.split(',') + label_path = opts[0] + opts = opts[1:] + + if ':' in label_path: + label, path = label_path.split(':')
(And here)
Quoted 8 lines
scripts/simplebench/bench-backup.py new file +8
-0+ bin_paths[label] = path + elif label_path in bin_paths: + label = label_path + path = bin_paths[label] + else: + path = label_path + label = f'q{i}' + bin_paths[label] = path
[...]
scripts/simplebench/bench-backup.py line 144
Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10
-0+if __name__ == '__main__': + p = argparse.ArgumentParser('Backup benchmark', epilog=''' +ENV format + + (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror] + + LABEL short name for the binary + PATH path to the binary + max-workers set x-perf.max-workers of backup job + use-copy-range set x-perf.disable-copy-range of backup job
s/disable/use/?
Max
Re: [PATCH v3 25/25] simplebench: add bench-backup.py
Quoted 20 lines · Max Reitz
Quoted 7 lines · Vladimir Sementsov-Ogievskiy
Add script to benchmark new backup architecture.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>---
scripts/simplebench/bench-backup.py| 165 ++++++++++++++++++++++++++++1 file changed, 165 insertions(+)create mode 100755
scripts/simplebench/bench-backup.pyLoose review, because I’m not too involved, and this is “just” a performance test, too.
Result: Looks good to me, two minor things below.
Quoted 5 lines
scripts/simplebench/bench-backup.py new file +0
-0[...]
Quoted 9 lines
scripts/simplebench/bench-backup.py line 43
Quoted 7 lines
scripts/simplebench/bench-backup.py new file +7
-0+def bench(args): + test_cases = [] + + sources = {} + targets = {} + for d in args.dir: + label, path = d.split(':')Should this be
d.split(':', 1)to accommodate path names with colons in them?
Yes, it makes sense.
Quoted 18 lines
Quoted 18 lines
+ sources[label] =
drv_file(path + '/test-source')+ targets[label] =drv_file(path + '/test-target')scripts/simplebench/bench-backup.py new file +15
-0+ + if args.nbd: + nbd = args.nbd.split(':') + host = nbd[0] + port = '10809' if len(nbd) == 1 else nbd[1] + drv = drv_nbd(host, port) + sources['nbd'] = drv + targets['nbd'] = drv + + for t in args.test: + src, dst = t.split(':') + + test_cases.append({ + 'id': t, + 'source': sources[src],
Quoted 8 lines
Quoted 6 lines
+ 'target': targets[dst]
+ })
+
+ binaries = [] # list of (<label>, <path>, [<options>]) + for i, q in enumerate(args.env):
+name_path=q.split(':')(Same here)
hmm here we can't distinguish path with ':' inside without a tag from tag:path.. but anyway, using ", 1" constantly is good. Possible failures will be more clear.
Quoted 34 lines
Quoted 10 lines
+ if
len(name_path)== 1:
+ label = f'q{i}'
+path_opts= name_path[0].split(',')
+ else:
+ label = name_path[0]
+path_opts= name_path[1].split(',')scripts/simplebench/bench-backup.py new file +3
-0+ + binaries.append((label, path_opts[0], path_opts[1:])) +scripts/simplebench/bench-backup.py line 85
Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10
-0+ test_envs = [] + + bin_paths = {} + for i, q in enumerate(args.env): + opts = q.split(',') + label_path = opts[0] + opts = opts[1:] + + if ':' in label_path: + label, path = label_path.split(':')(And here)
Quoted 8 lines
+ bin_paths[label] = path
+ eliflabel_pathinbin_paths:
+ label =label_path
+ path = bin_paths[label]
+ else:
+ path =label_path
+ label = f'q{i}'
+ bin_paths[label] = path[...]
Quoted 13 lines
scripts/simplebench/bench-backup.py line 144
Quoted 10 lines
scripts/simplebench/bench-backup.py new file +10
-0+if __name__ == '__main__': + p = argparse.ArgumentParser('Backup benchmark', epilog=''' +ENV format + + (LABEL:PATH|LABEL|PATH)[,max-workers=N][,use-copy-range=(on|off)][,mirror] + + LABEL short name for the binary + PATH path to the binary + max-workers set x-perf.max-workers of backup job + use-copy-range set x-perf.disable-copy-range of backup jobs/disable/use/?
yes
Signature · 2 lines
Best regards, Vladimir
ping
Quoted 143 lines · Vladimir Sementsov-Ogievskiy
Hi all!
The series turn backup into series of
block_copy_asynccalls, covering the whole disk, so we get block-status based paralallel async requests out of the box, which gives performance gain:All results are in seconds
----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------
A B C D E F mirror(old) backup(old) backup(old) backup(new) backup(new) backup(new) copy-range=on copy-range=off copy-range=on copy-range=on max-workers=1 hdd-ext4:hdd-ext4 19 20 21 ± 14% 19 51 ± 12% 22 ± 24% A+5% A+12% B+6% A+3% B-2% C-8% A+174% B+161% C+145% D+165% A+18% B+12% C+5% D+14% E-57% hdd-ext4:ssd-ext4 8.7 9.4 ± 3% 9.6 ± 2% 8.8 24 ± 2% 8.9 A+8% A+10% B+2% A+1% B-7% C-9% A+174% B+155% C+149% D+173% A+2% B-5% C-8% D+1% E-63% ssd-ext4:hdd-ext4 9 12 ± 9% 11 ± 7% 9.7 ± 7% 11 ± 2% 10 ± 3% A+36% A+28% B-6% A+7% B-21% C-16% A+21% B-11% C-5% D+13% A+16% B-14% C-9% D+8% E-4% ssd-ext4:ssd-ext4 4.4 11 ± 4% 10 ± 3% 4.7 5.7 10 ± 5% A+143% A+134% B-4% A+6% B-56% C-55% A+30% B-46% C-45% D+22% A+133% B-4% C-1% D+119% E+79% hdd-xfs:hdd-xfs 19 20 ± 3% 20 20 45 ± 4% 19 A+3% A+4% B+1% A+3% B+0% C-1% A+131% B+125% C+122% D+125% A-1% B-4% C-4% D-3% E-57% hdd-xfs:ssd-xfs 9.1 9.9 ± 4% 9.5 9.1 ± 3% 23 ± 2% 9.2 A+8% A+4% B-4% A+0% B-8% C-4% A+151% B+132% C+142% D+151% A+1% B-7% C-3% D+1% E-60% ssd-xfs:hdd-xfs 9.1 11 ± 9% 11 9.5 ± 4% 12 ± 22% 11 ± 3% A+16% A+22% B+6% A+4% B-10% C-15% A+32% B+14% C+8% D+26% A+18% B+2% C-4% D+13% E-10% ssd-xfs:ssd-xfs 4.1 8.7 ± 7% 9.2 ± 5% 4.5 ± 2% 5.7 ± 3% 9.7 ± 5% A+113% A+126% B+6% A+11% B-48% C-51% A+40% B-34% C-38% D+27% A+138% B+12% C+5% D+115% E+70% ssd-ext4:nbd 9.1 ± 2% 37 37 ± 2% 11 11 ± 3% 19 ± 2% A+302% A+304% B+1% A+18% B-71% C-71% A+18% B-71% C-71% D+0% A+106% B-49% C-49% D+74% E+75% nbd:ssd-ext4 9 30 ± 3% 31 9 9 17 A+237% A+245% B+2% A+0% B-70% C-71% A+0% B-70% C-71% D+0% A+93% B-43% C-44% D+93% E+93%----------------- ----------- ------------- -------------- --------------------- -------------------------------- ------------------------------------
Here column B is current backup and column D is new backup with default parameters.
Mirror is still faster, but we are very close to it.
v3:
01: add Max's r-b
02: change to perf.use-copy-range
03: add Max's r-b
04: - more explicit finish status of asyncblock_copy
-block_copy_asyncalways return non-NULL
- personal opaque for new cb
05: - new arguments added in this patch
- no default value for arguments inblock_copy_async()
06: new
07: - caller does_kick()by hand
- grammar in commit msg
- add new parameter in _this_ patch
- switch to oppositeignore_ratelimit
08: cancel now is async
09,10: add Max's r-b
11: changed a lot
12: add timeout
14: rebase on x-perf, keep r-b
15: rebase on x-perf
16: rebase on x-perf, keep r-b
17,18: new
19: now onlybackup.cis changed in this patch, changed a lot 20,21: new
22: rebased, keep r-b
23: new, split from 24
24: drop unrelated change (now patch23), keep r-b
25: changed a lot, explicitly specify options for each env (test table column)To run benchmark do the following:
prepare images:
In a directories, where you want to place source and target images, prepare images by:for img in test-source test-target; do
./qemu-img create -f raw $img 1000M;
./qemu-img bench -c 1000 -d 1 -f raw -s 1M -w --pattern=0xff $img
doneprepare similar image for nbd server, and start it somewhere by
qemu-nbd --persistent --nocache -f raw IMAGEThen, run benchmark, like this:
./bench-backup.py --env old:/work/src/qemu/up-backup-block-copy-master/build/qemu-system-x86_64,mirror old,copy-range=on old,copy-range=off new:../../build/qemu-system-x86_64 new,copy-range=on new,copy-range=on,max-workers=1 --dir hdd-ext4:/test-a hdd-xfs:/test-b ssd-ext4:/ssd ssd-xfs:/ssd-xfs --test $(for fs in ext4 xfs; do echo hdd-$fs:hdd-$fs hdd-$fs:ssd-$fs ssd-$fs:hdd-$fs ssd-$fs:ssd-$fs; done) --nbd 192.168.100.5 --test ssd-ext4:nbd nbd:ssd-ext4(you may simply reduce number of directories/test-cases, use --help for
help)Vladimir Sementsov-Ogievskiy (25):
iotests: 129 don't check backup "busy"
qapi: backup: add perf.use-copy-range parameterblock/block-copy: More explicitcall_stateblock/block-copy: implementblock_copy_asyncblock/block-copy: addmax_chunkandmax_workersparametersblock/block-copy: add list of all call-statesblock/block-copy: add ratelimit to block-copyblock/block-copy: addblock_copy_cancel
blockjob: addset_speedto BlockJobDriver
job: calljob_enterfromjob_user_pause
qapi: backup: add max-chunk and max-workers to x-perf struct
iotests: 56: prepare for backup over block-copy
iotests: 129: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 257: prepare for backup over block-copyblock/block-copy: makeprogress_bytes_callbackoptional block/backup: drop extra gotos frombackup_run()
backup: move to block-copy
qapi: backup: disablecopy_rangeby defaultblock/block-copy: drop unusedblock_copy_set_progress_callback()block/block-copy: drop unused argument ofblock_copy()simplebench/bench_block_job: use correct shebang line with python3
simplebench:bench_block_job: addcmd_optionsargument
simplebench: addbench-backup.pyqapi/block-core.json | 26 ++- block/backup-top.h | 1 + include/block/block-copy.h | 58 ++++- include/block/block_int.h | 3 + include/block/blockjob_int.h | 2 + block/backup-top.c | 6 +- block/backup.c | 233 ++++++++++++------- block/block-copy.c | 227 +++++++++++++++--- block/replication.c | 2 + blockdev.c | 14 ++ blockjob.c | 6 + job.c | 1 + scripts/simplebench/bench-backup.py | 165 +++++++++++++ scripts/simplebench/bench-example.py | 2 +- scripts/simplebench/bench_block_job.py | 13 +- tests/qemu-iotests/056 | 9 +- tests/qemu-iotests/129 | 3 +- tests/qemu-iotests/185 | 3 +- tests/qemu-iotests/185.out | 2 +- tests/qemu-iotests/219 | 13 +- tests/qemu-iotests/257 | 1 + tests/qemu-iotests/257.out | 306 ++++++++++++------------- 22 files changed, 798 insertions(+), 298 deletions(-)create mode 100755
scripts/simplebench/bench-backup.py
Signature · 2 lines
Best regards, Vladimir