Skip to content

backup performance: block_status + async

Vladimir Sementsov-Ogievskiy qemu-devel 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest23 patches 19 repliesIn review
  2. v325 patches 27 repliesSuperseded
  3. v220 patches 43 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v11 patch1 replyApplied

Changes in v4 from the cover letter

- add a lot of Max's r-b's, thanks!

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/23qapi: backup: add perf.use-copy-range parameterblock/backup-top.c block/backup-top.h block/backup.c +6 more+41 -60 replies
2/23block/block-copy: More explicit call_stateblock/block-copy.c+38 -130 replies
3/23block/block-copy: implement block_copy_asyncblock/block-copy.c include/block/block-copy.h+106 -40 replies
4/23block/block-copy: add max_chunk and max_workers parametersblock/block-copy.c include/block/block-copy.h+15 -20 replies
5/23block/block-copy: add list of all call-statesblock/block-copy.c+10 -10 replies
6/23block/block-copy: add ratelimit to block-copyblock/backup-top.c block/backup.c block/block-copy.c +1 more+51 -40 replies
7/23block/block-copy: add block_copy_cancelblock/block-copy.c include/block/block-copy.h+32 -50 replies
8/23blockjob: add set_speed to BlockJobDriverblockjob.c include/block/blockjob_int.h+8 -00 replies
9/23job: call job_enter from job_pausejob.c tests/qemu-iotests/109.out+27 -05 replies
10/23qapi: backup: add max-chunk and max-workers to x-perf structblock/backup.c block/replication.c blockdev.c +1 more+43 -80 replies
11/23iotests: 56: prepare for backup over block-copytests/qemu-iotests/056+7 -20 replies
12/23iotests: 185: prepare for backup over block-copytests/qemu-iotests/185 tests/qemu-iotests/185.out+4 -20 replies
13/23iotests: 219: prepare for backup over block-copytests/qemu-iotests/219+7 -60 replies
14/23iotests: 257: prepare for backup over block-copytests/qemu-iotests/257 tests/qemu-iotests/257.out+154 -1530 replies
15/23block/block-copy: make progress_bytes_callback optionalblock/block-copy.c+3 -10 replies
16/23block/backup: drop extra gotos from backup_run()block/backup.c+5 -70 replies
17/23backup: move to block-copyblock/backup.c+120 -670 replies
18/23qapi: backup: disable copy_range by defaultblockdev.c qapi/block-core.json+2 -20 replies
19/23block/block-copy: drop unused block_copy_set_progress_callback()block/block-copy.c include/block/block-copy.h+0 -210 replies
20/23block/block-copy: drop unused argument of block_copy()block/backup-top.c block/block-copy.c include/block/block-copy.h+4 -100 replies
21/23simplebench/bench_block_job: use correct shebang line with python3scripts/simplebench/bench_block_job.py+1 -10 replies
22/23simplebench: bench_block_job: add cmd_options argumentscripts/simplebench/bench-example.py scripts/simplebench/bench_block_job.py+8 -50 replies
23/23simplebench: add bench-backup.pyscripts/simplebench/bench-backup.py+167 -01 reply

Reviews trailers per patch

22 of 23 patches have a Reviewed-by or Acked-by of their own; the series has one for all of them.

PatchMax ReitzVladimir Sementsov-Ogievskiy
SeriesR-b
1/23R-b
2/23R-b
3/23R-b
4/23R-b
5/23R-b
6/23R-b
7/23R-b
8/23R-b
9/23R-b
10/23R-b
11/23R-b
12/23R-b
13/23R-b
14/23R-b
15/23R-b
16/23R-b
17/23R-b
18/23R-b
19/23R-b
20/23R-b
21/23R-b
22/23R-b
23/23

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH v4 00/23] backup performance: block_status + async

Hi Max!
I applied my series onto yours 129-fixing and found, that 129 fails for backup. And setting small max-chunk and even max-workers to 1 doesn't help! (setting speed like in v3 still helps).

And I found, that the problem is that really, the whole backup job goes during drain, because in new architecture we do just job_yield() during the whole background block-copy.

This leads to modifying the existing patch in the series, which does job_enter() from job_user_pause: we just need call job_enter() from job_pause() to cover not only user pauses but also drained_begin.

So, now I don't need any additional fixing of 129.

Changes in v4:
- add a lot of Max's r-b's, thanks!

03: fix over-80 line (in comment), add r-b
09: was "[PATCH v3 10/25] job: call job_enter from job_user_pause",
now changed to finally fix 129 iotest, drop r-b

10: squash-in additional wording on max-chunk, fix error message, keep r-b 17: drop extra include, assert job_is_cancelled() instead of check, add r-b 18: adjust commit message, add r-b
23: add comments and assertion, about the fact that test doesn't support
paths with colon inside
fix s/disable-copy-range/use-copy-range/

Vladimir Sementsov-Ogievskiy (23):
qapi: backup: add perf.use-copy-range parameter block/block-copy: More explicit call_state block/block-copy: implement block_copy_async block/block-copy: add max_chunk and max_workers parameters block/block-copy: add list of all call-states block/block-copy: add ratelimit to block-copy block/block-copy: add block_copy_cancel
blockjob: add set_speed to BlockJobDriver
job: call job_enter from job_pause
qapi: backup: add max-chunk and max-workers to x-perf struct
iotests: 56: prepare for backup over block-copy
iotests: 185: prepare for backup over block-copy
iotests: 219: prepare for backup over block-copy
iotests: 257: prepare for backup over block-copy block/block-copy: make progress_bytes_callback optional block/backup: drop extra gotos from backup_run()
backup: move to block-copy
qapi: backup: disable copy_range by default block/block-copy: drop unused block_copy_set_progress_callback() block/block-copy: drop unused argument of block_copy() simplebench/bench_block_job: use correct shebang line with python3
simplebench: bench_block_job: add cmd_options argument
simplebench: add bench-backup.py

 qapi/block-core.json                   |  28 ++-
 block/backup-top.h                     |   1 +
 include/block/block-copy.h             |  61 ++++-
 include/block/block_int.h              |   3 +
 include/block/blockjob_int.h           |   2 +
 block/backup-top.c                     |   6 +-
 block/backup.c                         | 233 ++++++++++++-------
 block/block-copy.c                     | 227 +++++++++++++++---
 block/replication.c                    |   2 +
 blockdev.c                             |  14 ++
 blockjob.c                             |   6 +
 job.c                                  |   3 +
 scripts/simplebench/bench-backup.py    | 167 ++++++++++++++
 scripts/simplebench/bench-example.py   |   2 +-
 scripts/simplebench/bench_block_job.py |  13 +-
 tests/qemu-iotests/056                 |   9 +-
 tests/qemu-iotests/109.out             |  24 ++
 tests/qemu-iotests/185                 |   3 +-
 tests/qemu-iotests/185.out             |   3 +-
 tests/qemu-iotests/219                 |  13 +-
 tests/qemu-iotests/257                 |   1 +
 tests/qemu-iotests/257.out             | 306 ++++++++++++-------------
 22 files changed, 830 insertions(+), 297 deletions(-)
 create mode 100755 scripts/simplebench/bench-backup.py
Signature · 1 line
2.29.2