Skip to content

block: incremental backup series

John Snow qemu-devel 8 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v14 Latest19 patches 8 repliesIn review
  2. v1317 patches 23 repliesSuperseded
  3. v1217 patches 37 repliesSuperseded
Earlier versions · 5
VersionPostedPatchesRepliesStatus
v1113 patches40 repliesSuperseded
v1013 patches8 repliesSuperseded
v910 patches16 repliesSuperseded
v810 patches13 repliesSuperseded
v710 patches16 repliesSuperseded

Changes in v14 from the cover letter

- 04: Edited description.
- 10: Fixed the add transaction. I hope.
- 18: Added a patch that allows bitmaps to be resized via bdrv_truncate().
- 19: Added a documentation file.

Patches in v14 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/19qapi: Add optional field "name" to block dirty bitmapblock.c block/mirror.c include/block/block.h +2 more+42 -50 replies
2/19qmp: Ensure consistent granularity typeblock.c block/mirror.c include/block/block.h +2 more+11 -100 replies
3/19qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-removeblock.c block/mirror.c blockdev.c +3 more+228 -91 reply
4/19block: Introduce bdrv_dirty_bitmap_granularity()block.c include/block/block.h+7 -21 reply
5/19hbitmap: add hbitmap_mergeinclude/qemu/hbitmap.h util/hbitmap.c+43 -00 replies
6/19qmp: Add block-dirty-bitmap-enable and block-dirty-bitmap-disableblock.c blockdev.c include/block/block.h +2 more+106 -00 replies
7/19block: Add bitmap successorsblock.c blockdev.c include/block/block.h +1 more+138 -10 replies
8/19qmp: Add support of "dirty-bitmap" sync mode for drive-backupblock.c block/backup.c block/mirror.c +6 more+172 -340 replies
9/19qmp: add block-dirty-bitmap-clearblock.c blockdev.c include/block/block.h +2 more+84 -00 replies
10/19qapi: Add transaction support to block-dirty-bitmap operationsblockdev.c qapi-schema.json+179 -11 reply
11/19qmp: Add dirty bitmap status fields in query-blockblock.c qapi/block-core.json+8 -10 replies
12/19block: add BdrvDirtyBitmap documentationblock.c+5 -50 replies
13/19block: Ensure consistent bitmap function prototypesblock.c block/backup.c block/mirror.c +3 more+26 -350 replies
14/19iotests: add invalid input incremental backup teststests/qemu-iotests/112 tests/qemu-iotests/112.out tests/qemu-iotests/group+95 -00 replies
15/19iotests: add simple incremental backup casetests/qemu-iotests/112 tests/qemu-iotests/112.out+124 -20 replies
16/19iotests: add transactional incremental backup testtests/qemu-iotests/112 tests/qemu-iotests/112.out+51 -20 replies
17/19iotests: add incremental backup failure recovery testtests/qemu-iotests/112 tests/qemu-iotests/112.out tests/qemu-iotests/iotests.py+62 -50 replies
18/19block: Resize bitmaps on bdrv_truncateblock.c include/block/block.h include/qemu/hbitmap.h +1 more+78 -02 replies
19/19docs: incremental backup documentationdocs/bitmaps.md+253 -02 replies

Reviews trailers per patch

16 of 19 patches have a Reviewed-by or Acked-by.

PatchEric BlakeMax ReitzStefan Hajnoczi
1/19R-b R-b
2/19R-b R-b
3/19R-b R-b
4/19R-b R-b
5/19R-b R-b
6/19R-b
7/19R-b
8/19R-b
9/19R-b
10/19
11/19R-b
12/19R-b
13/19R-b
14/19R-b
15/19R-b
16/19R-b
17/19R-b
18/19
19/19

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v14 thread

Cover letter [Qemu-devel] [PATCH v14 00/19] block: incremental backup series

This series requires: [PATCH v3] blkdebug: fix "once" rule

Salutations, weary traveler. Might I interest you in some incremental backups? Only slightly used, just like new!

This patchset enables the in-memory part of the incremental backup feature. Bitmap persistence and bitmap migration will be addressed in future patchsets by Vladimir Sementsov-Ogievskiy.

This series and most patches were originally by Fam Zheng, though I have corrupted them far beyond their original presentation.

Highlights (since v11):
- Renamed "node-ref" parameter to "node" in all QMP patches. (A rose by any other name? ...)
- Fuller, more robust iotests.
- Fixed progress percentages for incremental backup jobs.
- This patch relies upon a blkdebug fix for the iotests.
- The ability to resize bitmaps (as part of bdrv_truncate) has been included.
- Documentation for the user-perspective has been added. Developers are left to wonder what they did to deserve this.

V14:
- 04: Edited description.
- 10: Fixed the add transaction. I hope.
- 18: Added a patch that allows bitmaps to be resized via bdrv_truncate().
- 19: Added a documentation file.

V13:
- 02: New patch, trying to iron out type issues.
- 03: Patch no longer includes type fixes.
- 04: Patch no longer includes tpye fixes.
- 07: Error message fixes.
- 08: Removed superfluous empty lines
- Progress updates occur before cancellation check.
- 09: Fixed a critical bug where I added an extra '-' to docs.
- Corrected clear semantics (checks for disabled status)
- More full stops and newlines removed from error_setg.
- 10: Error message cleanup.

 -     disabled status check added to clear transaction
 -     Added version documentation for new transactions
 -     Refiddled the add transaction to not delete erroneously
       if we abort after an unsuccessful prepare
 - 13: Joined lines where possible.
 - 15: Removed unnecessary imports
 -     "%s" changed to "%i"
 -     'is 0' replaced by '== 0'
 -     removed .add_args(["-S"]) and hmp("c") and added a docstring.
 -     Check for '%' in dirnames.

- 16: removed hmp("c"), added a doctsring.
- xx Dropped blkdebug patch, was #16.
- 17: Removed hmp("c"), added a docstring.
- Moved all blkdebug config into QMP, because Max asked very nicely.

V12:
- Changed authorship from Fam Zheng to John Snow on most patches
- 02: Fix the error_setg leak in bdrv_dirty_bitmap_lookup
- Fix error phrasing in bdrv_dirty_bitmap_lookup
- Renamed "node-ref" to "node" for QMP commands.
- 03: Granularity helper no longer requires a BDS argument.
- 04: Return early if the second bitmap is empty.
- 05: Renamed the 'enabled' field to 'disabled to emphasize what the default operating state is.
- We now guard against bit sets or resets with the bitmap is
disabled, making it a more pure "read only" mode.
- Some documentation phrasing changes.
- 06: Removed explicit "frozen" state in favor of an implicit one. A successor present implies a frozen state.
- Updated all functions that target a single bitmap to use
assertions that the bitmap they are trying to modify is not frozen/disabled.
- Functions that target multiple bitmaps use only a conditional,
and will silently skip disabled bitmaps.
- thaw() function removed. It is implicitly handled in reclaim
and abdicate.
- Added check for return code of hbitmap_merge.
- Functions now check against enable OR disable when in frozen
state, for consistency and simplicity.
- Add "frozen" state documentation to remove/enable/disable
QMP commands.
- 07: Some documentation for bdrv_set_dirty_iter.

 -     Move function calls outside of assert()
 -     Cleanup the unused successor if we do not start the backup
 -     Version documentation added for dirty-bitmap to block-core.json
 -     Job progress is now reported for incremental backup jobs.

- 08: bdrv_dirty_bitmap_clear is now in its own patch, here.
- bdrv_dirty_bitmap_clear no longer takes a BDS argument.
- 09: Added a transaction for bdrv_dirty_bitmap_clear.
- 10: Change 'enabled' field to 'disabled' field, to match above decision in patch 05.
- 12: Removed extraneous BDS arguments from most bitmap functions.
- 13-15: New set of iotests.
- 16: blkdebug fix, already posted upstream.
- 17: Final iotest, testing failure case.

Fam Zheng (1):
qapi: Add optional field "name" to block dirty bitmap

John Snow (18):
qmp: Ensure consistent granularity type
qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
block: Introduce bdrv_dirty_bitmap_granularity()
hbitmap: add hbitmap_merge
qmp: Add block-dirty-bitmap-enable and block-dirty-bitmap-disable
block: Add bitmap successors
qmp: Add support of "dirty-bitmap" sync mode for drive-backup
qmp: add block-dirty-bitmap-clear
qapi: Add transaction support to block-dirty-bitmap operations
qmp: Add dirty bitmap status fields in query-block
block: add BdrvDirtyBitmap documentation
block: Ensure consistent bitmap function prototypes
iotests: add invalid input incremental backup tests
iotests: add simple incremental backup case
iotests: add transactional incremental backup test
iotests: add incremental backup failure recovery test
block: Resize bitmaps on bdrv_truncate
docs: incremental backup documentation

 block.c                       | 246 +++++++++++++++++++++++++--
 block/backup.c                | 149 +++++++++++++---
 block/mirror.c                |  46 ++---
 blockdev.c                    | 387 +++++++++++++++++++++++++++++++++++++++++-
 docs/bitmaps.md               | 253 +++++++++++++++++++++++++++
 hmp.c                         |   3 +-
 include/block/block.h         |  34 +++-
 include/block/block_int.h     |   4 +-
 include/qemu/hbitmap.h        |  21 +++
 migration/block.c             |   9 +-
 qapi-schema.json              |  10 +-
 qapi/block-core.json          | 122 ++++++++++++-
 qmp-commands.hx               |  92 +++++++++-
 tests/qemu-iotests/112        | 315 ++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/112.out    |   5 +
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |   6 +-
 util/hbitmap.c                |  79 +++++++++
 18 files changed, 1691 insertions(+), 91 deletions(-)
 create mode 100644 docs/bitmaps.md
 create mode 100644 tests/qemu-iotests/112
 create mode 100644 tests/qemu-iotests/112.out
Signature · 1 line
1.9.3