Skip to content

block: incremental backup transactions

John Snow qemu-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest0 of 10 patches 3 repliesIn review

Patches in v1 No applies recorded yet

None of the 10 patches the cover letter announces are in this archive.

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [Qemu-devel] [PATCH v2.5 00/10] block: incremental backup transactions

I am not prepared to send a v3 on this, primarily because I am still waffling on whether or not to do the code motion patch that is present in patch #8 of v2 of this series.

However, for the purposes of testing, reviewers may find it convenient to have a new version of this series that applies cleanly in concert with v6 of the transactionless series, so I am presenting an informal "version 2.5" of this series.

Patches are available on github: https://github.com/jnsnow/qemu/commits/incremental-transactions

And here is the v2.5 patchset diff:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/10:[0002] [FC] 'qapi: Add transaction support to block-dirty-bitmap operations' 002/10:[0021] [FC] 'iotests: add transactional incremental backup test' 003/10:[----] [--] 'block: rename BlkTransactionState and BdrvActionOps' 004/10:[----] [--] 'block: re-add BlkTransactionState'
005/10:[0010] [FC] 'block: add transactional callbacks feature' 006/10:[----] [--] 'block: add refcount to Job object'
007/10:[----] [-C] 'block: add delayed bitmap successor cleanup' 008/10:[0051] [FC] 'qmp: Add an implementation wrapper for qmp_drive_backup' 009/10:[0001] [FC] 'block: drive_backup transaction callback support' 010/10:[0033] [FC] 'iotests: 124 - transactional failure test'

01: Removed "Not yet implemented" line from bitmaps.md 02: Rebase fallout
05: Removed superfluous deletion loop pointed out by Max 08: Fallout from removing the code motion patch,
new forward declaration.
09: Fallout from removing the code motion patch,
new forward declaration.
10: Fallout from the rebase.

There are some more changes that need to happen to #10, primarily a bit more refactoring after the changes made to how images are checked.

Quoted 103 lines · John Snow

requires: [email protected]
"[PATCH v4 00/20] block: transactionless incremental backup series"

This series adds support for incremental backup primitives in QMP transactions. It requires my transactionless incremental backup series, currently at v4.

Patch 1 adds basic support for add and clear transactions. Patch 2 tests this basic support.
Patches 3-4 refactor transactions a little bit, to add clarity. Patch 5 adds the framework for error scenarios where only
some jobs that were launched by a transaction complete successfully, and we need to perform context-sensitive cleanup after the transaction itself has already completed.
Patches 6-7 add necessary bookkeeping information to backup job
data structures to take advantage of this new feature.
Patch 8 just moves code.
Patch 9 modifies qmp_drive_backup to support the new feature. Patch 10 implements the new feature for drive_backup transaction actions. Patch 11 tests the new feature.

Lingering questions:
- Is it worth it to add a post-transaction completion event to QMP that signifies all jobs launched by the transaction have completed? This would be of primary interest to libvirt, in particular, but only as a convenience feature.

Thank you,
--John Snow

v2:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/11:[0009] [FC] 'qapi: Add transaction support to block-dirty-bitmap operations' 002/11:[0036] [FC] 'iotests: add transactional incremental backup test' 003/11:[down] 'blockdev: rename BlkTransactionState and BdrvActionOps' 004/11:[down] 'block: re-add BlkTransactionState'
005/11:[0274] [FC] 'block: add transactional callbacks feature' 006/11:[0004] [FC] 'block: add refcount to Job object'
007/11:[0048] [FC] 'block: add delayed bitmap successor cleanup' 008/11:[down] 'block: move transactions beneath qmp interfaces' 009/11:[0084] [FC] 'qmp: Add an implementation wrapper for qmp_drive_backup' 010/11:[0050] [FC] 'block: drive_backup transaction callback support' 011/11:[0004] [FC] 'iotests: 124 - transactional failure test'

01: Fixed indentation.
Fixed QMP commands to behave with new bitmap_lookup from
transactionless-v4.
2.3 --> 2.4.
02: Folded in improvements to qemu-iotest 124 from transactional-v1. 03: NEW
04: NEW
05: A lot:
Don't delete the comma in the transaction actions config use g_new0 instead of g_malloc0
Phrasing: "retcode" --> "Return code"
Use GCC attributes to mark functions as unused until future patches. Added some data structure documentation.
Many structure and function renames, hopefully to improve readability. Use just one list for all Actions instead of two separate lists. Remove ActionState from the list upon deletion/decref
And many other small tweaks.
06: Comment phrasing.
07: Removed errp parameter from all functions introduced by this commit.

      bdrv_dirty_bitmap_decref --> bdrv_frozen_bitmap_decref

08: NEW
09: _drive_backup --> do_drive_backup()
Forward declarations removed.
10: Rebased on top of drastically modified #05.
Phrasing: "BackupBlockJob" instead of "BackupJob" in comments.
11: Removed extra parameters to wait_incremental() in
test_transaction_failure()

John Snow (11):
qapi: Add transaction support to block-dirty-bitmap operations
iotests: add transactional incremental backup test
block: rename BlkTransactionState and BdrvActionOps
block: re-add BlkTransactionState
block: add transactional callbacks feature
block: add refcount to Job object
block: add delayed bitmap successor cleanup
block: move transactions beneath qmp interfaces
qmp: Add an implementation wrapper for qmp_drive_backup
block: drive_backup transaction callback support
iotests: 124 - transactional failure test

  block.c                    |   65 +-
  block/backup.c             |   29 +-
  blockdev.c                 | 1599 ++++++++++++++++++++++++++++----------------
  blockjob.c                 |   18 +-
  include/block/block.h      |   10 +-
  include/block/block_int.h  |    8 +
  include/block/blockjob.h   |   21 +
  qapi-schema.json           |    6 +-
  tests/qemu-iotests/124     |  170 +++++
  tests/qemu-iotests/124.out |    4 +-
  10 files changed, 1313 insertions(+), 617 deletions(-)

Read the cover letter in its thread