Skip to content

Transactional block-graph modifying API

Vladimir Sementsov-Ogievskiy qemu-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v5 Latest45 patches 32 repliesIn review

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/45block: BlockDriver: add .filtered_child_is_backing fieldblock/commit.c block/mirror.c include/block/block_int-common.h+15 -01 reply
2/45block: introduce bdrv_open_file_child() helperblock.c block/blkdebug.c block/blklogwrites.c +21 more+94 -1001 reply
3/45block/blklogwrites: don't care to remove bs->file child on failureblock/blklogwrites.c+0 -41 reply
4/45test-bdrv-graph-mod: update test_parallel_perm_update test casetests/unit/test-bdrv-graph-mod.c+44 -262 replies
5/45tests-bdrv-drain: bdrv_replace_test driver: declare supports_backingtests/unit/test-bdrv-drain.c+1 -01 reply
6/45test-bdrv-graph-mod: fix filters to be filtersinclude/block/block_int-common.h tests/unit/test-bdrv-graph-mod.c+20 -91 reply
7/45block: document connection between child roles and bs->backing/bs->fileinclude/block/block-common.h+42 -01 reply
8/45block/snapshot: stress that we fallback to primary childblock/snapshot.c+10 -201 reply
9/45Revert "block: Let replace_child_noperm free children"block.c+23 -792 replies
10/45Revert "block: Let replace_child_tran keep indirect pointer"block.c+10 -710 replies
11/45Revert "block: Restructure remove_file_or_backing_child()"block.c+9 -120 replies
12/45Revert "block: Pass BdrvChild ** to replace_child_noperm"block.c+11 -120 replies
13/45block: Manipulate bs->file / bs->backing pointers in .attach/.detachblock.c block/raw-format.c block/snapshot.c +2 more+89 -972 replies
14/45block/snapshot: drop indirection around bdrv_snapshot_fallback_ptrblock/snapshot.c+17 -222 replies
15/45block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_childblock.c+8 -141 reply
16/45block: drop bdrv_detach_child()block.c+18 -271 reply
17/45block: drop bdrv_remove_filter_or_cow_childblock.c+1 -141 reply
18/45block: bdrv_refresh_perms(): allow external tranblock.c+20 -111 reply
19/45block: refactor bdrv_list_refresh_perms to allow any list of nodesblock.c+32 -191 reply
20/45block: make permission update functions publicblock.c include/block/block-global-state.h+7 -41 reply
21/45block: add bdrv_try_set_aio_context_tran transaction actionblock.c+48 -07 replies
22/45block: implemet bdrv_unref_tran()block.c include/block/block-common.h include/block/block-global-state.h +1 more+156 -802 replies
23/45blockdev: refactor transaction to use Transaction APIblockdev.c+186 -1310 replies
24/45blockdev: transactions: rename some thingsblockdev.c+15 -150 replies
25/45blockdev: qmp_transaction: refactor loop to classic forblockdev.c+3 -60 replies
26/45blockdev: transaction: refactor handling transaction propertiesblockdev.c+19 -690 replies
27/45blockdev: qmp_transaction: drop extra generic layerblockdev.c+89 -1890 replies
28/45qapi: block: add blockdev-del transaction actionblockdev.c qapi/block-core.json qapi/transaction.json+95 -150 replies
29/45block: introduce BDRV_O_NOPERM flagblock.c block/monitor/block-hmp-cmds.c blockdev.c +3 more+83 -312 replies
30/45block: bdrv_insert_node(): use BDRV_O_NOPERMblock.c+2 -20 replies
31/45qapi: block: add blockdev-add transaction actionblockdev.c qapi/transaction.json+60 -40 replies
32/45iotests: add blockdev-add-transactiontests/qemu-iotests/tests/blockdev-add-transaction tests/qemu-iotests/tests/blockdev-add-transaction.out+58 -00 replies
33/45block-backend: blk_root(): drop const specifier on return typeblock/block-backend.c include/sysemu/block-backend-global-state.h+2 -20 replies
34/45block/export: add blk_by_export_id()block/export/export.c include/sysemu/block-backend-global-state.h+19 -00 replies
35/45block: make bdrv_find_child() function publicblock.c blockdev.c include/block/block_int-io.h+14 -140 replies
36/45block: bdrv_replace_child_bs(): move to external transactionblock.c block/block-backend.c include/block/block-global-state.h+36 -170 replies
37/45qapi: add x-blockdev-replace commandblockdev.c qapi/block-core.json stubs/blk-by-qdev-id.c +1 more+137 -00 replies
38/45qapi: add x-blockdev-replace transaction actionblock.c blockdev.c include/block/block-global-state.h +1 more+42 -80 replies
39/45block: bdrv_get_xdbg_block_graph(): report export idsblock.c block/export/export.c include/block/export.h +2 more+28 -00 replies
40/45iotests.py: qemu_img_create: use imgfmt by defaulttests/qemu-iotests/iotests.py+2 -00 replies
41/45iotests.py: introduce VM.assert_edges_list() methodtests/qemu-iotests/iotests.py+17 -00 replies
42/45iotests.py: add VM.qmp_check() helpertests/qemu-iotests/iotests.py+4 -00 replies
43/45iotests: add filter-insertiontests/qemu-iotests/tests/filter-insertion tests/qemu-iotests/tests/filter-insertion.out+258 -00 replies
44/45block: bdrv_open_inherit: create BlockBackend only when necessaryblock.c+25 -230 replies
45/45block/copy-before-write: correct permission schemeblock/copy-before-write.c tests/qemu-iotests/tests/image-fleecing tests/qemu-iotests/tests/image-fleecing.out+23 -220 replies

Reviews trailers per patch

11 of 45 patches have a Reviewed-by or Acked-by.

PatchHanna Reitz
1/45R-b
2/45R-b
3/45R-b
4/45
5/45R-b
6/45R-b
7/45
8/45R-b
9/45
10/45
11/45
12/45
13/45
14/45
15/45
16/45R-b
17/45R-b
18/45R-b
19/45R-b
20/45R-b
21/45
22/45
23/45
24/45
25/45
26/45
27/45
28/45
29/45
30/45
31/45
32/45
33/45
34/45
35/45
36/45
37/45
38/45
39/45
40/45
41/45
42/45
43/45
44/45
45/45

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v5 thread

Cover letter [PATCH v5 00/45] Transactional block-graph modifying API

Hi all!

v5: rebased on master, sorry for the noise

That's a big series, which unites some of my previous ones, and completes them with necessary additions to finally implement block-graph modifying API. The series is called "v4" as it inherits "[PATCH v3 00/11] blockdev-replace" (among other things).

After this series, we have blockdev-add, blockdev-del and x-blockdev-replace transaction actions, which allows to insert and remove filters.

Additional challenge is to avoid intermediate permission update. That's and existing paradigm of block graph modifications: first do all the modifications and then refresh the permissions. Now we should bring this paradigm to block-graph modifying transactions: if several graph modifying commands are sequential in one transaction, permission are updated after the last of these commands. The application of this is possibility to correct copy-before-write filter permission requirements (see last patch).

I now unite all these things into one series because:
- they depend on each other and I have to rebase them together when something needs fix or refactoring
- just to resend with my new email address
If needed, parts may go in separate, and I can split them again if necessary.

So, what is here:

1. "[PATCH 00/14] block: cleanup backing and file handling" series, unchanged:

block: BlockDriver: add .filtered_child_is_backing field
block: introduce bdrv_open_file_child() helper block/blklogwrites: don't care to remove bs->file child on failure
test-bdrv-graph-mod: update test_parallel_perm_update test case
tests-bdrv-drain: bdrv_replace_test driver: declare supports_backing
test-bdrv-graph-mod: fix filters to be filters
block: document connection between child roles and

    bs->backing/bs->file

block/snapshot: stress that we fallback to primary child Revert "block: Let replace_child_noperm free children" Revert "block: Let replace_child_tran keep indirect pointer" Revert "block: Restructure remove_file_or_backing_child()" Revert "block: Pass BdrvChild ** to replace_child_noperm"
block: Manipulate bs->file / bs->backing pointers in .attach/.detach block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr

2. implement bdrv_unref_tran() - the key thing to implement blockdev-del transaction action later.
This part inherits from "[PATCH 00/14] block: blockdev-del force=false". Still force=false is not realized and qcow2 is untouched, as the target now is transactional removement.

block: refactor bdrv_remove_file_or_backing_child to bdrv_remove_child
block: drop bdrv_detach_child()
block: drop bdrv_remove_filter_or_cow_child
block: bdrv_refresh_perms(): allow external tran
block: refactor bdrv_list_refresh_perms to allow any list of nodes
block: make permission update functions public
block: add bdrv_try_set_aio_context_tran transaction action
block: implemet bdrv_unref_tran()

3. Move blockdev.c transactions to util/transactions.c API.

blockdev: refactor transaction to use Transaction API
blockdev: transactions: rename some things
blockdev: qmp_transaction: refactor loop to classic for
blockdev: transaction: refactor handling transaction properties
blockdev: qmp_transaction: drop extra generic layer

4. add blockdev-del transaction action

qapi: block: add blockdev-del transaction action

5. add blockdev-add transaction action (inherits from "[PATCH 0/2] blockdev-add transaction")

block: introduce BDRV_O_NOPERM flag
block: bdrv_insert_node(): use BDRV_O_NOPERM
qapi: block: add blockdev-add transaction action
iotests: add blockdev-add-transaction

6. add x-blockdev-replace command and transaction action (inherits from "[PATCH v3 00/11] blockdev-replace")

block-backend: blk_root(): drop const specifier on return type block/export: add blk_by_export_id()
block: make bdrv_find_child() function public
block: bdrv_replace_child_bs(): move to external transaction
qapi: add x-blockdev-replace command
qapi: add x-blockdev-replace transaction action
block: bdrv_get_xdbg_block_graph(): report export ids iotests.py: qemu_img_create: use imgfmt by default iotests.py: introduce VM.assert_edges_list() method iotests.py: add VM.qmp_check() helper
iotests: add filter-insertion

7. Correct permission scheme of copy-before-write filter, with help of new design of graph-modifying API.

block: bdrv_open_inherit: create BlockBackend only when necessary block/copy-before-write: correct permission scheme

 block.c                                       | 871 ++++++++++--------
 block/blkdebug.c                              |   9 +-
 block/blklogwrites.c                          |  11 +-
 block/blkreplay.c                             |   7 +-
 block/blkverify.c                             |   9 +-
 block/block-backend.c                         |  11 +-
 block/bochs.c                                 |   7 +-
 block/cloop.c                                 |   7 +-
 block/commit.c                                |   1 +
 block/copy-before-write.c                     |  24 +-
 block/copy-on-read.c                          |   9 +-
 block/crypto.c                                |  11 +-
 block/dmg.c                                   |   7 +-
 block/export/export.c                         |  31 +
 block/filter-compress.c                       |   6 +-
 block/mirror.c                                |   1 +
 block/monitor/block-hmp-cmds.c                |   2 +-
 block/parallels.c                             |   7 +-
 block/preallocate.c                           |   9 +-
 block/qcow.c                                  |   6 +-
 block/qcow2.c                                 |   8 +-
 block/qed.c                                   |   8 +-
 block/raw-format.c                            |   4 +-
 block/replication.c                           |   8 +-
 block/snapshot.c                              |  60 +-
 block/throttle.c                              |   8 +-
 block/vdi.c                                   |   7 +-
 block/vhdx.c                                  |   7 +-
 block/vmdk.c                                  |   7 +-
 block/vpc.c                                   |   7 +-
 blockdev.c                                    | 842 +++++++++--------
 include/block/block-common.h                  |  47 +-
 include/block/block-global-state.h            |  24 +-
 include/block/block_int-common.h              |  36 +-
 include/block/block_int-global-state.h        |   3 +-
 include/block/block_int-io.h                  |   1 +
 include/block/export.h                        |   1 +
 include/sysemu/block-backend-global-state.h   |   3 +-
 qapi/block-core.json                          |  73 +-
 qapi/transaction.json                         |  35 +-
 stubs/blk-by-qdev-id.c                        |   9 +
 stubs/blk-exp-find-by-blk.c                   |   9 +
 stubs/meson.build                             |   2 +
 tests/qemu-iotests/iotests.py                 |  23 +
 .../tests/blockdev-add-transaction            |  52 ++
 .../tests/blockdev-add-transaction.out        |   6 +
 tests/qemu-iotests/tests/filter-insertion     | 253 +++++
 tests/qemu-iotests/tests/filter-insertion.out |   5 +
 tests/qemu-iotests/tests/image-fleecing       |  20 +-
 tests/qemu-iotests/tests/image-fleecing.out   |   8 -
 tests/unit/test-bdrv-drain.c                  |  11 +-
 tests/unit/test-bdrv-graph-mod.c              |  94 +-
 52 files changed, 1725 insertions(+), 1002 deletions(-)
 create mode 100644 stubs/blk-by-qdev-id.c
 create mode 100644 stubs/blk-exp-find-by-blk.c
 create mode 100755 tests/qemu-iotests/tests/blockdev-add-transaction
 create mode 100644 tests/qemu-iotests/tests/blockdev-add-transaction.out
 create mode 100755 tests/qemu-iotests/tests/filter-insertion
 create mode 100644 tests/qemu-iotests/tests/filter-insertion.out
Signature · 1 line
2.35.1