Exposing backing-chain allocation over NBD
v3 cover Exposing backing-chain allocation over NBD
v2 was here:
https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg10926.html
Since then:
rebase to master
patches 1-3/5 of v2 are in pull request
patch 4/5 in v2 (1/6 here) includes improvements from Vladimir patches 3-6/6 here are new (and I hope to have public patches to libnbd's 'nbdinfo --map' to take advantage of it posted soon...)
Based-on: <[email protected]> ([PULL 0/8] NBD patches through 2020-10-08)
Also available at:
https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/nbd-alloc-depth-v3
001/6:[0056] [FC] 'nbd: Add new qemu:allocation-depth metacontext' 002/6:[0002] [FC] 'nbd: Add 'qemu-nbd -A' to expose allocation depth' 003/6:[down] 'nbd: Update qapi to support multiple bitmaps' 004/6:[down] 'nbd: Simplify qemu bitmap context name' 005/6:[down] 'nbd: Refactor counting of meta contexts' 006/6:[down] 'nbd: Allow export of multiple bitmaps for one device'
Eric Blake (6):
nbd: Add new qemu:allocation-depth metacontext
nbd: Add 'qemu-nbd -A' to expose allocation depth
nbd: Update qapi to support multiple bitmaps
nbd: Simplify qemu bitmap context name
nbd: Refactor counting of meta contexts
nbd: Allow export of multiple bitmaps for one device
docs/interop/nbd.txt | 23 +++- docs/tools/qemu-nbd.rst | 6 ++ qapi/block-core.json | 7 +- qapi/block-export.json | 20 +++- include/block/nbd.h | 8 +- blockdev-nbd.c | 20 +++- nbd/server.c | 210 +++++++++++++++++++++++++++++-------- qemu-nbd.c | 25 +++-- tests/qemu-iotests/291 | 6 +- tests/qemu-iotests/309 | 73 +++++++++++++ tests/qemu-iotests/309.out | 22 ++++ tests/qemu-iotests/group | 1 + 12 files changed, 350 insertions(+), 71 deletions(-) create mode 100755 tests/qemu-iotests/309 create mode 100644 tests/qemu-iotests/309.out
Signature · 1 line
2.28.0
PATCH v3 1/6 nbd: Add new qemu:allocation-depth metacontext
'qemu-img map' provides a way to determine which extents of an image come from the top layer vs. inherited from a backing chain. This is useful information worth exposing over NBD. There is a proposal to add a QMP command block-dirty-bitmap-populate which can create a dirty bitmap that reflects allocation information, at which point qemu:dirty-bitmap:NAME can expose that information via the creation of a temporary bitmap, but we can shorten the effort by adding a new qemu:allocation-depth context that does the same thing without an intermediate bitmap (this patch does not eliminate the need for that proposal, as it will have other uses as well).
For this patch, I just encoded a tri-state value (unallocated, from this layer, from any of the backing layers); an obvious extension would be to provide the actual depth in bits 31-4 while keeping bits 1-0 as a tri-state (leaving bits 3-2 unused, for ease of reading depth from a hex number). But this extension would require bdrv_is_allocated_above to return a depth number.
Note that this patch does not actually enable any way to request a server to enable this context; that will come in the next patch.
--- docs/interop/nbd.txt | 23 +++++++++-- include/block/nbd.h | 8 +++- nbd/server.c | 92 ++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 111 insertions(+), 12 deletions(-)
docs/interop/nbd.txt +19 -4
@@ -17,9 +17,9 @@ namespace "qemu". == "qemu" namespace == -The "qemu" namespace currently contains only one type of context, -related to exposing the contents of a dirty bitmap alongside the -associated disk contents. That context has the following form: +The "qemu" namespace currently contains two types of context. The +first is related to exposing the contents of a dirty bitmap alongside +the associated disk contents. That context has the following form: qemu:dirty-bitmap:<dirty-bitmap-export-name>
@@ -28,8 +28,22 @@ in reply for NBD_CMD_BLOCK_STATUS: bit 0: NBD_STATE_DIRTY, means that the extent is "dirty" +The second is related to exposing the source of various extents within +the image, with a single context named: + + qemu:allocation-depth + +In the allocation depth context, bits 0 and 1 form a tri-state value: + + bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, the extent is unallocated + bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the + top level of the image + bit 1 set: NBD_STATE_DEPTH_BACKING, the extent is inherited from a + backing layer + For NBD_OPT_LIST_META_CONTEXT the following queries are supported -in addition to "qemu:dirty-bitmap:<dirty-bitmap-export-name>": +in addition to the specific "qemu:allocation-depth" and +"qemu:dirty-bitmap:<dirty-bitmap-export-name>": * "qemu:" - returns list of all available metadata contexts in the namespace.
@@ -55,3 +69,4 @@ the operation of that feature. NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE * 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports, NBD_CMD_FLAG_FAST_ZERO +* 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth"
include/block/nbd.h +7 -1
@@ -1,5 +1,5 @@/* -* Copyright (C) 2016-2019 Red Hat, Inc. +* Copyright (C) 2016-2020 Red Hat, Inc. * Copyright (C) 2005 Anthony Liguori <[email protected]> * * Network Block Device
@@ -259,6 +259,12 @@ enum {/* Extent flags for qemu:dirty-bitmap in NBD_REPLY_TYPE_BLOCK_STATUS */ #define NBD_STATE_DIRTY (1 << 0) +/* Extent flags for qemu:allocation-depth in NBD_REPLY_TYPE_BLOCK_STATUS */ +#define NBD_STATE_DEPTH_UNALLOC (0 << 0) +#define NBD_STATE_DEPTH_LOCAL (1 << 0) +#define NBD_STATE_DEPTH_BACKING (2 << 0) +#define NBD_STATE_DEPTH_MASK (0x3) +static inline bool nbd_reply_type_is_error (int type) {return type & (1 <<15 );
nbd/server.c +85 -7
188 lines not on this page · raw message
Signature · 1 line
2.28.0
PATCH v3 2/6 nbd: Add 'qemu-nbd -A' to expose allocation depth
In reply to Eric BlakeAllow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add.
qemu as client can be hacked into viewing this new context by using the now-misnamed x-dirty-bitmap option when creating an NBD blockdev (even though our x- naming means we could rename it, I did not think it worth breaking back-compat of tools that have been using it while waiting for a better solution). It is worth noting the decoding of how such context information will appear in 'qemu-img map --output=json':
NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":trueNBD_STATE_DEPTH_LOCAL => "zero":false, "data":falseNBD_STATE_DEPTH_BACKING => "zero":true, "data":true
libnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;)
Message-Id: <[email protected]>
--- docs/tools/qemu-nbd.rst | 6 ++++ qapi/block-core.json | 7 ++-- qapi/block-export.json | 6 +++- blockdev-nbd.c | 2 ++ nbd/server.c | 2 ++ qemu-nbd.c | 14 ++++++-- tests/qemu-iotests/309 | 73 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/309.out | 22 ++++++++++++ tests/qemu-iotests/group | 1 + 9 files changed, 127 insertions(+), 6 deletions(-) create mode 100755 tests/qemu-iotests/309 create mode 100644 tests/qemu-iotests/309.out
docs/tools/qemu-nbd.rst +6 -0
@@ -72,6 +72,12 @@ driver options if ``--image-opts`` is specified. Export the disk as read-only. +.. option:: -A, --allocation-depth + + Expose allocation depth information via the + ``qemu:allocation-depth`` context accessible through + NBD_OPT_SET_META_CONTEXT. + .. option:: -B, --bitmap=NAME If *filename* has a qcow2 persistent bitmap *NAME*, expose
qapi/block-core.json +5 -2
@@ -3882,9 +3882,12 @@ # # @tls-creds: TLS credentials ID # -# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of +# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" or +# "qemu:allocation-depth" to query in place of the # traditional "base:allocation" block status (see -# NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0) +# NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and +# yes, naming this option x-context would have made +# more sense) (since 3.0) # # @reconnect-delay: On an unexpected disconnect, the nbd client tries to # connect again until succeeding or encountering a serious
qapi/block-export.json +5 -1
@@ -78,11 +78,15 @@ # NBD client can use NBD_OPT_SET_META_CONTEXT with # "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0) # +# @alloc: Also export the allocation map for @device, so the NBD client +# can use NBD_OPT_SET_META_CONTEXT with "qemu:allocation-depth" +# to inspect allocation details. (since 5.2) +# # Since: 5.0 ## { 'struct': 'BlockExportOptionsNbd', 'data': { '*name': 'str', '*description': 'str', - '*bitmap': 'str' } } + '*bitmap': 'str', '*alloc': 'bool' } } ## # @NbdServerAddOptions:
blockdev-nbd.c +2 -0
@@ -212,6 +212,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp) .description = g_strdup(arg->description), .has_bitmap = arg->has_bitmap, .bitmap = g_strdup(arg->bitmap), + .has_alloc = arg->alloc, + .alloc = arg->alloc, }, };
nbd/server.c +2 -0
@@ -1597,6 +1597,8 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,assert (strlen (exp->export_bitmap_context) <NBD_MAX_STRING_SIZE ); } + exp->alloc_context = arg->alloc; +blk_add_aio_context_notifier (blk, blk_aio_attached, blk_aio_detach, exp);QTAILQ_INSERT_TAIL (&exports, exp, next);
qemu-nbd.c +11 -3
@@ -99,6 +99,7 @@ static void usage(const char *name)"\n" "Exposing part of the image:\n" " -o, --offset=OFFSET offset into the image\n" +" -A, --allocation-depth expose the allocation depth\n" " -B, --bitmap=NAME expose a persistent dirty bitmap\n" "\n" "General purpose options:\n"
@@ -519,7 +520,7 @@ int main(int argc, char **argv)char *device =NULL ;QemuOpts *sn_opts =NULL ;const char *sn_id_or_name =NULL ; -const char *sopt ="hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L" ; +const char *sopt ="hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:AB:L" ;struct option lopt[] = { {"help" , no_argument,NULL ,'h' }, {"version" , no_argument,NULL ,'V' },
@@ -528,6 +529,7 @@ int main(int argc, char **argv) { "socket", required_argument, NULL, 'k' }, { "offset", required_argument, NULL, 'o' }, { "read-only", no_argument, NULL, 'r' }, + { "allocation-depth", no_argument, NULL, 'A' }, { "bitmap", required_argument, NULL, 'B' }, { "connect", required_argument, NULL, 'c' }, { "disconnect", no_argument, NULL, 'd' },
@@ -569,6 +571,7 @@ int main(int argc, char **argv)QDict *options =NULL ;const char *export_name =NULL ;/* defaults to "" later for server mode */ const char *export_description =NULL ; +bool alloc_depth = false;const char *bitmap =NULL ;const char *tlscredsid =NULL ;bool imageOpts = false;
@@ -694,6 +697,9 @@ int main(int argc, char **argv) readonly = true; flags &= ~BDRV_O_RDWR ;break ; +case 'A' : + alloc_depth = true; +break ;case 'B' : bitmap = optarg;break ;
@@ -791,8 +797,8 @@ int main(int argc, char **argv)exit (EXIT_FAILURE ); }if (export_name || export_description || dev_offset || - device || disconnect || fmt || sn_id_or_name || bitmap || - seen_aio || seen_discard || seen_cache) { + device || disconnect || fmt || sn_id_or_name || alloc_depth || + bitmap || seen_aio || seen_discard || seen_cache) {error_report ("List mode is incompatible with per-device settings" );exit (EXIT_FAILURE ); }
@@ -1078,6 +1084,8 @@ int main(int argc, char **argv) .description = g_strdup(export_description), .has_bitmap = !!bitmap, .bitmap = g_strdup(bitmap), + .has_alloc = alloc_depth, + .alloc = alloc_depth, }, };blk_exp_add (export_opts, &error_fatal);
tests/qemu-iotests/309 new file +73 -0
73 lines not on this page · raw message
tests/qemu-iotests/309.out new file +22 -0
@@ -0,0 +1,22 @@ +QA output created by 309 + +=== Initial image setup === + +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=4194304 +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT +wrote 2097152/2097152 bytes at offset 1048576 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Check allocation over NBD === + +[{ "start": 0, "length": 1048576, "depth": 1, "zero": false, "data": true, "offset": 327680}, +{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": 327680}, +{ "start": 3145728, "length": 1048576, "depth": 1, "zero": true, "data": false}] +[{ "start": 0, "length": 3145728, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, +{ "start": 3145728, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": true, "offset": OFFSET}, +{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": false}, +{ "start": 3145728, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": OFFSET}] +*** done
tests/qemu-iotests/group +1 -0
@@ -315,3 +315,4 @@ 304 rw quick 305 rw quick 307 rw quick export +309 rw auto quick
Signature · 1 line
2.28.0
Re: [PATCH v3 2/6] nbd: Add 'qemu-nbd -A' to expose allocation depth
Quoted 26 lines · Eric Blake
Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through
NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add.qemu as client can be hacked into viewing this new context by using the now-misnamed x-dirty-bitmap option when creating an NBD blockdev (even though our x- naming means we could rename it, I did not think it worth breaking back-compat of tools that have been using it while waiting for a better solution). It is worth noting the decoding of how such context information will appear in 'qemu-img map --output=json':
NBD_STATE_DEPTH_UNALLOC=> "zero":false, "data":trueNBD_STATE_DEPTH_LOCAL=> "zero":false, "data":falseNBD_STATE_DEPTH_BACKING=> "zero":true, "data":truelibnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;)
Signed-off-by Eric Blake <[email protected]>Message-Id: <[email protected]>
Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>[eblake: comment tweak suggested by Vladimir]Signed-off-by Eric Blake <[email protected]>---
[...]
qapi/block-core.json line 3885
Quoted 10 lines
qapi/block-core.json +1
-1@@ -3882,9 +3882,12 @@ # # @tls-creds: TLS credentials ID # -# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of +# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" or
What is a "metacontext"?
Aside: nbd.h spells it "meta context". The two spellings suggest different meanings to me. Using meta- as a prefix modifies the prefixed word, as in metadata (data that provides information about other data). Separated by space, meta becomes a noun. None of its common meanings seem to apply here. Recommend to pick an appropriate one of "metacontext", "meta-context", "meta context", and stick to it.
Quoted 30 lines
qapi/block-core.json +4
-1+# "qemu:allocation-depth" to query in place of the # traditional "base:allocation" block status (see -# NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0) +# NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and +# yes, naming this option x-context would have made +# more sense) (since 3.0) # # @reconnect-delay: On an unexpected disconnect, the nbd client tries to # connect again until succeeding or encountering a seriousqapi/block-export.json +5
-1@@ -78,11 +78,15 @@ # NBD client can use NBD_OPT_SET_META_CONTEXT with # "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0) # +# @alloc: Also export the allocation map for @device, so the NBD client +# can use NBD_OPT_SET_META_CONTEXT with "qemu:allocation-depth" +# to inspect allocation details. (since 5.2) +# # Since: 5.0 ## { 'struct': 'BlockExportOptionsNbd', 'data': { '*name': 'str', '*description': 'str', - '*bitmap': 'str' } } + '*bitmap': 'str', '*alloc': 'bool' } } ## # @NbdServerAddOptions:
[...]
Re: [PATCH v3 2/6] nbd: Add 'qemu-nbd -A' to expose allocation depth
Quoted 42 lines · Markus Armbruster
Quoted 26 lines · Eric Blake
Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through
NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add.qemu as client can be hacked into viewing this new context by using the now-misnamed x-dirty-bitmap option when creating an NBD blockdev (even though our x- naming means we could rename it, I did not think it worth breaking back-compat of tools that have been using it while waiting for a better solution). It is worth noting the decoding of how such context information will appear in 'qemu-img map --output=json':
NBD_STATE_DEPTH_UNALLOC=> "zero":false, "data":trueNBD_STATE_DEPTH_LOCAL=> "zero":false, "data":falseNBD_STATE_DEPTH_BACKING=> "zero":true, "data":truelibnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;)
Signed-off-by Eric Blake <[email protected]>Message-Id: <[email protected]>
Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>[eblake: comment tweak suggested by Vladimir]Signed-off-by Eric Blake <[email protected]>---
[...]
qapi/block-core.json line 3885
Quoted 10 lines
qapi/block-core.json +1
-1@@ -3882,9 +3882,12 @@ # # @tls-creds: TLS credentials ID # -# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of +# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" orWhat is a "metacontext"?
Quoted 6 lines
Aside:
nbd.hspells it "meta context". The two spellings suggest different meanings to me. Using meta- as a prefix modifies the prefixed word, as in metadata (data that provides information about other data). Separated by space, meta becomes a noun. None of its common meanings seem to apply here. Recommend to pick an appropriate one of "metacontext", "meta-context", "meta context", and stick to it.
In the NBD protocol it's called a "metadata context". This is abbreviated to "...META_CONTEXT" in the names of constants, eg. NBD_OPT_SET_META_CONTEXT. For the reasons you gave I don't think it's a very good name, but it's what we have.
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md#metadata-querying
What it actually is, is a string, like "base:allocation" or "qemu:dirty-bitmap:<NAME>" which selects which metadata you want to query in NBD_CMD_BLOCK_STATUS commands.
Rich.
Signature · 5 lines
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/
Re: [PATCH v3 2/6] nbd: Add 'qemu-nbd -A' to expose allocation depth
Quoted 22 lines · Richard W.M. Jones
Quoted 10 lines
qapi/block-core.json line 3885
Quoted 7 lines
+++
b/qapi/block-core.jsonqapi/block-core.json +1
-1@@ -3882,9 +3882,12 @@ # # @tls-creds: TLS credentials ID # -# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of +# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" orWhat is a "metacontext"?
Quoted 6 lines
Aside:
nbd.hspells it "meta context". The two spellings suggest different meanings to me. Using meta- as a prefix modifies the prefixed word, as in metadata (data that provides information about other data). Separated by space, meta becomes a noun. None of its common meanings seem to apply here. Recommend to pick an appropriate one of "metacontext", "meta-context", "meta context", and stick to it.In the NBD protocol it's called a "metadata context". This is abbreviated to "...META_CONTEXT" in the names of constants, eg.
NBD_OPT_SET_META_CONTEXT. For the reasons you gave I don't think it's a very good name, but it's what we have.
Quoted 5 lines
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md#metadata-querying
What it actually is, is a string, like "base:allocation" or "qemu:dirty-bitmap:<NAME>" which selects which metadata you want to query in
NBD_CMD_BLOCK_STATUScommands.
Sounds like documenting it as a "metadata context name" is probably the one to use then. I'll make that adjustment.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 3/6 nbd: Update qapi to support multiple bitmaps
In reply to Eric BlakeSince 'nbd-server-add' is deprecated, and 'block-export-add' is new to 5.2, we can still tweak the interface. Allowing 'bitmaps':['str'] is nicer than 'bitmap':'str'. This wires up the qapi and qemu-nbd changes to permit passing multiple bitmaps, but the actual support will require a further patch to the server.
--- qapi/block-export.json | 16 +++++++++++----- blockdev-nbd.c | 18 ++++++++++++++++-- nbd/server.c | 19 +++++++++++++------ qemu-nbd.c | 13 ++++++++----- 4 files changed, 48 insertions(+), 18 deletions(-)
qapi/block-export.json +11 -5
@@ -74,9 +74,9 @@ # @description: Free-form description of the export, up to 4096 bytes. # (Since 5.0) # -# @bitmap: Also export the dirty bitmap reachable from @device, so the -# NBD client can use NBD_OPT_SET_META_CONTEXT with -# "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0) +# @bitmaps: Also export each of the named dirty bitmaps reachable from +# @device, so the NBD client can use NBD_OPT_SET_META_CONTEXT with +# "qemu:dirty-bitmap:BITMAP" to inspect each bitmap. (since 5.2) # # @alloc: Also export the allocation map for @device, so the NBD client # can use NBD_OPT_SET_META_CONTEXT with "qemu:allocation-depth"
@@ -86,7 +86,7 @@ ## { 'struct': 'BlockExportOptionsNbd', 'data': { '*name': 'str', '*description': 'str', - '*bitmap': 'str', '*alloc': 'bool' } } + '*bitmaps': ['str'], '*alloc': 'bool' } } ## # @NbdServerAddOptions:
@@ -98,12 +98,18 @@ # @writable: Whether clients should be able to write to the device via the # NBD connection (default false). # +# @bitmap: Also export a single dirty bitmap reachable from @device, so the +# NBD client can use NBD_OPT_SET_META_CONTEXT with +# "qemu:dirty-bitmap:BITMAP" to inspect the bitmap (since 4.0); +# mutually exclusive with @bitmaps, and newer clients should use +# that instead. +# # Since: 5.0 ## { 'struct': 'NbdServerAddOptions', 'base': 'BlockExportOptionsNbd', 'data': { 'device': 'str', - '*writable': 'bool' } } + '*writable': 'bool', '*bitmap': 'str' } } ## # @nbd-server-add:
blockdev-nbd.c +16 -2
@@ -190,6 +190,20 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp)return ; } +/* +* New code should use the list 'bitmaps'; but until this code is +* gone, we must support the older single 'bitmap'. Use only one. +*/ +if (arg->has_bitmap) { +if (arg->has_bitmaps) { +error_setg (errp,"Can't mix 'bitmap' and 'bitmaps'" ); +return ; + } + arg->has_bitmaps = true; + arg->bitmaps =g_new0 (strList,1 ); + arg->bitmaps->value =g_strdup (arg->bitmap); + } +/* * block-export-add would default to the node-name, but we may have to use * the device name as a default here for compatibility.
@@ -210,8 +224,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error **errp) .name = g_strdup(arg->name), .has_description = arg->has_description, .description = g_strdup(arg->description), - .has_bitmap = arg->has_bitmap, - .bitmap = g_strdup(arg->bitmap), + .has_bitmaps = arg->has_bitmaps, + .bitmaps = g_steal_pointer(&arg->bitmaps), .has_alloc = arg->alloc, .alloc = arg->alloc, },
nbd/server.c +13 -6
@@ -1495,6 +1495,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,uint64_t perm, shared_perm;bool readonly = !exp_args->writable;bool shared = !exp_args->writable; +strList *bitmaps;int ret;assert (exp_args->type ==BLOCK_EXPORT_TYPE_NBD );
@@ -1556,12 +1557,18 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args, } exp->size =QEMU_ALIGN_DOWN (size,BDRV_SECTOR_SIZE ); -if (arg->bitmap) { +/* XXX Allow more than one bitmap */ +if (arg->bitmaps && arg->bitmaps->next) { +error_setg (errp,"multiple bitmaps per export not supported yet" ); +return -EOPNOTSUPP ; + } +for (bitmaps = arg->bitmaps; bitmaps; bitmaps = bitmaps->next) { +const char *bitmap = bitmaps->value;BlockDriverState *bs =blk_bs (blk);BdrvDirtyBitmap *bm =NULL ;while (bs) { - bm =bdrv_find_dirty_bitmap (bs, arg->bitmap); + bm =bdrv_find_dirty_bitmap (bs, bitmap);if (bm !=NULL ) {break ; }
@@ -1571,7 +1578,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,if (bm ==NULL ) { ret = -ENOENT ; -error_setg (errp,"Bitmap '%s' is not found" , arg->bitmap); +error_setg (errp,"Bitmap '%s' is not found" , bitmap); goto fail; }
@@ -1585,15 +1592,15 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args, ret = -EINVAL ;error_setg (errp,"Enabled bitmap '%s' incompatible with readonly export" , - arg->bitmap); + bitmap); goto fail; }bdrv_dirty_bitmap_set_busy (bm, true); exp->export_bitmap = bm; -assert (strlen (arg->bitmap) <=BDRV_BITMAP_MAX_NAME_SIZE ); +assert (strlen (bitmap) <=BDRV_BITMAP_MAX_NAME_SIZE ); exp->export_bitmap_context =g_strdup_printf ("qemu:dirty-bitmap:%s" , - arg->bitmap); + bitmap);assert (strlen (exp->export_bitmap_context) <NBD_MAX_STRING_SIZE ); }
qemu-nbd.c +8 -5
@@ -572,7 +572,7 @@ int main(int argc, char **argv)const char *export_name =NULL ;/* defaults to "" later for server mode */ const char *export_description =NULL ;bool alloc_depth = false; -const char *bitmap =NULL ; +strList *bitmaps =NULL , *tmp;const char *tlscredsid =NULL ;bool imageOpts = false;bool writethrough = true;
@@ -701,7 +701,10 @@ int main(int argc, char **argv) alloc_depth = true;break ;case 'B' : - bitmap = optarg; + tmp =g_new (strList,1 ); + tmp->value =g_strdup (optarg); + tmp->next = bitmaps; + bitmaps = tmp;break ;case 'k' : sockpath = optarg;
@@ -798,7 +801,7 @@ int main(int argc, char **argv) }if (export_name || export_description || dev_offset || device || disconnect || fmt || sn_id_or_name || alloc_depth || - bitmap || seen_aio || seen_discard || seen_cache) { + bitmaps || seen_aio || seen_discard || seen_cache) {error_report ("List mode is incompatible with per-device settings" );exit (EXIT_FAILURE ); }
@@ -1082,8 +1085,8 @@ int main(int argc, char **argv) .name = g_strdup(export_name), .has_description = !!export_description, .description = g_strdup(export_description), - .has_bitmap = !!bitmap, - .bitmap = g_strdup(bitmap), + .has_bitmaps = !!bitmaps, + .bitmaps = bitmaps, .has_alloc = alloc_depth, .alloc = alloc_depth, },
Signature · 1 line
2.28.0
PATCH v3 4/6 nbd: Simplify qemu bitmap context name
In reply to Eric BlakeEach dirty bitmap already knows its name; by reducing the scope of the places where we construct "qemu:dirty-bitmap:NAME" strings, tracking the name is more localized, and there are fewer per-export fields to worry about. This in turn will make it easier for an upcoming patch to export more than one bitmap at once.
--- nbd/server.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-)
nbd/server.c +9 -10
@@ -97,7 +97,6 @@ struct NBDExport {bool alloc_context;BdrvDirtyBitmap *export_bitmap; -char *export_bitmap_context; }; static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
@@ -882,14 +881,15 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, }if (nbd_strshift (&query,"dirty-bitmap:" )) { +const char *bm_name; +trace_nbd_negotiate_meta_query_parse ("dirty-bitmap:" );if (!meta->exp->export_bitmap) {trace_nbd_negotiate_meta_query_skip ("no dirty-bitmap exported" );return true; } -if (nbd_meta_empty_or_pattern (client, - meta->exp->export_bitmap_context + -strlen ("qemu:dirty-bitmap:" ), query)) { + bm_name =bdrv_dirty_bitmap_name (meta->exp->export_bitmap); +if (nbd_meta_empty_or_pattern (client, bm_name, query)) { meta->bitmap = true; }return true;
@@ -1025,8 +1025,11 @@ static int nbd_negotiate_meta_queries(NBDClient *client, }if (meta->bitmap) { - ret =nbd_negotiate_send_meta_context (client, - meta->exp->export_bitmap_context, +const char *bm_name =bdrv_dirty_bitmap_name (meta->exp->export_bitmap); + g_autofree char *context = g_strdup_printf("qemu:dirty-bitmap:%s", + bm_name); + + ret =nbd_negotiate_send_meta_context (client, context,NBD_META_ID_DIRTY_BITMAP , errp);if (ret <0 ) {
@@ -1599,9 +1602,6 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args,bdrv_dirty_bitmap_set_busy (bm, true); exp->export_bitmap = bm;assert (strlen (bitmap) <=BDRV_BITMAP_MAX_NAME_SIZE ); - exp->export_bitmap_context =g_strdup_printf ("qemu:dirty-bitmap:%s" , - bitmap); -assert (strlen (exp->export_bitmap_context) <NBD_MAX_STRING_SIZE ); } exp->alloc_context = arg->alloc;
@@ -1681,7 +1681,6 @@ static void nbd_export_delete(BlockExport *blk_exp)if (exp->export_bitmap) {bdrv_dirty_bitmap_set_busy (exp->export_bitmap, false); -g_free (exp->export_bitmap_context); } }
Signature · 1 line
2.28.0
PATCH v3 5/6 nbd: Refactor counting of meta contexts
In reply to Eric BlakeRather than open-code the count of negotiated contexts at several sites, embed it directly into the struct.
--- nbd/server.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)
nbd/server.c +11 -12
@@ -106,8 +106,7 @@ static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);* NBD_OPT_LIST_META_CONTEXT. */ typedef struct NBDExportMetaContexts { NBDExport *exp; - bool valid; /* means that negotiation of the option finished without - errors */ + size_t count; /* number of negotiated contexts */ bool base_allocation; /* export base:allocation context (block status) */ bool allocation_depth; /* export qemu:allocation-depth */ bool bitmap; /* export qemu:dirty-bitmap:<export bitmap name> */
@@ -448,7 +447,9 @@ static int nbd_negotiate_handle_list(NBDClient *client, Error **errp)static void nbd_check_meta_export (NBDClient *client) { - client->export_meta.valid &= client->exp == client->export_meta.exp; +if (client->exp != client->export_meta.exp) { + client->export_meta.count =0 ; + } }/* Send a reply to NBD_OPT_EXPORT_NAME.
@@ -956,6 +957,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,NBDExportMetaContexts local_meta;uint32_t nb_queries;int i; +size_t count =0 ;if (!client->structured_reply) { return nbd_opt_invalid(client, errp,
@@ -1013,6 +1015,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,if (ret <0 ) {return ret; } + count++; }if (meta->allocation_depth) {
@@ -1022,6 +1025,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,if (ret <0 ) {return ret; } + count++; }if (meta->bitmap) {
@@ -1035,11 +1039,12 @@ static int nbd_negotiate_meta_queries(NBDClient *client,if (ret <0 ) {return ret; } + count++; } ret =nbd_negotiate_send_rep (client,NBD_REP_ACK , errp);if (ret ==0 ) { - meta->valid = true; + meta->count = count; }return ret;
@@ -2400,15 +2405,9 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,return nbd_send_generic_reply (client, request->handle, -EINVAL ,"need non-zero length" , errp); } -if (client->export_meta.valid && - (client->export_meta.base_allocation || - client->export_meta.allocation_depth || - client->export_meta.bitmap)) - { +if (client->export_meta.count) {bool dont_fragment = request->flags &NBD_CMD_FLAG_REQ_ONE ; -int contexts_remaining = client->export_meta.base_allocation + - client->export_meta.allocation_depth + - client->export_meta.bitmap; +int contexts_remaining = client->export_meta.count;if (client->export_meta.base_allocation) { ret = nbd_co_send_block_status(client, request->handle,
Signature · 1 line
2.28.0
PATCH v3 6/6 nbd: Allow export of multiple bitmaps for one device
In reply to Eric BlakeWith this, 'qemu-nbd -B b0 -B b1 -f qcow2 img.qcow2' can let you sniff out multiple bitmaps from one server. qemu-img as client can still only read one bitmap per client connection, but other NBD clients (hello libnbd) can now read multiple bitmaps in a single pass.
--- nbd/server.c | 89 +++++++++++++++++++++++++++++------------- tests/qemu-iotests/291 | 6 +-- 2 files changed, 63 insertions(+), 32 deletions(-)
nbd/server.c +61 -28
@@ -96,7 +96,8 @@ struct NBDExport {Notifier eject_notifier;bool alloc_context; -BdrvDirtyBitmap *export_bitmap; +BdrvDirtyBitmap **export_bitmaps; +size_t nr_export_bitmaps; }; static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
@@ -109,7 +110,10 @@ typedef struct NBDExportMetaContexts {size_t count;/* number of negotiated contexts */ bool base_allocation;/* export base:allocation context (block status) */ bool allocation_depth;/* export qemu:allocation-depth */ -bool bitmap;/* export qemu:dirty-bitmap:<export bitmap name> */ +bool *bitmaps;/* +* export qemu:dirty-bitmap:<export bitmap name>, +* sized by exp->nr_export_bitmaps +*/ } NBDExportMetaContexts;struct NBDClient {
@@ -861,6 +865,8 @@ static bool nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta,static bool nbd_meta_qemu_query (NBDClient *client,NBDExportMetaContexts *meta,const char *query) { +size_t i; +if (!nbd_strshift (&query,"qemu:" )) {return false; }
@@ -869,7 +875,7 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,if (!*query) {if (client->opt ==NBD_OPT_LIST_META_CONTEXT ) { meta->allocation_depth = meta->exp->alloc_context; - meta->bitmap = !!meta->exp->export_bitmap; +memset (meta->bitmaps,1 , meta->exp->nr_export_bitmaps); }trace_nbd_negotiate_meta_query_parse ("empty" );return true;
@@ -882,17 +888,26 @@ static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, }if (nbd_strshift (&query,"dirty-bitmap:" )) { -const char *bm_name; -trace_nbd_negotiate_meta_query_parse ("dirty-bitmap:" ); -if (!meta->exp->export_bitmap) { -trace_nbd_negotiate_meta_query_skip ("no dirty-bitmap exported" ); +if (!*query) { +if (client->opt ==NBD_OPT_LIST_META_CONTEXT ) { +memset (meta->bitmaps,1 , meta->exp->nr_export_bitmaps); + } +trace_nbd_negotiate_meta_query_parse ("empty" );return true; } - bm_name =bdrv_dirty_bitmap_name (meta->exp->export_bitmap); -if (nbd_meta_empty_or_pattern (client, bm_name, query)) { - meta->bitmap = true; + +for (i =0 ; i < meta->exp->nr_export_bitmaps; i++) { +const char *bm_name; + + bm_name =bdrv_dirty_bitmap_name (meta->exp->export_bitmaps[i]); +if (strcmp (bm_name, query) ==0 ) { + meta->bitmaps[i] = true; +trace_nbd_negotiate_meta_query_parse (query); +return true; + } } +trace_nbd_negotiate_meta_query_skip ("no dirty-bitmap match" );return true; }
@@ -954,9 +969,10 @@ static int nbd_negotiate_meta_queries(NBDClient *client, {int ret; g_autofree char *export_name = NULL; -NBDExportMetaContexts local_meta; + g_autofree bool *bitmaps = NULL; +NBDExportMetaContexts local_meta = {0 };uint32_t nb_queries; -int i; +size_t i;size_t count =0 ;if (!client->structured_reply) {
@@ -971,6 +987,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client, meta = &local_meta; } +g_free (meta->bitmaps);memset (meta,0 ,sizeof (*meta)); ret =nbd_opt_read_name (client, &export_name,NULL , errp);
@@ -985,6 +1002,10 @@ static int nbd_negotiate_meta_queries(NBDClient *client,return nbd_opt_drop (client,NBD_REP_ERR_UNKNOWN , errp,"export '%s' not present" , sane_name); } + meta->bitmaps =g_new0 (bool, meta->exp->nr_export_bitmaps); +if (client->opt ==NBD_OPT_LIST_META_CONTEXT ) { + bitmaps = meta->bitmaps; + } ret =nbd_opt_read (client, &nb_queries,sizeof (nb_queries), false, errp);if (ret <=0 ) {
@@ -998,7 +1019,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,/* enable all known contexts */ meta->base_allocation = true; meta->allocation_depth = meta->exp->alloc_context; - meta->bitmap = !!meta->exp->export_bitmap; +memset (meta->bitmaps,1 , meta->exp->nr_export_bitmaps); }else {for (i =0 ; i < nb_queries; ++i) { ret =nbd_negotiate_meta_query (client, meta, errp);
@@ -1028,13 +1049,19 @@ static int nbd_negotiate_meta_queries(NBDClient *client, count++; } -if (meta->bitmap) { -const char *bm_name =bdrv_dirty_bitmap_name (meta->exp->export_bitmap); - g_autofree char *context = g_strdup_printf("qemu:dirty-bitmap:%s", - bm_name); +for (i =0 ; i < meta->exp->nr_export_bitmaps; i++) { +const char *bm_name; + g_autofree char *context = NULL; + +if (!meta->bitmaps[i]) { +continue ; + } + + bm_name =bdrv_dirty_bitmap_name (meta->exp->export_bitmaps[i]); + context =g_strdup_printf ("qemu:dirty-bitmap:%s" , bm_name); ret =nbd_negotiate_send_meta_context (client, context, -NBD_META_ID_DIRTY_BITMAP , +NBD_META_ID_DIRTY_BITMAP + i, errp);if (ret <0 ) {return ret;
@@ -1388,6 +1415,7 @@ void nbd_client_put(NBDClient *client)QTAILQ_REMOVE (&client->exp->clients, client, next);blk_exp_unref (&client->exp->common); } +g_free (client->export_meta.bitmaps);g_free (client); } }
@@ -1565,11 +1593,11 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args, } exp->size =QEMU_ALIGN_DOWN (size,BDRV_SECTOR_SIZE ); -/* XXX Allow more than one bitmap */ -if (arg->bitmaps && arg->bitmaps->next) { -error_setg (errp,"multiple bitmaps per export not supported yet" ); -return -EOPNOTSUPP ; +for (bitmaps = arg->bitmaps; bitmaps; bitmaps = bitmaps->next) { + exp->nr_export_bitmaps++; } + exp->export_bitmaps = g_new0(BdrvDirtyBitmap *, exp->nr_export_bitmaps); + exp->nr_export_bitmaps =0 ;for (bitmaps = arg->bitmaps; bitmaps; bitmaps = bitmaps->next) {const char *bitmap = bitmaps->value;BlockDriverState *bs =blk_bs (blk);
@@ -1605,7 +1633,7 @@ static int nbd_export_create(BlockExport *blk_exp, BlockExportOptions *exp_args, }bdrv_dirty_bitmap_set_busy (bm, true); - exp->export_bitmap = bm; + exp->export_bitmaps[exp->nr_export_bitmaps++] = bm;assert (strlen (bitmap) <=BDRV_BITMAP_MAX_NAME_SIZE ); }
@@ -1667,6 +1695,7 @@ static void nbd_export_request_shutdown(BlockExport *blk_exp)static void nbd_export_delete (BlockExport *blk_exp) { +size_t i;NBDExport *exp =container_of (blk_exp, NBDExport, common);assert (exp->name ==NULL );
@@ -1684,8 +1713,8 @@ static void nbd_export_delete(BlockExport *blk_exp) blk_aio_detach, exp); } -if (exp->export_bitmap) { -bdrv_dirty_bitmap_set_busy (exp->export_bitmap, false); +for (i =0 ; i < exp->nr_export_bitmaps; i++) { +bdrv_dirty_bitmap_set_busy (exp->export_bitmaps[i], false); } }
@@ -2332,6 +2361,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client,int flags;NBDExport *exp = client->exp;char *msg; +size_t i;switch (request->type) {case NBD_CMD_CACHE :
@@ -2435,12 +2465,15 @@ static coroutine_fn int nbd_handle_request(NBDClient *client, } } -if (client->export_meta.bitmap) { +for (i =0 ; i < client->exp->nr_export_bitmaps; i++) { +if (!client->export_meta.bitmaps[i]) { +continue ; + } ret =nbd_co_send_bitmap (client, request->handle, - client->exp->export_bitmap, + client->exp->export_bitmaps[i], request->from, request->len, dont_fragment, !--contexts_remaining, -NBD_META_ID_DIRTY_BITMAP , errp); +NBD_META_ID_DIRTY_BITMAP + i, errp);if (ret <0 ) {return ret; }
tests/qemu-iotests/291 +2 -4
@@ -107,16 +107,14 @@ echo # x-dirty-bitmap is a hack for reading bitmaps; it abuses block status to # report "data":false for portions of the bitmap which are set IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket" -nbd_server_start_unix_socket -r -f qcow2 -B b0 "$TEST_IMG" +nbd_server_start_unix_socket -r -f qcow2 \ + -B b0 -B b1 -B b2 -B b3 "$TEST_IMG" $QEMU_IMG map --output=json --image-opts \ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b0" | _filter_qemu_img_map -nbd_server_start_unix_socket -r -f qcow2 -B b1 "$TEST_IMG" $QEMU_IMG map --output=json --image-opts \ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b1" | _filter_qemu_img_map -nbd_server_start_unix_socket -r -f qcow2 -B b2 "$TEST_IMG" $QEMU_IMG map --output=json --image-opts \ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b2" | _filter_qemu_img_map -nbd_server_start_unix_socket -r -f qcow2 -B b3 "$TEST_IMG" $QEMU_IMG map --output=json --image-opts \ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b3" | _filter_qemu_img_map
Signature · 1 line
2.28.0