qemu-img info: Show protocol-level information
v2 cover qemu-img info: Show protocol-level information
Hi,
This series is a v2 to:
https://lists.nongnu.org/archive/html/qemu-block/2022-05/msg00042.html
Like v1, the purpose is to have qemu-img info print the extent-size-hint for images on filesystems that support it. In contrast to v1, it does so in a more complicated way.
v1 printed this information by adding protocol-specific information to ImageInfo, which Kevin disliked and instead proposed to have ImageInfo point to full ImageInfo objects for the respective node’s children.
I considered two possible solutions:
(A) Ignore the proposal, but not add protocol-specific information to ImageInfo. Instead, have img_info() itself try to find a clearly identifiable protocol node and print its driver-specific information alongside the rest of the information.
I discarded that idea because --output=json is supposed to produce a QAPI type, and so this additional information wouldn’t be there. It wouldn’t be great to print more information with --output=human than with --output=json.
(B) Somehow let qemu-img info print the block graph.
This implements (B). I decided against simply putting recursive ImageInfo objects into ImageInfo, for three reasons:
1. Extremely personal and unsubstantiated opinion: I don’t like it for block query functions to return a whole graph. I prefer query functions to work on single nodes and users to manually walk the graph if they need information about multiple nodes.
2. ImageInfo already has a link to the backing node’s ImageInfo. It would be strange to link the backing node’s ImageInfo twice (once in backing-image, once in the list of all child nodes).
3. query-named-block-nodes returns a list of BlockDeviceInfo objects, and every such object has an ImageInfo field. I think it would be a mistake for these ImageInfo fields to be full block subgraphs. Now, query-named-block-nodes already has a @flat parameter that can be used to suppress the backing-image information that is in ImageInfo. Still, it would be a bad idea to surprise users that don’t set it to true (it defaults to false) by blowing up the data they get back. We could add another parameter @nested that needs to be explicitly set to true or users will not get the child information; but having both @flat and @nested is kind of a bad interface.
So I decided to split a new structure BlockNodeInfo off of ImageInfo, where BlockNodeInfo contains everything that ImageInfo did except for the backing-image link. We can then create another structure BlockGraphInfo that builds on BlockNodeInfo, and in contrast to ImageInfo has link to all children instead of just backing-image. We then let qemu-img info output BlockGraphInfo objects, which works well because qemu-img info has always ensured the backing-image field would not be used (so the ImageInfo objects it emitted effectively always were what are now BlockNodeInfo objects).
(I hope this reorganization isn’t an incompatible change, but I’m not sure, I have to admit...)
This gets around having to deal with QMP changes relating to ImageInfo or BlockDeviceInfo (e.g. with query-named-block-nodes), and we don’t have to worry about the fact that the backing node’s ImageInfo were duplicated.
There is another potential duplication problem, though: VMDK’s format-specific info (ImageInfoSpecificVmdk) contains an array of ImageInfo objects for its extent files. Just like with backing-image, it seems not great to duplicate these links.
On closer inspection, however, it turns out that these objects aren’t actually the extent nodes’ ImageInfo data at all. These objects are filled by the VMDK driver with custom information that actually does not fit the ImageInfo structure, for example, the @format field is not a block driver type, but a VMDK format, like "SPARSE" or "FLAT". Therefore, the child links in the new BlockGraphInfo object actually would not link to duplicate information.
I decided to make that explicit by changing the extent information type from ImageInfo to a new VmdkExtentInfo type. I don’t know whether that is technically an incompatible change, and I don’t know whether it even matters. We can skip that type change, and this series should still work, but I feel like it would have been better for these objects to have had their own type from the start.
So the final state is that despite the QAPI changes, hopefully only the qemu-img info output changes, and it now prints every image node’s subgraph. This results in an output like the following (for a qcow2 image):
image: test.qcow2
file format: qcow2
virtual size: 64 MiB (67108864 bytes)
disk size: 196 KiBcluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
Child node '/file':
filename: test.qcow2
protocol type: file
file length: 192 KiB (197120 bytes)
disk size: 196 KiB
Format specific information:
extent size hint: 1048576
For reference, the output from v1 was this (with “extent size” corrected to “extent size hint”):
image: test.qcow2
file format: qcow2
virtual size: 64 MiB (67108864 bytes)
disk size: 196 KiBcluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
Protocol specific information:
extent size hint: 1048576
I think I still slightly prefer the output from v1, because the additional information is mostly just duplicated information (and thus clutters the output), but I can see that hard-adding protocol-specific information to ImageInfo wasn’t a good way to go about it (and I can’t find any better reasonable way to only print that driver-specific information (and nothing else) from the protocol node).
For completeness’s sake, git-backport-diff to v1:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/12:[----] [--] 'block: Improve empty format-specific info dump' 002/12:[0008] [FC] 'block/file: Add file-specific image info' 003/12:[down] 'block/vmdk: Change extent info type' 004/12:[down] 'block: Split BlockNodeInfo off of ImageInfo' 005/12:[down] 'qemu-img: Use BlockNodeInfo'
006/12:[down] 'block/qapi: Let bdrv_query_image_info() recurse' 007/12:[down] 'block/qapi: Introduce BlockGraphInfo' 008/12:[down] 'block/qapi: Add indentation to bdrv_node_info_dump()' 009/12:[down] 'iotests: Filter child node information' 010/12:[down] 'iotests/106, 214, 308: Read only one size line' 011/12:[down] 'qemu-img: Let info print block graph' 012/12:[down] 'qemu-img: Change info key names for protocol nodes'
Hanna Reitz (12):
block: Improve empty format-specific info dump block/file: Add file-specific image info block/vmdk: Change extent info type
block: Split BlockNodeInfo off of ImageInfo
qemu-img: Use BlockNodeInfo
block/qapi: Let bdrv_query_image_info() recurse block/qapi: Introduce BlockGraphInfo block/qapi: Add indentation to bdrv_node_info_dump()
iotests: Filter child node information iotests/106, 214, 308: Read only one size line
qemu-img: Let info print block graph
qemu-img: Change info key names for protocol nodes
qapi/block-core.json | 121 +++++++++++- include/block/qapi.h | 14 +- block/file-posix.c | 30 +++ block/monitor/block-hmp-cmds.c | 2 +- block/qapi.c | 319 ++++++++++++++++++++++++------- block/vmdk.c | 8 +- qemu-img.c | 76 +++++--- qemu-io-cmds.c | 5 +- tests/qemu-iotests/065 | 2 +- tests/qemu-iotests/106 | 4 +- tests/qemu-iotests/214 | 6 +- tests/qemu-iotests/302.out | 5 + tests/qemu-iotests/308 | 4 +- tests/qemu-iotests/common.filter | 22 ++- tests/qemu-iotests/common.rc | 22 ++- tests/qemu-iotests/iotests.py | 18 +- 16 files changed, 519 insertions(+), 139 deletions(-)
Signature · 1 line
2.35.3
PATCH v2 1/12 block: Improve empty format-specific info dump
When a block driver supports obtaining format-specific information, but that object only contains optional fields, it is possible that none of them are present, so that dump_qobject() (called by bdrv_image_info_specific_dump()) will not print anything.
The callers of bdrv_image_info_specific_dump() put a header above this information ("Format specific information:\n"), which will look strange when there is nothing below. Modify bdrv_image_info_specific_dump() to print this header instead of its callers, and only if there is indeed something to be printed.
--- include/block/qapi.h | 3 ++- block/qapi.c | 41 +++++++++++++++++++++++++++++++++++++---- qemu-io-cmds.c | 4 ++-- 3 files changed, 41 insertions(+), 7 deletions(-)
include/block/qapi.h +2 -1
@@ -40,6 +40,7 @@ void bdrv_query_image_info(BlockDriverState *bs, Error **errp);void bdrv_snapshot_dump (QEMUSnapshotInfo *sn); -void bdrv_image_info_specific_dump (ImageInfoSpecific *info_spec); +void bdrv_image_info_specific_dump (ImageInfoSpecific *info_spec, +const char *prefix);void bdrv_image_info_dump (ImageInfo *info);#endif
block/qapi.c +37 -4
58 lines not on this page · raw message
qemu-io-cmds.c +2 -2
@@ -1819,8 +1819,8 @@ static int info_f(BlockBackend *blk, int argc, char **argv)return -EIO ; }if (spec_info) { -printf ("Format specific information:\n" ); -bdrv_image_info_specific_dump (spec_info); +bdrv_image_info_specific_dump (spec_info, +"Format specific information:\n" );qapi_free_ImageInfoSpecific (spec_info); }
Signature · 1 line
2.35.3
Re: [PATCH v2 01/12] block: Improve empty format-specific info dump
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 18 lines
When a block driver supports obtaining format-specific information, but that object only contains optional fields, it is possible that none of them are present, so that
dump_qobject()(called bybdrv_image_info_specific_dump()) will not print anything.The callers of
bdrv_image_info_specific_dump()put a header above this information ("Format specific information:\n"), which will look strange when there is nothing below. Modifybdrv_image_info_specific_dump()to print this header instead of its callers, and only if there is indeed something to be printed.Signed-off-by Hanna Reitz <[email protected]>qemu-io-cmds.c +0
-0
qemu-io-cmds.c static int info_f(BlockBackend *blk, int argc, char **argv) · line 1825
Quoted 9 lines
qemu-io-cmds.c +2
-2@@ -1819,8 +1819,8 @@ static int info_f(BlockBackend *blk, int argc, char **argv)return -EIO ; }if (spec_info) { -printf ("Format specific information:\n" ); -bdrv_image_info_specific_dump (spec_info); +bdrv_image_info_specific_dump (spec_info, +"Format specific information:\n" );qapi_free_ImageInfoSpecific (spec_info); }
Interesting observation here: That qemu-io uses printf() instead of qemu_printf() for the top level, but then dump_qobject() (which results in qemu_printf()) for the format specific information, means that if you use the 'qemu-io' HMP command, you'll get half of the output on stdout and the other half in the monitor.
This series doesn't fix this, but the split makes a little more sense after this patch at least...
Kevin
Re: [PATCH v2 01/12] block: Improve empty format-specific info dump
Quoted 24 lines · Kevin Wolf
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 23 lines
When a block driver supports obtaining format-specific information, but that object only contains optional fields, it is possible that none of them are present, so that
dump_qobject()(called bybdrv_image_info_specific_dump()) will not print anything.The callers of
bdrv_image_info_specific_dump()put a header above this information ("Format specific information:\n"), which will look strange when there is nothing below. Modifybdrv_image_info_specific_dump()to print this header instead of its callers, and only if there is indeed something to be printed.Signed-off-by Hanna Reitz <[email protected]>qemu-io-cmds.c +1
-2@@ -1819,8 +1819,8 @@ static int info_f(BlockBackend *blk, int argc, char **argv)return -EIO ; }if (spec_info) { -printf ("Format specific information:\n" ); -bdrv_image_info_specific_dump (spec_info); + bdrv_image_info_specific_dump(spec_info,
Quoted 8 lines
qemu-io-cmds.c static int info_f(BlockBackend *blk, int argc, char **argv) · line 1825
Quoted 3 lines
qemu-io-cmds.c +1
-0+ "Format specific information:\n");qapi_free_ImageInfoSpecific (spec_info); }Interesting observation here: That qemu-io uses
printf()instead ofqemu_printf()for the top level, but thendump_qobject()(which results inqemu_printf()) for the format specific information, means that if you use the 'qemu-io' HMP command, you'll get half of the output on stdout and the other half in the monitor.
Hu. I can’t find a single instance of qemu_printf() in qemu-io-cmds.c, but then I assume all printf()s there should really be qemu_printf()?
Hanna
Quoted 4 lines
This series doesn't fix this, but the split makes a little more sense after this patch at least...
Kevin
Re: [PATCH v2 01/12] block: Improve empty format-specific info dump
Am 20.01.2023 um 14:35 hat Hanna Czenczek geschrieben:
Quoted 34 lines · Kevin Wolf
Quoted 24 lines
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 23 lines
When a block driver supports obtaining format-specific information, but that object only contains optional fields, it is possible that none of them are present, so that
dump_qobject()(called bybdrv_image_info_specific_dump()) will not print anything.The callers of
bdrv_image_info_specific_dump()put a header above this information ("Format specific information:\n"), which will look strange when there is nothing below. Modifybdrv_image_info_specific_dump()to print this header instead of its callers, and only if there is indeed something to be printed.Signed-off-by Hanna Reitz <[email protected]>qemu-io-cmds.c +1
-2@@ -1819,8 +1819,8 @@ static int info_f(BlockBackend *blk, int argc, char **argv)return -EIO ; }if (spec_info) { -printf ("Format specific information:\n" ); -bdrv_image_info_specific_dump (spec_info); + bdrv_image_info_specific_dump(spec_info,Quoted 8 lines
qemu-io-cmds.c static int info_f(BlockBackend *blk, int argc, char **argv) · line 1825
Quoted 3 lines
qemu-io-cmds.c +1
-0+ "Format specific information:\n");qapi_free_ImageInfoSpecific (spec_info); }Interesting observation here: That qemu-io uses
printf()instead ofqemu_printf()for the top level, but thendump_qobject()(which results inqemu_printf()) for the format specific information, means that if you use the 'qemu-io' HMP command, you'll get half of the output on stdout and the other half in the monitor.
Quoted 2 lines
Hu. I can’t find a single instance of
qemu_printf()inqemu-io-cmds.c, but then I assume all printf()s there should really beqemu_printf()?
That would probably be the most consistent way.
I expect it would change the output of some qemu-iotests cases, but we can explicitly print whatever was returned in QMP to keep the same information in the test output.
Kevin
PATCH v2 2/12 block/file: Add file-specific image info
In reply to Hanna ReitzAdd some (optional) information that the file driver can provide for image files, namely the extent size hint.
--- qapi/block-core.json | 26 ++++++++++++++++++++++++-- block/file-posix.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 2 deletions(-)
qapi/block-core.json +24 -2
@@ -139,16 +139,29 @@ '*encryption-format': 'RbdImageEncryptionFormat' } } +## +# @ImageInfoSpecificFile: +# +# @extent-size-hint: Extent size hint (if available) +# +# Since: 7.1 +## +{ 'struct': 'ImageInfoSpecificFile', + 'data': { + '*extent-size-hint': 'size' + } } + ## # @ImageInfoSpecificKind: # # @luks: Since 2.7 # @rbd: Since 6.1 +# @file: Since 7.1 # # Since: 1.7 ## { 'enum': 'ImageInfoSpecificKind', - 'data': [ 'qcow2', 'vmdk', 'luks', 'rbd' ] } + 'data': [ 'qcow2', 'vmdk', 'luks', 'rbd', 'file' ] } ## # @ImageInfoSpecificQCow2Wrapper:
@@ -185,6 +198,14 @@ { 'struct': 'ImageInfoSpecificRbdWrapper', 'data': { 'data': 'ImageInfoSpecificRbd' } } +## +# @ImageInfoSpecificFileWrapper: +# +# Since: 7.1 +## +{ 'struct': 'ImageInfoSpecificFileWrapper', + 'data': { 'data': 'ImageInfoSpecificFile' } } + ## # @ImageInfoSpecific: #
@@ -199,7 +220,8 @@ 'qcow2': 'ImageInfoSpecificQCow2Wrapper', 'vmdk': 'ImageInfoSpecificVmdkWrapper', 'luks': 'ImageInfoSpecificLUKSWrapper', - 'rbd': 'ImageInfoSpecificRbdWrapper' + 'rbd': 'ImageInfoSpecificRbdWrapper', + 'file': 'ImageInfoSpecificFileWrapper' } } ##
block/file-posix.c +30 -0
@@ -3083,6 +3083,34 @@ static int raw_get_info(BlockDriverState *bs, BlockDriverInfo *bdi)return 0 ; } +static ImageInfoSpecific *raw_get_specific_info (BlockDriverState *bs, +Error **errp) +{ +BDRVRawState *s = bs->opaque; +ImageInfoSpecificFile *file_info =g_new0 (ImageInfoSpecificFile,1 ); +ImageInfoSpecific *spec_info =g_new (ImageInfoSpecific,1 ); + + *spec_info = (ImageInfoSpecific ){ + .type =IMAGE_INFO_SPECIFIC_KIND_FILE , + .u.file.data = file_info, + }; + +#ifdef FS_IOC_FSGETXATTR + { +struct fsxattr attr; +int ret; + + ret =ioctl (s->fd,FS_IOC_FSGETXATTR , &attr); +if (!ret && attr.fsx_extsize !=0 ) { + file_info->has_extent_size_hint = true; + file_info->extent_size_hint = attr.fsx_extsize; + } + } +#endif + +return spec_info; +} +static BlockStatsSpecificFile get_blockstats_specific_file (BlockDriverState *bs) {BDRVRawState *s = bs->opaque;
@@ -3316,6 +3344,7 @@ BlockDriver bdrv_file = { .bdrv_co_truncate = raw_co_truncate, .bdrv_getlength = raw_getlength, .bdrv_get_info = raw_get_info, + .bdrv_get_specific_info = raw_get_specific_info, .bdrv_get_allocated_file_size = raw_get_allocated_file_size, .bdrv_get_specific_stats = raw_get_specific_stats,
@@ -3688,6 +3717,7 @@ static BlockDriver bdrv_host_device = { .bdrv_co_truncate = raw_co_truncate, .bdrv_getlength = raw_getlength, .bdrv_get_info = raw_get_info, + .bdrv_get_specific_info = raw_get_specific_info, .bdrv_get_allocated_file_size = raw_get_allocated_file_size, .bdrv_get_specific_stats = hdev_get_specific_stats,
Signature · 1 line
2.35.3
Re: [PATCH v2 02/12] block/file: Add file-specific image info
Quoted 8 lines · Hanna Reitz
Add some (optional) information that the file driver can provide for image files, namely the extent size hint.
Signed-off-by Hanna Reitz <[email protected]>--- qapi/block-core.json | 26 ++++++++++++++++++++++++-- block/file-posix.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 2 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
PATCH v2 3/12 block/vmdk: Change extent info type
In reply to Hanna ReitzVMDK's implementation of .bdrv_get_specific_info() returns information about its extent files, ostensibly in the form of ImageInfo objects. However, it does not get this information through bdrv_query_image_info(), but fills only a select few fields with custom information that does not always match the fields' purposes.
For example, @format, which is supposed to be a block driver name, is filled with the extent type, e.g. SPARSE or FLAT.
In ImageInfo, @compressed shows whether the data that can be seen in the image is stored in compressed form or not. For example, a compressed qcow2 image will store compressed data in its data file, but when accessing the qcow2 node, you will see normal data. This is not how VMDK uses the @compressed field for its extent files: Instead, it signifies whether accessing the extent file will yield compressed data (which the VMDK driver then (de-)compresses).
Create a new structure to represent the extent information. This allows us to clarify the fields' meanings, and it clearly shows that these are not complete ImageInfo objects. (That is, if a user wants an extent file's ImageInfo object, they will need to query it separately, and will not get it from ImageInfoSpecificVmdk.extents.)
Note that this removes the last use of ['ImageInfo'] (i.e. an array of ImageInfo objects), so the QAPI generator will no longer generate ImageInfoList by default. However, we use it in qemu-img.c, so we need to create a dummy object to force the generate to create that type, similarly to DummyForceArrays in machine.json (introduced in commit 9f08c8ec73878122ad4b061ed334f0437afaaa32 ("qapi: Lazy creation of array types")).
---
I'm not sure whether this is an incompatible change. I'm also not sure if it even matters whether it's an incompatible change (i.e. whether anyone cares).
I believe we can get away without this change. I find it useful to make it clear that (A) this extent information is not what you would find in other ImageInfo objects (i.e., I consider this a fix for ImageInfoSpecificVmdk's @extents field), and (B) that the upcoming BlockGraphInfo type will not duplicate the extent nodes' ImageInfo information, because those are actual ImageInfo objects.
We can probably replace this patch by clarifying all of this in documentation, but if possible I would prefer a syntactic clarification (as done here).
--- qapi/block-core.json | 38 +++++++++++++++++++++++++++++++++++++- block/vmdk.c | 8 ++++---- 2 files changed, 41 insertions(+), 5 deletions(-)
qapi/block-core.json +37 -1
@@ -124,7 +124,33 @@ 'create-type': 'str', 'cid': 'int', 'parent-cid': 'int', - 'extents': ['ImageInfo'] + 'extents': ['VmdkExtentInfo'] + } } + +## +# @VmdkExtentInfo: +# +# Information about a VMDK extent file +# +# @filename: Name of the extent file +# +# @format: Extent type (e.g. FLAT or SPARSE) +# +# @virtual-size: Number of bytes covered by this extent +# +# @cluster-size: Cluster size in bytes (for non-flat extents) +# +# @compressed: Whether this extent contains compressed data +# +# Since: 7.1 +## +{ 'struct': 'VmdkExtentInfo', + 'data': { + 'filename': 'str', + 'format': 'str', + 'virtual-size': 'int', + '*cluster-size': 'int', + '*compressed': 'bool' } } ##
@@ -5638,3 +5664,13 @@ 'data': { 'device': 'str', '*id': 'str', '*name': 'str'}, 'returns': 'SnapshotInfo', 'allow-preconfig': true } + +## +# @DummyBlockCoreForceArrays: +# +# Not used by QMP; hack to let us use ImageInfoList internally +# +# Since: 7.1 +## +{ 'struct': 'DummyBlockCoreForceArrays', + 'data': { 'unused-image-info': ['ImageInfo'] } }
block/vmdk.c +4 -4
@@ -2908,12 +2908,12 @@ static int vmdk_has_zero_init(BlockDriverState *bs)return 1 ; } -static ImageInfo *vmdk_get_extent_info (VmdkExtent *extent) +static VmdkExtentInfo *vmdk_get_extent_info (VmdkExtent *extent) { -ImageInfo *info =g_new0 (ImageInfo,1 ); +VmdkExtentInfo *info =g_new0 (VmdkExtentInfo,1 );bdrv_refresh_filename (extent->file->bs); - *info = (ImageInfo ){ + *info = (VmdkExtentInfo ){ .filename =g_strdup (extent->file->bs->filename), .format =g_strdup (extent->type), .virtual_size = extent->sectors *BDRV_SECTOR_SIZE ,
@@ -2992,7 +2992,7 @@ static ImageInfoSpecific *vmdk_get_specific_info(BlockDriverState *bs,int i;BDRVVmdkState *s = bs->opaque;ImageInfoSpecific *spec_info =g_new0 (ImageInfoSpecific,1 ); -ImageInfoList **tail; +VmdkExtentInfoList **tail; *spec_info = (ImageInfoSpecific ){ .type =IMAGE_INFO_SPECIFIC_KIND_VMDK ,
Signature · 1 line
2.35.3
Re: [PATCH v2 03/12] block/vmdk: Change extent info type
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 9 lines
VMDK's implementation of .bdrv_get_specific_info() returns information about its extent files, ostensibly in the form of ImageInfo objects. However, it does not get this information through
bdrv_query_image_info(), but fills only a select few fields with custom information that does not always match the fields' purposes.For example, @format, which is supposed to be a block driver name, is filled with the extent type, e.g. SPARSE or FLAT.
Quoted 7 lines
In ImageInfo, @compressed shows whether the data that can be seen in the image is stored in compressed form or not. For example, a compressed qcow2 image will store compressed data in its data file, but when accessing the qcow2 node, you will see normal data. This is not how VMDK uses the @compressed field for its extent files: Instead, it signifies whether accessing the extent file will yield compressed data (which the VMDK driver then (de-)compresses).
Actually, VMDK was the only user of the field in ImageInfo. qcow2 doesn't set the field at all because it would have to parse all L2 tables in order to set it.
So I suppose @compressed can now be removed from ImageInfo?
Kevin
Re: [PATCH v2 03/12] block/vmdk: Change extent info type
Quoted 17 lines · Kevin Wolf
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 9 lines
VMDK's implementation of .bdrv_get_specific_info() returns information about its extent files, ostensibly in the form of ImageInfo objects. However, it does not get this information through
bdrv_query_image_info(), but fills only a select few fields with custom information that does not always match the fields' purposes.For example, @format, which is supposed to be a block driver name, is filled with the extent type, e.g. SPARSE or FLAT.
Quoted 7 lines
In ImageInfo, @compressed shows whether the data that can be seen in the image is stored in compressed form or not. For example, a compressed qcow2 image will store compressed data in its data file, but when accessing the qcow2 node, you will see normal data. This is not how VMDK uses the @compressed field for its extent files: Instead, it signifies whether accessing the extent file will yield compressed data (which the VMDK driver then (de-)compresses).
Quoted 3 lines
Actually, VMDK was the only user of the field in ImageInfo. qcow2 doesn't set the field at all because it would have to parse all L2 tables in order to set it.
Right, makes sense.
Quoted 1 line
So I suppose @compressed can now be removed from ImageInfo?
I think so. Looks like it was indeed introduced specifically for VMDK’s extent information (cbe82d7fb32, f4c129a38a5) so that ImageInfo could be used there, which this patch here changes. So we should probably indeed remove the field – it did lead me to naïvely believe that it would have a meaning on images that actually support compression (like qcow2) after all.
Hanna
Re: [PATCH v2 03/12] block/vmdk: Change extent info type
In reply to Kevin WolfQuoted 18 lines · Kevin Wolf
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 9 lines
VMDK's implementation of .bdrv_get_specific_info() returns information about its extent files, ostensibly in the form of ImageInfo objects. However, it does not get this information through
bdrv_query_image_info(), but fills only a select few fields with custom information that does not always match the fields' purposes.For example, @format, which is supposed to be a block driver name, is filled with the extent type, e.g. SPARSE or FLAT.
Quoted 7 lines
In ImageInfo, @compressed shows whether the data that can be seen in the image is stored in compressed form or not. For example, a compressed qcow2 image will store compressed data in its data file, but when accessing the qcow2 node, you will see normal data. This is not how VMDK uses the @compressed field for its extent files: Instead, it signifies whether accessing the extent file will yield compressed data (which the VMDK driver then (de-)compresses).
Quoted 5 lines
Actually, VMDK was the only user of the field in ImageInfo. qcow2 doesn't set the field at all because it would have to parse all L2 tables in order to set it.
So I suppose @compressed can now be removed from ImageInfo?
I think you are okay for VMDK (the new struct has the same field names, but better meanings, and the on-the-wire representation for someone querying a known-VMDK image doesn't change). For non-VMDK, any code that was querying @compressed will break, but arguably no one was doing that since it would have always been false. If we want to be super-conservative, we deprecate the field now and only remove it from ImageInfo in a later release, but I'd rather trust Markus on this decision.
On a side note - would it be worth adding a bit to the qcow2 header (one of the compatible_feature bits seems best) which we set when writing a compressed cluster, so that on newer images, it is an O(1) probe of whether the image contains (or at least has contained in the past) a compressed cluster? Or is that going to add needless overhead for something we really don't need to know all that often?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [PATCH v2 03/12] block/vmdk: Change extent info type
Am 28.01.2023 um 00:06 hat Eric Blake geschrieben:
Quoted 25 lines · Kevin Wolf
Quoted 18 lines
Am 20.06.2022 um 18:26 hat Hanna Reitz geschrieben:
Quoted 9 lines
VMDK's implementation of .bdrv_get_specific_info() returns information about its extent files, ostensibly in the form of ImageInfo objects. However, it does not get this information through
bdrv_query_image_info(), but fills only a select few fields with custom information that does not always match the fields' purposes.For example, @format, which is supposed to be a block driver name, is filled with the extent type, e.g. SPARSE or FLAT.
Quoted 7 lines
In ImageInfo, @compressed shows whether the data that can be seen in the image is stored in compressed form or not. For example, a compressed qcow2 image will store compressed data in its data file, but when accessing the qcow2 node, you will see normal data. This is not how VMDK uses the @compressed field for its extent files: Instead, it signifies whether accessing the extent file will yield compressed data (which the VMDK driver then (de-)compresses).
Quoted 5 lines
Actually, VMDK was the only user of the field in ImageInfo. qcow2 doesn't set the field at all because it would have to parse all L2 tables in order to set it.
So I suppose @compressed can now be removed from ImageInfo?
Quoted 8 lines
I think you are okay for VMDK (the new struct has the same field names, but better meanings, and the on-the-wire representation for someone querying a known-VMDK image doesn't change). For non-VMDK, any code that was querying @compressed will break, but arguably no one was doing that since it would have always been false. If we want to be super-conservative, we deprecate the field now and only remove it from ImageInfo in a later release, but I'd rather trust Markus on this decision.
It is an optional field and VMDK is the only driver that ever provided it, and only in the context of extent information. In this case, the information on the wire stays the same after this patch.
So I don't think there is any visible difference for a client, apart from schema introspection?
Quoted 6 lines
On a side note - would it be worth adding a bit to the qcow2 header (one of the
compatible_featurebits seems best) which we set when writing a compressed cluster, so that on newer images, it is an O(1) probe of whether the image contains (or at least has contained in the past) a compressed cluster? Or is that going to add needless overhead for something we really don't need to know all that often?
I think the only use for it would be displaying it in 'qemu-img info'.
Would you combine two bits then? One for "compressed bit is valid" and one for "contains compressed clusters"? Because with one bit you can only distinguish "compressed" from "don't know", but there wouldn't be a way to say for certain that an image is uncompressed.
Kevin
PATCH v2 4/12 block: Split BlockNodeInfo off of ImageInfo
In reply to Hanna ReitzImageInfo sometimes contains flat information, and sometimes it does not. Split off a BlockNodeInfo struct, which only contains information about a single node and has no link to the backing image.
We do this so we can extend BlockNodeInfo to a BlockGraphInfo struct, which has links to all child nodes, not just the backing node. It would be strange to base BlockGraphInfo on ImageInfo, because then this extended struct would have two links to the backing node (one in BlockGraphInfo as one of all the child links, and one in ImageInfo).
Furthermore, it is quite common to ignore the backing-image field
altogether: bdrv_query_image_info() does not set it, and bdrv_image_info_dump() does not evaluate it. That signals that we should have different structs for describing a single node and one that has a link to the backing image.
Still, bdrv_query_image_info() and bdrv_image_info_dump() are not changed too much in this patch. Follow-up patches will handle them.
---
I hope this is not an incompatible change, but I'm not sure, I have to admit.
--- qapi/block-core.json | 22 +++++++++--- include/block/qapi.h | 3 ++ block/qapi.c | 86 ++++++++++++++++++++++++++++++++------------ 3 files changed, 84 insertions(+), 27 deletions(-)
qapi/block-core.json +17 -5
@@ -251,7 +251,7 @@ } } ## -# @ImageInfo: +# @BlockNodeInfo: # # Information about a QEMU image file #
@@ -279,22 +279,34 @@ # # @snapshots: list of VM snapshots # -# @backing-image: info of the backing image (since 1.6) -# # @format-specific: structure supplying additional format-specific # information (since 1.7) # # Since: 1.3 ## -{ 'struct': 'ImageInfo', +{ 'struct': 'BlockNodeInfo', 'data': {'filename': 'str', 'format': 'str', '*dirty-flag': 'bool', '*actual-size': 'int', 'virtual-size': 'int', '*cluster-size': 'int', '*encrypted': 'bool', '*compressed': 'bool', '*backing-filename': 'str', '*full-backing-filename': 'str', '*backing-filename-format': 'str', '*snapshots': ['SnapshotInfo'], - '*backing-image': 'ImageInfo', '*format-specific': 'ImageInfoSpecific' } } +## +# @ImageInfo: +# +# Information about a QEMU image file, and potentially its backing image +# +# @backing-image: info of the backing image +# +# Since: 7.1 +## +{ 'struct': 'ImageInfo', + 'base': 'BlockNodeInfo', + 'data': { + '*backing-image': 'ImageInfo' + } } + ## # @ImageCheck: #
include/block/qapi.h +3 -0
@@ -35,6 +35,9 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,int bdrv_query_snapshot_info_list (BlockDriverState *bs,SnapshotInfoList **p_list,Error **errp); +void bdrv_query_block_node_info (BlockDriverState *bs, +BlockNodeInfo **p_info, +Error **errp);void bdrv_query_image_info (BlockDriverState *bs,ImageInfo **p_info,Error **errp);
block/qapi.c +64 -22
124 lines not on this page · raw message
Signature · 1 line
2.35.3
PATCH v2 5/12 qemu-img: Use BlockNodeInfo
In reply to Hanna Reitzqemu-img info never uses ImageInfo's backing-image field, because it opens the backing chain one by one with BDRV_O_NO_BACKING, and prints all backing chain nodes' information consecutively. Use BlockNodeInfo to make it clear that we only print information about a single node, and that we are not using the backing-image field.
Notably, bdrv_image_info_dump() does not evaluate the backing-image field, so we can easily make it take a BlockNodeInfo pointer (and consequentially rename it to bdrv_node_info_dump()). It makes more sense this way, because again, the interface now makes it syntactically clear that backing-image is ignored by this function.
--- qapi/block-core.json | 4 +-- include/block/qapi.h | 2 +- block/monitor/block-hmp-cmds.c | 2 +- block/qapi.c | 2 +- qemu-img.c | 48 +++++++++++++++++----------------- 5 files changed, 29 insertions(+), 29 deletions(-)
qapi/block-core.json +2 -2
@@ -5680,9 +5680,9 @@ ## # @DummyBlockCoreForceArrays: # -# Not used by QMP; hack to let us use ImageInfoList internally +# Not used by QMP; hack to let us use BlockNodeInfoList internally # # Since: 7.1 ## { 'struct': 'DummyBlockCoreForceArrays', - 'data': { 'unused-image-info': ['ImageInfo'] } } + 'data': { 'unused-block-node-info': ['BlockNodeInfo'] } }
include/block/qapi.h +1 -1
@@ -45,5 +45,5 @@ void bdrv_query_image_info(BlockDriverState *bs,void bdrv_snapshot_dump (QEMUSnapshotInfo *sn);void bdrv_image_info_specific_dump (ImageInfoSpecific *info_spec,const char *prefix); -void bdrv_image_info_dump (ImageInfo *info); +void bdrv_node_info_dump (BlockNodeInfo *info);#endif
block/monitor/block-hmp-cmds.c +1 -1
@@ -734,7 +734,7 @@ static void print_block_info(Monitor *mon, BlockInfo *info,monitor_printf (mon,"\nImages:\n" ); image_info = inserted->image;while (1 ) { -bdrv_image_info_dump (image_info); +bdrv_node_info_dump (qapi_ImageInfo_base (image_info));if (image_info->has_backing_image) { image_info = image_info->backing_image; }else {
block/qapi.c +1 -1
@@ -865,7 +865,7 @@ void bdrv_image_info_specific_dump(ImageInfoSpecific *info_spec,visit_free (v); } -void bdrv_image_info_dump (ImageInfo *info) +void bdrv_node_info_dump (BlockNodeInfo *info) {char *size_buf, *dsize_buf;if (!info->has_actual_size) {
qemu-img.c +24 -24
132 lines not on this page · raw message
Signature · 1 line
2.35.3
PATCH v2 6/12 block/qapi: Let bdrv_query_image_info() recurse
In reply to Hanna ReitzThere is no real reason why bdrv_query_image_info() should generally not recurse. The ImageInfo struct has a pointer to the backing image, so it should generally be filled, unless the caller explicitly opts out.
This moves the recursing code from bdrv_block_device_info() into bdrv_query_image_info().
--- include/block/qapi.h | 2 + block/qapi.c | 94 +++++++++++++++++++++++++++----------------- 2 files changed, 59 insertions(+), 37 deletions(-)
include/block/qapi.h +2 -0
@@ -40,6 +40,8 @@ void bdrv_query_block_node_info(BlockDriverState *bs, Error **errp);void bdrv_query_image_info (BlockDriverState *bs,ImageInfo **p_info, +bool flat, +bool skip_implicit_filters,Error **errp);void bdrv_snapshot_dump (QEMUSnapshotInfo *sn);
block/qapi.c +57 -37
133 lines not on this page · raw message
Signature · 1 line
2.35.3
PATCH v2 7/12 block/qapi: Introduce BlockGraphInfo
In reply to Hanna ReitzIntroduce a new QAPI type BlockGraphInfo and an associated bdrv_query_block_graph_info() function that recursively gathers BlockNodeInfo objects through a block graph.
A follow-up patch is going to make "qemu-img info" use this to print information about all nodes that are (usually implicitly) opened for a given image file.
--- qapi/block-core.json | 35 ++++++++++++++++++++++++++++++++ include/block/qapi.h | 3 +++ block/qapi.c | 48 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 86 insertions(+)
qapi/block-core.json +35 -0
@@ -307,6 +307,41 @@ '*backing-image': 'ImageInfo' } } +## +# @BlockChildInfo: +# +# Information about all nodes in the block graph starting at some node, +# annotated with information about that node in relation to its parent. +# +# @name: Child name of the root node in the BlockGraphInfo struct, in its role +# as the child of some undescribed parent node +# +# @info: Block graph information starting at this node +# +# Since: 7.1 +## +{ 'struct': 'BlockChildInfo', + 'data': { + 'name': 'str', + 'info': 'BlockGraphInfo' + } } + +## +# @BlockGraphInfo: +# +# Information about all nodes in a block (sub)graph in the form of BlockNodeInfo +# data. +# The base BlockNodeInfo struct contains the information for the (sub)graph's +# root node. +# +# @children: Array of links to this node's child nodes' information +# +# Since: 7.1 +## +{ 'struct': 'BlockGraphInfo', + 'base': 'BlockNodeInfo', + 'data': { 'children': ['BlockChildInfo'] } } + ## # @ImageCheck: #
include/block/qapi.h +3 -0
@@ -43,6 +43,9 @@ void bdrv_query_image_info(BlockDriverState *bs,bool flat, bool skip_implicit_filters, Error **errp); +void bdrv_query_block_graph_info (BlockDriverState *bs, +BlockGraphInfo **p_info, +Error **errp);void bdrv_snapshot_dump (QEMUSnapshotInfo *sn); void bdrv_image_info_specific_dump(ImageInfoSpecific *info_spec,
block/qapi.c +48 -0
54 lines not on this page · raw message
Signature · 1 line
2.35.3
PATCH v2 8/12 block/qapi: Add indentation to bdrv_node_info_dump()
In reply to Hanna ReitzIn order to let qemu-img info present a block graph, add a parameter to bdrv_node_info_dump() and bdrv_image_info_specific_dump() so that the information of nodes below the root level can be given an indentation.
--- include/block/qapi.h | 5 ++-- block/monitor/block-hmp-cmds.c | 2 +- block/qapi.c | 47 +++++++++++++++++++--------------- qemu-img.c | 2 +- qemu-io-cmds.c | 3 ++- 5 files changed, 34 insertions(+), 25 deletions(-)
include/block/qapi.h +3 -2
@@ -49,6 +49,7 @@ void bdrv_query_block_graph_info(BlockDriverState *bs,void bdrv_snapshot_dump (QEMUSnapshotInfo *sn);void bdrv_image_info_specific_dump (ImageInfoSpecific *info_spec, -const char *prefix); -void bdrv_node_info_dump (BlockNodeInfo *info); +const char *prefix, +int indentation); +void bdrv_node_info_dump (BlockNodeInfo *info,int indentation);#endif
block/monitor/block-hmp-cmds.c +1 -1
@@ -734,7 +734,7 @@ static void print_block_info(Monitor *mon, BlockInfo *info,monitor_printf (mon,"\nImages:\n" ); image_info = inserted->image;while (1 ) { -bdrv_node_info_dump (qapi_ImageInfo_base (image_info)); +bdrv_node_info_dump (qapi_ImageInfo_base (image_info),0 );if (image_info->has_backing_image) { image_info = image_info->backing_image; }else {
block/qapi.c +27 -20
109 lines not on this page · raw message
qemu-img.c +1 -1
@@ -2843,7 +2843,7 @@ static void dump_human_image_info_list(BlockNodeInfoList *list) } delim = true; -bdrv_node_info_dump (elem->value); +bdrv_node_info_dump (elem->value,0 ); } }
qemu-io-cmds.c +2 -1
@@ -1820,7 +1820,8 @@ static int info_f(BlockBackend *blk, int argc, char **argv) }if (spec_info) {bdrv_image_info_specific_dump (spec_info, -"Format specific information:\n" ); +"Format specific information:\n" , +0 );qapi_free_ImageInfoSpecific (spec_info); }
Signature · 1 line
2.35.3
PATCH v2 9/12 iotests: Filter child node information
In reply to Hanna ReitzBefore we let qemu-img info print child node information, have common.filter, common.rc, and iotests.py filter it from the test output so we get as few reference output changes as possible.
--- tests/qemu-iotests/common.filter | 22 ++++++++++++++-------- tests/qemu-iotests/common.rc | 22 ++++++++++++++-------- tests/qemu-iotests/iotests.py | 18 +++++++++++++++--- 3 files changed, 43 insertions(+), 19 deletions(-)
tests/qemu-iotests/common.filter +14 -8
@@ -223,6 +223,7 @@ _filter_img_info() discard=0 regex_json_spec_start='^ *"format-specific": \{' + regex_json_child_start='^ *"children": \[' gsed -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \ -e "s#$TEST_DIR#TEST_DIR#g" \
@@ -251,20 +252,25 @@ _filter_img_info() -e 's/\(compression type: \)\(zlib\|zstd\)/\1COMPRESSION_TYPE/' \ -e "s/uuid: [-a-f0-9]\\+/uuid: 00000000-0000-0000-0000-000000000000/" | \ while IFS='' read -r line; do - if [[ $format_specific == 1 ]]; then - discard=0 - elif [[ $line == "Format specific information:" ]]; then - discard=1 - elif [[ $line =~ $regex_json_spec_start ]]; then - discard=2 - regex_json_spec_end="^${line%%[^ ]*}\\},? *$" + if [[ $discard == 0 ]]; then + if [[ $format_specific == 0 && $line == "Format specific information:" ]]; then + discard=1 + elif [[ $line =~ "Child node '/" ]]; then + discard=1 + elif [[ $line =~ $regex_json_spec_start ]]; then + discard=2 + regex_json_end="^${line%%[^ ]*}\\},? *$" + elif [[ $line =~ $regex_json_child_start ]]; then + discard=2 + regex_json_end="^${line%%[^ ]*}\\],? *$" + fi fi if [[ $discard == 0 ]]; then echo "$line" elif [[ $discard == 1 && ! $line ]]; then echo discard=0 - elif [[ $discard == 2 && $line =~ $regex_json_spec_end ]]; then + elif [[ $discard == 2 && $line =~ $regex_json_end ]]; then discard=0 fi done
tests/qemu-iotests/common.rc +14 -8
@@ -711,6 +711,7 @@ _img_info() discard=0 regex_json_spec_start='^ *"format-specific": \{' + regex_json_child_start='^ *"children": \[' $QEMU_IMG info $QEMU_IMG_EXTRA_ARGS "$@" "$TEST_IMG" 2>&1 | \ sed -e "s#$REMOTE_TEST_DIR#TEST_DIR#g" \ -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
@@ -721,20 +722,25 @@ _img_info() -e "/^disk size:/ D" \ -e "/actual-size/ D" | \ while IFS='' read -r line; do - if [[ $format_specific == 1 ]]; then - discard=0 - elif [[ $line == "Format specific information:" ]]; then - discard=1 - elif [[ $line =~ $regex_json_spec_start ]]; then - discard=2 - regex_json_spec_end="^${line%%[^ ]*}\\},? *$" + if [[ $discard == 0 ]]; then + if [[ $format_specific == 0 && $line == "Format specific information:" ]]; then + discard=1 + elif [[ $line =~ "Child node '/" ]]; then + discard=1 + elif [[ $format_specific == 0 && $line =~ $regex_json_spec_start ]]; then + discard=2 + regex_json_end="^${line%%[^ ]*}\\},? *$" + elif [[ $line =~ $regex_json_child_start ]]; then + discard=2 + regex_json_end="^${line%%[^ ]*}\\],? *$" + fi fi if [[ $discard == 0 ]]; then echo "$line" elif [[ $discard == 1 && ! $line ]]; then echo discard=0 - elif [[ $discard == 2 && $line =~ $regex_json_spec_end ]]; then + elif [[ $discard == 2 && $line =~ $regex_json_end ]]; then discard=0 fi done
tests/qemu-iotests/iotests.py +15 -3
@@ -329,7 +329,7 @@ def qemu_img_log(*args: str, check: bool = True def img_info_log(filename: str, filter_path: Optional[str] = None, use_image_opts: bool = False, extra_args: Sequence[str] = (), - check: bool = True, + check: bool = True, drop_child_info: bool = True, ) -> None: args = ['info'] if use_image_opts:
@@ -342,7 +342,7 @@ def img_info_log(filename: str, filter_path: Optional[str] = None, output = qemu_img(*args, check=check).stdout if not filter_path: filter_path = filename - log(filter_img_info(output, filter_path)) + log(filter_img_info(output, filter_path, drop_child_info)) def qemu_io_wrap_args(args: Sequence[str]) -> List[str]: if '-f' in args or '--image-opts' in args:
@@ -642,11 +642,23 @@ def _filter(_key, value): def filter_generated_node_ids(msg): return re.sub("#block[0-9]+", "NODE_NAME", msg) -def filter_img_info(output, filename): +def filter_img_info(output: str, filename: str, + drop_child_info: bool = True) -> str: lines = [] + drop_indented = False for line in output.split('\n'): if 'disk size' in line or 'actual-size' in line: continue + + # Drop child node info + if drop_indented: + if line.startswith(' '): + continue + drop_indented = False + if drop_child_info and "Child node '/" in line: + drop_indented = True + continue + line = line.replace(filename, 'TEST_IMG') line = filter_testfiles(line) line = line.replace(imgfmt, 'IMGFMT')
Signature · 1 line
2.35.3
PATCH v2 10/12 iotests/106, 214, 308: Read only one size line
In reply to Hanna ReitzThese tests read size information (sometimes disk size, sometimes virtual size) from qemu-img info's output. Once qemu-img starts printing info about child nodes, we are going to see multiple instances of that per image, but these tests are only interested in the first one, so use "head -n 1" to get it.
--- tests/qemu-iotests/106 | 4 ++-- tests/qemu-iotests/214 | 6 ++++-- tests/qemu-iotests/308 | 4 ++-- 3 files changed, 8 insertions(+), 6 deletions(-)
tests/qemu-iotests/106 +2 -2
@@ -66,7 +66,7 @@ for create_mode in off falloc full; do expected_size=$((expected_size + $GROWTH_SIZE)) fi - actual_size=$($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" | grep 'disk size') + actual_size=$($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" | grep 'disk size' | head -n 1) actual_size=$(echo "$actual_size" | sed -e 's/^[^0-9]*\([0-9]\+\).*$/\1/') # The actual size may exceed the expected size, depending on the file
@@ -105,7 +105,7 @@ for growth_mode in falloc full; do _make_test_img -o "extent_size_hint=0" 2G $QEMU_IMG resize -f "$IMGFMT" --preallocation=$growth_mode "$TEST_IMG" +${GROWTH_SIZE}K - actual_size=$($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" | grep 'disk size') + actual_size=$($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" | grep 'disk size' | head -n 1) actual_size=$(echo "$actual_size" | sed -e 's/^[^0-9]*\([0-9]\+\).*$/\1/') if [ $actual_size -lt $GROWTH_SIZE ]; then
tests/qemu-iotests/214 +4 -2
@@ -102,7 +102,8 @@ let data_size="8 * $cluster_size" $QEMU_IO -c "write -P 0xaa 0 $data_size" "$TEST_IMG" \ 2>&1 | _filter_qemu_io | _filter_testdir sizeA=$($QEMU_IMG info --output=json "$TEST_IMG" | - sed -n '/"actual-size":/ s/[^0-9]//gp') + sed -n '/"actual-size":/ s/[^0-9]//gp' | + head -n 1) _make_test_img 2M -o cluster_size=$cluster_size echo "Write compressed data:"
@@ -124,7 +125,8 @@ $QEMU_IO -c "write -P 0xcc $offset $data_size" "json:{\ _filter_qemu_io | _filter_testdir sizeB=$($QEMU_IMG info --output=json "$TEST_IMG" | - sed -n '/"actual-size":/ s/[^0-9]//gp') + sed -n '/"actual-size":/ s/[^0-9]//gp' | + head -n 1) if [ $sizeA -lt $sizeB ] then
tests/qemu-iotests/308 +2 -2
@@ -217,12 +217,12 @@ echo echo '=== Remove export ===' # Double-check that $EXT_MP appears as a non-empty file (the raw image) -$QEMU_IMG info -f raw "$EXT_MP" | grep 'virtual size' +$QEMU_IMG info -f raw "$EXT_MP" | grep 'virtual size' | head -n 1 fuse_export_del 'export-mp' # See that the file appears empty again -$QEMU_IMG info -f raw "$EXT_MP" | grep 'virtual size' +$QEMU_IMG info -f raw "$EXT_MP" | grep 'virtual size' | head -n 1 echo echo '=== Writable export ==='
Signature · 1 line
2.35.3
PATCH v2 11/12 qemu-img: Let info print block graph
In reply to Hanna ReitzFor every node in the backing chain, collect its BlockGraphInfo struct using bdrv_query_block_graph_info(). Print all nodes' information, indenting child nodes and labelling them with a path constructed from the child names leading to the node from the root (e.g. /file/file).
Note that we open each image with BDRV_O_NO_BACKING, so its backing child is omitted from this graph, and thus presented in the previous
manner: By simply concatenating all images' information, separated with blank lines.
This affects two iotests:
- 065: Here we try to get the format node's format specific information. The pre-patch code does so by taking all lines from "Format specific information:" until an empty line. This format specific information is no longer followed by an empty line, though, but by child node information, so limit the range by "Child node '/file':".
- 302: Calls qemu_img() for qemu-img info directly, which does not filter the output, so the child node information ends up in the output.
--- qapi/block-core.json | 4 +-- qemu-img.c | 69 ++++++++++++++++++++++++++------------ tests/qemu-iotests/065 | 2 +- tests/qemu-iotests/302.out | 5 +++ 4 files changed, 56 insertions(+), 24 deletions(-)
qapi/block-core.json +2 -2
@@ -5715,9 +5715,9 @@ ## # @DummyBlockCoreForceArrays: # -# Not used by QMP; hack to let us use BlockNodeInfoList internally +# Not used by QMP; hack to let us use BlockGraphInfoList internally # # Since: 7.1 ## { 'struct': 'DummyBlockCoreForceArrays', - 'data': { 'unused-block-node-info': ['BlockNodeInfo'] } } + 'data': { 'unused-block-graph-info': ['BlockGraphInfo'] } }
qemu-img.c +48 -21
152 lines not on this page · raw message
tests/qemu-iotests/065 +1 -1
@@ -56,7 +56,7 @@ class TestQemuImgInfo(TestImageInfoSpecific): def test_human(self): data = qemu_img('info', '--output=human', test_img).stdout.split('\n') data = data[(data.index('Format specific information:') + 1) - :data.index('')] + :data.index("Child node '/file':")] for field in data: self.assertTrue(re.match('^ {4}[^ ]', field) is not None) data = [line.strip() for line in data]
tests/qemu-iotests/302.out +5 -0
@@ -4,6 +4,11 @@ image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock file format: raw virtual size: 448 KiB (458752 bytes) disk size: unavailable +Child node '/file': + image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock + file format: nbd + virtual size: 448 KiB (458752 bytes) + disk size: unavailable === Converted image info === image: TEST_IMG
Signature · 1 line
2.35.3
PATCH v2 12/12 qemu-img: Change info key names for protocol nodes
In reply to Hanna ReitzCurrently, when querying a qcow2 image, qemu-img info reports something like this:
image: test.qcow2
file format: qcow2
virtual size: 64 MiB (67108864 bytes)
disk size: 196 KiBcluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
Child node '/file':
image: test.qcow2
file format: file
virtual size: 192 KiB (197120 bytes)
disk size: 196 KiB
Format specific information:
extent size hint: 1048576
Notably, the way the keys are named is specific for image files: The filename is shown under "image", the BDS driver under "file format", and the BDS length under "virtual size". This does not make much sense for nodes that are not actually supposed to be guest images, like the /file child node shown above.
Give bdrv_node_info_dump() a @protocol parameter that gives a hint that the respective node is probably just used for data storage and does not necessarily present the data for a VM guest disk. This renames the keys so that with this patch, the output becomes:
image: test.qcow2
[...]
Child node '/file':
filename: test.qcow2
protocol type: file
file length: 192 KiB (197120 bytes)
disk size: 196 KiB
Format specific information:
extent size hint: 1048576
(Perhaps we should also rename "Format specific information", but I could not come up with anything better that will not become problematic if we guess wrong with the protocol "heuristic".)
This change affects iotest 302, which has protocol node information in its reference output.
---
This patch is completely optional and just based on personal preference. I just don't like e.g. how the child node is called an "image" when it's pretty much unusable as a guest image, or how "virtual size" is there twice but with different values.
--- include/block/qapi.h | 2 +- block/monitor/block-hmp-cmds.c | 2 +- block/qapi.c | 39 ++++++++++++++++++++++++++++------ qemu-img.c | 3 ++- tests/qemu-iotests/302.out | 6 +++--- 5 files changed, 39 insertions(+), 13 deletions(-)
include/block/qapi.h +1 -1
@@ -51,5 +51,5 @@ void bdrv_snapshot_dump(QEMUSnapshotInfo *sn);void bdrv_image_info_specific_dump (ImageInfoSpecific *info_spec,const char *prefix,int indentation); -void bdrv_node_info_dump (BlockNodeInfo *info,int indentation); +void bdrv_node_info_dump (BlockNodeInfo *info,int indentation,bool protocol);#endif
block/monitor/block-hmp-cmds.c +1 -1
@@ -734,7 +734,7 @@ static void print_block_info(Monitor *mon, BlockInfo *info,monitor_printf (mon,"\nImages:\n" ); image_info = inserted->image;while (1 ) { -bdrv_node_info_dump (qapi_ImageInfo_base (image_info),0 ); +bdrv_node_info_dump (qapi_ImageInfo_base (image_info),0 , false);if (image_info->has_backing_image) { image_info = image_info->backing_image; }else {
block/qapi.c +32 -7
@@ -934,24 +934,49 @@ void bdrv_image_info_specific_dump(ImageInfoSpecific *info_spec,visit_free (v); } -void bdrv_node_info_dump (BlockNodeInfo *info,int indentation) +/** +* Print the given @info object in human-readable form. Every field is indented +* using the given @indentation (four spaces per indentation level). +* +* When using this to print a whole block graph, @protocol can be set to true to +* signify that the given information is associated with a protocol node, i.e. +* just data storage for an image, such that the data it presents is not really +* a full VM disk. If so, several fields change name: For example, "virtual +* size" is printed as "file length". +* (Consider a qcow2 image, which is represented by a qcow2 node and a file +* node. Printing a "virtual size" for the file node does not make sense, +* because without the qcow2 node, it is not really a guest disk, so it does not +* have a "virtual size". Therefore, we call it "file length" instead.) +* +* @protocol is ignored when @indentation is 0, because we take that to mean +* that the associated node is the root node in the queried block graph, and +* thus is always to be interpreted as a standalone guest disk. +*/ +void bdrv_node_info_dump (BlockNodeInfo *info,int indentation,bool protocol) {char *size_buf, *dsize_buf; g_autofree char *ind_s = g_strdup_printf("%*s", indentation * 4, ""); +if (indentation ==0 ) { +/* Top level, consider this a normal image */ + protocol = false; + } +if (!info->has_actual_size) { dsize_buf =g_strdup ("unavailable" ); }else { dsize_buf =size_to_str (info->actual_size); } size_buf =size_to_str (info->virtual_size); -qemu_printf ("%simage: %s\n" -"%sfile format: %s\n" -"%svirtual size: %s (%" PRId64" bytes)\n" +qemu_printf ("%s%s: %s\n" +"%s%s: %s\n" +"%s%s: %s (%" PRId64" bytes)\n" "%sdisk size: %s\n" , - ind_s, info->filename, - ind_s, info->format, - ind_s, size_buf, info->virtual_size, + ind_s, protocol ?"filename" :"image" , info->filename, + ind_s, protocol ?"protocol type" :"file format" , + info->format, + ind_s, protocol ?"file length" :"virtual size" , + size_buf, info->virtual_size, ind_s, dsize_buf);g_free (size_buf);g_free (dsize_buf);
qemu-img.c +2 -1
@@ -2837,7 +2837,8 @@ static void dump_human_image_info(BlockGraphInfo *info, int indentation, {BlockChildInfoList *children_list; -bdrv_node_info_dump (qapi_BlockGraphInfo_base (info), indentation); +bdrv_node_info_dump (qapi_BlockGraphInfo_base (info), indentation, + info->children ==NULL );for (children_list = info->children; children_list; children_list = children_list->next)
tests/qemu-iotests/302.out +3 -3
@@ -5,9 +5,9 @@ file format: raw virtual size: 448 KiB (458752 bytes) disk size: unavailable Child node '/file': - image: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock - file format: nbd - virtual size: 448 KiB (458752 bytes) + filename: nbd+unix:///exp?socket=SOCK_DIR/PID-nbd-sock + protocol type: nbd + file length: 448 KiB (458752 bytes) disk size: unavailable === Converted image info ===
Signature · 1 line
2.35.3
Quoted 5 lines · Hanna Reitz
Hi,
This series is a v2 to:
https://lists.nongnu.org/archive/html/qemu-block/2022-05/msg00042.html
Ping, it looks like this still applies (to the master branch and kevin’s block-next branch at least).
Hanna
Quoted 24 lines
So the final state is that despite the QAPI changes, hopefully only the qemu-img info output changes, and it now prints every image node’s subgraph. This results in an output like the following (for a qcow2 image):
image: test.qcow2
file format: qcow2
virtual size: 64 MiB (67108864 bytes)
disk size: 196 KiBcluster_size: 65536
Format specific information:
compat: 1.1
compression type: zlib
lazy refcounts: false
refcount bits: 16
corrupt: false
extended l2: false
Child node '/file':
filename: test.qcow2
protocol type: file
file length: 192 KiB (197120 bytes)
disk size: 196 KiB
Format specific information:
extent size hint: 1048576
Am 08.12.2022 um 13:24 hat Hanna Reitz geschrieben:
Quoted 9 lines · Hanna Reitz
Quoted 5 lines
Hi,
This series is a v2 to:
https://lists.nongnu.org/archive/html/qemu-block/2022-05/msg00042.html
Ping, it looks like this still applies (to the master branch and kevin’s block-next branch at least).
Not any more. :-)
But the conflicts seemed obvious enough, so I rebased it (including changing the "Since: 7.1" occurrences to 8.0) and applied it to my block branch. Thanks!
Kevin
Quoted 6 lines · Kevin Wolf
Am 08.12.2022 um 13:24 hat Hanna Reitz geschrieben:
Quoted 5 lines · Hanna Reitz
Quoted 4 lines
Hi,
This series is a v2 to:
Quoted 8 lines
Quoted 3 lines
Ping, it looks like this still applies (to the master branch and kevin’s block-next branch at least).
Not any more. :-)
But the conflicts seemed obvious enough, so I rebased it (including changing the "Since: 7.1" occurrences to 8.0) and applied it to my block branch.
Ah, yes. That I should have fixed.
Quoted 1 line
Thanks!
Thank you! :)
Hanna