qcow2: Let check -r all repair some snapshot bits
v3 cover qcow2: Let check -r all repair some snapshot bits
Hi,
The v1 cover letter explained this series’s purpose:
https://lists.nongnu.org/archive/html/qemu-block/2019-07/msg01290.html
The v2 cover letter explained the v2 changes:
https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00921.html
The only changes in v3 are:
- Patches 12 and 13: Added notes to the commit messages why it’s OK to not let the user choose which snapshots to drop
(I wanted to also address Eric’s idea of letting QCowSnapshot.extra_data_size be always >= sizeof(QCowSnapshotExtraData) and thus save the MAX(sizeof(extra), sn->extra_data_size) in qcow2_write_snapshots(), but that doesn’t really work, because then we have no way of knowing later whether the image is compliant and thus needs fixing or not. He gave me an R-b anyway, so I guess it’s fine. O:-))
This series now has R-bs from Eric on all patches. I’m only posting it because I felt a bit bad about just taking the series as-is and add the commit notes to 12 and 13 while applying it.
(Also, maybe there is someone who saw me have a bit of discussion with Eric and thus assumed I would definitely send a v3 that they could then review.)
Max Reitz (16):
include: Move endof() up from hw/virtio/virtio.h
qcow2: Use endof()
qcow2: Add Error ** to qcow2_read_snapshots()
qcow2: Keep unknown extra snapshot data
qcow2: Make qcow2_write_snapshots() public
qcow2: Put qcow2_upgrade() into its own function
qcow2: Write v3-compliant snapshot list on upgrade
qcow2: Separate qcow2_check_read_snapshot_table()
qcow2: Add qcow2_check_fix_snapshot_table()
qcow2: Fix broken snapshot table entries
qcow2: Keep track of the snapshot table length
qcow2: Fix overly long snapshot tables
qcow2: Repair snapshot table with too many entries
qcow2: Fix v3 snapshot table entry compliancy
iotests: Add peek_file* functions
iotests: Test qcow2's snapshot table handling
block/qcow2.h | 15 +- include/hw/virtio/virtio.h | 7 - include/qemu/compiler.h | 7 + block/qcow2-snapshot.c | 323 +++++++++++++++++++-- block/qcow2.c | 155 +++++++++-- hw/block/virtio-blk.c | 4 +- hw/net/virtio-net.c | 10 +- tests/qemu-iotests/261 | 523 +++++++++++++++++++++++++++++++++++ tests/qemu-iotests/261.out | 346 +++++++++++++++++++++++ tests/qemu-iotests/common.rc | 20 ++ tests/qemu-iotests/group | 1 + 11 files changed, 1354 insertions(+), 57 deletions(-) create mode 100755 tests/qemu-iotests/261 create mode 100644 tests/qemu-iotests/261.out
Signature · 1 line
2.21.0
PATCH v3 1/16 include: Move endof() up from hw/virtio/virtio.h
endof() is a useful macro, we can make use of it outside of virtio.
--- include/hw/virtio/virtio.h | 7 ------- include/qemu/compiler.h | 7 +++++++ hw/block/virtio-blk.c | 4 ++-- hw/net/virtio-net.c | 10 +++++----- 4 files changed, 14 insertions(+), 14 deletions(-)
include/hw/virtio/virtio.h +0 -7
@@ -37,13 +37,6 @@ static inline hwaddr vring_align(hwaddr addr,return QEMU_ALIGN_UP (addr, align); } -/* -* Calculate the number of bytes up to and including the given 'field' of -* 'container'. -*/ -#define virtio_endof (container, field) \ - (offsetof(container, field) + sizeof_field(container, field)) - typedef struct VirtIOFeature { uint64_t flags; size_t end;
include/qemu/compiler.h +7 -0
@@ -60,6 +60,13 @@#define sizeof_field (type, field) sizeof(((type *)0)->field) +/* +* Calculate the number of bytes up to and including the given 'field' of +* 'container'. +*/ +#define endof (container, field) \ + (offsetof(container, field) + sizeof_field(container, field)) +/* Convert from a base type to a parent type, with compile time checking. */ #ifdef __GNUC__#define DO_UPCAST (type, field, dev) ( __extension__ ( { \
hw/block/virtio-blk.c +2 -2
@@ -42,9 +42,9 @@*/ static VirtIOFeature feature_sizes[] = { {.flags =1ULL <<VIRTIO_BLK_F_DISCARD , - .end = virtio_endof(struct virtio_blk_config, discard_sector_alignment)}, + .end = endof(struct virtio_blk_config, discard_sector_alignment)}, {.flags =1ULL <<VIRTIO_BLK_F_WRITE_ZEROES , - .end = virtio_endof(struct virtio_blk_config, write_zeroes_may_unmap)}, + .end = endof(struct virtio_blk_config, write_zeroes_may_unmap)}, {} };
hw/net/virtio-net.c +5 -5
@@ -90,15 +90,15 @@ static inline __virtio16 *virtio_net_rsc_ext_num_dupacks(static VirtIOFeature feature_sizes[] = { {.flags =1ULL <<VIRTIO_NET_F_MAC , - .end = virtio_endof(struct virtio_net_config, mac)}, + .end = endof(struct virtio_net_config, mac)}, {.flags =1ULL <<VIRTIO_NET_F_STATUS , - .end = virtio_endof(struct virtio_net_config, status)}, + .end = endof(struct virtio_net_config, status)}, {.flags =1ULL <<VIRTIO_NET_F_MQ , - .end = virtio_endof(struct virtio_net_config, max_virtqueue_pairs)}, + .end = endof(struct virtio_net_config, max_virtqueue_pairs)}, {.flags =1ULL <<VIRTIO_NET_F_MTU , - .end = virtio_endof(struct virtio_net_config, mtu)}, + .end = endof(struct virtio_net_config, mtu)}, {.flags =1ULL <<VIRTIO_NET_F_SPEED_DUPLEX , - .end = virtio_endof(struct virtio_net_config, duplex)}, + .end = endof(struct virtio_net_config, duplex)}, {} };
Signature · 1 line
2.21.0
PATCH v3 2/16 qcow2: Use endof()
In reply to Max Reitz--- block/qcow2-snapshot.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
block/qcow2-snapshot.c +4 -3
@@ -92,11 +92,12 @@ int qcow2_read_snapshots(BlockDriverState *bs) } offset += extra_data_size; -if (extra_data_size >=8 ) { +if (extra_data_size >=endof (QCowSnapshotExtraData, + vm_state_size_large)) { sn->vm_state_size =be64_to_cpu (extra.vm_state_size_large); } -if (extra_data_size >=16 ) { +if (extra_data_size >=endof (QCowSnapshotExtraData, disk_size)) { sn->disk_size =be64_to_cpu (extra.disk_size); }else { sn->disk_size = bs->total_sectors *BDRV_SECTOR_SIZE ;
@@ -251,7 +252,7 @@ static int qcow2_write_snapshots(BlockDriverState *bs) }QEMU_BUILD_BUG_ON (offsetof(QCowHeader , snapshots_offset) != - offsetof(QCowHeader , nb_snapshots) +sizeof (header_data.nb_snapshots)); +endof (QCowHeader, nb_snapshots)); header_data.nb_snapshots =cpu_to_be32 (s->nb_snapshots); header_data.snapshots_offset =cpu_to_be64 (snapshots_offset);
Signature · 1 line
2.21.0
PATCH v3 3/16 qcow2: Add Error ** to qcow2_read_snapshots()
In reply to Max Reitz--- block/qcow2.h | 2 +- block/qcow2-snapshot.c | 7 ++++++- block/qcow2.c | 3 +-- 3 files changed, 8 insertions(+), 4 deletions(-)
block/qcow2.h +1 -1
@@ -708,7 +708,7 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs, Error **errp);void qcow2_free_snapshots (BlockDriverState *bs); -int qcow2_read_snapshots (BlockDriverState *bs); +int qcow2_read_snapshots (BlockDriverState *bs,Error **errp);/* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables,
block/qcow2-snapshot.c +6 -1
@@ -43,7 +43,7 @@ void qcow2_free_snapshots(BlockDriverState *bs) s->nb_snapshots =0 ; } -int qcow2_read_snapshots (BlockDriverState *bs) +int qcow2_read_snapshots (BlockDriverState *bs,Error **errp) {BDRVQcow2State *s = bs->opaque;QCowSnapshotHeader h;
@@ -68,6 +68,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) offset =ROUND_UP (offset,8 ); ret =bdrv_pread (bs->file, offset, &h,sizeof (h));if (ret <0 ) { +error_setg_errno (errp, -ret,"Failed to read snapshot table" ); goto fail; }
@@ -88,6 +89,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) ret =bdrv_pread (bs->file, offset, &extra,MIN (sizeof (extra), extra_data_size));if (ret <0 ) { +error_setg_errno (errp, -ret,"Failed to read snapshot table" ); goto fail; } offset += extra_data_size;
@@ -107,6 +109,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) sn->id_str =g_malloc (id_str_size +1 ); ret =bdrv_pread (bs->file, offset, sn->id_str, id_str_size);if (ret <0 ) { +error_setg_errno (errp, -ret,"Failed to read snapshot table" ); goto fail; } offset += id_str_size;
@@ -116,6 +119,7 @@ int qcow2_read_snapshots(BlockDriverState *bs) sn->name =g_malloc (name_size +1 ); ret =bdrv_pread (bs->file, offset, sn->name, name_size);if (ret <0 ) { +error_setg_errno (errp, -ret,"Failed to read snapshot table" ); goto fail; } offset += name_size;
@@ -123,6 +127,7 @@ int qcow2_read_snapshots(BlockDriverState *bs)if (offset - s->snapshots_offset >QCOW_MAX_SNAPSHOTS_SIZE ) { ret = -EFBIG ; +error_setg (errp,"Snapshot table is too big" ); goto fail; } }
block/qcow2.c +1 -2
@@ -1584,9 +1584,8 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, s->snapshots_offset = header.snapshots_offset; s->nb_snapshots = header.nb_snapshots; - ret =qcow2_read_snapshots (bs); + ret =qcow2_read_snapshots (bs, errp);if (ret <0 ) { -error_setg_errno (errp, -ret,"Could not read snapshots" ); goto fail; }
Signature · 1 line
2.21.0
PATCH v3 4/16 qcow2: Keep unknown extra snapshot data
In reply to Max ReitzThe qcow2 specification says to ignore unknown extra data fields in snapshot table entries. Currently, we discard it whenever we update the image, which is a bit different from "ignore".
This patch makes the qcow2 driver keep all unknown extra data fields when updating an image's snapshot table.
--- block/qcow2.h | 5 ++++ block/qcow2-snapshot.c | 61 +++++++++++++++++++++++++++++++++++------- 2 files changed, 56 insertions(+), 10 deletions(-)
block/qcow2.h +5 -0
@@ -61,6 +61,9 @@* space for snapshot names and IDs */ #define QCOW_MAX_SNAPSHOTS_SIZE (1024 * QCOW_MAX_SNAPSHOTS) +/* Maximum amount of extra data per snapshot table entry to accept */ +#define QCOW_MAX_SNAPSHOT_EXTRA_DATA 1024 +/* Bitmap header extension constraints */ #define QCOW2_MAX_BITMAPS 65535#define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS)
@@ -181,6 +184,8 @@ typedef struct QCowSnapshot {uint32_t date_sec;uint32_t date_nsec;uint64_t vm_clock_nsec; +uint32_t extra_data_size; +void *unknown_extra_data;/* Extra data past QCowSnapshotExtraData */ } QCowSnapshot;struct Qcow2Cache ;
block/qcow2-snapshot.c +51 -10
127 lines not on this page · raw message
Signature · 1 line
2.21.0
Re: [PATCH v3 04/16] qcow2: Keep unknown extra snapshot data
Quoted 15 lines · Max Reitz
The qcow2 specification says to ignore unknown extra data fields in snapshot table entries. Currently, we discard it whenever we update the image, which is a bit different from "ignore".
This patch makes the qcow2 driver keep all unknown extra data fields when updating an image's snapshot table.
Signed-off-by Max Reitz <[email protected]>Reviewed-by Eric Blake <[email protected]>---
/* Bitmap header extension constraints */ #define QCOW2_MAX_BITMAPS 65535 #define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS)
block/qcow2.h typedef struct QCowSnapshot { · line 188
Quoted 5 lines
block/qcow2.h +2
-0@@ -181,6 +184,8 @@ typedef struct QCowSnapshot {uint32_t date_sec;uint32_t date_nsec;uint64_t vm_clock_nsec; +uint32_t extra_data_size; +void *unknown_extra_data;/* Extra data past QCowSnapshotExtraData */
Would it be worth a comment change:
uint32_t extra_data_size; /* Size of all extra data, including QCowSnapshotExtraData */ void *unknown_extra_data; /* Data beyond QCowSnapshotExtraData, if any */
Either way, R-b stands.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v3 04/16] qcow2: Keep unknown extra snapshot data
Quoted 25 lines · Eric Blake
Quoted 20 lines · Max Reitz
The qcow2 specification says to ignore unknown extra data fields in snapshot table entries. Currently, we discard it whenever we update the image, which is a bit different from "ignore".
This patch makes the qcow2 driver keep all unknown extra data fields when updating an image's snapshot table.
Signed-off-by Max Reitz <[email protected]>Reviewed-by Eric Blake <[email protected]>---
/* Bitmap header extension constraints */
#defineQCOW2_MAX_BITMAPS65535
#defineQCOW2_MAX_BITMAP_DIRECTORY_SIZE(1024 *QCOW2_MAX_BITMAPS)block/qcow2.h +2
-0@@ -181,6 +184,8 @@ typedef struct QCowSnapshot { uint32_t date_sec; uint32_t date_nsec; uint64_t vm_clock_nsec; + uint32_t extra_data_size; + void *unknown_extra_data; /* Extra data pastQuoted 1 line
QCowSnapshotExtraData */
Would it be worth a comment change:
Quoted 5 lines
uint32_textra_data_size; /* Size of all extra data, including QCowSnapshotExtraData */
void*unknown_extra_data; /* Data beyond QCowSnapshotExtraData, if any */Either way, R-b stands.
Can’t hurt. Well, except that extra_data_size may or may not include QCowSnapshotExtraData, because maybe it isn’t fully present in the image.
Max
PATCH v3 5/16 qcow2: Make qcow2_write_snapshots() public
In reply to Max ReitzUpdating the snapshot list will be useful when upgrading a v2 image to v3, so we will need to call this function in qcow2.c.
--- block/qcow2.h | 1 + block/qcow2-snapshot.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
block/qcow2.h +1 -0
@@ -714,6 +714,7 @@ int qcow2_snapshot_load_tmp(BlockDriverState *bs,void qcow2_free_snapshots (BlockDriverState *bs);int qcow2_read_snapshots (BlockDriverState *bs,Error **errp); +int qcow2_write_snapshots (BlockDriverState *bs);/* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables,
block/qcow2-snapshot.c +1 -1
@@ -164,7 +164,7 @@ fail: }/* add at the end of the file a new list of snapshots */ -static int qcow2_write_snapshots (BlockDriverState *bs) +int qcow2_write_snapshots (BlockDriverState *bs) {BDRVQcow2State *s = bs->opaque;QCowSnapshot *sn;
Signature · 1 line
2.21.0
PATCH v3 6/16 qcow2: Put qcow2_upgrade() into its own function
In reply to Max ReitzThis does not make sense right now, but it will make sense once we need to do more than to just update s->qcow_version.
--- block/qcow2.c | 43 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 5 deletions(-)
block/qcow2.c +38 -5
@@ -4897,12 +4897,46 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,return 0 ; } +/* +* Upgrades an image's version. While newer versions encompass all +* features of older versions, some things may have to be presented +* differently. +*/ +static int qcow2_upgrade (BlockDriverState *bs,int target_version, +BlockDriverAmendStatusCB *status_cb,void *cb_opaque, +Error **errp) +{ +BDRVQcow2State *s = bs->opaque; +int current_version = s->qcow_version; +int ret; + +/* This is qcow2_upgrade(), not qcow2_downgrade() */ +assert (target_version > current_version); + +/* There are no other versions (yet) that you can upgrade to */ +assert (target_version ==3 ); + +status_cb (bs,0 ,1 , cb_opaque); + + s->qcow_version = target_version; + ret =qcow2_update_header (bs); +if (ret <0 ) { + s->qcow_version = current_version; +error_setg_errno (errp, -ret,"Failed to update the image header" ); +return ret; + } +status_cb (bs,1 ,1 , cb_opaque); + +return 0 ; +} +typedef enum Qcow2AmendOperation {/* This is the value Qcow2AmendHelperCBInfo::last_operation will be * statically initialized to so that the helper CB can discern the first * invocation from an operation change */ QCOW2_NO_OPERATION =0 , +QCOW2_UPGRADING ,QCOW2_CHANGING_REFCOUNT_ORDER ,QCOW2_DOWNGRADING , }Qcow2AmendOperation ;
@@ -5085,17 +5119,16 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, helper_cb_info = (Qcow2AmendHelperCBInfo ){ .original_status_cb = status_cb, .original_cb_opaque = cb_opaque, - .total_operations = (new_version < old_version) + .total_operations = (new_version != old_version) + (s->refcount_bits != refcount_bits) };/* Upgrade first (some features may require compat=1.1) */ if (new_version > old_version) { - s->qcow_version = new_version; - ret =qcow2_update_header (bs); + helper_cb_info.current_operation =QCOW2_UPGRADING ; + ret =qcow2_upgrade (bs, new_version, &qcow2_amend_helper_cb, + &helper_cb_info, errp);if (ret <0 ) { - s->qcow_version = old_version; -error_setg_errno (errp, -ret,"Failed to update the image header" );return ret; } }
Signature · 1 line
2.21.0
PATCH v3 7/16 qcow2: Write v3-compliant snapshot list on upgrade
In reply to Max Reitzqcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present.
qcow2_upgrade() therefore should update the snapshot table to ensure all entries have these extra data fields.
--- block/qcow2.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-)
block/qcow2.c +30 -2
@@ -4907,7 +4907,9 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version, Error **errp) {BDRVQcow2State *s = bs->opaque; +bool need_snapshot_update;int current_version = s->qcow_version; +int i;int ret;/* This is qcow2_upgrade(), not qcow2_downgrade() */
@@ -4916,7 +4918,33 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version,/* There are no other versions (yet) that you can upgrade to */ assert (target_version ==3 ); -status_cb (bs,0 ,1 , cb_opaque); +status_cb (bs,0 ,2 , cb_opaque); + +/* +* In v2, snapshots do not need to have extra data. v3 requires +* the 64-bit VM state size and the virtual disk size to be +* present. +* qcow2_write_snapshots() will always write the list in the +* v3-compliant format. +*/ + need_snapshot_update = false; +for (i =0 ; i < s->nb_snapshots; i++) { +if (s->snapshots[i].extra_data_size < +sizeof_field (QCowSnapshotExtraData, vm_state_size_large) + +sizeof_field (QCowSnapshotExtraData, disk_size)) + { + need_snapshot_update = true; +break ; + } + } +if (need_snapshot_update) { + ret =qcow2_write_snapshots (bs); +if (ret <0 ) { +error_setg_errno (errp, -ret,"Failed to update the snapshot table" ); +return ret; + } + } +status_cb (bs,1 ,2 , cb_opaque); s->qcow_version = target_version; ret =qcow2_update_header (bs);
@@ -4925,7 +4953,7 @@ static int qcow2_upgrade(BlockDriverState *bs, int target_version,error_setg_errno (errp, -ret,"Failed to update the image header" );return ret; } -status_cb (bs,1 ,1 , cb_opaque); +status_cb (bs,2 ,2 , cb_opaque);return 0 ; }
Signature · 1 line
2.21.0
Re: [PATCH v3 07/16] qcow2: Write v3-compliant snapshot list on upgrade
Quoted 16 lines · Max Reitz
qcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present.
qcow2_upgrade()therefore should update the snapshot table to ensure all entries have these extra data fields.Reported-by Eric Blake <[email protected]>Signed-off-by Max Reitz <[email protected]>Reviewed-by Eric Blake <[email protected]>--- block/qcow2.c | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-)
block/qcow2.c static int qcow2_upgrade(BlockDriverState *bs, int target_version, · line 4934
Quoted 5 lines
block/qcow2.c +5
-0+ need_snapshot_update = false; +for (i =0 ; i < s->nb_snapshots; i++) { +if (s->snapshots[i].extra_data_size < +sizeof_field (QCowSnapshotExtraData, vm_state_size_large) + +sizeof_field (QCowSnapshotExtraData, disk_size))
Shorter as:
if (s->snapshots[i].extra_data_size < sizeof(QCowSnapshotExtraData))
but that's stylistic, so R-b still stands.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v3 07/16] qcow2: Write v3-compliant snapshot list on upgrade
Quoted 23 lines · Eric Blake
Quoted 13 lines · Max Reitz
qcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size. Both are optional for v2 images, so they may not be present.
qcow2_upgrade()therefore should update the snapshot table to ensure all entries have these extra data fields.Reported-by Eric Blake <[email protected]>Signed-off-by Max Reitz <[email protected]>Reviewed-by Eric Blake <[email protected]>---
block/qcow2.c| 32 ++++++++++++++++++++++++++++++--Quoted 8 lines
1 file changed, 30 insertions(+), 2 deletions(-)+
need_snapshot_update= false;
+ for (i = 0; i <s->nb_snapshots; i++) {
+ if (s->snapshots[i].extra_data_size <
+sizeof_field(QCowSnapshotExtraData, vm_state_size_large)+ +sizeof_field(QCowSnapshotExtraData, disk_size))
Quoted 4 lines
Shorter as:
if (s->snapshots[i].extra_data_size < sizeof(QCowSnapshotExtraData))but that's stylistic, so R-b still stands.
Yes, but if we ever add fields to QCowSnapshotExtraData, we shouldn’t count them here. Therefore, I think we need to count exactly the fields that the standard says are mandatory in v3.
Max
Re: [PATCH v3 07/16] qcow2: Write v3-compliant snapshot list on upgrade
Quoted 6 lines · Max Reitz
Quoted 6 lines
Quoted 5 lines
+
need_snapshot_update= false;
+ for (i = 0; i <s->nb_snapshots; i++) {
+ if (s->snapshots[i].extra_data_size <
+sizeof_field(QCowSnapshotExtraData, vm_state_size_large)+ +sizeof_field(QCowSnapshotExtraData, disk_size))
Quoted 8 lines
Quoted 4 lines
Shorter as:
if (s->snapshots[i].extra_data_size < sizeof(QCowSnapshotExtraData))but that's stylistic, so R-b still stands.
Yes, but if we ever add fields to QCowSnapshotExtraData, we shouldn’t count them here. Therefore, I think we need to count exactly the fields that the standard says are mandatory in v3.
If we ever add more fields, I'd prefer that we did something like:
struct QCowSnapshotExtraV3Minimum {
uint64_t vm_state_size_large;
uint64_t disk_size;
};
struct QCow3SnapshotExtraFull {
struct QCowSnapshotExtraV3Minimum base;
new fields...;
};and use sane naming to get at extra members based on the expected types, rather than trying to piecemeal portions of a type based on size.
Until we actually DO add more fields, why do we have to complicate the current code?
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH v3 07/16] qcow2: Write v3-compliant snapshot list on upgrade
Quoted 28 lines · Eric Blake
Quoted 6 lines · Max Reitz
Quoted 6 lines
Quoted 5 lines
+
need_snapshot_update= false;
+ for (i = 0; i <s->nb_snapshots; i++) {
+ if (s->snapshots[i].extra_data_size <
+sizeof_field(QCowSnapshotExtraData, vm_state_size_large)+ +sizeof_field(QCowSnapshotExtraData, disk_size))Quoted 8 lines
Quoted 4 lines
Shorter as:
if (s->snapshots[i].extra_data_size < sizeof(QCowSnapshotExtraData))but that's stylistic, so R-b still stands.
Yes, but if we ever add fields to QCowSnapshotExtraData, we shouldn’t count them here. Therefore, I think we need to count exactly the fields that the standard says are mandatory in v3.
If we ever add more fields, I'd prefer that we did something like:
struct QCowSnapshotExtraV3Minimum {
uint64_tvm_state_size_large;
uint64_tdisk_size;
};
struct QCow3SnapshotExtraFull {
struct QCowSnapshotExtraV3Minimum base;
new fields...;
};
Quoted 5 lines
and use sane naming to get at extra members based on the expected types, rather than trying to piecemeal portions of a type based on size.
Until we actually DO add more fields, why do we have to complicate the current code?
I don’t think it’s complicated, I find it very expressive. There are two fields, we check whether they are present; why, that’s obvious, because those are the ones mandated by the standard.
If we just checked against sizeof(QCowSnapshotExtraData), I’d (as a naïve reader) ask myself what that has to do with the standard. I’d need to look into the structure definition and see that it currently contains exactly the fields that are mandated by the standard, and then I’d think “But what if we ever add fields to this structure?” The more verbose version avoids this problem.
And I’m not really inclined to take your proposal above right now, because that would mean having to touch a lot of code. I prefer this more verbose code over that.
Also, you explicitly agreed that the code in this patch is preferable to a plain sizeof(extra) in v2:
https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00942.html
Max
PATCH v3 8/16 qcow2: Separate qcow2_check_read_snapshot_table()
In reply to Max ReitzReading the snapshot table can fail. That is a problem when we want to repair the image.
Therefore, stop reading the snapshot table in qcow2_do_open() in check mode. Instead, add a new function qcow2_check_read_snapshot_table() that reads the snapshot table at a later point. In the future, we want to handle errors here and fix them.
--- block/qcow2.h | 4 +++ block/qcow2-snapshot.c | 58 ++++++++++++++++++++++++++++++++ block/qcow2.c | 76 ++++++++++++++++++++++++++++++++---------- 3 files changed, 120 insertions(+), 18 deletions(-)
block/qcow2.h +4 -0
@@ -716,6 +716,10 @@ void qcow2_free_snapshots(BlockDriverState *bs);int qcow2_read_snapshots (BlockDriverState *bs,Error **errp);int qcow2_write_snapshots (BlockDriverState *bs); +int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, +BdrvCheckResult *result, +BdrvCheckMode fix); +/* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create (BlockDriverState *bs,int num_tables,unsigned table_size);
block/qcow2-snapshot.c +58 -0
64 lines not on this page · raw message
block/qcow2.c +58 -18
99 lines not on this page · raw message
Signature · 1 line
2.21.0
PATCH v3 9/16 qcow2: Add qcow2_check_fix_snapshot_table()
In reply to Max Reitzqcow2_check_read_snapshot_table() can perform consistency checks, but it cannot fix everything. Specifically, it cannot allocate new clusters, because that should wait until the refcount structures are known to be consistent (i.e., after qcow2_check_refcounts()). Thus, it cannot call qcow2_write_snapshots().
Do that in qcow2_check_fix_snapshot_table(), which is called after qcow2_check_refcounts().
Currently, there is nothing that would set result->corruptions, so this is a no-op. A follow-up patch will change that.
--- block/qcow2.h | 3 +++ block/qcow2-snapshot.c | 25 +++++++++++++++++++++++++ block/qcow2.c | 9 ++++++++- 3 files changed, 36 insertions(+), 1 deletion(-)
block/qcow2.h +3 -0
@@ -719,6 +719,9 @@ int qcow2_write_snapshots(BlockDriverState *bs); int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs,BdrvCheckResult *result,BdrvCheckMode fix); +int coroutine_fn qcow2_check_fix_snapshot_table(BlockDriverState *bs, +BdrvCheckResult *result, +BdrvCheckMode fix);/* qcow2-cache.c functions */ Qcow2Cache *qcow2_cache_create(BlockDriverState *bs, int num_tables,
block/qcow2-snapshot.c +25 -0
@@ -380,6 +380,31 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs,return 0 ; } +int coroutine_fn qcow2_check_fix_snapshot_table(BlockDriverState *bs, +BdrvCheckResult *result, +BdrvCheckMode fix) +{ +BDRVQcow2State *s = bs->opaque; +int ret; + +if (result->corruptions && (fix &BDRV_FIX_ERRORS )) { +qemu_co_mutex_unlock (&s->lock); + ret =qcow2_write_snapshots (bs); +qemu_co_mutex_lock (&s->lock); +if (ret <0 ) { + result->check_errors++; +fprintf (stderr,"ERROR failed to update snapshot table: %s\n" , +strerror (-ret)); +return ret; + } + + result->corruptions_fixed += result->corruptions; + result->corruptions =0 ; + } + +return 0 ; +} +static void find_new_snapshot_id (BlockDriverState *bs,char *id_str,int id_str_size) {
block/qcow2.c +8 -1
@@ -597,13 +597,20 @@ static int coroutine_fn qcow2_co_check_locked(BlockDriverState *bs,memset (result,0 ,sizeof (*result)); ret =qcow2_check_read_snapshot_table (bs, &snapshot_res, fix); -qcow2_add_check_result (result, &snapshot_res, false);if (ret <0 ) { +qcow2_add_check_result (result, &snapshot_res, false);return ret; } ret =qcow2_check_refcounts (bs, &refcount_res, fix);qcow2_add_check_result (result, &refcount_res, true); +if (ret <0 ) { +qcow2_add_check_result (result, &snapshot_res, false); +return ret; + } + + ret =qcow2_check_fix_snapshot_table (bs, &snapshot_res, fix); +qcow2_add_check_result (result, &snapshot_res, false);if (ret <0 ) {return ret; }
Signature · 1 line
2.21.0
PATCH v3 10/16 qcow2: Fix broken snapshot table entries
In reply to Max ReitzThe only case where we currently reject snapshot table entries is when they have too much extra data. Fix them with qemu-img check -r all by counting it as a corruption, reducing their extra_data_size, and then letting qcow2_check_fix_snapshot_table() do the rest.
--- block/qcow2-snapshot.c | 67 +++++++++++++++++++++++++++++++++++------- 1 file changed, 56 insertions(+), 11 deletions(-)
block/qcow2-snapshot.c +56 -11
@@ -44,7 +44,23 @@ void qcow2_free_snapshots(BlockDriverState *bs) s->nb_snapshots =0 ; } -int qcow2_read_snapshots (BlockDriverState *bs,Error **errp) +/* +* If @repair is true, try to repair a broken snapshot table instead +* of just returning an error: +* +* - If there were snapshots with too much extra metadata, increment +* *extra_data_dropped for each. +* This requires the caller to eventually rewrite the whole snapshot +* table, which requires cluster allocation. Therefore, this should +* be done only after qcow2_check_refcounts() made sure the refcount +* structures are valid. +* (In the meantime, the image is still valid because +* qcow2_check_refcounts() does not do anything with snapshots' +* extra data.) +*/ +static int qcow2_do_read_snapshots (BlockDriverState *bs,bool repair, +int *extra_data_dropped, +Error **errp) {BDRVQcow2State *s = bs->opaque;QCowSnapshotHeader h;
@@ -64,6 +80,8 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) s->snapshots =g_new0 (QCowSnapshot, s->nb_snapshots);for (i =0 ; i < s->nb_snapshots; i++) { +bool truncate_unknown_extra_data = false; +/* Read statically sized part of the snapshot header */ offset =ROUND_UP (offset,8 ); ret =bdrv_pread (bs->file, offset, &h,sizeof (h));
@@ -86,10 +104,21 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) name_size =be16_to_cpu (h.name_size);if (sn->extra_data_size >QCOW_MAX_SNAPSHOT_EXTRA_DATA ) { - ret = -EFBIG ; -error_setg (errp,"Too much extra metadata in snapshot table " -"entry %i" , i); - goto fail; +if (!repair) { + ret = -EFBIG ; +error_setg (errp,"Too much extra metadata in snapshot table " +"entry %i" , i); +error_append_hint (errp,"You can force-remove this extra " +"metadata with qemu-img check -r all\n" ); + goto fail; + } + +fprintf (stderr,"Discarding too much extra metadata in snapshot " +"table entry %i (%" PRIu32" > %u)\n" , + i, sn->extra_data_size,QCOW_MAX_SNAPSHOT_EXTRA_DATA ); + + (*extra_data_dropped)++; + truncate_unknown_extra_data = true; }/* Read known extra data */
@@ -113,18 +142,26 @@ int qcow2_read_snapshots(BlockDriverState *bs, Error **errp) }if (sn->extra_data_size >sizeof (extra)) { -/* Store unknown extra data */ -size_t unknown_extra_data_size = - sn->extra_data_size -sizeof (extra); +uint64_t extra_data_end; +size_t unknown_extra_data_size; + + extra_data_end = offset + sn->extra_data_size -sizeof (extra); +if (truncate_unknown_extra_data) { + sn->extra_data_size =QCOW_MAX_SNAPSHOT_EXTRA_DATA ; + } + +/* Store unknown extra data */ + unknown_extra_data_size = sn->extra_data_size -sizeof (extra); sn->unknown_extra_data =g_malloc (unknown_extra_data_size); ret =bdrv_pread (bs->file, offset, sn->unknown_extra_data, unknown_extra_data_size);if (ret <0 ) { -error_setg_errno (errp, -ret,"Failed to read snapshot table" ); +error_setg_errno (errp, -ret, +"Failed to read snapshot table" ); goto fail; } - offset += unknown_extra_data_size; + offset = extra_data_end; }/* Read snapshot ID */
@@ -163,6 +200,11 @@ fail:return ret; } +int qcow2_read_snapshots (BlockDriverState *bs,Error **errp) +{ +return qcow2_do_read_snapshots (bs, false,NULL , errp); +} +/* add at the end of the file a new list of snapshots */ int qcow2_write_snapshots (BlockDriverState *bs) {
@@ -328,6 +370,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, {BDRVQcow2State *s = bs->opaque;Error *local_err =NULL ; +int extra_data_dropped =0 ;int ret;struct {uint32_t nb_snapshots;
@@ -363,7 +406,8 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, }qemu_co_mutex_unlock (&s->lock); - ret =qcow2_read_snapshots (bs, &local_err); + ret =qcow2_do_read_snapshots (bs, fix &BDRV_FIX_ERRORS , + &extra_data_dropped, &local_err);qemu_co_mutex_lock (&s->lock);if (ret <0 ) { result->check_errors++;
@@ -376,6 +420,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs,return ret; } + result->corruptions += extra_data_dropped;return 0 ; }
Signature · 1 line
2.21.0
PATCH v3 11/16 qcow2: Keep track of the snapshot table length
In reply to Max ReitzWhen repairing the snapshot table, we truncate entries that have too much extra data. This frees up space that we do not have to count towards the snapshot table size.
--- block/qcow2-snapshot.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
block/qcow2-snapshot.c +13 -1
@@ -68,6 +68,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair,QCowSnapshot *sn;int i, id_str_size, name_size;int64_t offset; +uint64_t table_length =0 ;int ret;if (!s->nb_snapshots) {
@@ -82,6 +83,8 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair,for (i =0 ; i < s->nb_snapshots; i++) {bool truncate_unknown_extra_data = false; + table_length =ROUND_UP (table_length,8 ); +/* Read statically sized part of the snapshot header */ offset =ROUND_UP (offset,8 ); ret =bdrv_pread (bs->file, offset, &h,sizeof (h));
@@ -184,7 +187,16 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair, offset += name_size; sn->name[name_size] ='\0' ; -if (offset - s->snapshots_offset >QCOW_MAX_SNAPSHOTS_SIZE ) { +/* Note that the extra data may have been truncated */ + table_length +=sizeof (h) + sn->extra_data_size + id_str_size + + name_size; +if (!repair) { +assert (table_length == offset - s->snapshots_offset); + } + +if (table_length >QCOW_MAX_SNAPSHOTS_SIZE || + offset - s->snapshots_offset >INT_MAX ) + { ret = -EFBIG ;error_setg (errp,"Snapshot table is too big" ); goto fail;
Signature · 1 line
2.21.0
PATCH v3 12/16 qcow2: Fix overly long snapshot tables
In reply to Max ReitzWe currently refuse to open qcow2 images with overly long snapshot tables. This patch makes qemu-img check -r all drop all offending entries past what we deem acceptable.
The user cannot choose which snapshots are removed. This is fine because we have chosen the maximum snapshot table size to be so large (64 MB) that it cannot be reasonably reached. If the snapshot table exceeds this size, the image has probably been corrupted in some way; in this case, it is most important to just make the image usable such that the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a plain "check" or "check -r leaks" will not delete any data.)
--- block/qcow2-snapshot.c | 88 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 78 insertions(+), 10 deletions(-)
block/qcow2-snapshot.c +78 -10
@@ -29,15 +29,24 @@#include "qemu/error-report.h" #include "qemu/cutils.h" +static void qcow2_free_single_snapshot (BlockDriverState *bs,int i) +{ +BDRVQcow2State *s = bs->opaque; + +assert (i >=0 && i < s->nb_snapshots); +g_free (s->snapshots[i].name); +g_free (s->snapshots[i].id_str); +g_free (s->snapshots[i].unknown_extra_data); +memset (&s->snapshots[i],0 ,sizeof (s->snapshots[i])); +} +void qcow2_free_snapshots (BlockDriverState *bs) {BDRVQcow2State *s = bs->opaque;int i;for (i =0 ; i < s->nb_snapshots; i++) { -g_free (s->snapshots[i].name); -g_free (s->snapshots[i].id_str); -g_free (s->snapshots[i].unknown_extra_data); +qcow2_free_single_snapshot (bs, i); }g_free (s->snapshots); s->snapshots =NULL ;
@@ -48,6 +57,14 @@ void qcow2_free_snapshots(BlockDriverState *bs)* If @repair is true, try to repair a broken snapshot table instead * of just returning an error: * +* - If the snapshot table was too long, set *nb_clusters_reduced to +* the number of snapshots removed off the end. +* The caller will update the on-disk nb_snapshots accordingly; +* this leaks clusters, but is safe. +* (The on-disk information must be updated before +* qcow2_check_refcounts(), because that function relies on +* s->nb_snapshots to reflect the on-disk value.) +* * - If there were snapshots with too much extra metadata, increment * *extra_data_dropped for each. * This requires the caller to eventually rewrite the whole snapshot
@@ -59,6 +76,7 @@ void qcow2_free_snapshots(BlockDriverState *bs)* extra data.) */ static int qcow2_do_read_snapshots (BlockDriverState *bs,bool repair, +int *nb_clusters_reduced,int *extra_data_dropped,Error **errp) {
@@ -67,7 +85,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair,QCowSnapshotExtraData extra;QCowSnapshot *sn;int i, id_str_size, name_size; -int64_t offset; +int64_t offset, pre_sn_offset;uint64_t table_length =0 ;int ret;
@@ -83,6 +101,7 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair,for (i =0 ; i < s->nb_snapshots; i++) {bool truncate_unknown_extra_data = false; + pre_sn_offset = offset; table_length =ROUND_UP (table_length,8 );/* Read statically sized part of the snapshot header */
@@ -197,9 +216,31 @@ static int qcow2_do_read_snapshots(BlockDriverState *bs, bool repair,if (table_length >QCOW_MAX_SNAPSHOTS_SIZE || offset - s->snapshots_offset >INT_MAX ) { - ret = -EFBIG ; -error_setg (errp,"Snapshot table is too big" ); - goto fail; +if (!repair) { + ret = -EFBIG ; +error_setg (errp,"Snapshot table is too big" ); +error_append_hint (errp,"You can force-remove all %u " +"overhanging snapshots with qemu-img check " +"-r all\n" , s->nb_snapshots - i); + goto fail; + } + +fprintf (stderr,"Discarding %u overhanging snapshots (snapshot " +"table is too big)\n" , s->nb_snapshots - i); + + *nb_clusters_reduced += (s->nb_snapshots - i); + +/* Discard current snapshot also */ +qcow2_free_single_snapshot (bs, i); + +/* +* This leaks all the rest of the snapshot table and the +* snapshots' clusters, but we run in check -r all mode, +* so qcow2_check_refcounts() will take care of it. +*/ + s->nb_snapshots = i; + offset = pre_sn_offset; +break ; } }
@@ -214,7 +255,7 @@ fail:int qcow2_read_snapshots (BlockDriverState *bs,Error **errp) { -return qcow2_do_read_snapshots (bs, false,NULL , errp); +return qcow2_do_read_snapshots (bs, false,NULL ,NULL , errp); }/* add at the end of the file a new list of snapshots */
@@ -382,6 +423,7 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, {BDRVQcow2State *s = bs->opaque;Error *local_err =NULL ; +int nb_clusters_reduced =0 ;int extra_data_dropped =0 ;int ret;struct {
@@ -419,7 +461,8 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs,qemu_co_mutex_unlock (&s->lock); ret =qcow2_do_read_snapshots (bs, fix &BDRV_FIX_ERRORS , - &extra_data_dropped, &local_err); + &nb_clusters_reduced, &extra_data_dropped, + &local_err);qemu_co_mutex_lock (&s->lock);if (ret <0 ) { result->check_errors++;
@@ -432,7 +475,32 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs,return ret; } - result->corruptions += extra_data_dropped; + result->corruptions += nb_clusters_reduced + extra_data_dropped; + +if (nb_clusters_reduced) { +/* +* Update image header now, because: +* (1) qcow2_check_refcounts() relies on s->nb_snapshots to be +* the same as what the image header says, +* (2) this leaks clusters, but qcow2_check_refcounts() will +* fix that. +*/ +assert (fix &BDRV_FIX_ERRORS ); + + snapshot_table_pointer.nb_snapshots =cpu_to_be32 (s->nb_snapshots); + ret =bdrv_pwrite_sync (bs->file, offsetof(QCowHeader , nb_snapshots), + &snapshot_table_pointer.nb_snapshots, +sizeof (snapshot_table_pointer.nb_snapshots)); +if (ret <0 ) { + result->check_errors++; +fprintf (stderr,"ERROR failed to update the snapshot count in the " +"image header: %s\n" ,strerror (-ret)); +return ret; + } + + result->corruptions_fixed += nb_clusters_reduced; + result->corruptions -= nb_clusters_reduced; + }return 0 ; }
Signature · 1 line
2.21.0
PATCH v3 13/16 qcow2: Repair snapshot table with too many entries
In reply to Max ReitzThe user cannot choose which snapshots are removed. This is fine because we have chosen the maximum snapshot table size to be so large (65536 entries) that it cannot be reasonably reached. If the snapshot table exceeds this size, the image has probably been corrupted in some way; in this case, it is most important to just make the image usable such that the user can copy off at least the active layer. (Also note that the snapshots will be removed only with "-r all", so a plain "check" or "check -r leaks" will not delete any data.)
--- block/qcow2-snapshot.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
block/qcow2-snapshot.c +14 -0
@@ -444,6 +444,14 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, s->snapshots_offset =be64_to_cpu (snapshot_table_pointer.snapshots_offset); s->nb_snapshots =be32_to_cpu (snapshot_table_pointer.nb_snapshots); +if (s->nb_snapshots >QCOW_MAX_SNAPSHOTS && (fix &BDRV_FIX_ERRORS )) { +fprintf (stderr,"Discarding %u overhanging snapshots\n" , + s->nb_snapshots -QCOW_MAX_SNAPSHOTS ); + + nb_clusters_reduced += s->nb_snapshots -QCOW_MAX_SNAPSHOTS ; + s->nb_snapshots =QCOW_MAX_SNAPSHOTS ; + } + ret = qcow2_validate_table(bs, s->snapshots_offset, s->nb_snapshots, sizeof(QCowSnapshotHeader), sizeof(QCowSnapshotHeader) * QCOW_MAX_SNAPSHOTS,
@@ -452,6 +460,12 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->check_errors++;error_reportf_err (local_err,"ERROR " ); +if (s->nb_snapshots >QCOW_MAX_SNAPSHOTS ) { +fprintf (stderr,"You can force-remove all %u overhanging snapshots " +"with qemu-img check -r all\n" , + s->nb_snapshots -QCOW_MAX_SNAPSHOTS ); + } +/* We did not read the snapshot table, so invalidate this information */ s->snapshots_offset =0 ; s->nb_snapshots =0 ;
Signature · 1 line
2.21.0
Re: [PATCH v3 13/16] qcow2: Repair snapshot table with too many entries
Quoted 11 lines · Max Reitz
The user cannot choose which snapshots are removed. This is fine because we have chosen the maximum snapshot table size to be so large (65536 entries) that it cannot be reasonably reached. If the snapshot table exceeds this size, the image has probably been corrupted in some way; in this case, it is most important to just make the image usable such that the user can copy off at least the active layer. (Also note that the snapshots will be removed only with "-r all", so a plain "check" or "check -r leaks" will not delete any data.)
Signed-off-by Max Reitz <[email protected]>Reviewed-by Eric Blake <[email protected]>
The updated commit messages are good.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 14/16 qcow2: Fix v3 snapshot table entry compliancy
In reply to Max Reitzqcow2 v3 images require every snapshot table entry to have at least 16 bytes of extra data. If they do not, let qemu-img check -r all fix it.
--- block/qcow2-snapshot.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
block/qcow2-snapshot.c +18 -0
@@ -516,6 +516,24 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->corruptions -= nb_clusters_reduced; } +/* +* All of v3 images' snapshot table entries need to have at least +* 16 bytes of extra data. +*/ +if (s->qcow_version >=3 ) { +int i; +for (i =0 ; i < s->nb_snapshots; i++) { +if (s->snapshots[i].extra_data_size < +sizeof_field (QCowSnapshotExtraData, vm_state_size_large) + +sizeof_field (QCowSnapshotExtraData, disk_size)) + { + result->corruptions++; +fprintf (stderr,"%s snapshot table entry %i is incomplete\n" , + fix &BDRV_FIX_ERRORS ?"Repairing" :"ERROR" , i); + } + } + } +return 0 ; }
Signature · 1 line
2.21.0
Re: [PATCH v3 14/16] qcow2: Fix v3 snapshot table entry compliancy
Quoted 18 lines · Max Reitz
qcow2 v3 images require every snapshot table entry to have at least 16 bytes of extra data. If they do not, let qemu-img check -r all fix it.
Signed-off-by Max Reitz <[email protected]>Reviewed-by Eric Blake <[email protected]>--- block/qcow2-snapshot.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)block/qcow2-snapshot.c +1
-0@@ -516,6 +516,24 @@ int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, result->corruptions -= nb_clusters_reduced; } +/*
block/qcow2-snapshot.c int coroutine_fn qcow2_check_read_snapshot_table(BlockDriverState *bs, · line 529
Quoted 10 lines
block/qcow2-snapshot.c +10
-0+* All of v3 images' snapshot table entries need to have at least +* 16 bytes of extra data. +*/ +if (s->qcow_version >=3 ) { +int i; +for (i =0 ; i < s->nb_snapshots; i++) { +if (s->snapshots[i].extra_data_size < +sizeof_field (QCowSnapshotExtraData, vm_state_size_large) + +sizeof_field (QCowSnapshotExtraData, disk_size)) + {
Another stylistic place where sizeof(QCowSnapshotExtraData) would be more compact. Doesn't change R-b
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH v3 15/16 iotests: Add peek_file* functions
In reply to Max Reitz--- tests/qemu-iotests/common.rc | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
tests/qemu-iotests/common.rc +20 -0
@@ -53,6 +53,26 @@ poke_file() printf "$3" | dd "of=$1" bs=1 "seek=$2" conv=notrunc &>/dev/null } +# peek_file_le 'test.img' 512 2 => 65534 +peek_file_le() +{ + # Wrap in echo $() to strip spaces + echo $(od -j"$2" -N"$3" --endian=little -An -vtu"$3" "$1") +} + +# peek_file_be 'test.img' 512 2 => 65279 +peek_file_be() +{ + # Wrap in echo $() to strip spaces + echo $(od -j"$2" -N"$3" --endian=big -An -vtu"$3" "$1") +} + +# peek_file_raw 'test.img' 512 2 => '\xff\xfe' +peek_file_raw() +{ + dd if="$1" bs=1 skip="$2" count="$3" status=none +} + if ! . ./common.config then
Signature · 1 line
2.21.0
PATCH v3 16/16 iotests: Test qcow2's snapshot table handling
In reply to Max ReitzAdd a test how our qcow2 driver handles extra data in snapshot table entries, and how it repairs overly long snapshot tables.
--- tests/qemu-iotests/261 | 523 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/261.out | 346 ++++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 870 insertions(+) create mode 100755 tests/qemu-iotests/261 create mode 100644 tests/qemu-iotests/261.out
tests/qemu-iotests/261 new file +523 -0
@@ -0,0 +1,523 @@ +#!/usr/bin/env bash +# +# Test case for qcow2's handling of extra data in snapshot table entries +# (and more generally, how certain cases of broken snapshot tables are +# handled) +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator [email protected] + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f "$TEST_IMG".v{2,3}.orig + rm -f "$TEST_DIR"/sn{0,1,2}{,-pre,-extra,-post} +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# This tests qocw2-specific low-level functionality +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux +# (1) We create a v2 image that supports nothing but refcount_bits=16 +# (2) We do some refcount management on our own which expects +# refcount_bits=16 +_unsupported_imgopts 'refcount_bits=\([^1]\|.\([^6]\|$\)\)' + +# Parameters: +# $1: image filename +# $2: snapshot table entry offset in the image +snapshot_table_entry_size() +{ + id_len=$(peek_file_be "$1" $(($2 + 12)) 2) + name_len=$(peek_file_be "$1" $(($2 + 14)) 2) + extra_len=$(peek_file_be "$1" $(($2 + 36)) 4) + + full_len=$((40 + extra_len + id_len + name_len)) + echo $(((full_len + 7) / 8 * 8)) +} + +# Parameter: +# $1: image filename +print_snapshot_table() +{ + nb_entries=$(peek_file_be "$1" 60 4) + offset=$(peek_file_be "$1" 64 8) + + echo "Snapshots in $1:" | _filter_testdir | _filter_imgfmt + + for ((i = 0; i < nb_entries; i++)); do + id_len=$(peek_file_be "$1" $((offset + 12)) 2) + name_len=$(peek_file_be "$1" $((offset + 14)) 2) + extra_len=$(peek_file_be "$1" $((offset + 36)) 4) + + extra_ofs=$((offset + 40)) + id_ofs=$((extra_ofs + extra_len)) + name_ofs=$((id_ofs + id_len)) + + echo " [$i]" + echo " ID: $(peek_file_raw "$1" $id_ofs $id_len)" + echo " Name: $(peek_file_raw "$1" $name_ofs $name_len)" + echo " Extra data size: $extra_len" + if [ $extra_len -ge 8 ]; then + echo " VM state size: $(peek_file_be "$1" $extra_ofs 8)" + fi + if [ $extra_len -ge 16 ]; then + echo " Disk size: $(peek_file_be "$1" $((extra_ofs + 8)) 8)" + fi + if [ $extra_len -gt 16 ]; then + echo ' Unknown extra data:' \ + "$(peek_file_raw "$1" $((extra_ofs + 16)) $((extra_len - 16)) \ + | tr -d '\0')" + fi + + offset=$((offset + $(snapshot_table_entry_size "$1" $offset))) + done +} + +# Mark clusters as allocated; works only in refblock 0 (i.e. before +# cluster #32768). +# Parameters: +# $1: Start offset of what to allocate +# $2: End offset (exclusive) +refblock0_allocate() +{ + reftable_ofs=$(peek_file_be "$TEST_IMG" 48 8) + refblock_ofs=$(peek_file_be "$TEST_IMG" $reftable_ofs 8) + + cluster=$(($1 / 65536)) + ecluster=$((($2 + 65535) / 65536)) + + while [ $cluster -lt $ecluster ]; do + if [ $cluster -ge 32768 ]; then + echo "*** Abort: Cluster $cluster exceeds refblock 0 ***" + exit 1 + fi + poke_file "$TEST_IMG" $((refblock_ofs + cluster * 2)) '\x00\x01' + cluster=$((cluster + 1)) + done +} + + +echo +echo '=== Create v2 template ===' +echo + +# Create v2 image with a snapshot table with three entries: +# [0]: No extra data (valid with v2, not valid with v3) +# [1]: Has extra data unknown to qemu +# [2]: Has the 64-bit VM state size, but not the disk size (again, +# valid with v2, not valid with v3) + +TEST_IMG="$TEST_IMG.v2.orig" IMGOPTS='compat=0.10' _make_test_img 64M +$QEMU_IMG snapshot -c sn0 "$TEST_IMG.v2.orig" +$QEMU_IMG snapshot -c sn1 "$TEST_IMG.v2.orig" +$QEMU_IMG snapshot -c sn2 "$TEST_IMG.v2.orig" + +# Copy out all existing snapshot table entries +sn_table_ofs=$(peek_file_be "$TEST_IMG.v2.orig" 64 8) + +# ofs: Snapshot table entry offset +# eds: Extra data size +# ids: Name + ID size +# len: Total entry length +sn0_ofs=$sn_table_ofs +sn0_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 36)) 4) +sn0_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn0_ofs + 14)) 2))) +sn0_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn0_ofs) +sn1_ofs=$((sn0_ofs + sn0_len)) +sn1_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 36)) 4) +sn1_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn1_ofs + 14)) 2))) +sn1_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn1_ofs) +sn2_ofs=$((sn1_ofs + sn1_len)) +sn2_eds=$(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 36)) 4) +sn2_ids=$(($(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 12)) 2) + + $(peek_file_be "$TEST_IMG.v2.orig" $((sn2_ofs + 14)) 2))) +sn2_len=$(snapshot_table_entry_size "$TEST_IMG.v2.orig" $sn2_ofs) + +# Data before extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-pre" bs=1 skip=$sn0_ofs count=40 \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-pre" bs=1 skip=$sn1_ofs count=40 \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-pre" bs=1 skip=$sn2_ofs count=40 \ + &> /dev/null + +# Extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-extra" bs=1 \ + skip=$((sn0_ofs + 40)) count=$sn0_eds &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-extra" bs=1 \ + skip=$((sn1_ofs + 40)) count=$sn1_eds &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-extra" bs=1 \ + skip=$((sn2_ofs + 40)) count=$sn2_eds &> /dev/null + +# Data after extra data +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn0-post" bs=1 \ + skip=$((sn0_ofs + 40 + sn0_eds)) count=$sn0_ids \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn1-post" bs=1 \ + skip=$((sn1_ofs + 40 + sn1_eds)) count=$sn1_ids \ + &> /dev/null +dd if="$TEST_IMG.v2.orig" of="$TEST_DIR/sn2-post" bs=1 \ + skip=$((sn2_ofs + 40 + sn2_eds)) count=$sn2_ids \ + &> /dev/null + +# Amend them, one by one +# Set sn0's extra data size to 0 +poke_file "$TEST_DIR/sn0-pre" 36 '\x00\x00\x00\x00' +truncate -s 0 "$TEST_DIR/sn0-extra" +# Grow sn0-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn0-pre") - 40)) \ + "$TEST_DIR/sn0-post" + +# Set sn1's extra data size to 42 +poke_file "$TEST_DIR/sn1-pre" 36 '\x00\x00\x00\x2a' +truncate -s 42 "$TEST_DIR/sn1-extra" +poke_file "$TEST_DIR/sn1-extra" 16 'very important data' +# Grow sn1-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn1-pre") - 82)) \ + "$TEST_DIR/sn1-post" + +# Set sn2's extra data size to 8 +poke_file "$TEST_DIR/sn2-pre" 36 '\x00\x00\x00\x08' +truncate -s 8 "$TEST_DIR/sn2-extra" +# Grow sn2-post to pad +truncate -s $(($(snapshot_table_entry_size "$TEST_DIR/sn2-pre") - 48)) \ + "$TEST_DIR/sn2-post" + +# Construct snapshot table +cat "$TEST_DIR"/sn0-{pre,extra,post} \ + "$TEST_DIR"/sn1-{pre,extra,post} \ + "$TEST_DIR"/sn2-{pre,extra,post} \ + | dd of="$TEST_IMG.v2.orig" bs=1 seek=$sn_table_ofs conv=notrunc \ + &> /dev/null + +# Done! +TEST_IMG="$TEST_IMG.v2.orig" _check_test_img +print_snapshot_table "$TEST_IMG.v2.orig" + +echo +echo '=== Upgrade to v3 ===' +echo + +cp "$TEST_IMG.v2.orig" "$TEST_IMG.v3.orig" +$QEMU_IMG amend -o compat=1.1 "$TEST_IMG.v3.orig" +TEST_IMG="$TEST_IMG.v3.orig" _check_test_img +print_snapshot_table "$TEST_IMG.v3.orig" + +echo +echo '=== Repair botched v3 ===' +echo + +# Force the v2 file to be v3. v3 requires each snapshot table entry +# to have at least 16 bytes of extra data, so it will not comply to +# the qcow2 v3 specification; but we can fix that. +cp "$TEST_IMG.v2.orig" "$TEST_IMG" + +# Set version +poke_file "$TEST_IMG" 4 '\x00\x00\x00\x03' +# Increase header length (necessary for v3) +poke_file "$TEST_IMG" 100 '\x00\x00\x00\x68' +# Set refcount order (necessary for v3) +poke_file "$TEST_IMG" 96 '\x00\x00\x00\x04' + +_check_test_img -r all +print_snapshot_table "$TEST_IMG" + + +# From now on, just test the qcow2 version we are supposed to test. +# (v3 by default, v2 by choice through $IMGOPTS.) +# That works because we always write all known extra data when +# updating the snapshot table, independent of the version. + +if echo "$IMGOPTS" | grep -q 'compat=\(0\.10\|v2\)' 2> /dev/null; then + subver=v2 +else + subver=v3 +fi + +echo +echo '=== Add new snapshot ===' +echo + +cp "$TEST_IMG.$subver.orig" "$TEST_IMG" +$QEMU_IMG snapshot -c sn3 "$TEST_IMG" +_check_test_img +print_snapshot_table "$TEST_IMG" + +echo +echo '=== Remove different snapshots ===' + +for sn in sn0 sn1 sn2; do + echo + echo "--- $sn ---" + + cp "$TEST_IMG.$subver.orig" "$TEST_IMG" + $QEMU_IMG snapshot -d $sn "$TEST_IMG" + _check_test_img + print_snapshot_table "$TEST_IMG" +done + +echo +echo '=== Reject too much unknown extra data ===' +echo + +cp "$TEST_IMG.$subver.orig" "$TEST_IMG" +$QEMU_IMG snapshot -c sn3 "$TEST_IMG" + +sn_table_ofs=$(peek_file_be "$TEST_IMG" 64 8) +sn0_ofs=$sn_table_ofs +sn1_ofs=$((sn0_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn0_ofs))) +sn2_ofs=$((sn1_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn1_ofs))) +sn3_ofs=$((sn2_ofs + $(snapshot_table_entry_size "$TEST_IMG" $sn2_ofs))) + +# 64 kB of extra data should be rejected +# (Note that this also induces a refcount error, because it spills +# over to the next cluster. That's a good way to test that we can +# handle simultaneous snapshot table and refcount errors.) +poke_file "$TEST_IMG" $((sn3_ofs + 36)) '\x00\x01\x00\x00' + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo '=== Snapshot table too big ===' +echo + +sn_table_ofs=$(peek_file_be "$TEST_IMG.v3.orig" 64 8) + +# Fill a snapshot with 1 kB of extra data, a 65535-char ID, and a +# 65535-char name, and repeat it as many times as necessary to fill +# 64 MB (the maximum supported by qemu) + +touch "$TEST_DIR/sn0" + +# Full size (fixed + extra + ID + name + padding) +sn_size=$((40 + 1024 + 65535 + 65535 + 2)) + +# We only need the fixed part, though. +truncate -s 40 "$TEST_DIR/sn0" + +# 65535-char ID string +poke_file "$TEST_DIR/sn0" 12 '\xff\xff' +# 65535-char name +poke_file "$TEST_DIR/sn0" 14 '\xff\xff' +# 1 kB of extra data +poke_file "$TEST_DIR/sn0" 36 '\x00\x00\x04\x00' + +# Create test image +_make_test_img 64M + +# Hook up snapshot table somewhere safe (at 1 MB) +poke_file "$TEST_IMG" 64 '\x00\x00\x00\x00\x00\x10\x00\x00' + +offset=1048576 +size_written=0 +sn_count=0 +while [ $size_written -le $((64 * 1048576)) ]; do + dd if="$TEST_DIR/sn0" of="$TEST_IMG" bs=1 seek=$offset conv=notrunc \ + &> /dev/null + offset=$((offset + sn_size)) + size_written=$((size_written + sn_size)) + sn_count=$((sn_count + 1)) +done +truncate -s "$offset" "$TEST_IMG" + +# Give the last snapshot (the one to be removed) an L1 table so we can +# see how that is handled when repairing the image +# (Put it two clusters before 1 MB, and one L2 table one cluster +# before 1 MB) +poke_file "$TEST_IMG" $((offset - sn_size + 0)) \ + '\x00\x00\x00\x00\x00\x0e\x00\x00' +poke_file "$TEST_IMG" $((offset - sn_size + 8)) \ + '\x00\x00\x00\x01' + +# Hook up the L2 table +poke_file "$TEST_IMG" $((1048576 - 2 * 65536)) \ + '\x80\x00\x00\x00\x00\x0f\x00\x00' + +# Make sure all of the clusters we just hooked up are allocated: +# - The snapshot table +# - The last snapshot's L1 and L2 table +refblock0_allocate $((1048576 - 2 * 65536)) $offset + +poke_file "$TEST_IMG" 60 \ + "$(printf '%08x' $sn_count | sed -e 's/\(..\)/\\x\1/g')" + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo "$((sn_count - 1)) snapshots should remain:" +echo " qemu-img info reports $(_img_info | grep -c '^ \{34\}') snapshots" +echo " Image header reports $(peek_file_be "$TEST_IMG" 60 4) snapshots" + +echo +echo '=== Snapshot table too big with one entry with too much extra data ===' +echo + +# For this test, we reuse the image from the previous case, which has +# a snapshot table that is right at the limit. +# Our layout looks like this: +# - (a number of snapshot table entries) +# - One snapshot with $extra_data_size extra data +# - One normal snapshot that breaks the 64 MB boundary +# - One normal snapshot beyond the 64 MB boundary +# +# $extra_data_size is calculated so that simply by virtue of it +# decreasing to 1 kB, the penultimate snapshot will fit into 64 MB +# limit again. The final snapshot will always be beyond the limit, so +# that we can see that the repair algorithm does still determine the +# limit to be somewhere, even when truncating one snapshot's extra +# data. + +# The last case has removed the last snapshot, so calculate +# $old_offset to get the current image's real length +old_offset=$((offset - sn_size)) + +# The layout from the previous test had one snapshot beyond the 64 MB +# limit; we want the same (after the oversized extra data has been +# truncated to 1 kB), so we drop the last three snapshots and +# construct them from scratch. +offset=$((offset - 3 * sn_size)) +sn_count=$((sn_count - 3)) + +# Assuming we had already written one of the three snapshots +# (necessary so we can calculate $extra_data_size next). +size_written=$((size_written - 2 * sn_size)) + +# Increase the extra data size so we go past the limit +# (The -1024 comes from the 1 kB of extra data we already have) +extra_data_size=$((64 * 1048576 + 8 - sn_size - (size_written - 1024))) + +poke_file "$TEST_IMG" $((offset + 36)) \ + "$(printf '%08x' $extra_data_size | sed -e 's/\(..\)/\\x\1/g')" + +offset=$((offset + sn_size - 1024 + extra_data_size)) +size_written=$((size_written - 1024 + extra_data_size)) +sn_count=$((sn_count + 1)) + +# Write the two normal snapshots +for ((i = 0; i < 2; i++)); do + dd if="$TEST_DIR/sn0" of="$TEST_IMG" bs=1 seek=$offset conv=notrunc \ + &> /dev/null + offset=$((offset + sn_size)) + size_written=$((size_written + sn_size)) + sn_count=$((sn_count + 1)) + + if [ $i = 0 ]; then + # Check that the penultimate snapshot is beyond the 64 MB limit + echo "Snapshot table size should equal $((64 * 1048576 + 8)):" \ + $size_written + echo + fi +done + +truncate -s $offset "$TEST_IMG" +refblock0_allocate $old_offset $offset + +poke_file "$TEST_IMG" 60 \ + "$(printf '%08x' $sn_count | sed -e 's/\(..\)/\\x\1/g')" + +# Print error +_img_info +echo +_check_test_img +echo + +# Just truncating the extra data should be sufficient to shorten the +# snapshot table so only one snapshot exceeds the extra size +_check_test_img -r all + +echo +echo '=== Too many snapshots ===' +echo + +# Create a v2 image, for speeds' sake: All-zero snapshot table entries +# are only valid in v2. +IMGOPTS='compat=0.10' _make_test_img 64M + +# Hook up snapshot table somewhere safe (at 1 MB) +poke_file "$TEST_IMG" 64 '\x00\x00\x00\x00\x00\x10\x00\x00' +# "Create" more than 65536 snapshots (twice that many here) +poke_file "$TEST_IMG" 60 '\x00\x02\x00\x00' + +# 40-byte all-zero snapshot table entries are valid snapshots, but +# only in v2 (v3 needs 16 bytes of extra data, so we would have to +# write 131072x '\x10'). +truncate -s $((1048576 + 40 * 131072)) "$TEST_IMG" + +# But let us give one of the snapshots to be removed an L1 table so +# we can see how that is handled when repairing the image. +# (Put it two clusters before 1 MB, and one L2 table one cluster +# before 1 MB) +poke_file "$TEST_IMG" $((1048576 + 40 * 65536 + 0)) \ + '\x00\x00\x00\x00\x00\x0e\x00\x00' +poke_file "$TEST_IMG" $((1048576 + 40 * 65536 + 8)) \ + '\x00\x00\x00\x01' + +# Hook up the L2 table +poke_file "$TEST_IMG" $((1048576 - 2 * 65536)) \ + '\x80\x00\x00\x00\x00\x0f\x00\x00' + +# Make sure all of the clusters we just hooked up are allocated: +# - The snapshot table +# - The last snapshot's L1 and L2 table +refblock0_allocate $((1048576 - 2 * 65536)) $((1048576 + 40 * 131072)) + +# Print error +_img_info +echo +_check_test_img +echo + +# Should be repairable +_check_test_img -r all + +echo +echo '65536 snapshots should remain:' +echo " qemu-img info reports $(_img_info | grep -c '^ \{34\}') snapshots" +echo " Image header reports $(peek_file_be "$TEST_IMG" 60 4) snapshots" + +# success, all done +echo "*** done" +status=0
tests/qemu-iotests/261.out new file +346 -0
346 lines not on this page · raw message
tests/qemu-iotests/group +1 -0
@@ -273,6 +273,7 @@ 256 rw quick 257 rw 258 rw quick +261 rw 262 rw quick migration 263 rw quick 265 rw auto quick
Signature · 1 line
2.21.0
Quoted 47 lines · Max Reitz
Hi,
The v1 cover letter explained this series’s purpose:
https://lists.nongnu.org/archive/html/qemu-block/2019-07/msg01290.htmlThe v2 cover letter explained the v2 changes:
https://lists.nongnu.org/archive/html/qemu-block/2019-08/msg00921.htmlThe only changes in v3 are:
- Patches 12 and 13: Added notes to the commit messages why it’s OK to not let the user choose which snapshots to drop(I wanted to also address Eric’s idea of letting QCowSnapshot.extra_data_size be always >= sizeof(QCowSnapshotExtraData) and thus save the
MAX(sizeof(extra), sn->extra_data_size)inqcow2_write_snapshots(), but that doesn’t really work, because then we have no way of knowing later whether the image is compliant and thus needs fixing or not. He gave me an R-b anyway, so I guess it’s fine. O:-))This series now has R-bs from Eric on all patches. I’m only posting it because I felt a bit bad about just taking the series as-is and add the commit notes to 12 and 13 while applying it.
(Also, maybe there is someone who saw me have a bit of discussion with Eric and thus assumed I would definitely send a v3 that they could then review.)Max Reitz (16):
include: Moveendof()up fromhw/virtio/virtio.h
qcow2: Useendof()
qcow2: Add Error ** toqcow2_read_snapshots()
qcow2: Keep unknown extra snapshot data
qcow2: Makeqcow2_write_snapshots()public
qcow2: Putqcow2_upgrade()into its own function
qcow2: Write v3-compliant snapshot list on upgrade
qcow2: Separateqcow2_check_read_snapshot_table()
qcow2: Addqcow2_check_fix_snapshot_table()
qcow2: Fix broken snapshot table entries
qcow2: Keep track of the snapshot table length
qcow2: Fix overly long snapshot tables
qcow2: Repair snapshot table with too many entries
qcow2: Fix v3 snapshot table entry compliancy
iotests: Add peek_file* functions
iotests: Test qcow2's snapshot table handling
Quoted 14 lines
block/qcow2.h | 15 +- include/hw/virtio/virtio.h | 7 - include/qemu/compiler.h | 7 + block/qcow2-snapshot.c | 323 +++++++++++++++++++-- block/qcow2.c | 155 +++++++++-- hw/block/virtio-blk.c | 4 +- hw/net/virtio-net.c | 10 +- tests/qemu-iotests/261 | 523 +++++++++++++++++++++++++++++++++++ tests/qemu-iotests/261.out | 346 +++++++++++++++++++++++ tests/qemu-iotests/common.rc | 20 ++ tests/qemu-iotests/group | 1 + 11 files changed, 1354 insertions(+), 57 deletions(-) create mode 100755 tests/qemu-iotests/261 create mode 100644 tests/qemu-iotests/261.out
Thanks for the review, applied to my block branch:
https://git.xanclic.moe/XanClic/qemu/commits/branch/block
Max