qcow2: External data files
PATCH 1/20 qemu-iotests: Test qcow2 preallocation modes
In reply to Kevin Wolf--- tests/qemu-iotests/243 | 63 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/243.out | 26 ++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 90 insertions(+) create mode 100755 tests/qemu-iotests/243 create mode 100644 tests/qemu-iotests/243.out
tests/qemu-iotests/243 new file +63 -0
@@ -0,0 +1,63 @@ +#!/bin/bash +# +# Test qcow2 preallocation +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator [email protected] + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +for mode in off metadata falloc full; do + + echo + echo "=== preallocation=$mode ===" + echo + + IMGOPTS="preallocation=$mode" _make_test_img 64M + + echo -n "File size: " + du -b $TEST_IMG | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + echo -n "Disk usage: " + [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + +done + +# success, all done +echo "*** done" +rm -f $seq.full +status=0
tests/qemu-iotests/243.out new file +26 -0
@@ -0,0 +1,26 @@ +QA output created by 243 + +=== preallocation=off === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=off +File size: 196616 +Disk usage: low + +=== preallocation=metadata === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=metadata +File size: 67436544 +Disk usage: low + +=== preallocation=falloc === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=falloc +File size: 67436544 +Disk usage: high + +=== preallocation=full === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=full +File size: 67436544 +Disk usage: high +*** done
tests/qemu-iotests/group +1 -0
@@ -241,3 +241,4 @@ 239 rw auto quick 240 auto quick 242 rw auto quick +243 rw auto quick
Signature · 1 line
2.20.1
cover qcow2: External data files
There are use cases where raw images are given (e.g. existing physical disks), but advanced features like dirty bitmaps or backing files are wanted that require use of a proper image format like qcow2.
This series adds an incompatible feature bit to qcow2 which allows to use an external data file: Metadata is kept in the qcow2 file like usual, but guest data is written to an external file. Clusters in the data file are not reference counted, instead we use a flat layout where host cluster offset == guest cluster offset. The external data file is therefore readable as a raw image (though writing to it invalidates the associated qcow2 metadata). Features that require refcounting such as internal snapshots or compression are not supposed in such setups.
Major changes since the RFC (many more minor changes):
- Added a 'data-file-raw' flag to the spec to keep the data file a valid self-consistent raw image (write_zeroes must always be propagated)
- Added QAPI documentation
- Added some test cases
- Implemented data file support or error paths for discard, check, compressed writes, snapshots
- Fixed qcow2_co_block_status()
- Rearranged qcow2_do_open() code for data files for better error handling (e.g. no more fallback to default data file if an explicit, but invalid data-file option is given)
git-backport-diff compared to the RFC:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/20:[down] 'qemu-iotests: Test qcow2 preallocation modes' 002/20:[down] 'qcow2: Simplify preallocation code'
003/20:[0027] [FC] 'qcow2: Extend spec for external data files' 004/20:[0014] [FC] 'qcow2: Basic definitions for external data files' 005/20:[----] [--] 'qcow2: Pass bs to qcow2_get_cluster_type()' 006/20:[----] [--] 'qcow2: Prepare qcow2_get_cluster_type() for external data file' 007/20:[----] [--] 'qcow2: Prepare count_contiguous_clusters() for external data file' 008/20:[----] [--] 'qcow2: Don't assume 0 is an invalid cluster offset' 009/20:[down] 'qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()' 010/20:[down] 'qcow2: Prepare qcow2_co_block_status() for data file' 011/20:[0024] [FC] 'qcow2: External file I/O'
012/20:[down] 'qcow2: Return error for snapshot operation with data file' 013/20:[down] 'qcow2: Support external data file in qemu-img check' 014/20:[0035] [FC] 'qcow2: Add basic data-file infrastructure' 015/20:[0003] [FC] 'qcow2: Creating images with external data file' 016/20:[0057] [FC] 'qcow2: Store data file name in the image' 017/20:[down] 'qcow2: Implement data-file-raw create option' 018/20:[down] 'qemu-iotests: Preallocation with external data file' 019/20:[down] 'qemu-iotests: General tests for qcow2 with external data file' 020/20:[down] 'qemu-iotests: amend with external data file'
Kevin Wolf (20):
qemu-iotests: Test qcow2 preallocation modes
qcow2: Simplify preallocation code
qcow2: Extend spec for external data files
qcow2: Basic definitions for external data files
qcow2: Pass bs to qcow2_get_cluster_type()
qcow2: Prepare qcow2_get_cluster_type() for external data file
qcow2: Prepare count_contiguous_clusters() for external data file
qcow2: Don't assume 0 is an invalid cluster offset
qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()
qcow2: Prepare qcow2_co_block_status() for data file
qcow2: External file I/O
qcow2: Return error for snapshot operation with data file
qcow2: Support external data file in qemu-img check
qcow2: Add basic data-file infrastructure
qcow2: Creating images with external data file
qcow2: Store data file name in the image
qcow2: Implement data-file-raw create option
qemu-iotests: Preallocation with external data file
qemu-iotests: General tests for qcow2 with external data file
qemu-iotests: amend with external data file
qapi/block-core.json | 26 ++- docs/interop/qcow2.txt | 38 ++++- block/qcow2.h | 66 ++++++-- include/block/block_int.h | 2 + block/qcow2-bitmap.c | 7 +- block/qcow2-cache.c | 6 +- block/qcow2-cluster.c | 182 ++++++++++++--------- block/qcow2-refcount.c | 88 ++++++++--- block/qcow2-snapshot.c | 22 ++- block/qcow2.c | 314 +++++++++++++++++++++++++++++++------ tests/qemu-iotests/031.out | 8 +- tests/qemu-iotests/036.out | 4 +- tests/qemu-iotests/061 | 45 +++++- tests/qemu-iotests/061.out | 103 +++++++++++- tests/qemu-iotests/082.out | 54 +++++++ tests/qemu-iotests/220.out | 2 +- tests/qemu-iotests/243 | 85 ++++++++++ tests/qemu-iotests/243.out | 58 +++++++ tests/qemu-iotests/244 | 200 +++++++++++++++++++++++ tests/qemu-iotests/244.out | 125 +++++++++++++++ tests/qemu-iotests/group | 2 + 21 files changed, 1243 insertions(+), 194 deletions(-) create mode 100755 tests/qemu-iotests/243 create mode 100644 tests/qemu-iotests/243.out create mode 100755 tests/qemu-iotests/244 create mode 100644 tests/qemu-iotests/244.out
Signature · 1 line
2.20.1
PATCH 3/20 qcow2: Extend spec for external data files
This adds external data file to the qcow2 spec as a new incompatible feature.
--- docs/interop/qcow2.txt | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-)
docs/interop/qcow2.txt +34 -4
@@ -97,7 +97,19 @@ in the description of a field. be written to (unless for regaining consistency). - Bits 2-63: Reserved (set to 0) + Bit 2: External data file bit. If this bit is set, an + external data file is used. Guest clusters are + then stored in the external data file. For such + images, clusters in the external data file are + not refcounted. The offset field in the + Standard Cluster Descriptor must match the + guest offset and neither compressed clusters + nor internal snapshots are supported. + + An external data file name header extension may + be present if this bit is set. + + Bits 3-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can
@@ -126,7 +138,17 @@ in the description of a field. bit is unset, the bitmaps extension data must be considered inconsistent. - Bits 1-63: Reserved (set to 0) + Bit 1: If this bit is set, the external data file can + be read as a consistent standalone raw image + without looking at the qcow2 metadata. + + Setting this bit has a performance impact for + some operations on the image (e.g. writing + zeros requires writing to the data file instead + of only setting the zero flag in the L2 table + entry) and conflicts with backing files. + + Bits 2-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width
@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file name other - Unknown header extension, can be safely ignored
@@ -437,6 +460,11 @@ L2 table entry: This information is only accurate in L2 tables that are reachable from the active L1 table. + With external data files, all guest clusters have an + implicit refcount of 1 (because of the fixed host = guest + mapping for guest cluster offsets), so this bit should be 1 + for all allocated clusters. + Standard Cluster Descriptor: Bit 0: If set to 1, the cluster reads as all zeros. The host
@@ -450,8 +478,10 @@ Standard Cluster Descriptor: 1 - 8: Reserved (set to 0) 9 - 55: Bits 9-55 of host cluster offset. Must be aligned to a - cluster boundary. If the offset is 0, the cluster is - unallocated. + cluster boundary. If the offset is 0 and bit 63 is clear, + the cluster is unallocated. The offset may only be 0 with + bit 63 set (indicating a host cluster offset of 0) when an + external data file is used. 56 - 61: Reserved (set to 0)
Signature · 1 line
2.20.1
PATCH 5/20 qcow2: Pass bs to qcow2_get_cluster_type()
In reply to Kevin Wolf--- block/qcow2.h | 3 ++- block/qcow2-cluster.c | 37 +++++++++++++++++++------------------ block/qcow2-refcount.c | 10 +++++----- 3 files changed, 26 insertions(+), 24 deletions(-)
block/qcow2.h +2 -1
@@ -510,7 +510,8 @@ static inline int64_t qcow2_vm_state_offset(BDRVQcow2State *s)return (int64_t )s->l1_vm_state_index << (s->cluster_bits + s->l2_bits); } -static inline QCow2ClusterType qcow2_get_cluster_type (uint64_t l2_entry) +static inline QCow2ClusterType qcow2_get_cluster_type (BlockDriverState *bs, +uint64_t l2_entry) {if (l2_entry &QCOW_OFLAG_COMPRESSED ) {return QCOW2_CLUSTER_COMPRESSED ;
block/qcow2-cluster.c +19 -18
@@ -380,8 +380,8 @@ fail:* as contiguous. (This allows it, for example, to stop at the first compressed * cluster which may require a different handling) */ -static int count_contiguous_clusters (int nb_clusters,int cluster_size, -uint64_t *l2_slice,uint64_t stop_flags) +static int count_contiguous_clusters (BlockDriverState *bs,int nb_clusters, +int cluster_size,uint64_t *l2_slice,uint64_t stop_flags) {int i;QCow2ClusterType first_cluster_type;
@@ -394,7 +394,7 @@ static int count_contiguous_clusters(int nb_clusters, int cluster_size, }/* must be allocated */ - first_cluster_type =qcow2_get_cluster_type (first_entry); + first_cluster_type =qcow2_get_cluster_type (bs, first_entry);assert (first_cluster_type ==QCOW2_CLUSTER_NORMAL || first_cluster_type ==QCOW2_CLUSTER_ZERO_ALLOC );
@@ -412,7 +412,8 @@ static int count_contiguous_clusters(int nb_clusters, int cluster_size,* Checks how many consecutive unallocated clusters in a given L2 * slice have the same cluster type. */ -static int count_contiguous_clusters_unallocated (int nb_clusters, +static int count_contiguous_clusters_unallocated (BlockDriverState *bs, +int nb_clusters,uint64_t *l2_slice,QCow2ClusterType wanted_type) {
@@ -422,7 +423,7 @@ static int count_contiguous_clusters_unallocated(int nb_clusters, wanted_type == QCOW2_CLUSTER_UNALLOCATED);for (i =0 ; i < nb_clusters; i++) {uint64_t entry =be64_to_cpu (l2_slice[i]); -QCow2ClusterType type =qcow2_get_cluster_type (entry); +QCow2ClusterType type =qcow2_get_cluster_type (bs, entry);if (type != wanted_type) {break ;
@@ -595,7 +596,7 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,* true */ assert (nb_clusters <=INT_MAX ); - type =qcow2_get_cluster_type (*cluster_offset); + type =qcow2_get_cluster_type (bs, *cluster_offset);if (s->qcow_version <3 && (type ==QCOW2_CLUSTER_ZERO_PLAIN || type ==QCOW2_CLUSTER_ZERO_ALLOC )) { qcow2_signal_corruption(bs, true, -1, -1, "Zero cluster entry found"
@@ -613,14 +614,14 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,case QCOW2_CLUSTER_ZERO_PLAIN :case QCOW2_CLUSTER_UNALLOCATED :/* how many empty clusters ? */ - c =count_contiguous_clusters_unallocated (nb_clusters, + c =count_contiguous_clusters_unallocated (bs, nb_clusters, &l2_slice[l2_index], type); *cluster_offset =0 ;break ;case QCOW2_CLUSTER_ZERO_ALLOC :case QCOW2_CLUSTER_NORMAL :/* how many allocated clusters ? */ - c =count_contiguous_clusters (nb_clusters, s->cluster_size, + c =count_contiguous_clusters (bs, nb_clusters, s->cluster_size, &l2_slice[l2_index],QCOW_OFLAG_ZERO ); *cluster_offset &=L2E_OFFSET_MASK ;if (offset_into_cluster (s, *cluster_offset)) {
@@ -1013,14 +1014,14 @@ void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m)* write, but require COW to be performed (this includes yet unallocated space, * which must copy from the backing file) */ -static int count_cow_clusters (BDRVQcow2State *s,int nb_clusters, +static int count_cow_clusters (BlockDriverState *bs,int nb_clusters,uint64_t *l2_slice,int l2_index) {int i;for (i =0 ; i < nb_clusters; i++) {uint64_t l2_entry =be64_to_cpu (l2_slice[l2_index + i]); -QCow2ClusterType cluster_type =qcow2_get_cluster_type (l2_entry); +QCow2ClusterType cluster_type =qcow2_get_cluster_type (bs, l2_entry);switch (cluster_type) {case QCOW2_CLUSTER_NORMAL :
@@ -1165,7 +1166,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset, cluster_offset =be64_to_cpu (l2_slice[l2_index]);/* Check how many clusters are already allocated and don't need COW */ -if (qcow2_get_cluster_type (cluster_offset) ==QCOW2_CLUSTER_NORMAL +if (qcow2_get_cluster_type (bs, cluster_offset) ==QCOW2_CLUSTER_NORMAL && (cluster_offset &QCOW_OFLAG_COPIED )) {/* If a specific host_offset is required, check it */
@@ -1189,7 +1190,7 @@ static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,/* We keep all QCOW_OFLAG_COPIED clusters */ keep_clusters = -count_contiguous_clusters (nb_clusters, s->cluster_size, +count_contiguous_clusters (bs, nb_clusters, s->cluster_size, &l2_slice[l2_index],QCOW_OFLAG_COPIED |QCOW_OFLAG_ZERO );assert (keep_clusters <= nb_clusters);
@@ -1324,7 +1325,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,if (entry &QCOW_OFLAG_COMPRESSED ) { nb_clusters =1 ; }else { - nb_clusters =count_cow_clusters (s, nb_clusters, l2_slice, l2_index); + nb_clusters =count_cow_clusters (bs, nb_clusters, l2_slice, l2_index); }/* This function is only called when there were no non-COW clusters, so if
@@ -1332,7 +1333,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,* wrong with our code. */ assert (nb_clusters >0 ); -if (qcow2_get_cluster_type (entry) ==QCOW2_CLUSTER_ZERO_ALLOC && +if (qcow2_get_cluster_type (bs, entry) ==QCOW2_CLUSTER_ZERO_ALLOC && (entry &QCOW_OFLAG_COPIED ) && (!*host_offset ||start_of_cluster (s, *host_offset) == (entry &L2E_OFFSET_MASK )))
@@ -1352,7 +1353,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,* would be fine, too, but count_cow_clusters() above has limited * nb_clusters already to a range of COW clusters */ preallocated_nb_clusters = -count_contiguous_clusters (nb_clusters, s->cluster_size, +count_contiguous_clusters (bs, nb_clusters, s->cluster_size, &l2_slice[l2_index],QCOW_OFLAG_COPIED );assert (preallocated_nb_clusters >0 );
@@ -1616,7 +1617,7 @@ static int discard_in_l2_slice(BlockDriverState *bs, uint64_t offset,* If full_discard is true, the sector should not read back as zeroes, * but rather fall through to the backing file. */ -switch (qcow2_get_cluster_type (old_l2_entry)) { +switch (qcow2_get_cluster_type (bs, old_l2_entry)) {case QCOW2_CLUSTER_UNALLOCATED :if (full_discard || !bs->backing) {continue ;
@@ -1729,7 +1730,7 @@ static int zero_in_l2_slice(BlockDriverState *bs, uint64_t offset,* Minimize L2 changes if the cluster already reads back as * zeroes with correct allocation. */ - cluster_type =qcow2_get_cluster_type (old_offset); + cluster_type =qcow2_get_cluster_type (bs, old_offset);if (cluster_type ==QCOW2_CLUSTER_ZERO_PLAIN || (cluster_type ==QCOW2_CLUSTER_ZERO_ALLOC && !unmap)) {continue ;
@@ -1871,7 +1872,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,uint64_t l2_entry =be64_to_cpu (l2_slice[j]);int64_t offset = l2_entry &L2E_OFFSET_MASK ;QCow2ClusterType cluster_type = -qcow2_get_cluster_type (l2_entry); +qcow2_get_cluster_type (bs, l2_entry);if (cluster_type !=QCOW2_CLUSTER_ZERO_PLAIN && cluster_type !=QCOW2_CLUSTER_ZERO_ALLOC ) {
block/qcow2-refcount.c +5 -5
@@ -1157,7 +1157,7 @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry, {BDRVQcow2State *s = bs->opaque; -switch (qcow2_get_cluster_type (l2_entry)) { +switch (qcow2_get_cluster_type (bs, l2_entry)) {case QCOW2_CLUSTER_COMPRESSED : {int nb_csectors;
@@ -1300,7 +1300,7 @@ int qcow2_update_snapshot_refcount(BlockDriverState *bs, entry &= ~QCOW_OFLAG_COPIED ; offset = entry &L2E_OFFSET_MASK ; -switch (qcow2_get_cluster_type (entry)) { +switch (qcow2_get_cluster_type (bs, entry)) {case QCOW2_CLUSTER_COMPRESSED : nb_csectors = ((entry >> s->csize_shift) & s->csize_mask) +1 ;
@@ -1582,7 +1582,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,for (i =0 ; i < s->l2_size; i++) { l2_entry =be64_to_cpu (l2_table[i]); -switch (qcow2_get_cluster_type (l2_entry)) { +switch (qcow2_get_cluster_type (bs, l2_entry)) {case QCOW2_CLUSTER_COMPRESSED :/* Compressed clusters don't have QCOW_OFLAG_COPIED */ if (l2_entry &QCOW_OFLAG_COPIED ) {
@@ -1633,7 +1633,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,/* Correct offsets are cluster aligned */ if (offset_into_cluster (s, offset)) { -if (qcow2_get_cluster_type (l2_entry) == +if (qcow2_get_cluster_type (bs, l2_entry) ==QCOW2_CLUSTER_ZERO_ALLOC ) { fprintf(stderr, "%s offset=%" PRIx64 ": Preallocated zero "
@@ -1868,7 +1868,7 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res,for (j =0 ; j < s->l2_size; j++) {uint64_t l2_entry =be64_to_cpu (l2_table[j]);uint64_t data_offset = l2_entry &L2E_OFFSET_MASK ; -QCow2ClusterType cluster_type =qcow2_get_cluster_type (l2_entry); +QCow2ClusterType cluster_type =qcow2_get_cluster_type (bs, l2_entry);if (cluster_type ==QCOW2_CLUSTER_NORMAL || cluster_type ==QCOW2_CLUSTER_ZERO_ALLOC ) {
Signature · 1 line
2.20.1
PATCH 6/20 qcow2: Prepare qcow2_get_cluster_type() for external data file
In reply to Kevin Wolf--- block/qcow2.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
block/qcow2.h +9 -1
@@ -521,7 +521,15 @@ static inline QCow2ClusterType qcow2_get_cluster_type(BlockDriverState *bs, }return QCOW2_CLUSTER_ZERO_PLAIN ; }else if (!(l2_entry &L2E_OFFSET_MASK )) { -return QCOW2_CLUSTER_UNALLOCATED ; +/* Offset 0 generally means unallocated, but it is ambiguous with +* external data files because 0 is a valid offset there. However, all +* clusters in external data files always have refcount 1, so we can +* rely on QCOW_OFLAG_COPIED to disambiguate. */ +if (has_data_file (bs) && (l2_entry &QCOW_OFLAG_COPIED )) { +return QCOW2_CLUSTER_NORMAL ; + }else { +return QCOW2_CLUSTER_UNALLOCATED ; + } }else {return QCOW2_CLUSTER_NORMAL ; }
Signature · 1 line
2.20.1
PATCH 2/20 qcow2: Simplify preallocation code
In reply to Kevin WolfImage creation already involves a bdrv_co_truncate() call, which allows to specify a preallocation mode. Just pass the right mode there and remove the code that is made redundant by this.
--- block/qcow2.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-)
block/qcow2.c +1 -27
@@ -2940,19 +2940,6 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) goto out; } -if (qcow2_opts->preallocation ==PREALLOC_MODE_FULL || - qcow2_opts->preallocation ==PREALLOC_MODE_FALLOC ) - { -int64_t prealloc_size = -qcow2_calc_prealloc_size (qcow2_opts->size, cluster_size, - refcount_order); - - ret =blk_truncate (blk, prealloc_size, qcow2_opts->preallocation, errp); -if (ret <0 ) { - goto out; - } - } -/* Write the header */ QEMU_BUILD_BUG_ON ((1 <<MIN_CLUSTER_BITS ) <sizeof (*header)); header =g_malloc0 (cluster_size);
@@ -3034,7 +3021,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) }/* Okay, now that we have a valid image, let's give it the right size */ - ret =blk_truncate (blk, qcow2_opts->size,PREALLOC_MODE_OFF , errp); + ret =blk_truncate (blk, qcow2_opts->size, qcow2_opts->preallocation, errp);if (ret <0 ) {error_prepend (errp,"Could not resize image: " ); goto out;
@@ -3066,19 +3053,6 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } } -/* And if we're supposed to preallocate metadata, do that now */ -if (qcow2_opts->preallocation !=PREALLOC_MODE_OFF ) { -BDRVQcow2State *s =blk_bs (blk)->opaque; -qemu_co_mutex_lock (&s->lock); - ret =preallocate_co (blk_bs (blk),0 , qcow2_opts->size); -qemu_co_mutex_unlock (&s->lock); - -if (ret <0 ) { -error_setg_errno (errp, -ret,"Could not preallocate metadata" ); - goto out; - } - } -blk_unref (blk); blk =NULL ;
Signature · 1 line
2.20.1
PATCH 9/20 qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()
In reply to Kevin Wolfqcow2_alloc_compressed_cluster_offset() used to return the cluster offset for success and 0 for error. This doesn't only conflict with 0 as a valid host offset, but also loses the error code.
Similar to the change made to qcow2_alloc_cluster_offset() for uncompressed clusters in commit 148da7ea9d6, make the function return 0/-errno and return the allocated cluster offset in a by-reference parameter.
--- block/qcow2.h | 7 ++++--- block/qcow2-cluster.c | 28 +++++++++++++--------------- block/qcow2.c | 19 ++++++++----------- tests/qemu-iotests/220.out | 2 +- 4 files changed, 26 insertions(+), 30 deletions(-)
block/qcow2.h +4 -3
@@ -647,9 +647,10 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, uint64_t offset,int qcow2_alloc_cluster_offset (BlockDriverState *bs,uint64_t offset,unsigned int *bytes,uint64_t *host_offset,QCowL2Meta **m); -uint64_t qcow2_alloc_compressed_cluster_offset (BlockDriverState *bs, -uint64_t offset, -int compressed_size); +int qcow2_alloc_compressed_cluster_offset (BlockDriverState *bs, +uint64_t offset, +int compressed_size, +uint64_t *host_offset);int qcow2_alloc_cluster_link_l2 (BlockDriverState *bs,QCowL2Meta *m);void qcow2_alloc_cluster_abort (BlockDriverState *bs,QCowL2Meta *m);
block/qcow2-cluster.c +13 -15
@@ -736,19 +736,16 @@ static int get_cluster_table(BlockDriverState *bs, uint64_t offset,/* * alloc_compressed_cluster_offset * -* For a given offset of the disk image, return cluster offset in -* qcow2 file. -* -* If the offset is not found, allocate a new compressed cluster. -* -* Return the cluster offset if successful, -* Return 0, otherwise. +* For a given offset on the virtual disk, allocate a new compressed cluster +* and put the host offset of the cluster into *host_offset. If a cluster is +* already allocated at the offset, return an error. * +* Return 0 on success and -errno in error cases */ - -uint64_t qcow2_alloc_compressed_cluster_offset (BlockDriverState *bs, -uint64_t offset, -int compressed_size) +int qcow2_alloc_compressed_cluster_offset (BlockDriverState *bs, +uint64_t offset, +int compressed_size, +uint64_t *host_offset) {BDRVQcow2State *s = bs->opaque;int l2_index, ret;
@@ -758,7 +755,7 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, ret =get_cluster_table (bs, offset, &l2_slice, &l2_index);if (ret <0 ) { -return 0 ; +return ret; }/* Compression can't overwrite anything. Fail if the cluster was already
@@ -766,13 +763,13 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, cluster_offset =be64_to_cpu (l2_slice[l2_index]);if (cluster_offset &L2E_OFFSET_MASK ) {qcow2_cache_put (s->l2_table_cache, (void **) &l2_slice); -return 0 ; +return -EIO ; } cluster_offset =qcow2_alloc_bytes (bs, compressed_size);if (cluster_offset <0 ) {qcow2_cache_put (s->l2_table_cache, (void **) &l2_slice); -return 0 ; +return cluster_offset; } nb_csectors = ((cluster_offset + compressed_size - 1) >> 9) -
@@ -790,7 +787,8 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, l2_slice[l2_index] =cpu_to_be64 (cluster_offset);qcow2_cache_put (s->l2_table_cache, (void **) &l2_slice); -return cluster_offset; + *host_offset = cluster_offset & s->cluster_offset_mask; +return 0 ; }static int perform_cow (BlockDriverState *bs,QCowL2Meta *m)
block/qcow2.c +8 -11
@@ -3885,17 +3885,16 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset,int ret;size_t out_len;uint8_t *buf, *out_buf; -int64_t cluster_offset; +uint64_t cluster_offset;if (bytes ==0 ) {/* align end of file to a sector boundary to ease reading with sector based I/Os */ - cluster_offset =bdrv_getlength (bs->file->bs); -if (cluster_offset <0 ) { -return cluster_offset; +int64_t len =bdrv_getlength (bs->file->bs); +if (len <0 ) { +return len; } -return bdrv_co_truncate (bs->file, cluster_offset,PREALLOC_MODE_OFF , -NULL ); +return bdrv_co_truncate (bs->file, len,PREALLOC_MODE_OFF ,NULL ); }if (offset_into_cluster (s, offset)) {
@@ -3932,14 +3931,12 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, }qemu_co_mutex_lock (&s->lock); - cluster_offset = -qcow2_alloc_compressed_cluster_offset (bs, offset, out_len); -if (!cluster_offset) { + ret =qcow2_alloc_compressed_cluster_offset (bs, offset, out_len, + &cluster_offset); +if (ret <0 ) {qemu_co_mutex_unlock (&s->lock); - ret = -EIO ; goto fail; } - cluster_offset &= s->cluster_offset_mask; ret =qcow2_pre_write_overlap_check (bs,0 , cluster_offset, out_len);qemu_co_mutex_unlock (&s->lock);
tests/qemu-iotests/220.out +1 -1
@@ -38,7 +38,7 @@ wrote 2097152/2097152 bytes at offset 37748736 No errors were found on the image. image size 39845888 == Trying to write compressed cluster == -write failed: Input/output error +write failed: File too large image size 562949957615616 == Writing normal cluster == wrote 2097152/2097152 bytes at offset 0
Signature · 1 line
2.20.1
PATCH 13/20 qcow2: Support external data file in qemu-img check
In reply to Kevin WolfFor external data files, data clusters must be excluded from the refcount calculations. Instead, an implicit refcount of 1 is assumed for the COPIED flag.
Compressed clusters and internal snapshots are incompatible with external data files, so print an error if they are in use for images with an external data file.
--- block/qcow2-refcount.c | 41 ++++++++++++++++++++++++++++++----------- 1 file changed, 30 insertions(+), 11 deletions(-)
block/qcow2-refcount.c +30 -11
@@ -1605,6 +1605,13 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, res->corruptions++; } +if (has_data_file (bs)) { +fprintf (stderr,"ERROR compressed cluster %d with data file, " +"entry=0x%" PRIx64"\n" , i, l2_entry); + res->corruptions++; +break ; + } +/* Mark cluster as used */ nb_csectors = ((l2_entry >> s->csize_shift) & s->csize_mask) +1 ;
@@ -1695,11 +1702,13 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, }/* Mark cluster as used */ - ret =qcow2_inc_refcounts_imrt (bs, res, - refcount_table, refcount_table_size, - offset, s->cluster_size); -if (ret <0 ) { - goto fail; +if (!has_data_file (bs)) { + ret =qcow2_inc_refcounts_imrt (bs, res, refcount_table, + refcount_table_size, + offset, s->cluster_size); +if (ret <0 ) { + goto fail; + } }break ; }
@@ -1884,12 +1893,16 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res,if (cluster_type ==QCOW2_CLUSTER_NORMAL || cluster_type ==QCOW2_CLUSTER_ZERO_ALLOC ) { - ret =qcow2_get_refcount (bs, - data_offset >> s->cluster_bits, - &refcount); -if (ret <0 ) { -/* don't print message nor increment check_errors */ -continue ; +if (has_data_file (bs)) { + refcount =1 ; + }else { + ret =qcow2_get_refcount (bs, + data_offset >> s->cluster_bits, + &refcount); +if (ret <0 ) { +/* don't print message nor increment check_errors */ +continue ; + } }if ((refcount ==1 ) != ((l2_entry &QCOW_OFLAG_COPIED ) !=0 )) { fprintf(stderr, "%s OFLAG_COPIED data cluster: "
@@ -2083,6 +2096,12 @@ static int calculate_refcounts(BlockDriverState *bs, BdrvCheckResult *res, }/* snapshots */ +if (has_data_file (bs) && s->nb_snapshots) { +fprintf (stderr,"ERROR %d snapshots in image with data file\n" , + s->nb_snapshots); + res->corruptions++; + } +for (i =0 ; i < s->nb_snapshots; i++) { sn = s->snapshots + i;if (offset_into_cluster (s, sn->l1_table_offset)) {
Signature · 1 line
2.20.1
PATCH 14/20 qcow2: Add basic data-file infrastructure
In reply to Kevin WolfThis adds a .bdrv_open option to specify the external data file node.
--- qapi/block-core.json | 7 ++++++- block/qcow2.h | 4 +++- block/qcow2.c | 34 ++++++++++++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 4 deletions(-)
qapi/block-core.json +6 -1
@@ -3080,6 +3080,10 @@ # encrypted images, except when doing a metadata-only # probe of the image. (since 2.10) # +# @data-file: reference to or definition of the external data file. +# This may only be specified for images that require an +# external data file. (since 4.0) +# # Since: 2.9 ## { 'struct': 'BlockdevOptionsQcow2',
@@ -3094,7 +3098,8 @@ '*l2-cache-entry-size': 'int', '*refcount-cache-size': 'int', '*cache-clean-interval': 'int', - '*encrypt': 'BlockdevQcow2Encryption' } } + '*encrypt': 'BlockdevQcow2Encryption', + '*data-file': 'BlockdevRef' } } ## # @SshHostKeyCheckMode:
block/qcow2.h +3 -1
@@ -91,6 +91,7 @@#define DEFAULT_CLUSTER_SIZE 65536 +#define QCOW2_OPT_DATA_FILE "data-file"#define QCOW2_OPT_LAZY_REFCOUNTS "lazy-refcounts"#define QCOW2_OPT_DISCARD_REQUEST "pass-discard-request"#define QCOW2_OPT_DISCARD_SNAPSHOT "pass-discard-snapshot"
@@ -205,7 +206,8 @@ enum {QCOW2_INCOMPAT_DATA_FILE =1 <<QCOW2_INCOMPAT_DATA_FILE_BITNR ,QCOW2_INCOMPAT_MASK =QCOW2_INCOMPAT_DIRTY - | QCOW2_INCOMPAT_CORRUPT, + |QCOW2_INCOMPAT_CORRUPT + | QCOW2_INCOMPAT_DATA_FILE, };/* Compatible feature bits */
block/qcow2.c +32 -2
@@ -1441,8 +1441,31 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } -/* TODO Open external data file */ - s->data_file = bs->file; +/* Open external data file */ + s->data_file =bdrv_open_child (NULL , options,"data-file" , bs, &child_file, + true, &local_err); +if (local_err) { +error_propagate (errp, local_err); + ret = -EINVAL ; + goto fail; + } + +if (s->incompatible_features &QCOW2_INCOMPAT_DATA_FILE ) { +if (!s->data_file) { +error_setg (errp,"'data-file' is required for this image" ); + ret = -EINVAL ; + goto fail; + } + }else { +if (s->data_file) { +error_setg (errp,"'data-file' can only be set for images with an " +"external data file" ); + ret = -EINVAL ; + goto fail; + }else { + s->data_file = bs->file; + } + }/* qcow2_read_extension may have set up the crypto context * if the crypt method needs a header region, some methods
@@ -1615,6 +1638,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options,return ret; fail: +if (has_data_file (bs)) { +bdrv_unref_child (bs, s->data_file); + }g_free (s->unknown_header_fields);cleanup_unknown_header_ext (bs);qcow2_free_snapshots (bs);
@@ -2234,6 +2260,10 @@ static void qcow2_close(BlockDriverState *bs)g_free (s->image_backing_file);g_free (s->image_backing_format); +if (has_data_file (bs)) { +bdrv_unref_child (bs, s->data_file); + } +qcow2_refcount_close (bs);qcow2_free_snapshots (bs); }
Signature · 1 line
2.20.1
PATCH 18/20 qemu-iotests: Preallocation with external data file
In reply to Kevin WolfTest that preallocating metadata results in a somewhat larger qcow2 file, but preallocating data only affects the disk usage of the data file and the qcow2 file stays small.
--- tests/qemu-iotests/243 | 22 ++++++++++++++++++++++ tests/qemu-iotests/243.out | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+)
tests/qemu-iotests/243 +22 -0
@@ -29,6 +29,7 @@ status=1 # failure is the default! _cleanup() { _cleanup_test_img + rm -f $TEST_IMG.data } trap "_cleanup; exit \$status" 0 1 2 3 15
@@ -57,6 +58,27 @@ for mode in off metadata falloc full; do done +for mode in off metadata falloc full; do + + echo + echo "=== External data file: preallocation=$mode ===" + echo + + IMGOPTS="data_file=$TEST_IMG.data,preallocation=$mode" _make_test_img 64M + + echo -n "qcow2 file size: " + du -b $TEST_IMG | cut -f1 + echo -n "data file size: " + du -b $TEST_IMG.data | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + echo -n "qcow2 disk usage: " + [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + echo -n "data disk usage: " + [ $(du -B1 $TEST_IMG.data | cut -f1) -lt 1048576 ] && echo "low" || echo "high" + +done + # success, all done echo "*** done" rm -f $seq.full
tests/qemu-iotests/243.out +32 -0
@@ -23,4 +23,36 @@ Disk usage: high Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 preallocation=full File size: 67436544 Disk usage: high + +=== External data file: preallocation=off === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=off +qcow2 file size: 196616 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: low + +=== External data file: preallocation=metadata === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=metadata +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: low + +=== External data file: preallocation=falloc === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=falloc +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: high + +=== External data file: preallocation=full === + +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 data_file=TEST_DIR/t.IMGFMT.data preallocation=full +qcow2 file size: 327680 +data file size: 67108864 +qcow2 disk usage: low +data disk usage: high *** done
Signature · 1 line
2.20.1
PATCH 7/20 qcow2: Prepare count_contiguous_clusters() for external data file
In reply to Kevin WolfOffset 0 can be valid for normal (allocated) clusters now, so use qcow2_get_cluster_type() instead.
--- block/qcow2-cluster.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +2 -2
@@ -389,12 +389,12 @@ static int count_contiguous_clusters(BlockDriverState *bs, int nb_clusters,uint64_t first_entry =be64_to_cpu (l2_slice[0 ]);uint64_t offset = first_entry & mask; -if (!offset) { + first_cluster_type =qcow2_get_cluster_type (bs, first_entry); +if (first_cluster_type ==QCOW2_CLUSTER_UNALLOCATED ) {return 0 ; }/* must be allocated */ - first_cluster_type =qcow2_get_cluster_type (bs, first_entry);assert (first_cluster_type ==QCOW2_CLUSTER_NORMAL || first_cluster_type ==QCOW2_CLUSTER_ZERO_ALLOC );
Signature · 1 line
2.20.1
PATCH 10/20 qcow2: Prepare qcow2_co_block_status() for data file
In reply to Kevin WolfOffset 0 cannot be assumed to mean an unallocated cluster any more. Instead, the cluster type needs to be checked.
*file must refer to the data file instead of the image file if a valid offset is returned from qcow2_co_block_status().
--- block/qcow2.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
block/qcow2.c +2 -2
@@ -1817,11 +1817,11 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, *pnum = bytes; -if (cluster_offset !=0 && ret !=QCOW2_CLUSTER_COMPRESSED && +if ((ret ==QCOW2_CLUSTER_NORMAL || ret ==QCOW2_CLUSTER_ZERO_ALLOC ) && !s->crypto) { index_in_cluster = offset & (s->cluster_size -1 ); *map = cluster_offset | index_in_cluster; - *file = bs->file->bs; + *file = s->data_file->bs; status |=BDRV_BLOCK_OFFSET_VALID ; }if (ret ==QCOW2_CLUSTER_ZERO_PLAIN || ret ==QCOW2_CLUSTER_ZERO_ALLOC ) {
Signature · 1 line
2.20.1
PATCH 20/20 qemu-iotests: amend with external data file
In reply to Kevin Wolf--- tests/qemu-iotests/061 | 45 ++++++++++++++++++- tests/qemu-iotests/061.out | 89 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 133 insertions(+), 1 deletion(-)
tests/qemu-iotests/061 +44 -1
@@ -28,7 +28,8 @@ status=1 # failure is the default! _cleanup() { - _cleanup_test_img + _cleanup_test_img + rm -f $TEST_IMG.data } trap "_cleanup; exit \$status" 0 1 2 3 15
@@ -250,6 +251,48 @@ $QEMU_IMG snapshot -c foo "$TEST_IMG" $QEMU_IMG amend -p -o "compat=0.10" "$TEST_IMG" _check_test_img +echo +echo "=== Testing version downgrade with external data file ===" +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data" _make_test_img 64M +$QEMU_IMG amend -o "compat=0.10" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +echo +echo "=== Try changing the external data file ===" +echo +IMGOPTS="compat=1.1" _make_test_img 64M +$QEMU_IMG amend -o "data_file=foo" "$TEST_IMG" + +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data" _make_test_img 64M +$QEMU_IMG amend -o "data_file=foo" "$TEST_IMG" +_img_info --format-specific +TEST_IMG="data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" _img_info --format-specific --image-opts + +echo +$QEMU_IMG amend -o "data_file=" --image-opts "data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" +_img_info --format-specific +TEST_IMG="data-file.filename=$TEST_IMG.data,file.filename=$TEST_IMG" _img_info --format-specific --image-opts + +echo +echo "=== Clearing and setting data-file-raw ===" +echo +IMGOPTS="compat=1.1,data_file=$TEST_IMG.data,data_file_raw=on" _make_test_img 64M +$QEMU_IMG amend -o "data_file_raw=on" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +$QEMU_IMG amend -o "data_file_raw=off" "$TEST_IMG" +_img_info --format-specific +_check_test_img + +$QEMU_IMG amend -o "data_file_raw=on" "$TEST_IMG" +_img_info --format-specific +_check_test_img + + # success, all done echo "*** done" rm -f $seq.full
tests/qemu-iotests/061.out +89 -0
93 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 8/20 qcow2: Don't assume 0 is an invalid cluster offset
In reply to Kevin WolfThe cluster allocation code uses 0 as an invalid offset that is used in case of errors or as "offset not yet determined". With external data files, a host cluster offset of 0 becomes valid, though.
Define a constant INV_OFFSET (which is not cluster aligned and will therefore never be a valid offset) that can be used for such purposes.
This removes the additional host_offset == 0 check that commit ff52aab2df5 introduced; the confusion between an invalid offset and (erroneous) allocation at offset 0 is removed with this change.
--- block/qcow2.h | 2 ++ block/qcow2-cluster.c | 59 ++++++++++++++++++++----------------------- 2 files changed, 29 insertions(+), 32 deletions(-)
block/qcow2.h +2 -0
@@ -463,6 +463,8 @@ typedef enum QCow2MetadataOverlap {#define REFT_OFFSET_MASK 0xfffffffffffffe00ULL +#define INV_OFFSET (-1ULL) +static inline bool has_data_file (BlockDriverState *bs) {BDRVQcow2State *s = bs->opaque;
block/qcow2-cluster.c +27 -32
143 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 12/20 qcow2: Return error for snapshot operation with data file
In reply to Kevin WolfInternal snapshots and an external data file are incompatible because snapshots require refcounting and non-linear mapping. Return an error for all of the snapshot operations if an external data file is in use.
--- block/qcow2-snapshot.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
block/qcow2-snapshot.c +15 -0
@@ -353,6 +353,10 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)return -EFBIG ; } +if (has_data_file (bs)) { +return -ENOTSUP ; + } +memset (sn,0 ,sizeof (*sn));/* Generate an ID */
@@ -466,6 +470,10 @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char *snapshot_id)int ret;uint64_t *sn_l1_table =NULL ; +if (has_data_file (bs)) { +return -ENOTSUP ; + } +/* Search the snapshot */ snapshot_index =find_snapshot_by_id_or_name (bs, snapshot_id);if (snapshot_index <0 ) {
@@ -599,6 +607,10 @@ int qcow2_snapshot_delete(BlockDriverState *bs,QCowSnapshot sn;int snapshot_index, ret; +if (has_data_file (bs)) { +return -ENOTSUP ; + } +/* Search the snapshot */ snapshot_index =find_snapshot_by_id_and_name (bs, snapshot_id, name);if (snapshot_index <0 ) {
@@ -670,6 +682,9 @@ int qcow2_snapshot_list(BlockDriverState *bs, QEMUSnapshotInfo **psn_tab)QCowSnapshot *sn;int i; +if (has_data_file (bs)) { +return -ENOTSUP ; + }if (!s->nb_snapshots) { *psn_tab =NULL ;return s->nb_snapshots;
Signature · 1 line
2.20.1
PATCH 4/20 qcow2: Basic definitions for external data files
In reply to Kevin WolfThis adds basic constants, struct fields and helper function for external data file support to the implementation.
QCOW2_INCOMPAT_MASK and QCOW2_AUTOCLEAR_MASK are not updated yet so that opening images with an external data file still fails (we don't handle them correctly yet).
--- block/qcow2.h | 32 ++++++++++++++++++++++---------- block/qcow2.c | 9 +++++++++ tests/qemu-iotests/031.out | 8 ++++---- tests/qemu-iotests/036.out | 4 ++-- tests/qemu-iotests/061.out | 14 +++++++------- 5 files changed, 44 insertions(+), 23 deletions(-)
block/qcow2.h +22 -10
@@ -197,13 +197,15 @@ enum {/* Incompatible feature bits */ enum { -QCOW2_INCOMPAT_DIRTY_BITNR =0 , -QCOW2_INCOMPAT_CORRUPT_BITNR =1 , -QCOW2_INCOMPAT_DIRTY =1 <<QCOW2_INCOMPAT_DIRTY_BITNR , -QCOW2_INCOMPAT_CORRUPT =1 <<QCOW2_INCOMPAT_CORRUPT_BITNR , - -QCOW2_INCOMPAT_MASK =QCOW2_INCOMPAT_DIRTY - |QCOW2_INCOMPAT_CORRUPT , +QCOW2_INCOMPAT_DIRTY_BITNR =0 , +QCOW2_INCOMPAT_CORRUPT_BITNR =1 , +QCOW2_INCOMPAT_DATA_FILE_BITNR =2 , +QCOW2_INCOMPAT_DIRTY =1 <<QCOW2_INCOMPAT_DIRTY_BITNR , +QCOW2_INCOMPAT_CORRUPT =1 <<QCOW2_INCOMPAT_CORRUPT_BITNR , +QCOW2_INCOMPAT_DATA_FILE =1 <<QCOW2_INCOMPAT_DATA_FILE_BITNR , + +QCOW2_INCOMPAT_MASK =QCOW2_INCOMPAT_DIRTY + |QCOW2_INCOMPAT_CORRUPT , };/* Compatible feature bits */
@@ -216,10 +218,12 @@ enum {/* Autoclear feature bits */ enum { -QCOW2_AUTOCLEAR_BITMAPS_BITNR =0 , -QCOW2_AUTOCLEAR_BITMAPS =1 <<QCOW2_AUTOCLEAR_BITMAPS_BITNR , +QCOW2_AUTOCLEAR_BITMAPS_BITNR =0 , +QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR =1 , +QCOW2_AUTOCLEAR_BITMAPS =1 <<QCOW2_AUTOCLEAR_BITMAPS_BITNR , +QCOW2_AUTOCLEAR_DATA_FILE_RAW =1 <<QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR , -QCOW2_AUTOCLEAR_MASK =QCOW2_AUTOCLEAR_BITMAPS , +QCOW2_AUTOCLEAR_MASK =QCOW2_AUTOCLEAR_BITMAPS , };enum qcow2_discard_type {
@@ -340,6 +344,8 @@ typedef struct BDRVQcow2State {CoQueue compress_wait_queue;int nb_compress_threads; + +BdrvChild *data_file; } BDRVQcow2State; typedef struct Qcow2COWRegion {
@@ -457,6 +463,12 @@ typedef enum QCow2MetadataOverlap {#define REFT_OFFSET_MASK 0xfffffffffffffe00ULL +static inline bool has_data_file (BlockDriverState *bs) +{ +BDRVQcow2State *s = bs->opaque; +return (s->data_file != bs->file); +} +static inline int64_t start_of_cluster (BDRVQcow2State *s,int64_t offset) {return offset & ~(s->cluster_size -1 );
block/qcow2.c +9 -0
@@ -73,6 +73,7 @@ typedef struct {#define QCOW2_EXT_MAGIC_FEATURE_TABLE 0x6803f857#define QCOW2_EXT_MAGIC_CRYPTO_HEADER 0x0537be77#define QCOW2_EXT_MAGIC_BITMAPS 0x23852875 +#define QCOW2_EXT_MAGIC_DATA_FILE 0x44415441 static int coroutine_fn qcow2_co_preadv_compressed(BlockDriverState *bs,
@@ -1440,6 +1441,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, goto fail; } +/* TODO Open external data file */ + s->data_file = bs->file; +/* qcow2_read_extension may have set up the crypto context * if the crypt method needs a header region, some methods * don't need header extensions, so must check here
@@ -2428,6 +2432,11 @@ int qcow2_update_header(BlockDriverState *bs) .bit = QCOW2_INCOMPAT_CORRUPT_BITNR, .name = "corrupt bit", }, + { + .type = QCOW2_FEAT_TYPE_INCOMPATIBLE, + .bit = QCOW2_INCOMPAT_DATA_FILE_BITNR, + .name = "external data file", + }, { .type = QCOW2_FEAT_TYPE_COMPATIBLE, .bit = QCOW2_COMPAT_LAZY_REFCOUNTS_BITNR,
tests/qemu-iotests/031.out +4 -4
@@ -117,7 +117,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> Header extension:
@@ -150,7 +150,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> Header extension:
@@ -164,7 +164,7 @@ No errors were found on the image. magic 0x514649fb version 3 -backing_file_offset 0x148 +backing_file_offset 0x178 backing_file_size 0x17 cluster_bits 16 size 67108864
@@ -188,7 +188,7 @@ data 'host_device' Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> Header extension:
tests/qemu-iotests/036.out +2 -2
@@ -58,7 +58,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary>
@@ -86,7 +86,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> *** done
tests/qemu-iotests/061.out +7 -7
@@ -26,7 +26,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> magic 0x514649fb
@@ -84,7 +84,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> magic 0x514649fb
@@ -144,7 +144,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> ERROR cluster 5 refcount=0 reference=1
@@ -199,7 +199,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> magic 0x514649fb
@@ -268,7 +268,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> read 65536/65536 bytes at offset 44040192
@@ -306,7 +306,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> ERROR cluster 5 refcount=0 reference=1
@@ -335,7 +335,7 @@ header_length 104 Header extension: magic 0x6803f857 -length 144 +length 192 data <binary> read 131072/131072 bytes at offset 0
Signature · 1 line
2.20.1
PATCH 15/20 qcow2: Creating images with external data file
In reply to Kevin WolfThis adds a .bdrv_create option to use an external data file.
--- qapi/block-core.json | 4 ++++ include/block/block_int.h | 1 + block/qcow2.c | 26 ++++++++++++++++++++++++++ 3 files changed, 31 insertions(+)
qapi/block-core.json +4 -0
@@ -4135,6 +4135,9 @@ # Driver specific image creation options for qcow2. # # @file Node to create the image format on +# @data-file Node to use as an external data file in which all guest +# data is stored so that only metadata remains in the qcow2 +# file (since: 4.0) # @size Size of the virtual disk in bytes # @version Compatibility level (default: v3) # @backing-file File name of the backing file if a backing file
@@ -4150,6 +4153,7 @@ ## { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', + '*data-file': 'BlockdevRef', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str',
include/block/block_int.h +1 -0
@@ -56,6 +56,7 @@#define BLOCK_OPT_NOCOW "nocow"#define BLOCK_OPT_OBJECT_SIZE "object_size"#define BLOCK_OPT_REFCOUNT_BITS "refcount_bits" +#define BLOCK_OPT_DATA_FILE "data_file"#define BLOCK_PROBE_BUF_SIZE 512
block/qcow2.c +26 -0
@@ -2868,6 +2868,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp)*/ BlockBackend *blk =NULL ;BlockDriverState *bs =NULL ; +BlockDriverState *data_bs =NULL ;QCowHeader *header;size_t cluster_size;int version;
@@ -2964,6 +2965,20 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) } refcount_order =ctz32 (qcow2_opts->refcount_bits); +if (qcow2_opts->data_file) { +if (version <3 ) { +error_setg (errp,"External data files are only supported with " +"compatibility level 1.1 and above (use version=v3 or " +"greater)" ); + ret = -EINVAL ; + goto out; + } + data_bs =bdrv_open_blockdev_ref (qcow2_opts->data_file, errp); +if (bs ==NULL ) { + ret = -EIO ; + goto out; + } + }/* Create BlockBackend to write to the image */ blk =blk_new (BLK_PERM_WRITE |BLK_PERM_RESIZE ,BLK_PERM_ALL );
@@ -3002,6 +3017,10 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) header->compatible_features |=cpu_to_be64 (QCOW2_COMPAT_LAZY_REFCOUNTS ); } +if (data_bs) { + header->incompatible_features |= +cpu_to_be64 (QCOW2_INCOMPAT_DATA_FILE ); + } ret =blk_pwrite (blk,0 , header, cluster_size,0 );g_free (header);
@@ -3032,6 +3051,9 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) options =qdict_new ();qdict_put_str (options,"driver" ,"qcow2" );qdict_put_str (options,"file" , bs->node_name); +if (data_bs) { +qdict_put_str (options,"data-file" , data_bs->node_name); + } blk =blk_new_open (NULL ,NULL , options,BDRV_O_RDWR |BDRV_O_RESIZE |BDRV_O_NO_FLUSH , &local_err);
@@ -3104,6 +3126,9 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) options =qdict_new ();qdict_put_str (options,"driver" ,"qcow2" );qdict_put_str (options,"file" , bs->node_name); +if (data_bs) { +qdict_put_str (options,"data-file" , data_bs->node_name); + } blk =blk_new_open (NULL ,NULL , options,BDRV_O_RDWR |BDRV_O_NO_BACKING |BDRV_O_NO_IO , &local_err);
@@ -3117,6 +3142,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) out:blk_unref (blk);bdrv_unref (bs); +bdrv_unref (data_bs);return ret; }
Signature · 1 line
2.20.1
PATCH 11/20 qcow2: External file I/O
In reply to Kevin WolfThis changes the qcow2 implementation to direct all guest data I/O to s->data_file rather than bs->file, while metadata I/O still uses bs->file. At the moment, this is still always the same, but soon we'll add options to set s->data_file to an external data file.
--- block/qcow2.h | 2 +- block/qcow2-bitmap.c | 7 +++--- block/qcow2-cache.c | 6 ++--- block/qcow2-cluster.c | 46 +++++++++++++++++++++++++++++++------ block/qcow2-refcount.c | 39 +++++++++++++++++++++++-------- block/qcow2-snapshot.c | 7 +++--- block/qcow2.c | 52 +++++++++++++++++++++++++++++++++--------- 7 files changed, 122 insertions(+), 37 deletions(-)
block/qcow2.h +1 -1
@@ -622,7 +622,7 @@ void qcow2_process_discards(BlockDriverState *bs, int ret);int qcow2_check_metadata_overlap (BlockDriverState *bs,int ign,int64_t offset,int64_t size);int qcow2_pre_write_overlap_check (BlockDriverState *bs,int ign,int64_t offset, -int64_t size); +int64_t size,bool data_file); int qcow2_inc_refcounts_imrt(BlockDriverState *bs, BdrvCheckResult *res, void **refcount_table, int64_t *refcount_table_size,
block/qcow2-bitmap.c +4 -3
@@ -778,7 +778,8 @@ static int bitmap_list_store(BlockDriverState *bs, Qcow2BitmapList *bm_list,* directory in-place (actually, turn-off the extension), which is checked * in qcow2_check_metadata_overlap() */ ret =qcow2_pre_write_overlap_check ( - bs, in_place ?QCOW2_OL_BITMAP_DIRECTORY :0 , dir_offset, dir_size); + bs, in_place ?QCOW2_OL_BITMAP_DIRECTORY :0 , dir_offset, dir_size, + false);if (ret <0 ) { goto fail; }
@@ -1224,7 +1225,7 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,memset (buf + write_size,0 , s->cluster_size - write_size); } - ret =qcow2_pre_write_overlap_check (bs,0 , off, s->cluster_size); + ret =qcow2_pre_write_overlap_check (bs,0 , off, s->cluster_size, false);if (ret <0 ) {error_setg_errno (errp, -ret,"Qcow2 overlap check failed" ); goto fail;
@@ -1292,7 +1293,7 @@ static int store_bitmap(BlockDriverState *bs, Qcow2Bitmap *bm, Error **errp) } ret =qcow2_pre_write_overlap_check (bs,0 , tb_offset, - tb_size *sizeof (tb[0 ])); + tb_size *sizeof (tb[0 ]), false);if (ret <0 ) {error_setg_errno (errp, -ret,"Qcow2 overlap check failed" ); goto fail;
block/qcow2-cache.c +3 -3
@@ -205,13 +205,13 @@ static int qcow2_cache_entry_flush(BlockDriverState *bs, Qcow2Cache *c, int i)if (c == s->refcount_block_cache) { ret =qcow2_pre_write_overlap_check (bs,QCOW2_OL_REFCOUNT_BLOCK , - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); }else if (c == s->l2_table_cache) { ret =qcow2_pre_write_overlap_check (bs,QCOW2_OL_ACTIVE_L2 , - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); }else { ret =qcow2_pre_write_overlap_check (bs,0 , - c->entries[i].offset, c->table_size); + c->entries[i].offset, c->table_size, false); }if (ret <0 ) {
block/qcow2-cluster.c +39 -7
117 lines not on this page · raw message
block/qcow2-refcount.c +30 -9
@@ -1156,8 +1156,20 @@ void qcow2_free_any_clusters(BlockDriverState *bs, uint64_t l2_entry, int nb_clusters, enum qcow2_discard_type type) {BDRVQcow2State *s = bs->opaque; +QCow2ClusterType ctype =qcow2_get_cluster_type (bs, l2_entry); -switch (qcow2_get_cluster_type (bs, l2_entry)) { +if (has_data_file (bs)) { +if (s->discard_passthrough[type] && + (ctype ==QCOW2_CLUSTER_NORMAL || + ctype ==QCOW2_CLUSTER_ZERO_ALLOC )) + { +bdrv_pdiscard (s->data_file, l2_entry &L2E_OFFSET_MASK , + nb_clusters << s->cluster_bits); + } +return ; + } + +switch (ctype) {case QCOW2_CLUSTER_COMPRESSED : {int nb_csectors;
@@ -1649,7 +1661,7 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, l2_table[i] =cpu_to_be64 (l2_entry); ret =qcow2_pre_write_overlap_check (bs,QCOW2_OL_ACTIVE_L2 |QCOW2_OL_INACTIVE_L2 , - l2e_offset,sizeof (uint64_t )); + l2e_offset,sizeof (uint64_t ), false);if (ret <0 ) {fprintf (stderr,"ERROR: Overlap check failed\n" ); res->check_errors++;
@@ -1898,7 +1910,8 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res,if (l2_dirty) { ret =qcow2_pre_write_overlap_check (bs,QCOW2_OL_ACTIVE_L2 , - l2_offset, s->cluster_size); + l2_offset, s->cluster_size, + false);if (ret <0 ) {fprintf (stderr,"ERROR: Could not write L2 table; metadata " "overlap check failed: %s\n" ,strerror (-ret));
@@ -2366,7 +2379,7 @@ write_refblocks: } ret =qcow2_pre_write_overlap_check (bs,0 , refblock_offset, - s->cluster_size); + s->cluster_size, false);if (ret <0 ) {fprintf (stderr,"ERROR writing refblock: %s\n" ,strerror (-ret)); goto fail;
@@ -2417,7 +2430,8 @@ write_refblocks: } ret =qcow2_pre_write_overlap_check (bs,0 , reftable_offset, - reftable_size *sizeof (uint64_t )); + reftable_size *sizeof (uint64_t ), + false);if (ret <0 ) {fprintf (stderr,"ERROR writing reftable: %s\n" ,strerror (-ret)); goto fail;
@@ -2751,10 +2765,15 @@ QEMU_BUILD_BUG_ON(QCOW2_OL_MAX_BITNR != ARRAY_SIZE(metadata_ol_names));* overlaps; or a negative value (-errno) on error. */ int qcow2_pre_write_overlap_check (BlockDriverState *bs,int ign,int64_t offset, -int64_t size) +int64_t size,bool data_file) { -int ret =qcow2_check_metadata_overlap (bs, ign, offset, size); +int ret; + +if (data_file &&has_data_file (bs)) { +return 0 ; + } + ret =qcow2_check_metadata_overlap (bs, ign, offset, size);if (ret <0 ) {return ret; }else if (ret >0 ) {
@@ -2855,7 +2874,8 @@ static int flush_refblock(BlockDriverState *bs, uint64_t **reftable,if (reftable_index < *reftable_size && (*reftable)[reftable_index]) { offset = (*reftable)[reftable_index]; - ret =qcow2_pre_write_overlap_check (bs,0 , offset, s->cluster_size); + ret =qcow2_pre_write_overlap_check (bs,0 , offset, s->cluster_size, + false);if (ret <0 ) {error_setg_errno (errp, -ret,"Overlap check failed" );return ret;
@@ -3121,7 +3141,8 @@ int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,/* Write the new reftable */ ret =qcow2_pre_write_overlap_check (bs,0 , new_reftable_offset, - new_reftable_size *sizeof (uint64_t )); + new_reftable_size *sizeof (uint64_t ), + false);if (ret <0 ) {error_setg_errno (errp, -ret,"Overlap check failed" ); goto done;
block/qcow2-snapshot.c +4 -3
@@ -184,7 +184,7 @@ static int qcow2_write_snapshots(BlockDriverState *bs)/* The snapshot list position has not yet been updated, so these clusters * must indeed be completely free */ - ret =qcow2_pre_write_overlap_check (bs,0 , offset, snapshots_size); + ret =qcow2_pre_write_overlap_check (bs,0 , offset, snapshots_size, false);if (ret <0 ) { goto fail; }
@@ -389,7 +389,7 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info) } ret =qcow2_pre_write_overlap_check (bs,0 , sn->l1_table_offset, - s->l1_size *sizeof (uint64_t )); + s->l1_size *sizeof (uint64_t ), false);if (ret <0 ) { goto fail; }
@@ -528,7 +528,8 @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char *snapshot_id) } ret =qcow2_pre_write_overlap_check (bs,QCOW2_OL_ACTIVE_L1 , - s->l1_table_offset, cur_l1_bytes); + s->l1_table_offset, cur_l1_bytes, + false);if (ret <0 ) { goto fail; }
block/qcow2.c +41 -11
136 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH 16/20 qcow2: Store data file name in the image
In reply to Kevin WolfRather than requiring that the external data file node is passed explicitly when creating the qcow2 node, store the filename in the designated header extension during .bdrv_create and read it from there as a default during .bdrv_open.
--- qapi/block-core.json | 8 +++- block/qcow2.h | 1 + block/qcow2.c | 94 +++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/082.out | 27 +++++++++++ 4 files changed, 128 insertions(+), 2 deletions(-)
qapi/block-core.json +7 -1
@@ -59,6 +59,9 @@ # # @compat: compatibility level # +# @data-file: the filename of the external data file that is stored in the +# image and used as a default for opening the image (since: 4.0) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for
@@ -76,6 +79,7 @@ { 'struct': 'ImageInfoSpecificQCow2', 'data': { 'compat': 'str', + '*data-file': 'str', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int',
@@ -3082,7 +3086,9 @@ # # @data-file: reference to or definition of the external data file. # This may only be specified for images that require an -# external data file. (since 4.0) +# external data file. If it is not specified for such +# an image, the data file name is loaded from the image +# file. (since 4.0) # # Since: 2.9 ##
block/qcow2.h +1 -0
@@ -343,6 +343,7 @@ typedef struct BDRVQcow2State {* override) */ char *image_backing_file;char *image_backing_format; +char *image_data_file;CoQueue compress_wait_queue;int nb_compress_threads;
block/qcow2.c +93 -1
178 lines not on this page · raw message
tests/qemu-iotests/082.out +27 -0
@@ -48,6 +48,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -69,6 +70,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -90,6 +92,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -111,6 +114,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -132,6 +136,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -153,6 +158,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -174,6 +180,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -195,6 +202,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -231,6 +239,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -304,6 +313,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -325,6 +335,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -346,6 +357,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -367,6 +379,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -388,6 +401,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -409,6 +423,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -430,6 +445,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -451,6 +467,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -487,6 +504,7 @@ Supported options: backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -568,6 +586,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -590,6 +609,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -612,6 +632,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -634,6 +655,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -656,6 +678,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -678,6 +701,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -700,6 +724,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -722,6 +747,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -761,6 +787,7 @@ Creation options for 'qcow2': backing_fmt=<str> - Image format of the base image cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) + data_file=<str> - File name of an external data file encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
Signature · 1 line
2.20.1
PATCH 17/20 qcow2: Implement data-file-raw create option
In reply to Kevin WolfProvide an option to force QEMU to always keep the external data file consistent as a standalone read-only raw image.
At the moment, this means making sure that write_zeroes requests are forwarded to the data file instead of just updating the metadata, and checking that no backing file is used.
--- qapi/block-core.json | 9 ++++++ block/qcow2.h | 9 +++++- include/block/block_int.h | 1 + block/qcow2-cluster.c | 10 +++++++ block/qcow2.c | 56 ++++++++++++++++++++++++++++++++++++-- tests/qemu-iotests/082.out | 27 ++++++++++++++++++ 6 files changed, 109 insertions(+), 3 deletions(-)
qapi/block-core.json +9 -0
@@ -62,6 +62,10 @@ # @data-file: the filename of the external data file that is stored in the # image and used as a default for opening the image (since: 4.0) # +# @data-file-raw: True if the external data file must stay valid as a +# standalone (read-only) raw image without looking at qcow2 +# metadata (since: 4.0) +# # @lazy-refcounts: on or off; only valid for compat >= 1.1 # # @corrupt: true if the image has been marked corrupt; only valid for
@@ -80,6 +84,7 @@ 'data': { 'compat': 'str', '*data-file': 'str', + '*data-file-raw': 'bool', '*lazy-refcounts': 'bool', '*corrupt': 'bool', 'refcount-bits': 'int',
@@ -4144,6 +4149,9 @@ # @data-file Node to use as an external data file in which all guest # data is stored so that only metadata remains in the qcow2 # file (since: 4.0) +# @data-file-raw True if the external data file must stay valid as a +# standalone (read-only) raw image without looking at qcow2 +# metadata (default: false; since: 4.0) # @size Size of the virtual disk in bytes # @version Compatibility level (default: v3) # @backing-file File name of the backing file if a backing file
@@ -4160,6 +4168,7 @@ { 'struct': 'BlockdevCreateOptionsQcow2', 'data': { 'file': 'BlockdevRef', '*data-file': 'BlockdevRef', + '*data-file-raw': 'bool', 'size': 'size', '*version': 'BlockdevQcow2Version', '*backing-file': 'str',
block/qcow2.h +8 -1
@@ -225,7 +225,8 @@ enum {QCOW2_AUTOCLEAR_BITMAPS =1 <<QCOW2_AUTOCLEAR_BITMAPS_BITNR ,QCOW2_AUTOCLEAR_DATA_FILE_RAW =1 <<QCOW2_AUTOCLEAR_DATA_FILE_RAW_BITNR , - QCOW2_AUTOCLEAR_MASK = QCOW2_AUTOCLEAR_BITMAPS, +QCOW2_AUTOCLEAR_MASK =QCOW2_AUTOCLEAR_BITMAPS + | QCOW2_AUTOCLEAR_DATA_FILE_RAW, }; enum qcow2_discard_type {
@@ -474,6 +475,12 @@ static inline bool has_data_file(BlockDriverState *bs)return (s->data_file != bs->file); } +static inline bool data_file_is_raw (BlockDriverState *bs) +{ +BDRVQcow2State *s = bs->opaque; +return !!(s->autoclear_features &QCOW2_AUTOCLEAR_DATA_FILE_RAW ); +} +static inline int64_t start_of_cluster (BDRVQcow2State *s,int64_t offset) {return offset & ~(s->cluster_size -1 );
include/block/block_int.h +1 -0
@@ -57,6 +57,7 @@#define BLOCK_OPT_OBJECT_SIZE "object_size"#define BLOCK_OPT_REFCOUNT_BITS "refcount_bits"#define BLOCK_OPT_DATA_FILE "data_file" +#define BLOCK_OPT_DATA_FILE_RAW "data_file_raw"#define BLOCK_PROBE_BUF_SIZE 512
block/qcow2-cluster.c +10 -0
@@ -1783,6 +1783,16 @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,int64_t cleared;int ret; +/* If we have to stay in sync with an external data file, zero out +* s->data_file first. */ +if (data_file_is_raw (bs)) { +assert (has_data_file (bs)); + ret =bdrv_co_pwrite_zeroes (s->data_file, offset, bytes, flags); +if (ret <0 ) { +return ret; + } + } +/* Caller must pass aligned values, except at image end */ assert (QEMU_IS_ALIGNED (offset, s->cluster_size)); assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) ||
block/qcow2.c +54 -2
116 lines not on this page · raw message
tests/qemu-iotests/082.out +27 -0
@@ -49,6 +49,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -71,6 +72,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -93,6 +95,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -115,6 +118,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -137,6 +141,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -159,6 +164,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -181,6 +187,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -203,6 +210,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -240,6 +248,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -314,6 +323,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -336,6 +346,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -358,6 +369,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -380,6 +392,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -402,6 +415,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -424,6 +438,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -446,6 +461,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -468,6 +484,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -505,6 +522,7 @@ Supported options: cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -587,6 +605,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -610,6 +629,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -633,6 +653,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -656,6 +677,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -679,6 +701,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -702,6 +725,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -725,6 +749,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -748,6 +773,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
@@ -788,6 +814,7 @@ Creation options for 'qcow2': cluster_size=<size> - qcow2 cluster size compat=<str> - Compatibility level (0.10 or 1.1) data_file=<str> - File name of an external data file + data_file_raw=<bool (on/off)> - The external data file must stay valid as a raw image encrypt.cipher-alg=<str> - Name of encryption cipher algorithm encrypt.cipher-mode=<str> - Name of encryption cipher mode encrypt.format=<str> - Encrypt the image, format choices: 'aes', 'luks'
Signature · 1 line
2.20.1
PATCH 19/20 qemu-iotests: General tests for qcow2 with external data file
In reply to Kevin Wolf--- tests/qemu-iotests/244 | 200 +++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/244.out | 125 +++++++++++++++++++++++ tests/qemu-iotests/group | 1 + 3 files changed, 326 insertions(+) create mode 100755 tests/qemu-iotests/244 create mode 100644 tests/qemu-iotests/244.out
tests/qemu-iotests/244 new file +200 -0
@@ -0,0 +1,200 @@ +#!/bin/bash +# +# Test qcow2 with external data files +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator [email protected] + +seq=$(basename $0) +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + rm -f $TEST_IMG.data + rm -f $TEST_IMG.src +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux + +echo +echo "=== Create and open image with external data file ===" +echo + +echo "With data file name in the image:" +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M +_check_test_img + +$QEMU_IO -c "open $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "Data file required, but without data file name in the image:" +$QEMU_IMG amend -odata_file= $TEST_IMG + +$QEMU_IO -c "open $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "Setting data-file for an image with internal data:" +_make_test_img 64M + +$QEMU_IO -c "open -odata-file.filename=$TEST_IMG.data $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir +$QEMU_IO -c "open -odata-file.filename=inexistent $TEST_IMG" -c "read -P 0 0 64k" 2>&1 | _filter_qemu_io | _filter_testdir + +echo +echo "=== Conflicting features ===" +echo + +echo "Convert to compressed target with data file:" +TEST_IMG="$TEST_IMG.src" _make_test_img 64M + +$QEMU_IO -c 'write -P 0x11 0 1M' \ + -f $IMGFMT "$TEST_IMG.src" | + _filter_qemu_io + +$QEMU_IMG convert -f $IMGFMT -O $IMGFMT -c -odata_file="$TEST_IMG.data" \ + "$TEST_IMG.src" "$TEST_IMG" + +echo +echo "Convert uncompressed, then write compressed data manually:" +$QEMU_IMG convert -f $IMGFMT -O $IMGFMT -odata_file="$TEST_IMG.data" \ + "$TEST_IMG.src" "$TEST_IMG" +$QEMU_IMG compare "$TEST_IMG.src" "$TEST_IMG" + +$QEMU_IO -c 'write -c -P 0x22 0 1M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +echo "Take an internal snapshot:" + +$QEMU_IMG snapshot -c test "$TEST_IMG" +_check_test_img + +echo +echo "=== Standalone image with external data file (efficient) ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M + +echo -n "qcow2 file size before I/O: " +du -b $TEST_IMG | cut -f1 + +# Create image with the following layout +# 0-1 MB: Unallocated +# 1-2 MB: Written (pattern 0x11) +# 2-3 MB: Discarded +# 3-4 MB: Zero write over discarded space +# 4-5 MB: Zero write over written space +# 5-6 MB: Zero write over unallocated space + +echo +$QEMU_IO -c 'write -P 0x11 1M 4M' \ + -c 'discard 2M 2M' \ + -c 'write -z 3M 3M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +$QEMU_IMG map --output=json "$TEST_IMG" + +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 4M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io + +# Zero clusters are only marked as such in the qcow2 metadata, but contain +# stale data in the external data file +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 2M' \ + -c 'read -P 0x11 4M 1M' \ + -c 'read -P 0 5M 1M' \ + -f raw "$TEST_IMG.data" | + _filter_qemu_io + + +echo -n "qcow2 file size after I/O: " +du -b $TEST_IMG | cut -f1 + +echo +echo "=== Standalone image with external data file (valid raw) ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data,data_file_raw=on" _make_test_img 64M + +echo -n "qcow2 file size before I/O: " +du -b $TEST_IMG | cut -f1 + +echo +$QEMU_IO -c 'write -P 0x11 1M 4M' \ + -c 'discard 2M 2M' \ + -c 'write -z 3M 3M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io +_check_test_img + +echo +$QEMU_IMG map --output=json "$TEST_IMG" + +echo +$QEMU_IO -c 'read -P 0 0 1M' \ + -c 'read -P 0x11 1M 1M' \ + -c 'read -P 0 2M 4M' \ + -f $IMGFMT "$TEST_IMG" | + _filter_qemu_io + +echo +$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.data" + +echo -n "qcow2 file size after I/O: " +du -b $TEST_IMG | cut -f1 + +echo +echo "=== bdrv_co_block_status test for file and offset=0 ===" +echo + +IMGOPTS="data_file=$TEST_IMG.data" _make_test_img 64M + +$QEMU_IO -c 'write -P 0x11 0 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io +$QEMU_IO -c 'read -P 0x11 0 1M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io +$QEMU_IMG map --output=human "$TEST_IMG" | _filter_testdir +$QEMU_IMG map --output=json "$TEST_IMG" + +# success, all done +echo "*** done" +rm -f $seq.full +status=0
tests/qemu-iotests/244.out new file +125 -0
125 lines not on this page · raw message
tests/qemu-iotests/group +1 -0
@@ -242,3 +242,4 @@ 240 auto quick 242 rw auto quick 243 rw auto quick +244 rw auto quick
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH 03/20] qcow2: Extend spec for external data files
In reply to Kevin WolfQuoted 37 lines · Kevin Wolf
This adds external data file to the qcow2 spec as a new incompatible feature.
Signed-off-by Kevin Wolf <[email protected]>--- docs/interop/qcow2.txt | 38 ++++++++++++++++++++++++++++++++++---- 1 file changed, 34 insertions(+), 4 deletions(-)docs/interop/qcow2.txt +13
-1@@ -97,7 +97,19 @@ in the description of a field. be written to (unless for regaining consistency). - Bits 2-63: Reserved (set to 0) + Bit 2: External data file bit. If this bit is set, an + external data file is used. Guest clusters are + then stored in the external data file. For such + images, clusters in the external data file are + not refcounted. The offset field in the + Standard Cluster Descriptor must match the + guest offset and neither compressed clusters + nor internal snapshots are supported. + + An external data file name header extension may + be present if this bit is set. + + Bits 3-63: Reserved (set to 0) 80 - 87: compatible_features Bitmask of compatible features. An implementation can@@ -126,7 +138,17 @@ in the description of a field. bit is unset, the bitmaps extension data must be considered inconsistent.
docs/interop/qcow2.txt in the description of a field. · line 149
Quoted 10 lines
docs/interop/qcow2.txt +9
-1- Bits 1-63: Reserved (set to 0) + Bit 1: If this bit is set, the external data file can + be read as a consistent standalone raw image + without looking at the qcow2 metadata. + + Setting this bit has a performance impact for + some operations on the image (e.g. writing + zeros requires writing to the data file instead + of only setting the zero flag in the L2 table + entry) and conflicts with backing files.
Is it an error if an image has this bit set but not the 'external data file bit' in the incompatible bits?
docs/interop/qcow2.txt be stored. Each extension has a structure like the following: · line 173
Quoted 10 lines
docs/interop/qcow2.txt +3
-0+ + Bits 2-63: Reserved (set to 0) 96 - 99: refcount_order Describes the width of a reference count block entry (width@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file name
Likewise, should it be an explicit error if this header is present without the 'external data file bit'?
Quoted 16 lines
docs/interop/qcow2.txt +5
-0other - Unknown header extension, can be safely ignored@@ -437,6 +460,11 @@ L2 table entry: This information is only accurate in L2 tables that are reachable from the active L1 table. + With external data files, all guest clusters have an + implicit refcount of 1 (because of the fixed host = guest + mapping for guest cluster offsets), so this bit should be 1 + for all allocated clusters. + Standard Cluster Descriptor: Bit 0: If set to 1, the cluster reads as all zeros. The host
docs/interop/qcow2.txt Standard Cluster Descriptor: · line 484
Quoted 9 lines
docs/interop/qcow2.txt +4
-2@@ -450,8 +478,10 @@ Standard Cluster Descriptor: 1 - 8: Reserved (set to 0) 9 - 55: Bits 9-55 of host cluster offset. Must be aligned to a - cluster boundary. If the offset is 0, the cluster is - unallocated. + cluster boundary. If the offset is 0 and bit 63 is clear, + the cluster is unallocated. The offset may only be 0 with + bit 63 set (indicating a host cluster offset of 0) when an + external data file is used.
Looks good, modulo any additions you want to make based on answering my questions above.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [Qemu-block] [PATCH 03/20] qcow2: Extend spec for external data files
In reply to Kevin Wolfdocs/interop/qcow2.txt in the description of a field. · line 110
Quoted 12 lines · Kevin Wolf
docs/interop/qcow2.txt +11
-1- Bits 2-63: Reserved (set to 0) + Bit 2: External data file bit. If this bit is set, an + external data file is used. Guest clusters are + then stored in the external data file. For such + images, clusters in the external data file are + not refcounted. The offset field in the + Standard Cluster Descriptor must match the + guest offset and neither compressed clusters + nor internal snapshots are supported. + + An external data file name header extension may + be present if this bit is set.
This sentence is clearer if the header extension name is made prominent:
An "external data file name" header extension
Or
An External Data File Name header extension
(In the previous paragraph Standard Cluster Descriptor is also capitalized.)
docs/interop/qcow2.txt be stored. Each extension has a structure like the following: · line 173
Quoted 5 lines
docs/interop/qcow2.txt +1
-0@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file name
This new header extension isn't described in this patch?
Re: [Qemu-devel] [Qemu-block] [PATCH 03/20] qcow2: Extend spec for external data files
Quoted 16 lines · Stefan Hajnoczi
docs/interop/qcow2.txt in the description of a field. · line 110
Quoted 12 lines · Kevin Wolf
docs/interop/qcow2.txt +11
-1- Bits 2-63: Reserved (set to 0) + Bit 2: External data file bit. If this bit is set, an + external data file is used. Guest clusters are + then stored in the external data file. For such + images, clusters in the external data file are + not refcounted. The offset field in the + Standard Cluster Descriptor must match the + guest offset and neither compressed clusters + nor internal snapshots are supported. + + An external data file name header extension may + be present if this bit is set.This sentence is clearer if the header extension name is made prominent:
Quoted 8 lines
An "external data file name" header extension
Or
An External Data File Name header extension
(In the previous paragraph Standard Cluster Descriptor is also capitalized.)
Indeed, so I like the latter suggestion.
Quoted 8 lines
docs/interop/qcow2.txt be stored. Each extension has a structure like the following: · line 173
Quoted 6 lines
docs/interop/qcow2.txt +1
-0@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file nameThis new header extension isn't described in this patch?
I asked the same on v1, and the answer (which remains valid) is that neither is 0xe2792aca Backing file format name. (In other words, both extensions are simple enough as a single file name to be implicitly described by the reference to the header in the earlier text). Making both explicit wouldn't hurt my feelings, but I don't see it as a showstopper to the patch as-is.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 01/20] qemu-iotests: Test qcow2 preallocation modes
In reply to Kevin Wolftests/qemu-iotests/243 line 51
Quoted 9 lines · Kevin Wolf
Signed-off-by Kevin Wolf <[email protected]>---
+++ b/tests/qemu-iotests/243
tests/qemu-iotests/243 new file +2
-0@@ -0,0 +1,63 @@ + + echo -n "File size: "
echo -n is non-portable (bash has shopt -s xpg_echo); better is to just use printf. See commit b43671f8
tests/qemu-iotests/243 line 55
Quoted 4 lines
tests/qemu-iotests/243 new file +4
-0+ du -b $TEST_IMG | cut -f1 + + # Can't use precise numbers here because they differ between filesystems + echo -n "Disk usage: "
and again
Quoted 1 line
+ [ $(du -B1 $TEST_IMG | cut -f1) -lt 1048576 ] && echo "low" || echo "high"
du -b/-B are not portable to POSIX, but you did limit the test to just Linux where they are fairly likely, so I don't have any better suggestions. I like the masking into high/low to work around file system granularity quirks as well as image metadata size differences.
As using printf instead of echo -n is trivial,
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [Qemu-block] [PATCH 03/20] qcow2: Extend spec for external data files
In reply to Eric BlakeQuoted 10 lines · Eric Blake
Quoted 8 lines · Stefan Hajnoczi
docs/interop/qcow2.txt be stored. Each extension has a structure like the following: · line 173
Quoted 5 lines · Kevin Wolf
docs/interop/qcow2.txt +1
-0@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file nameThis new header extension isn't described in this patch?
Quoted 6 lines
I asked the same on v1, and the answer (which remains valid) is that neither is
0xe2792acaBacking file format name. (In other words, both extensions are simple enough as a single file name to be implicitly described by the reference to the header in the earlier text). Making both explicit wouldn't hurt my feelings, but I don't see it as a showstopper to the patch as-is.
The spec should make the representation clear. Is it a NUL-terminated string or is the length dictated by the header extension length field?
Otherwise implementors are forced to look at the QEMU source code or guess based on hex dumping example files :(.
Stefan
Re: [Qemu-devel] [Qemu-block] [PATCH 03/20] qcow2: Extend spec for external data files
Quoted 18 lines · Stefan Hajnoczi
Quoted 10 lines · Eric Blake
Quoted 8 lines · Stefan Hajnoczi
docs/interop/qcow2.txt be stored. Each extension has a structure like the following: · line 173
Quoted 5 lines · Kevin Wolf
docs/interop/qcow2.txt +1
-0@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file nameThis new header extension isn't described in this patch?
Quoted 6 lines
I asked the same on v1, and the answer (which remains valid) is that neither is
0xe2792acaBacking file format name. (In other words, both extensions are simple enough as a single file name to be implicitly described by the reference to the header in the earlier text). Making both explicit wouldn't hurt my feelings, but I don't see it as a showstopper to the patch as-is.
Quoted 2 lines
The spec should make the representation clear. Is it a NUL-terminated string or is the length dictated by the header extension length field?
My understanding is length determined by the header field, with optional NUL padding out to the alignment boundary (but that also means that it does NOT necessarily have a trailing NUL on disk if sizing matches alignment). But yes, being explicit never hurts.
Quoted 3 lines
Otherwise implementors are forced to look at the QEMU source code or guess based on hex dumping example files :(.
Indeed, cleaning up the existing Backing file format name is worth doing (at which point this should follow suit). But it still sounds like a separate patch, at which point it becomes a question of ordering - if the cleanup lands first, then this needs to rebase to do the same; if this lands first, then the cleanup does both headers at once.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [Qemu-block] [PATCH 03/20] qcow2: Extend spec for external data files
Am 06.03.2019 um 13:43 hat Eric Blake geschrieben:
Quoted 28 lines · Stefan Hajnoczi
Quoted 18 lines · Eric Blake
Quoted 10 lines · Stefan Hajnoczi
Quoted 8 lines · Kevin Wolf
docs/interop/qcow2.txt be stored. Each extension has a structure like the following: · line 173
Quoted 5 lines
docs/interop/qcow2.txt +1
-0@@ -148,6 +170,7 @@ be stored. Each extension has a structure like the following: 0x6803f857 - Feature name table 0x23852875 - Bitmaps extension 0x0537be77 - Full disk encryption header pointer + 0x44415441 - External data file nameThis new header extension isn't described in this patch?
Quoted 6 lines
I asked the same on v1, and the answer (which remains valid) is that neither is
0xe2792acaBacking file format name. (In other words, both extensions are simple enough as a single file name to be implicitly described by the reference to the header in the earlier text). Making both explicit wouldn't hurt my feelings, but I don't see it as a showstopper to the patch as-is.Quoted 2 lines
The spec should make the representation clear. Is it a NUL-terminated string or is the length dictated by the header extension length field?
My understanding is length determined by the header field, with optional NUL padding out to the alignment boundary (but that also means that it does NOT necessarily have a trailing NUL on disk if sizing matches alignment). But yes, being explicit never hurts.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
Otherwise implementors are forced to look at the QEMU source code or guess based on hex dumping example files :(.
Indeed, cleaning up the existing Backing file format name is worth doing (at which point this should follow suit). But it still sounds like a separate patch, at which point it becomes a question of ordering - if the cleanup lands first, then this needs to rebase to do the same; if this lands first, then the cleanup does both headers at once.
Maybe add a new section "String header extensions" that covers both?
If this remains the only patch in the series that would need a significant change, I'd prefer a follow-up patch indeed.
Kevin
Re: [Qemu-devel] [Qemu-block] [PATCH 03/20] qcow2: Extend spec for external data files
Quoted 4 lines · Kevin Wolf
Maybe add a new section "String header extensions" that covers both?
If this remains the only patch in the series that would need a significant change, I'd prefer a follow-up patch indeed.
A follow-up patch sounds good.
Stefan
Am 27.02.2019 um 18:22 hat Kevin Wolf geschrieben:
Quoted 15 lines
There are use cases where raw images are given (e.g. existing physical disks), but advanced features like dirty bitmaps or backing files are wanted that require use of a proper image format like qcow2.
This series adds an incompatible feature bit to qcow2 which allows to use an external data file: Metadata is kept in the qcow2 file like usual, but guest data is written to an external file. Clusters in the data file are not reference counted, instead we use a flat layout where host cluster offset == guest cluster offset. The external data file is therefore readable as a raw image (though writing to it invalidates the associated qcow2 metadata). Features that require refcounting such as internal snapshots or compression are not supposed in such setups.
Major changes since the RFC (many more minor changes):
Quoted 10 lines
- Added a 'data-file-raw' flag to the spec to keep the data file a valid self-consistent raw image (
write_zeroesmust always be propagated)
- Added QAPI documentation
- Added some test cases
- Implemented data file support or error paths for discard, check, compressed writes, snapshots
- Fixedqcow2_co_block_status()
- Rearrangedqcow2_do_open()code for data files for better error handling (e.g. no more fallback to default data file if an explicit, but invalid data-file option is given)
Thanks, made the trivial changes suggested during review and applied to the block branch.
Kevin