qcow2: include LUKS payload overhead in qemu-img measure
v2 cover qcow2: include LUKS payload overhead in qemu-img measure
v2:
* Take into account all "encrypt." image creation options since the LUKS payload size varies depending on the cipher [Max]
The LUKS payload has a significant size (>1 MB). It must be included in the qemu-img measure calculation so we arrive at the correct minimum image size.
Stefan Hajnoczi (2):
qcow2: include LUKS payload overhead in qemu-img measure
iotests: add LUKS payload overhead to 178 qemu-img measure test
block/qcow2.c | 72 +++++++++++++++++++++++++++++++- tests/qemu-iotests/178 | 8 ++++ tests/qemu-iotests/178.out.qcow2 | 24 +++++++++++ 3 files changed, 103 insertions(+), 1 deletion(-)
Signature · 1 line
2.20.1
PATCH v2 1/2 qcow2: include LUKS payload overhead in qemu-img measure
LUKS encryption reserves clusters for its own payload data. The size of this area must be included in the qemu-img measure calculation so that we arrive at the correct minimum required image size.
(Ab)use the qcrypto_block_create() API to determine the payload overhead. We discard the payload data that qcrypto thinks will be written to the image.
--- block/qcow2.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-)
block/qcow2.c +71 -1
@@ -4231,6 +4231,60 @@ static coroutine_fn int qcow2_co_flush_to_os(BlockDriverState *bs)return ret; } +static ssize_t qcow2_measure_crypto_hdr_init_func (QCryptoBlock *block, +size_t headerlen,void *opaque,Error **errp) +{ +size_t *headerlenp = opaque; + +/* Stash away the payload size */ + *headerlenp = headerlen; +return 0 ; +} + +static ssize_t qcow2_measure_crypto_hdr_write_func (QCryptoBlock *block, +size_t offset,const uint8_t *buf,size_t buflen, +void *opaque,Error **errp) +{ +/* Discard the bytes, we're not actually writing to an image */ +return buflen; +} + +/* Determine the number of bytes for the LUKS payload */ +static bool qcow2_measure_luks_headerlen (QemuOpts *opts,size_t *len, +Error **errp) +{ +QDict *opts_qdict; +QDict *cryptoopts_qdict; +QCryptoBlockCreateOptions *cryptoopts; +QCryptoBlock *crypto; + +/* Extract "encrypt." options into a qdict */ + opts_qdict =qemu_opts_to_qdict (opts,NULL ); +qdict_extract_subqdict (opts_qdict, &cryptoopts_qdict,"encrypt." ); +qobject_unref (opts_qdict); + +/* Build QCryptoBlockCreateOptions object from qdict */ +qdict_put_str (cryptoopts_qdict,"format" ,"luks" ); + cryptoopts =block_crypto_create_opts_init (cryptoopts_qdict, errp); +qobject_unref (cryptoopts_qdict); +if (!cryptoopts) { +return false; + } + +/* Fake LUKS creation in order to determine the payload size */ + crypto =qcrypto_block_create (cryptoopts,"encrypt." , + qcow2_measure_crypto_hdr_init_func, + qcow2_measure_crypto_hdr_write_func, + len, errp); +qapi_free_QCryptoBlockCreateOptions (cryptoopts); +if (!crypto) { +return false; + } + +qcrypto_block_free (crypto); +return true; +} +static BlockMeasureInfo *qcow2_measure (QemuOpts *opts,BlockDriverState *in_bs,Error **errp) {
@@ -4240,11 +4294,13 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs,uint64_t virtual_size;/* disk size as seen by guest */ uint64_t refcount_bits;uint64_t l2_tables; +uint64_t luks_payload_size =0 ;size_t cluster_size;int version;char *optstr;PreallocMode prealloc;bool has_backing_file; +bool has_luks;/* Parse image creation options */ cluster_size =qcow2_opt_get_cluster_size_del (opts, &local_err);
@@ -4274,6 +4330,20 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, has_backing_file = !!optstr;g_free (optstr); + optstr =qemu_opt_get_del (opts,BLOCK_OPT_ENCRYPT_FORMAT ); + has_luks = optstr &&strcmp (optstr,"luks" ) ==0 ; +g_free (optstr); + +if (has_luks) { +size_t headerlen; + +if (!qcow2_measure_luks_headerlen (opts, &headerlen, &local_err)) { + goto err; + } + + luks_payload_size =ROUND_UP (headerlen, cluster_size); + } + virtual_size =qemu_opt_get_size_del (opts,BLOCK_OPT_SIZE ,0 ); virtual_size =ROUND_UP (virtual_size, cluster_size);
@@ -4344,7 +4414,7 @@ static BlockMeasureInfo *qcow2_measure(QemuOpts *opts, BlockDriverState *in_bs, info =g_new (BlockMeasureInfo,1 ); info->fully_allocated =qcow2_calc_prealloc_size (virtual_size, cluster_size, -ctz32 (refcount_bits)); +ctz32 (refcount_bits)) + luks_payload_size;/* Remove data clusters that are not required. This overestimates the * required size because metadata needed for the fully allocated file is
Signature · 1 line
2.20.1
Quoted 4 lines · Stefan Hajnoczi
LUKS encryption reserves clusters for its own payload data. The size of this area must be included in the qemu-img measure calculation so that we arrive at the correct minimum required image size.
Quoted 8 lines
(Ab)use the
qcrypto_block_create()API to determine the payload overhead. We discard the payload data that qcrypto thinks will be written to the image.Signed-off-by Stefan Hajnoczi <[email protected]>--- block/qcow2.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-)
PATCH v2 2/2 iotests: add LUKS payload overhead to 178 qemu-img measure test
In reply to Stefan HajnocziThe previous patch includes the LUKS payload overhead into the qemu-img measure calculation for qcow2. Update qemu-iotests 178 to exercise this new code path.
--- tests/qemu-iotests/178 | 8 ++++++++ tests/qemu-iotests/178.out.qcow2 | 24 ++++++++++++++++++++++++ 2 files changed, 32 insertions(+)
tests/qemu-iotests/178 +8 -0
@@ -142,6 +142,14 @@ for ofmt in human json; do # The backing file doesn't need to exist :) $QEMU_IMG measure --output=$ofmt -o backing_file=x \ -f "$fmt" -O "$IMGFMT" "$TEST_IMG" + + echo + echo "== $fmt input image and LUKS encryption ==" + echo + $QEMU_IMG measure --output=$ofmt \ + --object secret,id=sec0,data=base \ + -o encrypt.format=luks,encrypt.key-secret=sec0 \ + -f "$fmt" -O "$IMGFMT" "$TEST_IMG" fi echo
tests/qemu-iotests/178.out.qcow2 +24 -0
@@ -68,6 +68,11 @@ converted image file size in bytes: 458752 required size: 1074135040 fully allocated size: 1074135040 +== qcow2 input image and LUKS encryption == + +required size: 2686976 +fully allocated size: 1076232192 + == qcow2 input image and preallocation (human) == required size: 1074135040
@@ -114,6 +119,11 @@ converted image file size in bytes: 524288 required size: 1074135040 fully allocated size: 1074135040 +== raw input image and LUKS encryption == + +required size: 2686976 +fully allocated size: 1076232192 + == raw input image and preallocation (human) == required size: 1074135040
@@ -205,6 +215,13 @@ converted image file size in bytes: 458752 "fully-allocated": 1074135040 } +== qcow2 input image and LUKS encryption == + +{ + "required": 2686976, + "fully-allocated": 1076232192 +} + == qcow2 input image and preallocation (json) == {
@@ -263,6 +280,13 @@ converted image file size in bytes: 524288 "fully-allocated": 1074135040 } +== raw input image and LUKS encryption == + +{ + "required": 2686976, + "fully-allocated": 1076232192 +} + == raw input image and preallocation (json) == {
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v2 2/2] iotests: add LUKS payload overhead to 178 qemu-img measure test
Quoted 18 lines · Stefan Hajnoczi
The previous patch includes the LUKS payload overhead into the qemu-img measure calculation for qcow2. Update qemu-iotests 178 to exercise this new code path.
Reviewed-by Max Reitz <[email protected]>Reviewed-by Philippe Mathieu-Daudé <[email protected]>Signed-off-by Stefan Hajnoczi <[email protected]>--- tests/qemu-iotests/178 | 8 ++++++++ tests/qemu-iotests/178.out.qcow2 | 24 ++++++++++++++++++++++++ 2 files changed, 32 insertions(+)tests/qemu-iotests/178 +0
-0@@ -142,6 +142,14 @@ for ofmt in human json; do # The backing file doesn't need to exist :)
tests/qemu-iotests/178 for ofmt in human json; do · line 152
Quoted 10 lines
tests/qemu-iotests/178 +8
-0$QEMU_IMG measure --output=$ofmt -o backing_file=x \ -f "$fmt" -O "$IMGFMT" "$TEST_IMG" + + echo + echo "== $fmt input image and LUKS encryption ==" + echo + $QEMU_IMG measure --output=$ofmt \ + --object secret,id=sec0,data=base \ + -o encrypt.format=luks,encrypt.key-secret=sec0 \ + -f "$fmt" -O "$IMGFMT" "$TEST_IMG"
With the change in patch 1, it would be possible to set encrypt.iter-time=10, for example (default for the LUKS tests). For me, that decreases the test's runtime from 24 to 15 seconds.
Pretty please? :-)
Max
Quoted 3 lines
tests/qemu-iotests/178 +0
-0fi echo
Re: [Qemu-devel] [PATCH v2 2/2] iotests: add LUKS payload overhead to 178 qemu-img measure test
Quoted 22 lines · Max Reitz
Quoted 6 lines · Stefan Hajnoczi
tests/qemu-iotests/178 +0
-0@@ -142,6 +142,14 @@ for ofmt in human json; do # The backing file doesn't need to exist :)tests/qemu-iotests/178 for ofmt in human json; do · line 152
Quoted 10 lines
tests/qemu-iotests/178 +8
-0$QEMU_IMG measure --output=$ofmt -o backing_file=x \ -f "$fmt" -O "$IMGFMT" "$TEST_IMG" + + echo + echo "== $fmt input image and LUKS encryption ==" + echo + $QEMU_IMG measure --output=$ofmt \ + --object secret,id=sec0,data=base \ + -o encrypt.format=luks,encrypt.key-secret=sec0 \ + -f "$fmt" -O "$IMGFMT" "$TEST_IMG"With the change in patch 1, it would be possible to set encrypt.iter-time=10, for example (default for the LUKS tests). For me, that decreases the test's runtime from 24 to 15 seconds.
Quoted 1 line
Pretty please? :-)
Will fix in v3.
Stefan