vmdk: Fix zero cluster handling
cover vmdk: Fix zero cluster handling
What I was really investigating is why 055 was so slow. I couldn't solve that, but instead I found out that our VMDK code for zero clusters and write_zeroes was completely broken. Apart from segfaults when zero clusters were actually enabled, this caused a compressed backup target to result in a bigger file than uncompressed with VMDK.
This series tries to fix it (with one bonus performance patch).
Kevin Wolf (6):
vmdk: Rename VmdkMetaData.valid to new_allocation
vmdk: Fix zero cluster allocation
vmdk: Fix partial overwrite of zero cluster
vmdk: Don't update L2 table for zero write on zero cluster
vmdk: Flush only once in vmdk_L2update()
iotests: vmdk: Enable zeroed_grained=on by default
block/vmdk.c | 47 +++++++++++++++++++++++++--------------- tests/qemu-iotests/059 | 6 ++--- tests/qemu-iotests/check | 3 +++ 3 files changed, 35 insertions(+), 21 deletions(-)
Signature · 1 line
2.25.3
PATCH 1/6 vmdk: Rename VmdkMetaData.valid to new_allocation
m_data is used for zero clusters even though valid == 0. It really only means that a new cluster was allocated in the image file. Rename it to reflect this.
While at it, change it from int to bool, too.
--- block/vmdk.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
block/vmdk.c +4 -4
@@ -180,7 +180,7 @@ typedef struct VmdkMetaData {unsigned int l1_index;unsigned int l2_index;unsigned int l2_offset; -int valid; +bool new_allocation;uint32_t *l2_cache_entry; } VmdkMetaData;
@@ -1492,7 +1492,7 @@ static int get_cluster_offset(BlockDriverState *bs,unsigned int l2_size_bytes = extent->l2_size * extent->entry_size;if (m_data) { - m_data->valid =0 ; + m_data->new_allocation = false; }if (extent->flat) { *cluster_offset = extent->flat_start_offset;
@@ -1630,7 +1630,7 @@ static int get_cluster_offset(BlockDriverState *bs,return ret; }if (m_data) { - m_data->valid =1 ; + m_data->new_allocation = true; m_data->l1_index = l1_index; m_data->l2_index = l2_index; m_data->l2_offset = l2_offset;
@@ -2021,7 +2021,7 @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset,if (ret) {return ret; } -if (m_data.valid) { +if (m_data.new_allocation) {/* update L2 tables */ if (vmdk_L2update (extent, &m_data, cluster_offset >> BDRV_SECTOR_BITS)
Signature · 1 line
2.25.3
Re: [PATCH 1/6] vmdk: Rename VmdkMetaData.valid to new_allocation
Quoted 11 lines · Kevin Wolf
m_datais used for zero clusters even though valid == 0. It really only means that a new cluster was allocated in the image file. Rename it to reflect this.While at it, change it from int to bool, too.
Signed-off-by Kevin Wolf <[email protected]>--- block/vmdk.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 2/6 vmdk: Fix zero cluster allocation
In reply to Kevin Wolfm_data must be contain valid data even for zero clusters when no cluster was allocated in the image file. Without this, zero writes segfault with images that have zeroed_grain=on.
For zero writes, we don't want to allocate a cluster in the image file even in compressed files.
--- block/vmdk.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
block/vmdk.c +7 -5
@@ -1572,6 +1572,12 @@ static int get_cluster_offset(BlockDriverState *bs, extent->l2_cache_counts[min_index] =1 ; found: l2_index = ((offset >>9 ) / extent->cluster_sectors) % extent->l2_size; +if (m_data) { + m_data->l1_index = l1_index; + m_data->l2_index = l2_index; + m_data->l2_offset = l2_offset; + m_data->l2_cache_entry = ((uint32_t *)l2_table) + l2_index; + }if (extent->sesparse) { cluster_sector =le64_to_cpu (((uint64_t *)l2_table)[l2_index]);
@@ -1631,10 +1637,6 @@ static int get_cluster_offset(BlockDriverState *bs, }if (m_data) { m_data->new_allocation = true; - m_data->l1_index = l1_index; - m_data->l2_index = l2_index; - m_data->l2_offset = l2_offset; - m_data->l2_cache_entry = ((uint32_t *)l2_table) + l2_index; } } *cluster_offset = cluster_sector <<BDRV_SECTOR_BITS ;
@@ -1990,7 +1992,7 @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset,error_report ("Could not write to allocated cluster" " for streamOptimized" );return -EIO ; - }else { + }else if (!zeroed) {/* allocate */ ret =get_cluster_offset (bs, extent, &m_data, offset, true, &cluster_offset,0 ,0 );
Signature · 1 line
2.25.3
Re: [PATCH 2/6] vmdk: Fix zero cluster allocation
Quoted 1 line · Kevin Wolf
m_datamust be contain valid data even for zero clusters when no cluster
s/be //
Quoted 7 lines
was allocated in the image file. Without this, zero writes segfault with images that have zeroed_grain=on.
For zero writes, we don't want to allocate a cluster in the image file even in compressed files.
Fixes 524089bce43fd1cd3daaca979872451efa2cf7c6
Nearly 4 years ago, and claims to fix a different regression. Wow, we aren't doing too well with vmdk.
Quoted 5 lines
Signed-off-by Kevin Wolf <[email protected]>--- block/vmdk.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH 2/6] vmdk: Fix zero cluster allocation
In reply to Kevin WolfQuoted 3 lines · Kevin Wolf
m_datamust be contain valid data even for zero clusters when no cluster was allocated in the image file. Without this, zero writes segfault with images that have zeroed_grain=on.
zero_grained=on ?
Quoted 9 lines
For zero writes, we don't want to allocate a cluster in the image file even in compressed files.
Fixes 524089bce43fd1cd3daaca979872451efa2cf7c6Signed-off-by Kevin Wolf <[email protected]>--- block/vmdk.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH 2/6] vmdk: Fix zero cluster allocation
Am 30.04.2020 um 16:19 hat Eric Blake geschrieben:
Quoted 6 lines · Kevin Wolf
Quoted 3 lines
m_datamust be contain valid data even for zero clusters when no cluster was allocated in the image file. Without this, zero writes segfault with images that have zeroed_grain=on.zero_grained=on ?
No, zeroed_grain is the actual name of the option.
I don't really know what a grain is in VMDK terminology, but about the only thing that felt healthy about the code I touched was that it has whole-grain buffers. :-)
Kevin
PATCH 3/6 vmdk: Fix partial overwrite of zero cluster
In reply to Kevin WolfWhen overwriting a zero cluster, we must not perform copy-on-write from the backing file, but from a zeroed buffer.
--- block/vmdk.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
block/vmdk.c +12 -6
@@ -1340,7 +1340,9 @@ static void vmdk_refresh_limits(BlockDriverState *bs, Error **errp)* get_whole_cluster * * Copy backing file's cluster that covers @sector_num, otherwise write zero, -* to the cluster at @cluster_sector_num. +* to the cluster at @cluster_sector_num. If @zeroed is true, we're overwriting +* a zeroed cluster in the current layer and must not copy data from the +* backing file. * * If @skip_start_sector < @skip_end_sector, the relative range * [@skip_start_sector, @skip_end_sector) is not copied or written, and leave
@@ -1351,18 +1353,21 @@ static int get_whole_cluster(BlockDriverState *bs,uint64_t cluster_offset, uint64_t offset, uint64_t skip_start_bytes, - uint64_t skip_end_bytes) + uint64_t skip_end_bytes, + bool zeroed) { int ret = VMDK_OK;int64_t cluster_bytes;uint8_t *whole_grain; +bool copy_from_backing;/* For COW, align request sector_num to cluster start */ cluster_bytes = extent->cluster_sectors <<BDRV_SECTOR_BITS ; offset =QEMU_ALIGN_DOWN (offset, cluster_bytes); whole_grain =qemu_blockalign (bs, cluster_bytes); + copy_from_backing = bs->backing && !zeroed; -if (!bs->backing) { +if (!copy_from_backing) {memset (whole_grain,0 , skip_start_bytes);memset (whole_grain + skip_end_bytes,0 , cluster_bytes - skip_end_bytes); }
@@ -1377,7 +1382,7 @@ static int get_whole_cluster(BlockDriverState *bs,/* Read backing data before skip range */ if (skip_start_bytes >0 ) { -if (bs->backing) { +if (copy_from_backing) {/* qcow2 emits this on bs->file instead of bs->backing */ BLKDBG_EVENT (extent->file,BLKDBG_COW_READ ); ret = bdrv_pread(bs->backing, offset, whole_grain,
@@ -1397,7 +1402,7 @@ static int get_whole_cluster(BlockDriverState *bs, }/* Read backing data after skip range */ if (skip_end_bytes < cluster_bytes) { -if (bs->backing) { +if (copy_from_backing) {/* qcow2 emits this on bs->file instead of bs->backing */ BLKDBG_EVENT (extent->file,BLKDBG_COW_READ ); ret = bdrv_pread(bs->backing, offset + skip_end_bytes,
@@ -1631,7 +1636,8 @@ static int get_cluster_offset(BlockDriverState *bs,* or inappropriate VM shutdown. */ ret =get_whole_cluster (bs, extent, cluster_sector *BDRV_SECTOR_SIZE , - offset, skip_start_bytes, skip_end_bytes); + offset, skip_start_bytes, skip_end_bytes, + zeroed);if (ret) {return ret; }
Signature · 1 line
2.25.3
Re: [PATCH 3/6] vmdk: Fix partial overwrite of zero cluster
Quoted 8 lines · Kevin Wolf
When overwriting a zero cluster, we must not perform copy-on-write from the backing file, but from a zeroed buffer.
Signed-off-by Kevin Wolf <[email protected]>--- block/vmdk.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 4/6 vmdk: Don't update L2 table for zero write on zero cluster
In reply to Kevin WolfIf a cluster is already zeroed, we don't have to call vmdk_L2update(), which is rather slow because it flushes the image file.
--- block/vmdk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
block/vmdk.c +1 -1
@@ -2013,7 +2013,7 @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset, offset_in_cluster == 0 && n_bytes >= extent->cluster_sectors * BDRV_SECTOR_SIZE) { n_bytes = extent->cluster_sectors *BDRV_SECTOR_SIZE ; -if (!zero_dry_run) { +if (!zero_dry_run && ret !=VMDK_ZEROED ) {/* update L2 tables */ if (vmdk_L2update (extent, &m_data,VMDK_GTE_ZEROED ) !=VMDK_OK ) {
Signature · 1 line
2.25.3
Re: [PATCH 4/6] vmdk: Don't update L2 table for zero write on zero cluster
Quoted 8 lines · Kevin Wolf
If a cluster is already zeroed, we don't have to call
vmdk_L2update(), which is rather slow because it flushes the image file.Signed-off-by Kevin Wolf <[email protected]>--- block/vmdk.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 14 lines
block/vmdk.c +1
-1@@ -2013,7 +2013,7 @@ static int vmdk_pwritev(BlockDriverState *bs, uint64_t offset, offset_in_cluster == 0 && n_bytes >= extent->cluster_sectors * BDRV_SECTOR_SIZE) { n_bytes = extent->cluster_sectors *BDRV_SECTOR_SIZE ; -if (!zero_dry_run) { +if (!zero_dry_run && ret !=VMDK_ZEROED ) {/* update L2 tables */ if (vmdk_L2update (extent, &m_data,VMDK_GTE_ZEROED ) !=VMDK_OK ) {
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 5/6 vmdk: Flush only once in vmdk_L2update()
In reply to Kevin WolfIf we have a backup L2 table, we currently flush once after writing to the active L2 table and again after writing to the backup table. A single flush is enough and makes things a little less slow.
--- block/vmdk.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
block/vmdk.c +5 -2
@@ -1435,7 +1435,7 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data, offset =cpu_to_le32 (offset);/* update L2 table */ BLKDBG_EVENT (extent->file,BLKDBG_L2_UPDATE ); -if (bdrv_pwrite_sync (extent->file, +if (bdrv_pwrite (extent->file, ((int64_t )m_data->l2_offset *512 ) + (m_data->l2_index *sizeof (offset)), &offset,sizeof (offset)) <0 ) {
@@ -1444,13 +1444,16 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data,/* update backup L2 table */ if (extent->l1_backup_table_offset !=0 ) { m_data->l2_offset = extent->l1_backup_table[m_data->l1_index]; -if (bdrv_pwrite_sync (extent->file, +if (bdrv_pwrite (extent->file, ((int64_t )m_data->l2_offset *512 ) + (m_data->l2_index *sizeof (offset)), &offset,sizeof (offset)) <0 ) {return VMDK_ERROR ; } } +if (bdrv_flush (extent->file->bs) <0 ) { +return VMDK_ERROR ; + }if (m_data->l2_cache_entry) { *m_data->l2_cache_entry = offset; }
Signature · 1 line
2.25.3
Re: [PATCH 5/6] vmdk: Flush only once in vmdk_L2update()
Quoted 14 lines · Kevin Wolf
If we have a backup L2 table, we currently flush once after writing to the active L2 table and again after writing to the backup table. A single flush is enough and makes things a little less slow.
Signed-off-by Kevin Wolf <[email protected]>--- block/vmdk.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)block/vmdk.c +0
-0
block/vmdk.c static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data, · line 1441
Quoted 8 lines
block/vmdk.c +1
-1@@ -1435,7 +1435,7 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data, offset =cpu_to_le32 (offset);/* update L2 table */ BLKDBG_EVENT (extent->file,BLKDBG_L2_UPDATE ); -if (bdrv_pwrite_sync (extent->file, +if (bdrv_pwrite (extent->file, ((int64_t )m_data->l2_offset *512 ) + (m_data->l2_index *sizeof (offset)), &offset,sizeof (offset)) <0 ) {
Worth reindenting ther est of the function call?
Quoted 5 lines
block/vmdk.c +0
-1@@ -1444,13 +1444,16 @@ static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data,/* update backup L2 table */ if (extent->l1_backup_table_offset !=0 ) { m_data->l2_offset = extent->l1_backup_table[m_data->l1_index]; - if (bdrv_pwrite_sync(extent->file,
block/vmdk.c static int vmdk_L2update(VmdkExtent *extent, VmdkMetaData *m_data, · line 1456
Quoted 10 lines
block/vmdk.c +4
-0+if (bdrv_pwrite (extent->file, ((int64_t )m_data->l2_offset *512 ) + (m_data->l2_index *sizeof (offset)), &offset,sizeof (offset)) <0 ) {return VMDK_ERROR ; } } +if (bdrv_flush (extent->file->bs) <0 ) { +return VMDK_ERROR ; + }
But indentation doesn't affect correctness.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
PATCH 6/6 iotests: vmdk: Enable zeroed_grained=on by default
In reply to Kevin WolfIn order to avoid bitrot in the zero cluster code in VMDK, enable zero_grained=on by default for the tests.
--- tests/qemu-iotests/059 | 6 +++--- tests/qemu-iotests/check | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-)
tests/qemu-iotests/059 +3 -3
@@ -41,9 +41,9 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt vmdk _supported_proto file _supported_os Linux -_unsupported_imgopts "subformat=monolithicFlat" \ - "subformat=twoGbMaxExtentFlat" \ - "subformat=twoGbMaxExtentSparse" + +# We test all kinds of VMDK options here, so ignore user-specified options +IMGOPTS="" capacity_offset=16 granularity_offset=20
tests/qemu-iotests/check +3 -0
@@ -546,6 +546,9 @@ fi if [ "$IMGFMT" == "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=" > /dev/null); then IMGOPTS=$(_optstr_add "$IMGOPTS" "iter-time=10") fi +if [ "$IMGFMT" == "vmdk" ] && ! (echo "$IMGOPTS" | grep "zeroed_grain=" > /dev/null); then + IMGOPTS=$(_optstr_add "$IMGOPTS" "zeroed_grain=on") +fi if [ -z "$SAMPLE_IMG_DIR" ]; then SAMPLE_IMG_DIR="$source_iotests/sample_images"
Signature · 1 line
2.25.3
Re: [PATCH 6/6] iotests: vmdk: Enable zeroed_grained=on by default
Quoted 2 lines · Kevin Wolf
In order to avoid bitrot in the zero cluster code in VMDK, enable zero_grained=on by default for the tests.
Here, you spell it zero_grained=on,
Quoted 6 lines
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/059 | 6 +++--- tests/qemu-iotests/check | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-)
So you're changing the default for better coverage and speed, but ensuring that 59 still covers the (slower) zero_grained=off. Seems reasonable.
tests/qemu-iotests/check fi · line 549
Quoted 6 lines
+++ b/tests/qemu-iotests/check
tests/qemu-iotests/check +1
-0@@ -546,6 +546,9 @@ fi if [ "$IMGFMT" == "luks" ] && ! (echo "$IMGOPTS" | grep "iter-time=" > /dev/null); then IMGOPTS=$(_optstr_add "$IMGOPTS" "iter-time=10") fi +if [ "$IMGFMT" == "vmdk" ] && ! (echo "$IMGOPTS" | grep "zeroed_grain=" > /dev/null); then
Here, zeroed_grain=. Which is it?
tests/qemu-iotests/check fi · line 550
Quoted 1 line
tests/qemu-iotests/check +1
-0+ IMGOPTS=$(_optstr_add "$IMGOPTS" "zeroed_grain=on")
As a native speaker, my inclination is zero_grained; but I don't know the VMDK spec well enough to know if this is something in the spec, or just a term that qemu invented. And since we already have existing usage of one spelling, switching the spelling now would require a deprecation period and is separate from this patch.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [PATCH 6/6] iotests: vmdk: Enable zeroed_grained=on by default
Am 30.04.2020 um 16:22 hat Eric Blake geschrieben:
Quoted 5 lines · Kevin Wolf
Quoted 2 lines
In order to avoid bitrot in the zero cluster code in VMDK, enable zero_grained=on by default for the tests.
Here, you spell it zero_grained=on,
Thanks for spotting this, the typo is in the commit message.
Quoted 9 lines
Quoted 6 lines
Signed-off-by Kevin Wolf <[email protected]>--- tests/qemu-iotests/059 | 6 +++--- tests/qemu-iotests/check | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-)So you're changing the default for better coverage and speed, but ensuring that 59 still covers the (slower) zero_grained=off. Seems reasonable.
The real reason why I'm changing 059 is that zeroed_grain=on works only with some subformats and the test case tests many different subformats, including those for which it doesn't work.
Kevin
Am 30.04.2020 um 15:30 hat Kevin Wolf geschrieben:
Quoted 7 lines
What I was really investigating is why 055 was so slow. I couldn't solve that, but instead I found out that our VMDK code for zero clusters and
write_zeroeswas completely broken. Apart from segfaults when zero clusters were actually enabled, this caused a compressed backup target to result in a bigger file than uncompressed with VMDK.This series tries to fix it (with one bonus performance patch).
Thanks for the review, fixed up the commit messages and applied.
If you were curious about the VMDK terminology, I looked it up and the basic terms translate to qcow2 like this:
* grain directory = L1 table
* grain table = L2 table
* grain = cluster
"zeroed-grain GTE (grain table entry)" is the exact term used in the VMDK spec for what we would call a zero cluster in qcow2.
Kevin