Minor fixes about compressed clusters with OFLAG_COPIED
cover Minor fixes about compressed clusters with OFLAG_COPIED
Hi,
while reviewing one previous patch about data corruption and compressed clusters we discussed that the documentation doesn't clarify that L2 entries for compressed clusters are not supposed to have the OFLAG_COPIED bit set.
Here's a patch to update the documentation and another one to fix the error message that "qemu-img check" produces.
Regards,
Berto
Alberto Garcia (2):
Fix error message about compressed clusters with OFLAG_COPIED specs/qcow2: Clarify that compressed clusters have the COPIED bit
reset
block/qcow2-refcount.c | 4 ++-- docs/interop/qcow2.txt | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-)
Signature · 1 line
2.11.0
PATCH 1/2 Fix error message about compressed clusters with OFLAG_COPIED
Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset.
This patch fixes the error message and reports the offset of the compressed cluster.
--- block/qcow2-refcount.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
block/qcow2-refcount.c +2 -2
@@ -1577,9 +1577,9 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,case QCOW2_CLUSTER_COMPRESSED :/* Compressed clusters don't have QCOW_OFLAG_COPIED */ if (l2_entry &QCOW_OFLAG_COPIED ) { -fprintf (stderr,"ERROR: cluster %" PRId64": " +fprintf (stderr,"ERROR: coffset=0x%" PRIx64": " "copied flag must never be set for compressed " -"clusters\n" , l2_entry >> s->cluster_bits); +"clusters\n" , l2_entry & s->cluster_offset_mask); l2_entry &= ~QCOW_OFLAG_COPIED ; res->corruptions++; }
Signature · 1 line
2.11.0
PATCH 2/2 specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
In reply to Alberto GarciaCompressed clusters are not supposed to have the COPIED bit set, but this is not made explicit in the specs, so let's document it.
--- docs/interop/qcow2.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
docs/interop/qcow2.txt +4 -4
@@ -400,10 +400,10 @@ L2 table entry: 62: 0 for standard clusters 1 for compressed clusters - 63: 0 for a cluster that is unused or requires COW, 1 if its - refcount is exactly one. This information is only accurate - in L2 tables that are reachable from the active L1 - table. + 63: 0 for clusters that are unused, compressed or require COW. + 1 for standard clusters whose refcount is exactly one. + This information is only accurate in L2 tables + that are reachable from the active L1 table. Standard Cluster Descriptor:
Signature · 1 line
2.11.0
Re: [Qemu-devel] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED
In reply to Alberto GarciaQuoted 7 lines · Alberto Garcia
Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset.
Quoted 8 lines
This patch fixes the error message and reports the offset of the compressed cluster.
Signed-off-by Alberto Garcia <[email protected]>--- block/qcow2-refcount.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Do we have iotests coverage of this?
Should the qcow2 spec be explicit that OFLAG_COPIED should never be set on a compressed cluster? The current documentation for L2 table entry mentions that bit 63 is '1' for a cluster with a refcount of exactly 1 if it is an L2 table reachable from the active L1 table, with no mention of a restriction that it must not be set when bit 62 is set. Or is it feasible that although qemu itself (should) never set OFLAG_COPIED on a compressed cluster, that some third-party tool could still validly set the bit for a compressed cluster that happens to occupy a host cluster with a refcount of exactly 1 (and where writing to that cluster could be done by replacing the cluster in-place with uncompressed data, or by again writing compressed data - compression is rather wasteful in that sense as the tail of the host cluster is unused, and the only way to use the tail of the cluster is with another compressed cluster at which point the refcount is no longer 1).
Quoted 5 lines
block/qcow2-refcount.c +0
-0
block/qcow2-refcount.c static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, · line 1583
Quoted 9 lines
block/qcow2-refcount.c +2
-2@@ -1577,9 +1577,9 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,case QCOW2_CLUSTER_COMPRESSED :/* Compressed clusters don't have QCOW_OFLAG_COPIED */ if (l2_entry &QCOW_OFLAG_COPIED ) { -fprintf (stderr,"ERROR: cluster %" PRId64": " +fprintf (stderr,"ERROR: coffset=0x%" PRIx64": " "copied flag must never be set for compressed " -"clusters\n" , l2_entry >> s->cluster_bits); +"clusters\n" , l2_entry & s->cluster_offset_mask); l2_entry &= ~QCOW_OFLAG_COPIED ;
At any rate, regardless of the answers to my questions, the error message cleanup makes sense.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 2/2] specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
In reply to Alberto GarciaQuoted 14 lines · Alberto Garcia
Compressed clusters are not supposed to have the COPIED bit set, but this is not made explicit in the specs, so let's document it.
Signed-off-by Alberto Garcia <[email protected]>--- docs/interop/qcow2.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)docs/interop/qcow2.txt +0
-0@@ -400,10 +400,10 @@ L2 table entry: 62: 0 for standard clusters
docs/interop/qcow2.txt L2 table entry: · line 406
Quoted 10 lines
docs/interop/qcow2.txt +4
-41 for compressed clusters - 63: 0 for a cluster that is unused or requires COW, 1 if its - refcount is exactly one. This information is only accurate - in L2 tables that are reachable from the active L1 - table. + 63: 0 for clusters that are unused, compressed or require COW. + 1 for standard clusters whose refcount is exactly one. + This information is only accurate in L2 tables + that are reachable from the active L1 table.
This matches what qemu outputs, so the question becomes whether it is technically necessary to make this requirement mandatory for 3rd-party implementations. But I'm in favor of the tighter wording, as it gets rather hairy to check whether exactly one compressed cluster is occupying a host cluster, plus I don't want to think about what happens if a compressed cluster with the bit set crosses a host cluster boundary (does it mean that compressed cluster is the only [remaining] source of data for BOTH host clusters at once, where both the head of the first host cluster and tail of the second host cluster is unused?)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED
In reply to Eric BlakeQuoted 11 lines · Eric Blake
Quoted 10 lines · Alberto Garcia
Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset.
This patch fixes the error message and reports the offset of the compressed cluster.
Quoted 7 lines
Quoted 5 lines
Signed-off-by Alberto Garcia <[email protected]>--- block/qcow2-refcount.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)Do we have iotests coverage of this?
I have one half-written, but I was thinking to put it in 214, so I'll wait until Max's patch is merged.
Berto
Max Reitz
AppliedQuoted 14 lines · Alberto Garcia
Hi,
while reviewing one previous patch about data corruption and compressed clusters we discussed that the documentation doesn't clarify that L2 entries for compressed clusters are not supposed to have the
OFLAG_COPIEDbit set.Here's a patch to update the documentation and another one to fix the error message that "qemu-img check" produces.
Regards,
Berto
Quoted 8 lines
Alberto Garcia (2):
Fix error message about compressed clusters withOFLAG_COPIEDspecs/qcow2: Clarify that compressed clusters have the COPIED bit
resetblock/qcow2-refcount.c | 4 ++-- docs/interop/qcow2.txt | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-)
Thanks, applied to my block-next tree:
https://github.com/XanClic/qemu/commits/block-next
Max