Skip to contentSkip to conversation

Minor fixes about compressed clusters with OFLAG_COPIED

PATCH Applied Alberto Garcia 3 participants 7 messages Apr 10, 2018 – Apr 13, 2018 Block layer

lore
Share & tools
lore
#0 lore raw

cover Minor fixes about compressed clusters with OFLAG_COPIED

Hi,

while reviewing one previous patch about data corruption and compressed clusters we discussed that the documentation doesn't clarify that L2 entries for compressed clusters are not supposed to have the OFLAG_COPIED bit set.

Here's a patch to update the documentation and another one to fix the error message that "qemu-img check" produces.

Regards,

Berto

Alberto Garcia (2):
Fix error message about compressed clusters with OFLAG_COPIED specs/qcow2: Clarify that compressed clusters have the COPIED bit
reset

 block/qcow2-refcount.c | 4 ++--
 docs/interop/qcow2.txt | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)
Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/2 Fix error message about compressed clusters with OFLAG_COPIED

Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset.

This patch fixes the error message and reports the offset of the compressed cluster.

---
 block/qcow2-refcount.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
block/qcow2-refcount.c +2 -2
index 6b8b63514a..2dc23005b7 100644
@@ -1577,9 +1577,9 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
         case QCOW2_CLUSTER_COMPRESSED:
             /* Compressed clusters don't have QCOW_OFLAG_COPIED */
             if (l2_entry & QCOW_OFLAG_COPIED) {
-                fprintf(stderr, "ERROR: cluster %" PRId64 ": "
+                fprintf(stderr, "ERROR: coffset=0x%" PRIx64 ": "
                     "copied flag must never be set for compressed "
-                    "clusters\n", l2_entry >> s->cluster_bits);
+                    "clusters\n", l2_entry & s->cluster_offset_mask);
                 l2_entry &= ~QCOW_OFLAG_COPIED;
                 res->corruptions++;
             }
Signature · 1 line
2.11.0
#4 lore raw

PATCH 2/2 specs/qcow2: Clarify that compressed clusters have the COPIED bit reset

In reply to Alberto Garcia

Compressed clusters are not supposed to have the COPIED bit set, but this is not made explicit in the specs, so let's document it.

---
 docs/interop/qcow2.txt | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
docs/interop/qcow2.txt +4 -4
index feb711fb6a..8e1547ded2 100644
@@ -400,10 +400,10 @@ L2 table entry:
               62:   0 for standard clusters
                     1 for compressed clusters
 
-              63:   0 for a cluster that is unused or requires COW, 1 if its
-                    refcount is exactly one. This information is only accurate
-                    in L2 tables that are reachable from the active L1
-                    table.
+              63:   0 for clusters that are unused, compressed or require COW.
+                    1 for standard clusters whose refcount is exactly one.
+                    This information is only accurate in L2 tables
+                    that are reachable from the active L1 table.
 
 Standard Cluster Descriptor:
 
Signature · 1 line
2.11.0
Reviewed-by on v1 1/2
#2 lore raw

Re: [Qemu-devel] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED

In reply to Alberto Garcia
Quoted 7 lines · Alberto Garcia

Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset.

Quoted 8 lines

This patch fixes the error message and reports the offset of the compressed cluster.

---
 block/qcow2-refcount.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Do we have iotests coverage of this?

Should the qcow2 spec be explicit that OFLAG_COPIED should never be set on a compressed cluster? The current documentation for L2 table entry mentions that bit 63 is '1' for a cluster with a refcount of exactly 1 if it is an L2 table reachable from the active L1 table, with no mention of a restriction that it must not be set when bit 62 is set. Or is it feasible that although qemu itself (should) never set OFLAG_COPIED on a compressed cluster, that some third-party tool could still validly set the bit for a compressed cluster that happens to occupy a host cluster with a refcount of exactly 1 (and where writing to that cluster could be done by replacing the cluster in-place with uncompressed data, or by again writing compressed data - compression is rather wasteful in that sense as the tail of the host cluster is unused, and the only way to use the tail of the cluster is with another compressed cluster at which point the refcount is no longer 1).

Quoted 5 lines
block/qcow2-refcount.c +0 -0
index 6b8b63514a..2dc23005b7 100644

block/qcow2-refcount.c static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res, · line 1583

Quoted 9 lines
block/qcow2-refcount.c +2 -2
@@ -1577,9 +1577,9 @@ static int check_refcounts_l2(BlockDriverState *bs, BdrvCheckResult *res,
         case QCOW2_CLUSTER_COMPRESSED:
             /* Compressed clusters don't have QCOW_OFLAG_COPIED */
             if (l2_entry & QCOW_OFLAG_COPIED) {
-                fprintf(stderr, "ERROR: cluster %" PRId64 ": "
+                fprintf(stderr, "ERROR: coffset=0x%" PRIx64 ": "
                     "copied flag must never be set for compressed "
-                    "clusters\n", l2_entry >> s->cluster_bits);
+                    "clusters\n", l2_entry & s->cluster_offset_mask);
                 l2_entry &= ~QCOW_OFLAG_COPIED;

At any rate, regardless of the answers to my questions, the error message cleanup makes sense.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
Reviewed-by on v1 2/2
#5 lore raw

Re: [Qemu-devel] [PATCH 2/2] specs/qcow2: Clarify that compressed clusters have the COPIED bit reset

In reply to Alberto Garcia
Quoted 14 lines · Alberto Garcia

Compressed clusters are not supposed to have the COPIED bit set, but this is not made explicit in the specs, so let's document it.

---
 docs/interop/qcow2.txt | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
docs/interop/qcow2.txt +0 -0
index feb711fb6a..8e1547ded2 100644
@@ -400,10 +400,10 @@ L2 table entry:
               62:   0 for standard clusters

docs/interop/qcow2.txt L2 table entry: · line 406

Quoted 10 lines
docs/interop/qcow2.txt +4 -4
                     1 for compressed clusters
 
-              63:   0 for a cluster that is unused or requires COW, 1 if its
-                    refcount is exactly one. This information is only accurate
-                    in L2 tables that are reachable from the active L1
-                    table.
+              63:   0 for clusters that are unused, compressed or require COW.
+                    1 for standard clusters whose refcount is exactly one.
+                    This information is only accurate in L2 tables
+                    that are reachable from the active L1 table.

This matches what qemu outputs, so the question becomes whether it is technically necessary to make this requirement mandatory for 3rd-party implementations. But I'm in favor of the tighter wording, as it gets rather hairy to check whether exactly one compressed cluster is occupying a host cluster, plus I don't want to think about what happens if a compressed cluster with the bit set crosses a host cluster boundary (does it mean that compressed cluster is the only [remaining] source of data for BOTH host clusters at once, where both the head of the first host cluster and tail of the second host cluster is unused?)

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
#3 lore raw

Re: [Qemu-devel] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED

In reply to Eric Blake
Quoted 11 lines · Eric Blake
Quoted 10 lines · Alberto Garcia

Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report the offset instead. Plus, the calculation is wrong and it uses the raw L2 entry as if it was simply an offset.

This patch fixes the error message and reports the offset of the compressed cluster.

Quoted 7 lines
Quoted 5 lines
---
 block/qcow2-refcount.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Do we have iotests coverage of this?

I have one half-written, but I was thinking to put it in 214, so I'll wait until Max's patch is merged.

Berto

Applied
#6 lore raw
In reply to Alberto Garcia
Quoted 14 lines · Alberto Garcia

Hi,

while reviewing one previous patch about data corruption and compressed clusters we discussed that the documentation doesn't clarify that L2 entries for compressed clusters are not supposed to have the OFLAG_COPIED bit set.

Here's a patch to update the documentation and another one to fix the error message that "qemu-img check" produces.

Regards,

Berto

Quoted 8 lines

Alberto Garcia (2):
Fix error message about compressed clusters with OFLAG_COPIED specs/qcow2: Clarify that compressed clusters have the COPIED bit
reset

 block/qcow2-refcount.c | 4 ++--
 docs/interop/qcow2.txt | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

Thanks, applied to my block-next tree:

https://github.com/XanClic/qemu/commits/block-next

Max