Skip to contentSkip to conversation

qcow2: Set zero flag for discarded clusters

PATCH Superseded Kevin Wolf 3 participants 8 messages Feb 8, 2014 – Feb 14, 2014 Block layer

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

PATCH 1/1 qcow2: Set zero flag for discarded clusters

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
 block/qcow2-cluster.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +20 -2
index 25d45d1..008bc04 100644
@@ -1333,13 +1333,31 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
         uint64_t old_offset;
 
         old_offset = be64_to_cpu(l2_table[l2_index + i]);
-        if ((old_offset & L2E_OFFSET_MASK) == 0) {
+
+        /*
+         * Make sure that a discarded area reads back as zeroes for v3 images
+         * (we cannot do it for v2 without actually writing a zero-filled
+         * buffer). We can skip the operation if the cluster is already marked
+         * as zero, or if it's unallocated and we don't have a backing file.
+         *
+         * TODO We might want to use bdrv_get_block_status(bs) here, but we're
+         * holding s->lock, so that doesn't work today.
+         */
+        if (!!(old_offset & QCOW_OFLAG_ZERO)) {
+            continue;
+        }
+
+        if ((old_offset & L2E_OFFSET_MASK) == 0 && !bs->backing_hd) {
             continue;
         }
 
         /* First remove L2 entries */
         qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
-        l2_table[l2_index + i] = cpu_to_be64(0);
+        if (s->version >= 3) {
+            l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
+        } else {
+            l2_table[l2_index + i] = cpu_to_be64(0);
+        }
 
         /* Then decrease the refcount */
         qcow2_free_any_clusters(bs, old_offset, 1, type);
Signature · 1 line
1.8.1.4
Reviewed-by on v1 1/1
#1 lore raw
Quoted 7 lines · Kevin Wolf

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
  block/qcow2-cluster.c | 22 ++++++++++++++++++++--
  1 file changed, 20 insertions(+), 2 deletions(-)
#2 lore raw
In reply to Kevin Wolf
Quoted 29 lines · Kevin Wolf

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
  block/qcow2-cluster.c | 22 ++++++++++++++++++++--
  1 file changed, 20 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +12 -1
index 25d45d1..008bc04 100644
@@ -1333,13 +1333,31 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
          uint64_t old_offset;
 
          old_offset = be64_to_cpu(l2_table[l2_index + i]);
-        if ((old_offset & L2E_OFFSET_MASK) == 0) {
+
+        /*
+         * Make sure that a discarded area reads back as zeroes for v3 images
+         * (we cannot do it for v2 without actually writing a zero-filled
+         * buffer). We can skip the operation if the cluster is already marked
+         * as zero, or if it's unallocated and we don't have a backing file.
+         *
+         * TODO We might want to use bdrv_get_block_status(bs) here, but we're
+         * holding s->lock, so that doesn't work today.
+         */
+        if (!!(old_offset & QCOW_OFLAG_ZERO)) {
+            continue;

block/qcow2-cluster.c static int discard_single_l2(BlockDriverState *bs, uint64_t offset, · line 1356

Quoted 10 lines
block/qcow2-cluster.c +4 -1
+        }
+
+        if ((old_offset & L2E_OFFSET_MASK) == 0 && !bs->backing_hd) {
              continue;
          }
 
          /* First remove L2 entries */
          qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
-        l2_table[l2_index + i] = cpu_to_be64(0);
+        if (s->version >= 3) {

Oh, I revoke my reviewed-by from earlier. This should probably be "s->qcow_version"; otherwise, it won't compile.

Max

Quoted 7 lines
block/qcow2-cluster.c +4 -0
+            l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
+        } else {
+            l2_table[l2_index + i] = cpu_to_be64(0);
+        }
 
          /* Then decrease the refcount */
          qcow2_free_any_clusters(bs, old_offset, 1, type);
author
#3 lore raw

Am 08.02.2014 um 17:17 hat Max Reitz geschrieben:

Quoted 33 lines · Kevin Wolf
Quoted 32 lines

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
 block/qcow2-cluster.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +15 -1
index 25d45d1..008bc04 100644
@@ -1333,13 +1333,31 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
         uint64_t old_offset;
         old_offset = be64_to_cpu(l2_table[l2_index + i]);
-        if ((old_offset & L2E_OFFSET_MASK) == 0) {
+
+        /*
+         * Make sure that a discarded area reads back as zeroes for v3 images
+         * (we cannot do it for v2 without actually writing a zero-filled
+         * buffer). We can skip the operation if the cluster is already marked
+         * as zero, or if it's unallocated and we don't have a backing file.
+         *
+         * TODO We might want to use bdrv_get_block_status(bs) here, but we're
+         * holding s->lock, so that doesn't work today.
+         */
+        if (!!(old_offset & QCOW_OFLAG_ZERO)) {
+            continue;
+        }
+
+        if ((old_offset & L2E_OFFSET_MASK) == 0 && !bs->backing_hd) {
             continue;
Quoted 8 lines

block/qcow2-cluster.c static int discard_single_l2(BlockDriverState *bs, uint64_t offset, · line 1356

Quoted 5 lines
block/qcow2-cluster.c +1 -1
         }
         /* First remove L2 entries */
         qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
-        l2_table[l2_index + i] = cpu_to_be64(0);
+        if (s->version >= 3) {

Oh, I revoke my reviewed-by from earlier. This should probably be "s->qcow_version"; otherwise, it won't compile.

Indeed. Makes we wonder what source it was that I tested... :-/

Kevin

author
#4 lore raw

PATCH v2 1/1 qcow2: Set zero flag for discarded clusters

In reply to Max Reitz

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
 block/qcow2-cluster.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +20 -2
index 25d45d1..9461969 100644
@@ -1333,13 +1333,31 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
         uint64_t old_offset;
 
         old_offset = be64_to_cpu(l2_table[l2_index + i]);
-        if ((old_offset & L2E_OFFSET_MASK) == 0) {
+
+        /*
+         * Make sure that a discarded area reads back as zeroes for v3 images
+         * (we cannot do it for v2 without actually writing a zero-filled
+         * buffer). We can skip the operation if the cluster is already marked
+         * as zero, or if it's unallocated and we don't have a backing file.
+         *
+         * TODO We might want to use bdrv_get_block_status(bs) here, but we're
+         * holding s->lock, so that doesn't work today.
+         */
+        if (!!(old_offset & QCOW_OFLAG_ZERO)) {
+            continue;
+        }
+
+        if ((old_offset & L2E_OFFSET_MASK) == 0 && !bs->backing_hd) {
             continue;
         }
 
         /* First remove L2 entries */
         qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
-        l2_table[l2_index + i] = cpu_to_be64(0);
+        if (s->qcow_version >= 3) {
+            l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
+        } else {
+            l2_table[l2_index + i] = cpu_to_be64(0);
+        }
 
         /* Then decrease the refcount */
         qcow2_free_any_clusters(bs, old_offset, 1, type);
Signature · 1 line
1.8.1.4
#5 lore raw
Quoted 8 lines · Kevin Wolf

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
 block/qcow2-cluster.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

Thanks, applied v2 to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan

#6 lore raw
In reply to Kevin Wolf
Quoted 37 lines · Kevin Wolf

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
 block/qcow2-cluster.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +15 -1
index 25d45d1..9461969 100644
@@ -1333,13 +1333,31 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
         uint64_t old_offset;
 
         old_offset = be64_to_cpu(l2_table[l2_index + i]);
-        if ((old_offset & L2E_OFFSET_MASK) == 0) {
+
+        /*
+         * Make sure that a discarded area reads back as zeroes for v3 images
+         * (we cannot do it for v2 without actually writing a zero-filled
+         * buffer). We can skip the operation if the cluster is already marked
+         * as zero, or if it's unallocated and we don't have a backing file.
+         *
+         * TODO We might want to use bdrv_get_block_status(bs) here, but we're
+         * holding s->lock, so that doesn't work today.
+         */
+        if (!!(old_offset & QCOW_OFLAG_ZERO)) {
+            continue;
+        }
+
+        if ((old_offset & L2E_OFFSET_MASK) == 0 && !bs->backing_hd) {
             continue;
         }
 
         /* First remove L2 entries */

block/qcow2-cluster.c static int discard_single_l2(BlockDriverState *bs, uint64_t offset, · line 1363

Quoted 10 lines
block/qcow2-cluster.c +5 -1
         qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
-        l2_table[l2_index + i] = cpu_to_be64(0);
+        if (s->qcow_version >= 3) {
+            l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
+        } else {
+            l2_table[l2_index + i] = cpu_to_be64(0);
+        }
 
         /* Then decrease the refcount */
         qcow2_free_any_clusters(bs, old_offset, 1, type);

Oops, this breaks qemu-iotests 046. I have dropped it from the pull request.

046 1s ... - output mismatch (see 046.out.bad)

046.out.bad renamed from 046.out +4 -0
@@ -187,10 +187,12 @@
 24 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 8192/8192 bytes at offset 516096
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Pattern verification failed at offset 524288, 24576 bytes
 read 24576/24576 bytes at offset 524288
 24 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 8192/8192 bytes at offset 548864
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Pattern verification failed at offset 557056, 8192 bytes
 read 8192/8192 bytes at offset 557056
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 57344/57344 bytes at offset 565248
@@ -199,10 +201,12 @@
 24 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 8192/8192 bytes at offset 647168
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Pattern verification failed at offset 655360, 24576 bytes
 read 24576/24576 bytes at offset 655360
 24 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 8192/8192 bytes at offset 679936
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Pattern verification failed at offset 688128, 8192 bytes
 read 8192/8192 bytes at offset 688128
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 57344/57344 bytes at offset 696320

Failures: 046
Failed 1 of 1 tests

author
#7 lore raw

Am 14.02.2014 um 18:05 hat Stefan Hajnoczi geschrieben:

Quoted 46 lines · Kevin Wolf
Quoted 45 lines

Instead of making the backing file contents visible again after a discard request, set the zero flag if possible (i.e. on version >= 3).

---
 block/qcow2-cluster.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
block/qcow2-cluster.c +20 -2
index 25d45d1..9461969 100644
@@ -1333,13 +1333,31 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
         uint64_t old_offset;
 
         old_offset = be64_to_cpu(l2_table[l2_index + i]);
-        if ((old_offset & L2E_OFFSET_MASK) == 0) {
+
+        /*
+         * Make sure that a discarded area reads back as zeroes for v3 images
+         * (we cannot do it for v2 without actually writing a zero-filled
+         * buffer). We can skip the operation if the cluster is already marked
+         * as zero, or if it's unallocated and we don't have a backing file.
+         *
+         * TODO We might want to use bdrv_get_block_status(bs) here, but we're
+         * holding s->lock, so that doesn't work today.
+         */
+        if (!!(old_offset & QCOW_OFLAG_ZERO)) {
+            continue;
+        }
+
+        if ((old_offset & L2E_OFFSET_MASK) == 0 && !bs->backing_hd) {
             continue;
         }
 
         /* First remove L2 entries */
         qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
-        l2_table[l2_index + i] = cpu_to_be64(0);
+        if (s->qcow_version >= 3) {
+            l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
+        } else {
+            l2_table[l2_index + i] = cpu_to_be64(0);
+        }
 
Quoted 4 lines

block/qcow2-cluster.c static int discard_single_l2(BlockDriverState *bs, uint64_t offset, · line 1363

Quoted 2 lines
block/qcow2-cluster.c +0 -0
         /* Then decrease the refcount */
         qcow2_free_any_clusters(bs, old_offset, 1, type);

Oops, this breaks qemu-iotests 046. I have dropped it from the pull request.

Okay. This is kind of nasty, because we now have different expected results for v2 and v3 images. I'll have to see how the test case is fixed in the best way. Perhaps it's best not to check discarded areas at all, because strictly speaking they are undefined.

Kevin