Skip to contentSkip to conversation

qcow2: Stop overwriting compressed clusters one by one

PATCH In review Alberto Garcia 3 participants 4 messages Sep 11, 2019 – Sep 12, 2019 Block layer

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 qcow2: Stop overwriting compressed clusters one by one

handle_alloc() tries to find as many contiguous clusters that need copy-on-write as possible in order to allocate all of them at the same time.

However, compressed clusters are only overwritten one by one, so let's say that we have an image with 1024 consecutive compressed clusters:

   qemu-img create -f qcow2 hd.qcow2 64M
   for f in `seq 0 64 65472`; do
      qemu-io -c "write -c ${f}k 64k" hd.qcow2
   done

In this case trying to overwrite the whole image with one large write request results in 1024 separate allocations:

qemu-io -c "write 0 64M" hd.qcow2

This restriction comes from commit 095a9c58ce12afeeb90c2 from 2018. Nowadays QEMU can overwrite multiple compressed clusters just fine, and in fact it already does: as long as the first cluster that handle_alloc() finds is not compressed, all other compressed clusters in the same batch will be overwritten in one go:

   qemu-img create -f qcow2 hd.qcow2 64M
   qemu-io -c "write -z 0 64k" hd.qcow2
   for f in `seq 64 64 65472`; do
      qemu-io -c "write -c ${f}k 64k" hd.qcow2
   done

Compared to the previous one, overwriting this image on my computer goes from 8.35s down to 230ms.

---
 block/qcow2-cluster.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
block/qcow2-cluster.c +1 -7
index f09cc992af..dcacd3c450 100644
@@ -1351,13 +1351,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
     }
 
     entry = be64_to_cpu(l2_slice[l2_index]);
-
-    /* For the moment, overwrite compressed clusters one by one */
-    if (entry & QCOW_OFLAG_COMPRESSED) {
-        nb_clusters = 1;
-    } else {
-        nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index);
-    }
+    nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index);
 
     /* This function is only called when there were no non-COW clusters, so if
      * we can't find any unallocated or COW clusters either, something is
Signature · 1 line
2.20.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 12 lines · Alberto Garcia

handle_alloc() tries to find as many contiguous clusters that need copy-on-write as possible in order to allocate all of them at the same time.

However, compressed clusters are only overwritten one by one, so let's say that we have an image with 1024 consecutive compressed clusters:

   qemu-img create -f qcow2 hd.qcow2 64M
   for f in `seq 0 64 65472`; do
      qemu-io -c "write -c ${f}k 64k" hd.qcow2
   done
Quoted 6 lines

In this case trying to overwrite the whole image with one large write request results in 1024 separate allocations:

   qemu-io -c "write 0 64M" hd.qcow2

This restriction comes from commit 095a9c58ce12afeeb90c2 from 2018.

You accidentally typed a reasonably modern date. It's from *2008*!

Quoted 29 lines

Nowadays QEMU can overwrite multiple compressed clusters just fine, and in fact it already does: as long as the first cluster that handle_alloc() finds is not compressed, all other compressed clusters in the same batch will be overwritten in one go:

   qemu-img create -f qcow2 hd.qcow2 64M
   qemu-io -c "write -z 0 64k" hd.qcow2
   for f in `seq 64 64 65472`; do
      qemu-io -c "write -c ${f}k 64k" hd.qcow2
   done

Compared to the previous one, overwriting this image on my computer goes from 8.35s down to 230ms.

---
 block/qcow2-cluster.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
block/qcow2-cluster.c +0 -1
index f09cc992af..dcacd3c450 100644
@@ -1351,13 +1351,7 @@ static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
     }
 
     entry = be64_to_cpu(l2_slice[l2_index]);
-

block/qcow2-cluster.c static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset, · line 1357

Quoted 10 lines
block/qcow2-cluster.c +1 -6
-    /* For the moment, overwrite compressed clusters one by one */
-    if (entry & QCOW_OFLAG_COMPRESSED) {
-        nb_clusters = 1;
-    } else {
-        nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index);
-    }
+    nb_clusters = count_cow_clusters(bs, nb_clusters, l2_slice, l2_index);
 
     /* This function is only called when there were no non-COW clusters, so if
      * we can't find any unallocated or COW clusters either, something is

Well, given that count_cow_clusters already works this way, this doesn't break anything that wasn't broken before, at least.

#2 lore raw
Quoted 3 lines · John Snow
Quoted 1 line

This restriction comes from commit 095a9c58ce12afeeb90c2 from 2018.

You accidentally typed a reasonably modern date. It's from *2008*!

Oh my, and I reviewed the message 3 times ... if this one gets committed please correct the date.

Berto

#3 lore raw

Am 12.09.2019 um 09:25 hat Alberto Garcia geschrieben:

Quoted 7 lines · John Snow
Quoted 3 lines
Quoted 1 line

This restriction comes from commit 095a9c58ce12afeeb90c2 from 2018.

You accidentally typed a reasonably modern date. It's from *2008*!

Oh my, and I reviewed the message 3 times ... if this one gets committed please correct the date.

It was actually one of the very first QEMU patches I reviewed. :-)

Thanks, fixed the year and applied to the block branch.

Kevin