Skip to contentSkip to conversation

qemu-img convert: Don't pre-zero images

PATCH In review Kevin Wolf 3 participants 3 messages Jun 22, 2020

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

PATCH 1/1 qemu-img convert: Don't pre-zero images

Since commit 5a37b60a61c, qemu-img create will pre-zero the target image if it isn't already zero-initialised (most importantly, for host block devices, but also iscsi etc.), so that writing explicit zeros wouldn't be necessary later.

This could speed up the operation significantly, in particular when the source image file was only sparsely populated. However, it also means that some block are written twice: Once when pre-zeroing them, and then when they are overwritten with actual data. On a full image, the pre-zeroing is wasted work because everything will be overwritten.

In practice, write_zeroes typically turns out faster than writing explicit zero buffers, but slow enough that first zeroing everything and then overwriting parts can be a significant net loss.

Meanwhile, qemu-img convert was rewritten in 690c7301600 and zero blocks are now written to the target using bdrv_co_pwrite_zeroes() if the target could be pre-zeroed. This way we already make use of the faster write_zeroes operation, but avoid writing any blocks twice.

Remove the pre-zeroing because these days this former optimisation has actually turned into a pessimisation in the common case.

---
 qemu-img.c | 9 ---------
 1 file changed, 9 deletions(-)
qemu-img.c +0 -9
index d7e846e607..bdb9f6aa46 100644
@@ -2084,15 +2084,6 @@ static int convert_do_copy(ImgConvertState *s)
         s->has_zero_init = bdrv_has_zero_init(blk_bs(s->target));
     }
 
-    if (!s->has_zero_init && !s->target_has_backing &&
-        bdrv_can_write_zeroes_with_unmap(blk_bs(s->target)))
-    {
-        ret = blk_make_zero(s->target, BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK);
-        if (ret == 0) {
-            s->has_zero_init = true;
-        }
-    }
-
     /* Allocate buffer for copied data. For compressed images, only one cluster
      * can be copied at a time. */
     if (s->compressed) {
Signature · 1 line
2.25.4
Tested-by on v1 1/1
#1 lore raw
Quoted 21 lines · Kevin Wolf

Since commit 5a37b60a61c, qemu-img create will pre-zero the target image if it isn't already zero-initialised (most importantly, for host block devices, but also iscsi etc.), so that writing explicit zeros wouldn't be necessary later.

This could speed up the operation significantly, in particular when the source image file was only sparsely populated. However, it also means that some block are written twice: Once when pre-zeroing them, and then when they are overwritten with actual data. On a full image, the pre-zeroing is wasted work because everything will be overwritten.

In practice, write_zeroes typically turns out faster than writing explicit zero buffers, but slow enough that first zeroing everything and then overwriting parts can be a significant net loss.

Meanwhile, qemu-img convert was rewritten in 690c7301600 and zero blocks are now written to the target using bdrv_co_pwrite_zeroes() if the target could be pre-zeroed. This way we already make use of the faster write_zeroes operation, but avoid writing any blocks twice.

Quoted 5 lines

Remove the pre-zeroing because these days this former optimisation has actually turned into a pessimisation in the common case.

Thanks, you can also add:

Quoted 27 lines
---
 qemu-img.c | 9 ---------
 1 file changed, 9 deletions(-)
qemu-img.c +0 -10
index d7e846e607..bdb9f6aa46 100644
@@ -2084,15 +2084,6 @@ static int convert_do_copy(ImgConvertState *s)
         s->has_zero_init = bdrv_has_zero_init(blk_bs(s->target));
     }
 
-    if (!s->has_zero_init && !s->target_has_backing &&
-        bdrv_can_write_zeroes_with_unmap(blk_bs(s->target)))
-    {
-        ret = blk_make_zero(s->target, BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK);
-        if (ret == 0) {
-            s->has_zero_init = true;
-        }
-    }
-
     /* Allocate buffer for copied data. For compressed images, only one cluster
      * can be copied at a time. */
     if (s->compressed) {
--

2.25.4

Reviewed-by on v1 1/1
#2 lore raw
In reply to Kevin Wolf
Quoted 15 lines · Kevin Wolf

Since commit 5a37b60a61c, qemu-img create will pre-zero the target image if it isn't already zero-initialised (most importantly, for host block devices, but also iscsi etc.), so that writing explicit zeros wouldn't be necessary later.

This could speed up the operation significantly, in particular when the source image file was only sparsely populated. However, it also means that some block are written twice: Once when pre-zeroing them, and then when they are overwritten with actual data. On a full image, the pre-zeroing is wasted work because everything will be overwritten.

In practice, write_zeroes typically turns out faster than writing explicit zero buffers, but slow enough that first zeroing everything and then overwriting parts can be a significant net loss.

Quoted 7 lines

Meanwhile, qemu-img convert was rewritten in 690c7301600 and zero blocks are now written to the target using bdrv_co_pwrite_zeroes() if the target could be pre-zeroed. This way we already make use of the faster write_zeroes operation, but avoid writing any blocks twice.

Remove the pre-zeroing because these days this former optimisation has actually turned into a pessimisation in the common case.

Indeed, it is both data-dependent (how sparse is the source) and target-dependent (does the reduced I/O from a single pre-zeroing pass outweigh the time spent on non-zero portions). I'm fine with dropping this optimization, as we do not have proper heuristics to reliably detect when it is actually going to be a win. (It would be nice if we _could_ have reliable heuristics, but that's a story for another day)

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org