qemu-img convert: Don't pre-zero images
PATCH 1/1 qemu-img convert: Don't pre-zero images
Since commit 5a37b60a61c, qemu-img create will pre-zero the target image if it isn't already zero-initialised (most importantly, for host block devices, but also iscsi etc.), so that writing explicit zeros wouldn't be necessary later.
This could speed up the operation significantly, in particular when the source image file was only sparsely populated. However, it also means that some block are written twice: Once when pre-zeroing them, and then when they are overwritten with actual data. On a full image, the pre-zeroing is wasted work because everything will be overwritten.
In practice, write_zeroes typically turns out faster than writing explicit zero buffers, but slow enough that first zeroing everything and then overwriting parts can be a significant net loss.
Meanwhile, qemu-img convert was rewritten in 690c7301600 and zero blocks are now written to the target using bdrv_co_pwrite_zeroes() if the target could be pre-zeroed. This way we already make use of the faster write_zeroes operation, but avoid writing any blocks twice.
Remove the pre-zeroing because these days this former optimisation has actually turned into a pessimisation in the common case.
--- qemu-img.c | 9 --------- 1 file changed, 9 deletions(-)
qemu-img.c +0 -9
@@ -2084,15 +2084,6 @@ static int convert_do_copy(ImgConvertState *s) s->has_zero_init =bdrv_has_zero_init (blk_bs (s->target)); } -if (!s->has_zero_init && !s->target_has_backing && -bdrv_can_write_zeroes_with_unmap (blk_bs (s->target))) - { - ret =blk_make_zero (s->target,BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ); -if (ret ==0 ) { - s->has_zero_init = true; - } - } -/* Allocate buffer for copied data. For compressed images, only one cluster * can be copied at a time. */ if (s->compressed) {
Signature · 1 line
2.25.4
Quoted 21 lines · Kevin Wolf
Since commit
5a37b60a61c, qemu-img create will pre-zero the target image if it isn't already zero-initialised (most importantly, for host block devices, but also iscsi etc.), so that writing explicit zeros wouldn't be necessary later.This could speed up the operation significantly, in particular when the source image file was only sparsely populated. However, it also means that some block are written twice: Once when pre-zeroing them, and then when they are overwritten with actual data. On a full image, the pre-zeroing is wasted work because everything will be overwritten.
In practice,
write_zeroestypically turns out faster than writing explicit zero buffers, but slow enough that first zeroing everything and then overwriting parts can be a significant net loss.Meanwhile, qemu-img convert was rewritten in
690c7301600and zero blocks are now written to the target usingbdrv_co_pwrite_zeroes()if the target could be pre-zeroed. This way we already make use of the fasterwrite_zeroesoperation, but avoid writing any blocks twice.
Quoted 5 lines
Remove the pre-zeroing because these days this former optimisation has actually turned into a pessimisation in the common case.
Reported-by Nir Soffer <[email protected]>Signed-off-by Kevin Wolf <[email protected]>
Thanks, you can also add:
Quoted 27 lines
--- qemu-img.c | 9 --------- 1 file changed, 9 deletions(-)qemu-img.c +0
-10@@ -2084,15 +2084,6 @@ static int convert_do_copy(ImgConvertState *s) s->has_zero_init =bdrv_has_zero_init (blk_bs (s->target)); } -if (!s->has_zero_init && !s->target_has_backing && -bdrv_can_write_zeroes_with_unmap (blk_bs (s->target))) - { - ret =blk_make_zero (s->target,BDRV_REQ_MAY_UNMAP |BDRV_REQ_NO_FALLBACK ); -if (ret ==0 ) { - s->has_zero_init = true; - } - } -/* Allocate buffer for copied data. For compressed images, only one cluster * can be copied at a time. */ if (s->compressed) { --2.25.4
Quoted 15 lines · Kevin Wolf
Since commit
5a37b60a61c, qemu-img create will pre-zero the target image if it isn't already zero-initialised (most importantly, for host block devices, but also iscsi etc.), so that writing explicit zeros wouldn't be necessary later.This could speed up the operation significantly, in particular when the source image file was only sparsely populated. However, it also means that some block are written twice: Once when pre-zeroing them, and then when they are overwritten with actual data. On a full image, the pre-zeroing is wasted work because everything will be overwritten.
In practice,
write_zeroestypically turns out faster than writing explicit zero buffers, but slow enough that first zeroing everything and then overwriting parts can be a significant net loss.
Quoted 7 lines
Meanwhile, qemu-img convert was rewritten in
690c7301600and zero blocks are now written to the target usingbdrv_co_pwrite_zeroes()if the target could be pre-zeroed. This way we already make use of the fasterwrite_zeroesoperation, but avoid writing any blocks twice.Remove the pre-zeroing because these days this former optimisation has actually turned into a pessimisation in the common case.
Indeed, it is both data-dependent (how sparse is the source) and target-dependent (does the reduced I/O from a single pre-zeroing pass outweigh the time spent on non-zero portions). I'm fine with dropping this optimization, as we do not have proper heuristics to reliably detect when it is actually going to be a win. (It would be nice if we _could_ have reliable heuristics, but that's a story for another day)
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org