qemu-img: add preallocation=full
v3 cover qemu-img: add preallocation=full
This series implements full image preallocation to create a non-sparse image file at creation time, both for raw and qcow2 format. The purpose is to avoid performance deterioration of the guest cause by sparse image.
v3: - Fix comments to v2 by Fam.
- qcow2: first fallocate disk space, then allocate metadata. This avoids the problem in v2 that bdrv_preallocate may clear all information in metadata. This does not necessarily map all data clusters sequentially but does keep information in metadata. Peter, is this acceptable?
Hu Tao (6):
block: introduce prealloc_mode
block: add BlockDriver.bdrv_preallocate.
block/raw-posix: implement bdrv_preallocate
raw-posix: Add full image preallocation option
qcow2: implement bdrv_preallocate
qcow2: Add full image preallocation option
block.c | 13 +++++++++++ block/qcow2.c | 33 ++++++++++++++++++++++------ block/raw-posix.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++ include/block/block.h | 7 ++++++ include/block/block_int.h | 3 +++ 5 files changed, 105 insertions(+), 7 deletions(-)
Signature · 1 line
1.7.11.7
RFC PATCH v3 1/6 block: introduce prealloc_mode
This patch prepares for the subsequent patches.
--- block/qcow2.c | 6 +++--- include/block/block.h | 6 ++++++ 2 files changed, 9 insertions(+), 3 deletions(-)
block/qcow2.c +3 -3
@@ -1614,7 +1614,7 @@ static int qcow2_create(const char *filename, QEMUOptionParameter *options,uint64_t sectors =0 ;int flags =0 ;size_t cluster_size =DEFAULT_CLUSTER_SIZE ; -int prealloc =0 ; +int prealloc =PREALLOC_OFF ;int version =3 ;Error *local_err =NULL ;int ret;
@@ -1635,9 +1635,9 @@ static int qcow2_create(const char *filename, QEMUOptionParameter *options, } }else if (!strcmp (options->name,BLOCK_OPT_PREALLOC )) {if (!options->value.s || !strcmp (options->value.s,"off" )) { - prealloc =0 ; + prealloc =PREALLOC_OFF ; }else if (!strcmp (options->value.s,"metadata" )) { - prealloc =1 ; + prealloc =PREALLOC_METADATA ; }else {error_setg (errp,"Invalid preallocation mode: '%s'" , options->value.s);
include/block/block.h +6 -0
@@ -527,4 +527,10 @@ int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag);int bdrv_debug_resume (BlockDriverState *bs,const char *tag);bool bdrv_debug_is_suspended (BlockDriverState *bs,const char *tag); +enum prealloc_mode { +PREALLOC_OFF =0 , +PREALLOC_METADATA , +PREALLOC_FULL , +}; +#endif
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [RFC PATCH v3 1/6] block: introduce prealloc_mode
include/block/block.h int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag); · line 530
Quoted 9 lines · Hu Tao
include/block/block.h +1
-0@@ -527,4 +527,10 @@ int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag);int bdrv_debug_resume (BlockDriverState *bs,const char *tag);bool bdrv_debug_is_suspended (BlockDriverState *bs,const char *tag); +enum prealloc_mode {
Please follow modern QEMU coding style and use CamelCase: PreallocMode
It would be nice to use PreallocMode instead int for variables.
Re: [Qemu-devel] [RFC PATCH v3 1/6] block: introduce prealloc_mode
In reply to Hu TaoQuoted 11 lines · Hu Tao
This patch prepares for the subsequent patches.
Signed-off-by Hu Tao <[email protected]>--- block/qcow2.c | 6 +++--- include/block/block.h | 6 ++++++ 2 files changed, 9 insertions(+), 3 deletions(-)+++
b/include/block/block.h
include/block/block.h int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag); · line 534
Quoted 8 lines
include/block/block.h +5
-0@@ -527,4 +527,10 @@ int bdrv_debug_remove_breakpoint(BlockDriverState *bs, const char *tag);int bdrv_debug_resume (BlockDriverState *bs,const char *tag);bool bdrv_debug_is_suspended (BlockDriverState *bs,const char *tag); +enum prealloc_mode { +PREALLOC_OFF =0 , +PREALLOC_METADATA , +PREALLOC_FULL , +};
Why not make this a QAPI enum:
{ 'enum': 'PreallocMode', 'data': ['off', 'metadata', 'full'] }
After all, this is the sort of thing where we will eventually want to select the prealloc mode of new files being created by online operations such as snapshot creation.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
RFC PATCH v3 2/6 block: add BlockDriver.bdrv_preallocate.
In reply to Hu TaoThis field is used to preallocate disk space for block device.
--- block.c | 13 +++++++++++++ include/block/block.h | 1 + include/block/block_int.h | 3 +++ 3 files changed, 17 insertions(+)
block.c +13 -0
@@ -3216,6 +3216,19 @@ bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs)return false; } +int bdrv_preallocate (BlockDriverState *bs,int64_t offset,int64_t length) +{ +if (bs->backing_hd) { +return -ENOTSUP ; + } + +if (bs->drv->bdrv_preallocate) { +return bs->drv->bdrv_preallocate (bs, offset, length); + } + +return -ENOTSUP ; +} + typedef struct BdrvCoGetBlockStatusData { BlockDriverState *bs; BlockDriverState *base;
include/block/block.h +1 -0
@@ -349,6 +349,7 @@ int bdrv_discard(BlockDriverState *bs, int64_t sector_num, int nb_sectors);int bdrv_co_discard (BlockDriverState *bs,int64_t sector_num,int nb_sectors);int bdrv_has_zero_init_1 (BlockDriverState *bs);int bdrv_has_zero_init (BlockDriverState *bs); +int bdrv_preallocate (BlockDriverState *bs,int64_t offset,int64_t length);bool bdrv_unallocated_blocks_are_zero (BlockDriverState *bs);bool bdrv_can_write_zeroes_with_unmap (BlockDriverState *bs); int64_t bdrv_get_block_status(BlockDriverState *bs, int64_t sector_num,
include/block/block_int.h +3 -0
@@ -232,6 +232,9 @@ struct BlockDriver {*/ int (*bdrv_has_zero_init)(BlockDriverState *bs); +int (*bdrv_preallocate)(BlockDriverState *bs,int64_t offset, +int64_t length); +QLIST_ENTRY (BlockDriver) list; };
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [RFC PATCH v3 2/6] block: add BlockDriver.bdrv_preallocate.
block.c bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs) · line 3223
Quoted 13 lines · Hu Tao
block.c +5
-0@@ -3216,6 +3216,19 @@ bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs)return false; } +int bdrv_preallocate (BlockDriverState *bs,int64_t offset,int64_t length) +{ +if (bs->backing_hd) { +return -ENOTSUP ; + }
Depending on the image file format it may be possible to preallocate metadata while using a backing file. Why prevent this?
Re: [Qemu-devel] [RFC PATCH v3 2/6] block: add BlockDriver.bdrv_preallocate.
Quoted 9 lines · Stefan Hajnoczi
Quoted 8 lines · Hu Tao
block.c +0
-0@@ -3216,6 +3216,19 @@ bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs)return false; }
Quoted 8 lines
block.c bool bdrv_can_write_zeroes_with_unmap(BlockDriverState *bs) · line 3223
Quoted 5 lines
block.c +5
-0+int bdrv_preallocate (BlockDriverState *bs,int64_t offset,int64_t length) +{ +if (bs->backing_hd) { +return -ENOTSUP ; + }Depending on the image file format it may be possible to preallocate metadata while using a backing file. Why prevent this?
I thought in the case we have no need to preallocate forbacking file. But yes, we can also preallocate for bs when there is backing file. Thanks!
RFC PATCH v3 3/6 block/raw-posix: implement bdrv_preallocate
In reply to Hu Tao--- block/raw-posix.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
block/raw-posix.c +34 -0
@@ -1160,6 +1160,39 @@ static int64_t raw_get_allocated_file_size(BlockDriverState *bs)return (int64_t )st.st_blocks *512 ; } +#ifdef __linux__ +static int raw_preallocate2 (int fd,int64_t offset,int64_t length) +{ +int ret =-1 ; + + ret =fallocate (fd,0 , offset, length); + +/* fallback to posix_fallocate() if fallocate() is not supported */ +if (ret <0 && (errno ==ENOSYS || errno ==EOPNOTSUPP )) { + ret =posix_fallocate (fd, offset, length); + } + +return ret; +} +#else +static int raw_preallocate2 (int fd,int64_t offset,int64_t length) +{ +return posix_fallocate (fd, offset, length); +} +#endif + +static int raw_preallocate (BlockDriverState *bs,int64_t offset,int64_t length) +{ +BDRVRawState *s = bs->opaque; +int64_t len =bdrv_getlength (bs); + +if (offset + length <0 || offset + length > len) { +return -EINVAL ; + } + +return raw_preallocate2 (s->fd, offset, length); +} +static int raw_create (const char *filename,QEMUOptionParameter *options,Error **errp) {
@@ -1356,6 +1389,7 @@ static BlockDriver bdrv_file = { .bdrv_close = raw_close, .bdrv_create = raw_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, + .bdrv_preallocate = raw_preallocate, .bdrv_co_get_block_status = raw_co_get_block_status, .bdrv_co_write_zeroes = raw_co_write_zeroes,
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [RFC PATCH v3 3/6] block/raw-posix: implement bdrv_preallocate
block/raw-posix.c static int64_t raw_get_allocated_file_size(BlockDriverState *bs) · line 1175
Quoted 12 lines · Hu Tao
block/raw-posix.c +12
-0+static int raw_preallocate2 (int fd,int64_t offset,int64_t length) +{ +int ret =-1 ; + + ret =fallocate (fd,0 , offset, length); + +/* fallback to posix_fallocate() if fallocate() is not supported */ +if (ret <0 && (errno ==ENOSYS || errno ==EOPNOTSUPP )) { + ret =posix_fallocate (fd, offset, length); + } + +return ret;
Return value semantics differ between the two functions:
* fallocate - return 0 or -1 with errno set
* posix_fallocate - return 0 or error number (without using errno!)
Please make it consistent. Usually in QEMU we return 0 on success and -errno on failure.
Re: [Qemu-devel] [RFC PATCH v3 3/6] block/raw-posix: implement bdrv_preallocate
Quoted 18 lines · Stefan Hajnoczi
block/raw-posix.c static int64_t raw_get_allocated_file_size(BlockDriverState *bs) · line 1175
Quoted 12 lines · Hu Tao
block/raw-posix.c +12
-0+static int raw_preallocate2 (int fd,int64_t offset,int64_t length) +{ +int ret =-1 ; + + ret =fallocate (fd,0 , offset, length); + +/* fallback to posix_fallocate() if fallocate() is not supported */ +if (ret <0 && (errno ==ENOSYS || errno ==EOPNOTSUPP )) { + ret =posix_fallocate (fd, offset, length); + } + +return ret;Return value semantics differ between the two functions:
* fallocate - return 0 or -1 with errno set
*posix_fallocate- return 0 or error number (without using errno!)
Quoted 2 lines
Please make it consistent. Usually in QEMU we return 0 on success and -errno on failure.
Thanks for catching this!
RFC PATCH v3 4/6 raw-posix: Add full image preallocation option
In reply to Hu TaoThis patch adds a new option preallocation for raw format, and implements full preallocation.
--- block/raw-posix.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)
block/raw-posix.c +22 -0
@@ -1199,11 +1199,22 @@ static int raw_create(const char *filename, QEMUOptionParameter *options,int fd;int result =0 ;int64_t total_size =0 ; +int prealloc =PREALLOC_OFF ;/* Read out options */ while (options && options->name) {if (!strcmp (options->name,BLOCK_OPT_SIZE )) { total_size = options->value.n /BDRV_SECTOR_SIZE ; + }else if (!strcmp (options->name,BLOCK_OPT_PREALLOC )) { +if (!options->value.s || !strcmp (options->value.s,"off" )) { + prealloc =PREALLOC_OFF ; + }else if (!strcmp (options->value.s,"full" )) { + prealloc =PREALLOC_FULL ; + }else { +error_setg (errp,"Invalid preallocation mode: '%s'" , + options->value.s); +return -EINVAL ; + } } options++; }
@@ -1218,6 +1229,12 @@ static int raw_create(const char *filename, QEMUOptionParameter *options, result = -errno;error_setg_errno (errp, -result,"Could not resize file" ); } +if (prealloc ==PREALLOC_FULL && +raw_preallocate2 (fd,0 , total_size *BDRV_SECTOR_SIZE ) !=0 ) { + result = -errno; +error_setg_errno (errp, -result, +"Could not preallocate data for the new file" ); + }if (qemu_close (fd) !=0 ) { result = -errno;error_setg_errno (errp, -result,"Could not close the new file" );
@@ -1373,6 +1390,11 @@ static QEMUOptionParameter raw_create_options[] = { .type = OPT_SIZE, .help ="Virtual disk size" }, + { + .name = BLOCK_OPT_PREALLOC, + .type = OPT_STRING, + .help ="Preallocation mode (allowed values: off, full)" + }, { NULL } };
Signature · 1 line
1.7.11.7
RFC PATCH v3 5/6 qcow2: implement bdrv_preallocate
In reply to Hu Tao--- block/qcow2.c | 7 +++++++ 1 file changed, 7 insertions(+)
block/qcow2.c +7 -0
@@ -2188,6 +2188,12 @@ static int qcow2_amend_options(BlockDriverState *bs,return 0 ; } +static int qcow2_preallocate (BlockDriverState *bs,int64_t offset, +int64_t length) +{ +return bdrv_preallocate (bs->file, offset, length); +} +static QEMUOptionParameter qcow2_create_options[] = { { .name =BLOCK_OPT_SIZE ,
@@ -2242,6 +2248,7 @@ static BlockDriver bdrv_qcow2 = { .bdrv_reopen_prepare = qcow2_reopen_prepare, .bdrv_create = qcow2_create, .bdrv_has_zero_init = bdrv_has_zero_init_1, + .bdrv_preallocate = qcow2_preallocate, .bdrv_co_get_block_status = qcow2_co_get_block_status, .bdrv_set_key = qcow2_set_key, .bdrv_make_empty = qcow2_make_empty,
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [RFC PATCH v3 5/6] qcow2: implement bdrv_preallocate
Quoted 10 lines · Hu Tao
Signed-off-by Hu Tao <[email protected]>--- block/qcow2.c | 7 +++++++ 1 file changed, 7 insertions(+)block/qcow2.c +0
-0
block/qcow2.c static int qcow2_amend_options(BlockDriverState *bs, · line 2195
Quoted 8 lines
block/qcow2.c +5
-0@@ -2188,6 +2188,12 @@ static int qcow2_amend_options(BlockDriverState *bs,return 0 ; } +static int qcow2_preallocate (BlockDriverState *bs,int64_t offset, +int64_t length) +{ +return bdrv_preallocate (bs->file, offset, length); +}
What about qcow2-level preallocation (metadata)?
I'm not sure what the meaning of offset and length are here - are they supposed to be virtual disk LBAs. They are being passed through to bs->file so they actually become physical file offsets.
RFC PATCH v3 6/6 qcow2: Add full image preallocation option
In reply to Hu TaoThis adds a preallocation=full mode to qcow2 image creation, which creates a non-sparse image file.
--- block/qcow2.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
block/qcow2.c +16 -4
@@ -1386,7 +1386,7 @@ static int qcow2_change_backing_file(BlockDriverState *bs,return qcow2_update_header (bs); } -static int preallocate (BlockDriverState *bs) +static int preallocate (BlockDriverState *bs,enum prealloc_mode mode) {uint64_t nb_sectors;uint64_t offset;
@@ -1395,9 +1395,19 @@ static int preallocate(BlockDriverState *bs)int ret;QCowL2Meta *meta; +assert (mode !=PREALLOC_OFF ); + nb_sectors =bdrv_getlength (bs) >>9 ; offset =0 ; +if (mode ==PREALLOC_FULL ) { + ret =bdrv_preallocate (bs->file,0 ,bdrv_getlength (bs)); +if (ret <0 ) { +return ret; + } + } + +/* allocate metadata */ while (nb_sectors) { num =MIN (nb_sectors,INT_MAX >>9 ); ret = qcow2_alloc_cluster_offset(bs, offset, 0, num, &num,
@@ -1577,11 +1587,11 @@ static int qcow2_create2(const char *filename, int64_t total_size, } } -/* And if we're supposed to preallocate metadata, do that now */ +/* And if we're supposed to preallocate data, do that now */ if (prealloc) {BDRVQcowState *s = bs->opaque;qemu_co_mutex_lock (&s->lock); - ret =preallocate (bs); + ret =preallocate (bs, prealloc);qemu_co_mutex_unlock (&s->lock);if (ret <0 ) {error_setg_errno (errp, -ret,"Could not preallocate metadata" );
@@ -1638,6 +1648,8 @@ static int qcow2_create(const char *filename, QEMUOptionParameter *options, prealloc =PREALLOC_OFF ; }else if (!strcmp (options->value.s,"metadata" )) { prealloc =PREALLOC_METADATA ; + }else if (!strcmp (options->value.s,"full" )) { + prealloc =PREALLOC_FULL ; }else {error_setg (errp,"Invalid preallocation mode: '%s'" , options->value.s);
@@ -2229,7 +2241,7 @@ static QEMUOptionParameter qcow2_create_options[] = { { .name = BLOCK_OPT_PREALLOC, .type = OPT_STRING, - .help ="Preallocation mode (allowed values: off, metadata)" + .help ="Preallocation mode (allowed values: off, metadata, full)" }, { .name = BLOCK_OPT_LAZY_REFCOUNTS,
Signature · 1 line
1.7.11.7
Re: [Qemu-devel] [RFC PATCH v3 6/6] qcow2: Add full image preallocation option
Quoted 6 lines · Hu Tao
block/qcow2.c +1
-1-static int preallocate (BlockDriverState *bs) +static int preallocate (BlockDriverState *bs,enum prealloc_mode mode) {uint64_t nb_sectors;uint64_t offset;
block/qcow2.c static int preallocate(BlockDriverState *bs) · line 1404
Quoted 10 lines
block/qcow2.c +4
-0@@ -1395,9 +1395,19 @@ static int preallocate(BlockDriverState *bs)int ret;QCowL2Meta *meta; +assert (mode !=PREALLOC_OFF ); + nb_sectors =bdrv_getlength (bs) >>9 ; offset =0 ; +if (mode ==PREALLOC_FULL ) { + ret =bdrv_preallocate (bs->file,0 ,bdrv_getlength (bs));
This doesn't make sense since it fails to factor in qcow2 metadata and the file layout.
Quoted 3 lines · Hu Tao
This series implements full image preallocation to create a non-sparse image file at creation time, both for raw and qcow2 format. The purpose is to avoid performance deterioration of the guest cause by sparse image.
I'm not sure the new bdrv_preallocate() interface is necessary. Can qcow2_create() simply pass a preallocate=full option to bdrv_create_file()?
It's simpler if we avoid bdrv_preallocate(), just keep it a .bdrv_create() option. That way we also avoid the question of how exactly preallocate functions on an image file that already has blocks allocated.
So what I imagine is:
1. Add preallocation=full support to raw-posix.c
2. Add preallocation=full support to qcow2.c, calculate image file size including metadata for bdrv_create_file().
CCing Eric Blake so libvirt can consider how full preallocation interacts with their safezero() preallocation. If QEMU handles preallocation can we skip safezero()? Is there a concern about full preallocation in QEMU taking a long time without progress report?
Stefan
Quoted 19 lines · Stefan Hajnoczi
Quoted 3 lines · Hu Tao
This series implements full image preallocation to create a non-sparse image file at creation time, both for raw and qcow2 format. The purpose is to avoid performance deterioration of the guest cause by sparse image.
I'm not sure the new
bdrv_preallocate()interface is necessary. Canqcow2_create()simply pass a preallocate=full option tobdrv_create_file()?It's simpler if we avoid
bdrv_preallocate(), just keep it a .bdrv_create() option. That way we also avoid the question of how exactly preallocate functions on an image file that already has blocks allocated.So what I imagine is:
1. Add preallocation=full support toraw-posix.c
2. Add preallocation=full support toqcow2.c, calculate image file size including metadata forbdrv_create_file().
Quoted 4 lines
CCing Eric Blake so libvirt can consider how full preallocation interacts with their
safezero()preallocation. If QEMU handles preallocation can we skipsafezero()? Is there a concern about full preallocation in QEMU taking a long time without progress report?
If libvirt can detect that qemu is new enough to do preallocation, then libvirt can skip its own preallocation. Which means you need to make sure that QMP can probe the existence of this new feature (probably already possible via query-command-line-options, but see also my reply to 1/6 about starting to put this in QAPI).
The fact that preallocation is sometimes long running is definitely worth considering; but it raises the more generic issue of how to turn any long running blocking command into an asynchronous job that gets started, then has both polling and event-notification progress until completion, as well as the option of an early abort if it is taking too long.
And there's my long-standing gripe that qemu-img doesn't support -p progress meters for enough of its already-existing long-running commands, which makes it harder for libvirt to provide wrappers around qemu-img operations when manipulating images not associated with a running VM.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Quoted 8 lines · Stefan Hajnoczi
Quoted 3 lines · Hu Tao
This series implements full image preallocation to create a non-sparse image file at creation time, both for raw and qcow2 format. The purpose is to avoid performance deterioration of the guest cause by sparse image.
I'm not sure the new
bdrv_preallocate()interface is necessary. Canqcow2_create()simply pass a preallocate=full option tobdrv_create_file()?
Looks simpler. I'll try this.
Quoted 17 lines
It's simpler if we avoid
bdrv_preallocate(), just keep it a .bdrv_create() option. That way we also avoid the question of how exactly preallocate functions on an image file that already has blocks allocated.So what I imagine is:
1. Add preallocation=full support toraw-posix.c
2. Add preallocation=full support toqcow2.c, calculate image file size including metadata forbdrv_create_file().CCing Eric Blake so libvirt can consider how full preallocation interacts with their
safezero()preallocation. If QEMU handles preallocation can we skipsafezero()? Is there a concern about full preallocation in QEMU taking a long time without progress report?Stefan