block: seriously improve savevm/loadvm performance
v8 cover block: seriously improve savevm/loadvm performance
This series do standard basic things:
- it creates intermediate buffer for all writes from QEMU migration code to QCOW2 image,
- this buffer is sent to disk asynchronously, allowing several writes to run in parallel.
In general, migration code is fantastically inefficent (by observation), buffers are not aligned and sent with arbitrary pieces, a lot of time less than 100 bytes at a chunk, which results in read-modify-write operations with non-cached operations. It should also be noted that all operations are performed into unallocated image blocks, which also suffer due to partial writes to such new clusters.
This patch series is an implementation of idea discussed in the RFC posted by Denis Plotnikov
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg01925.html
Results with this series over NVME are better than original code
original rfc this cached: 1.79s 2.38s 1.27s non-cached: 3.29s 1.31s 0.81s
Changes from v7:
- dropped lock from LoadVMState
- fixed assert in last patch
- dropped patch 1 as queued
Changes from v6:
- blk_load_vmstate kludges added (patchew problem fixed)
Changes from v5:
- loadvm optimizations added with Vladimir comments included
Changes from v4:
- added patch 4 with blk_save_vmstate() cleanup
- added R-By
- bdrv_flush_vmstate -> bdrv_finalize_vmstate
- fixed return code of bdrv_co_do_save_vmstate
- fixed typos in comments (Eric, thanks!)
- fixed patchew warnings
Changes from v3:
- rebased to master
- added patch 3 which removes aio_task_pool_wait_one()
- added R-By to patch 1
- patch 4 is rewritten via bdrv_run_co
- error path in blk_save_vmstate() is rewritten to call bdrv_flush_vmstate unconditionally
- added some comments
- fixes initialization in bdrv_co_vmstate_save_task_entry as suggested
Changes from v2:
- code moved from QCOW2 level to generic block level
- created bdrv_flush_vmstate helper to fix 022, 029 tests
- added recursive for bs->file in bdrv_co_flush_vmstate (fix 267)
- fixed blk_save_vmstate helper
- fixed coroutine wait as Vladimir suggested with waiting fixes from me
Changes from v1:
- patchew warning fixed
- fixed validation that only 1 waiter is allowed in patch 1
PATCH v8 1/6 block/aio_task: allow start/wait task from any coroutine
From: Vladimir Sementsov-Ogievskiy <[email protected]>
Currently, aio task pool assumes that there is a main coroutine, which creates tasks and wait for them. Let's remove the restriction by using CoQueue. Code becomes clearer, interface more obvious.
--- block/aio_task.c | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-)
block/aio_task.c +6 -15
@@ -27,11 +27,10 @@#include "block/aio_task.h" struct AioTaskPool { -Coroutine *main_co;int status;int max_busy_tasks;int busy_tasks; -bool waiting; +CoQueue waiters; }; static void coroutine_fn aio_task_co(void *opaque)
@@ -52,31 +51,23 @@ static void coroutine_fn aio_task_co(void *opaque)g_free (task); -if (pool->waiting) { - pool->waiting = false; -aio_co_wake (pool->main_co); - } +qemu_co_queue_restart_all (&pool->waiters); } void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool) {assert (pool->busy_tasks >0 ); -assert (qemu_coroutine_self () == pool->main_co); - pool->waiting = true; -qemu_coroutine_yield (); +qemu_co_queue_wait (&pool->waiters,NULL ); -assert (!pool->waiting);assert (pool->busy_tasks < pool->max_busy_tasks); } void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool) { -if (pool->busy_tasks < pool->max_busy_tasks) { -return ; +while (pool->busy_tasks >= pool->max_busy_tasks) { +aio_task_pool_wait_one (pool); } - -aio_task_pool_wait_one (pool); } void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool)
@@ -98,8 +89,8 @@ AioTaskPool *coroutine_fn aio_task_pool_new(int max_busy_tasks) {AioTaskPool *pool =g_new0 (AioTaskPool,1 ); - pool->main_co =qemu_coroutine_self (); pool->max_busy_tasks = max_busy_tasks; +qemu_co_queue_init (&pool->waiters);return pool; }
Signature · 1 line
2.17.1
PATCH v8 2/6 block/aio_task: drop aio_task_pool_wait_one() helper
In reply to Denis V. LunevIt is not used outside the module.
Actually there are 2 kind of waiters:
- for a slot and
- for all tasks to finish
This patch limits external API to listed types.
--- block/aio_task.c | 13 ++----------- include/block/aio_task.h | 1 - 2 files changed, 2 insertions(+), 12 deletions(-)
block/aio_task.c +2 -11
@@ -54,26 +54,17 @@ static void coroutine_fn aio_task_co(void *opaque)qemu_co_queue_restart_all (&pool->waiters); } -void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool) -{ -assert (pool->busy_tasks >0 ); - -qemu_co_queue_wait (&pool->waiters,NULL ); - -assert (pool->busy_tasks < pool->max_busy_tasks); -} - void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool) {while (pool->busy_tasks >= pool->max_busy_tasks) { -aio_task_pool_wait_one (pool); +qemu_co_queue_wait (&pool->waiters,NULL ); } } void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool) {while (pool->busy_tasks >0 ) { -aio_task_pool_wait_one (pool); +qemu_co_queue_wait (&pool->waiters,NULL ); } }
include/block/aio_task.h +0 -1
@@ -48,7 +48,6 @@ bool aio_task_pool_empty(AioTaskPool *pool); void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *task); void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool); -void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool); void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool);#endif /* BLOCK_AIO_TASK_H */
Signature · 1 line
2.17.1
PATCH v8 3/6 block/block-backend: remove always true check from blk_save_vmstate
In reply to Denis V. Lunevbdrv_save_vmstate() returns either error with negative return value or size. Thus this check is useless.
--- block/block-backend.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
block/block-backend.c +1 -1
@@ -2188,7 +2188,7 @@ int blk_save_vmstate(BlockBackend *blk, const uint8_t *buf,return ret; } -if (ret == size && !blk->enable_write_cache) { +if (!blk->enable_write_cache) { ret =bdrv_flush (blk_bs (blk)); }
Signature · 1 line
2.17.1
PATCH v8 4/6 block, migration: add bdrv_finalize_vmstate helper
In reply to Denis V. LunevRight now bdrv_fclose() is just calling bdrv_flush().
The problem is that migration code is working inefficiently from block layer terms and are frequently called for very small pieces of unaligned data. Block layer is capable to work this way, but this is very slow.
This patch is a preparation for the introduction of the intermediate buffer at block driver state. It would be beneficial to separate conventional bdrv_flush() from closing QEMU file from migration code.
The patch also forces bdrv_finalize_vmstate() operation inside synchronous blk_save_vmstate() operation. This helper is used from qemu-io only.
--- block/block-backend.c | 6 +++++- block/io.c | 15 +++++++++++++++ include/block/block.h | 5 +++++ migration/savevm.c | 4 ++++ 4 files changed, 29 insertions(+), 1 deletion(-)
block/block-backend.c +5 -1
@@ -2177,16 +2177,20 @@ int blk_truncate(BlockBackend *blk, int64_t offset, bool exact,int blk_save_vmstate (BlockBackend *blk,const uint8_t *buf,int64_t pos,int size) { -int ret; +int ret, ret2;if (!blk_is_available (blk)) {return -ENOMEDIUM ; } ret =bdrv_save_vmstate (blk_bs (blk), buf, pos, size); + ret2 =bdrv_finalize_vmstate (blk_bs (blk));if (ret <0 ) {return ret; } +if (ret2 <0 ) { +return ret2; + }if (!blk->enable_write_cache) { ret =bdrv_flush (blk_bs (blk));
block/io.c +15 -0
@@ -2724,6 +2724,21 @@ int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos)return bdrv_rw_vmstate (bs, qiov, pos, true); } +static int coroutine_fn bdrv_co_finalize_vmstate(BlockDriverState *bs) +{ +return 0 ; +} + +static int coroutine_fn bdrv_finalize_vmstate_co_entry(void *opaque) +{ +return bdrv_co_finalize_vmstate (opaque); +} + +int bdrv_finalize_vmstate (BlockDriverState *bs) +{ +return bdrv_run_co (bs, bdrv_finalize_vmstate_co_entry, bs); +} +/**************************************************************/ /* async I/Os */
include/block/block.h +5 -0
@@ -567,6 +567,11 @@ int bdrv_save_vmstate(BlockDriverState *bs, const uint8_t *buf,int bdrv_load_vmstate (BlockDriverState *bs,uint8_t *buf,int64_t pos,int size); +/* +* bdrv_finalize_vmstate() is mandatory to commit vmstate changes if +* bdrv_save_vmstate() was ever called. +*/ +int bdrv_finalize_vmstate (BlockDriverState *bs); void bdrv_img_create(const char *filename, const char *fmt, const char *base_filename, const char *base_fmt,
migration/savevm.c +4 -0
@@ -150,6 +150,10 @@ static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,static int bdrv_fclose (void *opaque,Error **errp) { +int err =bdrv_finalize_vmstate (opaque); +if (err <0 ) { +return err; + }return bdrv_flush (opaque); }
Signature · 1 line
2.17.1
Re: [PATCH 4/6] block, migration: add bdrv_finalize_vmstate helper
Quoted 36 lines · Denis V. Lunev
Right now
bdrv_fclose()is just callingbdrv_flush().The problem is that migration code is working inefficiently from block layer terms and are frequently called for very small pieces of unaligned data. Block layer is capable to work this way, but this is very slow.
This patch is a preparation for the introduction of the intermediate buffer at block driver state. It would be beneficial to separate conventional
bdrv_flush()from closing QEMU file from migration code.The patch also forces
bdrv_finalize_vmstate()operation inside synchronousblk_save_vmstate()operation. This helper is used from qemu-io only.Signed-off-by Denis V. Lunev <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>CC Kevin Wolf <[email protected]>CC Max Reitz <[email protected]>CC Stefan Hajnoczi <[email protected]>CC Fam Zheng <[email protected]>CC Juan Quintela <[email protected]>CC "Dr. David Alan Gilbert" <[email protected]>CC Denis Plotnikov <[email protected]>--- block/block-backend.c | 6 +++++- block/io.c | 15 +++++++++++++++ include/block/block.h | 5 +++++ migration/savevm.c | 4 ++++ 4 files changed, 29 insertions(+), 1 deletion(-)migration/savevm.c +0
-0
migration/savevm.c static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos, · line 155
Quoted 6 lines
migration/savevm.c +3
-0@@ -150,6 +150,10 @@ static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,static int bdrv_fclose (void *opaque,Error **errp) { +int err =bdrv_finalize_vmstate (opaque); +if (err <0 ) { +return err;
This is returning an error without having populating 'errp' which means the caller will be missing error diagnosis
migration/savevm.c static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos, · line 158
Quoted 3 lines
migration/savevm.c +1
-0+ }return bdrv_flush (opaque); }
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 4/6] block, migration: add bdrv_finalize_vmstate helper
Quoted 37 lines · Daniel P. Berrangé
Quoted 36 lines · Denis V. Lunev
Right now
bdrv_fclose()is just callingbdrv_flush().The problem is that migration code is working inefficiently from block layer terms and are frequently called for very small pieces of unaligned data. Block layer is capable to work this way, but this is very slow.
This patch is a preparation for the introduction of the intermediate buffer at block driver state. It would be beneficial to separate conventional
bdrv_flush()from closing QEMU file from migration code.The patch also forces
bdrv_finalize_vmstate()operation inside synchronousblk_save_vmstate()operation. This helper is used from qemu-io only.Signed-off-by Denis V. Lunev <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>CC Kevin Wolf <[email protected]>CC Max Reitz <[email protected]>CC Stefan Hajnoczi <[email protected]>CC Fam Zheng <[email protected]>CC Juan Quintela <[email protected]>CC "Dr. David Alan Gilbert" <[email protected]>CC Denis Plotnikov <[email protected]>--- block/block-backend.c | 6 +++++- block/io.c | 15 +++++++++++++++ include/block/block.h | 5 +++++ migration/savevm.c | 4 ++++ 4 files changed, 29 insertions(+), 1 deletion(-)migration/savevm.c +0
-0@@ -150,6 +150,10 @@ static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
Quoted 7 lines
migration/savevm.c static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos, · line 155
Quoted 5 lines
migration/savevm.c +3
-0static int bdrv_fclose (void *opaque,Error **errp) { +int err =bdrv_finalize_vmstate (opaque); +if (err <0 ) { +return err;This is returning an error without having populating 'errp' which means the caller will be missing error diagnosis
but this behaves exactly like the branch below,bdrv_flush() could return error too and errp
is not filled in the same way.
Den
Quoted 5 lines
migration/savevm.c static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos, · line 158
Quoted 3 lines
migration/savevm.c +1
-0+ }return bdrv_flush (opaque); }Regards,
Daniel
Re: [PATCH 4/6] block, migration: add bdrv_finalize_vmstate helper
Quoted 46 lines · Denis V. Lunev
Quoted 37 lines · Daniel P. Berrangé
Quoted 36 lines · Denis V. Lunev
Right now
bdrv_fclose()is just callingbdrv_flush().The problem is that migration code is working inefficiently from block layer terms and are frequently called for very small pieces of unaligned data. Block layer is capable to work this way, but this is very slow.
This patch is a preparation for the introduction of the intermediate buffer at block driver state. It would be beneficial to separate conventional
bdrv_flush()from closing QEMU file from migration code.The patch also forces
bdrv_finalize_vmstate()operation inside synchronousblk_save_vmstate()operation. This helper is used from qemu-io only.Signed-off-by Denis V. Lunev <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>CC Kevin Wolf <[email protected]>CC Max Reitz <[email protected]>CC Stefan Hajnoczi <[email protected]>CC Fam Zheng <[email protected]>CC Juan Quintela <[email protected]>CC "Dr. David Alan Gilbert" <[email protected]>CC Denis Plotnikov <[email protected]>--- block/block-backend.c | 6 +++++- block/io.c | 15 +++++++++++++++ include/block/block.h | 5 +++++ migration/savevm.c | 4 ++++ 4 files changed, 29 insertions(+), 1 deletion(-)migration/savevm.c +0
-0@@ -150,6 +150,10 @@ static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,Quoted 7 lines
migration/savevm.c static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos, · line 155
Quoted 5 lines
migration/savevm.c +3
-0static int bdrv_fclose (void *opaque,Error **errp) { +int err =bdrv_finalize_vmstate (opaque); +if (err <0 ) { +return err;This is returning an error without having populating 'errp' which means the caller will be missing error diagnosis
Quoted 3 lines
but this behaves exactly like the branch below,
bdrv_flush()could return error too and errp
is not filled in the same way.
Doh, it seems the only caller passes NULL for the errp too, so it is a redundant parameter. So nothing wrong with your patch after all.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 4/6] block, migration: add bdrv_finalize_vmstate helper
Quoted 47 lines · Daniel P. Berrangé
Quoted 46 lines · Denis V. Lunev
Quoted 44 lines · Daniel P. Berrangé
Quoted 35 lines · Denis V. Lunev
Right now
bdrv_fclose()is just callingbdrv_flush().The problem is that migration code is working inefficiently from block layer terms and are frequently called for very small pieces of unaligned data. Block layer is capable to work this way, but this is very slow.
This patch is a preparation for the introduction of the intermediate buffer at block driver state. It would be beneficial to separate conventional
bdrv_flush()from closing QEMU file from migration code.The patch also forces
bdrv_finalize_vmstate()operation inside synchronousblk_save_vmstate()operation. This helper is used from qemu-io only.Signed-off-by Denis V. Lunev <[email protected]>Reviewed-by Vladimir Sementsov-Ogievskiy <[email protected]>CC Kevin Wolf <[email protected]>CC Max Reitz <[email protected]>CC Stefan Hajnoczi <[email protected]>CC Fam Zheng <[email protected]>CC Juan Quintela <[email protected]>CC "Dr. David Alan Gilbert" <[email protected]>CC Denis Plotnikov <[email protected]>--- block/block-backend.c | 6 +++++- block/io.c | 15 +++++++++++++++ include/block/block.h | 5 +++++ migration/savevm.c | 4 ++++ 4 files changed, 29 insertions(+), 1 deletion(-)migration/savevm.c +0
-0migration/savevm.c static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos, · line 155
Quoted 6 lines
migration/savevm.c +3
-0@@ -150,6 +150,10 @@ static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,static int bdrv_fclose (void *opaque,Error **errp) { +int err =bdrv_finalize_vmstate (opaque); +if (err <0 ) { +return err;This is returning an error without having populating 'errp' which means the caller will be missing error diagnosis
Quoted 7 lines
Quoted 3 lines
but this behaves exactly like the branch below,
bdrv_flush()could return error too and errp
is not filled in the same way.Doh, it seems the only caller passes NULL for the errp too, so it is a redundant parameter. So nothing wrong with your patch after all.
Not setting an error on failure is plainly wrong.
If it works because all callers pass NULL, then the obvious fix is to drop the @errp parameter.
I agree it's not this patch's fault. It needs fixing anyway. If you have to respin for some other reason, including a fix would be nice.
PATCH v8 5/6 block/io: improve savevm performance
In reply to Denis V. LunevThis patch does 2 standard basic things:
- it creates intermediate buffer for all writes from QEMU migration code to block driver,
- this buffer is sent to disk asynchronously, allowing several writes to run in parallel.
Thus bdrv_vmstate_write() is becoming asynchronous. All pending operations completion are performed in newly invented bdrv_finalize_vmstate().
In general, migration code is fantastically inefficent (by observation), buffers are not aligned and sent with arbitrary pieces, a lot of time less than 100 bytes at a chunk, which results in read-modify-write operations if target file descriptor is opened with O_DIRECT. It should also be noted that all operations are performed into unallocated image blocks, which also suffer due to partial writes to such new clusters even on cached file descriptors.
Snapshot creation time (2 GB Fedora-31 VM running over NVME storage):
original fixed cached: 1.79s 1.27s non-cached: 3.29s 0.81s
The difference over HDD would be more significant :)
--- block/io.c | 126 +++++++++++++++++++++++++++++++++++++- include/block/block_int.h | 8 +++ 2 files changed, 132 insertions(+), 2 deletions(-)
block/io.c +124 -2
@@ -26,6 +26,7 @@#include "trace.h" #include "sysemu/block-backend.h" #include "block/aio-wait.h" +#include "block/aio_task.h" #include "block/blockjob.h" #include "block/blockjob_int.h" #include "block/block_int.h"
@@ -33,6 +34,7 @@#include "qapi/error.h" #include "qemu/error-report.h" #include "qemu/main-loop.h" +#include "qemu/units.h" #include "sysemu/replay.h" /* Maximum bounce buffer for copy-on-read and write zeroes, in bytes */
@@ -2640,6 +2642,103 @@ typedef struct BdrvVmstateCo {bool is_read; } BdrvVmstateCo; +typedef struct BdrvVMStateTask { +AioTask task; + +BlockDriverState *bs; +int64_t offset; +void *buf; +size_t bytes; +}BdrvVMStateTask ; + +typedef struct BdrvSaveVMState { +AioTaskPool *pool; +BdrvVMStateTask *t; +}BdrvSaveVMState ; + + +static coroutine_fn int bdrv_co_vmstate_save_task_entry(AioTask *task) +{ +int err =0 ; +BdrvVMStateTask *t =container_of (task, BdrvVMStateTask, task); + +if (t->bytes !=0 ) { +QEMUIOVector qiov =QEMU_IOVEC_INIT_BUF (qiov, t->buf, t->bytes); + +bdrv_inc_in_flight (t->bs); + err = t->bs->drv->bdrv_save_vmstate (t->bs, &qiov, t->offset); +bdrv_dec_in_flight (t->bs); + } + +qemu_vfree (t->buf); +return err; +} + +static BdrvVMStateTask *bdrv_vmstate_task_create (BlockDriverState *bs, +int64_t pos,size_t size) +{ +BdrvVMStateTask *t =g_new (BdrvVMStateTask,1 ); + + *t = (BdrvVMStateTask ) { + .task.func = bdrv_co_vmstate_save_task_entry, + .buf =qemu_blockalign (bs, size), + .offset = pos, + .bs = bs, + }; + +return t; +} + +static int bdrv_co_do_save_vmstate (BlockDriverState *bs,QEMUIOVector *qiov, +int64_t pos) +{ +BdrvSaveVMState *state = bs->savevm_state; +BdrvVMStateTask *t; +size_t buf_size =MAX (bdrv_get_cluster_size (bs),1 * MiB); +size_t to_copy, off; + +if (state ==NULL ) { + state =g_new (BdrvSaveVMState,1 ); + *state = (BdrvSaveVMState ) { + .pool =aio_task_pool_new (BDRV_VMSTATE_WORKERS_MAX ), + .t =bdrv_vmstate_task_create (bs, pos, buf_size), + }; + + bs->savevm_state = state; + } + +if (aio_task_pool_status (state->pool) <0 ) { +/* +* The operation as a whole is unsuccessful. Prohibit all futher +* operations. If we clean here, new useless ops will come again. +* Thus we rely on caller for cleanup here. +*/ +return aio_task_pool_status (state->pool); + } + + t = state->t; +if (t->offset + t->bytes != pos) { +/* Normally this branch is not reachable from migration */ +return bs->drv->bdrv_save_vmstate (bs, qiov, pos); + } + + off =0 ; +while (1 ) { + to_copy =MIN (qiov->size - off, buf_size - t->bytes); +qemu_iovec_to_buf (qiov, off, t->buf + t->bytes, to_copy); + t->bytes += to_copy; +if (t->bytes < buf_size) { +return 0 ; + } + +aio_task_pool_start_task (state->pool, &t->task); + + pos += to_copy; + off += to_copy; + state->t = t =bdrv_vmstate_task_create (bs, pos, buf_size); + } +} + static int coroutine_fnbdrv_co_rw_vmstate (BlockDriverState *bs,QEMUIOVector *qiov,int64_t pos,bool is_read)
@@ -2655,7 +2754,7 @@ bdrv_co_rw_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos,if (is_read) { ret = drv->bdrv_load_vmstate (bs, qiov, pos); }else { - ret = drv->bdrv_save_vmstate (bs, qiov, pos); + ret =bdrv_co_do_save_vmstate (bs, qiov, pos); } }else if (bs->file) { ret =bdrv_co_rw_vmstate (bs->file->bs, qiov, pos, is_read);
@@ -2726,7 +2825,30 @@ int bdrv_readv_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, int64_t pos) static int coroutine_fn bdrv_co_finalize_vmstate(BlockDriverState *bs) { -return 0 ; +int err; +BdrvSaveVMState *state = bs->savevm_state; + +if (bs->drv->bdrv_save_vmstate ==NULL && bs->file !=NULL ) { +return bdrv_co_finalize_vmstate (bs->file->bs); + } +if (state ==NULL ) { +return 0 ; + } + +if (aio_task_pool_status (state->pool) >=0 ) { +/* We are on success path, commit last chunk if possible */ +aio_task_pool_start_task (state->pool, &state->t->task); + } + +aio_task_pool_wait_all (state->pool); + err =aio_task_pool_status (state->pool); + +aio_task_pool_free (state->pool); +g_free (state); + + bs->savevm_state =NULL ; + +return err; } static int coroutine_fn bdrv_finalize_vmstate_co_entry(void *opaque)
include/block/block_int.h +8 -0
@@ -61,6 +61,8 @@#define BLOCK_PROBE_BUF_SIZE 512 +#define BDRV_VMSTATE_WORKERS_MAX 8 +enum BdrvTrackedRequestType {BDRV_TRACKED_READ ,BDRV_TRACKED_WRITE ,
@@ -808,6 +810,9 @@ struct BdrvChild {QLIST_ENTRY (BdrvChild) next_parent; }; + +typedef struct BdrvSaveVMState BdrvSaveVMState ; +/* * Note: the function bdrv_append() copies and swaps contents of * BlockDriverStates, so if you add new fields to this struct, please
@@ -971,6 +976,9 @@ struct BlockDriverState {/* BdrvChild links to this node may never be frozen */ bool never_freeze; + +/* Intermediate buffer for VM state saving from snapshot creation code */ +BdrvSaveVMState *savevm_state; };struct BlockBackendRootState {
Signature · 1 line
2.17.1
PATCH v8 6/6 block/io: improve loadvm performance
In reply to Denis V. LunevThis patch creates intermediate buffer for reading from block driver state and performs read-ahead to this buffer. Snapshot code performs reads sequentially and thus we know what offsets will be required and when they will become not needed.
Results are fantastic. Switch to snapshot times of 2GB Fedora 31 VM over NVME storage are the following:
original fixed cached: 1.84s 1.16s non-cached: 12.74s 1.27s
The difference over HDD would be more significant :)
--- block/block-backend.c | 12 +- block/io.c | 239 +++++++++++++++++++++++++++++++++++++- include/block/block_int.h | 3 + 3 files changed, 250 insertions(+), 4 deletions(-)
block/block-backend.c +11 -1
@@ -2201,11 +2201,21 @@ int blk_save_vmstate(BlockBackend *blk, const uint8_t *buf,int blk_load_vmstate (BlockBackend *blk,uint8_t *buf,int64_t pos,int size) { +int ret, ret2; +if (!blk_is_available (blk)) {return -ENOMEDIUM ; } -return bdrv_load_vmstate (blk_bs (blk), buf, pos, size); + ret =bdrv_load_vmstate (blk_bs (blk), buf, pos, size); + ret2 =bdrv_finalize_vmstate (blk_bs (blk)); +if (ret <0 ) { +return ret; + } +if (ret2 <0 ) { +return ret2; + } +return ret; }int blk_probe_blocksizes (BlockBackend *blk,BlockSizes *bsz)
block/io.c +236 -3
268 lines not on this page · raw message
include/block/block_int.h +3 -0
@@ -812,6 +812,7 @@ struct BdrvChild {typedef struct BdrvSaveVMState BdrvSaveVMState ; +typedef struct BdrvLoadVMState BdrvLoadVMState ;/* * Note: the function bdrv_append() copies and swaps contents of
@@ -979,6 +980,8 @@ struct BlockDriverState {/* Intermediate buffer for VM state saving from snapshot creation code */ BdrvSaveVMState *savevm_state; +/* Intermediate buffer for VM state loading */ +BdrvLoadVMState *loadvm_state; };struct BlockBackendRootState {
Signature · 1 line
2.17.1
Re: [PATCH 6/6] block/io: improve loadvm performance
Quoted 13 lines · Denis V. Lunev
This patch creates intermediate buffer for reading from block driver state and performs read-ahead to this buffer. Snapshot code performs reads sequentially and thus we know what offsets will be required and when they will become not needed.
Results are fantastic. Switch to snapshot times of 2GB Fedora 31 VM over NVME storage are the following:
original fixedcached: 1.84s 1.16s non-cached: 12.74s 1.27sThe difference over HDD would be more significant:)
Quoted 8 lines
Signed-off-by Denis V. Lunev<[email protected]>CC Vladimir Sementsov-Ogievskiy<[email protected]>CC Kevin Wolf<[email protected]>CC Max Reitz<[email protected]>CC Stefan Hajnoczi<[email protected]>CC Fam Zheng<[email protected]>CC Juan Quintela<[email protected]>CC Denis Plotnikov<[email protected]>
Signature · 2 lines
Best regards, Vladimir
Quoted 61 lines · Denis V. Lunev
This series do standard basic things:
- it creates intermediate buffer for all writes from QEMU migration code to QCOW2 image,
- this buffer is sent to disk asynchronously, allowing several writes to run in parallel.In general, migration code is fantastically inefficent (by observation), buffers are not aligned and sent with arbitrary pieces, a lot of time less than 100 bytes at a chunk, which results in read-modify-write operations with non-cached operations. It should also be noted that all operations are performed into unallocated image blocks, which also suffer due to partial writes to such new clusters.
This patch series is an implementation of idea discussed in the RFC posted by Denis Plotnikov
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg01925.html
Results with this series over NVME are better than original codeoriginal rfc this cached: 1.79s 2.38s 1.27s non-cached: 3.29s 1.31s 0.81sChanges from v7:
- dropped lock from LoadVMState
- fixed assert in last patch
- dropped patch 1 as queuedChanges from v6:
-blk_load_vmstatekludges added (patchew problem fixed)Changes from v5:
- loadvm optimizations added with Vladimir comments includedChanges from v4:
- added patch 4 withblk_save_vmstate()cleanup
- added R-By
-bdrv_flush_vmstate->bdrv_finalize_vmstate
- fixed return code ofbdrv_co_do_save_vmstate
- fixed typos in comments (Eric, thanks!)
- fixed patchew warningsChanges from v3:
- rebased to master
- added patch 3 which removesaio_task_pool_wait_one()
- added R-By to patch 1
- patch 4 is rewritten viabdrv_run_co
- error path inblk_save_vmstate()is rewritten to callbdrv_flush_vmstateunconditionally
- added some comments
- fixes initialization inbdrv_co_vmstate_save_task_entryas suggestedChanges from v2:
- code moved from QCOW2 level to generic block level
- createdbdrv_flush_vmstatehelper to fix 022, 029 tests
- added recursive forbs->fileinbdrv_co_flush_vmstate(fix 267)
- fixedblk_save_vmstatehelper
- fixed coroutine wait as Vladimir suggested with waiting fixes from meChanges from v1:
- patchew warning fixed
- fixed validation that only 1 waiter is allowed in patch 1
Quoted 11 lines
Signed-off-by Denis V. Lunev <[email protected]>CC Kevin Wolf <[email protected]>CC Max Reitz <[email protected]>CC Stefan Hajnoczi <[email protected]>CC Fam Zheng <[email protected]>CC Juan Quintela <[email protected]>CC "Dr. David Alan Gilbert" <[email protected]>CC Vladimir Sementsov-Ogievskiy <[email protected]>CC Denis Plotnikov <[email protected]>
ping
Quoted 73 lines · Denis V. Lunev
Quoted 61 lines · Denis V. Lunev
This series do standard basic things:
- it creates intermediate buffer for all writes from QEMU migration code to QCOW2 image,
- this buffer is sent to disk asynchronously, allowing several writes to run in parallel.In general, migration code is fantastically inefficent (by observation), buffers are not aligned and sent with arbitrary pieces, a lot of time less than 100 bytes at a chunk, which results in read-modify-write operations with non-cached operations. It should also be noted that all operations are performed into unallocated image blocks, which also suffer due to partial writes to such new clusters.
This patch series is an implementation of idea discussed in the RFC posted by Denis Plotnikov
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg01925.html
Results with this series over NVME are better than original codeoriginal rfc this cached: 1.79s 2.38s 1.27s non-cached: 3.29s 1.31s 0.81sChanges from v7:
- dropped lock from LoadVMState
- fixed assert in last patch
- dropped patch 1 as queuedChanges from v6:
-blk_load_vmstatekludges added (patchew problem fixed)Changes from v5:
- loadvm optimizations added with Vladimir comments includedChanges from v4:
- added patch 4 withblk_save_vmstate()cleanup
- added R-By
-bdrv_flush_vmstate->bdrv_finalize_vmstate
- fixed return code ofbdrv_co_do_save_vmstate
- fixed typos in comments (Eric, thanks!)
- fixed patchew warningsChanges from v3:
- rebased to master
- added patch 3 which removesaio_task_pool_wait_one()
- added R-By to patch 1
- patch 4 is rewritten viabdrv_run_co
- error path inblk_save_vmstate()is rewritten to callbdrv_flush_vmstateunconditionally
- added some comments
- fixes initialization inbdrv_co_vmstate_save_task_entryas suggestedChanges from v2:
- code moved from QCOW2 level to generic block level
- createdbdrv_flush_vmstatehelper to fix 022, 029 tests
- added recursive forbs->fileinbdrv_co_flush_vmstate(fix 267)
- fixedblk_save_vmstatehelper
- fixed coroutine wait as Vladimir suggested with waiting fixes from meChanges from v1:
- patchew warning fixed
- fixed validation that only 1 waiter is allowed in patch 1Quoted 11 lines
Signed-off-by Denis V. Lunev <[email protected]>CC Kevin Wolf <[email protected]>CC Max Reitz <[email protected]>CC Stefan Hajnoczi <[email protected]>CC Fam Zheng <[email protected]>CC Juan Quintela <[email protected]>CC "Dr. David Alan Gilbert" <[email protected]>CC Vladimir Sementsov-Ogievskiy <[email protected]>CC Denis Plotnikov <[email protected]>
Quoted 1 line
ping
ping V2
Quoted 13 lines · Denis V. Lunev
This series do standard basic things:
- it creates intermediate buffer for all writes from QEMU migration code to QCOW2 image,
- this buffer is sent to disk asynchronously, allowing several writes to run in parallel.In general, migration code is fantastically inefficent (by observation), buffers are not aligned and sent with arbitrary pieces, a lot of time less than 100 bytes at a chunk, which results in read-modify-write operations with non-cached operations. It should also be noted that all operations are performed into unallocated image blocks, which also suffer due to partial writes to such new clusters.
Quoted 7 lines
This patch series is an implementation of idea discussed in the RFC posted by Denis Plotnikov
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg01925.html
Results with this series over NVME are better than original codeoriginal rfc this cached: 1.79s 2.38s 1.27s non-cached: 3.29s 1.31s 0.81s
Kevin and Max: Is this going through one of your trees?
Stefan