qcow2: async handling of fragmented io
v3 cover qcow2: async handling of fragmented io
Hi all!
Here is an asynchronous scheme for handling fragmented qcow2 reads and writes. Both qcow2 read and write functions loops through sequential portions of data. The series aim it to parallelize these loops iterations.
It improves performance for fragmented qcow2 images, I've tested it as described below.
v3 (by Max's comments) [perf results not updated]:
01: - use coroutine_fn where appropriate !!!!!!!!!!!!!!!!!!!!!!!
- add aio_task_pool_free
- add some comments
- move header to include/block
- s/wait_done/waiting
02: - Rewrite note about decryption in guest buffers [thx to Eric]
- separate g_assert_not_reached for QCOW2_CLUSTER_ZERO_*
- drop return after g_assert_not_reached
03: - drop bytes_done and correctly use qiov_offset
- fix comment
04: - move QCOW2_MAX_WORKERS to block/qcow2.h
- initialize ret in qcow2_co_preadv_part
Based-on: https://github.com/stefanha/qemu/commits/block
v2: changed a lot, as
1. a lot of preparations around locks, hd_qiovs, threads for encryption are done
2. I decided to create separate file with async request handling API, to reuse it for backup, stream and copy-on-read to improve their performance too. Mirror and qemu-img convert has their own async request handling, may be we'll be able finally merge all these similar code into one feature.
Note that not all API calls used in qcow2, some will be needed on following steps for parallelizing other io loops.
About testing:
I have four 4G qcow2 images (with default 64k block size) on my ssd disk:
t-seq.qcow2 - sequentially written qcow2 image
t-reverse.qcow2 - filled by writing 64k portions from end to the start t-rand.qcow2 - filled by writing 64k portions (aligned) in random order t-part-rand.qcow2 - filled by shuffling order of 64k writes in 1m clusters (see source code of image generation in the end for details)
and I've done several runs like the following (sequential io by 1mb chunks):
out=/tmp/block; echo > $out; cat /tmp/files | while read file; do for wr in {"","-w"}; do echo "$file" $wr; ./qemu-img bench -c 4096 -d 1 -f qcow2 -n -s 1m -t none $wr "$file" | grep 'Run completed in' | awk '{print $4}' >> $out; done; done
short info about parameters:
-w - do writes (otherwise do reads)
-c - count of blocks
-s - block size
-t none - disable cache
-n - native aio
-d 1 - don't use parallel requests provided by qemu-img bench itself
results:
+---------------------------+---------+---------+
| file | master | async |
+---------------------------+---------+---------+
| /ssd/t-part-rand.qcow2 | 14.671 | 9.193 |
+---------------------------+---------+---------+
| /ssd/t-part-rand.qcow2 -w | 11.434 | 8.621 |
+---------------------------+---------+---------+
| /ssd/t-rand.qcow2 | 20.421 | 10.05 |
+---------------------------+---------+---------+
| /ssd/t-rand.qcow2 -w | 11.097 | 8.915 |
+---------------------------+---------+---------+
| /ssd/t-reverse.qcow2 | 17.515 | 9.407 |
+---------------------------+---------+---------+
| /ssd/t-reverse.qcow2 -w | 11.255 | 8.649 |
+---------------------------+---------+---------+
| /ssd/t-seq.qcow2 | 9.081 | 9.072 |
+---------------------------+---------+---------+
| /ssd/t-seq.qcow2 -w | 8.761 | 8.747 |
+---------------------------+---------+---------+
| /tmp/t-part-rand.qcow2 | 41.179 | 41.37 |
+---------------------------+---------+---------+
| /tmp/t-part-rand.qcow2 -w | 54.097 | 55.323 |
+---------------------------+---------+---------+
| /tmp/t-rand.qcow2 | 711.899 | 514.339 |
+---------------------------+---------+---------+
| /tmp/t-rand.qcow2 -w | 546.259 | 642.114 |
+---------------------------+---------+---------+
| /tmp/t-reverse.qcow2 | 86.065 | 96.522 |
+---------------------------+---------+---------+
| /tmp/t-reverse.qcow2 -w | 46.557 | 48.499 |
+---------------------------+---------+---------+
| /tmp/t-seq.qcow2 | 33.804 | 33.862 |
+---------------------------+---------+---------+
| /tmp/t-seq.qcow2 -w | 34.299 | 34.233 |
+---------------------------+---------+---------+Performance gain is obvious, especially for read and especially for ssd. For hdd there is a degradation for reverse case, but this is the most impossible case and seems not critical.
How images are generated:
==== gen-writes ======
#!/usr/bin/env python
import random
import sys
size = 4 * 1024 * 1024 * 1024
block = 64 * 1024
block2 = 1024 * 1024
arg = sys.argv[1]
if arg in ('rand', 'reverse', 'seq'):
writes = list(range(0, size, block))
if arg == 'rand':
random.shuffle(writes)
elif arg == 'reverse':
writes.reverse()
elif arg == 'part-rand':
writes = []
for off in range(0, size, block2):
wr = list(range(off, off + block2, block))
random.shuffle(wr)
writes.extend(wr)
elif arg != 'seq':
sys.exit(1)
for w in writes:
print 'write -P 0xff {} {}'.format(w, block)
print 'q'
==========================
===== gen-test-images.sh =====
#!/bin/bash
IMG_PATH=/ssd
for name in seq reverse rand part-rand; do
IMG=$IMG_PATH/t-$name.qcow2
echo createing $IMG ...
rm -f $IMG
qemu-img create -f qcow2 $IMG 4G
gen-writes $name | qemu-io $IMG
done
==============================Vladimir Sementsov-Ogievskiy (4):
block: introduce aio task pool
block/qcow2: refactor qcow2_co_preadv_part
block/qcow2: refactor qcow2_co_pwritev_part
block/qcow2: introduce parallel subrequest handling in read and write
qapi/block-core.json | 2 +- block/qcow2.h | 3 + include/block/aio_task.h | 54 +++++ block/aio_task.c | 124 +++++++++++ block/qcow2.c | 461 +++++++++++++++++++++++++-------------- block/Makefile.objs | 2 + block/trace-events | 1 + 7 files changed, 486 insertions(+), 161 deletions(-) create mode 100644 include/block/aio_task.h create mode 100644 block/aio_task.c
Signature · 1 line
2.18.0
PATCH v3 1/4 block: introduce aio task pool
Common interface for aio task loops. To be used for improving performance of synchronous io loops in qcow2, block-stream, copy-on-read, and may be other places.
--- include/block/aio_task.h | 54 +++++++++++++++++ block/aio_task.c | 124 +++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 2 + 3 files changed, 180 insertions(+) create mode 100644 include/block/aio_task.h create mode 100644 block/aio_task.c
include/block/aio_task.h new file +54 -0
@@ -0,0 +1,54 @@ +/* +* Aio tasks loops +* +* Copyright (c) 2019 Virtuozzo International GmbH. +* +* Permission is hereby granted, free of charge, to any person obtaining a copy +* of this software and associated documentation files (the "Software"), to deal +* in the Software without restriction, including without limitation the rights +* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +* copies of the Software, and to permit persons to whom the Software is +* furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +* THE SOFTWARE. +*/ + +#ifndef BLOCK_AIO_TASK_H +#define BLOCK_AIO_TASK_H + +#include "qemu/coroutine.h" + +typedef struct AioTaskPool AioTaskPool ; +typedef struct AioTask AioTask ; +typedef int coroutine_fn (*AioTaskFunc)(AioTask *task); +struct AioTask { +AioTaskPool *pool; +AioTaskFunc func; +int ret; +}; + +AioTaskPool *aio_task_pool_new (int max_busy_tasks); +void aio_task_pool_free (AioTaskPool *); + +/* error code of failed task or 0 if all is OK */ +int aio_task_pool_status (AioTaskPool *pool); + +bool aio_task_pool_empty (AioTaskPool *pool); + +/* User provides filled @task, however task->pool will be set automatically */ +void coroutine_fn aio_task_pool_start_task(AioTaskPool *pool, AioTask *task); + +void coroutine_fn aio_task_pool_wait_slot(AioTaskPool *pool); +void coroutine_fn aio_task_pool_wait_one(AioTaskPool *pool); +void coroutine_fn aio_task_pool_wait_all(AioTaskPool *pool); + +#endif /* BLOCK_AIO_TASK_H */
block/aio_task.c new file +124 -0
124 lines not on this page · raw message
block/Makefile.objs +2 -0
@@ -40,6 +40,8 @@ block-obj-y += throttle.o copy-on-read.o block-obj-y += crypto.o +block-obj-y += aio_task.o + common-obj-y += stream.o nfs.o-libs := $(LIBNFS_LIBS)
Signature · 1 line
2.18.0
Re: [Qemu-devel] [PATCH v3 1/4] block: introduce aio task pool
Quoted 18 lines · Vladimir Sementsov-Ogievskiy
Common interface for aio task loops. To be used for improving performance of synchronous io loops in qcow2, block-stream, copy-on-read, and may be other places.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- include/block/aio_task.h | 54 +++++++++++++++++ block/aio_task.c | 124 +++++++++++++++++++++++++++++++++++++++ block/Makefile.objs | 2 + 3 files changed, 180 insertions(+) create mode 100644 include/block/aio_task.h create mode 100644 block/aio_task.cinclude/block/aio_task.h new file +0
-0
[...]
Quoted 1 line
+AioTaskPool *aio_task_pool_new(int
max_busy_tasks);
Because aio_task_pool_wait_one() asserts that it runs in the same coroutine as aio_task_pool_new(), this should be a coroutine_fn as well. O:-)
But I don’t want to be responsible for breaking your “1” key (assuming you have the exclamation mark there):
PATCH v3 2/4 block/qcow2: refactor qcow2_co_preadv_part
In reply to Vladimir Sementsov-OgievskiyFurther patch will run partial requests of iterations of qcow2_co_preadv in parallel for performance reasons. To prepare for this, separate part which may be parallelized into separate function (qcow2_co_preadv_task).
While being here, also separate encrypted clusters reading to own function, like it is done for compressed reading.
--- qapi/block-core.json | 2 +- block/qcow2.c | 205 +++++++++++++++++++++++-------------------- 2 files changed, 111 insertions(+), 96 deletions(-)
qapi/block-core.json +1 -1
@@ -3266,7 +3266,7 @@ 'pwritev_rmw_tail', 'pwritev_rmw_after_tail', 'pwritev', 'pwritev_zero', 'pwritev_done', 'empty_image_prepare', 'l1_shrink_write_table', 'l1_shrink_free_l2_clusters', - 'cor_write', 'cluster_alloc_space', 'none'] } + 'cor_write', 'cluster_alloc_space', 'none', 'read_encrypted'] } ## # @BlkdebugIOType:
block/qcow2.c +110 -95
238 lines not on this page · raw message
Signature · 1 line
2.18.0
Re: [Qemu-devel] [PATCH v3 2/4] block/qcow2: refactor qcow2_co_preadv_part
Quoted 5 lines · Vladimir Sementsov-Ogievskiy
Further patch will run partial requests of iterations of
qcow2_co_preadvin parallel for performance reasons. To prepare for this, separate part which may be parallelized into separate function (qcow2_co_preadv_task).
Quoted 8 lines
While being here, also separate encrypted clusters reading to own function, like it is done for compressed reading.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- qapi/block-core.json | 2 +- block/qcow2.c | 205 +++++++++++++++++++++++-------------------- 2 files changed, 111 insertions(+), 96 deletions(-)
PATCH v3 3/4 block/qcow2: refactor qcow2_co_pwritev_part
In reply to Vladimir Sementsov-OgievskiySimilarly to previous commit, prepare for parallelizing write-loop iterations.
--- block/qcow2.c | 153 +++++++++++++++++++++++++++++--------------------- 1 file changed, 89 insertions(+), 64 deletions(-)
block/qcow2.c +89 -64
@@ -2234,6 +2234,87 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta)return 0 ; } +/* +* qcow2_co_pwritev_task +* Called with s->lock unlocked +* l2meta - if not NULL, qcow2_co_do_pwritev() will consume it. Caller must not +* use it somehow after qcow2_co_pwritev_task() call +*/ +static coroutine_fn int qcow2_co_pwritev_task(BlockDriverState *bs, +uint64_t file_cluster_offset, +uint64_t offset,uint64_t bytes, +QEMUIOVector *qiov, +uint64_t qiov_offset, +QCowL2Meta *l2meta) +{ +int ret; +BDRVQcow2State *s = bs->opaque; +void *crypt_buf =NULL ; +int offset_in_cluster =offset_into_cluster (s, offset); +QEMUIOVector encrypted_qiov; + +if (bs->encrypted) { +assert (s->crypto); +assert (bytes <=QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); + crypt_buf =qemu_try_blockalign (bs->file->bs, bytes); +if (crypt_buf ==NULL ) { + ret = -ENOMEM ; + goto out_unlocked; + } +qemu_iovec_to_buf (qiov, qiov_offset, crypt_buf, bytes); + +if (qcow2_co_encrypt (bs, file_cluster_offset, offset, + crypt_buf, bytes) <0 ) { + ret = -EIO ; + goto out_unlocked; + } + +qemu_iovec_init_buf (&encrypted_qiov, crypt_buf, bytes); + qiov = &encrypted_qiov; + qiov_offset =0 ; + } + +/* Try to efficiently initialize the physical space with zeroes */ + ret =handle_alloc_space (bs, l2meta); +if (ret <0 ) { + goto out_unlocked; + } + +/* +* If we need to do COW, check if it's possible to merge the +* writing of the guest data together with that of the COW regions. +* If it's not possible (or not necessary) then write the +* guest data now. +*/ +if (!merge_cow (offset, bytes, qiov, qiov_offset, l2meta)) { +BLKDBG_EVENT (bs->file,BLKDBG_WRITE_AIO ); +trace_qcow2_writev_data (qemu_coroutine_self (), + file_cluster_offset + offset_in_cluster); + ret =bdrv_co_pwritev_part (s->data_file, + file_cluster_offset + offset_in_cluster, + bytes, qiov, qiov_offset,0 ); +if (ret <0 ) { + goto out_unlocked; + } + } + +qemu_co_mutex_lock (&s->lock); + + ret =qcow2_handle_l2meta (bs, &l2meta, true); + goto out_locked; + +out_unlocked: +qemu_co_mutex_lock (&s->lock); + +out_locked: +qcow2_handle_l2meta (bs, &l2meta, false); +qemu_co_mutex_unlock (&s->lock); + +qemu_vfree (crypt_buf); + +return ret; +} + static coroutine_fn int qcow2_co_pwritev_part(BlockDriverState *bs,uint64_t offset,uint64_t bytes,QEMUIOVector *qiov,size_t qiov_offset,int flags)
@@ -2243,15 +2324,10 @@ static coroutine_fn int qcow2_co_pwritev_part(int ret;unsigned int cur_bytes;/* number of sectors in current iteration */ uint64_t cluster_offset; -QEMUIOVector encrypted_qiov; -uint64_t bytes_done =0 ; -uint8_t *cluster_data =NULL ;QCowL2Meta *l2meta =NULL ;trace_qcow2_writev_start_req (qemu_coroutine_self (), offset, bytes); -qemu_co_mutex_lock (&s->lock); -while (bytes !=0 ) { l2meta =NULL ;
@@ -2265,6 +2341,8 @@ static coroutine_fn int qcow2_co_pwritev_part( - offset_in_cluster); } +qemu_co_mutex_lock (&s->lock); + ret =qcow2_alloc_cluster_offset (bs, offset, &cur_bytes, &cluster_offset, &l2meta);if (ret <0 ) {
@@ -2282,73 +2360,20 @@ static coroutine_fn int qcow2_co_pwritev_part(qemu_co_mutex_unlock (&s->lock); -if (bs->encrypted) { -assert (s->crypto); -if (!cluster_data) { - cluster_data =qemu_try_blockalign (bs->file->bs, -QCOW_MAX_CRYPT_CLUSTERS - * s->cluster_size); -if (cluster_data ==NULL ) { - ret = -ENOMEM ; - goto out_unlocked; - } - } - -assert (cur_bytes <=QCOW_MAX_CRYPT_CLUSTERS * s->cluster_size); -qemu_iovec_to_buf (qiov, qiov_offset + bytes_done, - cluster_data, cur_bytes); - -if (qcow2_co_encrypt (bs, cluster_offset, offset, - cluster_data, cur_bytes) <0 ) { - ret = -EIO ; - goto out_unlocked; - } - -qemu_iovec_init_buf (&encrypted_qiov, cluster_data, cur_bytes); - } - -/* Try to efficiently initialize the physical space with zeroes */ - ret =handle_alloc_space (bs, l2meta); + ret =qcow2_co_pwritev_task (bs, cluster_offset, offset, cur_bytes, + qiov, qiov_offset, l2meta); + l2meta =NULL ;/* l2meta is consumed by qcow2_co_pwritev_task() */ if (ret <0 ) { - goto out_unlocked; - } - -/* If we need to do COW, check if it's possible to merge the -* writing of the guest data together with that of the COW regions. -* If it's not possible (or not necessary) then write the -* guest data now. */ -if (!merge_cow (offset, cur_bytes, - bs->encrypted ? &encrypted_qiov : qiov, - bs->encrypted ?0 : qiov_offset + bytes_done, l2meta)) - { -BLKDBG_EVENT (bs->file,BLKDBG_WRITE_AIO ); -trace_qcow2_writev_data (qemu_coroutine_self (), - cluster_offset + offset_in_cluster); - ret =bdrv_co_pwritev_part ( - s->data_file, cluster_offset + offset_in_cluster, cur_bytes, - bs->encrypted ? &encrypted_qiov : qiov, - bs->encrypted ?0 : qiov_offset + bytes_done,0 ); -if (ret <0 ) { - goto out_unlocked; - } - } - -qemu_co_mutex_lock (&s->lock); - - ret =qcow2_handle_l2meta (bs, &l2meta, true); -if (ret) { - goto out_locked; + goto fail_nometa; } bytes -= cur_bytes; offset += cur_bytes; - bytes_done += cur_bytes; + qiov_offset += cur_bytes;trace_qcow2_writev_done_part (qemu_coroutine_self (), cur_bytes); } ret =0 ; - goto out_locked; -out_unlocked:qemu_co_mutex_lock (&s->lock); out_locked:
@@ -2356,7 +2381,7 @@ out_locked:qemu_co_mutex_unlock (&s->lock); -qemu_vfree (cluster_data); +fail_nometa:trace_qcow2_writev_done_req (qemu_coroutine_self (), ret);return ret;
Signature · 1 line
2.18.0
Re: [Qemu-devel] [PATCH v3 3/4] block/qcow2: refactor qcow2_co_pwritev_part
Quoted 13 lines · Vladimir Sementsov-Ogievskiy
Similarly to previous commit, prepare for parallelizing write-loop iterations.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/qcow2.c | 153 +++++++++++++++++++++++++++++--------------------- 1 file changed, 89 insertions(+), 64 deletions(-)block/qcow2.c +0
-0
block/qcow2.c static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta) · line 2240
Quoted 7 lines
block/qcow2.c +4
-0@@ -2234,6 +2234,87 @@ static int handle_alloc_space(BlockDriverState *bs, QCowL2Meta *l2meta)return 0 ; } +/* +* qcow2_co_pwritev_task +* Called with s->lock unlocked +* l2meta - if not NULL, qcow2_co_do_pwritev() will consume it. Caller must not
You missed this instance of “qcow2_co_do_pwritev()”.
With that fixed:
PATCH v3 4/4 block/qcow2: introduce parallel subrequest handling in read and write
In reply to Vladimir Sementsov-OgievskiyIt improves performance for fragmented qcow2 images.
--- block/qcow2.h | 3 ++ block/qcow2.c | 125 ++++++++++++++++++++++++++++++++++++++++----- block/trace-events | 1 + 3 files changed, 117 insertions(+), 12 deletions(-)
block/qcow2.h +3 -0
@@ -65,6 +65,9 @@#define QCOW2_MAX_BITMAPS 65535#define QCOW2_MAX_BITMAP_DIRECTORY_SIZE (1024 * QCOW2_MAX_BITMAPS) +/* Maximum of parallel sub-request per guest request */ +#define QCOW2_MAX_WORKERS 8 +/* indicate that the refcount of the referenced cluster is exactly one. */ #define QCOW_OFLAG_COPIED (1ULL << 63)/* indicate that the cluster is compressed (they never have the copied flag) */
block/qcow2.c +113 -12
198 lines not on this page · raw message
block/trace-events +1 -0
@@ -61,6 +61,7 @@ file_paio_submit(void *acb, void *opaque, int64_t offset, int count, int type) " file_copy_file_range(void *bs, int src, int64_t src_off, int dst, int64_t dst_off, int64_t bytes, int flags, int64_t ret) "bs %p src_fd %d offset %"PRIu64" dst_fd %d offset %"PRIu64" bytes %"PRIu64" flags %d ret %"PRId64 # qcow2.c +qcow2_add_task(void *co, void *bs, void *pool, const char *action, int cluster_type, uint64_t file_cluster_offset, uint64_t offset, uint64_t bytes, void *qiov, size_t qiov_offset) "co %p bs %p pool %p: %s: cluster_type %d file_cluster_offset %" PRIu64 " offset %" PRIu64 " bytes %" PRIu64 " qiov %p qiov_offset %zu" qcow2_writev_start_req(void *co, int64_t offset, int bytes) "co %p offset 0x%" PRIx64 " bytes %d" qcow2_writev_done_req(void *co, int ret) "co %p ret %d" qcow2_writev_start_part(void *co) "co %p"
Signature · 1 line
2.18.0
Re: [Qemu-devel] [PATCH v3 4/4] block/qcow2: introduce parallel subrequest handling in read and write
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
It improves performance for fragmented qcow2 images.
Signed-off-by Vladimir Sementsov-Ogievskiy <[email protected]>--- block/qcow2.h | 3 ++ block/qcow2.c | 125 ++++++++++++++++++++++++++++++++++++++++----- block/trace-events | 1 + 3 files changed, 117 insertions(+), 12 deletions(-)
Quoted 1 line · Vladimir Sementsov-Ogievskiy
01: - use
coroutine_fnwhere appropriate !!!!!!!!!!!!!!!!!!!!!!!
:-)
Quoted 5 lines · Max Reitz
Quoted 1 line · Vladimir Sementsov-Ogievskiy
01: - use
coroutine_fnwhere appropriate !!!!!!!!!!!!!!!!!!!!!!!:-)
Ahahaha, I'll explain:
When comparing v2 vs v3 and writing this difference script I noticed that I added coroutine_fn marks only to .h and not to .c. So I added some of "!!!" to not foreget to fix it.
Signature · 2 lines
Best regards, Vladimir
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Hi all!
Here is an asynchronous scheme for handling fragmented qcow2 reads and writes. Both qcow2 read and write functions loops through sequential portions of data. The series aim it to parallelize these loops iterations.
It improves performance for fragmented qcow2 images, I've tested it as described below.
Looks good to me, but I can’t take it yet because I need to wait for Stefan’s branch to be merged, of course.
Speaking of which, why didn’t you add any tests for the *_part() methods? I find it a bit unsettling that nothing would have caught the bug you had in v2 in patch 3.
Max
Quoted 10 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Hi all!
Here is an asynchronous scheme for handling fragmented qcow2 reads and writes. Both qcow2 read and write functions loops through sequential portions of data. The series aim it to parallelize these loops iterations.
It improves performance for fragmented qcow2 images, I've tested it as described below.
Quoted 7 lines
Looks good to me, but I can’t take it yet because I need to wait for Stefan’s branch to be merged, of course.
Speaking of which, why didn’t you add any tests for the
*_part()methods? I find it a bit unsettling that nothing would have caught the bug you had in v2 in patch 3.
Hmm, any test with write to fragmented area should have caught it.. Ok, I'll think of something.
Signature · 2 lines
Best regards, Vladimir
Quoted 14 lines · Vladimir Sementsov-Ogievskiy
Quoted 13 lines · Max Reitz
Quoted 8 lines · Vladimir Sementsov-Ogievskiy
Hi all!
Here is an asynchronous scheme for handling fragmented qcow2 reads and writes. Both qcow2 read and write functions loops through sequential portions of data. The series aim it to parallelize these loops iterations.
It improves performance for fragmented qcow2 images, I've tested it as described below.Looks good to me, but I can’t take it yet because I need to wait for Stefan’s branch to be merged, of course.
Quoted 9 lines
Quoted 4 lines
Speaking of which, why didn’t you add any tests for the
*_part()methods? I find it a bit unsettling that nothing would have caught the bug you had in v2 in patch 3.Hmm, any test with write to fragmented area should have caught it.. Ok, I'll think of something.
And now I see that it's not trivial to make such a test:
1. qcow2 write is broken when we give nonzero qiov_offset to it, but only qcow2_write calls bdrv_co_pwritev_part, so we need to have a test where qcow2 is a file child for qcow2
2. Then, the bug leads to the beginning of the qiov will be written to all parts. But our testing tool qemu-io has only "write -P" command with buffer filled with the same one byte, so we can't catch it
Signature · 2 lines
Best regards, Vladimir