Protect the block layer with a rwlock: part 2
cover Protect the block layer with a rwlock: part 2
Please read "Protect the block layer with a rwlock: part 1" for an additional introduction and aim of this series.
This second part aims to add the graph rdlock to the BlockDriver functions that already run in coroutine context and are classified as IO. Such functions will recursively traverse the BlockDriverState graph, therefore they need to be protected with the rdlock.
Based-on: <[email protected]>
Thank you,
Emanuele
Emanuele Giuseppe Esposito (6):
block: assert that bdrv_co_create is always called with graph rdlock
taken
block: assert that BlockDriver->bdrv_co_{amend/create} are called with
graph rdlock taken
block: assert that BlockDriver->bdrv_co_copy_range_{from/to} is always
called with graph rdlock taken
block/dirty-bitmap: assert that BlockDriver->bdrv_co_*_dirty_bitmap
are always called with graph rdlock taken
block/io: assert that BlockDriver->bdrv_co_*_snapshot_* are always
called with graph rdlock taken
block: assert that BlockDriver->bdrv_co_delete_file is always called
with graph rdlock taken
block.c | 2 ++ block/amend.c | 1 + block/block-backend.c | 2 ++ block/create.c | 1 + block/dirty-bitmap.c | 2 ++ block/io.c | 7 +++++++ include/block/block_int-common.h | 14 +++++++++++++- qemu-img.c | 4 +++- 8 files changed, 31 insertions(+), 2 deletions(-)
Signature · 1 line
2.31.1
PATCH 1/6 block: assert that bdrv_co_create is always called with graph rdlock taken
This function is either called by bdrv_create(), which always takes care of creating a new coroutine, or by bdrv_create_file(), which is only called by BlockDriver->bdrv_co_create_opts callbacks, invoked by bdrv_co_create().
Protecting bdrv_co_create() implies that BlockDriver->bdrv_co_create_opts is always called with graph rdlock taken.
--- block.c | 1 + include/block/block_int-common.h | 1 + 2 files changed, 2 insertions(+)
block.c +1 -0
@@ -537,6 +537,7 @@ int coroutine_fn bdrv_co_create(BlockDriver *drv, const char *filename,assert (qemu_in_coroutine ());assert (*errp ==NULL );assert (drv); +assert_bdrv_graph_readable ();if (!drv->bdrv_co_create_opts) { error_setg(errp, "Driver '%s' does not support image creation",
include/block/block_int-common.h +1 -0
@@ -253,6 +253,7 @@ struct BlockDriver { int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,Error **errp); +/* Called with graph rdlock taken */ int coroutine_fn (*bdrv_co_create_opts)(BlockDriver *drv, const char *filename, QemuOpts *opts,
Signature · 1 line
2.31.1
PATCH 2/6 block: assert that BlockDriver->bdrv_co_{amend/create} are called with graph rdlock taken
In reply to Emanuele Giuseppe EspositoBoth functions are only called by Job->run() callbacks, therefore they must take the lock in the *_run() implementation.
--- block/amend.c | 1 + block/create.c | 1 + include/block/block_int-common.h | 2 ++ 3 files changed, 4 insertions(+)
block/amend.c +1 -0
@@ -45,6 +45,7 @@ static int coroutine_fn blockdev_amend_run(Job *job, Error **errp) {BlockdevAmendJob *s =container_of (job, BlockdevAmendJob, common);int ret; +GRAPH_RDLOCK_GUARD ();job_progress_set_remaining (&s->common,1 ); ret = s->bs->drv->bdrv_co_amend (s->bs, s->opts, s->force, errp);
block/create.c +1 -0
@@ -43,6 +43,7 @@ static int coroutine_fn blockdev_create_run(Job *job, Error **errp)int ret;GLOBAL_STATE_CODE (); +GRAPH_RDLOCK_GUARD ();job_progress_set_remaining (&s->common,1 ); ret = s->drv->bdrv_co_create (s->opts, errp);
include/block/block_int-common.h +2 -0
@@ -251,6 +251,7 @@ struct BlockDriver { Error **errp);void (*bdrv_close)(BlockDriverState *bs); +/* Called with graph rdlock taken */ int coroutine_fn (*bdrv_co_create)(BlockdevCreateOptions *opts,Error **errp);/* Called with graph rdlock taken */
@@ -471,6 +472,7 @@ struct BlockDriver {int (*bdrv_probe)(const uint8_t *buf,int buf_size,const char *filename); +/* Called with graph rdlock taken. */ int coroutine_fn (*bdrv_co_amend)(BlockDriverState *bs, BlockdevAmendOptions *opts, bool force,
Signature · 1 line
2.31.1
PATCH 3/6 block: assert that BlockDriver->bdrv_co_copy_range_{from/to} is always called with graph rdlock taken
In reply to Emanuele Giuseppe EspositoThe only non-protected caller is convert_co_copy_range(), all other callers are BlockDriver callbacks that already take the rdlock.
--- block/block-backend.c | 2 ++ block/io.c | 5 +++++ include/block/block_int-common.h | 4 ++++ qemu-img.c | 4 +++- 4 files changed, 14 insertions(+), 1 deletion(-)
block/block-backend.c +2 -0
@@ -2631,6 +2631,8 @@ int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in,if (r) {return r; } + +GRAPH_RDLOCK_GUARD ();return bdrv_co_copy_range (blk_in->root, off_in, blk_out->root, off_out, bytes, read_flags, write_flags);
block/io.c +5 -0
@@ -3165,6 +3165,7 @@ static int coroutine_fn bdrv_co_copy_range_internal( {BdrvTrackedRequest req;int ret; +assert_bdrv_graph_readable ();/* TODO We can support BDRV_REQ_NO_FALLBACK here */ assert (!(read_flags &BDRV_REQ_NO_FALLBACK ));
@@ -3246,6 +3247,7 @@ int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, int64_t src_offset, BdrvRequestFlags write_flags) {IO_CODE (); +assert_bdrv_graph_readable ();trace_bdrv_co_copy_range_from (src, src_offset, dst, dst_offset, bytes, read_flags, write_flags); return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset,
@@ -3263,6 +3265,7 @@ int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, int64_t src_offset, BdrvRequestFlags write_flags) {IO_CODE (); +assert_bdrv_graph_readable ();trace_bdrv_co_copy_range_to (src, src_offset, dst, dst_offset, bytes, read_flags, write_flags); return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset,
@@ -3275,6 +3278,8 @@ int coroutine_fn bdrv_co_copy_range(BdrvChild *src, int64_t src_offset, BdrvRequestFlags write_flags) {IO_CODE (); +assert_bdrv_graph_readable (); +return bdrv_co_copy_range_from (src, src_offset, dst, dst_offset, bytes, read_flags, write_flags);
include/block/block_int-common.h +4 -0
@@ -574,6 +574,8 @@ struct BlockDriver {* * See the comment of bdrv_co_copy_range for the parameter and return value * semantics. +* +* Called with graph rdlock taken. */ int coroutine_fn (*bdrv_co_copy_range_from)(BlockDriverState *bs, BdrvChild *src,
@@ -592,6 +594,8 @@ struct BlockDriver {* * See the comment of bdrv_co_copy_range for the parameter and return value * semantics. +* +* Called with graph rdlock taken. */ int coroutine_fn (*bdrv_co_copy_range_to)(BlockDriverState *bs, BdrvChild *src,
qemu-img.c +3 -1
@@ -2027,7 +2027,9 @@ retry:if (s->ret == -EINPROGRESS ) {if (copy_range) { - ret =convert_co_copy_range (s, sector_num, n); + WITH_GRAPH_RDLOCK_GUARD() { + ret =convert_co_copy_range (s, sector_num, n); + }if (ret) { s->copy_range = false; goto retry;
Signature · 1 line
2.31.1
PATCH 4/6 block/dirty-bitmap: assert that BlockDriver->bdrv_co_*_dirty_bitmap are always called with graph rdlock taken
In reply to Emanuele Giuseppe EspositoThe only callers are the respective bdrv_*_dirty_bitmap() functions that take care of creating a new coroutine (that already takes the graph rdlock).
--- block/dirty-bitmap.c | 2 ++ include/block/block_int-common.h | 2 ++ 2 files changed, 4 insertions(+)
block/dirty-bitmap.c +2 -0
@@ -392,6 +392,7 @@ int coroutine_fn bdrv_co_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name, Error **errp) { +assert_bdrv_graph_readable ();if (bs->drv && bs->drv->bdrv_co_remove_persistent_dirty_bitmap) {return bs->drv->bdrv_co_remove_persistent_dirty_bitmap (bs, name, errp); }
@@ -413,6 +414,7 @@ bdrv_co_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name, uint32_t granularity, Error **errp) { BlockDriver *drv = bs->drv; +assert_bdrv_graph_readable ();if (!drv) { error_setg_errno(errp, ENOMEDIUM,
include/block/block_int-common.h +2 -0
@@ -789,9 +789,11 @@ struct BlockDriver {void (*bdrv_drain_end)(BlockDriverState *bs);bool (*bdrv_supports_persistent_dirty_bitmap)(BlockDriverState *bs); +/* Called with graph rdlock held. */ bool coroutine_fn (*bdrv_co_can_store_new_dirty_bitmap)(BlockDriverState *bs,const char *name,uint32_t granularity,Error **errp); +/* Called with graph rdlock held. */ int coroutine_fn (*bdrv_co_remove_persistent_dirty_bitmap)(BlockDriverState *bs,const char *name,Error **errp); };
Signature · 1 line
2.31.1
PATCH 5/6 block/io: assert that BlockDriver->bdrv_co_*_snapshot_* are always called with graph rdlock taken
In reply to Emanuele Giuseppe EspositoThe only callers are other callback functions that already run with the graph rdlock taken.
--- block/io.c | 2 ++ include/block/block_int-common.h | 3 +++ 2 files changed, 5 insertions(+)
block/io.c +2 -0
@@ -3449,6 +3449,7 @@ bdrv_co_preadv_snapshot(BdrvChild *child, int64_t offset, int64_t bytes,BlockDriver *drv = bs->drv;int ret;IO_CODE (); +assert_bdrv_graph_readable ();if (!drv) {return -ENOMEDIUM ;
@@ -3474,6 +3475,7 @@ bdrv_co_snapshot_block_status(BlockDriverState *bs,BlockDriver *drv = bs->drv;int ret;IO_CODE (); +assert_bdrv_graph_readable ();if (!drv) {return -ENOMEDIUM ;
include/block/block_int-common.h +3 -0
@@ -652,9 +652,12 @@ struct BlockDriver {* - be able to select a specific snapshot * - receive the snapshot's actual length (which may differ from bs's * length) +* +* Called with graph rdlock taken. */ int coroutine_fn (*bdrv_co_preadv_snapshot)(BlockDriverState *bs,int64_t offset,int64_t bytes,QEMUIOVector *qiov,size_t qiov_offset); +/* Called with graph rdlock taken. */ int coroutine_fn (*bdrv_co_snapshot_block_status)(BlockDriverState *bs,bool want_zero,int64_t offset,int64_t bytes,int64_t *pnum,int64_t *map,BlockDriverState **file);
Signature · 1 line
2.31.1
PATCH 6/6 block: assert that BlockDriver->bdrv_co_delete_file is always called with graph rdlock taken
In reply to Emanuele Giuseppe EspositoThe only callers are other callback functions that already run with the graph rdlock taken.
--- block.c | 1 + include/block/block_int-common.h | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
block.c +1 -0
@@ -747,6 +747,7 @@ int coroutine_fn bdrv_co_delete_file(BlockDriverState *bs, Error **errp)IO_CODE ();assert (bs !=NULL ); +assert_bdrv_graph_readable ();if (!bs->drv) {error_setg (errp,"Block node '%s' is not opened" , bs->filename);
include/block/block_int-common.h +1 -1
@@ -681,7 +681,7 @@ struct BlockDriver {*/ int coroutine_fn (*bdrv_co_flush)(BlockDriverState *bs); -/* Delete a created file. */ +/* Delete a created file. Called with graph rdlock taken. */ int coroutine_fn (*bdrv_co_delete_file)(BlockDriverState *bs,Error **errp);
Signature · 1 line
2.31.1
Ok, as I expected simple changes in a previous based-on serie provoke a cascade of changes that inevitably affect these patches too.
While I strongly suggest to have an initial look at these patches because it gives an idea on what am I trying to accomplish, I would not go looking at nitpicks and trivial errors that came up from the based-on series (ie "just as in the previous serie, fix this").
The order of the series is:
1. Still more coroutine and various fixes in block layer
2. Protect the block layer with a rwlock: part 1
3. Protect the block layer with a rwlock: part 2
4. Protect the block layer with a rwlock: part 3
Thank you,
Emanuele
Am 16/11/2022 um 14:53 schrieb Emanuele Giuseppe Esposito:
Quoted 37 lines
Please read "Protect the block layer with a rwlock: part 1" for an additional introduction and aim of this series.
This second part aims to add the graph rdlock to the BlockDriver functions that already run in coroutine context and are classified as IO. Such functions will recursively traverse the BlockDriverState graph, therefore they need to be protected with the rdlock.
Based-on: <[email protected]>
Thank you,
EmanueleEmanuele Giuseppe Esposito (6):
block: assert thatbdrv_co_createis always called with graph rdlock
taken
block: assert that BlockDriver->bdrv_co_{amend/create} are called with
graph rdlock taken
block: assert that BlockDriver->bdrv_co_copy_range_{from/to} is always
called with graph rdlock taken
block/dirty-bitmap: assert that BlockDriver->bdrv_co_*_dirty_bitmap
are always called with graph rdlock taken
block/io: assert that BlockDriver->bdrv_co_*_snapshot_* are always
called with graph rdlock taken
block: assert thatBlockDriver->bdrv_co_delete_fileis always called
with graph rdlock takenblock.c | 2 ++ block/amend.c | 1 + block/block-backend.c | 2 ++ block/create.c | 1 + block/dirty-bitmap.c | 2 ++ block/io.c | 7 +++++++ include/block/block_int-common.h | 14 +++++++++++++- qemu-img.c | 4 +++- 8 files changed, 31 insertions(+), 2 deletions(-)