refactor submit_bio_wait and bio await helpers v3
Subsystems (2)
Hi Jens,
this series factors common code between submit_bio_wait and bio_await_chain into a common helper, and then uses that in XFS as well instead of open coding such functionality. It also cleans up the submit or kill logic in the ioctl handlers to share more code.
There is another places in btrfs that could be refactored to use this, although it is non-trivial, and I plan to add more users of this helper to XFS in the future.
Changes since v2:
- spelling fixes
- rename the kick callback to submit
Changes since v1:
- preserve (and extend) setting REQ_SYNC
Diffstat:
block/bio.c | 81 ++++++++++++++++++++++++++++++++------------------- block/blk-lib.c | 16 +--------- block/blk.h | 2 - block/ioctl.c | 11 +----- fs/xfs/xfs_zone_gc.c | 38 +++++++++-------------- include/linux/bio.h | 2 + 6 files changed, 73 insertions(+), 77 deletions(-)
PATCH 1/5 xfs: fix number of GC bvecs
GC scratch allocations can wrap around and use the same buffer twice, and the current code fails to account for that. So far this worked due to rounding in the block layer, but changes to the bio allocator drop the over-provisioning and generic/256 or generic/361 will now usually fail when running against the current block tree.
Simplify the allocation to always pass the maximum value that is easier to verify, as a saving of up to one bvec per allocation isn't worth the effort to verify a complicated calculated value.
--- fs/xfs/xfs_zone_gc.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
fs/xfs/xfs_zone_gc.c +10 -9
@@ -670,7 +670,6 @@ xfs_zone_gc_start_chunk(struct xfs_inode *ip;struct bio *bio;xfs_daddr_t daddr; -unsigned int len;bool is_seq;if (xfs_is_shutdown (mp))
@@ -685,15 +684,16 @@ xfs_zone_gc_start_chunk(return false; } - len =XFS_FSB_TO_B (mp, irec.rm_blockcount); - bio =bio_alloc_bioset (bdev, -min (howmany (len,XFS_GC_BUF_SIZE ) +1 ,XFS_GC_NR_BUFS ), -REQ_OP_READ ,GFP_NOFS , &data->bio_set); - +/* +* Scratch allocation can wrap around to the same buffer again, +* provision an extra bvec for that case. +*/ + bio =bio_alloc_bioset (bdev,XFS_GC_NR_BUFS +1 ,REQ_OP_READ ,GFP_NOFS , + &data->bio_set); chunk = container_of(bio, struct xfs_gc_bio, bio); chunk->ip = ip; chunk->offset =XFS_FSB_TO_B (mp, irec.rm_offset); - chunk->len = len; + chunk->len =XFS_FSB_TO_B (mp, irec.rm_blockcount); chunk->old_startblock =xfs_rgbno_to_rtb (iter->victim_rtg, irec.rm_startblock); chunk->new_daddr = daddr;
@@ -707,8 +707,9 @@ xfs_zone_gc_start_chunk( bio->bi_iter.bi_sector =xfs_rtb_to_daddr (mp, chunk->old_startblock); bio->bi_end_io = xfs_zone_gc_end_io;xfs_zone_gc_add_data (chunk); - data->scratch_head = (data->scratch_head + len) % data->scratch_size; - data->scratch_available -= len; + data->scratch_head = + (data->scratch_head + chunk->len) % data->scratch_size; + data->scratch_available -= chunk->len;XFS_STATS_INC (mp, xs_gc_read_calls);
Signature · 1 line
2.47.3
PATCH 2/5 block: unify the synchronous bi_end_io callbacks
In reply to Christoph HellwigPut the bio in bio_await_chain after waiting for the completion, and share the now identical callbacks between submit_bio_wait and bio_await_chain.
--- block/bio.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-)
block/bio.c +3 -8
@@ -1462,7 +1462,7 @@ void bio_iov_iter_unbounce(struct bio *bio, bool is_error, bool mark_dirty)bio_iov_iter_unbounce_read (bio, is_error, mark_dirty); } -static void submit_bio_wait_endio (struct bio *bio) +static void bio_wait_end_io (struct bio *bio) {complete (bio->bi_private); }
@@ -1484,7 +1484,7 @@ int submit_bio_wait(struct bio *bio) bio->bi_bdev->bd_disk->lockdep_map); bio->bi_private = &done; - bio->bi_end_io = submit_bio_wait_endio; + bio->bi_end_io = bio_wait_end_io; bio->bi_opf |=REQ_SYNC ;submit_bio (bio);blk_wait_io (&done);
@@ -1523,12 +1523,6 @@ int bdev_rw_virt(struct block_device *bdev, sector_t sector, void *data, }EXPORT_SYMBOL_GPL (bdev_rw_virt); -static void bio_wait_end_io (struct bio *bio) -{ -complete (bio->bi_private); -bio_put (bio); -} -/* * bio_await_chain - ends @bio and waits for every chained bio to complete */
@@ -1541,6 +1535,7 @@ void bio_await_chain(struct bio *bio) bio->bi_end_io = bio_wait_end_io;bio_endio (bio);blk_wait_io (&done); +bio_put (bio); }void __bio_advance (struct bio *bio,unsigned bytes)
Signature · 1 line
2.47.3
Re: [PATCH 2/5] block: unify the synchronous bi_end_io callbacks
Quoted 7 lines · Christoph Hellwig
Put the bio in
bio_await_chainafter waiting for the completion, and share the now identical callbacks betweensubmit_bio_waitandbio_await_chain.Signed-off-by Christoph Hellwig<[email protected]>Reviewed-by Bart Van Assche<[email protected]>Reviewed-by Damien Le Moal<[email protected]>
Looks good.
-ck
PATCH 3/5 block: factor out a bio_await helper
In reply to Christoph HellwigAdd a new helper to wait for a bio and anything chained off it to complete synchronously after submitting it. This factors common code out of submit_bio_wait and bio_await_chain and will also be useful for file system code and thus is exported.
Note that this will now set REQ_SYNC also for the bio_await case for consistency. Nothing should look at the flag in the end_io handler, but if something does having the flag set makes more sense.
--- block/bio.c | 53 +++++++++++++++++++++++++++++++-------------- include/linux/bio.h | 2 ++ 2 files changed, 39 insertions(+), 16 deletions(-)
block/bio.c +37 -16
@@ -1468,17 +1468,20 @@ static void bio_wait_end_io(struct bio *bio) }/** -* submit_bio_wait - submit a bio, and wait until it completes -* @bio: The &struct bio which describes the I/O +* bio_await - call a function on a bio, and wait until it completes +* @bio: the bio which describes the I/O +* @submit: function called to submit the bio +* @priv: private data passed to @submit * -* Simple wrapper around submit_bio(). Returns 0 on success, or the error from -* bio_endio() on failure. +* Wait for the bio as well as any bio chained off it after executing the +* passed in callback @submit. The wait for the bio is set up before calling +* @submit to ensure that the completion is captured. If @submit is %NULL, +* submit_bio() is used instead to submit the bio. * -* WARNING: Unlike to how submit_bio() is usually used, this function does not -* result in bio reference to be consumed. The caller must drop the reference -* on his own. +* Note: this overrides the bi_private and bi_end_io fields in the bio. */ -int submit_bio_wait (struct bio *bio) +void bio_await (struct bio *bio,void *priv, +void (*submit)(struct bio *bio,void *priv)) {DECLARE_COMPLETION_ONSTACK_MAP (done, bio->bi_bdev->bd_disk->lockdep_map);
@@ -1486,13 +1489,37 @@ int submit_bio_wait(struct bio *bio) bio->bi_private = &done; bio->bi_end_io = bio_wait_end_io; bio->bi_opf |=REQ_SYNC ; -submit_bio (bio); +if (submit) +submit (bio, priv); +else +submit_bio (bio);blk_wait_io (&done); +} +EXPORT_SYMBOL_GPL (bio_await); +/** +* submit_bio_wait - submit a bio, and wait until it completes +* @bio: The &struct bio which describes the I/O +* +* Simple wrapper around submit_bio(). Returns 0 on success, or the error from +* bio_endio() on failure. +* +* WARNING: Unlike to how submit_bio() is usually used, this function does not +* result in bio reference to be consumed. The caller must drop the reference +* on his own. +*/ +int submit_bio_wait (struct bio *bio) +{ +bio_await (bio,NULL ,NULL );return blk_status_to_errno (bio->bi_status); }EXPORT_SYMBOL (submit_bio_wait); +static void bio_endio_cb (struct bio *bio,void *priv) +{ +bio_endio (bio); +} +/** * bdev_rw_virt - synchronously read into / write from kernel mapping * @bdev: block device to access
@@ -1528,13 +1555,7 @@ EXPORT_SYMBOL_GPL(bdev_rw_virt);*/ void bio_await_chain (struct bio *bio) { -DECLARE_COMPLETION_ONSTACK_MAP (done, - bio->bi_bdev->bd_disk->lockdep_map); - - bio->bi_private = &done; - bio->bi_end_io = bio_wait_end_io; -bio_endio (bio); -blk_wait_io (&done); +bio_await (bio,NULL , bio_endio_cb);bio_put (bio); }
include/linux/bio.h +2 -0
@@ -432,6 +432,8 @@ extern void bio_uninit(struct bio *);void bio_reset (struct bio *bio,struct block_device *bdev,blk_opf_t opf);void bio_reuse (struct bio *bio,blk_opf_t opf);void bio_chain (struct bio *,struct bio *); +void bio_await (struct bio *bio,void *priv, +void (*submit)(struct bio *bio,void *priv));int __must_check bio_add_page (struct bio *bio,struct page *page,unsigned len,unsigned off);
Signature · 1 line
2.47.3
Re: [PATCH 3/5] block: factor out a bio_await helper
Quoted 38 lines · Christoph Hellwig
/**
- *submit_bio_wait- submit a bio, and wait until it completes
- * @bio: The &struct bio which describes the I/O
+ *bio_await- call a function on a bio, and wait until it completes
+ * @bio: the bio which describes the I/O
+ * @submit: function called to submit the bio
+ * @priv: private data passed to @submit
*
- * Simple wrapper aroundsubmit_bio(). Returns 0 on success, or the error from
- *bio_endio()on failure.
+ * Wait for the bio as well as any bio chained off it after executing the
+ * passed in callback @submit. The wait for the bio is set up before calling
+ * @submit to ensure that the completion is captured. If @submit is %NULL,
+ *submit_bio()is used instead to submit the bio.
*
- * WARNING: Unlike to howsubmit_bio()is usually used, this function does not
- * result in bio reference to be consumed. The caller must drop the reference
- * on his own.
+ * Note: this overrides thebi_privateandbi_end_iofields in the bio. */block/bio.c +8
-2-int submit_bio_wait (struct bio *bio) +void bio_await (struct bio *bio,void *priv, +void (*submit)(struct bio *bio,void *priv)) {DECLARE_COMPLETION_ONSTACK_MAP (done, bio->bi_bdev->bd_disk->lockdep_map);@@ -1486,13 +1489,37 @@ int submit_bio_wait(struct bio *bio) bio->bi_private = &done; bio->bi_end_io = bio_wait_end_io; bio->bi_opf |=REQ_SYNC ; -submit_bio (bio); +if (submit) +submit (bio, priv); +else +submit_bio (bio);blk_wait_io (&done); +} +EXPORT_SYMBOL_GPL (bio_await);
[ ... ]
block/bio.c EXPORT_SYMBOL_GPL(bdev_rw_virt); · line 1560
Quoted 12 lines
void bio_await_chain(struct bio *bio) {block/bio.c +1
-7-DECLARE_COMPLETION_ONSTACK_MAP (done, - bio->bi_bdev->bd_disk->lockdep_map); - - bio->bi_private = &done; - bio->bi_end_io = bio_wait_end_io; -bio_endio (bio); -blk_wait_io (&done); +bio_await (bio,NULL , bio_endio_cb);bio_put (bio); }
The comment above bio_await() says that the callback function passed as third argument submits the bio. The above code passes bio_end_cb as the "submit" argument. bio_end_cb() does something else than submitting a
bio. How about renaming the "submit" argument of bio_await() into
something else, e.g. "cb" or "callback" to make its role more clear?
Thanks,
Bart.
Re: [PATCH 3/5] block: factor out a bio_await helper
Quoted 6 lines · Bart Van Assche
Quoted 1 line
}The comment above
bio_await()says that the callback function passed as third argument submits the bio. The above code passesbio_end_cbas the "submit" argument.bio_end_cb()does something else than submitting a bio. How about renaming the "submit" argument ofbio_await()into something else, e.g. "cb" or "callback" to make its role more clear?
It was called kick and Damien objected politely. A direct completion isn't really distinguishable from a submission anyway, submit_bio could complete the bio with an error before doing much.
PATCH 4/5 block: add a bio_submit_or_kill helper
In reply to Christoph HellwigFactor the common logic for the ioctl helpers to either submit a bio or end if the process is being killed. As this is now the only user of bio_await_chain, open code that.
--- block/bio.c | 23 ++++++++++++++--------- block/blk-lib.c | 16 ++-------------- block/blk.h | 2 +- block/ioctl.c | 11 ++--------- 4 files changed, 19 insertions(+), 33 deletions(-)
block/bio.c +14 -9
@@ -1520,6 +1520,20 @@ static void bio_endio_cb(struct bio *bio, void *priv)bio_endio (bio); } +/* +* Submit @bio synchronously, or call bio_endio on it if the current process +* is being killed. +*/ +int bio_submit_or_kill (struct bio *bio,unsigned int flags) +{ +if ((flags &BLKDEV_ZERO_KILLABLE ) &&fatal_signal_pending (current)) { +bio_await (bio,NULL , bio_endio_cb); +return -EINTR ; + } + +return submit_bio_wait (bio); +} +/** * bdev_rw_virt - synchronously read into / write from kernel mapping * @bdev: block device to access
@@ -1550,15 +1564,6 @@ int bdev_rw_virt(struct block_device *bdev, sector_t sector, void *data, }EXPORT_SYMBOL_GPL (bdev_rw_virt); -/* -* bio_await_chain - ends @bio and waits for every chained bio to complete -*/ -void bio_await_chain (struct bio *bio) -{ -bio_await (bio,NULL , bio_endio_cb); -bio_put (bio); -} -void __bio_advance (struct bio *bio,unsigned bytes) {if (bio_integrity (bio))
block/blk-lib.c +2 -14
@@ -155,13 +155,7 @@ static int blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector,__blkdev_issue_write_zeroes (bdev, sector, nr_sects, gfp, &bio, flags, limit);if (bio) { -if ((flags &BLKDEV_ZERO_KILLABLE ) && -fatal_signal_pending (current)) { -bio_await_chain (bio); -blk_finish_plug (&plug); -return -EINTR ; - } - ret =submit_bio_wait (bio); + ret =bio_submit_or_kill (bio, flags);bio_put (bio); }blk_finish_plug (&plug);
@@ -236,13 +230,7 @@ static int blkdev_issue_zero_pages(struct block_device *bdev, sector_t sector,blk_start_plug (&plug);__blkdev_issue_zero_pages (bdev, sector, nr_sects, gfp, &bio, flags);if (bio) { -if ((flags &BLKDEV_ZERO_KILLABLE ) && -fatal_signal_pending (current)) { -bio_await_chain (bio); -blk_finish_plug (&plug); -return -EINTR ; - } - ret =submit_bio_wait (bio); + ret =bio_submit_or_kill (bio, flags);bio_put (bio); }blk_finish_plug (&plug);
block/blk.h +1 -1
@@ -55,7 +55,7 @@ bool __blk_freeze_queue_start(struct request_queue *q, struct task_struct *owner);int __bio_queue_enter (struct request_queue *q,struct bio *bio);void submit_bio_noacct_nocheck (struct bio *bio,bool split); -void bio_await_chain (struct bio *bio); +int bio_submit_or_kill (struct bio *bio,unsigned int flags);static inline bool blk_try_enter_queue (struct request_queue *q,bool pm) {
block/ioctl.c +2 -9
@@ -153,13 +153,7 @@ static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, nr_sects = len >>SECTOR_SHIFT ;blk_start_plug (&plug); -while (1 ) { -if (fatal_signal_pending (current)) { -if (prev) -bio_await_chain (prev); - err = -EINTR ; - goto out_unplug; - } +while (!fatal_signal_pending (current)) { bio =blk_alloc_discard_bio (bdev, §or, &nr_sects,GFP_KERNEL );if (!bio)
@@ -167,12 +161,11 @@ static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, prev =bio_chain_and_submit (prev, bio); }if (prev) { - err =submit_bio_wait (prev); + err =bio_submit_or_kill (prev,BLKDEV_ZERO_KILLABLE );if (err == -EOPNOTSUPP ) err =0 ;bio_put (prev); } -out_unplug:blk_finish_plug (&plug); fail:filemap_invalidate_unlock (bdev->bd_mapping);
Signature · 1 line
2.47.3
Re: [PATCH 4/5] block: add a bio_submit_or_kill helper
Quoted 3 lines · Christoph Hellwig
Factor the common logic for the ioctl helpers to either submit a bio or end if the process is being killed. As this is now the only user of
bio_await_chain, open code that.
Re: [PATCH 4/5] block: add a bio_submit_or_kill helper
In reply to Christoph HellwigQuoted 23 lines · Christoph Hellwig
block/ioctl.c +1
-7@@ -153,13 +153,7 @@ static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, nr_sects = len >>SECTOR_SHIFT ;blk_start_plug (&plug); -while (1 ) { -if (fatal_signal_pending (current)) { -if (prev) -bio_await_chain (prev); - err = -EINTR ; - goto out_unplug; - } +while (!fatal_signal_pending (current)) { bio =blk_alloc_discard_bio (bdev, §or, &nr_sects,GFP_KERNEL );if (!bio)@@ -167,12 +161,11 @@ static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, prev =bio_chain_and_submit (prev, bio); }if (prev) {
block/ioctl.c static int blk_ioctl_discard(struct block_device *bdev, blk_mode_t mode, · line 171
Quoted 10 lines
block/ioctl.c +1
-2- err =submit_bio_wait (prev); + err =bio_submit_or_kill (prev,BLKDEV_ZERO_KILLABLE );if (err == -EOPNOTSUPP ) err =0 ;bio_put (prev); } -out_unplug:blk_finish_plug (&plug); fail:filemap_invalidate_unlock (bdev->bd_mapping);
Without this change, if a fatal signal is pending before the loop starts, -EINTR is returned. With this patch applied, 0 is returned, isn't it?
Thanks,
Bart.
Re: [PATCH 4/5] block: add a bio_submit_or_kill helper
Quoted 3 lines · Bart Van Assche
Without this change, if a fatal signal is pending before the loop starts,
-EINTRis returned. With this patch applied, 0 is returned, isn't it?
Yes, I'll look into a fix.
Re: [PATCH 4/5] block: add a bio_submit_or_kill helper
In reply to Bart Van AsscheQuoted 3 lines · Bart Van Assche
Without this change, if a fatal signal is pending before the loop starts,
-EINTRis returned. With this patch applied, 0 is returned, isn't it?
I looked at this a bit, and it turns that while this indeed changes the behavior for discard, the other ioctls had the return 0 when nothing was submitted before the thread got killed behavior, and this makes discard match them.
Given that we won't return from the system call for a fatal kill it should not matter. Adding Keith who wrote all this to confirm.
PATCH 5/5 xfs: use bio_await in xfs_zone_gc_reset_sync
In reply to Christoph HellwigReplace the open-coded bio wait logic with the new bio_await helper.
--- fs/xfs/xfs_zone_gc.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-)
fs/xfs/xfs_zone_gc.c +5 -14
@@ -900,9 +900,10 @@ xfs_zone_gc_finish_reset(static void xfs_submit_zone_reset_bio ( -struct xfs_rtgroup *rtg, -struct bio *bio) +struct bio *bio, +void *priv) { +struct xfs_rtgroup *rtg = priv;struct xfs_mount *mp =rtg_mount (rtg);trace_xfs_zone_reset (rtg);
@@ -934,26 +935,16 @@ xfs_submit_zone_reset_bio(submit_bio (bio); } -static void xfs_bio_wait_endio (struct bio *bio) -{ -complete (bio->bi_private); -} -int xfs_zone_gc_reset_sync (struct xfs_rtgroup *rtg) { -DECLARE_COMPLETION_ONSTACK (done);struct bio bio;int error;bio_init (&bio,rtg_mount (rtg)->m_rtdev_targp->bt_bdev,NULL ,0 ,REQ_OP_ZONE_RESET |REQ_SYNC ); - bio.bi_private = &done; - bio.bi_end_io = xfs_bio_wait_endio; -xfs_submit_zone_reset_bio (rtg, &bio); -wait_for_completion_io (&done); - +bio_await (&bio, rtg, xfs_submit_zone_reset_bio); error =blk_status_to_errno (bio.bi_status);bio_uninit (&bio);return error;
@@ -990,7 +981,7 @@ xfs_zone_gc_reset_zones( chunk->data = data;WRITE_ONCE (chunk->state,XFS_GC_BIO_NEW );list_add_tail (&chunk->entry, &data->resetting); -xfs_submit_zone_reset_bio (rtg, bio); +xfs_submit_zone_reset_bio (bio, rtg); }while (next); }
Signature · 1 line
2.47.3
Re: [PATCH 5/5] xfs: use bio_await in xfs_zone_gc_reset_sync
Quoted 4 lines · Christoph Hellwig
Replace the open-coded bio wait logic with the new
bio_awaithelper.Signed-off-by Christoph Hellwig<[email protected]>Reviewed-by Damien Le Moal<[email protected]>
Looks good.
-ck
Quoted 11 lines · Christoph Hellwig
this series factors common code between
submit_bio_waitandbio_await_chaininto a common helper, and then uses that in XFS as well instead of open coding such functionality. It also cleans up the submit or kill logic in the ioctl handlers to share more code.There is another places in btrfs that could be refactored to use this, although it is non-trivial, and I plan to add more users of this helper to XFS in the future.
[...]
Applied, thanks!
[1/5] xfs: fix number of GC bvecs
commit: 452c8f6cbd0ef1408474a875c5c4149a02c7610f
[2/5] block: unify the synchronous bi_end_io callbacks
commit: 65565ca5f99b42fe62b9a10117cca04f4311dc66
[3/5] block: factor out a bio_await helper
commit: 6fa747550e35f0a74e649b19d97055988a25b2e4
[4/5] block: add a bio_submit_or_kill helper
commit: 92c3737a2473ff5b83f90f5c1b353a27492a10f2
[5/5] xfs: use bio_await in xfs_zone_gc_reset_sync
commit: 2d148a214b24b4a2525f649cced0c3e9e57281cdBest regards,
Signature · 1 line
Jens Axboe