Don't allow REQ_NOWAIT for bio splitting
Subsystems (10)
Jens Axboe
maintainercover Don't allow REQ_NOWAIT for bio splitting
Hi,
If we end up needing to split a bio that is marked with REQ_NOWAIT, it's entirely possible that any part of the split bio will hit a failure trying to get submitted. If this happens, then the entire request ends up being errored with BLK_STS_AGAIN, even if any parts of this bio has been read or written to the device/file.
Rather than end up in this odd state, disallow splitting with REQ_NOWAIT. If we end the entire request upfront with EAGAIN, then we don't end up with a partially written IO that still returns EAGAIN.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerPATCH 1/2 block: handle bio_split_to_limits() NULL return
This can't happen right now, but in preparation for allowing bio_split_to_limits() returning NULL if it ended the bio, check for it in all the callers.
--- block/blk-merge.c | 4 +++- block/blk-mq.c | 5 ++++- drivers/block/drbd/drbd_req.c | 2 ++ drivers/block/ps3vram.c | 2 ++ drivers/md/dm.c | 2 ++ drivers/md/md.c | 2 ++ drivers/nvme/host/multipath.c | 2 ++ drivers/s390/block/dcssblk.c | 2 ++ 8 files changed, 19 insertions(+), 2 deletions(-)
block/blk-merge.c +3 -1
@@ -358,11 +358,13 @@ struct bio *__bio_split_to_limits(struct bio *bio,default : split =bio_split_rw (bio, lim, nr_segs, bs,get_max_io_size (bio, lim) <<SECTOR_SHIFT ); +if (IS_ERR (split)) +return NULL ;break ; }if (split) { -/* there isn't chance to merge the splitted bio */ +/* there isn't chance to merge the split bio */ split->bi_opf |=REQ_NOMERGE ;blkcg_bio_issue_init (split);
block/blk-mq.c +4 -1
@@ -2951,8 +2951,11 @@ void blk_mq_submit_bio(struct bio *bio)blk_status_t ret; bio =blk_queue_bounce (bio, q); -if (bio_may_exceed_limits (bio, &q->limits)) +if (bio_may_exceed_limits (bio, &q->limits)) { bio =__bio_split_to_limits (bio, &q->limits, &nr_segs); +if (!bio) +return ; + }if (!bio_integrity_prep (bio))return ;
drivers/block/drbd/drbd_req.c +2 -0
@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)struct drbd_device *device = bio->bi_bdev->bd_disk->private_data; bio =bio_split_to_limits (bio); +if (!bio) +return ;/* * what we "blindly" assume:
drivers/block/ps3vram.c +2 -0
@@ -587,6 +587,8 @@ static void ps3vram_submit_bio(struct bio *bio)dev_dbg (&dev->core,"%s\n" , __func__); bio =bio_split_to_limits (bio); +if (!bio) +return ;spin_lock_irq (&priv->lock); busy = !bio_list_empty (&priv->list);
drivers/md/dm.c +2 -0
@@ -1742,6 +1742,8 @@ static void dm_split_and_process_bio(struct mapped_device *md,* otherwise associated queue_limits won't be imposed. */ bio =bio_split_to_limits (bio); +if (!bio) +return ; }init_clone_info (&ci, md, map, bio, is_abnormal);
drivers/md/md.c +2 -0
@@ -455,6 +455,8 @@ static void md_submit_bio(struct bio *bio) } bio =bio_split_to_limits (bio); +if (!bio) +return ;if (mddev->ro ==MD_RDONLY &&unlikely (rw ==WRITE )) {if (bio_sectors (bio) !=0 )
drivers/nvme/host/multipath.c +2 -0
@@ -376,6 +376,8 @@ static void nvme_ns_head_submit_bio(struct bio *bio)* pool from the original queue to allocate the bvecs from. */ bio =bio_split_to_limits (bio); +if (!bio) +return ; srcu_idx =srcu_read_lock (&head->srcu); ns =nvme_find_path (head);
drivers/s390/block/dcssblk.c +2 -0
@@ -865,6 +865,8 @@ dcssblk_submit_bio(struct bio *bio)unsigned long bytes_done; bio =bio_split_to_limits (bio); +if (!bio) +return ; bytes_done =0 ; dev_info = bio->bi_bdev->bd_disk->private_data;
Signature · 1 line
2.39.0
Jens Axboe
maintainerPATCH 2/2 block: don't allow splitting of a REQ_NOWAIT bio
In reply to Jens AxboeIf we split a bio marked with REQ_NOWAIT, then we can trigger spurious EAGAIN if constituent parts of that split bio end up failing request allocations. Parts will complete just fine, but just a single failure in one of the chained bios will yield an EAGAIN final result for the parent bio.
Return EAGAIN early if we end up needing to split such a bio, which allows for saner recovery handling.
--- block/blk-merge.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
block/blk-merge.c +10 -0
@@ -309,6 +309,16 @@ static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim, *segs = nsegs;return NULL ; split: +/* +* We can't sanely support splitting for a REQ_NOWAIT bio. End it +* with EAGAIN if splitting is required and return an error pointer. +*/ +if (bio->bi_opf &REQ_NOWAIT ) { + bio->bi_status =BLK_STS_AGAIN ; +bio_endio (bio); +return ERR_PTR (-EAGAIN ); + } + *segs = nsegs;/*
Signature · 1 line
2.39.0
Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio
Quoted 8 lines · Jens Axboe
If we split a bio marked with
REQ_NOWAIT, then we can trigger spurious EAGAIN if constituent parts of that split bio end up failing request allocations. Parts will complete just fine, but just a single failure in one of the chained bios will yield an EAGAIN final result for the parent bio.Return EAGAIN early if we end up needing to split such a bio, which allows for saner recovery handling.
We're losing some performance here for large-ish single depth IO with nvme. We can get a little back by forcing to use the async worker earlier in the dispatch instead of getting all the way to the bio splitting, but the overhead to check for the condition (which is arbitrary decision anyway since we don't know the queue limits at io_uring prep time) mostly negates the gain.
It's probably fine, though, since you can still hit peak b/w with just a little higher qdepth. This patch is a simple way to handle the problem, so looks good to me.
Jens Axboe
maintainerRe: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio
Quoted 10 lines · Keith Busch
Quoted 8 lines · Jens Axboe
If we split a bio marked with
REQ_NOWAIT, then we can trigger spurious EAGAIN if constituent parts of that split bio end up failing request allocations. Parts will complete just fine, but just a single failure in one of the chained bios will yield an EAGAIN final result for the parent bio.Return EAGAIN early if we end up needing to split such a bio, which allows for saner recovery handling.
Quoted 6 lines
We're losing some performance here for large-ish single depth IO with nvme. We can get a little back by forcing to use the async worker earlier in the dispatch instead of getting all the way to the bio splitting, but the overhead to check for the condition (which is arbitrary decision anyway since we don't know the queue limits at
io_uringprep time) mostly negates the gain.
Yes, it's not perfect - for perfection, we'd need to be able to either arbitrarily retry parts of the split bio if we can't get a tag. Or reserve tags for this request when doing the splits. Either one of those would require extensive surgery to achieve. In my testing, the cost is low enough that I think we can live with this for now.
Quoted 5 lines
It's probably fine, though, since you can still hit peak b/w with just a little higher qdepth. This patch is a simple way to handle the problem, so looks good to me.
Reviewed-by Keith Busch <[email protected]>
Thanks!
Signature · 1 line
Jens Axboe
Re: [PATCH 1/2] block: handle bio_split_to_limits() NULL return
In reply to Jens AxboeQuoted 21 lines · Jens Axboe
This can't happen right now, but in preparation for allowing
bio_split_to_limits()returning NULL if it ended the bio, check for it in all the callers.Signed-off-by Jens Axboe <[email protected]>--- block/blk-merge.c | 4 +++- block/blk-mq.c | 5 ++++- drivers/block/drbd/drbd_req.c | 2 ++ drivers/block/ps3vram.c | 2 ++ drivers/md/dm.c | 2 ++ drivers/md/md.c | 2 ++ drivers/nvme/host/multipath.c | 2 ++ drivers/s390/block/dcssblk.c | 2 ++ 8 files changed, 19 insertions(+), 2 deletions(-)block/blk-merge.c +0
-0
block/blk-merge.c struct bio *__bio_split_to_limits(struct bio *bio, · line 362
Quoted 5 lines
block/blk-merge.c +2
-0@@ -358,11 +358,13 @@ struct bio *__bio_split_to_limits(struct bio *bio,default : split =bio_split_rw (bio, lim, nr_segs, bs,get_max_io_size (bio, lim) <<SECTOR_SHIFT ); +if (IS_ERR (split)) +return NULL ;
Can we decide on either passing an ERR_PTR or NULL and do it through the whole stack?
drivers/block/drbd/drbd_req.c void drbd_submit_bio(struct bio *bio) · line 1611
Quoted 10 lines
drivers/block/drbd/drbd_req.c +2
-0@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)struct drbd_device *device = bio->bi_bdev->bd_disk->private_data; bio =bio_split_to_limits (bio); +if (!bio) +return ;
So for the callers in drivers, do we need thee checks for drivers that don't even support REQ_NOWAIT?
Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio
In reply to Jens Axboeblock/blk-merge.c static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim, · line 320
Quoted 10 lines · Jens Axboe
split:
block/blk-merge.c +9
-0+/* +* We can't sanely support splitting for a REQ_NOWAIT bio. End it +* with EAGAIN if splitting is required and return an error pointer. +*/ +if (bio->bi_opf &REQ_NOWAIT ) { + bio->bi_status =BLK_STS_AGAIN ; +bio_endio (bio); +return ERR_PTR (-EAGAIN ); + }
Hmm. Just completing the bio here seems a little dangerous in terms of ownership. What speaks against letting the caller do it?
Re: [PATCH 1/2] block: handle bio_split_to_limits() NULL return
In reply to Christoph HellwigPer the question in the other thread: these are my comments to it.
Quoted 15 lines · Christoph Hellwig
block/blk-merge.c struct bio *__bio_split_to_limits(struct bio *bio, · line 362
Quoted 4 lines
block/blk-merge.c +2
-0split =bio_split_rw (bio, lim, nr_segs, bs,get_max_io_size (bio, lim) <<SECTOR_SHIFT ); +if (IS_ERR (split)) +return NULL ;Can we decide on either passing an
ERR_PTRor NULL and do it through the whole stack?Quoted 7 lines
drivers/block/drbd/drbd_req.c +0
-0@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)struct drbd_device *device = bio->bi_bdev->bd_disk->private_data;
Quoted 6 lines
drivers/block/drbd/drbd_req.c void drbd_submit_bio(struct bio *bio) · line 1611
Quoted 3 lines
drivers/block/drbd/drbd_req.c +2
-0bio =bio_split_to_limits (bio); +if (!bio) +return ;So for the callers in drivers, do we need thee checks for drivers that don't even support
REQ_NOWAIT?
---end quoted text---
Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio
In reply to Christoph HellwigAnd this is the other comment.
Quoted 14 lines · Christoph Hellwig
block/blk-merge.c static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim, · line 320
Quoted 10 lines · Jens Axboe
split:
block/blk-merge.c +9
-0+/* +* We can't sanely support splitting for a REQ_NOWAIT bio. End it +* with EAGAIN if splitting is required and return an error pointer. +*/ +if (bio->bi_opf &REQ_NOWAIT ) { + bio->bi_status =BLK_STS_AGAIN ; +bio_endio (bio); +return ERR_PTR (-EAGAIN ); + }Hmm. Just completing the bio here seems a little dangerous in terms of ownership. What speaks against letting the caller do it?
---end quoted text---
Jens Axboe
maintainerRe: [PATCH 1/2] block: handle bio_split_to_limits() NULL return
In reply to Christoph HellwigQuoted 17 lines · Christoph Hellwig
Quoted 16 lines · Jens Axboe
This can't happen right now, but in preparation for allowing
bio_split_to_limits()returning NULL if it ended the bio, check for it in all the callers.Signed-off-by Jens Axboe <[email protected]>--- block/blk-merge.c | 4 +++- block/blk-mq.c | 5 ++++- drivers/block/drbd/drbd_req.c | 2 ++ drivers/block/ps3vram.c | 2 ++ drivers/md/dm.c | 2 ++ drivers/md/md.c | 2 ++ drivers/nvme/host/multipath.c | 2 ++ drivers/s390/block/dcssblk.c | 2 ++ 8 files changed, 19 insertions(+), 2 deletions(-)
Quoted 13 lines
block/blk-merge.c struct bio *__bio_split_to_limits(struct bio *bio, · line 362
Quoted 10 lines
block/blk-merge.c +2
-0@@ -358,11 +358,13 @@ struct bio *__bio_split_to_limits(struct bio *bio,default : split =bio_split_rw (bio, lim, nr_segs, bs,get_max_io_size (bio, lim) <<SECTOR_SHIFT ); +if (IS_ERR (split)) +return NULL ;Can we decide on either passing an
ERR_PTRor NULL and do it through the whole stack?
We need the error return here as we could just return NULL and it not be an error, but for further down the stack I feel like returning an error that you can't do anything with (as it's already been dealt with) would be confusing. That's why I did it that way.
Quoted 7 lines
Quoted 7 lines
drivers/block/drbd/drbd_req.c +0
-0@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)struct drbd_device *device = bio->bi_bdev->bd_disk->private_data;
Quoted 6 lines
drivers/block/drbd/drbd_req.c void drbd_submit_bio(struct bio *bio) · line 1611
Quoted 3 lines
drivers/block/drbd/drbd_req.c +2
-0bio =bio_split_to_limits (bio); +if (!bio) +return ;So for the callers in drivers, do we need thee checks for drivers that don't even support
REQ_NOWAIT?
Good point, probably not, we should be erroring these out before they reach the driver.
Doesn't hurt though, it would not necessarily be obvious that you'd now need to start checking bio_split_to_limits() return values when you set NOWAIT on the queue.
Signature · 1 line
Jens Axboe