Skip to contentSkip to conversation

Don't allow REQ_NOWAIT for bio splitting

PATCH In review Jens Axboe 3 participants 10 messages Jan 4, 2023 – Jan 18, 2023

Subsystems (10)
lore
Share & tools
lore

Jens Axboe

author maintainer
#0 lore raw

cover Don't allow REQ_NOWAIT for bio splitting

Hi,

If we end up needing to split a bio that is marked with REQ_NOWAIT, it's entirely possible that any part of the split bio will hit a failure trying to get submitted. If this happens, then the entire request ends up being errored with BLK_STS_AGAIN, even if any parts of this bio has been read or written to the device/file.

Rather than end up in this odd state, disallow splitting with REQ_NOWAIT. If we end the entire request upfront with EAGAIN, then we don't end up with a partially written IO that still returns EAGAIN.

Signature · 1 line
Jens Axboe
author maintainer
#1 lore raw

PATCH 1/2 block: handle bio_split_to_limits() NULL return

This can't happen right now, but in preparation for allowing bio_split_to_limits() returning NULL if it ended the bio, check for it in all the callers.

---
 block/blk-merge.c             | 4 +++-
 block/blk-mq.c                | 5 ++++-
 drivers/block/drbd/drbd_req.c | 2 ++
 drivers/block/ps3vram.c       | 2 ++
 drivers/md/dm.c               | 2 ++
 drivers/md/md.c               | 2 ++
 drivers/nvme/host/multipath.c | 2 ++
 drivers/s390/block/dcssblk.c  | 2 ++
 8 files changed, 19 insertions(+), 2 deletions(-)
block/blk-merge.c +3 -1
index 35a8f75cc45d..071c5f8cf0cf 100644
@@ -358,11 +358,13 @@ struct bio *__bio_split_to_limits(struct bio *bio,
 	default:
 		split = bio_split_rw(bio, lim, nr_segs, bs,
 				get_max_io_size(bio, lim) << SECTOR_SHIFT);
+		if (IS_ERR(split))
+			return NULL;
 		break;
 	}
 
 	if (split) {
-		/* there isn't chance to merge the splitted bio */
+		/* there isn't chance to merge the split bio */
 		split->bi_opf |= REQ_NOMERGE;
 
 		blkcg_bio_issue_init(split);
block/blk-mq.c +4 -1
index c5cf0dbca1db..2c49b4151da1 100644
@@ -2951,8 +2951,11 @@ void blk_mq_submit_bio(struct bio *bio)
 	blk_status_t ret;
 
 	bio = blk_queue_bounce(bio, q);
-	if (bio_may_exceed_limits(bio, &q->limits))
+	if (bio_may_exceed_limits(bio, &q->limits)) {
 		bio = __bio_split_to_limits(bio, &q->limits, &nr_segs);
+		if (!bio)
+			return;
+	}
 
 	if (!bio_integrity_prep(bio))
 		return;
drivers/block/drbd/drbd_req.c +2 -0
index eb14ec8ec04c..e36216d50753 100644
@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)
 	struct drbd_device *device = bio->bi_bdev->bd_disk->private_data;
 
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;
 
 	/*
 	 * what we "blindly" assume:
drivers/block/ps3vram.c +2 -0
index c76e0148eada..574e470b220b 100644
@@ -587,6 +587,8 @@ static void ps3vram_submit_bio(struct bio *bio)
 	dev_dbg(&dev->core, "%s\n", __func__);
 
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;
 
 	spin_lock_irq(&priv->lock);
 	busy = !bio_list_empty(&priv->list);
drivers/md/dm.c +2 -0
index e1ea3a7bd9d9..b424a6ee27ba 100644
@@ -1742,6 +1742,8 @@ static void dm_split_and_process_bio(struct mapped_device *md,
 		 * otherwise associated queue_limits won't be imposed.
 		 */
 		bio = bio_split_to_limits(bio);
+		if (!bio)
+			return;
 	}
 
 	init_clone_info(&ci, md, map, bio, is_abnormal);
drivers/md/md.c +2 -0
index 775f1dde190a..8af639296b3c 100644
@@ -455,6 +455,8 @@ static void md_submit_bio(struct bio *bio)
 	}
 
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;
 
 	if (mddev->ro == MD_RDONLY && unlikely(rw == WRITE)) {
 		if (bio_sectors(bio) != 0)
drivers/nvme/host/multipath.c +2 -0
index c03093b6813c..fc39d01e7b63 100644
@@ -376,6 +376,8 @@ static void nvme_ns_head_submit_bio(struct bio *bio)
 	 * pool from the original queue to allocate the bvecs from.
 	 */
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;
 
 	srcu_idx = srcu_read_lock(&head->srcu);
 	ns = nvme_find_path(head);
drivers/s390/block/dcssblk.c +2 -0
index b392b9f5482e..c0f85ffb2b62 100644
@@ -865,6 +865,8 @@ dcssblk_submit_bio(struct bio *bio)
 	unsigned long bytes_done;
 
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;
 
 	bytes_done = 0;
 	dev_info = bio->bi_bdev->bd_disk->private_data;
Signature · 1 line
2.39.0
#2 lore raw

Re: [PATCH 1/2] block: handle bio_split_to_limits() NULL return

Quoted 21 lines · Jens Axboe

This can't happen right now, but in preparation for allowing bio_split_to_limits() returning NULL if it ended the bio, check for it in all the callers.

---
 block/blk-merge.c             | 4 +++-
 block/blk-mq.c                | 5 ++++-
 drivers/block/drbd/drbd_req.c | 2 ++
 drivers/block/ps3vram.c       | 2 ++
 drivers/md/dm.c               | 2 ++
 drivers/md/md.c               | 2 ++
 drivers/nvme/host/multipath.c | 2 ++
 drivers/s390/block/dcssblk.c  | 2 ++
 8 files changed, 19 insertions(+), 2 deletions(-)
block/blk-merge.c +0 -0
index 35a8f75cc45d..071c5f8cf0cf 100644

block/blk-merge.c struct bio *__bio_split_to_limits(struct bio *bio, · line 362

Quoted 5 lines
block/blk-merge.c +2 -0
@@ -358,11 +358,13 @@ struct bio *__bio_split_to_limits(struct bio *bio,
 	default:
 		split = bio_split_rw(bio, lim, nr_segs, bs,
 				get_max_io_size(bio, lim) << SECTOR_SHIFT);
+		if (IS_ERR(split))
+			return NULL;

Can we decide on either passing an ERR_PTR or NULL and do it through the whole stack?

drivers/block/drbd/drbd_req.c void drbd_submit_bio(struct bio *bio) · line 1611

Quoted 10 lines
drivers/block/drbd/drbd_req.c +2 -0
index eb14ec8ec04c..e36216d50753 100644
@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)
 	struct drbd_device *device = bio->bi_bdev->bd_disk->private_data;
 
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;

So for the callers in drivers, do we need thee checks for drivers that don't even support REQ_NOWAIT?

#3 lore raw

Re: [PATCH 1/2] block: handle bio_split_to_limits() NULL return

Per the question in the other thread: these are my comments to it.

Quoted 15 lines · Christoph Hellwig

block/blk-merge.c struct bio *__bio_split_to_limits(struct bio *bio, · line 362

Quoted 4 lines
block/blk-merge.c +2 -0
 		split = bio_split_rw(bio, lim, nr_segs, bs,
 				get_max_io_size(bio, lim) << SECTOR_SHIFT);
+		if (IS_ERR(split))
+			return NULL;

Can we decide on either passing an ERR_PTR or NULL and do it through the whole stack?

Quoted 7 lines
drivers/block/drbd/drbd_req.c +0 -0
index eb14ec8ec04c..e36216d50753 100644
@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)
 	struct drbd_device *device = bio->bi_bdev->bd_disk->private_data;
 
Quoted 6 lines

drivers/block/drbd/drbd_req.c void drbd_submit_bio(struct bio *bio) · line 1611

Quoted 3 lines
drivers/block/drbd/drbd_req.c +2 -0
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;

So for the callers in drivers, do we need thee checks for drivers that don't even support REQ_NOWAIT?

---end quoted text---

author maintainer
#4 lore raw

Re: [PATCH 1/2] block: handle bio_split_to_limits() NULL return

In reply to Christoph Hellwig
Quoted 17 lines · Christoph Hellwig
Quoted 16 lines · Jens Axboe

This can't happen right now, but in preparation for allowing bio_split_to_limits() returning NULL if it ended the bio, check for it in all the callers.

---
 block/blk-merge.c             | 4 +++-
 block/blk-mq.c                | 5 ++++-
 drivers/block/drbd/drbd_req.c | 2 ++
 drivers/block/ps3vram.c       | 2 ++
 drivers/md/dm.c               | 2 ++
 drivers/md/md.c               | 2 ++
 drivers/nvme/host/multipath.c | 2 ++
 drivers/s390/block/dcssblk.c  | 2 ++
 8 files changed, 19 insertions(+), 2 deletions(-)
Quoted 13 lines

block/blk-merge.c struct bio *__bio_split_to_limits(struct bio *bio, · line 362

Quoted 10 lines
block/blk-merge.c +2 -0
index 35a8f75cc45d..071c5f8cf0cf 100644
@@ -358,11 +358,13 @@ struct bio *__bio_split_to_limits(struct bio *bio,
 	default:
 		split = bio_split_rw(bio, lim, nr_segs, bs,
 				get_max_io_size(bio, lim) << SECTOR_SHIFT);
+		if (IS_ERR(split))
+			return NULL;

Can we decide on either passing an ERR_PTR or NULL and do it through the whole stack?

We need the error return here as we could just return NULL and it not be an error, but for further down the stack I feel like returning an error that you can't do anything with (as it's already been dealt with) would be confusing. That's why I did it that way.

Quoted 7 lines
Quoted 7 lines
drivers/block/drbd/drbd_req.c +0 -0
index eb14ec8ec04c..e36216d50753 100644
@@ -1607,6 +1607,8 @@ void drbd_submit_bio(struct bio *bio)
 	struct drbd_device *device = bio->bi_bdev->bd_disk->private_data;
 
Quoted 6 lines

drivers/block/drbd/drbd_req.c void drbd_submit_bio(struct bio *bio) · line 1611

Quoted 3 lines
drivers/block/drbd/drbd_req.c +2 -0
 	bio = bio_split_to_limits(bio);
+	if (!bio)
+		return;

So for the callers in drivers, do we need thee checks for drivers that don't even support REQ_NOWAIT?

Good point, probably not, we should be erroring these out before they reach the driver.

Doesn't hurt though, it would not necessarily be obvious that you'd now need to start checking bio_split_to_limits() return values when you set NOWAIT on the queue.

Signature · 1 line
Jens Axboe
author maintainer
#5 lore raw

PATCH 2/2 block: don't allow splitting of a REQ_NOWAIT bio

In reply to Jens Axboe

If we split a bio marked with REQ_NOWAIT, then we can trigger spurious EAGAIN if constituent parts of that split bio end up failing request allocations. Parts will complete just fine, but just a single failure in one of the chained bios will yield an EAGAIN final result for the parent bio.

Return EAGAIN early if we end up needing to split such a bio, which allows for saner recovery handling.

---
 block/blk-merge.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
block/blk-merge.c +10 -0
index 071c5f8cf0cf..b7c193d67185 100644
@@ -309,6 +309,16 @@ static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim,
 	*segs = nsegs;
 	return NULL;
 split:
+	/*
+	 * We can't sanely support splitting for a REQ_NOWAIT bio. End it
+	 * with EAGAIN if splitting is required and return an error pointer.
+	 */
+	if (bio->bi_opf & REQ_NOWAIT) {
+		bio->bi_status = BLK_STS_AGAIN;
+		bio_endio(bio);
+		return ERR_PTR(-EAGAIN);
+	}
+
 	*segs = nsegs;
 
 	/*
Signature · 1 line
2.39.0
maintainer Reviewed-by on v1 2/2
#6 lore raw

Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio

Quoted 8 lines · Jens Axboe

If we split a bio marked with REQ_NOWAIT, then we can trigger spurious EAGAIN if constituent parts of that split bio end up failing request allocations. Parts will complete just fine, but just a single failure in one of the chained bios will yield an EAGAIN final result for the parent bio.

Return EAGAIN early if we end up needing to split such a bio, which allows for saner recovery handling.

We're losing some performance here for large-ish single depth IO with nvme. We can get a little back by forcing to use the async worker earlier in the dispatch instead of getting all the way to the bio splitting, but the overhead to check for the condition (which is arbitrary decision anyway since we don't know the queue limits at io_uring prep time) mostly negates the gain.

It's probably fine, though, since you can still hit peak b/w with just a little higher qdepth. This patch is a simple way to handle the problem, so looks good to me.

author maintainer
#7 lore raw

Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio

Quoted 10 lines · Keith Busch
Quoted 8 lines · Jens Axboe

If we split a bio marked with REQ_NOWAIT, then we can trigger spurious EAGAIN if constituent parts of that split bio end up failing request allocations. Parts will complete just fine, but just a single failure in one of the chained bios will yield an EAGAIN final result for the parent bio.

Return EAGAIN early if we end up needing to split such a bio, which allows for saner recovery handling.

Quoted 6 lines

We're losing some performance here for large-ish single depth IO with nvme. We can get a little back by forcing to use the async worker earlier in the dispatch instead of getting all the way to the bio splitting, but the overhead to check for the condition (which is arbitrary decision anyway since we don't know the queue limits at io_uring prep time) mostly negates the gain.

Yes, it's not perfect - for perfection, we'd need to be able to either arbitrarily retry parts of the split bio if we can't get a tag. Or reserve tags for this request when doing the splits. Either one of those would require extensive surgery to achieve. In my testing, the cost is low enough that I think we can live with this for now.

Quoted 5 lines

It's probably fine, though, since you can still hit peak b/w with just a little higher qdepth. This patch is a simple way to handle the problem, so looks good to me.

Thanks!

Signature · 1 line
Jens Axboe
#8 lore raw

Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio

In reply to Jens Axboe

block/blk-merge.c static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim, · line 320

Quoted 10 lines · Jens Axboe

split:

block/blk-merge.c +9 -0
+	/*
+	 * We can't sanely support splitting for a REQ_NOWAIT bio. End it
+	 * with EAGAIN if splitting is required and return an error pointer.
+	 */
+	if (bio->bi_opf & REQ_NOWAIT) {
+		bio->bi_status = BLK_STS_AGAIN;
+		bio_endio(bio);
+		return ERR_PTR(-EAGAIN);
+	}

Hmm. Just completing the bio here seems a little dangerous in terms of ownership. What speaks against letting the caller do it?

#9 lore raw

Re: [PATCH 2/2] block: don't allow splitting of a REQ_NOWAIT bio

And this is the other comment.

Quoted 14 lines · Christoph Hellwig

block/blk-merge.c static struct bio *bio_split_rw(struct bio *bio, const struct queue_limits *lim, · line 320

Quoted 10 lines · Jens Axboe

split:

block/blk-merge.c +9 -0
+	/*
+	 * We can't sanely support splitting for a REQ_NOWAIT bio. End it
+	 * with EAGAIN if splitting is required and return an error pointer.
+	 */
+	if (bio->bi_opf & REQ_NOWAIT) {
+		bio->bi_status = BLK_STS_AGAIN;
+		bio_endio(bio);
+		return ERR_PTR(-EAGAIN);
+	}

Hmm. Just completing the bio here seems a little dangerous in terms of ownership. What speaks against letting the caller do it?

---end quoted text---