Skip to contentSkip to conversation

cleanup bio page releasing and fix a page leak

PATCH In review Christoph Hellwig 5 participants 23 messages May 2, 2019 – May 7, 2019

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Jens,

this series cleans up the various direct I/O and pass through routines by switching them over to common bio helpers. For the block device simple case this also fixes a page leak if we were using bvec iters.

The last page just unconditionally applies the no page ref behavior for bvec iters. I looked at all the callers, and there is none that drops the pre-required references before completing the request. Probably not suitable for so late in the merge, but I wanted to get it out.

#1 lore raw

PATCH 1/8 block: move the BIO_NO_PAGE_REF check into bio_release_pages

Move the BIO_NO_PAGE_REF check into bio_release_pages instead of duplicating it in both callers.

Also make the function available outside of bio.c so that we can reuse it in other direct I/O implementations.

---
 block/bio.c         | 11 ++++++-----
 include/linux/bio.h |  1 +
 2 files changed, 7 insertions(+), 5 deletions(-)
block/bio.c +6 -5
index 683cbb40f051..96ddffa49881 100644
@@ -855,11 +855,14 @@ static void bio_get_pages(struct bio *bio)
 		get_page(bvec->bv_page);
 }
 
-static void bio_release_pages(struct bio *bio)
+void bio_release_pages(struct bio *bio)
 {
 	struct bvec_iter_all iter_all;
 	struct bio_vec *bvec;
 
+	if (bio_flagged(bio, BIO_NO_PAGE_REF))
+		return;
+
 	bio_for_each_segment_all(bvec, bio, iter_all)
 		put_page(bvec->bv_page);
 }
@@ -1692,8 +1695,7 @@ static void bio_dirty_fn(struct work_struct *work)
 		next = bio->bi_private;
 
 		bio_set_pages_dirty(bio);
-		if (!bio_flagged(bio, BIO_NO_PAGE_REF))
-			bio_release_pages(bio);
+		bio_release_pages(bio);
 		bio_put(bio);
 	}
 }
@@ -1709,8 +1711,7 @@ void bio_check_pages_dirty(struct bio *bio)
 			goto defer;
 	}
 
-	if (!bio_flagged(bio, BIO_NO_PAGE_REF))
-		bio_release_pages(bio);
+	bio_release_pages(bio);
 	bio_put(bio);
 	return;
 defer:
include/linux/bio.h +1 -0
index ea73df36529a..d5699a54328e 100644
@@ -427,6 +427,7 @@ bool __bio_try_merge_page(struct bio *bio, struct page *page,
 void __bio_add_page(struct bio *bio, struct page *page,
 		unsigned int len, unsigned int off);
 int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter);
+void bio_release_pages(struct bio *bio);
 struct rq_map_data;
 extern struct bio *bio_map_user_iov(struct request_queue *,
 				    struct iov_iter *, gfp_t);
Signature · 1 line
2.20.1
Reviewed-by on v1 1/8
#2 lore raw

Re: [PATCH 1/8] block: move the BIO_NO_PAGE_REF check into bio_release_pages

Looks good,

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#3 lore raw

PATCH 2/8 block: use bio_release_pages in bio_unmap_user

In reply to Christoph Hellwig

Use bio_release_pages and bio_set_pages_dirty instead of open coding them.

---
 block/bio.c | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)
block/bio.c +3 -19
index 96ddffa49881..a6862b954350 100644
@@ -1445,24 +1445,6 @@ struct bio *bio_map_user_iov(struct request_queue *q,
 	return ERR_PTR(ret);
 }
 
-static void __bio_unmap_user(struct bio *bio)
-{
-	struct bio_vec *bvec;
-	struct bvec_iter_all iter_all;
-
-	/*
-	 * make sure we dirty pages we wrote to
-	 */
-	bio_for_each_segment_all(bvec, bio, iter_all) {
-		if (bio_data_dir(bio) == READ)
-			set_page_dirty_lock(bvec->bv_page);
-
-		put_page(bvec->bv_page);
-	}
-
-	bio_put(bio);
-}
-
 /**
  *	bio_unmap_user	-	unmap a bio
  *	@bio:		the bio being unmapped
@@ -1474,7 +1456,9 @@ static void __bio_unmap_user(struct bio *bio)
  */
 void bio_unmap_user(struct bio *bio)
 {
-	__bio_unmap_user(bio);
+	bio_set_pages_dirty(bio);
+	bio_release_pages(bio);
+	bio_put(bio);
 	bio_put(bio);
 }
 
Signature · 1 line
2.20.1
#4 lore raw

Re: [PATCH 2/8] block: use bio_release_pages in bio_unmap_user

Quoted 38 lines · Christoph Hellwig

Use bio_release_pages and bio_set_pages_dirty instead of open coding them.

---
 block/bio.c | 22 +++-------------------
 1 file changed, 3 insertions(+), 19 deletions(-)
block/bio.c +0 -18
index 96ddffa49881..a6862b954350 100644
@@ -1445,24 +1445,6 @@ struct bio *bio_map_user_iov(struct request_queue *q,
 	return ERR_PTR(ret);
 }
 
-static void __bio_unmap_user(struct bio *bio)
-{
-	struct bio_vec *bvec;
-	struct bvec_iter_all iter_all;
-
-	/*
-	 * make sure we dirty pages we wrote to
-	 */
-	bio_for_each_segment_all(bvec, bio, iter_all) {
-		if (bio_data_dir(bio) == READ)
-			set_page_dirty_lock(bvec->bv_page);
-
-		put_page(bvec->bv_page);
-	}
-
-	bio_put(bio);
-}
-
 /**
  *	bio_unmap_user	-	unmap a bio
  *	@bio:		the bio being unmapped

block/bio.c static void __bio_unmap_user(struct bio *bio) · line 1459

Quoted 5 lines
block/bio.c +1 -1
@@ -1474,7 +1456,9 @@ static void __bio_unmap_user(struct bio *bio)
  */
 void bio_unmap_user(struct bio *bio)
 {
-	__bio_unmap_user(bio);
+	bio_set_pages_dirty(bio);

Doesn't this need to be :

if (bio_data_dir(bio) == READ)
bio_set_pages_dirty()

Quoted 6 lines
block/bio.c +2 -0
+	bio_release_pages(bio);
+	bio_put(bio);
 	bio_put(bio);
 }
 
#5 lore raw

Re: [PATCH 2/8] block: use bio_release_pages in bio_unmap_user

Quoted 11 lines · Nikolay Borisov

block/bio.c static void __bio_unmap_user(struct bio *bio) · line 1459

Quoted 6 lines
block/bio.c +1 -1
@@ -1474,7 +1456,9 @@ static void __bio_unmap_user(struct bio *bio)
  */
 void bio_unmap_user(struct bio *bio)
 {
-	__bio_unmap_user(bio);
+	bio_set_pages_dirty(bio);

Doesn't this need to be :

if (bio_data_dir(bio) == READ)
bio_set_pages_dirty()

Yes, indeed.

#6 lore raw

PATCH 3/8 block: use bio_release_pages in bio_map_user_iov

In reply to Christoph Hellwig

Use bio_release_pages instead of open coding it.

---
 block/bio.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
block/bio.c +1 -5
index a6862b954350..3938e179a530 100644
@@ -1370,8 +1370,6 @@ struct bio *bio_map_user_iov(struct request_queue *q,
 	int j;
 	struct bio *bio;
 	int ret;
-	struct bio_vec *bvec;
-	struct bvec_iter_all iter_all;
 
 	if (!iov_iter_count(iter))
 		return ERR_PTR(-EINVAL);
@@ -1438,9 +1436,7 @@ struct bio *bio_map_user_iov(struct request_queue *q,
 	return bio;
 
  out_unmap:
-	bio_for_each_segment_all(bvec, bio, iter_all) {
-		put_page(bvec->bv_page);
-	}
+	bio_release_pages(bio);
 	bio_put(bio);
 	return ERR_PTR(ret);
 }
Signature · 1 line
2.20.1
Reviewed-by on v1 3/8
#7 lore raw

Re: [PATCH 3/8] block: use bio_release_pages in bio_map_user_iov

Looks good,

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#8 lore raw

PATCH 4/8 iomap: use bio_release_pages in iomap_dio_bio_end_io

In reply to Christoph Hellwig

Use bio_release_pages instead of duplicating it.

---
 fs/iomap.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
fs/iomap.c +1 -7
index 12a656271076..a7bff5b2e1e8 100644
@@ -1588,13 +1588,7 @@ static void iomap_dio_bio_end_io(struct bio *bio)
 	if (should_dirty) {
 		bio_check_pages_dirty(bio);
 	} else {
-		if (!bio_flagged(bio, BIO_NO_PAGE_REF)) {
-			struct bvec_iter_all iter_all;
-			struct bio_vec *bvec;
-
-			bio_for_each_segment_all(bvec, bio, iter_all)
-				put_page(bvec->bv_page);
-		}
+		bio_release_pages(bio);
 		bio_put(bio);
 	}
 }
Signature · 1 line
2.20.1
#9 lore raw

Re: [PATCH 4/8] iomap: use bio_release_pages in iomap_dio_bio_end_io

fs/iomap.c static void iomap_dio_bio_end_io(struct bio *bio) · line 1591

Quoted 12 lines · Christoph Hellwig
fs/iomap.c +1 -7
@@ -1588,13 +1588,7 @@ static void iomap_dio_bio_end_io(struct bio *bio)
 	if (should_dirty) {
 		bio_check_pages_dirty(bio);
 	} else {
-		if (!bio_flagged(bio, BIO_NO_PAGE_REF)) {
-			struct bvec_iter_all iter_all;
-			struct bio_vec *bvec;
-
-			bio_for_each_segment_all(bvec, bio, iter_all)
-				put_page(bvec->bv_page);
-		}
+		bio_release_pages(bio);

Shouldn't this rather be:

		if (!bio_flagged(bio, BIO_NO_PAGE_REF)) 
			bio_release_pages(bio);
Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
Reviewed-by
#10 lore raw

Re: [PATCH 4/8] iomap: use bio_release_pages in iomap_dio_bio_end_io

Quoted 5 lines · Johannes Thumshirn

Shouldn't this rather be:

		if (!bio_flagged(bio, BIO_NO_PAGE_REF)) 
			bio_release_pages(bio);

OK I apparently can't remember between 3 emails, sorry

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#11 lore raw

PATCH 5/8 block_dev: use bio_release_pages in blkdev_bio_end_io

In reply to Christoph Hellwig

Use bio_release_pages instead of duplicating it.

---
 fs/block_dev.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
fs/block_dev.c +1 -7
index 8abc6570d29f..a59ebea9d125 100644
@@ -335,13 +335,7 @@ static void blkdev_bio_end_io(struct bio *bio)
 	if (should_dirty) {
 		bio_check_pages_dirty(bio);
 	} else {
-		if (!bio_flagged(bio, BIO_NO_PAGE_REF)) {
-			struct bvec_iter_all iter_all;
-			struct bio_vec *bvec;
-
-			bio_for_each_segment_all(bvec, bio, iter_all)
-				put_page(bvec->bv_page);
-		}
+		bio_release_pages(bio);
 		bio_put(bio);
 	}
 }
Signature · 1 line
2.20.1
Reviewed-by on v1 5/8
#12 lore raw

Re: [PATCH 5/8] block_dev: use bio_release_pages in blkdev_bio_end_io

Looks good,

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#13 lore raw

PATCH 6/8 block_dev: use bio_release_pages in __blkdev_direct_IO_simple

In reply to Christoph Hellwig

Use bio_release_pages instead of a manual put_pages loop that fails to take BIO_NO_PAGE_REF into account. Also use bio_set_pages_dirty for the rest of the former loop to reduce code duplication.

Fixes 399254aaf489 ("block: add BIO_NO_PAGE_REF flag")
---
 fs/block_dev.c | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)
fs/block_dev.c +4 -7
index a59ebea9d125..4fc2377884a3 100644
@@ -204,13 +204,12 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
 {
 	struct file *file = iocb->ki_filp;
 	struct block_device *bdev = I_BDEV(bdev_file_inode(file));
-	struct bio_vec inline_vecs[DIO_INLINE_BIO_VECS], *vecs, *bvec;
+	struct bio_vec inline_vecs[DIO_INLINE_BIO_VECS], *vecs;
 	loff_t pos = iocb->ki_pos;
 	bool should_dirty = false;
 	struct bio bio;
 	ssize_t ret;
 	blk_qc_t qc;
-	struct bvec_iter_all iter_all;
 
 	if ((pos | iov_iter_alignment(iter)) &
 	    (bdev_logical_block_size(bdev) - 1))
@@ -260,11 +259,9 @@ __blkdev_direct_IO_simple(struct kiocb *iocb, struct iov_iter *iter,
 	}
 	__set_current_state(TASK_RUNNING);
 
-	bio_for_each_segment_all(bvec, &bio, iter_all) {
-		if (should_dirty && !PageCompound(bvec->bv_page))
-			set_page_dirty_lock(bvec->bv_page);
-		put_page(bvec->bv_page);
-	}
+	if (should_dirty)
+		bio_set_pages_dirty(&bio);
+	bio_release_pages(&bio);
 
 	if (unlikely(bio.bi_status))
 		ret = blk_status_to_errno(bio.bi_status);
Signature · 1 line
2.20.1
Reviewed-by on v1 6/8
#14 lore raw

Re: [PATCH 6/8] block_dev: use bio_release_pages in __blkdev_direct_IO_simple

Looks good,

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#15 lore raw

PATCH 7/8 direct-io: use bio_release_pages in dio_bio_complete

In reply to Christoph Hellwig

Use bio_release_pages and bio_set_pages_dirty instead of open coding them.

---
 fs/direct-io.c | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)
fs/direct-io.c +3 -12
index fbe885d68035..62d0594a4622 100644
@@ -537,7 +537,6 @@ static struct bio *dio_await_one(struct dio *dio)
  */
 static blk_status_t dio_bio_complete(struct dio *dio, struct bio *bio)
 {
-	struct bio_vec *bvec;
 	blk_status_t err = bio->bi_status;
 
 	if (err) {
@@ -550,17 +549,9 @@ static blk_status_t dio_bio_complete(struct dio *dio, struct bio *bio)
 	if (dio->is_async && dio->op == REQ_OP_READ && dio->should_dirty) {
 		bio_check_pages_dirty(bio);	/* transfers ownership */
 	} else {
-		struct bvec_iter_all iter_all;
-
-		bio_for_each_segment_all(bvec, bio, iter_all) {
-			struct page *page = bvec->bv_page;
-
-			if (dio->op == REQ_OP_READ && !PageCompound(page) &&
-					dio->should_dirty)
-				set_page_dirty_lock(page);
-			put_page(page);
-		}
-		bio_put(bio);
+		if (dio->op == REQ_OP_READ && dio->should_dirty)
+			bio_set_pages_dirty(bio);
+		bio_release_pages(bio);
 	}
 	return err;
 }
Signature · 1 line
2.20.1
Reviewed-by on v1 7/8
#16 lore raw

Re: [PATCH 7/8] direct-io: use bio_release_pages in dio_bio_complete

Looks good,

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#17 lore raw

PATCH 8/8 block: never take page references for ITER_BVEC

In reply to Christoph Hellwig

If we pass pages through an iov_iter we always already have a reference in the caller. Thus remove the ITER_BVEC_FLAG_NO_REF and don't take reference to pages by default for bvec backed iov_iters.

---
 block/bio.c          | 14 +-------------
 drivers/block/loop.c | 16 ++++------------
 fs/io_uring.c        |  3 ---
 include/linux/uio.h  |  8 --------
 4 files changed, 5 insertions(+), 36 deletions(-)
block/bio.c +1 -13
index 3938e179a530..e999d530d863 100644
@@ -846,15 +846,6 @@ int bio_add_page(struct bio *bio, struct page *page,
 }
 EXPORT_SYMBOL(bio_add_page);
 
-static void bio_get_pages(struct bio *bio)
-{
-	struct bvec_iter_all iter_all;
-	struct bio_vec *bvec;
-
-	bio_for_each_segment_all(bvec, bio, iter_all)
-		get_page(bvec->bv_page);
-}
-
 void bio_release_pages(struct bio *bio)
 {
 	struct bvec_iter_all iter_all;
@@ -967,11 +958,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
 			ret = __bio_iov_iter_get_pages(bio, iter);
 	} while (!ret && iov_iter_count(iter) && !bio_full(bio));
 
-	if (iov_iter_bvec_no_ref(iter))
+	if (is_bvec)
 		bio_set_flag(bio, BIO_NO_PAGE_REF);
-	else if (is_bvec)
-		bio_get_pages(bio);
-
 	return bio->bi_vcnt ? 0 : ret;
 }
 
drivers/block/loop.c +4 -12
index 102d79575895..c20710e617c2 100644
@@ -264,20 +264,12 @@ lo_do_transfer(struct loop_device *lo, int cmd,
 	return ret;
 }
 
-static inline void loop_iov_iter_bvec(struct iov_iter *i,
-		unsigned int direction, const struct bio_vec *bvec,
-		unsigned long nr_segs, size_t count)
-{
-	iov_iter_bvec(i, direction, bvec, nr_segs, count);
-	i->type |= ITER_BVEC_FLAG_NO_REF;
-}
-
 static int lo_write_bvec(struct file *file, struct bio_vec *bvec, loff_t *ppos)
 {
 	struct iov_iter i;
 	ssize_t bw;
 
-	loop_iov_iter_bvec(&i, WRITE, bvec, 1, bvec->bv_len);
+	iov_iter_bvec(&i, WRITE, bvec, 1, bvec->bv_len);
 
 	file_start_write(file);
 	bw = vfs_iter_write(file, &i, ppos, 0);
@@ -355,7 +347,7 @@ static int lo_read_simple(struct loop_device *lo, struct request *rq,
 	ssize_t len;
 
 	rq_for_each_segment(bvec, rq, iter) {
-		loop_iov_iter_bvec(&i, READ, &bvec, 1, bvec.bv_len);
+		iov_iter_bvec(&i, READ, &bvec, 1, bvec.bv_len);
 		len = vfs_iter_read(lo->lo_backing_file, &i, &pos, 0);
 		if (len < 0)
 			return len;
@@ -396,7 +388,7 @@ static int lo_read_transfer(struct loop_device *lo, struct request *rq,
 		b.bv_offset = 0;
 		b.bv_len = bvec.bv_len;
 
-		loop_iov_iter_bvec(&i, READ, &b, 1, b.bv_len);
+		iov_iter_bvec(&i, READ, &b, 1, b.bv_len);
 		len = vfs_iter_read(lo->lo_backing_file, &i, &pos, 0);
 		if (len < 0) {
 			ret = len;
@@ -563,7 +555,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
 	}
 	atomic_set(&cmd->ref, 2);
 
-	loop_iov_iter_bvec(&iter, rw, bvec, nr_bvec, blk_rq_bytes(rq));
+	iov_iter_bvec(&iter, rw, bvec, nr_bvec, blk_rq_bytes(rq));
 	iter.iov_offset = offset;
 
 	cmd->iocb.ki_pos = pos;
fs/io_uring.c +0 -3
index f65f85d89217..f7eb63a5b3db 100644
@@ -853,9 +853,6 @@ static int io_import_fixed(struct io_ring_ctx *ctx, int rw,
 	iov_iter_bvec(iter, rw, imu->bvec, imu->nr_bvecs, offset + len);
 	if (offset)
 		iov_iter_advance(iter, offset);
-
-	/* don't drop a reference to these pages */
-	iter->type |= ITER_BVEC_FLAG_NO_REF;
 	return 0;
 }
 
include/linux/uio.h +0 -8
index f184af1999a8..bace8fd40d0c 100644
@@ -23,9 +23,6 @@ struct kvec {
 };
 
 enum iter_type {
-	/* set if ITER_BVEC doesn't hold a bv_page ref */
-	ITER_BVEC_FLAG_NO_REF = 2,
-
 	/* iter types */
 	ITER_IOVEC = 4,
 	ITER_KVEC = 8,
@@ -93,11 +90,6 @@ static inline unsigned char iov_iter_rw(const struct iov_iter *i)
 	return i->type & (READ | WRITE);
 }
 
-static inline bool iov_iter_bvec_no_ref(const struct iov_iter *i)
-{
-	return (i->type & ITER_BVEC_FLAG_NO_REF) != 0;
-}
-
 /*
  * Total number of bytes covered by an iovec.
  *
Signature · 1 line
2.20.1
Reviewed-by on v1 8/8
#18 lore raw

Re: [PATCH 8/8] block: never take page references for ITER_BVEC

Looks good,

Signature · 6 lines
Johannes Thumshirn                            SUSE Labs Filesystems
[email protected]                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
#19 lore raw

Re: [PATCH 8/8] block: never take page references for ITER_BVEC

In reply to Christoph Hellwig
Quoted 128 lines · Christoph Hellwig

If we pass pages through an iov_iter we always already have a reference in the caller. Thus remove the ITER_BVEC_FLAG_NO_REF and don't take reference to pages by default for bvec backed iov_iters.

---
 block/bio.c          | 14 +-------------
 drivers/block/loop.c | 16 ++++------------
 fs/io_uring.c        |  3 ---
 include/linux/uio.h  |  8 --------
 4 files changed, 5 insertions(+), 36 deletions(-)
block/bio.c +1 -13
index 3938e179a530..e999d530d863 100644
@@ -846,15 +846,6 @@ int bio_add_page(struct bio *bio, struct page *page,
 }
 EXPORT_SYMBOL(bio_add_page);
 
-static void bio_get_pages(struct bio *bio)
-{
-	struct bvec_iter_all iter_all;
-	struct bio_vec *bvec;
-
-	bio_for_each_segment_all(bvec, bio, iter_all)
-		get_page(bvec->bv_page);
-}
-
 void bio_release_pages(struct bio *bio)
 {
 	struct bvec_iter_all iter_all;
@@ -967,11 +958,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
 			ret = __bio_iov_iter_get_pages(bio, iter);
 	} while (!ret && iov_iter_count(iter) && !bio_full(bio));
 
-	if (iov_iter_bvec_no_ref(iter))
+	if (is_bvec)
 		bio_set_flag(bio, BIO_NO_PAGE_REF);
-	else if (is_bvec)
-		bio_get_pages(bio);
-
 	return bio->bi_vcnt ? 0 : ret;
 }
 
drivers/block/loop.c +4 -12
index 102d79575895..c20710e617c2 100644
@@ -264,20 +264,12 @@ lo_do_transfer(struct loop_device *lo, int cmd,
 	return ret;
 }
 
-static inline void loop_iov_iter_bvec(struct iov_iter *i,
-		unsigned int direction, const struct bio_vec *bvec,
-		unsigned long nr_segs, size_t count)
-{
-	iov_iter_bvec(i, direction, bvec, nr_segs, count);
-	i->type |= ITER_BVEC_FLAG_NO_REF;
-}
-
 static int lo_write_bvec(struct file *file, struct bio_vec *bvec, loff_t *ppos)
 {
 	struct iov_iter i;
 	ssize_t bw;
 
-	loop_iov_iter_bvec(&i, WRITE, bvec, 1, bvec->bv_len);
+	iov_iter_bvec(&i, WRITE, bvec, 1, bvec->bv_len);
 
 	file_start_write(file);
 	bw = vfs_iter_write(file, &i, ppos, 0);
@@ -355,7 +347,7 @@ static int lo_read_simple(struct loop_device *lo, struct request *rq,
 	ssize_t len;
 
 	rq_for_each_segment(bvec, rq, iter) {
-		loop_iov_iter_bvec(&i, READ, &bvec, 1, bvec.bv_len);
+		iov_iter_bvec(&i, READ, &bvec, 1, bvec.bv_len);
 		len = vfs_iter_read(lo->lo_backing_file, &i, &pos, 0);
 		if (len < 0)
 			return len;
@@ -396,7 +388,7 @@ static int lo_read_transfer(struct loop_device *lo, struct request *rq,
 		b.bv_offset = 0;
 		b.bv_len = bvec.bv_len;
 
-		loop_iov_iter_bvec(&i, READ, &b, 1, b.bv_len);
+		iov_iter_bvec(&i, READ, &b, 1, b.bv_len);
 		len = vfs_iter_read(lo->lo_backing_file, &i, &pos, 0);
 		if (len < 0) {
 			ret = len;
@@ -563,7 +555,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loop_cmd *cmd,
 	}
 	atomic_set(&cmd->ref, 2);
 
-	loop_iov_iter_bvec(&iter, rw, bvec, nr_bvec, blk_rq_bytes(rq));
+	iov_iter_bvec(&iter, rw, bvec, nr_bvec, blk_rq_bytes(rq));
 	iter.iov_offset = offset;
 
 	cmd->iocb.ki_pos = pos;
fs/io_uring.c +0 -3
index f65f85d89217..f7eb63a5b3db 100644
@@ -853,9 +853,6 @@ static int io_import_fixed(struct io_ring_ctx *ctx, int rw,
 	iov_iter_bvec(iter, rw, imu->bvec, imu->nr_bvecs, offset + len);
 	if (offset)
 		iov_iter_advance(iter, offset);
-
-	/* don't drop a reference to these pages */
-	iter->type |= ITER_BVEC_FLAG_NO_REF;
 	return 0;
 }
 
include/linux/uio.h +0 -3
index f184af1999a8..bace8fd40d0c 100644
@@ -23,9 +23,6 @@ struct kvec {
 };
 
 enum iter_type {
-	/* set if ITER_BVEC doesn't hold a bv_page ref */
-	ITER_BVEC_FLAG_NO_REF = 2,
-
 	/* iter types */
 	ITER_IOVEC = 4,
 	ITER_KVEC = 8,
@@ -93,11 +90,6 @@ static inline unsigned char iov_iter_rw(const struct iov_iter *i)
 	return i->type & (READ | WRITE);

include/linux/uio.h static inline unsigned char iov_iter_rw(const struct iov_iter *i) · line 95

Quoted 10 lines
include/linux/uio.h +0 -5
 }
 
-static inline bool iov_iter_bvec_no_ref(const struct iov_iter *i)
-{
-	return (i->type & ITER_BVEC_FLAG_NO_REF) != 0;
-}
-
 /*
  * Total number of bytes covered by an iovec.
  *

I remember that this way is the initial version of Jens' patch, however kernel bug is triggered:

https://lore.kernel.org/linux-block/[email protected]/

Or maybe I miss some recent changes, could you explain it a bit?

Thanks,
Ming

#20 lore raw

Re: [PATCH 8/8] block: never take page references for ITER_BVEC

Quoted 6 lines · Ming Lei

I remember that this way is the initial version of Jens' patch, however kernel bug is triggered:

https://lore.kernel.org/linux-block/[email protected]/

Or maybe I miss some recent changes, could you explain it a bit?

I'm not even sure how the original would crash.. When I run the rest locally it gets and EBUSY from ioctl LOOP_SET_FD, so I'm not sure what it tends up testing in the end.

#21 lore raw

Re: [PATCH 8/8] block: never take page references for ITER_BVEC

Quoted 11 lines · Christoph Hellwig
Quoted 6 lines · Ming Lei

I remember that this way is the initial version of Jens' patch, however kernel bug is triggered:

https://lore.kernel.org/linux-block/[email protected]/

Or maybe I miss some recent changes, could you explain it a bit?

I'm not even sure how the original would crash.. When I run the rest locally it gets and EBUSY from ioctl LOOP_SET_FD, so I'm not sure what it tends up testing in the end.

I also did another run with KASAN enabled and a clean environment, this gives no EBUSY and still works fine. I still don't understand what the original problem might have been here, though.

maintainer
#22 lore raw
In reply to Christoph Hellwig
Quoted 7 lines · Christoph Hellwig

Hi Jens,

this series cleans up the various direct I/O and pass through routines by switching them over to common bio helpers. For the block device simple case this also fixes a page leak if we were using bvec iters.

Quoted 5 lines

The last page just unconditionally applies the no page ref behavior for bvec iters. I looked at all the callers, and there is none that drops the pre-required references before completing the request. Probably not suitable for so late in the merge, but I wanted to get it out.

I'm guessing you'll spin a v2 of this due to the issue that Nikolay pointed out. I'd suggest we then just queue it up for later in the merge window inclusion.

Signature · 1 line
Jens Axboe