Skip to contentSkip to conversation

iomap: Submit BIOs at the end of each extent

PATCH In review Matthew Wilcox 2 participants 4 messages Mar 19, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iomap: Submit BIOs at the end of each extent

From: "Matthew Wilcox (Oracle)" <[email protected]>

By definition, an extent covers a range of consecutive blocks, so it would be quite rare to be able to just add pages to the BIO from a previous range. The only case we can think of is a mapped extent followed by a hole extent, followed by another mapped extent which has been allocated immediately after the first extent. We believe this to be an unlikely layout for a filesystem to choose and, since the queue is plugged, those two BIOs would be merged by the block layer.

The reason we care is that ext2/ext4 choose to lay out blocks 0-11 consecutively, followed by the indirect block, and we want to merge those two BIOs. If we don't submit the data BIO before asking the filesystem for the next extent, then the indirect BIO will be submitted first, and waited for, leading to inefficient I/O patterns.

---
 fs/iomap/buffered-io.c | 5 +++++
 1 file changed, 5 insertions(+)
fs/iomap/buffered-io.c +5 -0
index 83438b3257de..8d26920ddf00 100644
@@ -388,6 +388,11 @@ iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length,
 				ctx, iomap, srcmap);
 	}
 
+	if (ctx->bio) {
+		submit_bio(ctx->bio);
+		ctx->bio = NULL;
+	}
+
 	return done;
 }
 
Signature · 1 line
2.25.1
#1 lore raw
Quoted 7 lines · Matthew Wilcox

From: "Matthew Wilcox (Oracle)" <[email protected]>

By definition, an extent covers a range of consecutive blocks, so it would be quite rare to be able to just add pages to the BIO from a previous range. The only case we can think of is a mapped extent followed by a hole extent, followed by another mapped extent which has been allocated immediately after the first extent. We believe this to

Well... userspace can induce that with fallocate(INSERT_RANGE). :)

Quoted 19 lines

be an unlikely layout for a filesystem to choose and, since the queue is plugged, those two BIOs would be merged by the block layer.

The reason we care is that ext2/ext4 choose to lay out blocks 0-11 consecutively, followed by the indirect block, and we want to merge those two BIOs. If we don't submit the data BIO before asking the filesystem for the next extent, then the indirect BIO will be submitted first, and waited for, leading to inefficient I/O patterns.

---
 fs/iomap/buffered-io.c | 5 +++++
 1 file changed, 5 insertions(+)
fs/iomap/buffered-io.c +0 -0
index 83438b3257de..8d26920ddf00 100644

fs/iomap/buffered-io.c iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length, · line 394

Quoted 7 lines
fs/iomap/buffered-io.c +4 -0
@@ -388,6 +388,11 @@ iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length,
 				ctx, iomap, srcmap);
 	}
 
+	if (ctx->bio) {
+		submit_bio(ctx->bio);
+		ctx->bio = NULL;
+	}

Makes sense, but could we have a quick comment here to capture why we're submitting the bio here?

/*
* Submit the bio now so that we neither combine IO requests for
* non-adjacent ranges nor interleave data and metadata requests. */

--D

Quoted 7 lines
fs/iomap/buffered-io.c +1 -0
+
 	return done;
 }
 
Signature · 1 line
2.25.1
#2 lore raw
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Matthew Wilcox

From: "Matthew Wilcox (Oracle)" <[email protected]>

By definition, an extent covers a range of consecutive blocks, so it would be quite rare to be able to just add pages to the BIO from a previous range. The only case we can think of is a mapped extent followed by a hole extent, followed by another mapped extent which has been allocated immediately after the first extent. We believe this to

Well... userspace can induce that with fallocate(INSERT_RANGE). :)

It's not impossible, of course ... just unlikely. Nobody actually uses INSERT_RANGE anyway.

Quoted 30 lines
Quoted 19 lines

be an unlikely layout for a filesystem to choose and, since the queue is plugged, those two BIOs would be merged by the block layer.

The reason we care is that ext2/ext4 choose to lay out blocks 0-11 consecutively, followed by the indirect block, and we want to merge those two BIOs. If we don't submit the data BIO before asking the filesystem for the next extent, then the indirect BIO will be submitted first, and waited for, leading to inefficient I/O patterns.

---
 fs/iomap/buffered-io.c | 5 +++++
 1 file changed, 5 insertions(+)
fs/iomap/buffered-io.c +0 -0
index 83438b3257de..8d26920ddf00 100644

fs/iomap/buffered-io.c iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length, · line 394

Quoted 7 lines
fs/iomap/buffered-io.c +4 -0
@@ -388,6 +388,11 @@ iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length,
 				ctx, iomap, srcmap);
 	}
 
+	if (ctx->bio) {
+		submit_bio(ctx->bio);
+		ctx->bio = NULL;
+	}

Makes sense, but could we have a quick comment here to capture why we're submitting the bio here?

Quoted 4 lines

/*
* Submit the bio now so that we neither combine IO requests for
* non-adjacent ranges nor interleave data and metadata requests. */

How about:

* Submitting the bio here leads to better I/O patterns for
* filesystems which need to do metadata reads to find the
* next range.

I also realised we can add:

@@ -454,8 +459,6 @@ iomap_readpages(struct address_space *mapping, struct list_head *pages,

        }
        ret = 0;
 done:
-       if (ctx.bio)
-               submit_bio(ctx.bio);
        if (ctx.cur_page) {
                if (!ctx.cur_page_in_bio)
                        unlock_page(ctx.cur_page);

since we always subit the bio in readpages_actor.

#3 lore raw
Quoted 50 lines · Matthew Wilcox
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Matthew Wilcox

From: "Matthew Wilcox (Oracle)" <[email protected]>

By definition, an extent covers a range of consecutive blocks, so it would be quite rare to be able to just add pages to the BIO from a previous range. The only case we can think of is a mapped extent followed by a hole extent, followed by another mapped extent which has been allocated immediately after the first extent. We believe this to

Well... userspace can induce that with fallocate(INSERT_RANGE). :)

It's not impossible, of course ... just unlikely. Nobody actually uses INSERT_RANGE anyway.

Quoted 30 lines
Quoted 19 lines

be an unlikely layout for a filesystem to choose and, since the queue is plugged, those two BIOs would be merged by the block layer.

The reason we care is that ext2/ext4 choose to lay out blocks 0-11 consecutively, followed by the indirect block, and we want to merge those two BIOs. If we don't submit the data BIO before asking the filesystem for the next extent, then the indirect BIO will be submitted first, and waited for, leading to inefficient I/O patterns.

---
 fs/iomap/buffered-io.c | 5 +++++
 1 file changed, 5 insertions(+)
fs/iomap/buffered-io.c +0 -0
index 83438b3257de..8d26920ddf00 100644

fs/iomap/buffered-io.c iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length, · line 394

Quoted 7 lines
fs/iomap/buffered-io.c +4 -0
@@ -388,6 +388,11 @@ iomap_readahead_actor(struct inode *inode, loff_t pos, loff_t length,
 				ctx, iomap, srcmap);
 	}
 
+	if (ctx->bio) {
+		submit_bio(ctx->bio);
+		ctx->bio = NULL;
+	}

Makes sense, but could we have a quick comment here to capture why we're submitting the bio here?

Quoted 4 lines

/*
* Submit the bio now so that we neither combine IO requests for
* non-adjacent ranges nor interleave data and metadata requests. */

Quoted 5 lines

How about:

* Submitting the bio here leads to better I/O patterns for
* filesystems which need to do metadata reads to find the
* next range.

Oooh I like your version better.

Quoted 12 lines

I also realised we can add:

(quoted diff) +0 -2
@@ -454,8 +459,6 @@ iomap_readpages(struct address_space *mapping, struct list_head *pages,
        }
        ret = 0;
 done:
-       if (ctx.bio)
-               submit_bio(ctx.bio);
        if (ctx.cur_page) {
                if (!ctx.cur_page_in_bio)
                        unlock_page(ctx.cur_page);
Quoted 1 line

since we always subit the bio in readpages_actor.

<nod> I'll keep an eye out for v2. :)

--D