Skip to contentSkip to conversation

fs: allow short direct-io reads to be completed via buffered IO

PATCH In review Josef Bacik 2 participants 4 messages May 3, 2010 – May 4, 2010

Subsystems (4)
Memory management VFS Btrfs file system Page cache
lore
Share & tools
lore

Josef Bacik

author
#0 lore raw

PATCH 1/3 fs: allow short direct-io reads to be completed via buffered IO

This is similar to what already happens in the write case. If we have a short read while doing O_DIRECT, instead of just returning, fallthrough and try to read the rest via buffered IO. BTRFS needs this because if we encounter a compressed or inline extent during DIO, we need to fallback on buffered. If the extent is compressed we need to read the entire thing into memory and de-compress it into the users pages. I have tested this with fsx and everything works great. Thanks,

Signed-off-by Josef Bacik <[email protected]>
---
 mm/filemap.c |   23 ++++++++++++++++++-----
 1 files changed, 18 insertions(+), 5 deletions(-)
mm/filemap.c +18 -5
index 140ebda..423b439 100644
@@ -1263,7 +1263,7 @@ generic_file_aio_read(struct kiocb *iocb, const struct iovec *iov,
 {
 	struct file *filp = iocb->ki_filp;
 	ssize_t retval;
-	unsigned long seg;
+	unsigned long seg = 0;
 	size_t count;
 	loff_t *ppos = &iocb->ki_pos;
 

@@ -1290,21 +1290,34 @@ generic_file_aio_read(struct kiocb *iocb, const struct iovec *iov,
 				retval = mapping->a_ops->direct_IO(READ, iocb,
 							iov, pos, nr_segs);
 			}
-			if (retval > 0)
+			if (retval > 0) {
 				*ppos = pos + retval;
-			if (retval) {
+				count -= retval;
+			}
+			if (retval < 0 || !count) {
 				file_accessed(filp);
 				goto out;
 			}
 		}
 	}
 
+	count = retval;
 	for (seg = 0; seg < nr_segs; seg++) {
 		read_descriptor_t desc;
+		loff_t offset = 0;
+
+		if (count) {
+			if (count > iov[seg].iov_len) {
+				count -= iov[seg].iov_len;
+				continue;
+			}
+			offset = count;
+			count = 0;
+		}
 
 		desc.written = 0;
-		desc.arg.buf = iov[seg].iov_base;
-		desc.count = iov[seg].iov_len;
+		desc.arg.buf = iov[seg].iov_base + offset;
+		desc.count = iov[seg].iov_len - offset;
 		if (desc.count == 0)
 			continue;
 		desc.error = 0;
Signature · 1 line
1.6.6.1
#1 lore raw
Quoted 7 lines · Josef Bacik

This is similar to what already happens in the write case. If we have a short read while doing O_DIRECT, instead of just returning, fallthrough and try to read the rest via buffered IO. BTRFS needs this because if we encounter a compressed or inline extent during DIO, we need to fallback on buffered. If the extent is compressed we need to read the entire thing into memory and de-compress it into the users pages. I have tested this with fsx and everything works great. Thanks,

Won't this mean that any direct IO read that spans EOF (i.e. get a short read) now attempt a buffered IO (that will fail) before returning?

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
author
#2 lore raw
Quoted 9 lines · Dave Chinner
Quoted 7 lines · Josef Bacik

This is similar to what already happens in the write case. If we have a short read while doing O_DIRECT, instead of just returning, fallthrough and try to read the rest via buffered IO. BTRFS needs this because if we encounter a compressed or inline extent during DIO, we need to fallback on buffered. If the extent is compressed we need to read the entire thing into memory and de-compress it into the users pages. I have tested this with fsx and everything works great. Thanks,

Quoted 3 lines

Won't this mean that any direct IO read that spans EOF (i.e. get a short read) now attempt a buffered IO (that will fail) before returning?

Hmm yeah you are right. What would be an acceptable way to avoid this, do a

if (retval || !count || ppos >= i_size_read(inode))

	goto out;

type thing? Thanks,

Josef

#3 lore raw
Quoted 14 lines · Josef Bacik
Quoted 9 lines · Dave Chinner
Quoted 7 lines · Josef Bacik

This is similar to what already happens in the write case. If we have a short read while doing O_DIRECT, instead of just returning, fallthrough and try to read the rest via buffered IO. BTRFS needs this because if we encounter a compressed or inline extent during DIO, we need to fallback on buffered. If the extent is compressed we need to read the entire thing into memory and de-compress it into the users pages. I have tested this with fsx and everything works great. Thanks,

Quoted 3 lines

Won't this mean that any direct IO read that spans EOF (i.e. get a short read) now attempt a buffered IO (that will fail) before returning?

Quoted 6 lines

Hmm yeah you are right. What would be an acceptable way to avoid this, do a

if (retval || !count || ppos >= i_size_read(inode))

	goto out;

type thing? Thanks,

Yes, that looks like it would work to me. Might be worth a comment, though.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]