Skip to contentSkip to conversation

generic_file_sendpage

PATCH In review Jan Blunck 6 participants 11 messages Jul 15, 2005 – Jul 21, 2005

Subsystems (4)
lore
Share & tools
lore

Jan Blunck

author
#0 lore raw

PATCH 1/1 generic_file_sendpage

This is a generic sendpage() for regular files.

With generic_file_sendpage() it is possible to use sendfile() on file targets, instead of only sending data to sockets.

This implementation is basically an extension of Joern's original patch (http://marc.theaimsgroup.com/?l=linux-kernel&m=109455958522766&w=2) but is honoring signals. I also removed some unnecessary code: no IOVs, no AIO.

qemu-debian:/home/root/sendfile_file# time ./fastcp 100mb test
real    0m11.037s
user    0m0.010s
sys     0m7.600s
qemu-debian:/home/root/sendfile_file# time cp 100mb test
real    0m13.342s
user    0m0.400s
sys     0m9.080s

Comments please,
Jan

 fs/ext2/file.c     |    1 
 fs/ext3/file.c     |    1 
 include/linux/fs.h |    2 
 mm/filemap.c       |  170 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 174 insertions(+)
include/linux/fs.h +2 -0
Index: linux-2.6/include/linux/fs.h
===================================================================
@@ -1502,6 +1502,8 @@ extern ssize_t do_sync_write(struct file
 ssize_t generic_file_write_nolock(struct file *file, const struct iovec *iov,
 				unsigned long nr_segs, loff_t *ppos);
 extern ssize_t generic_file_sendfile(struct file *, loff_t *, size_t, read_actor_t, void *);
+extern ssize_t generic_file_sendpage(struct file *, struct page *, int, size_t,
+				     loff_t *, int);
 extern void do_generic_mapping_read(struct address_space *mapping,
 				    struct file_ra_state *, struct file *,
 				    loff_t *, read_descriptor_t *, read_actor_t);
mm/filemap.c +170 -0

188 lines not on this page · raw message

fs/ext2/file.c +1 -0
Index: linux-2.6/fs/ext2/file.c
===================================================================
@@ -53,6 +53,7 @@ struct file_operations ext2_file_operati
 	.readv		= generic_file_readv,
 	.writev		= generic_file_writev,
 	.sendfile	= generic_file_sendfile,
+	.sendpage	= generic_file_sendpage,
 };
 
 #ifdef CONFIG_EXT2_FS_XIP
fs/ext3/file.c +1 -0
Index: linux-2.6/fs/ext3/file.c
===================================================================
@@ -119,6 +119,7 @@ struct file_operations ext3_file_operati
 	.release	= ext3_release_file,
 	.fsync		= ext3_sync_file,
 	.sendfile	= generic_file_sendfile,
+	.sendpage	= generic_file_sendpage,
 };
 
 struct inode_operations ext3_file_inode_operations = {
#1 lore raw

mm/filemap.c int remove_suid(struct dentry *dentry) · line 1747

Quoted 13 lines · Jan Blunck

This is a generic sendpage() for regular files.

mm/filemap.c +9 -0
+static inline size_t
+filemap_copy_from_kernel(struct page *page, unsigned long offset,
+			 const char *buf, unsigned bytes)
+{
+	char *kaddr;
+
+	kaddr = kmap(page);
+	memcpy(kaddr + offset, buf, bytes);
+	kunmap(page);

Use kmap_atomic().

Move the flush_dcache_page() into here.

Quoted 9 lines
mm/filemap.c +9 -0
+static ssize_t
+__generic_kernel_file_write(struct file *file, const char *buf,
+			    size_t count, loff_t *ppos)
+{
+	struct address_space * mapping = file->f_mapping;
+	struct address_space_operations *a_ops = mapping->a_ops;
+	struct inode 	*inode = mapping->host;
+	long		status = 0;
+	loff_t		pos;

mm/filemap.c generic_file_direct_write(struct kiocb * · line 1928

Quoted 10 lines
mm/filemap.c +10 -0
+	struct page	*page;
+	struct page	*cached_page = NULL;
+	const int	isblk = S_ISBLK(inode->i_mode);
+	ssize_t		written;
+	ssize_t		err;
+	size_t		bytes;
+	struct pagevec	lru_pvec;
+
+	/* There is no sane reason to use O_DIRECT */
+	BUG_ON(file->f_flags & O_DIRECT);

err, this seems like an easy way for people to make the kernel go BUG.

mm/filemap.c generic_file_direct_write(struct kiocb * · line 1931

Quoted 2 lines
mm/filemap.c +2 -0
+	if (unlikely(signal_pending(current)))
+		return -EINTR;

This doesn't help. The reason we've avoided file-to-file sendfile() is that it can cause applications to get uninterruptibly stuck in the kernel for ages. This code doesn't solve that problem. It needs to handle signal_pending() inside the main loop.

And it probably needs to return a sane value (number of bytes copied) rather than -EINTR.

I don't know if we want to add this feature, really. It's such a specialised thing.

#2 lore raw
Quoted 5 lines · Andrew Morton

mm/filemap.c generic_file_direct_write(struct kiocb * · line 1928

Quoted 3 lines
mm/filemap.c +3 -0
+
+	/* There is no sane reason to use O_DIRECT */
+	BUG_ON(file->f_flags & O_DIRECT);

err, this seems like an easy way for people to make the kernel go BUG.

Is there a sane use for O_DIRECT in combination with sendfile()?

If not, I'd like to change sys_sendfile() and return -EINVAL for O_DIRECT file descriptors.

Quoted 10 lines

mm/filemap.c generic_file_direct_write(struct kiocb * · line 1931

Quoted 2 lines
mm/filemap.c +2 -0
+	if (unlikely(signal_pending(current)))
+		return -EINTR;

This doesn't help. The reason we've avoided file-to-file sendfile() is that it can cause applications to get uninterruptibly stuck in the kernel for ages. This code doesn't solve that problem. It needs to handle signal_pending() inside the main loop.

And it probably needs to return a sane value (number of bytes copied) rather than -EINTR.

Makes sense.

Quoted 2 lines

I don't know if we want to add this feature, really. It's such a specialised thing.

With union mount and cowlink, there are two users already. cp(1) could use it as well, even if the improvement is quite minimal.

Jörn

Signature · 2 lines
All art is but imitation of nature.
-- Lucius Annaeus Seneca
#3 lore raw
Quoted 5 lines
Quoted 2 lines

I don't know if we want to add this feature, really. It's such a specialised thing.

With union mount and cowlink, there are two users already. cp(1) could use it as well, even if the improvement is quite minimal.

FTP PUT could use this too - currently, only FTPGETs can use sendfile because the target had to be a socket.
(With FTP PUT, the src is a sock, dst is a filedescriptor.)

Jan Engelhardt

#4 lore raw
Quoted 10 lines · Jan Engelhardt
Quoted 6 lines
Quoted 3 lines

I don't know if we want to add this feature, really. It's such a specialised thing.

With union mount and cowlink, there are two users already. cp(1) could use it as well, even if the improvement is quite minimal.

FTP PUT could use this too - currently, only FTPGETs can use sendfile because the target had to be a socket.
(With FTP PUT, the src is a sock, dst is a filedescriptor.)

No, FTP PUT _cannot_ use it, exactly because sendfile() can't do anything but file sources (and not even all file sources - it can only do regular filesystems that use the page cache).

This is why I want to get rid of sendfile(). It's a fundamentally broken interface. Really. In contrast, the pipe buffers _can_ be used for direct socket->file interfaces.

Now, even pipe buffers obviously won't actually be really "zero-copy":
you'll end up needing one copy to align the result, since the incoming network packets will obviously not be nice page-sized and page-aligned things. But they won't need the "copy to user space" and "copy back from user space into kernel space", so it will be a question of _minimal_ copy (and avoiding unnecessary user space VM work).

Linus

#5 lore raw
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

With union mount and cowlink, there are two users already. cp(1) could use it as well, even if the improvement is quite minimal.

FTP PUT could use this too - ...

Quoted 7 lines

No, FTP PUT _cannot_ use it, exactly because sendfile() can't do anything but file sources (and not even all file sources - it can only do regular filesystems that use the page cache).

This is why I want to get rid of sendfile(). It's a fundamentally broken interface. Really. In contrast, the pipe buffers _can_ be used for direct socket->file interfaces.

How will userspace access these pipe buffers?

Jan Engelhardt

#6 lore raw
Quoted 6 lines · Jan Engelhardt
Quoted 4 lines

This is why I want to get rid of sendfile(). It's a fundamentally broken interface. Really. In contrast, the pipe buffers _can_ be used for direct socket->file interfaces.

How will userspace access these pipe buffers?

You can fill them from a user-space buffer with "write()", and you can read them into a user-space buffer with "read()".

It's really a pipe.

The idea of the pipe buffers is that you can _also_ send them to other file descriptors, or fill them from other file descriptors, without having to copy the data to/from user space.

Linus

---
See the example patch that I posted months ago (and that I referred to in my other email, here is that thing repeated in case you missed it):

Quoted 11 lines

For anybody interested in zero-copy work, here's a LWN write-up of some of the original discussion:

        http://lwn.net/Articles/118750/

and my (very ugly) example patch can be found for example here:

        http://groups-beta.google.com/group/linux.kernel/msg/782bd9e5cb647207?hl=en&

(it's not a a complete implementation, but it shows how to go from a file _to_ a pipe buffer, but not back to a file again).

#7 lore raw
In reply to Jörn Engel
Quoted 23 lines · Jörn Engel
Quoted 5 lines · Andrew Morton

mm/filemap.c generic_file_direct_write(struct kiocb * · line 1928

Quoted 3 lines
mm/filemap.c +3 -0
+
+	/* There is no sane reason to use O_DIRECT */
+	BUG_ON(file->f_flags & O_DIRECT);

err, this seems like an easy way for people to make the kernel go BUG.

Is there a sane use for O_DIRECT in combination with sendfile()?

If not, I'd like to change sys_sendfile() and return -EINVAL for O_DIRECT file descriptors.

Quoted 10 lines

mm/filemap.c generic_file_direct_write(struct kiocb * · line 1931

Quoted 2 lines
mm/filemap.c +2 -0
+	if (unlikely(signal_pending(current)))
+		return -EINTR;

This doesn't help. The reason we've avoided file-to-file sendfile() is that it can cause applications to get uninterruptibly stuck in the kernel for ages. This code doesn't solve that problem. It needs to handle signal_pending() inside the main loop.

And it probably needs to return a sane value (number of bytes copied) rather than -EINTR.

Quoted 7 lines

Makes sense.

Quoted 2 lines

I don't know if we want to add this feature, really. It's such a specialised thing.

With union mount and cowlink, there are two users already. cp(1) could use it as well, even if the improvement is quite minimal.

you might soon add linux-vserver to the list, as we will be using this for a special version of the COW links (to break unified links on write) ...

best,
Herbert

Quoted 10 lines

Jörn

Signature · 7 lines
All art is but imitation of nature.
-- Lucius Annaeus Seneca
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
author
#8 lore raw

Re: [PATCH] generic_file_sendpage (updated patch)

In reply to Andrew Morton
Quoted 14 lines · Andrew Morton

Review

Quoted 10 lines
(quoted diff) +9 -0
+static inline size_t
+filemap_copy_from_kernel(struct page *page, unsigned long offset,
+			 const char *buf, unsigned bytes)
+{
+	char *kaddr;
+
+	kaddr = kmap(page);
+	memcpy(kaddr + offset, buf, bytes);
+	kunmap(page);

Use kmap_atomic().

Quoted 2 lines

Move the flush_dcache_page() into here.

Ok

Quoted 7 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+
+	/* There is no sane reason to use O_DIRECT */
+	BUG_ON(file->f_flags & O_DIRECT);

err, this seems like an easy way for people to make the kernel go BUG.

Hmm, yeah ... -EINVAL should do it.

Quoted 9 lines

Review

Quoted 3 lines
(quoted diff) +2 -0
+	if (unlikely(signal_pending(current)))
+		return -EINTR;

This doesn't help. The reason we've avoided file-to-file sendfile() is that it can cause applications to get uninterruptibly stuck in the kernel for ages. This code doesn't solve that problem. It needs to handle signal_pending() inside the main loop.

The sendfile() mainloop is in generic_file_sendfile(). generic_file_sendpage() is called via the file_send_actor() from do_generic_file_read(). Therefore generic_file_sendpage() is called for every page which is written to the file. Since signals are checked before every page this is solving the uninterruptible problem. IMHO, the sendpage() should not be interrupted in between a single page.

Quoted 3 lines

And it probably needs to return a sane value (number of bytes copied) rather than -EINTR.

The return value of sendfile() is not the same as the return value of sendpage(). This is done in file_send_actor().

Quoted 3 lines

I don't know if we want to add this feature, really. It's such a specialised thing.

This issue comes up every now and than. I know of at least two users: COW links and my union mounts. I think this solution is quite good. I tested it under qemu and it is working for me.

Jan

BTW: I'm out for the weekend now.

 fs/ext2/file.c     |    1 
 fs/ext3/file.c     |    1 
 include/linux/fs.h |    2 
 mm/filemap.c       |  172 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 176 insertions(+)
include/linux/fs.h +2 -0
Index: linux-2.6/include/linux/fs.h
===================================================================
@@ -1502,6 +1502,8 @@ extern ssize_t do_sync_write(struct file
 ssize_t generic_file_write_nolock(struct file *file, const struct iovec *iov,
 				unsigned long nr_segs, loff_t *ppos);
 extern ssize_t generic_file_sendfile(struct file *, loff_t *, size_t, read_actor_t, void *);
+extern ssize_t generic_file_sendpage(struct file *, struct page *, int, size_t,
+				     loff_t *, int);
 extern void do_generic_mapping_read(struct address_space *mapping,
 				    struct file_ra_state *, struct file *,
 				    loff_t *, read_descriptor_t *, read_actor_t);
mm/filemap.c +172 -0

190 lines not on this page · raw message

fs/ext2/file.c +1 -0
Index: linux-2.6/fs/ext2/file.c
===================================================================
@@ -53,6 +53,7 @@ struct file_operations ext2_file_operati
 	.readv		= generic_file_readv,
 	.writev		= generic_file_writev,
 	.sendfile	= generic_file_sendfile,
+	.sendpage	= generic_file_sendpage,
 };
 
 #ifdef CONFIG_EXT2_FS_XIP
fs/ext3/file.c +1 -0
Index: linux-2.6/fs/ext3/file.c
===================================================================
@@ -119,6 +119,7 @@ struct file_operations ext3_file_operati
 	.release	= ext3_release_file,
 	.fsync		= ext3_sync_file,
 	.sendfile	= generic_file_sendfile,
+	.sendpage	= generic_file_sendpage,
 };
 
 struct inode_operations ext3_file_inode_operations = {
#9 lore raw
In reply to Andrew Morton
Quoted 3 lines · Andrew Morton

I don't know if we want to add this feature, really. It's such a specialised thing.

It is, in this format, and I agree - I don't want to add it.

What I'd really like to see is somebody taking a look at my old "pipe as a zero-copy buffer" patches, which as an interface allowed arbitrary data to be copied between _any_ file descriptors, and allowed you to do things like mix input sources quite naturally (ie you could write a header first from a user-space buffer, then the contents of a file, and then push the result out to a socket, all with zero-copy).

"sendfile()" in general I think has been a mistake. It's too specialized, and the interface has always sucked. As Andrew pointed out, it actually needs to limit the number of buffers in flight partly because otherwise you have uninterruptible kernel work etc etc.

But more importantly, sendfile() was always broken as a interface for receiving data from anything but a page-cache based filesystem. That means that it's totally useless for a lot of things. The pipe-buffer thing ends up being a totally generic "in-kernel buffer" interface, and is a _lot_ more flexible.

For anybody interested in zero-copy work, here's a LWN write-up of some of the original discussion:

	http://lwn.net/Articles/118750/

and my (very ugly) example patch can be found for example here:

	http://groups-beta.google.com/group/linux.kernel/msg/782bd9e5cb647207?hl=en&

(it's not a a complete implementation, but it shows how to go from a file _to_ a pipe buffer, but not back to a file again).

I really want to get _rid_ of sendfile, not make any more of it.

Linus

author
#10 lore raw
Quoted 3 lines · Linus Torvalds

"sendfile()" in general I think has been a mistake. It's too specialized, and the interface has always sucked.

Ok, you're right. I will have a look at the pipe buffer stuff.

Quoted 2 lines

As Andrew pointed out, it actually
needs to limit the number of buffers in flight partly because otherwise you have uninterruptible kernel work etc etc.

Hmm, sendfile() uses do_generic_mapping_read() which is calling the file_send_actor() per page. The actor calls ->sendpage(). I don't see any situation how this could lead to uninterruptible kernel work when the sendpage() itself is checking for signals. There are only 2 pages in flight in this case.

Jan