Skip to contentSkip to conversation

: sendfile: remove flags paramter of do_splice_direct()

PATCH In review Changli Gao 3 participants 8 messages May 26, 2010 – May 30, 2010 VFS

lore
Share & tools
lore
#0 lore raw

PATCH v2 3/4 : sendfile: remove flags paramter of do_splice_direct()

remove flags paramter of do_splice_direct().

the flags parameter of do_splice_direct() doesn't mean non-block read of in file, but non-block read/write of pipe. As the reader and writer of process internal pipe is the process itself, we don't need this flags.

----

 fs/read_write.c    |   15 ++-------------
 fs/splice.c        |   19 +++----------------
 include/linux/fs.h |    2 +-
 3 files changed, 6 insertions(+), 30 deletions(-)
fs/read_write.c +2 -13
index 113386d..0688e05 100644
@@ -796,7 +796,7 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
 	struct inode * in_inode, * out_inode;
 	loff_t pos;
 	ssize_t retval;
-	int fput_needed_in, fput_needed_out, fl;
+	int fput_needed_in, fput_needed_out;
 
 	/*
 	 * Get input file, and verify that it is ok..
@@ -846,18 +846,7 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
 		count = max - pos;
 	}
 
-	fl = 0;
-#if 0
-	/*
-	 * We need to debate whether we can enable this or not. The
-	 * man page documents EAGAIN return for the output at least,
-	 * and the application is arguably buggy if it doesn't expect
-	 * EAGAIN on a non-blocking file descriptor.
-	 */
-	if (in_file->f_flags & O_NONBLOCK)
-		fl = SPLICE_F_NONBLOCK;
-#endif
-	retval = do_splice_direct(in_file, ppos, out_file, count, fl);
+	retval = do_splice_direct(in_file, ppos, out_file, count);
 
 	if (retval > 0) {
 		add_rchar(current, retval);
fs/splice.c +3 -16
index eb602cb..57172ca 100644
@@ -1125,7 +1125,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd,
 	long ret, bytes;
 	umode_t i_mode;
 	size_t len;
-	int i, flags;
+	int i;
 
 	/*
 	 * We require the input being a regular file, as we don't want to
@@ -1162,29 +1162,18 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd,
 	ret = 0;
 	bytes = 0;
 	len = sd->total_len;
-	flags = sd->flags;
-
-	/*
-	 * Don't block on output, we have to drain the direct pipe.
-	 */
-	sd->flags &= ~SPLICE_F_NONBLOCK;
 
 	while (len) {
 		size_t read_len;
 		loff_t pos = sd->pos, prev_pos = pos;
 
-		ret = do_splice_to(in, &pos, pipe, len, flags);
+		ret = do_splice_to(in, &pos, pipe, len, 0);
 		if (unlikely(ret <= 0))
 			goto out_release;
 
 		read_len = ret;
 		sd->total_len = read_len;
 
-		/*
-		 * NOTE: nonblocking mode only applies to the input. We
-		 * must not do the output in nonblocking mode as then we
-		 * could get stuck data in the internal pipe:
-		 */
 		ret = actor(pipe, sd);
 		if (unlikely(ret <= 0)) {
 			sd->pos = prev_pos;
@@ -1242,7 +1231,6 @@ static int direct_splice_actor(struct pipe_inode_info *pipe,
  * @ppos:	input file offset
  * @out:	file to splice to
  * @len:	number of bytes to splice
- * @flags:	splice modifier flags
  *
  * Description:
  *    For use by do_sendfile(). splice can easily emulate sendfile, but
@@ -1252,12 +1240,11 @@ static int direct_splice_actor(struct pipe_inode_info *pipe,
  *
  */
 long do_splice_direct(struct file *in, loff_t *ppos, struct file *out,
-		      size_t len, unsigned int flags)
+		      size_t len)
 {
 	struct splice_desc sd = {
 		.len		= len,
 		.total_len	= len,
-		.flags		= flags,
 		.pos		= *ppos,
 		.u.file		= out,
 	};
include/linux/fs.h +1 -1
index 4079ef9..eb86433 100644
@@ -2224,7 +2224,7 @@ extern ssize_t generic_file_splice_write(struct pipe_inode_info *,
 extern ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe,
 		struct file *out, loff_t *, size_t len, unsigned int flags);
 extern long do_splice_direct(struct file *in, loff_t *ppos, struct file *out,
-		size_t len, unsigned int flags);
+		size_t len);
 
 extern void
 file_ra_state_init(struct file_ra_state *ra, struct address_space *mapping);
#1 lore raw
Quoted 4 lines · Changli Gao

remove flags paramter of do_splice_direct().

the flags parameter of do_splice_direct() doesn't mean non-block read of in file,

Actually, it does. Look at the SPLICE_F_NONBLOCK usage in __generic_splice_file_read().

Quoted 126 lines

but non-block read/write of pipe. As the reader and writer of process
internal pipe is the process itself, we don't need this flags.

----

 fs/read_write.c    |   15 ++-------------
 fs/splice.c        |   19 +++----------------
 include/linux/fs.h |    2 +-
 3 files changed, 6 insertions(+), 30 deletions(-)
fs/read_write.c +2 -13
index 113386d..0688e05 100644
@@ -796,7 +796,7 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
 	struct inode * in_inode, * out_inode;
 	loff_t pos;
 	ssize_t retval;
-	int fput_needed_in, fput_needed_out, fl;
+	int fput_needed_in, fput_needed_out;
 
 	/*
 	 * Get input file, and verify that it is ok..
@@ -846,18 +846,7 @@ static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
 		count = max - pos;
 	}
 
-	fl = 0;
-#if 0
-	/*
-	 * We need to debate whether we can enable this or not. The
-	 * man page documents EAGAIN return for the output at least,
-	 * and the application is arguably buggy if it doesn't expect
-	 * EAGAIN on a non-blocking file descriptor.
-	 */
-	if (in_file->f_flags & O_NONBLOCK)
-		fl = SPLICE_F_NONBLOCK;
-#endif
-	retval = do_splice_direct(in_file, ppos, out_file, count, fl);
+	retval = do_splice_direct(in_file, ppos, out_file, count);
 
 	if (retval > 0) {
 		add_rchar(current, retval);
fs/splice.c +3 -16
index eb602cb..57172ca 100644
@@ -1125,7 +1125,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd,
 	long ret, bytes;
 	umode_t i_mode;
 	size_t len;
-	int i, flags;
+	int i;
 
 	/*
 	 * We require the input being a regular file, as we don't want to
@@ -1162,29 +1162,18 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd,
 	ret = 0;
 	bytes = 0;
 	len = sd->total_len;
-	flags = sd->flags;
-
-	/*
-	 * Don't block on output, we have to drain the direct pipe.
-	 */
-	sd->flags &= ~SPLICE_F_NONBLOCK;
 
 	while (len) {
 		size_t read_len;
 		loff_t pos = sd->pos, prev_pos = pos;
 
-		ret = do_splice_to(in, &pos, pipe, len, flags);
+		ret = do_splice_to(in, &pos, pipe, len, 0);
 		if (unlikely(ret <= 0))
 			goto out_release;
 
 		read_len = ret;
 		sd->total_len = read_len;
 
-		/*
-		 * NOTE: nonblocking mode only applies to the input. We
-		 * must not do the output in nonblocking mode as then we
-		 * could get stuck data in the internal pipe:
-		 */
 		ret = actor(pipe, sd);
 		if (unlikely(ret <= 0)) {
 			sd->pos = prev_pos;
@@ -1242,7 +1231,6 @@ static int direct_splice_actor(struct pipe_inode_info *pipe,
  * @ppos:	input file offset
  * @out:	file to splice to
  * @len:	number of bytes to splice
- * @flags:	splice modifier flags
  *
  * Description:
  *    For use by do_sendfile(). splice can easily emulate sendfile, but
@@ -1252,12 +1240,11 @@ static int direct_splice_actor(struct pipe_inode_info *pipe,
  *
  */
 long do_splice_direct(struct file *in, loff_t *ppos, struct file *out,
-		      size_t len, unsigned int flags)
+		      size_t len)
 {
 	struct splice_desc sd = {
 		.len		= len,
 		.total_len	= len,
-		.flags		= flags,
 		.pos		= *ppos,
 		.u.file		= out,
 	};
include/linux/fs.h +1 -2
index 4079ef9..eb86433 100644
@@ -2224,7 +2224,7 @@ extern ssize_t generic_file_splice_write(struct pipe_inode_info *,
 extern ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe,
 		struct file *out, loff_t *, size_t len, unsigned int flags);
 extern long do_splice_direct(struct file *in, loff_t *ppos, struct file *out,
-		size_t len, unsigned int flags);
+		size_t len);
 
 extern void
 file_ra_state_init(struct file_ra_state *ra, struct address_space *mapping);
--

To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
Quoted 9 lines · Miklos Szeredi
Quoted 4 lines · Changli Gao

remove flags paramter of do_splice_direct().

the flags parameter of do_splice_direct() doesn't mean non-block read of in file,

Actually, it does.  Look at the SPLICE_F_NONBLOCK usage in __generic_splice_file_read().

Oh, I checked the code again. You're right. However, why don't we check the flags of in file instead in __generic_file_splice_read()?

Signature · 2 lines
Regards,
Changli Gao([email protected])
#3 lore raw
Quoted 7 lines · Changli Gao
Quoted 6 lines · Miklos Szeredi
Quoted 4 lines · Changli Gao

remove flags paramter of do_splice_direct().

the flags parameter of do_splice_direct() doesn't mean non-block read of in file,

Quoted 7 lines
Quoted 3 lines

Actually, it does.  Look at the SPLICE_F_NONBLOCK usage in __generic_splice_file_read().

Oh, I checked the code again. You're right. However, why don't we check the flags of in file instead in __generic_file_splice_read()?

I am afraid that __generic_file_splice_read() should not use SPLICE_F_NONBLOCK.

Here is the comment from the commit 42324c62704365d6a3e89138dea55909d2f26afe :

<<EOF
Linus introduced SPLICE_F_NONBLOCK in commit
29e350944fdc2dfca102500790d8ad6d6ff4f69d
(splice: add SPLICE_F_NONBLOCK flag )

It doesn't make the splice itself necessarily nonblocking (because the actual file descriptors that are spliced from/to may block unless they have the O_NONBLOCK flag set), but it makes the splice pipe operations nonblocking.

Linus intention was clear : let SPLICE_F_NONBLOCK control the splice pipe mode only
EOF

And I have greped the whole code:

fs/splice.c:            if (spd->flags & SPLICE_F_NONBLOCK) {
fs/splice.c:                    if (flags & SPLICE_F_NONBLOCK) {
fs/splice.c:            if (sd->flags & SPLICE_F_NONBLOCK)
fs/splice.c:                    if (flags & SPLICE_F_NONBLOCK) {
fs/splice.c:            if (flags & SPLICE_F_NONBLOCK) {
fs/splice.c:                    if (flags & SPLICE_F_NONBLOCK) {
fs/splice.c:    if (!ret && ipipe->waiting_writers && (flags &

SPLICE_F_NONBLOCK))
fs/splice.c: * applicable one is SPLICE_F_NONBLOCK. include/linux/splice.h:#define SPLICE_F_NONBLOCK (0x02) /* don't block on the pipe splicing (but */

kernel/relay.c:                 if (flags & SPLICE_F_NONBLOCK)
kernel/trace/trace.c:           if (flags & SPLICE_F_NONBLOCK)

besides the __generic_file_splice_read(), there are two users: relay and trace. I believe they all misuse this flag.

Signature · 6 lines
Regards,
Changli Gao([email protected])
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#4 lore raw

[ Removed [email protected] since I'm getting errors in sending to that address ]

Quoted 34 lines · Changli Gao
Quoted 7 lines · Changli Gao
Quoted 6 lines · Miklos Szeredi
Quoted 4 lines · Changli Gao

remove flags paramter of do_splice_direct().

the flags parameter of do_splice_direct() doesn't mean non-block read of in file,

Quoted 7 lines
Quoted 3 lines

Actually, it does. Look at the SPLICE_F_NONBLOCK usage in __generic_splice_file_read().

Oh, I checked the code again. You're right. However, why don't we check the flags of in file instead in __generic_file_splice_read()?

I am afraid that __generic_file_splice_read() should not use SPLICE_F_NONBLOCK.

Here is the comment from the commit 42324c62704365d6a3e89138dea55909d2f26afe :

<<EOF
Linus introduced SPLICE_F_NONBLOCK in commit
29e350944fdc2dfca102500790d8ad6d6ff4f69d
(splice: add SPLICE_F_NONBLOCK flag )

It doesn't make the splice itself necessarily nonblocking (because the actual file descriptors that are spliced from/to may block unless they have the O_NONBLOCK flag set), but it makes the splice pipe operations nonblocking.

Linus intention was clear : let SPLICE_F_NONBLOCK control the splice pipe mode only
EOF

Quoted 9 lines

And I have greped the whole code:

kernel/trace/trace.c: if (flags & SPLICE_F_NONBLOCK)

besides the __generic_file_splice_read(), there are two users: relay and trace. I believe they all misuse this flag.

The trace splice never blocks anyway. Since we have no good method to wake up a waiter. That is, we never know if it is safe to call wakeup, because a tracepoint may be in a section where the rq lock is held.

The only thing that passing SPLICE_F_NONBLOCK does in that code is to return -EAGAIN when there is nothing in the buffer. Otherwise, it returns 0.

-- Steve

#5 lore raw
Quoted 49 lines · Steven Rostedt

[ Removed [email protected] since I'm getting errors in sending to that address ]

Quoted 34 lines · Changli Gao
Quoted 7 lines · Changli Gao
Quoted 6 lines · Miklos Szeredi
Quoted 4 lines · Changli Gao

remove flags paramter of do_splice_direct().

the flags parameter of do_splice_direct() doesn't mean non-block read of in file,

Quoted 7 lines
Quoted 3 lines

Actually, it does.  Look at the SPLICE_F_NONBLOCK usage in __generic_splice_file_read().

Oh, I checked the code again. You're right. However, why don't we check the flags of in file instead in __generic_file_splice_read()?

I am afraid that __generic_file_splice_read() should not use SPLICE_F_NONBLOCK.

Here is the comment from the commit 42324c62704365d6a3e89138dea55909d2f26afe :

<<EOF
Linus introduced  SPLICE_F_NONBLOCK in commit
29e350944fdc2dfca102500790d8ad6d6ff4f69d
(splice: add SPLICE_F_NONBLOCK flag )

  It doesn't make the splice itself necessarily nonblocking (because the actual file descriptors that are spliced from/to may block unless they have the O_NONBLOCK flag set), but it makes the splice pipe operations nonblocking.

Linus intention was clear : let SPLICE_F_NONBLOCK control the splice pipe mode only
EOF

Quoted 9 lines

And I have greped the whole code:

kernel/trace/trace.c:           if (flags & SPLICE_F_NONBLOCK)

besides the __generic_file_splice_read(), there are two users: relay and trace. I believe they all misuse this flag.

Quoted 8 lines

The trace splice never blocks anyway. Since we have no good method to wake up a waiter. That is, we never know if it is safe to call wakeup, because a tracepoint may be in a section where the rq lock is held.

The only thing that passing SPLICE_F_NONBLOCK does in that code is to return -EAGAIN when there is nothing in the buffer. Otherwise, it returns 0.

But why not use (file->f_flags & O_NONBLOCK) instead, as tracing_splice_read_pipe() does?

Signature · 6 lines
Regards,
Changli Gao([email protected])
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#6 lore raw
Quoted 6 lines · Changli Gao
Quoted 2 lines · Steven Rostedt

[ Removed [email protected] since I'm getting errors in sending to that address ]

But why not use (file->f_flags & O_NONBLOCK) instead, as tracing_splice_read_pipe() does?

Probably because I copied the code from something that used SPLICE_F_NONBLOCK ;-)

But userspace passes that flag too. Although, maybe I just don't understand the API fully.

-- Steve

#7 lore raw
Quoted 3 lines · Steven Rostedt

Probably because I copied the code from something that used SPLICE_F_NONBLOCK ;-)

From the relay API implementation?

Quoted 3 lines

But userspace passes that flag too. Although, maybe I just don't understand the API fully.

this flag is a little confusing. It is much like:

flags = fcntl(pipe_fd, F_GETFL);
fcntl(pipe_fd, F_SETFL, flags | O_NONBLOCK);
splice(..pipe_fd..., 0); // without O_NONBLOCK
fcntl(pipe_fd, F_SETFL, falgs);

In fact, setting O_NONBLOCK is the only way of letting splice pipe operations non-blockable currently. When splicing, splice doesn't refer to the pipe file's f_flags. :(

Signature · 2 lines
Regards,
Changli Gao([email protected])