Skip to contentSkip to conversation

splice support #2

PATCH [email protected] 5 participants 8 messages Mar 31, 2006 – Apr 4, 2006

lore
Share & tools
lore
#0 lore raw

Linus and Jens,

Stephen Rothwell just pointed me at the new splice() interface. Looks really nice!

One comment though. Could we add a off_t to splice to make it more like pwrite() ? Otherwise threads could get painful with race conditions between the seek and the splice() call.

Either that or add psplice() like this:

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

where ofs sets the offset to read from fdin.

Cheers, Tridge

#1 lore raw
Quoted 9 lines · On 3/31/06, [email protected] <[email protected]> wrote:

Linus and Jens,

Stephen Rothwell just pointed me at the new splice() interface. Looks really nice!

One comment though. Could we add a off_t to splice to make it more like pwrite() ? Otherwise threads could get painful with race conditions between the seek and the splice() call.

Quoted 7 lines

Either that or add psplice() like this:

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

where ofs sets the offset to read from fdin.

Cheers, Tridge

Maybe "ssize_t psplice(int fdin, int fdout, size_t len, off_t fdin_ofs, unsigned flags);"
would be better at documenting that the offset is for the input? Or maybe adding also a fdout_ofs for when fdout is also a file...

--
Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/
[email protected]
[email protected]

Every day, every year
you have to work
you have to study
you have to scene.

#2 lore raw

Antonio,

Quoted 2 lines

Maybe "ssize_t psplice(int fdin, int fdout, size_t len, off_t fdin_ofs, unsigned flags);"

yep, thats clearer

I think ideally splice() would just be changed to always take a off_t as the interface is so new hopefully no apps depend on it yet. Then we don't need psplice().

A while ago we got bitten with tdb in Samba and systems that only have write(), and no pwrite(). If a process forks then accesses the db in both parent and child the shared seek pointer semantics of inherited file descriptors causes races with seek/write. It would be nice if splice() was immune from this from the start.

If people want it both ways, then an alternative would be to have a SPLICE_FLAG_OFS_CURRENT flag to splice(), which would tell splice() to ignore the off_t and use the current seek ptr. I guess that would save some book keeping for some apps that know they don't have any seek races and want to always write at the current position. I wouldn't need this, but some people might find it convenient.

Quoted 2 lines

would be better at documenting that the offset is for the input? Or maybe adding also a fdout_ofs for when fdout is also a file...

I thought about that, but Stephen convinced me it was a bad idea, as the semantics with whats in the pipe buffer at any one time gets too messy if the output is a socket. I guess you could return -EINVAL if you used the fdout_ofs and its not a seekable file, but that seems to be overloading the system call too much for my liking.

Cheers, Tridge

#3 lore raw
In reply to [email protected]
Quoted 9 lines · On Fri, Mar 31 2006, [email protected] wrote:

Linus and Jens,

Stephen Rothwell just pointed me at the new splice() interface. Looks really nice!

One comment though. Could we add a off_t to splice to make it more like pwrite() ? Otherwise threads could get painful with race conditions between the seek and the splice() call.

Quoted 5 lines

Either that or add psplice() like this:

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

where ofs sets the offset to read from fdin.

I definitely see some valid reasons for adding a file offset instead of using ->f_pos, I'll leave that decision up to Linus though. Linus?

Signature · 1 line
Jens Axboe
#4 lore raw
Quoted 5 lines · Jens Axboe
Quoted 2 lines

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

I definitely see some valid reasons for adding a file offset instead of using ->f_pos, I'll leave that decision up to Linus though. Linus?

I think a file offset is fine, the one thing holding me back was just the interface. One file offset per fd? Or just have the rule that the file offset is for the "non-pipe" device?

Linus

#5 lore raw
Quoted 4 lines · Linus Torvalds
Quoted 1 line · Jens Axboe
Quoted 1 line
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

I definitely see some valid reasons for adding a file offset instead of using ->f_pos, I'll leave that decision up to Linus though. Linus?

I think a file offset is fine, the one thing holding me back was just the interface. One file offset per fd? Or just have the rule that the file offset is for the "non-pipe" device?

Intuitively, I'd expect the offset to be tied to the non-pipe if I were to eg see this for the first time. So my vote would go for that.

Signature · 1 line
Jens Axboe
#6 lore raw
Quoted 7 lines · Jens Axboe
Quoted 5 lines · Linus Torvalds
Quoted 2 lines · Jens Axboe
Quoted 1 line

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

I definitely see some valid reasons for adding a file offset instead of using ->f_pos, I'll leave that decision up to Linus though. Linus?

I think a file offset is fine, the one thing holding me back was just the interface. One file offset per fd? Or just have the rule that the file offset is for the "non-pipe" device?

Intuitively, I'd expect the offset to be tied to the non-pipe if I were to eg see this for the first time. So my vote would go for that.

Eee! That means that splice(file_fd, pipe_fd, 1000) and splice(pipe_fd, file_fd, 1000) have different semantics. It also would seem to prevent ever implementing direct file-to-file splicing.

--Andy

#7 lore raw
Quoted 17 lines · Andy Lutomirski
Quoted 16 lines · Jens Axboe
Quoted 12 lines · Linus Torvalds
Quoted 5 lines · Jens Axboe
Quoted 2 lines

ssize_t psplice(int fdin, int fdout, size_t len, off_t ofs, unsigned flags);

I definitely see some valid reasons for adding a file offset instead of using ->f_pos, I'll leave that decision up to Linus though. Linus?

I think a file offset is fine, the one thing holding me back was just the interface. One file offset per fd? Or just have the rule that the file offset is for the "non-pipe" device?

Quoted 7 lines
Quoted 3 lines

Intuitively, I'd expect the offset to be tied to the non-pipe if I were to eg see this for the first time. So my vote would go for that.

Eee! That means that splice(file_fd, pipe_fd, 1000) and splice(pipe_fd, file_fd, 1000) have different semantics. It also would seem to prevent ever implementing direct file-to-file splicing.

The semantics as written: (fdin, fdout, len). But yes the offset gets ugly, unless you add two offsets. And you are right it would not go well with file -> file splicing, which btw is already implemented (just not in 2.6.17-rc1) along with file -> socket.

Signature · 1 line
Jens Axboe