splice: Fix corruption of spliced data after splice() returns
A message that is not in the archive (sent off-list or not yet received): [email protected]
Alejandro Colomar
maintainerRFC PATCH 1/4 splice: Fix corruption of spliced data after splice() returns
Hi Randy!
Thanks! I'll FWD to the linux-man@ mailing list too.
Cheers,
Alex
-------- Forwarded Message --------
Subject: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns
Date: Wed, 19 Jul 2023 17:36:03 -0700
From: Randy Dunlap <[email protected]>
To: Alejandro Colomar <[email protected]>, Michael Kerrisk <[email protected]>
FYI:
-------- Forwarded Message --------
Subject: Re: [RFC PATCH 1/4] splice: Fix corruption of spliced data after splice() returns
Date: Wed, 19 Jul 2023 17:00:17 -0700
From: Linus Torvalds <[email protected]>
To: Matt Whitlock <[email protected]>
CC: Matthew Wilcox <[email protected]>, Miklos Szeredi <[email protected]>, David Howells <[email protected]>, [email protected], Dave Chinner <[email protected]>, Jens Axboe <[email protected]>, [email protected], [email protected], [email protected], Christoph Hellwig <[email protected]>, [email protected]
Quoted 5 lines · Matt Whitlock
Then that is my request. This entire complaint/discussion/argument would have been avoided if splice(2) had contained a sentence like this one from sendfile(2):
Quoted 7 lines
"If
out_fdrefers to a socket or pipe with zero-copy support, callers must ensure the transferred portions of the file referred to byin_fdremain unmodified until the reader on the other end ofout_fdhas consumed the transferred data."That is a clear warning of the perils of the implementation under the hood, and it could/should be copied, more or less verbatim, to splice(2).
Ack. Internally in the kernel, the two really have always been more or less of intermingled.
In fact, I think splice()/sendfile()/tee() could - and maybe should - actually be a single man-page to make it clear that they are all facets of the same thing.
The issues with TCP_CORK exist for splice too, for example, for exactly the same reasons.
And while SPLICE_F_MORE exists, it only deals with multiple splice() calls, it doesn't deal with the "I wrote a header before I even started using splice()" case that is the one that is mentioned for sendfile().
Or course, technically TCP_CORK exists for plain write() use as well, but there the portable and historical fix is simply to use writev() and send it all in one go.
So it's hopefully only when you use sendfile() and splice() that you end up with "oh, but I have multiple different *kinds* of sources, and I want to cork things until I've dealt with them all".
Linus
Alejandro Colomar
maintainerHi Randy, Linus,
Quoted 11 lines · Linus Torvalds
Oh, and none of this discussion has had any man-page people associated with it.
So in order to maybe get the documentation clarified, let's add them.
Alejandro, Michael, see
https://lore.kernel.org/lkml/[email protected]/
for the original email (and thread).
Linus
Thank you both for pointing to that! I've been checking, and the splice.2 source code already has some hidden comments right after a paragraph.
(see <https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/tree/man2/splice.2#n215>)
I think we can append a paragraph like the suggested one after the one that starts in line 216. Can you clarify if the requirement of taking care of not modifying the buffer applied to just splice(2) or also tee(2) and/or vmsplice(2). BTW, if you want to send a patch, that's very welcome.
Cheers,
Alex
Quoted 62 lines · Alejandro Colomar
Hi Randy!
Thanks! I'll FWD to the linux-man@ mailing list too.
Cheers,
Alex-------- Forwarded Message --------
Subject: Fwd: [RFC PATCH 1/4] splice: Fix corruption of spliced data aftersplice()returns
Date: Wed, 19 Jul 2023 17:36:03 -0700
From: Randy Dunlap <[email protected]>
To: Alejandro Colomar <[email protected]>, Michael Kerrisk <[email protected]>FYI:
-------- Forwarded Message --------
Subject: Re: [RFC PATCH 1/4] splice: Fix corruption of spliced data aftersplice()returns
Date: Wed, 19 Jul 2023 17:00:17 -0700
From: Linus Torvalds <[email protected]>
To: Matt Whitlock <[email protected]>
CC: Matthew Wilcox <[email protected]>, Miklos Szeredi <[email protected]>, David Howells <[email protected]>, [email protected], Dave Chinner <[email protected]>, Jens Axboe <[email protected]>, [email protected], [email protected], [email protected], Christoph Hellwig <[email protected]>, [email protected]Quoted 5 lines · Matt Whitlock
Then that is my request. This entire complaint/discussion/argument would have been avoided if splice(2) had contained a sentence like this one from sendfile(2):
Quoted 7 lines
"If
out_fdrefers to a socket or pipe with zero-copy support, callers must ensure the transferred portions of the file referred to byin_fdremain unmodified until the reader on the other end ofout_fdhas consumed the transferred data."That is a clear warning of the perils of the implementation under the hood, and it could/should be copied, more or less verbatim, to splice(2).
Ack. Internally in the kernel, the two really have always been more or less of intermingled.
In fact, I think splice()/sendfile()/tee() could - and maybe should - actually be a single man-page to make it clear that they are all facets of the same thing.
The issues with
TCP_CORKexist for splice too, for example, for exactly the same reasons.And while
SPLICE_F_MOREexists, it only deals with multiplesplice()calls, it doesn't deal with the "I wrote a header before I even started usingsplice()" case that is the one that is mentioned forsendfile().Or course, technically
TCP_CORKexists for plainwrite()use as well, but there the portable and historical fix is simply to usewritev()and send it all in one go.So it's hopefully only when you use
sendfile()andsplice()that you end up with "oh, but I have multiple different *kinds* of sources, and I want to cork things until I've dealt with them all".Linus
Signature · 2 lines
<http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5