sendfile() and UDP socket
Subsystems (3)
PATCH 1/1 sendfile() and UDP socket
Hi All,
Sendfile() over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you run sendfile() with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.
net/ipv4/udp.c | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-)
net/ipv4/udp.c +15 -0
@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) {struct udp_sock *up =udp_sk (sk); +struct inet_sock *inet =inet_sk (sk);int ret; +int fragheaderlen; +struct ip_options *opt =NULL ; + +lock_sock (sk); +if (inet->cork.flags &IPCORK_OPT ) + opt = inet->cork.opt; + fragheaderlen =sizeof (struct iphdr ) + (opt ? opt->optlen :0 ); + +if (inet->cork.length + size >=0xFFFF - fragheaderlen) { + ret =udp_push_pending_frames (sk); +if (ret) + goto out; + } +release_sock (sk);if (!up->pending) {struct msghdr msg = { .msg_flags = flags|MSG_MORE };
Quoted 28 lines · Johann Baudy
Hi All,
Sendfile()over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you runsendfile()with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.Signed-off-by Johann Baudy <[email protected]>net/ipv4/udp.c | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-)net/ipv4/udp.c +8
-0@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) {struct udp_sock *up =udp_sk (sk); +struct inet_sock *inet =inet_sk (sk);int ret; +int fragheaderlen; +struct ip_options *opt =NULL ; + +lock_sock (sk); +if (inet->cork.flags &IPCORK_OPT ) + opt = inet->cork.opt; + fragheaderlen =sizeof (struct iphdr ) + (opt ? opt->optlen :0 );
net/ipv4/udp.c int udp_sendpage(struct sock *sk, struct page *page, int offset, · line 764
Quoted 10 lines
net/ipv4/udp.c +7
-0+ +if (inet->cork.length + size >=0xFFFF - fragheaderlen) { + ret =udp_push_pending_frames (sk); +if (ret) + goto out; + } +release_sock (sk);if (!up->pending) {struct msghdr msg = { .msg_flags = flags|MSG_MORE };
Hi,
I wonder if it is slightly nicer to do without the opt variable. I _think_ its safe to access inet->cork.opt->optlen based on the (inet->cork.flags & IPCORK_OPT) check.
net/ipv4/udp.c +14 -0
@@ -743,7 +743,21 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) {struct udp_sock *up =udp_sk (sk); +struct inet_sock *inet =inet_sk (sk);int ret; +int fragheaderlen; + + fragheaderlen =sizeof (struct iphdr ); +lock_sock (sk); +if (inet->cork.flags &IPCORK_OPT ) + fragheaderlen += inet->cork.opt->optlen; + +if (inet->cork.length + size >=0xFFFF - fragheaderlen) { + ret =udp_push_pending_frames (sk); +if (ret) + goto out; + } +release_sock (sk);if (!up->pending) {struct msghdr msg = { .msg_flags = flags|MSG_MORE };
Signature · 3 lines
Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
Quoted 51 lines · Simon Horman
Quoted 28 lines · Johann Baudy
Hi All,
Sendfile()over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you runsendfile()with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.Signed-off-by Johann Baudy <[email protected]>net/ipv4/udp.c | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-)net/ipv4/udp.c +8
-0@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) {struct udp_sock *up =udp_sk (sk); +struct inet_sock *inet =inet_sk (sk);int ret; +int fragheaderlen; +struct ip_options *opt =NULL ; + +lock_sock (sk); +if (inet->cork.flags &IPCORK_OPT ) + opt = inet->cork.opt; + fragheaderlen =sizeof (struct iphdr ) + (opt ? opt->optlen :0 );net/ipv4/udp.c int udp_sendpage(struct sock *sk, struct page *page, int offset, · line 764
Quoted 10 lines
net/ipv4/udp.c +7
-0+ +if (inet->cork.length + size >=0xFFFF - fragheaderlen) { + ret =udp_push_pending_frames (sk); +if (ret) + goto out; + } +release_sock (sk);if (!up->pending) {struct msghdr msg = { .msg_flags = flags|MSG_MORE };Hi,
I wonder if it is slightly nicer to do without the opt variable. I _think_ its safe to access
inet->cork.opt->optlenbased on the (inet->cork.flags&IPCORK_OPT) check.net/ipv4/udp.c +0
-0
net/ipv4/udp.c int udp_sendpage(struct sock *sk, struct page *page, int offset, · line 748
Quoted 6 lines
net/ipv4/udp.c +2
-0@@ -743,7 +743,21 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) {struct udp_sock *up =udp_sk (sk); +struct inet_sock *inet =inet_sk (sk);int ret; +int fragheaderlen;
Also, I wonder if this should be an unsigned int
as both sizeof(struct iphdr) and inet->cork.opt->optlen are unsigned, though of different width.
Quoted 15 lines
net/ipv4/udp.c +12
-0+ + fragheaderlen =sizeof (struct iphdr ); +lock_sock (sk); +if (inet->cork.flags &IPCORK_OPT ) + fragheaderlen += inet->cork.opt->optlen; + +if (inet->cork.length + size >=0xFFFF - fragheaderlen) { + ret =udp_push_pending_frames (sk); +if (ret) + goto out; + } +release_sock (sk);if (!up->pending) {struct msghdr msg = { .msg_flags = flags|MSG_MORE };
Signature · 3 lines
Simon Horman VA Linux Systems Japan K.K., Sydney, Australia Satellite Office H: www.vergenet.net/~horms/ W: www.valinux.co.jp/en
Hi, Johann,
Quoted 5 lines
Hi All,
Sendfile()over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you runsendfile()with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.
UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.
You can pass "offset" of the file you are going to send and "count" to be sent to sendfile systemcall, so you can split the file into several pieces and send each of them.
If you want send a large file over UDP, the typical code will be like:
while (...) {
setsockopt(fd, UDP_CORK, 1);
sendmsg(fd, &apl_header, sizeof(apl_header));
offset += sendfile(fd, offset, count);
setsockopt(fd, UDP_CORK, 0);
}or:
while (...) {
sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE);
offset += sendfile(fd, offset, count);
}Quoted 38 lines
Signed-off-by Johann Baudy <[email protected]>net/ipv4/udp.c | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-)net/ipv4/udp.c +15
-1@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) {struct udp_sock *up =udp_sk (sk); +struct inet_sock *inet =inet_sk (sk);int ret; +int fragheaderlen; +struct ip_options *opt =NULL ; + +lock_sock (sk); +if (inet->cork.flags &IPCORK_OPT ) + opt = inet->cork.opt; + fragheaderlen =sizeof (struct iphdr ) + (opt ? opt->optlen :0 ); + +if (inet->cork.length + size >=0xFFFF - fragheaderlen) { + ret =udp_push_pending_frames (sk); +if (ret) + goto out; + } +release_sock (sk);if (!up->pending) {struct msghdr msg = { .msg_flags = flags|MSG_MORE }; --To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Le mardi 16 septembre 2008 15:01:17 Hirokazu Takahashi, vous avez écrit :
Quoted 5 lines
UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.
Also, why use UDP for this... If you want stream semantics, why not use TCP or SCTP instead?
Quoted 6 lines
If you want send a large file over UDP, the typical code will be like:
while (...) { sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE); offset += sendfile(fd, offset, count); }
Correct me if I am wrong, but... Unless you have a big MTU (as _not_ in 1500 bytes :D), doing an extra syscall might be slower than copying data in a single vectorized sendmsg() syscall.
Signature · 2 lines
Rémi Denis-Courmont http://www.remlab.net/
Hi,
Quoted 8 lines
Quoted 5 lines
UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.
Also, why use UDP for this... If you want stream semantics, why not use TCP or SCTP instead?
I think a lot of VoIP and video streaming services are working on UDP. Linux NFS over UDP also uses this feature.
Quoted 10 lines
Quoted 6 lines
If you want send a large file over UDP, the typical code will be like:
while (...) { sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE); offset += sendfile(fd, offset, count); }Correct me if I am wrong, but... Unless you have a big MTU (as _not_ in 1500 bytes :D), doing an extra syscall might be slower than copying data in a single vectorized
sendmsg()syscall.
That's not true.
Even if the MTU is small, you can send a UDP message whose size can be up to 64KB at once. They will be split into several IP packets without any copies.
And more, copying data will pollute the cache memory much, which cannot be ignored if you want send tons of data of files.
Thanks,
Hirokazu Takahashi.
Le vendredi 19 septembre 2008 15:28:54 Hirokazu Takahashi, vous avez écrit :
Quoted 10 lines
Quoted 8 lines
Quoted 5 lines
UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.
Also, why use UDP for this... If you want stream semantics, why not use TCP or SCTP instead?
I think a lot of VoIP and video streaming services are working on UDP.
VoIP uses lots of small packets, considering the typical packetization times. Using send(MSG_MORE) + sendfile would definitely be slower than a single sendmsg() in such case, because the per-packet memcpy() will be quite short.
Video streaming typically does send lots of large packets, and might well read the data from a mmap-able file. But video streaming protocols such as RTP typically try to avoid fragmentation, so large sendfile() won't work.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
If you want send a large file over UDP, the typical code will be like:
while (...) { sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE); offset += sendfile(fd, offset, count); }
Quoted 8 lines
Quoted 3 lines
Correct me if I am wrong, but... Unless you have a big MTU (as _not_ in 1500 bytes :D), doing an extra syscall might be slower than copying data in a single vectorized
sendmsg()syscall.That's not true.
Even if the MTU is small, you can send a UDP message whose size can be up to 64KB at once. They will be split into several IP packets without any copies.
And you will encounter fragmentation, which sucks at high data rates. As for low rates, you typically would not bother to optimize memcpy. But more importantly, you should not assume that the other end is only talking to you - it might as well be talking to many different people and not appreciate if all of them send lots of fragments.
And lets not get into how poorly fragmentation works through middleboxes...
Quoted 2 lines
And more, copying data will pollute the cache memory much, which cannot be ignored if you want send tons of data of files.
Signature · 2 lines
Rémi Denis-Courmont http://www.remlab.net/
David Miller
maintainerFrom: Johann Baudy <[email protected]>
Date: Sun, 14 Sep 2008 12:25:56 +0200
Quoted 7 lines
Sendfile()over UDP socket are currently limited to ~ 64KBytes file (max cork.length). Indeed, if you runsendfile()with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.Signed-off-by Johann Baudy <[email protected]>
Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.
If you look, this is what things like NFS using SUNRPC over UDP do. They have a transmission unit for the data transfer and use that for each "send".
A sendfile() with length >= 64K is the same as a sendmsg() with such a length, which is defined as:
if (len > 0xFFFF) return -EMSGSIZE;
So we could technically even return an error for this sendfile() over UDP case.
Quoted 4 lines · David Miller
Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.
But isn't the whole idea of the sendfile() is to send a file no matter what underlying media is?
Quoted 3 lines
If you look, this is what things like NFS using SUNRPC over UDP do. They have a transmission unit for the data transfer and use that for each "send".
That's maybe because udp_sendpage() does not support sending pending data if new packet is too big to attach?
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerFrom: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 04:21:57 +0400
Quoted 8 lines · David Miller
Quoted 4 lines
Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.
But isn't the whole idea of the
sendfile()is to send a file no matter what underlying media is?
It's a way to fabricate a send() directly from the page cache.
Quoted 2 lines
That's maybe because
udp_sendpage()does not support sending pending data if new packet is too big to attach?
I don't think so.
It's simply enforcing the wsize/rsize that's configured for the mount. And this is exactly deciding what the UDP segment size should be.
Quoted 9 lines · David Miller
Quoted 7 lines
Quoted 4 lines
Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.
But isn't the whole idea of the
sendfile()is to send a file no matter what underlying media is?It's a way to fabricate a
send()directly from the page cache.
And to send exactly required number of bytes (or size of the cache)? To send a single page (combined to several other pages) we have simple ->sendpage() callback, which should not return error when it is asked to send a data and it can do it by actually submitting two packets without special tcp-like processing of the segments.
Quoted 7 lines
Quoted 2 lines
That's maybe because
udp_sendpage()does not support sending pending data if new packet is too big to attach?I don't think so.
It's simply enforcing the wsize/rsize that's configured for the mount. And this is exactly deciding what the UDP segment size should be.
And what if it is just a result of the knowledge on how udp_sendpage() behaves, so code adjusts packet sizes in advance?
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerFrom: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 05:08:34 +0400
Quoted 9 lines · David Miller
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines
Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.
But isn't the whole idea of the
sendfile()is to send a file no matter what underlying media is?
Quoted 7 lines
Quoted 1 line
It's a way to fabricate a
send()directly from the page cache.And to send exactly required number of bytes (or size of the cache)? To send a single page (combined to several other pages) we have simple ->sendpage() callback, which should not return error when it is asked to send a data and it can do it by actually submitting two packets without special tcp-like processing of the segments.
You're basically throwing away the difference between datagram and stream socket semantics.
I don't see what else I can explain if you cannot see that this is significant.
Quoted 11 lines · David Miller
Quoted 5 lines
And to send exactly required number of bytes (or size of the cache)? To send a single page (combined to several other pages) we have simple ->sendpage() callback, which should not return error when it is asked to send a data and it can do it by actually submitting two packets without special tcp-like processing of the segments.
You're basically throwing away the difference between datagram and stream socket semantics.
I don't see what else I can explain if you cannot see that this is significant.
Hey David, that's getting the wrong direction :)
Do not make decision backed by what you read or decided to think before that instead of making it clear.
Stream socket means that whatever data we put to it, it will become completely boundary-free, in that regard, that receiving side will not be able to get original sending packet sizes (without too much efforts).
Datagram just preservs the boundaries and that's what we have with this patch. Previously we accumulated a single segment upto predefined size and sent it when it is complete. Now we are able to sent it when it is complete and start creating new segment without returning the error.
Like:
previosly: A + A + A + A + send + return
now: A + A + A + A + send + A + A and so on, so effectively nothing changes except maybe time when segment is being sent: previously it was on the full size and now it is when new packet does not fit that predefined size.
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerFrom: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 08:19:29 +0400
Quoted 3 lines
Stream socket means that whatever data we put to it, it will become completely boundary-free, in that regard, that receiving side will not be able to get original sending packet sizes (without too much efforts).
Right.
Quoted 2 lines
Datagram just preservs the boundaries and that's what we have with this patch.
Not exactly.
Each and every send() operation on a datagram socket must result in exactly one packet. sendfile() was following this rule, when it returned successfully only one single packet was emitted.
The new sendfile() semantics are outside of this model.
Quoted 10 lines · David Miller
Quoted 2 lines
Datagram just preservs the boundaries and that's what we have with this patch.
Not exactly.
Each and every
send()operation on a datagram socket must result in exactly one packet.sendfile()was following this rule, when it returned successfully only one single packet was emitted.The new
sendfile()semantics are outside of this model.
That's for send(), not sendfile(). The latter now works like lots of sends() while previously it worked as single send().
Why sendfile() should be completely different compared to stream case? Obviously send() has to be different and it is, but sendfile() is just a bunch of sends over the pages in the cache, so let's allow it to be multiple sends and not only single packet send.
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerFrom: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 08:40:52 +0400
Quoted 1 line
Why
sendfile()should be completely different compared to stream case?
Because datagram sockets are completely different from stream sockets.
You program them differently, segmentation is made by the socket user not within by the protocol itself.
sendfile() should behave in a way congruent to the other data transfer APIs of the BSD socket layer.
Quoted 7 lines · David Miller
Because datagram sockets are completely different from stream sockets.
You program them differently, segmentation is made by the socket user not within by the protocol itself.
sendfile()should behave in a way congruent to the other data transfer APIs of the BSD socket layer.
So effectively you are saying, that sendfile() is just a pure send(), but with diferent arguments? I.e. it is not supposed to send the whole data it points to, but as much as possible according to send() standard?
Well, it may be a right or wrong decision, and in my opinion sendfile() is very diferent than send() since it should require the whole data to be sent, i.e. being like a loop of send()s, but since sendfile() is effectively a very new approach, it could have different behaviour rules.
Because of this, Johann, this patch will not be applied, but thanks a lot for your work, we made a clear meaning on interface usage.
Signature · 1 line
Evgeniy Polyakov
David Miller
maintainerFrom: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 09:49:20 +0400
Quoted 6 lines · David Miller
Quoted 5 lines
Because datagram sockets are completely different from stream sockets.
You program them differently, segmentation is made by the socket user not within by the protocol itself.
Quoted 6 lines
Quoted 2 lines
sendfile()should behave in a way congruent to the other data transfer APIs of the BSD socket layer.So effectively you are saying, that
sendfile()is just a puresend(), but with diferent arguments? I.e. it is not supposed to send the whole data it points to, but as much as possible according tosend()standard?
It already makes this, guess what happens when socket error occurs midstream during sendfile(), even for TCP?
First, we return length successfully sent.
User has to retry sendfile() call with remaining length, and at this point they will immediately get the socket error return value.
This was broken at one point and I remember applying the fix for this several years ago :-)
Quoted 5 lines
Well, it may be a right or wrong decision, and in my opinion
sendfile()is very diferent thansend()since it should require the whole data to be sent, i.e. being like a loop of send()s, but sincesendfile()is effectively a very new approach, it could have different behaviour rules.
It is in fact exactly and precisely like send(). It must even give the same error return semantics as other socket data transfer calls do. See above.
The user has to have a resending loop _anyways_, in order to do correct error handling.
Nothing is gained from the proposal, really. It can only harm application developers into thinking that segmentation over datagram sockets is not their responsibility, when it absolutely is.
Quoted 7 lines · David Miller
Quoted 3 lines
So effectively you are saying, that
sendfile()is just a puresend(), but with diferent arguments? I.e. it is not supposed to send the whole data it points to, but as much as possible according tosend()standard?It already makes this, guess what happens when socket error occurs midstream during
sendfile(), even for TCP?
Quoted 7 lines
First, we return length successfully sent.
User has to retry
sendfile()call with remaining length, and at this point they will immediately get the socket error return value.This was broken at one point and I remember applying the fix for this several years ago :-)
Well, still tcp errors are completely different from udp ones: the former ony result in a real problem or interrupt/timeout in the simple case. UDP adds 'it is too big to be transferred' one. And instead of just send the previous frame and start new one, we return error so that user could do the same with additional syscall.
Quoted 9 lines
Quoted 5 lines
Well, it may be a right or wrong decision, and in my opinion
sendfile()is very diferent thansend()since it should require the whole data to be sent, i.e. being like a loop of send()s, but sincesendfile()is effectively a very new approach, it could have different behaviour rules.It is in fact exactly and precisely like
send(). It must even give the same error return semantics as other socket data transfer calls do. See above.
That's the main difference on how you and me look at sendfile().
Quoted 6 lines
The user has to have a resending loop _anyways_, in order to do correct error handling.
Nothing is gained from the proposal, really. It can only harm application developers into thinking that segmentation over datagram sockets is not their responsibility, when it absolutely is.
Which basically means that sendfile() for udp is exactly send() of the mapped data. And it is not about segmentation, since packets are correctly segmented with this proposal: packet is rounded to the submitted page. This is about optimization compared to send() of the data. You basically draw a line, that with UDP it is impossible to do.
Signature · 1 line
Evgeniy Polyakov
Quoted 7 lines · Evgeniy Polyakov
Quoted 3 lines · David Miller
It is in fact exactly and precisely like
send(). It must even give the same error return semantics as other socket data transfer calls do. See above.That's the main difference on how you and me look at
sendfile().
I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtQuoted 4 lines · Herbert Xu
I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.
One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.
Signature · 1 line
Evgeniy Polyakov
Quoted 8 lines · Evgeniy Polyakov
Quoted 4 lines · Herbert Xu
I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.
One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.
Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)
So what application needs this?
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtQuoted 2 lines · Herbert Xu
Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)
It still deserves living if it is not highly priority traffic like video dataflow.
Quoted 1 line
So what application needs this?
Johann did not show his exact usage scenario, but from what I got I concluded, that it is kind of a video sensor (or some other data which is allowed to be lost), which has to add a header
to the frame and submit it to the network without any copy because of hardware limitation for the memory bus. Sensor can put data via DMA to the needed location. His first (and parallel to this one) idea was to extend packet socket to allow send of the mapped data.
I proposed to do the same with sendfile, i.e. dma data to the mapped area of some file in the ramdisk, add header, do it for multiple frames and then send given file using sendfile().
Signature · 1 line
Evgeniy Polyakov
Quoted 7 lines · Herbert Xu
Quoted 6 lines · Evgeniy Polyakov
Quoted 4 lines · Herbert Xu
I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.
Quoted 7 lines
Quoted 2 lines
One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.
Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)
So what application needs this?
It seems it might be useful for a video server. The one thing that seems to be missing from the sendfile() semantics is a message size to be used for splitting the file into UDP datagrams, but this could be provided by a separate ioctl(), and could default to the largest message size that would fit in the MTU.
-Bill
Using skb_get() to recycle skbs
In reply to Evgeniy PolyakovIn our netdev driver, we preallocate a pool of skb which are used by our h/w to DMA frames.
Normally, when we indicate a packet to stack (netif_rx), the stack ends up freeing it. If we do a skb_get() prior to that, then the stack just reduces the use_count, and our driver is able to re-use it. We do have to use caution to see that stack has dropped it's reference (atomic_read(skb->users) <= 1) prior to returning the buffer to our h/w.
Is this a valid approach? It so it could save some CPU cycles which are used in alloc_skb(). If this can be done, how come more drivers in tree are not doing it, is there any flip side to it? One could be that stack may not free it in time (we could starve the hardware of free buffers), but in that case we can drop our reference, and let the stack free it up, and force an allocation instead of reuse.
Thanks for your response.
Ram Natarajan
Emulex Corporation
Re: Using skb_get() to recycle skbs
Hi.
Quoted 11 lines · On Thu, Sep 25, 2008 at 06:03:03AM -0700, [email protected]…
In our netdev driver, we preallocate a pool of skb which are used by our h/w to DMA frames.
Normally, when we indicate a packet to stack (
netif_rx), the stack ends up freeing it. If we do askb_get()prior to that, then the stack just reduces theuse_count, and our driver is able to re-use it. We do have to use caution to see that stack has dropped it's reference (atomic_read(skb->users)<= 1) prior to returning the buffer to our h/w.
Quoted 9 lines
Is this a valid approach? It so it could save some CPU cycles which are used in
alloc_skb(). If this can be done, how come more drivers in tree are not doing it, is there any flip side to it? One could be that stack may not free it in time (we could starve the hardware of free buffers), but in that case we can drop our reference, and let the stack free it up, and force an allocation instead of reuse.
Now stack allocates new skb and copy data from old one to the new, since it belives that skb is shared between different users and thus can not be modified in the rx path. You can check how skb_share_check() is called in ip_rcv() and similar receiving functions.
Signature · 1 line
Evgeniy Polyakov
Hi All,
I'm sorry for this late reply but I was out of Internet access during one month. So if my understanding is correct, sendfile() over UDP can't be used to send a file > 64 ko in one system call, as each request must fit in one UDP packet.
However is it expected that sendfile() returns 64Ko but sends nothing over the network when we are performing this particular case (file>64ko)? should it return 0?
Thanks in advance,
Johann
Quoted 28 lines · Bill Fink
Quoted 24 lines · Herbert Xu
Quoted 8 lines · Evgeniy Polyakov
Quoted 6 lines · Herbert Xu
Quoted 4 lines
I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.
Quoted 7 lines
Quoted 2 lines
One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.
Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)
So what application needs this?
It seems it might be useful for a video server. The one thing that seems to be missing from the
sendfile()semantics is a message size to be used for splitting the file into UDP datagrams, but this could be provided by a separateioctl(), and could default to the largest message size that would fit in the MTU.
Quoted 7 lines
Quoted 1 line
-Bill
--
Johann Baudy
[email protected]
--
Johann Baudy
[email protected]