Skip to contentSkip to conversation

sendfile() and UDP socket

PATCH In review Johann Baudy 9 participants 27 messages Sep 14, 2008 – Oct 14, 2008 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sendfile() and UDP socket

Hi All,

Sendfile() over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you run sendfile() with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.

 net/ipv4/udp.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)
net/ipv4/udp.c +15 -0
index 8e42fbb..64e0857 100644
@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
 		 size_t size, int flags)
 {
 	struct udp_sock *up = udp_sk(sk);
+	struct inet_sock *inet = inet_sk(sk);
 	int ret;
+	int fragheaderlen;
+	struct ip_options *opt = NULL;
+
+	lock_sock(sk);
+	if (inet->cork.flags & IPCORK_OPT)
+		opt = inet->cork.opt;
+	fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0);
+
+	if (inet->cork.length + size >= 0xFFFF - fragheaderlen) {
+		ret = udp_push_pending_frames(sk);
+		if (ret)
+			goto out;
+	}
+	release_sock(sk);
 
 	if (!up->pending) {
 		struct msghdr msg = {	.msg_flags = flags|MSG_MORE };
#1 lore raw
Quoted 28 lines · Johann Baudy

Hi All,

Sendfile() over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you run sendfile() with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.

 net/ipv4/udp.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)
net/ipv4/udp.c +8 -0
index 8e42fbb..64e0857 100644
@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
 		 size_t size, int flags)
 {
 	struct udp_sock *up = udp_sk(sk);
+	struct inet_sock *inet = inet_sk(sk);
 	int ret;
+	int fragheaderlen;
+	struct ip_options *opt = NULL;
+
+	lock_sock(sk);
+	if (inet->cork.flags & IPCORK_OPT)
+		opt = inet->cork.opt;
+	fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0);

net/ipv4/udp.c int udp_sendpage(struct sock *sk, struct page *page, int offset, · line 764

Quoted 10 lines
net/ipv4/udp.c +7 -0
+
+	if (inet->cork.length + size >= 0xFFFF - fragheaderlen) {
+		ret = udp_push_pending_frames(sk);
+		if (ret)
+			goto out;
+	}
+	release_sock(sk);
 
 	if (!up->pending) {
 		struct msghdr msg = {	.msg_flags = flags|MSG_MORE };

Hi,

I wonder if it is slightly nicer to do without the opt variable. I _think_ its safe to access inet->cork.opt->optlen based on the (inet->cork.flags & IPCORK_OPT) check.

net/ipv4/udp.c +14 -0
index 8e42fbb..969f6cd 100644
@@ -743,7 +743,21 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
 		 size_t size, int flags)
 {
 	struct udp_sock *up = udp_sk(sk);
+	struct inet_sock *inet = inet_sk(sk);
 	int ret;
+	int fragheaderlen;
+
+	fragheaderlen = sizeof(struct iphdr);
+	lock_sock(sk);
+	if (inet->cork.flags & IPCORK_OPT)
+		fragheaderlen += inet->cork.opt->optlen;
+
+	if (inet->cork.length + size >= 0xFFFF - fragheaderlen) {
+		ret = udp_push_pending_frames(sk);
+		if (ret)
+			goto out;
+	}
+	release_sock(sk);
 
 	if (!up->pending) {
 		struct msghdr msg = {	.msg_flags = flags|MSG_MORE };
Signature · 3 lines
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en
#2 lore raw
Quoted 51 lines · Simon Horman
Quoted 28 lines · Johann Baudy

Hi All,

Sendfile() over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you run sendfile() with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.

 net/ipv4/udp.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)
net/ipv4/udp.c +8 -0
index 8e42fbb..64e0857 100644
@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
 		 size_t size, int flags)
 {
 	struct udp_sock *up = udp_sk(sk);
+	struct inet_sock *inet = inet_sk(sk);
 	int ret;
+	int fragheaderlen;
+	struct ip_options *opt = NULL;
+
+	lock_sock(sk);
+	if (inet->cork.flags & IPCORK_OPT)
+		opt = inet->cork.opt;
+	fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0);

net/ipv4/udp.c int udp_sendpage(struct sock *sk, struct page *page, int offset, · line 764

Quoted 10 lines
net/ipv4/udp.c +7 -0
+
+	if (inet->cork.length + size >= 0xFFFF - fragheaderlen) {
+		ret = udp_push_pending_frames(sk);
+		if (ret)
+			goto out;
+	}
+	release_sock(sk);
 
 	if (!up->pending) {
 		struct msghdr msg = {	.msg_flags = flags|MSG_MORE };

Hi,

I wonder if it is slightly nicer to do without the opt variable. I _think_ its safe to access inet->cork.opt->optlen based on the (inet->cork.flags & IPCORK_OPT) check.

net/ipv4/udp.c +0 -0
index 8e42fbb..969f6cd 100644

net/ipv4/udp.c int udp_sendpage(struct sock *sk, struct page *page, int offset, · line 748

Quoted 6 lines
net/ipv4/udp.c +2 -0
@@ -743,7 +743,21 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
 		 size_t size, int flags)
 {
 	struct udp_sock *up = udp_sk(sk);
+	struct inet_sock *inet = inet_sk(sk);
 	int ret;
+	int fragheaderlen;

Also, I wonder if this should be an unsigned int
as both sizeof(struct iphdr) and inet->cork.opt->optlen are unsigned, though of different width.

Quoted 15 lines
net/ipv4/udp.c +12 -0
+
+	fragheaderlen = sizeof(struct iphdr);
+	lock_sock(sk);
+	if (inet->cork.flags & IPCORK_OPT)
+		fragheaderlen += inet->cork.opt->optlen;
+
+	if (inet->cork.length + size >= 0xFFFF - fragheaderlen) {
+		ret = udp_push_pending_frames(sk);
+		if (ret)
+			goto out;
+	}
+	release_sock(sk);
 
 	if (!up->pending) {
 		struct msghdr msg = {	.msg_flags = flags|MSG_MORE };
Signature · 3 lines
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en
#3 lore raw
In reply to Johann Baudy

Hi, Johann,

Quoted 5 lines

Hi All,

Sendfile() over UDP socket are currently limited to ~ 64KBytes file (max cork.length).
Indeed, if you run sendfile() with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.

UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.

You can pass "offset" of the file you are going to send and "count" to be sent to sendfile systemcall, so you can split the file into several pieces and send each of them.

If you want send a large file over UDP, the typical code will be like:

while (...) {
        setsockopt(fd, UDP_CORK, 1);
        sendmsg(fd, &apl_header, sizeof(apl_header));
        offset += sendfile(fd, offset, count);
        setsockopt(fd, UDP_CORK, 0);
}

or:

while (...) {
        sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE);
        offset += sendfile(fd, offset, count);
}
Quoted 38 lines
 net/ipv4/udp.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)
net/ipv4/udp.c +15 -1
index 8e42fbb..64e0857 100644
@@ -743,7 +743,22 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
 		 size_t size, int flags)
 {
 	struct udp_sock *up = udp_sk(sk);
+	struct inet_sock *inet = inet_sk(sk);
 	int ret;
+	int fragheaderlen;
+	struct ip_options *opt = NULL;
+
+	lock_sock(sk);
+	if (inet->cork.flags & IPCORK_OPT)
+		opt = inet->cork.opt;
+	fragheaderlen = sizeof(struct iphdr) + (opt ? opt->optlen : 0);
+
+	if (inet->cork.length + size >= 0xFFFF - fragheaderlen) {
+		ret = udp_push_pending_frames(sk);
+		if (ret)
+			goto out;
+	}
+	release_sock(sk);
 
 	if (!up->pending) {
 		struct msghdr msg = {	.msg_flags = flags|MSG_MORE };
 
 
--

To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

Le mardi 16 septembre 2008 15:01:17 Hirokazu Takahashi, vous avez écrit :

Quoted 5 lines

UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.

Also, why use UDP for this... If you want stream semantics, why not use TCP or SCTP instead?

Quoted 6 lines

If you want send a large file over UDP, the typical code will be like:

while (...) {
        sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE);
        offset += sendfile(fd, offset, count);
}

Correct me if I am wrong, but... Unless you have a big MTU (as _not_ in 1500 bytes :D), doing an extra syscall might be slower than copying data in a single vectorized sendmsg() syscall.

Signature · 2 lines
Rémi Denis-Courmont
http://www.remlab.net/
#5 lore raw

Hi,

Quoted 8 lines
Quoted 5 lines

UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.

Also, why use UDP for this... If you want stream semantics, why not use TCP or SCTP instead?

I think a lot of VoIP and video streaming services are working on UDP. Linux NFS over UDP also uses this feature.

Quoted 10 lines
Quoted 6 lines

If you want send a large file over UDP, the typical code will be like:

while (...) {
        sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE);
        offset += sendfile(fd, offset, count);
}

Correct me if I am wrong, but... Unless you have a big MTU (as _not_ in 1500 bytes :D), doing an extra syscall might be slower than copying data in a single vectorized sendmsg() syscall.

That's not true.
Even if the MTU is small, you can send a UDP message whose size can be up to 64KB at once. They will be split into several IP packets without any copies.

And more, copying data will pollute the cache memory much, which cannot be ignored if you want send tons of data of files.

Thanks,
Hirokazu Takahashi.

#6 lore raw

Le vendredi 19 septembre 2008 15:28:54 Hirokazu Takahashi, vous avez écrit :

Quoted 10 lines
Quoted 8 lines
Quoted 5 lines

UDP is a datagram protocol, so I think applications using UDP should care about the size of packets they are going to send rather than expecting that the messages will be split into several packets automatically. If some of the packets have lost, it will be really hard for the applications to re-create the same ones to send again.

Also, why use UDP for this... If you want stream semantics, why not use TCP or SCTP instead?

I think a lot of VoIP and video streaming services are working on UDP.

VoIP uses lots of small packets, considering the typical packetization times. Using send(MSG_MORE) + sendfile would definitely be slower than a single sendmsg() in such case, because the per-packet memcpy() will be quite short.

Video streaming typically does send lots of large packets, and might well read the data from a mmap-able file. But video streaming protocols such as RTP typically try to avoid fragmentation, so large sendfile() won't work.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines

If you want send a large file over UDP, the typical code will be like:

while (...) {
        sendmsg(fd, &apl_header, sizeof(apl_header), MSG_MORE);
        offset += sendfile(fd, offset, count);
}
Quoted 8 lines
Quoted 3 lines

Correct me if I am wrong, but... Unless you have a big MTU (as _not_ in 1500 bytes :D), doing an extra syscall might be slower than copying data in a single vectorized sendmsg() syscall.

That's not true.
Even if the MTU is small, you can send a UDP message whose size can be up to 64KB at once. They will be split into several IP packets without any copies.

And you will encounter fragmentation, which sucks at high data rates. As for low rates, you typically would not bother to optimize memcpy. But more importantly, you should not assume that the other end is only talking to you - it might as well be talking to many different people and not appreciate if all of them send lots of fragments.

And lets not get into how poorly fragmentation works through middleboxes...

Quoted 2 lines

And more, copying data will pollute the cache memory much, which cannot be ignored if you want send tons of data of files.

Signature · 2 lines
Rémi Denis-Courmont
http://www.remlab.net/
maintainer
#7 lore raw
In reply to Johann Baudy

From: Johann Baudy <[email protected]>
Date: Sun, 14 Sep 2008 12:25:56 +0200

Quoted 7 lines

Sendfile() over UDP socket are currently limited to ~ 64KBytes file (max cork.length). Indeed, if you run sendfile() with a file size > 64KBytes over UDP socket, system call will stop and return ~64KBytes without sending anything on the network. This patch is pushing ongoing frames when frames buffer is full, to prevent overflow.

Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.

If you look, this is what things like NFS using SUNRPC over UDP do. They have a transmission unit for the data transfer and use that for each "send".

A sendfile() with length >= 64K is the same as a sendmsg() with such a length, which is defined as:

	if (len > 0xFFFF)
		return -EMSGSIZE;

So we could technically even return an error for this sendfile() over UDP case.

#8 lore raw
Quoted 4 lines · David Miller

Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.

But isn't the whole idea of the sendfile() is to send a file no matter what underlying media is?

Quoted 3 lines

If you look, this is what things like NFS using SUNRPC over UDP do. They have a transmission unit for the data transfer and use that for each "send".

That's maybe because udp_sendpage() does not support sending pending data if new packet is too big to attach?

Signature · 1 line
	Evgeniy Polyakov
maintainer
#9 lore raw

From: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 04:21:57 +0400

Quoted 8 lines · David Miller
Quoted 4 lines

Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.

But isn't the whole idea of the sendfile() is to send a file no matter what underlying media is?

It's a way to fabricate a send() directly from the page cache.

Quoted 2 lines

That's maybe because udp_sendpage() does not support sending pending data if new packet is too big to attach?

I don't think so.

It's simply enforcing the wsize/rsize that's configured for the mount. And this is exactly deciding what the UDP segment size should be.

#10 lore raw
Quoted 9 lines · David Miller
Quoted 7 lines
Quoted 4 lines

Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.

But isn't the whole idea of the sendfile() is to send a file no matter what underlying media is?

It's a way to fabricate a send() directly from the page cache.

And to send exactly required number of bytes (or size of the cache)? To send a single page (combined to several other pages) we have simple ->sendpage() callback, which should not return error when it is asked to send a data and it can do it by actually submitting two packets without special tcp-like processing of the segments.

Quoted 7 lines
Quoted 2 lines

That's maybe because udp_sendpage() does not support sending pending data if new packet is too big to attach?

I don't think so.

It's simply enforcing the wsize/rsize that's configured for the mount. And this is exactly deciding what the UDP segment size should be.

And what if it is just a result of the knowledge on how udp_sendpage() behaves, so code adjusts packet sizes in advance?

Signature · 1 line
	Evgeniy Polyakov
maintainer
#11 lore raw

From: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 05:08:34 +0400

Quoted 9 lines · David Miller
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

Applications which work over datagram protocols must perform their own segmentation. It is not like doing a send over a stream protocol like TCP, where you can use whatever length you want for send calls and segmentation is done for the application.

But isn't the whole idea of the sendfile() is to send a file no matter what underlying media is?

Quoted 7 lines
Quoted 1 line

It's a way to fabricate a send() directly from the page cache.

And to send exactly required number of bytes (or size of the cache)? To send a single page (combined to several other pages) we have simple ->sendpage() callback, which should not return error when it is asked to send a data and it can do it by actually submitting two packets without special tcp-like processing of the segments.

You're basically throwing away the difference between datagram and stream socket semantics.

I don't see what else I can explain if you cannot see that this is significant.

#12 lore raw
Quoted 11 lines · David Miller
Quoted 5 lines

And to send exactly required number of bytes (or size of the cache)? To send a single page (combined to several other pages) we have simple ->sendpage() callback, which should not return error when it is asked to send a data and it can do it by actually submitting two packets without special tcp-like processing of the segments.

You're basically throwing away the difference between datagram and stream socket semantics.

I don't see what else I can explain if you cannot see that this is significant.

Hey David, that's getting the wrong direction :)
Do not make decision backed by what you read or decided to think before that instead of making it clear.

Stream socket means that whatever data we put to it, it will become completely boundary-free, in that regard, that receiving side will not be able to get original sending packet sizes (without too much efforts).

Datagram just preservs the boundaries and that's what we have with this patch. Previously we accumulated a single segment upto predefined size and sent it when it is complete. Now we are able to sent it when it is complete and start creating new segment without returning the error.

Like:
previosly: A + A + A + A + send + return
now: A + A + A + A + send + A + A and so on, so effectively nothing changes except maybe time when segment is being sent: previously it was on the full size and now it is when new packet does not fit that predefined size.

Signature · 1 line
	Evgeniy Polyakov
maintainer
#13 lore raw

From: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 08:19:29 +0400

Quoted 3 lines

Stream socket means that whatever data we put to it, it will become completely boundary-free, in that regard, that receiving side will not be able to get original sending packet sizes (without too much efforts).

Right.

Quoted 2 lines

Datagram just preservs the boundaries and that's what we have with this patch.

Not exactly.

Each and every send() operation on a datagram socket must result in exactly one packet. sendfile() was following this rule, when it returned successfully only one single packet was emitted.

The new sendfile() semantics are outside of this model.

#14 lore raw
Quoted 10 lines · David Miller
Quoted 2 lines

Datagram just preservs the boundaries and that's what we have with this patch.

Not exactly.

Each and every send() operation on a datagram socket must result in exactly one packet. sendfile() was following this rule, when it returned successfully only one single packet was emitted.

The new sendfile() semantics are outside of this model.

That's for send(), not sendfile(). The latter now works like lots of sends() while previously it worked as single send().

Why sendfile() should be completely different compared to stream case? Obviously send() has to be different and it is, but sendfile() is just a bunch of sends over the pages in the cache, so let's allow it to be multiple sends and not only single packet send.

Signature · 1 line
	Evgeniy Polyakov
maintainer
#15 lore raw

From: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 08:40:52 +0400

Quoted 1 line

Why sendfile() should be completely different compared to stream case?

Because datagram sockets are completely different from stream sockets.

You program them differently, segmentation is made by the socket user not within by the protocol itself.

sendfile() should behave in a way congruent to the other data transfer APIs of the BSD socket layer.

#16 lore raw
Quoted 7 lines · David Miller

Because datagram sockets are completely different from stream sockets.

You program them differently, segmentation is made by the socket user not within by the protocol itself.

sendfile() should behave in a way congruent to the other data transfer APIs of the BSD socket layer.

So effectively you are saying, that sendfile() is just a pure send(), but with diferent arguments? I.e. it is not supposed to send the whole data it points to, but as much as possible according to send() standard?

Well, it may be a right or wrong decision, and in my opinion sendfile() is very diferent than send() since it should require the whole data to be sent, i.e. being like a loop of send()s, but since sendfile() is effectively a very new approach, it could have different behaviour rules.

Because of this, Johann, this patch will not be applied, but thanks a lot for your work, we made a clear meaning on interface usage.

Signature · 1 line
	Evgeniy Polyakov
maintainer
#17 lore raw

From: Evgeniy Polyakov <[email protected]>
Date: Mon, 22 Sep 2008 09:49:20 +0400

Quoted 6 lines · David Miller
Quoted 5 lines

Because datagram sockets are completely different from stream sockets.

You program them differently, segmentation is made by the socket user not within by the protocol itself.

Quoted 6 lines
Quoted 2 lines

sendfile() should behave in a way congruent to the other data transfer APIs of the BSD socket layer.

So effectively you are saying, that sendfile() is just a pure send(), but with diferent arguments? I.e. it is not supposed to send the whole data it points to, but as much as possible according to send() standard?

It already makes this, guess what happens when socket error occurs midstream during sendfile(), even for TCP?

First, we return length successfully sent.

User has to retry sendfile() call with remaining length, and at this point they will immediately get the socket error return value.

This was broken at one point and I remember applying the fix for this several years ago :-)

Quoted 5 lines

Well, it may be a right or wrong decision, and in my opinion sendfile() is very diferent than send() since it should require the whole data to be sent, i.e. being like a loop of send()s, but since sendfile() is effectively a very new approach, it could have different behaviour rules.

It is in fact exactly and precisely like send(). It must even give the same error return semantics as other socket data transfer calls do. See above.

The user has to have a resending loop _anyways_, in order to do correct error handling.

Nothing is gained from the proposal, really. It can only harm application developers into thinking that segmentation over datagram sockets is not their responsibility, when it absolutely is.

#18 lore raw
Quoted 7 lines · David Miller
Quoted 3 lines

So effectively you are saying, that sendfile() is just a pure send(), but with diferent arguments? I.e. it is not supposed to send the whole data it points to, but as much as possible according to send() standard?

It already makes this, guess what happens when socket error occurs midstream during sendfile(), even for TCP?

Quoted 7 lines

First, we return length successfully sent.

User has to retry sendfile() call with remaining length, and at this point they will immediately get the socket error return value.

This was broken at one point and I remember applying the fix for this several years ago :-)

Well, still tcp errors are completely different from udp ones: the former ony result in a real problem or interrupt/timeout in the simple case. UDP adds 'it is too big to be transferred' one. And instead of just send the previous frame and start new one, we return error so that user could do the same with additional syscall.

Quoted 9 lines
Quoted 5 lines

Well, it may be a right or wrong decision, and in my opinion sendfile() is very diferent than send() since it should require the whole data to be sent, i.e. being like a loop of send()s, but since sendfile() is effectively a very new approach, it could have different behaviour rules.

It is in fact exactly and precisely like send(). It must even give the same error return semantics as other socket data transfer calls do. See above.

That's the main difference on how you and me look at sendfile().

Quoted 6 lines

The user has to have a resending loop _anyways_, in order to do correct error handling.

Nothing is gained from the proposal, really. It can only harm application developers into thinking that segmentation over datagram sockets is not their responsibility, when it absolutely is.

Which basically means that sendfile() for udp is exactly send() of the mapped data. And it is not about segmentation, since packets are correctly segmented with this proposal: packet is rounded to the submitted page. This is about optimization compared to send() of the data. You basically draw a line, that with UDP it is impossible to do.

Signature · 1 line
	Evgeniy Polyakov
#19 lore raw
Quoted 7 lines · Evgeniy Polyakov
Quoted 3 lines · David Miller

It is in fact exactly and precisely like send(). It must even give the same error return semantics as other socket data transfer calls do. See above.

That's the main difference on how you and me look at sendfile().

I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#20 lore raw
Quoted 4 lines · Herbert Xu

I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.

One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.

Signature · 1 line
	Evgeniy Polyakov
#21 lore raw
Quoted 8 lines · Evgeniy Polyakov
Quoted 4 lines · Herbert Xu

I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.

One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.

Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)

So what application needs this?

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#22 lore raw
Quoted 2 lines · Herbert Xu

Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)

It still deserves living if it is not highly priority traffic like video dataflow.

Quoted 1 line

So what application needs this?

Johann did not show his exact usage scenario, but from what I got I concluded, that it is kind of a video sensor (or some other data which is allowed to be lost), which has to add a header
to the frame and submit it to the network without any copy because of hardware limitation for the memory bus. Sensor can put data via DMA to the needed location. His first (and parallel to this one) idea was to extend packet socket to allow send of the mapped data.
I proposed to do the same with sendfile, i.e. dma data to the mapped area of some file in the ramdisk, add header, do it for multiple frames and then send given file using sendfile().

Signature · 1 line
	Evgeniy Polyakov
#23 lore raw
In reply to Herbert Xu
Quoted 7 lines · Herbert Xu
Quoted 6 lines · Evgeniy Polyakov
Quoted 4 lines · Herbert Xu

I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.

Quoted 7 lines
Quoted 2 lines

One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.

Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)

So what application needs this?

It seems it might be useful for a video server. The one thing that seems to be missing from the sendfile() semantics is a message size to be used for splitting the file into UDP datagrams, but this could be provided by a separate ioctl(), and could default to the largest message size that would fit in the MTU.

-Bill

A message that is not in the archive (sent off-list or not yet received): [email protected]

#25 lore raw

Hi All,

I'm sorry for this late reply but I was out of Internet access during one month. So if my understanding is correct, sendfile() over UDP can't be used to send a file > 64 ko in one system call, as each request must fit in one UDP packet.

However is it expected that sendfile() returns 64Ko but sends nothing over the network when we are performing this particular case (file>64ko)? should it return 0?

Thanks in advance,
Johann

Quoted 28 lines · Bill Fink
Quoted 24 lines · Herbert Xu
Quoted 8 lines · Evgeniy Polyakov
Quoted 6 lines · Herbert Xu
Quoted 4 lines

I think this dicussion is moot since the only time you want to use sendfile is for bulk transfers and these days anybody designing new applications that does bulk transfers over UDP should be taken out and shot.

Quoted 7 lines
Quoted 2 lines

One can protect himself pointing how slow may be memory bus in some hardware setup, which completely does not allow to perform any copy.

Yes but bulk transfers over UDP is a bad idea regardless of how slow your bus is :)

So what application needs this?

It seems it might be useful for a video server. The one thing that seems to be missing from the sendfile() semantics is a message size to be used for splitting the file into UDP datagrams, but this could be provided by a separate ioctl(), and could default to the largest message size that would fit in the MTU.

Quoted 7 lines
Quoted 1 line

-Bill

--
Johann Baudy
[email protected]

--
Johann Baudy
[email protected]

#26 lore raw

Using skb_get() to recycle skbs

In reply to Evgeniy Polyakov

In our netdev driver, we preallocate a pool of skb which are used by our h/w to DMA frames.

Normally, when we indicate a packet to stack (netif_rx), the stack ends up freeing it. If we do a skb_get() prior to that, then the stack just reduces the use_count, and our driver is able to re-use it. We do have to use caution to see that stack has dropped it's reference (atomic_read(skb->users) <= 1) prior to returning the buffer to our h/w.

Is this a valid approach? It so it could save some CPU cycles which are used in alloc_skb(). If this can be done, how come more drivers in tree are not doing it, is there any flip side to it? One could be that stack may not free it in time (we could starve the hardware of free buffers), but in that case we can drop our reference, and let the stack free it up, and force an allocation instead of reuse.

Thanks for your response.

Ram Natarajan
Emulex Corporation

#27 lore raw

Re: Using skb_get() to recycle skbs

Hi.

Quoted 11 lines · On Thu, Sep 25, 2008 at 06:03:03AM -0700, [email protected]…

In our netdev driver, we preallocate a pool of skb which are used by our h/w to DMA frames.

Normally, when we indicate a packet to stack (netif_rx), the stack ends up freeing it. If we do a skb_get() prior to that, then the stack just reduces the use_count, and our driver is able to re-use it. We do have to use caution to see that stack has dropped it's reference (atomic_read(skb->users) <= 1) prior to returning the buffer to our h/w.

Quoted 9 lines

Is this a valid approach? It so it could save some CPU cycles which are used in alloc_skb(). If this can be done, how come more drivers in tree are not doing it, is there any flip side to it? One could be that stack may not free it in time (we could starve the hardware of free buffers), but in that case we can drop our reference, and let the stack free it up, and force an allocation instead of reuse.

Now stack allocates new skb and copy data from old one to the new, since it belives that skb is shared between different users and thus can not be modified in the rx path. You can check how skb_share_check() is called in ip_rcv() and similar receiving functions.

Signature · 1 line
	Evgeniy Polyakov