Skip to contentSkip to conversation

min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

DISCUSSION Ben Greear 2 participants 5 messages Dec 10, 2001

lore
Share & tools
lore

Ben Greear

author
#0 lore raw

This relates to my earlier question about setting the threshold at which select returns that a (UDP) socket is writable.

It appears that UDP sockets are hardwired at 2048 bytes...

From linux/include/net/sock.h:

....
#define SOCK_MIN_SNDBUF 2048
#define SOCK_MIN_RCVBUF 256
/* Must be less or equal SOCK_MIN_SNDBUF */
#define SOCK_MIN_WRITE_SPACE SOCK_MIN_SNDBUF

/*
* Default write policy as shown to user space via poll/select/SIGIO
* Kernel internally doesn't use the MIN_WRITE_SPACE threshold.
*/
static inline int sock_writeable(struct sock *sk)

{
	return sock_wspace(sk) >= SOCK_MIN_WRITE_SPACE;
}

It appears that this issue could be fixed in several ways. One would be to make SOCK_MIN_WRITE_SPACE a global variable and set it through the proc file system (or an IOCTL, etc...). That is not very flexible, but easy. It sounds more likely to break existing programs though..

A more flexible way would be to set the value on a per sock basis (in the sock structure), which is probably the right way, but may require a new IOCTL defined
in user-space... This way should be completely backwards compatible, as the default value can be 2048 as it is now.

Fixing this little problem will be very useful for anyone using larger UDP packet sizes and poll/select. Remember that GigE supports >>2048 MTU, so using larger UDP packets becomes a more useful thing, even in the real world :)

As a side note, it appears that TCP is much smarter about this, taking the current load into account: If there is very little in the current write queue, then it will say the socket is writable at a very small threshold. However, if the write queue is almost full, tcp will make select/poll wait longer. This type of thing could also work for UDP, and the user can indirectly determine the behaviour by setting the write-queue size appropriately large...

Thanks,
Ben

Signature · 3 lines
Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear
#1 lore raw
Quoted 6 lines · Ben Greear

This relates to my earlier question about setting the threshold at which select returns that a (UDP) socket is writable.

It appears that UDP sockets are hardwired at 2048 bytes...

From linux/include/net/sock.h:

    int len = 0x8000;

    setsockopt(s, SOL_SOCKET, SO_SNDBUF, &len, sizeof(len));
    setsockopt(s, SOL_SOCKET, SO_RCVBUF, &len, sizeof(len));

Doesn't this work?

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

I was going to compile a list of innovations that could be attributed to Microsoft. Once I realized that Ctrl-Alt-Del was handled in the BIOS, I found that there aren't any.

author
#2 lore raw
Quoted 11 lines · Richard B. Johnson
Quoted 7 lines · Ben Greear

This relates to my earlier question about setting the threshold at which select returns that a (UDP) socket is writable.

It appears that UDP sockets are hardwired at 2048 bytes...

From linux/include/net/sock.h:

Quoted 7 lines
    int len = 0x8000;

    setsockopt(s, SOL_SOCKET, SO_SNDBUF, &len, sizeof(len));
    setsockopt(s, SOL_SOCKET, SO_RCVBUF, &len, sizeof(len));

Doesn't this work?

That sets the queue sizes bigger, but suppose this:

I have 4M queue size. I have 4M-2k bytes already in the queue (2k free). I have a 4k UDP buffer to write. I call select and it says the socket is writable. However, in this case I cannot actually write to the socket because I have only 2 of the 4k that I need... Now, I can detect the failure to send and re-transmit, but that basically gets me into a tight loop because select keeps saying I can write, and I keep trying. The tight loop is doubly bad because the machine is already highly stressed or it's buffers would never be so full....

I want select to only say I can write when I'm at XX (say, 64k) bytes of free buffer-queue space...

Ben

Quoted 11 lines

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

I was going to compile a list of innovations that could be attributed to Microsoft. Once I realized that Ctrl-Alt-Del was handled in the BIOS, I found that there aren't any.

Signature · 3 lines
Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear
#3 lore raw
Quoted 36 lines · Ben Greear
Quoted 11 lines · Richard B. Johnson
Quoted 7 lines · Ben Greear

This relates to my earlier question about setting the threshold at which select returns that a (UDP) socket is writable.

It appears that UDP sockets are hardwired at 2048 bytes...

From linux/include/net/sock.h:

Quoted 7 lines
    int len = 0x8000;

    setsockopt(s, SOL_SOCKET, SO_SNDBUF, &len, sizeof(len));
    setsockopt(s, SOL_SOCKET, SO_RCVBUF, &len, sizeof(len));

Doesn't this work?

That sets the queue sizes bigger, but suppose this:

I have 4M queue size. I have 4M-2k bytes already in the queue (2k free). I have a 4k UDP buffer to write. I call select and it says the socket is writable. However, in this case I cannot actually write to the socket because I have only 2 of the 4k that I need... Now, I can detect the failure to send and re-transmit, but that basically gets me into a tight loop because select keeps saying I can write, and I keep trying. The tight loop is doubly bad because the machine is already highly stressed or it's buffers would never be so full....

Quoted 5 lines

I want select to only say I can write when I'm at XX (say, 64k) bytes of free buffer-queue space...

Ben

If you have a 4M queue size, it appears as though you are trying to use UDP where TCP should have been used. Normally, what you call the queue size, is set to contain you largest packet you will ever want to send. With this in mind, you don't even know if a fragmented packet can be routed if it's more than 64k in length so you would never try to send something larger than that under UDP.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

I was going to compile a list of innovations that could be attributed to Microsoft. Once I realized that Ctrl-Alt-Del was handled in the BIOS, I found that there aren't any.

author
#4 lore raw
Quoted 19 lines · Richard B. Johnson
Quoted 10 lines · Ben Greear

I have 4M queue size. I have 4M-2k bytes already in the queue (2k free). I have a 4k UDP buffer to write. I call select and it says the socket is writable. However, in this case I cannot actually write to the socket because I have only 2 of the 4k that I need... Now, I can detect the failure to send and re-transmit, but that basically gets me into a tight loop because select keeps saying I can write, and I keep trying. The tight loop is doubly bad because the machine is already highly stressed or it's buffers would never be so full....

Quoted 6 lines

I want select to only say I can write when I'm at XX (say, 64k) bytes of free buffer-queue space...

Ben

Quoted 6 lines

If you have a 4M queue size, it appears as though you are trying to use UDP where TCP should have been used. Normally, what you call the queue size, is set to contain you largest packet you will ever want to send. With this in mind, you don't even know if a fragmented packet can be routed if it's more than 64k in length so you would never try to send something larger than that under UDP.

Assume that I actually want to do what I said I did! :)

I never try to send
a packet bigger than 64k, the protocol doesn't handle it. But, I may try to send 100000 60k UDP packets in very fast succession...which could fill up my send queue.

Signature · 3 lines
Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear