datagram queue length
(Posted a few days ago to c.os.l.networking; no replies there.)
I seem to be running into a limit of 64 queued datagrams. This isn't a data buffer size; varying the size of the datagram makes no difference in the observed queue size. If more datagrams are sent before some are read, they are silently dropped. (By "silently," I mean, "tcpdump doesn't record these as dropped packets.")
This only happens when the sending and receiving processes are on different machines, btw.
Can anyone tell me where this magic 64 number comes from, so I can increase it?
Python demo attached.
-Jonathan
# <receive udp requests>
# start this, then immediately start the other # _on another machine_ import socket, time
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.bind(('', 3001))
time.sleep(5)
while True:
data, client_addr = sock.recvfrom(8192)
print data# <separate process to send stuff>
import socket
for i in range(200):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.sendto('a' * 100, 0, ('***other machine ip***', 3001)) sock.close()
Quoted 11 lines · Jonathan Ellis
(Posted a few days ago to c.os.l.networking; no replies there.)
I seem to be running into a limit of 64 queued datagrams. This isn't a data buffer size; varying the size of the datagram makes no difference in the observed queue size. If more datagrams are sent before some are read, they are silently dropped. (By "silently," I mean, "tcpdump doesn't record these as dropped packets.")
This only happens when the sending and receiving processes are on different machines, btw.
Quoted 7 lines
Can anyone tell me where this magic 64 number comes from, so I can increase it?
Python demo attached.
-Jonathan
Your datagram receiver isn't keeping up with your datagram transmitter. If you increase the number of datagrams that are being queued, you will still encounter the same problem, but after more datagrams are stored. You need to either throttle the transmitter or speed up the receiver.
In your test code, you deliberately don't receive anything for 5 seconds. What do you expect? You are lucky you get one datagram!
Quoted 17 lines
# <receive udp requests>
# start this, then immediately start the other # _on another machine_ import socket, timesock =
socket.socket(socket.AF_INET, socket.SOCK_DGRAM)sock.bind(('', 3001))
time.sleep(5)while True:
data, client_addr = sock.recvfrom(8192) print data# <separate process to send stuff>
import socket
Quoted 10 lines
for i in range(200):
sock =socket.socket(socket.AF_INET, socket.SOCK_DGRAM)sock.sendto('a' * 100, 0, ('***other machine ip***', 3001))sock.close()
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Cheers,
Dick Johnson
Penguin : Linux version 2.6.12 on an i686 machine (5537.79 BogoMips). Warning : 98.36% of all statistics are fiction.
.
I apologize for the following. I tried to kill it with the above dot :
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
Quoted 10 lines · linux-os (Dick Johnson)
Quoted 5 lines
I seem to be running into a limit of 64 queued datagrams. This isn't a data buffer size; varying the size of the datagram makes no difference in the observed queue size. If more datagrams are sent before some are read, they are silently dropped. (By "silently," I mean, "tcpdump doesn't record these as dropped packets.")
Your datagram receiver isn't keeping up with your datagram transmitter. If you increase the number of datagrams that are being queued, you will still encounter the same problem, but after more datagrams are stored.
Right -- except that my consumer is quite fast enough in the average case; it's only the worst case where it can't keep up. Extending the queue would allow it to catch up with such bursts of activity without dropping requests. The low- and mid- hanging fruit has already been picked as far as consumer optimization goes; anything remaining is quite high indeed.
Quoted 2 lines
In your test code, you deliberately don't receive anything for 5 seconds. What do you expect?
I expected to demonstrate the problem. :)
-Jonathan