Excessive TCP retransmits over lossless, high latency link
[Oops, don't group-reply to the other one which I sent to vger.rutgers.edu by mistake :-)]
First let me describe the link.
The kernel is standard 2.4.7, i686 on a laptop, all standard settings.
I have been downloading a large email over a "9600 baud" mobile phone link, over infrared (from phone to computer). Protocols: PPP over IRComm over IRDA at 115200 baud. The modem is in the phone (the phone responds to AT commands and provides data, over the IRComm protocol). The phone is a Nokia 6210e.
Packets arrive _very_ slowly -- about 1 packet every 5.2 seconds. (Showing that "9600 baud" is even worse false advertising than "56k").
The ping time during this download is ~23 seconds. I've seen it go up to >60 seconds. When the link is idle, the ping time oscillates beteen 1 second and 1.5 seconds. All the responses seem to arrive eventually, and in order.
(The irdaping time is about 0.2 seconds, when the IRDA link is idle. When the link is active, irdaping doesn't see any response, except for once when the call terminated, perhaps due to irdaping. At least, we see that the IRDA link time is negligable).
So we can see that the mail download is acting to fill up the router queues. Everyone here knows that TCP does tend to fill router queues, even when that pessimises performance. Quite why anybody designs routers with 60 second queues is unclear to me, but they do.
Even with this, I can see that the large mail is downloading far slower than it ought to.
The appended "tcpdump -i ppp0 -n" trace shows an excessive number of retransmits from the remote POP server. The retransmits are triggered by excessive duplicate ACKs from the local client. By excessive, I mean lots of retransmits of the same data.
The interesting thing is that there isn't any evidence of packet loss.
It appears there is a stable cycle of duplicate ACKs and duplicate retransmits which doesn't go away, even without packet loss.
I think this is fixable problem at the Linux client end. Linux is sending many duplicate ACKs in response to data that it has already received. Normally, duplicate ACKs are useful to indicate that a packet is missing and should be retransmitted. Those ACKs have a sequence number that is higher than the packet which triggers them.
In this case, there is no missing data, and all the duplicate ACKs do is cause the server to resend the same data, repeatedly and redundantly.
If this can happen over a low loss but very slow and high latency link, I guess it can happen over normal links too, but I haven't seen an example.
Is there some /proc/sys setting to fix this, a kernel patch, or is it perhaps fixed in a newer kernel already?
thanks,
-- Jamie
ps. please don't send me any large emails ATM, thx, they take about 30 minutes per 100k to download :-)
17:39:38.872919 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 4294964376:4294965836(1460) ack 1 win 8760 (DF) 17:39:38.873003 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 1460 win 62780 (DF) 17:39:44.122914 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 4294965836:0(1460) ack 1 win 8760 (DF) 17:39:44.123006 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 1460 win 62780 (DF) 17:39:49.372887 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 0:1460(1460) ack 1 win 8760 (DF) 17:39:49.372986 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 1460 win 62780 (DF) 17:39:54.612626 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 1460:2920(1460) ack 1 win 8760 (DF) 17:39:54.612712 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 2920 win 62780 (DF) 17:39:57.742873 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 2920:3812(892) ack 1 win 8760 (DF) 17:39:57.742962 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 3812 win 62780 (DF) 17:40:03.482921 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 3812:5272(1460) ack 1 win 8760 (DF) 17:40:03.483010 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:08.732901 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 1460:2920(1460) ack 1 win 8760 (DF) 17:40:08.732991 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:13.972868 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 1460:2920(1460) ack 1 win 8760 (DF) 17:40:13.972954 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:19.712883 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 1460:2920(1460) ack 1 win 8760 (DF) 17:40:19.712975 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:24.923064 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 1460:2920(1460) ack 1 win 8760 (DF) 17:40:24.923157 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:30.102989 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 2920:4380(1460) ack 1 win 8760 (DF) 17:40:30.103082 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:35.312885 < 194.217.242.22.pop3 > 193.237.65.219.32772: . 3812:5272(1460) ack 1 win 8760 (DF) 17:40:35.312978 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 5272 win 62780 (DF) 17:40:40.522880 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 5272:6671(1399) ack 1 win 8760 (DF) 17:40:40.522968 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 1:1(0) ack 6671 win 62780 (DF) 17:40:40.632511 > 193.237.65.219.32772 > 194.217.242.22.pop3: P 1:10(9) ack 6671 win 62780 (DF) 17:40:45.301126 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 5272:6671(1399) ack 1 win 8760 (DF) 17:40:45.301236 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 10:10(0) ack 6671 win 62780 (DF) 17:40:50.503077 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 5272:6671(1399) ack 1 win 8760 (DF) 17:40:50.503179 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 10:10(0) ack 6671 win 62780 (DF) 17:40:55.723023 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 5272:6671(1399) ack 1 win 8760 (DF) 17:40:55.723123 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 10:10(0) ack 6671 win 62780 (DF) 17:40:57.402665 > 193.237.65.219.32772 > 194.217.242.22.pop3: P 1:10(9) ack 6671 win 62780 (DF) 17:41:00.522888 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 5272:6671(1399) ack 1 win 8760 (DF) 17:41:00.523004 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 10:10(0) ack 6671 win 62780 (DF) 17:41:00.522899 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6671:6677(6) ack 10 win 8760 (DF) 17:41:00.523073 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 10:10(0) ack 6677 win 62780 (DF) 17:41:00.523535 > 193.237.65.219.32772 > 194.217.242.22.pop3: P 10:27(17) ack 6677 win 62780 (DF) 17:41:00.542914 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6671:6677(6) ack 10 win 8760 (DF) 17:41:00.543035 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 27:27(0) ack 6677 win 62780 (DF) 17:41:00.542921 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6671:6677(6) ack 10 win 8760 (DF) 17:41:00.543091 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 27:27(0) ack 6677 win 62780 (DF) 17:41:00.560282 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6671:6677(6) ack 10 win 8760 (DF) 17:41:00.560382 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 27:27(0) ack 6677 win 62780 (DF) 17:41:06.792902 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6677:8137(1460) ack 27 win 8760 (DF) 17:41:06.793004 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 27:27(0) ack 8137 win 62780 (DF) 17:41:08.882933 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 8137:8706(569) ack 27 win 8760 (DF) 17:41:08.883022 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 27:27(0) ack 8706 win 62780 (DF) 17:41:08.904657 > 193.237.65.219.32772 > 194.217.242.22.pop3: P 27:36(9) ack 8706 win 62780 (DF) 17:41:13.592879 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6677:8137(1460) ack 27 win 8760 (DF) 17:41:13.592967 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 36:36(0) ack 8706 win 62780 (DF) 17:41:18.819646 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 6677:8137(1460) ack 27 win 8760 (DF) 17:41:18.819733 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 36:36(0) ack 8706 win 62780 (DF) 17:41:20.912868 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 8137:8706(569) ack 27 win 8760 (DF) 17:41:20.912983 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 36:36(0) ack 8706 win 62780 (DF) 17:41:20.912875 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 8706:8712(6) ack 36 win 8760 (DF) 17:41:20.913050 > 193.237.65.219.32772 > 194.217.242.22.pop3: . 36:36(0) ack 8712 win 62780 (DF) 17:41:20.912884 < 194.217.242.22.pop3 > 193.237.65.219.32772: P 8706:8712(6) ack 36 win 8760 (DF)
Quoted 4 lines · Jamie Lokier
The appended "tcpdump -i ppp0 -n" trace shows an excessive number of retransmits from the remote POP server. The retransmits are triggered by excessive duplicate ACKs from the local client. By excessive, I mean lots of retransmits of the same data.
The duplicate ACKs should not cause any retransmits (unless the sender is badly broken), because they contain a high enough ACK number.
The problem is really that a TCP sender cannot recover from a too short RTT estimate; if the RTT is longer and it doesn't get any acks it'll assume packet loss and never has a chance to find out about the longer RTT, because that only works with new ACKs.
The standard initial RTT is 3 seconds; but your RTT is 5.2s.
What you can do is to change the initial RTT on the sender side. On Linux it can be done with the "irtt" option of route or the equivalent one of iproute2. Most other OS have a similar way to configure the IRTT.
-Andi
Hello!
Quoted 1 line
The interesting thing is that there isn't any evidence of packet loss.
Why did you disable bith sacks and timestamps? Exactly to get maximal damage from long delay link?
Quoted 2 lines
Is there some
/proc/syssetting to fix this, a kernel patch, or is it perhaps fixed in a newer kernel already?
No patches to block send required ACKs exist of course. :-)
All the problem is at sender, it mispredicts rtt.
What OS is sender? If it is linux too, try to use default configuration not playing with /proc/sys/net/tcp_*, especially with timestamps and sacks and the situation should rectify.
Also, please, send full (binary!) tcpdump from SYN and to FIN. Andi says right thing, but I am still puzzled why rtt is miscalculated it should be estimated correctly.
Well, and if sender is not linux... no ideas.
Alexey
Hello!
Quoted 2 lines
packet loss and never has a chance to find out about the longer RTT, because that only works with new ACKs.
Karn algo calculates right rtt after log(rtt/3sec) retransmits. It is not so big number. In this case, it is 1 in fact. The only effect of irtt setting is avoiding one retransmission.
Alexey
Quoted 8 lines · Andi Kleen
Quoted 4 lines · Jamie Lokier
The appended "tcpdump -i ppp0 -n" trace shows an excessive number of retransmits from the remote POP server. The retransmits are triggered by excessive duplicate ACKs from the local client. By excessive, I mean lots of retransmits of the same data.
The duplicate ACKs should not cause any retransmits (unless the sender is badly broken), because they contain a high enough ACK number.
We are receiving frames with sequence number <= the ACK we're sending.
When the remote end receives those ACKs, it is seeing a series of duplicate ACKs for data it sent about 5 frames ago. So of course it retransmits the data starting from 5 frames ago. As it receives a whole series of the same duplicate ACK, it retransmits from the same place each time.
I don't see what is broken about the remote end in this case.
Quoted 6 lines
The problem is really that a TCP sender cannot recover from a too short RTT estimate; if the RTT is longer and it doesn't get any acks it'll assume packet loss and never has a chance to find out about the longer RTT, because that only works with new ACKs.
The standard initial RTT is 3 seconds; but your RTT is 5.2s.
Actually, 5.2s is the most common packet interarrival time. The RTT is more like 23s!
Quoted 3 lines
What you can do is to change the initial RTT on the sender side. On Linux it can be done with the "irtt" option of route or the equivalent one of iproute2. Most other OS have a similar way to configure the IRTT.
Unfortunately I cannot change the IRTT on my ISP's mail server, or for that matter on anyone's web server.
Are you saying that the IRTT must be larger than the actual RTT, and the estimates can only go down? I thought IRTT was a rough guess and it was expected to go either up or down.
-- Jamie
Quoted 8 lines · [email protected] wrote:
Quoted 1 line
The interesting thing is that there isn't any evidence of packet loss.
Why did you disable bith sacks and timestamps? Exactly to get maximal damage from long delay link?
What OS is sender? If it is linux too, try to use default configuration not playing with
/proc/sys/net/tcp_*, especially with timestamps and sacks and the situation should rectify.
Unfortunately the other machine is my ISP, who I do not control. I have no idea what the sender OS is, but it's obviously not a standard recent Linux judging from the lack of SACK.
Quoted 1 line
Also, please, send full (binary!) tcpdump from SYN and to FIN.
I have send myself a 30k mail and produced a trace of receiving it. A binary trace with "tcpdump -n -w" is attached, and a text version of it with "tcpdump -r pktlog2" is appended below.
Quoted 1 line
Well, and if sender is not linux... no ideas.
I am wondering if not sending so many duplicate ACKs would help the broken sender (I know, that is against RFC793 but hey if it works...).
For the sake of a Linux test, I shall try proxying this stream via a Linux 2.4.2 box I have on the net, and see if it is an improvement.
-- Jamie
Quoted 1 line
I don't see what is broken about the remote end in this case.
The remote end is probably not broken, at least in this case. This looks like an artifact of a disgustingly large queue, and a very slow link. When the time to transmit tiny packets in the initial handshake is much smaller than the time to transmit a full size frame, retransmission timers can get confused. A complete trace would settle this.
I strongly recommend setting the mtu of your ppp0 interface down to 576 (or smaller) to reduce the time it takes to transfer a full size frame, decrease the likelihood that frames suffer corruption, and allow acknowledgements more often than every 5 seconds. This is a setting in your ppp configuration, don't do this using ifconfig.
Don't take my word for it, see RFC1144, section 5.2: a good MTU is chosen so that a full size frame is transferred in 200ms.
RFC1144, written by Van Jacobson:
To illustrate, note that for a 9600 bps line with header compression there is essentially no benefit in increasing the MTU beyond 200 bytes: If the MTU is increased to 576, the average delay increases by 188% while throughput only improves by 3% (from 96 to 99%).
Besides, if you do what you propose (not ack old data with the next byte expected) you risk stalling the connection entirely.
-neil
Hello!
Quoted 1 line
binary trace with "tcpdump -n -w" is attached,
I am afraid you forgot it.
Quoted 2 lines
I am wondering if not sending so many duplicate ACKs would help the broken sender (I know, that is against RFC793 but hey if it works...).
If it works, hack kernel not to send dupacks. What is the problem? :-) Yes, on such links fast retransmit is not useful in any case.
Actually, you can make something from your side: if rtt is so high, most likely this means excessive queueing. You can fight this advertising smaller window (f.e. adding option "window N" to route). If intepacket gap is 5sec and rtt 23, you can divide window by 23/5 and rtt should drop to some more reasonable number. "Theoretical" bandwidth will reduce, but practical will grow.
Quoted 2 lines
For the sake of a Linux test, I shall try proxying this stream via a Linux 2.4.2 box I have on the net, and see if it is an improvement.
Yes, it is intersting.
Alexey
Quoted 2 lines
Packets arrive _very_ slowly -- about 1 packet every 5.2 seconds. (Showing that "9600 baud" is even worse false advertising than "56k").
With my 7110 I get close to 8000bps throughput . To get that I ended up having to fix various flow control setting so the tiny buffers on the phone didnt keep overflowing
Quoted 1 line
The interesting thing is that there isn't any evidence of packet loss.
The modem/phone link does its own retransmit and recovery sequences. That means that on a very flaky link you _will_ see long pauses with retransmits at link layer going on
Quoted 3 lines · [email protected] wrote:
Quoted 1 line
binary trace with "tcpdump -n -w" is attached,
I am afraid you forgot it.
I have attached various files to this message. Hopefully the names make it clear which they are.
Quoted 3 lines
Quoted 2 lines
I am wondering if not sending so many duplicate ACKs would help the broken sender (I know, that is against RFC793 but hey if it works...).
If it works, hack kernel not to send dupacks. What is the problem? :-)
I hate rebooting :-)
Quoted 1 line
Yes, on such links fast retransmit is not useful in any case.
Should I turn of /proc/sys/net/ipv4/tcp_fack then?
Quoted 2 lines
Actually, you can make something from your side: if rtt is so high, most likely this means excessive queueing.
Yes, definitely. Btw, I saw a ping round trip time of 162s just now.
Quoted 11 lines
You can fight this
advertising smaller window (f.e. adding option "window N" to route). If intepacket gap is 5sec and rtt 23, you can divide window by 23/5 and rtt should drop to some more reasonable number. "Theoretical" bandwidth will reduce, but practical will grow.Quoted 2 lines
For the sake of a Linux test, I shall try proxying this stream via a Linux 2.4.2 box I have on the net, and see if it is an improvement.
Yes, it is intersting.
I've tried it, with no special settings (mtu or window or sack etc.) at either end. Local end is Linux 2.4.7, remote is 2.4.2-2 from Red Hat 7.1. The remote is proxying a connection to the final mailserver.
I saw very few retransmits in a single message download. SACK appears occasionally. I don't really understand the local reaction to SACK, or why a SACK option appears in one ACK sent locally and not the following ACK, even though the SACK mentions data that does not arrive between the two locally sent ACKs.
The throughput difference was obvious: POP3 negotiation + 30k message + headers took:
5 min 31 sec downloading unknown OS -> Linux 2.4.7
2 min 15 sec downloading Linux 2.4.2 -> Linux 2.4.7Looking at the traces, I expect the improvement on the straight data part is better than 2:1, because I saw so many retransmits with the other OS and hardly any with Linux as the server.
If someone would like to do an OS type probe on sdps.demon.co.uk, we will know which OS should be avoided for this kind of connection ;-)
It is still very slow considering it is advertised as "9600 baud" though :-(
cheers,
-- Jamie
19:43:06.892991 > 193.237.65.219.32855 > 158.152.1.58.domain: 48716+ A? sdps.demon.co.uk. (34) (DF)
19:43:08.952861 < 158.152.1.58.domain > 193.237.65.219.32855: 48716 5/4/4 CNAME pop3.mail.demon.net., A 194.217.242.59, A 194.217.242.22, (266) 19:43:08.989850 > 193.237.65.219.32855 > 158.152.1.58.domain: 48717+ AAAA? sdps.demon.co.uk. (34) (DF)
19:43:10.525449 < 158.152.1.58.domain > 193.237.65.219.32855: 48717 1/1/0 CNAME pop3.mail.demon.net. (114)
19:43:10.526084 > 193.237.65.219.32855 > 158.152.1.58.domain: 48718+ A? sdps.demon.co.uk. (34) (DF)
19:43:12.572859 < 158.152.1.58.domain > 193.237.65.219.32855: 48718 5/4/4 CNAME pop3.mail.demon.net., A 194.217.242.59, A 194.217.242.22, (266) 19:43:12.573433 > 193.237.65.219.32809 > 194.217.242.59.pop3: S 2296392844:2296392844(0) win 5840 <mss 1460,sackOK,timestamp 1404252 0,nop,wscale 0> (DF) 19:43:13.615650 < 194.217.242.59.pop3 > 193.237.65.219.32809: S 2152317245:2152317245(0) ack 2296392845 win 8760 <mss 1460> (DF) 19:43:13.615773 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 1:1(0) ack 1 win 5840 (DF)
19:43:14.632866 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 1:44(43) ack 1 win 8760 (DF)
19:43:14.632961 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 1:1(0) ack 44 win 5840 (DF)
19:43:14.649264 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 1:25(24) ack 44 win 5840 (DF)
19:43:15.672850 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 44:50(6) ack 25 win 8760 (DF)
19:43:15.685004 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 25:40(15) ack 50 win 5840 (DF)
19:43:16.702917 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 50:50(0) ack 40 win 8760 (DF)
19:43:17.203063 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 50:56(6) ack 40 win 8760 (DF)
19:43:17.242650 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 40:40(0) ack 56 win 5840 (DF)
19:43:20.222880 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 40:46(6) ack 56 win 5840 (DF)
19:43:21.233068 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 56:69(13) ack 46 win 8760 (DF)
19:43:21.233354 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 46:46(0) ack 69 win 5840 (DF)
19:43:21.251867 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 46:52(6) ack 69 win 5840 (DF)
19:43:22.272869 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 69:124(55) ack 52 win 8760 (DF)
19:43:22.308400 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 52:58(6) ack 124 win 5840 (DF)
19:43:23.322929 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 124:151(27) ack 58 win 8760 (DF)
19:43:23.349075 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 58:74(16) ack 151 win 5840 (DF)
19:43:27.102700 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 58:74(16) ack 151 win 5840 (DF)
19:43:29.732891 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 151:1611(1460) ack 74 win 8760 (DF)
19:43:29.733159 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 1611 win 8760 (DF)
19:43:34.982965 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 1611:3071(1460) ack 74 win 8760 (DF)
19:43:35.022675 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 3071 win 11680 (DF)
19:43:40.262962 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 3071:4531(1460) ack 74 win 8760 (DF)
19:43:40.263062 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 4531 win 14600 (DF)
19:43:45.512900 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 4531:5991(1460) ack 74 win 8760 (DF)
19:43:45.512991 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 5991 win 17520 (DF)
19:43:45.512905 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 5991:5991(0) ack 74 win 8760 (DF)
19:43:50.752866 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 151:1611(1460) ack 74 win 8760 (DF)
19:43:50.752954 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 5991 win 17520 (DF)
19:43:55.952900 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 1611:3071(1460) ack 74 win 8760 (DF)
19:43:55.952987 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 5991 win 17520 (DF)
19:44:01.162979 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 3071:4531(1460) ack 74 win 8760 (DF)
19:44:01.163072 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 5991 win 17520 (DF)
19:44:06.382817 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 5991:7451(1460) ack 74 win 8760 (DF)
19:44:06.382909 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 7451 win 20440 (DF)
19:44:11.602458 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 3071:4531(1460) ack 74 win 8760 (DF)
19:44:11.602546 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 7451 win 20440 (DF)
19:44:16.812904 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 4531:5991(1460) ack 74 win 8760 (DF)
19:44:16.812990 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 7451 win 20440 (DF)
19:44:22.002981 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 5991:7451(1460) ack 74 win 8760 (DF)
19:44:22.003076 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 7451 win 20440 (DF)
19:44:25.642892 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 7451:8342(891) ack 74 win 8760 (DF)
19:44:25.642985 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 8342 win 23360 (DF)
19:44:30.852962 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 5991:7451(1460) ack 74 win 8760 (DF)
19:44:30.853052 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 8342 win 23360 (DF)
19:44:36.062951 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 5991:7451(1460) ack 74 win 8760 (DF)
19:44:36.063046 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 8342 win 23360 (DF)
19:44:41.282732 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 5991:7451(1460) ack 74 win 8760 (DF)
19:44:41.282825 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 8342 win 23360 (DF)
19:44:44.402896 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 7451:8342(891) ack 74 win 8760 (DF)
19:44:44.402995 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 8342 win 23360 (DF)
19:44:49.612909 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 8342:9802(1460) ack 74 win 8760 (DF)
19:44:49.612996 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 9802 win 26280 (DF)
19:44:54.832979 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 9802:11262(1460) ack 74 win 8760 (DF)
19:44:54.833070 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 11262 win 29200 (DF)
19:45:00.052742 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 7451:8911(1460) ack 74 win 8760 (DF)
19:45:00.052831 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 11262 win 29200 (DF)
19:45:05.262884 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 7451:8911(1460) ack 74 win 8760 (DF)
19:45:05.262978 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 11262 win 29200 (DF)
19:45:10.482889 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 8342:9802(1460) ack 74 win 8760 (DF)
19:45:10.482979 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 11262 win 29200 (DF)
19:45:15.702883 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 11262:12722(1460) ack 74 win 8760 (DF)
19:45:15.702975 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 12722 win 32120 (DF)
19:45:20.922616 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 8342:9802(1460) ack 74 win 8760 (DF)
19:45:20.922706 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 12722 win 32120 (DF)
19:45:26.132863 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 8342:9802(1460) ack 74 win 8760 (DF)
19:45:26.132958 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 12722 win 32120 (DF)
19:45:31.352814 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 9802:11262(1460) ack 74 win 8760 (DF)
19:45:31.352898 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 12722 win 32120 (DF)
19:45:36.572622 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 12722:14182(1460) ack 74 win 8760 (DF)
19:45:36.572709 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 14182 win 35040 (DF)
19:45:41.772897 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 14182:15642(1460) ack 74 win 8760 (DF)
19:45:41.772985 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 15642 win 37960 (DF)
19:45:46.982994 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 11262:12722(1460) ack 74 win 8760 (DF)
19:45:46.983083 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 15642 win 37960 (DF)
19:45:50.102890 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 15642:16533(891) ack 74 win 8760 (DF)
19:45:50.102989 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 16533 win 37960 (DF)
19:45:55.328662 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 11262:12722(1460) ack 74 win 8760 (DF)
19:45:55.328755 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 16533 win 37960 (DF)
19:46:00.534606 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 12722:14182(1460) ack 74 win 8760 (DF)
19:46:00.534691 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 16533 win 37960 (DF)
19:46:05.777455 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 16533:17993(1460) ack 74 win 8760 (DF)
19:46:05.777543 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 17993 win 40880 (DF)
19:46:11.002902 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 17993:19453(1460) ack 74 win 8760 (DF)
19:46:11.002992 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:16.252870 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 12722:14182(1460) ack 74 win 8760 (DF)
19:46:16.252965 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:21.482878 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 12722:14182(1460) ack 74 win 8760 (DF)
19:46:21.482967 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:26.732886 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 14182:15642(1460) ack 74 win 8760 (DF)
19:46:26.732977 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:31.982906 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 15642:17102(1460) ack 74 win 8760 (DF)
19:46:31.982992 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:37.234530 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 16533:17993(1460) ack 74 win 8760 (DF)
19:46:37.234617 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:42.472760 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 17993:19453(1460) ack 74 win 8760 (DF)
19:46:42.472856 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 19453 win 43800 (DF)
19:46:47.687936 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 19453:20913(1460) ack 74 win 8760 (DF)
19:46:47.688029 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 20913 win 46720 (DF)
19:46:52.932888 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 20913:22373(1460) ack 74 win 8760 (DF)
19:46:52.932976 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 22373 win 49640 (DF)
19:46:58.172866 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 19453:20913(1460) ack 74 win 8760 (DF)
19:46:58.172953 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 22373 win 49640 (DF)
19:47:03.402904 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 22373:23833(1460) ack 74 win 8760 (DF)
19:47:03.402993 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 23833 win 52560 (DF)
19:47:08.652863 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 19453:20913(1460) ack 74 win 8760 (DF)
19:47:08.652952 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 23833 win 52560 (DF)
19:47:14.414511 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 19453:20913(1460) ack 74 win 8760 (DF)
19:47:14.414604 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 23833 win 52560 (DF)
19:47:19.622913 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 20913:22373(1460) ack 74 win 8760 (DF)
19:47:19.623003 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 23833 win 52560 (DF)
19:47:23.282880 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 23833:24725(892) ack 74 win 8760 (DF)
19:47:23.282980 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 24725 win 52560 (DF)
19:47:28.522888 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 22373:23833(1460) ack 74 win 8760 (DF)
19:47:28.522976 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 24725 win 52560 (DF)
19:47:34.272886 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 24725:26185(1460) ack 74 win 8760 (DF)
19:47:34.272981 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 26185 win 55480 (DF)
19:47:40.002881 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 23833:25293(1460) ack 74 win 8760 (DF)
19:47:40.002971 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 26185 win 55480 (DF)
19:47:45.202902 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 26185:27645(1460) ack 74 win 8760 (DF)
19:47:45.202998 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 27645 win 58400 (DF)
19:47:50.394404 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 27645:29105(1460) ack 74 win 8760 (DF)
19:47:50.394500 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 29105 win 61320 (DF)
19:47:56.121837 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 23833:25293(1460) ack 74 win 8760 (DF)
19:47:56.121944 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 29105 win 61320 (DF)
19:48:01.302918 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 24725:26185(1460) ack 74 win 8760 (DF)
19:48:01.303011 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 29105 win 61320 (DF)
19:48:07.022900 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 24725:26185(1460) ack 74 win 8760 (DF)
19:48:07.022992 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 29105 win 61320 (DF)
19:48:12.212914 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 26185:27645(1460) ack 74 win 8760 (DF)
19:48:12.213001 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 29105 win 61320 (DF)
19:48:17.432888 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 27645:29105(1460) ack 74 win 8760 (DF)
19:48:17.432985 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 29105 win 61320 (DF)
19:48:22.642896 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 29105:30565(1460) ack 74 win 8760 (DF)
19:48:22.642984 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 30565 win 62780 (DF)
19:48:24.212871 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 30565:31088(523) ack 74 win 8760 (DF)
19:48:24.212966 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 74:74(0) ack 31088 win 62780 (DF)
19:48:24.245885 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 74:82(8) ack 31088 win 62780 (DF)
19:48:29.452732 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 29105:30565(1460) ack 74 win 8760 (DF)
19:48:29.452827 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 82:82(0) ack 31088 win 62780 (DF)
19:48:31.762704 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 74:82(8) ack 31088 win 62780 (DF)
19:48:34.882885 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 29105:30565(1460) ack 74 win 8760 (DF)
19:48:34.882977 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 82:82(0) ack 31088 win 62780 (DF)
19:48:36.452866 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 30565:31088(523) ack 74 win 8760 (DF)
19:48:36.452978 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 82:82(0) ack 31088 win 62780 (DF)
19:48:36.452873 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 31088:31094(6) ack 82 win 8760 (DF)
19:48:36.453045 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 82:82(0) ack 31094 win 62780 (DF)
19:48:36.464298 > 193.237.65.219.32809 > 194.217.242.59.pop3: P 82:88(6) ack 31094 win 62780 (DF)
19:48:36.474321 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 31088:31094(6) ack 82 win 8760 (DF)
19:48:36.474421 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 88:88(0) ack 31094 win 62780 (DF)
19:48:36.482923 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 31094:31094(0) ack 82 win 8760 (DF)
19:48:36.502875 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 31088:31094(6) ack 82 win 8760 (DF)
19:48:36.502936 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 88:88(0) ack 31094 win 62780 (DF)
19:48:36.502880 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 31088:31094(6) ack 82 win 8760 (DF)
19:48:36.502992 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 88:88(0) ack 31094 win 62780 (DF)
19:48:37.546707 < 194.217.242.59.pop3 > 193.237.65.219.32809: P 31094:31100(6) ack 88 win 8760 (DF)
19:48:37.546812 > 193.237.65.219.32809 > 194.217.242.59.pop3: . 88:88(0) ack 31100 win 62780 (DF)
19:48:37.546714 < 194.217.242.59.pop3 > 193.237.65.219.32809: F 31100:31100(0) ack 88 win 8760 (DF)
19:48:37.556924 > 193.237.65.219.32809 > 194.217.242.59.pop3: F 88:88(0) ack 31101 win 62780 (DF)
19:48:38.571482 < 194.217.242.59.pop3 > 193.237.65.219.32809: . 31101:31101(0) ack 89 win 8760 (DF)
20:49:10.535879 > 193.237.65.219.32857 > 158.152.1.58.domain: 4313+ A? sdps.demon.co.uk. (34) (DF)
20:49:12.607387 < 158.152.1.58.domain > 193.237.65.219.32857: 4313 5/4/4 CNAME pop3.mail.demon.net., A 194.217.242.58, A 194.217.242.59, (266) 20:49:13.112862 < 194.217.242.7.50453 > 193.237.65.219.smtp: S 3979673482:3979673482(0) win 8760 <mss 1460> (DF)
20:49:13.114345 > 193.237.65.219 > 194.217.242.7: icmp: 193.237.65.219 tcp port smtp unreachable [tos 0xc0]
20:49:13.666321 > 193.237.65.219.32833 > 217.33.12.64.4000: S 2192994338:2192994338(0) win 5840 <mss 1460,sackOK,timestamp 1800361 0,nop,wscale 0> (DF) 20:49:14.689811 < 217.33.12.64.4000 > 193.237.65.219.32833: S 1422559437:1422559437(0) ack 2192994339 win 5792 <mss 1460,sackOK,timestamp 18895656 1800361,nop,wscale 0> (DF) 20:49:14.689916 > 193.237.65.219.32833 > 217.33.12.64.4000: . 1:1(0) ack 1 win 5840 <nop,nop,timestamp 1800463 18895656> (DF)
20:49:15.722871 < 217.33.12.64.4000 > 193.237.65.219.32833: P 1:44(43) ack 1 win 5792 <nop,nop,timestamp 18895765 1800463> (DF)
20:49:15.722963 > 193.237.65.219.32833 > 217.33.12.64.4000: . 1:1(0) ack 44 win 5840 <nop,nop,timestamp 1800567 18895765> (DF)
20:49:15.759946 > 193.237.65.219.32833 > 217.33.12.64.4000: P 1:25(24) ack 44 win 5840 <nop,nop,timestamp 1800570 18895765> (DF)
20:49:16.792932 < 217.33.12.64.4000 > 193.237.65.219.32833: . 44:44(0) ack 25 win 5792 <nop,nop,timestamp 18895869 1800570> (DF)
20:49:16.792940 < 217.33.12.64.4000 > 193.237.65.219.32833: P 44:50(6) ack 25 win 5792 <nop,nop,timestamp 18895871 1800570> (DF)
20:49:16.813410 > 193.237.65.219.32833 > 217.33.12.64.4000: P 25:40(15) ack 50 win 5840 <nop,nop,timestamp 1800676 18895871> (DF)
20:49:17.832858 < 217.33.12.64.4000 > 193.237.65.219.32833: . 50:50(0) ack 40 win 5792 <nop,nop,timestamp 18895977 1800676> (DF)
20:49:18.342863 < 217.33.12.64.4000 > 193.237.65.219.32833: P 50:56(6) ack 40 win 5792 <nop,nop,timestamp 18895986 1800676> (DF)
20:49:18.382680 > 193.237.65.219.32833 > 217.33.12.64.4000: . 40:40(0) ack 56 win 5840 <nop,nop,timestamp 1800833 18895986> (DF)
20:49:21.377055 > 193.237.65.219.32833 > 217.33.12.64.4000: P 40:46(6) ack 56 win 5840 <nop,nop,timestamp 1801132 18895986> (DF)
20:49:22.902855 < 217.33.12.64.4000 > 193.237.65.219.32833: . 56:56(0) ack 46 win 5792 <nop,nop,timestamp 18896469 1801132> (DF)
20:49:23.412846 < 217.33.12.64.4000 > 193.237.65.219.32833: P 56:69(13) ack 46 win 5792 <nop,nop,timestamp 18896471 1801132> (DF)
20:49:23.412935 > 193.237.65.219.32833 > 217.33.12.64.4000: . 46:46(0) ack 69 win 5840 <nop,nop,timestamp 1801336 18896471> (DF)
20:49:23.433965 > 193.237.65.219.32833 > 217.33.12.64.4000: P 46:52(6) ack 69 win 5840 <nop,nop,timestamp 1801338 18896471> (DF)
20:49:24.462871 < 217.33.12.64.4000 > 193.237.65.219.32833: P 69:124(55) ack 52 win 5792 <nop,nop,timestamp 18896638 1801338> (DF) 20:49:24.511822 > 193.237.65.219.32833 > 217.33.12.64.4000: . 52:52(0) ack 124 win 5840 <nop,nop,timestamp 1801445 18896638> (DF)
20:49:24.517953 > 193.237.65.219.32833 > 217.33.12.64.4000: P 52:58(6) ack 124 win 5840 <nop,nop,timestamp 1801446 18896638> (DF)
20:49:25.532880 < 217.33.12.64.4000 > 193.237.65.219.32833: P 124:151(27) ack 58 win 5792 <nop,nop,timestamp 18896752 1801446> (DF) 20:49:25.532976 > 193.237.65.219.32833 > 217.33.12.64.4000: . 58:58(0) ack 151 win 5840 <nop,nop,timestamp 1801548 18896752> (DF)
20:49:25.581259 > 193.237.65.219.32833 > 217.33.12.64.4000: P 58:74(16) ack 151 win 5840 <nop,nop,timestamp 1801552 18896752> (DF) 20:49:26.605437 < 217.33.12.64.4000 > 193.237.65.219.32833: . 151:151(0) ack 74 win 5792 <nop,nop,timestamp 18896860 1801552> (DF) 20:49:32.302918 < 217.33.12.64.4000 > 193.237.65.219.32833: . 151:1599(1448) ack 74 win 5792 <nop,nop,timestamp 18896860 1801552> (DF) 20:49:32.303020 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 1599 win 8688 <nop,nop,timestamp 1802225 18896860> (DF) 20:49:32.302923 < 217.33.12.64.4000 > 193.237.65.219.32833: P 1599:1611(12) ack 74 win 5792 <nop,nop,timestamp 18896860 1801552> (DF) 20:49:32.303081 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 1611 win 8688 <nop,nop,timestamp 1802225 18896860> (DF) 20:49:39.072902 < 217.33.12.64.4000 > 193.237.65.219.32833: . 1611:3059(1448) ack 74 win 5792 <nop,nop,timestamp 18897529 1802225> (DF) 20:49:39.073002 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 3059 win 11584 <nop,nop,timestamp 1802902 18897529> (DF) 20:49:44.319061 < 217.33.12.64.4000 > 193.237.65.219.32833: . 3059:4507(1448) ack 74 win 5792 <nop,nop,timestamp 18897529 1802225> (DF) 20:49:44.319159 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 4507 win 14480 <nop,nop,timestamp 1803426 18897529> (DF) 20:49:49.561886 < 217.33.12.64.4000 > 193.237.65.219.32833: . 4507:5955(1448) ack 74 win 5792 <nop,nop,timestamp 18897529 1802225> (DF) 20:49:49.561979 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 5955 win 17376 <nop,nop,timestamp 1803950 18897529> (DF) 20:49:54.823304 < 217.33.12.64.4000 > 193.237.65.219.32833: . 5955:7403(1448) ack 74 win 5792 <nop,nop,timestamp 18897529 1802225> (DF) 20:49:54.823394 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 7403 win 20272 <nop,nop,timestamp 1804477 18897529> (DF) 20:50:00.072892 < 217.33.12.64.4000 > 193.237.65.219.32833: . 7403:8851(1448) ack 74 win 5792 <nop,nop,timestamp 18898211 1802902> (DF) 20:50:00.072990 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 8851 win 23168 <nop,nop,timestamp 1805002 18898211> (DF) 20:50:05.341269 < 217.33.12.64.4000 > 193.237.65.219.32833: . 8851:10299(1448) ack 74 win 5792 <nop,nop,timestamp 18898211 1802902> (DF) 20:50:05.341362 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 10299 win 26064 <nop,nop,timestamp 1805528 18898211> (DF) 20:50:10.602470 < 217.33.12.64.4000 > 193.237.65.219.32833: P 10299:11747(1448) ack 74 win 5792 <nop,nop,timestamp 18898735 1803426> (DF) 20:50:10.602561 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 11747 win 28960 <nop,nop,timestamp 1806054 18898735> (DF) 20:50:14.272884 < 217.33.12.64.4000 > 193.237.65.219.32833: P 11747:12722(975) ack 74 win 5792 <nop,nop,timestamp 18898735 1803426> (DF) 20:50:14.272983 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 12722 win 31856 <nop,nop,timestamp 1806422 18898735> (DF) 20:50:19.523616 < 217.33.12.64.4000 > 193.237.65.219.32833: . 12722:14170(1448) ack 74 win 5792 <nop,nop,timestamp 18899260 1803950> (DF) 20:50:19.523710 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 14170 win 34752 <nop,nop,timestamp 1806947 18899260> (DF) 20:50:24.785028 < 217.33.12.64.4000 > 193.237.65.219.32833: . 14170:15618(1448) ack 74 win 5792 <nop,nop,timestamp 18899260 1803950> (DF) 20:50:24.785119 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 15618 win 37648 <nop,nop,timestamp 1807473 18899260> (DF) 20:50:30.032878 < 217.33.12.64.4000 > 193.237.65.219.32833: . 15618:17066(1448) ack 74 win 5792 <nop,nop,timestamp 18899787 1804477> (DF) 20:50:30.032969 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 17066 win 40544 <nop,nop,timestamp 1807998 18899787> (DF) 20:50:35.302882 < 217.33.12.64.4000 > 193.237.65.219.32833: . 17066:18514(1448) ack 74 win 5792 <nop,nop,timestamp 18899787 1804477> (DF) 20:50:35.302970 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 18514 win 43440 <nop,nop,timestamp 1808525 18899787> (DF) 20:50:40.562865 < 217.33.12.64.4000 > 193.237.65.219.32833: . 18514:19962(1448) ack 74 win 5792 <nop,nop,timestamp 18900315 1805002> (DF) 20:50:40.562955 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 19962 win 46336 <nop,nop,timestamp 1809051 18900315> (DF) 20:50:45.792976 < 217.33.12.64.4000 > 193.237.65.219.32833: . 19962:21410(1448) ack 74 win 5792 <nop,nop,timestamp 18900315 1805002> (DF) 20:50:45.793066 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 21410 win 49232 <nop,nop,timestamp 1809574 18900315> (DF) 20:50:51.552902 < 217.33.12.64.4000 > 193.237.65.219.32833: P 21410:22858(1448) ack 74 win 5792 <nop,nop,timestamp 18900851 1805528> (DF) 20:50:51.552994 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 22858 win 52128 <nop,nop,timestamp 1810150 18900851> (DF) 20:50:56.792977 < 217.33.12.64.4000 > 193.237.65.219.32833: . 22858:24306(1448) ack 74 win 5792 <nop,nop,timestamp 18900851 1805528> (DF) 20:50:56.793066 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 24306 win 55024 <nop,nop,timestamp 1810674 18900851> (DF) 20:51:02.032865 < 217.33.12.64.4000 > 193.237.65.219.32833: . 24306:25754(1448) ack 74 win 5792 <nop,nop,timestamp 18901377 1806054> (DF) 20:51:02.032955 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 25754 win 57920 <nop,nop,timestamp 1811198 18901377> (DF) 20:51:07.242888 < 217.33.12.64.4000 > 193.237.65.219.32833: . 25754:27202(1448) ack 74 win 5792 <nop,nop,timestamp 18901377 1806054> (DF) 20:51:07.242982 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 27202 win 60816 <nop,nop,timestamp 1811719 18901377> (DF) 20:51:12.962860 < 217.33.12.64.4000 > 193.237.65.219.32833: . 27202:28650(1448) ack 74 win 5792 <nop,nop,timestamp 18901743 1806422> (DF) 20:51:12.962946 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 28650 win 63712 <nop,nop,timestamp 1812291 18901743> (DF) 20:51:18.182724 < 217.33.12.64.4000 > 193.237.65.219.32833: . 28650:30098(1448) ack 74 win 5792 <nop,nop,timestamp 18901743 1806422> (DF) 20:51:18.182819 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 30098 win 63712 <nop,nop,timestamp 1812813 18901743> (DF) 20:51:21.822887 < 217.33.12.64.4000 > 193.237.65.219.32833: P 30098:31062(964) ack 74 win 5792 <nop,nop,timestamp 18902269 1806947> (DF) 20:51:21.822974 > 193.237.65.219.32833 > 217.33.12.64.4000: . 74:74(0) ack 31062 win 63712 <nop,nop,timestamp 1813177 18902269> (DF) 20:51:21.890387 > 193.237.65.219.32833 > 217.33.12.64.4000: P 74:82(8) ack 31062 win 63712 <nop,nop,timestamp 1813183 18902269> (DF) 20:51:22.922519 < 217.33.12.64.4000 > 193.237.65.219.32833: . 31062:31062(0) ack 82 win 5792 <nop,nop,timestamp 18908505 1813183> (DF) 20:51:22.922527 < 217.33.12.64.4000 > 193.237.65.219.32833: P 31062:31068(6) ack 82 win 5792 <nop,nop,timestamp 18908507 1813183> (DF) 20:51:22.922648 > 193.237.65.219.32833 > 217.33.12.64.4000: . 82:82(0) ack 31068 win 63712 <nop,nop,timestamp 1813287 18908507> (DF) 20:51:22.948645 > 193.237.65.219.32833 > 217.33.12.64.4000: P 82:88(6) ack 31068 win 63712 <nop,nop,timestamp 1813289 18908507> (DF) 20:51:23.992468 < 217.33.12.64.4000 > 193.237.65.219.32833: P 31068:31074(6) ack 88 win 5792 <nop,nop,timestamp 18908616 1813289> (DF) 20:51:23.992475 < 217.33.12.64.4000 > 193.237.65.219.32833: F 31074:31074(0) ack 88 win 5792 <nop,nop,timestamp 18908616 1813289> (DF) 20:51:23.992729 > 193.237.65.219.32833 > 217.33.12.64.4000: F 88:88(0) ack 31075 win 63712 <nop,nop,timestamp 1813394 18908616> (DF) 20:51:25.012863 < 217.33.12.64.4000 > 193.237.65.219.32833: . 31075:31075(0) ack 89 win 5792 <nop,nop,timestamp 18908718 1813394> (DF)
Quoted 4 lines · Jamie Lokier
If someone would like to do an OS type probe on sdps.demon.co.uk, we will know which OS should be avoided for this kind of connection ;-)
it's running solaris 2.5 or 2.5.1
Signature · 2 lines
Lukas Beeler <[email protected]> GPG Fingerprint: 8030 1C2F 66C5 9D80 AA31 6604 7D4D 0A67 68D8 B67E
Hello!
Quoted 1 line
I hate rebooting :-)
Relax. Really, this will not change anything. After more careful look into the first tcpdump, I did not find any signs of false fast retransmits.
All the problem is due to wrong rtt estimator at sender.
Quoted 3 lines
Quoted 1 line
Yes, on such links fast retransmit is not useful in any case.
Should I turn of
/proc/sys/net/ipv4/tcp_fackthen?
No. Defaults are defaults, because they are the best defaults. :-)
Quoted 1 line
Yes, definitely. Btw, I saw a ping round trip time of 162s just now.
I do not understand, do you share this link with someone or ping over tcp connection?
If the last is true, reduce window to 4K, maximum 8K. Default 64K, combined with misconfigured queues and/or with broken error correction is disaster.
Actually, you dump shows that window is not open.
Quoted 5 lines
I saw very few retransmits in a single message download. SACK appears occasionally. I don't really understand the local reaction to SACK, or why a SACK option appears in one ACK sent locally and not the following ACK, even though the SACK mentions data that does not arrive between the two locally sent ACKs.
But I do not see _any_ sacks in your tcpdumps.
Quoted 5 lines
The throughput difference was obvious: POP3 negotiation + 30k message + headers took:
5 min 31 sec downloading unknown OS -> Linux 2.4.7 2 min 15 sec downloading Linux 2.4.2 -> Linux 2.4.7
It is dominated by rtt, one rtt per segment. It is very strange that cwnd does not want to open. Maybe, it is worth to tcpdump at proxy.
Alexey
Quoted 4 lines · [email protected] wrote:
Quoted 1 line
Yes, definitely. Btw, I saw a ping round trip time of 162s just now.
I do not understand, do you share this link with someone or ping over tcp connection?
I was doing ping with the TCP connection going. When there is no TCP connection, ping time is 1-1.5 seconds.
Quoted 7 lines
Quoted 5 lines
I saw very few retransmits in a single message download. SACK appears occasionally. I don't really understand the local reaction to SACK, or why a SACK option appears in one ACK sent locally and not the following ACK, even though the SACK mentions data that does not arrive between the two locally sent ACKs.
But I do not see _any_ sacks in your tcpdumps.
Sorry, you are right. I did see an sack, but not in this trace.
Quoted 8 lines
Quoted 5 lines
The throughput difference was obvious: POP3 negotiation + 30k message + headers took:
5 min 31 sec downloading unknown OS -> Linux 2.4.7 2 min 15 sec downloading Linux 2.4.2 -> Linux 2.4.7It is dominated by rtt, one rtt per segment. It is very strange that cwnd does not want to open. Maybe, it is worth to tcpdump at proxy.
Do you mean that you want to see the Linux -> Linux connection, with tcpdumps at both ends?
-- Jamie
Hello!
Quoted 2 lines
Do you mean that you want to see the Linux -> Linux connection, with tcpdumps at both ends?
At the end of sender. Receiver side is not so interesting.
Well, if you were able to make it with solaris this would be also interesting. It would be clear how exactly it miscalculates rtt at least. :-) But Linux is interesting too. Theoretically it should show better times even with so short transfer. cwnd stays at 1 and this is very strange.
Alexey