Skip to contentSkip to conversation

TCP stack behaviour question

DISCUSSION Stuart MacDonald 3 participants 4 messages Sep 18, 2006 – Sep 20, 2006

lore
Share & tools
lore
#0 lore raw

From: Stuart MacDonald [mailto:[email protected]]

Quoted 8 lines

What happened was this: I had a run where I captured output with tcpdump. My original post was based on that, and the results of the debug output from my app. For whatever reason, it appears the stack didn't generate all of the packets it should have. When the log showed a second-last to last retransmit time of about 27 seconds, and then a gap of about 400 to the very next packet of any kind, I assumed that meant the stack had given up on the retransmits when it appears something else was going on.

I did another run and confirmed this. The tcpdump capture shows that seven retransmits are sent, obeying the exponential backoff. Then something odd happens. Instead of the 8th retransmit at 7th + 26.88 seconds, there is an arp at 7th + 4.159722 seconds. There are three arps in fact, each one second apart and directed to the MAC of the powered-off machine. After this there are further arps (in groups of three one second apart), but they are broadcast and have a backoff schedule.

The kernel debugging shows that tcp_write_timeout() and tcp_retransmit_timer() are still being called though, right up to what would be the 16th retransmit.

I suppose that the TCP retransmits aren't being sent because the ethernet and/or IP layers don't know what's going on, which is what's producing the arps. Is that correct? Is that documented anywhere?

I was expecting to see all 15 retransmits, and was confused when I didn't see them.

..Stu

#1 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Stuart" == Stuart MacDonald <[email protected]> writes:

Stuart> I suppose that the TCP retransmits aren't being sent because Stuart> the ethernet and/or IP layers don't know what's going on, Stuart> which is what's producing the arps. Is that correct?

It seems correct. You cannot expect TCP packets to be sent if the target is supposedly on a directly connected network and ARP cannot get its MAC address. What should the IP layer put as the MAC address if it is unknown?

You may want to run another test with another unreachable target located after a router, so that the MAC address of the router is used on the wire. You should see all the TCP retransmits you expect to see.

Sam

Signature · 1 line
Samuel Tardieu -- [email protected] -- http://www.rfc1149.net/
#2 lore raw

From: On Behalf Of Samuel Tardieu

Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Stuart" == Stuart MacDonald <[email protected]> writes:

Stuart> I suppose that the TCP retransmits aren't being sent because Stuart> the ethernet and/or IP layers don't know what's going on, Stuart> which is what's producing the arps. Is that correct?

It seems correct. You cannot expect TCP packets to be sent if the target is supposedly on a directly connected network and ARP cannot get its MAC address. What should the IP layer put as the MAC address if it is unknown?

I'd agree, but the MAC is/was known. Only part way through the sequence of retransmits (with MAC filled in) do the arps start. Also, tellingly maybe, the first three arps are sent directly to the unreachable MAC. Only after those are unanswered are broadcast arps attempted.

Ah, interesting, see below. ***

Quoted 3 lines

You may want to run another test with another unreachable target located after a router, so that the MAC address of the router is used on the wire. You should see all the TCP retransmits you expect to see.

I'll try that if I have time, but I agree, I expect to see all the retransmits in that case.

*** I found arp(7) and read up on it while I was typing. And now I see something interesting in the tcpdump; my app is actually talking on two TCP connections at the same time. Both are in retransmit phase, and the first arp is 5 seconds (delay_first_probe_time) after an _aggregate total_ of 15 retransmits (being the two original unanswered packets and 7 and 6 retransmits of each).

My reading of tcp(7)'s documentation of tcp_retries2 is that tcp_retries2 is a per-TCP packet count. My tcpdump seems to show that it is in fact a global count. Which is correct?

..Stu

#3 lore raw
Quoted 9 lines · Stuart MacDonald

*** I found arp(7) and read up on it while I was typing. And now I see something interesting in the tcpdump; my app is actually talking on two TCP connections at the same time. Both are in retransmit phase, and the first arp is 5 seconds (delay_first_probe_time) after an _aggregate total_ of 15 retransmits (being the two original unanswered packets and 7 and 6 retransmits of each).

Quoted 3 lines

My reading of tcp(7)'s documentation of tcp_retries2 is that tcp_retries2 is a per-TCP packet count. My tcpdump seems to show that it is in fact a global count. Which is correct?

The ARP layer keeps track of what neighbours are reachable and doesn't transmit packets to unreachable ones before they answer unicast or broadcast ARP. This is a state machine borrowed from IPv6.

There is nothing global.

-Andi