Skip to contentSkip to conversation

Failure to send fragmented IP packet in case of missing ARP entry

DISCUSSION Andrei Dolnikov 2 participants 4 messages Sep 10, 2012 – Sep 12, 2012

Subsystems (3)
Networking Network drivers IPv4/IPv6
lore
Share & tools
lore

Andrei Dolnikov

author
#0 lore raw

Hello all,

The following issue is observed on most Linux distributions:
Transmission of fragmented IP packets in case of missing ARP entry for destination IP fails.
Actually ARP request is sent, and, once ARP response is received, only few queued fragments are transmitted. Remaining fragments are lost. It can be easily reproduced as follows:

    # arp -d <dst IP>
    # ping -s 65000 -c 1 <dst IP>
Ping result is: "1 packets transmitted, 0 received, 100% packet loss, time 0ms".

The latest kernel version I tried was 3.5.0-1 x86_64, but I also was able to reproduce it with 3.2.x, 3.0.x and 2.6.32.
It doesn't depend on hardware: was able to reproduce with VMWare Player, Intel based laptop, Intel Atom and ARM based custom boards.
As I'm not a networking standards expert I'm not sure if it's a real bug or acceptable behaviour, but decided to raise the issue here as I can't reproduce this anomaly with the Windows 7 PC.

Thanks,
Andrei.

#1 lore raw
Quoted 21 lines · Andrei Dolnikov

Hello all,

The following issue is observed on most Linux distributions:
Transmission of fragmented IP packets in case of missing ARP entry for destination IP fails.
Actually ARP request is sent, and, once ARP response is received, only few queued fragments are transmitted. Remaining fragments are lost. It can be easily reproduced as follows:

     # arp -d <dst IP>
     # ping -s 65000 -c 1 <dst IP>
Ping result is: "1 packets transmitted, 0 received, 100% packet loss, 
time 0ms".

The latest kernel version I tried was 3.5.0-1 x86_64, but I also was able to reproduce it with 3.2.x, 3.0.x and 2.6.32.
It doesn't depend on hardware: was able to reproduce with VMWare Player, Intel based laptop, Intel Atom and ARM based custom boards. As I'm not a networking standards expert I'm not sure if it's a real bug or acceptable behaviour, but decided to raise the issue here as I can't reproduce this anomaly with the Windows 7 PC.

Quoted 3 lines

Thanks,
Andrei.
--

Its a bit better with linux-3.3, with commit
8b5c171bb3dc0686b2647a84e990199c5faa9ef8
(neigh: new unresolved queue limits)

+neigh/default/unres_qlen_bytes - INTEGER

+       The maximum number of bytes which may be used by packets
+       queued for each unresolved address by other network layers.
+       (added in linux 3.3)
+
+neigh/default/unres_qlen - INTEGER
+       The maximum number of packets which may be queued for each
+       unresolved address by other network layers.
+       (deprecated in linux 3.3) : use unres_qlen_bytes instead.

Problem is : unres_qlen_bytes default value is 65536, so its a bit too small once you take into account truesize overhead

I guess following patch would be needed :

net/ipv4/arp.c +1 -1
index 4780045..3395bb6 100644
@@ -171,7 +171,7 @@ struct neigh_table arp_tbl = {
 		.gc_staletime		= 60 * HZ,
 		.reachable_time		= 30 * HZ,
 		.delay_probe_time	= 5 * HZ,
-		.queue_len_bytes	= 64*1024,
+		.queue_len_bytes	= 64 * SKB_TRUESIZE(1024),
 		.ucast_probes		= 3,
 		.mcast_probes		= 3,
 		.anycast_delay		= 1 * HZ,
#2 lore raw
Quoted 51 lines · Eric Dumazet
Quoted 21 lines · Andrei Dolnikov

Hello all,

The following issue is observed on most Linux distributions:
Transmission of fragmented IP packets in case of missing ARP entry for destination IP fails.
Actually ARP request is sent, and, once ARP response is received, only few queued fragments are transmitted. Remaining fragments are lost. It can be easily reproduced as follows:

     # arp -d <dst IP>
     # ping -s 65000 -c 1 <dst IP>
Ping result is: "1 packets transmitted, 0 received, 100% packet loss, 
time 0ms".

The latest kernel version I tried was 3.5.0-1 x86_64, but I also was able to reproduce it with 3.2.x, 3.0.x and 2.6.32.
It doesn't depend on hardware: was able to reproduce with VMWare Player, Intel based laptop, Intel Atom and ARM based custom boards. As I'm not a networking standards expert I'm not sure if it's a real bug or acceptable behaviour, but decided to raise the issue here as I can't reproduce this anomaly with the Windows 7 PC.

Quoted 3 lines

Thanks,
Andrei.
--

Its a bit better with linux-3.3, with commit
8b5c171bb3dc0686b2647a84e990199c5faa9ef8
(neigh: new unresolved queue limits)

(quoted diff) +9 -0
+neigh/default/unres_qlen_bytes - INTEGER
+       The maximum number of bytes which may be used by packets
+       queued for each unresolved address by other network layers.
+       (added in linux 3.3)
+
+neigh/default/unres_qlen - INTEGER
+       The maximum number of packets which may be queued for each
+       unresolved address by other network layers.
+       (deprecated in linux 3.3) : use unres_qlen_bytes instead.

Problem is : unres_qlen_bytes default value is 65536, so its a bit too small once you take into account truesize overhead

I guess following patch would be needed :

net/ipv4/arp.c +0 -0
index 4780045..3395bb6 100644

net/ipv4/arp.c struct neigh_table arp_tbl = { · line 177

Quoted 8 lines
net/ipv4/arp.c +1 -1
@@ -171,7 +171,7 @@ struct neigh_table arp_tbl = {
 		.gc_staletime		= 60 * HZ,
 		.reachable_time		= 30 * HZ,
 		.delay_probe_time	= 5 * HZ,
-		.queue_len_bytes	= 64*1024,
+		.queue_len_bytes	= 64 * SKB_TRUESIZE(1024),
 		.ucast_probes		= 3,
 		.mcast_probes		= 3,
 		.anycast_delay		= 1 * HZ,

In the mean time, you can also do

echo 50 >/proc/sys/net/ipv4/neigh/eth0/unres_qlen

(change eth0 by the name of your interface)

author
#3 lore raw

Works for me.

Thank you!
Andrei.

Quoted 67 lines · Eric Dumazet
Quoted 50 lines · Eric Dumazet
Quoted 21 lines · Andrei Dolnikov

Hello all,

The following issue is observed on most Linux distributions:
Transmission of fragmented IP packets in case of missing ARP entry for destination IP fails.
Actually ARP request is sent, and, once ARP response is received, only few queued fragments are transmitted. Remaining fragments are lost. It can be easily reproduced as follows:

      # arp -d <dst IP>
      # ping -s 65000 -c 1 <dst IP>
Ping result is: "1 packets transmitted, 0 received, 100% packet loss,
time 0ms".

The latest kernel version I tried was 3.5.0-1 x86_64, but I also was able to reproduce it with 3.2.x, 3.0.x and 2.6.32.
It doesn't depend on hardware: was able to reproduce with VMWare Player, Intel based laptop, Intel Atom and ARM based custom boards. As I'm not a networking standards expert I'm not sure if it's a real bug or acceptable behaviour, but decided to raise the issue here as I can't reproduce this anomaly with the Windows 7 PC.

Quoted 3 lines

Thanks,
Andrei.
--

Its a bit better with linux-3.3, with commit
8b5c171bb3dc0686b2647a84e990199c5faa9ef8
(neigh: new unresolved queue limits)

(quoted diff) +9 -0
+neigh/default/unres_qlen_bytes - INTEGER
+       The maximum number of bytes which may be used by packets
+       queued for each unresolved address by other network layers.
+       (added in linux 3.3)
+
+neigh/default/unres_qlen - INTEGER
+       The maximum number of packets which may be queued for each
+       unresolved address by other network layers.
+       (deprecated in linux 3.3) : use unres_qlen_bytes instead.

Problem is : unres_qlen_bytes default value is 65536, so its a bit too small once you take into account truesize overhead

I guess following patch would be needed :

net/ipv4/arp.c +0 -0
index 4780045..3395bb6 100644

net/ipv4/arp.c struct neigh_table arp_tbl = { · line 177

Quoted 8 lines
net/ipv4/arp.c +1 -1
@@ -171,7 +171,7 @@ struct neigh_table arp_tbl = {
  		.gc_staletime		= 60 * HZ,
  		.reachable_time		= 30 * HZ,
  		.delay_probe_time	= 5 * HZ,
-		.queue_len_bytes	= 64*1024,
+		.queue_len_bytes	= 64 * SKB_TRUESIZE(1024),
  		.ucast_probes		= 3,
  		.mcast_probes		= 3,
  		.anycast_delay		= 1 * HZ,

In the mean time, you can also do

echo 50 >/proc/sys/net/ipv4/neigh/eth0/unres_qlen

(change eth0 by the name of your interface)