Skip to contentSkip to conversation

2.4.28(+?): Strange ARP problem

DISCUSSION Scott Doty 3 participants 6 messages Jan 13, 2005 – Jan 14, 2005 Networking

lore
Share & tools
lore

Scott Doty

author
#0 lore raw

Hi,

We use Linux extensively here at Sonic.net. Our web servers have two NIC's -- a NIC with a public IP address, and a NIC on our SAN (with NetApps).

When we tried to upgrade to 2.4.28, we encountered a problem with NetApp reachability, which turns out to have been a problem with ARP: we were seeing two ARP entries for the NetApp IP's. One would be correct, and one would be "incomplete".

Occasionally, a system would glom onto the incomplete entry, and NFS connectivity would tank. This doesn't happen with 2.4.27.

We'd like to upgrade to 2.4.29-rc2, but we have much trepidation about doing so. I certainly don't want to treat the list as "our own personal help desk" (as warned about in the FAQ), but was hoping someone could shed some light on the problem. I think either myself or one of our guys can write a patch to fix it, if someone would point us in the right direction.

Thank you,

-Scott

#1 lore raw
Quoted 13 lines · Scott Doty

Hi,

We use Linux extensively here at Sonic.net. Our web servers have two NIC's -- a NIC with a public IP address, and a NIC on our SAN (with NetApps).

When we tried to upgrade to 2.4.28, we encountered a problem with NetApp reachability, which turns out to have been a problem with ARP: we were seeing two ARP entries for the NetApp IP's. One would be correct, and one would be "incomplete".

Occasionally, a system would glom onto the incomplete entry, and NFS connectivity would tank. This doesn't happen with 2.4.27.

Quoted 7 lines

We'd like to upgrade to 2.4.29-rc2, but we have much trepidation about doing so. I certainly don't want to treat the list as "our own personal help desk" (as warned about in the FAQ), but was hoping someone could shed some light on the problem. I think either myself or one of our guys can write a patch to fix it, if someone would point us in the right direction.

Thank you,

Scott,

I have no idea of what might be causing such regression - I see a few ARP related changelogs on v2.4.28-rc2:

o [IPV4]: Set ARP hw type correctly for BOOTP over FDDI
o [IPV4]: Permit the official ARP hw type in SIOCSARP for FDDI

Maybe you can try earlier v2.4.28's (-rc1 for one) to check where the problem starts to happen?

David, Herbert, any ideas?

author
#2 lore raw
Quoted 2 lines · Marcelo Tosatti

Maybe you can try earlier v2.4.28's (-rc1 for one) to check where the problem starts to happen?

Good plan, we'll try different kernel versions to pin down exactly where the problem starts.

-Scott

#3 lore raw
In reply to Marcelo Tosatti
Quoted 3 lines · Marcelo Tosatti

Maybe you can try earlier v2.4.28's (-rc1 for one) to check where the problem starts to happen?

The symptom sounds like the bug in the 2.4 backport of the neighbour hash updates. In neigh_create, hash_val needs to be computed inside the lock (and after the growing), not outside.

Someone even posted a patch for it. I'll dig it up tonight if it doesn't show up by then.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#5 lore raw
In reply to Scott Doty
Quoted 2 lines · Scott Doty

I just built a patch from your description -- it's attached.

Yep, that looks good. Please let us know if that fixes your problems.

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author
#4 lore raw
In reply to Herbert Xu
Quoted 11 lines · Herbert Xu
Quoted 3 lines · Marcelo Tosatti

Maybe you can try earlier v2.4.28's (-rc1 for one) to check where the problem starts to happen?

The symptom sounds like the bug in the 2.4 backport of the neighbour hash updates. In neigh_create, hash_val needs to be computed inside the lock (and after the growing), not outside.

Someone even posted a patch for it. I'll dig it up tonight if it doesn't show up by then.

I just built a patch from your description -- it's attached.

-Scott

net/core/neighbour.c +3 -2
@@ -427,11 +427,12 @@
 
 	n->confirmed = jiffies - (n->parms->base_reachable_time<<1);
 
-	hash_val = tbl->hash(pkey, dev) & tbl->hash_mask;
-
 	write_lock_bh(&tbl->lock);
 	if (atomic_read(&tbl->entries) > (tbl->hash_mask + 1))
 		neigh_hash_grow(tbl, (tbl->hash_mask + 1) << 1);
+
+	hash_val = tbl->hash(pkey, dev) & tbl->hash_mask;
+
 	for (n1 = tbl->hash_buckets[hash_val]; n1; n1 = n1->next) {
 		if (dev == n1->dev &&
 		    memcmp(n1->primary_key, pkey, key_len) == 0) {