Skip to contentSkip to conversation

ARP limit ?

LARTC Roberto Nibali 5 participants 10 messages Aug 11, 2004 – Aug 12, 2004

lore
Share & tools
lore
#0 lore raw

Hello,

Quoted 1 line

yep, i'm using them, i needed to know the max and anyone experiencing

Those are int and nothing in the kernel code prevents them having MAX_INT-1 assigned as a value. However you have a rather serious networking problem anyway if you exceed the predefined gc* values.

Some more information (besides reading the related kernel code):

http://www.rstack.net/arp.html
http://www.rstack.net/tuning_proc_for_arp.html

Quoted 1 line

problems with very big arp's

Yes, the time spent in the GC to reach the equilibrium is rather high with a high number of stale cache entries. Once gc_tresh3 threshold kicks in you get an aggressive table flush concurrent with the new neighbour entries.

Quoted 1 line

arpd is marked as depricated or so!?

No, not at all ;).

Best regards,
Roberto Nibali, ratz

Signature · 5 lines
echo 
'[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#2 lore raw
In reply to Roberto Nibali
Quoted 2 lines · On Wed, Aug 11, 2004 at 10:49:23AM +0300, [email protected] wrote:

what is the limit on the arp cache entires ?! Does someone have very big LANs with linux-routers ? How big ? (i mean flat L2 network, not segmented)

It is configurable with /proc/sys/net/ipv4/neigh/default/gc_thresh{1,2,3}. I don't know what's the exact limit.

Quoted 1 line

thanx

Bye,

Peter Surda (Shurdeek) <[email protected]>, ICQ 10236103, +436505122023

Signature · 4 lines
                Where do you think you're going today?
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#3 lore raw
In reply to Roberto Nibali

In case of big LAN ( more than 512 hosts ) you must use arpd daemon .

Quoted 10 lines · On Wednesday 11 August 2004 10:49, [email protected] wrote:

what is the limit on the arp cache entires ?!
Does someone have very big LANs with linux-routers ? How big ? (i mean flat L2 network, not segmented)

I want to know how much can my net can scale ?

thanx
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 5 lines
Best Regards,
Nachko Halachev
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#4 lore raw
In reply to Roberto Nibali
Quoted 1 line · Nachko Halachev

In case of big LAN ( more than 512 hosts ) you must use arpd daemon .

No, you don't have to, see my previous post.

Bye,

Peter Surda (Shurdeek) <[email protected]>, ICQ 10236103, +436505122023

Signature · 4 lines
                Where do you think you're going today?
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#5 lore raw
In reply to Roberto Nibali

Hi,

Quoted 5 lines

from make config --> arpd -> help

....
This code is experimental and also obsolete.
.....

Yeah well, such entries are sprinkled all over the kernel. Fact is that it's still in the 2.6.x kernel series, which means that even though it was thought to be experimental and obsolete, it has not been ripped out of any stable kernel drops. To bo honest, I don't see the experimental part as the locking looks correct and netlink sockets are used to communicate, which is a big plus as well. I have only check for 2 minutes though, relevant code excerpts inlined for viewing pleasure:

#ifdef CONFIG_ARPD
         if (notify && neigh->parms->app_probes)
                 neigh_app_notify(neigh);
#endif
#ifdef CONFIG_ARPD
void neigh_app_ns(struct neighbour *n)
{
         struct sk_buff *skb;
         struct nlmsghdr  *nlh;
         int size = NLMSG_SPACE(sizeof(struct ndmsg)+256);

         skb = alloc_skb(size, GFP_ATOMIC);
         if (!skb)
                 return;

         if (neigh_fill_info(skb, n, 0, 0, RTM_GETNEIGH) < 0) {
                 kfree_skb(skb);
                 return;
         }
         nlh = (struct nlmsghdr*)skb->data;
         nlh->nlmsg_flags = NLM_F_REQUEST;
         NETLINK_CB(skb).dst_groups = RTMGRP_NEIGH;
         netlink_broadcast(rtnl, skb, 0, RTMGRP_NEIGH, GFP_ATOMIC);
}
static void neigh_app_notify(struct neighbour *n)
{
         struct sk_buff *skb;
         struct nlmsghdr  *nlh;
         int size = NLMSG_SPACE(sizeof(struct ndmsg)+256);

         skb = alloc_skb(size, GFP_ATOMIC);
         if (!skb)
                 return;

         if (neigh_fill_info(skb, n, 0, 0, RTM_NEWNEIGH) < 0) {
                 kfree_skb(skb);
                 return;
         }
         nlh = (struct nlmsghdr*)skb->data;
         NETLINK_CB(skb).dst_groups = RTMGRP_NEIGH;
         netlink_broadcast(rtnl, skb, 0, RTMGRP_NEIGH, GFP_ATOMIC);
}

#endif /* CONFIG_ARPD */

Quoted 3 lines

2.4.22
That is why I try to use bogger gc_*
thanx for the links.. reading now..

gc_* is of course the way to go and as I've stated before, I would rather think of a misconcepted network architecture when seing neighbour table overflows and fix that flaw instead of using arpd. I found myself back a couple of times in a situation where I had to fiddle with the proc-fs values in a load balanced environment using asymmetric routing.

The reason why it is marked obsolete is because most probably noone really is using it since people run sane network environments or use the proc-fs tunables.

Best regards,
Roberto Nibali, ratz

Signature · 5 lines
echo 
'[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#6 lore raw
In reply to Roberto Nibali

yep, i'm using them, i needed to know the max and anyone experiencing problems with very big arp's
arpd is marked as depricated or so!?

Quoted 8 lines · On Wed, Aug 11, 2004 at 10:49:23AM +0300, [email protected] wrote:
Quoted 2 lines

what is the limit on the arp cache entires ?! Does someone have very big LANs with linux-routers ? How big ? (i mean flat L2 network, not segmented)

It is configurable with /proc/sys/net/ipv4/neigh/default/gc_thresh{1,2,3}. I don't know what's the exact limit.

Quoted 1 line

thanx

Bye,

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#7 lore raw
In reply to Roberto Nibali

from make config --> arpd -> help

....
This code is experimental and also obsolete.
.....

2.4.22
That is why I try to use bogger gc_*
thanx for the links.. reading now..

Quoted 20 lines

Hello,

Quoted 1 line

yep, i'm using them, i needed to know the max and anyone experiencing

Those are int and nothing in the kernel code prevents them having MAX_INT-1 assigned as a value. However you have a rather serious networking problem anyway if you exceed the predefined gc* values.

Some more information (besides reading the related kernel code):

http://www.rstack.net/arp.html
http://www.rstack.net/tuning_proc_for_arp.html

Quoted 1 line

problems with very big arp's

Yes, the time spent in the GC to reach the equilibrium is rather high with a high number of stale cache entries. Once gc_tresh3 threshold kicks in you get an aggressive table flush concurrent with the new neighbour entries.

Quoted 13 lines
Quoted 1 line

arpd is marked as depricated or so!?

No, not at all ;).

Best regards,
Roberto Nibali, ratz

Signature · 5 lines
echo 
'[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#8 lore raw
In reply to Roberto Nibali
Quoted 10 lines
Quoted 4 lines

2.4.22
That is why I try to use bogger gc_*
thanx for the links.. reading now..

gc_* is of course the way to go and as I've stated before, I would rather think of a misconcepted network architecture when seing neighbour table overflows and fix that flaw instead of using arpd. I found myself back a couple of times in a situation where I had to fiddle with the proc-fs values in a load balanced environment using asymmetric routing.

]- i know it is not good to have big LAN's, but I'm in situation if I move
to more router oriented network, the things will become much more problematic. The routers itself will become the biggest botlleneck (much worse than big ARP tables).. and also will limit my ability to balance the network (CATV).. with two words better stick to big ARP tables rather than introduce other weak links.. and wait until
it becomes big enought and I can logicaly separate small blocks of the
network behind routers.. and still have big ARP (but not enourmous :"))

One additional question...if I deploy parallel router i.e. before :

<big LAN>----[router]---Internet

after :

<big LAN>----[router1]---Internet

                    |--[router2]--|

so that part of the LAN is routed via router1 and part of it over router2.. if I follow my thoughts the ARP will spread over the routers and as consequence router1 will shrink its arp table.
(big LAN - is phisicaly one net, but logicaly/IP several subnets).. router1 will make arp-request only for its IP-subnets, but not for those that router2 take care of.

are my thoghts correct...

tia

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#9 lore raw
In reply to Roberto Nibali

You would be better off if you could separate the LAN by using different channels in the CATV system. Then you actually have a smaller network and the physical and logical connectivity are identical. However, if the memory of the router is sufficient to contain the large ARP table and the bandwidth is sufficient, your large LAN should be ok.

It is not generally desireable to have multiple subnets on the same physical network, because the broadcasts can become confusing.

Quoted 43 lines · [email protected] wrote:
Quoted 9 lines
Quoted 3 lines

2.4.22
That is why I try to use bogger gc_*
thanx for the links.. reading now..

gc_* is of course the way to go and as I've stated before, I would rather think of a misconcepted network architecture when seing neighbour table overflows and fix that flaw instead of using arpd. I found myself back a couple of times in a situation where I had to fiddle with the proc-fs values in a load balanced environment using asymmetric routing.

]- i know it is not good to have big LAN's, but I'm in situation if I move
to more router oriented network, the things will become much more problematic. The routers itself will become the biggest botlleneck (much worse than big ARP tables).. and also will limit my ability to balance the network (CATV).. with two words better stick to big ARP tables rather than introduce other weak links.. and wait until
it becomes big enought and I can logicaly separate small blocks of the
network behind routers.. and still have big ARP (but not enourmous :"))

One additional question...if I deploy parallel router i.e. before :

<big LAN>----[router]---Internet

after :

<big LAN>----[router1]---Internet

                    |--[router2]--|

so that part of the LAN is routed via router1 and part of it over router2.. if I follow my thoughts the ARP will spread over the routers and as consequence router1 will shrink its arp table.
(big LAN - is phisicaly one net, but logicaly/IP several subnets).. router1 will make arp-request only for its IP-subnets, but not for those that router2 take care of.

are my thoghts correct...

tia

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Signature · 6 lines
     Lawrence MacIntyre     865.574.8696     [email protected]
                Oak Ridge National Laboratory
High Performance Information Infrastructure Technology Group
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/