: should we consider arp missed max during bond_ab_arp_probe()?
Subsystems (2)
Hi Jay,
Our QE reported that, when there is no active slave during bond_ab_arp_probe(), the slaves send the arp probe message one by one. This will flap the switch's mac table quickly, sometimes even make the switch stop learning mac address. So should we consider the arp missed max during bond_ab_arp_probe()? i.e. each slave has more chances to send probe messages before switch to another slave. What do you think?
Thanks
Hangbin
Quoted 8 lines · Wed, Nov 06, 2024 at 08:39:48AM CET, [email protected] wrote:
Hi Jay,
Our QE reported that, when there is no active slave during
bond_ab_arp_probe(), the slaves send the arp probe message one by one. This will flap the switch's mac table quickly, sometimes even make the switch stop learning mac address. So should we consider the arp missed max duringbond_ab_arp_probe()? i.e. each slave has more chances to send probe messages before switch to another slave. What do you think?
Out of curiosity, is anyone still using AB mode in real life? And if yes, any idea why exacly?
Quoted 4 lines
Thanks
Hangbin
Quoted 11 lines · Jiri Pirko
Quoted 8 lines · Wed, Nov 06, 2024 at 08:39:48AM CET, [email protected] wrote:
Hi Jay,
Our QE reported that, when there is no active slave during
bond_ab_arp_probe(), the slaves send the arp probe message one by one. This will flap the switch's mac table quickly, sometimes even make the switch stop learning mac address. So should we consider the arp missed max duringbond_ab_arp_probe()? i.e. each slave has more chances to send probe messages before switch to another slave. What do you think?Out of curiosity, is anyone still using AB mode in real life? And if
Based on our analyse, in year 2024, there are 53.8% users using 802.3ad mode, 41.6% users using active-backup mode. 2.5% users using round-robin mode.
Quoted 1 line
yes, any idea why exacly?
I think they just want to make sure there is a backup for the link.
Thanks
Hangbin
Quoted 11 lines · Wed, Nov 06, 2024 at 10:25:30AM CET, [email protected] wrote:
Quoted 10 lines · Jiri Pirko
Quoted 8 lines · Wed, Nov 06, 2024 at 08:39:48AM CET, [email protected] wrote:
Hi Jay,
Our QE reported that, when there is no active slave during
bond_ab_arp_probe(), the slaves send the arp probe message one by one. This will flap the switch's mac table quickly, sometimes even make the switch stop learning mac address. So should we consider the arp missed max duringbond_ab_arp_probe()? i.e. each slave has more chances to send probe messages before switch to another slave. What do you think?
Quoted 8 lines
Quoted 1 line
Out of curiosity, is anyone still using AB mode in real life? And if
Based on our analyse, in year 2024, there are 53.8% users using 802.3ad mode, 41.6% users using active-backup mode. 2.5% users using round-robin mode.
Quoted 1 line
yes, any idea why exacly?
I think they just want to make sure there is a backup for the link.
Why don't they use LACP? You can have backup there as well.
Quoted 3 lines
Thanks
Hangbin
Quoted 3 lines · Jiri Pirko
Quoted 1 line
I think they just want to make sure there is a backup for the link.
Why don't they use LACP? You can have backup there as well.
Some users don't want to configure switches. Specifically, some large-scale users don't want or don't allow to maintain the same number of switches.
Hangbin
Jakub Kicinski
maintainerQuoted 10 lines · On Wed, 6 Nov 2024 15:40:39 +0100 Jiri Pirko wrote:
Quoted 8 lines
Quoted 1 line
Out of curiosity, is anyone still using AB mode in real life? And if
Based on our analyse, in year 2024, there are 53.8% users using 802.3ad mode, 41.6% users using active-backup mode. 2.5% users using round-robin mode.
Quoted 1 line
yes, any idea why exacly?
I think they just want to make sure there is a backup for the link.
Why don't they use LACP? You can have backup there as well.
FWIW I was asked to help with A/B setups in the past for facility networks (building sensor etc). Those guys wanted to do as little networking as possible, and had very low bandwidth needs but still wanted the redundancy. Basic active/backup was a good fit.
Quoted 8 lines · Hangbin Liu
Hi Jay,
Our QE reported that, when there is no active slave during
bond_ab_arp_probe(), the slaves send the arp probe message one by one. This will flap the switch's mac table quickly, sometimes even make the switch stop learning mac address. So should we consider the arp missed max duringbond_ab_arp_probe()? i.e. each slave has more chances to send probe messages before switch to another slave. What do you think?
Well, "quickly" here depends entirely on what the value ofarp_interval is. It's been quite a while since I looked into the details of this particular behavior, but at the time I didn't see the switches I had issue flap warnings. If memory serves, I usually tested with arp_interval in the realm of 100ms, with anywhere from 2 to 6 interfaces in the bond.
What settings are you using for the bond, and what model of
switch exhibits the behavior you describe?
That said, the intent of the current implementation is to cycle
through the interfaces in the bond relatively quickly when no interfaces are up, under the theory that such behavior finds an available interface in the minimum time.
I'm not necessarily opposed to having each probe "step," so to
speak, perform multiple ARP probe checks. However, I wonder if this is a complicated workaround for not wanting to change a configuration setting on a switch, and it would only make things better by chance (i.e., that the probes just happen to now take long enough to not run afoul of the switch's time limit for some flap parameter).
-J
---
-Jay Vosburgh, [email protected]
Quoted 18 lines · Jay Vosburgh
Quoted 8 lines · Hangbin Liu
Hi Jay,
Our QE reported that, when there is no active slave during
bond_ab_arp_probe(), the slaves send the arp probe message one by one. This will flap the switch's mac table quickly, sometimes even make the switch stop learning mac address. So should we consider the arp missed max duringbond_ab_arp_probe()? i.e. each slave has more chances to send probe messages before switch to another slave. What do you think?Well, "quickly" here depends entirely on what the value of
arp_intervalis. It's been quite a while since I looked into the details of this particular behavior, but at the time I didn't see the switches I had issue flap warnings. If memory serves, I usually tested witharp_intervalin the realm of 100ms, with anywhere from 2 to 6 interfaces in the bond.
Quoted 2 lines
What settings are you using for the bond, and what model of
switch exhibits the behavior you describe?
In our network, we have a cisco 9364 switch. Which will disable mac learning for 120 seconds if 6 MAC moves in 30 seconds[1] by default.
Quoted 6 lines
That said, the intent of the current implementation is to cycle
through the interfaces in the bond relatively quickly when no interfaces are up, under the theory that such behavior finds an available interface in the minimum time.
Quoted 6 lines
I'm not necessarily opposed to having each probe "step," so to
speak, perform multiple ARP probe checks. However, I wonder if this is a complicated workaround for not wanting to change a configuration setting on a switch, and it would only make things better by chance (i.e., that the probes just happen to now take long enough to not run afoul of the switch's time limit for some flap parameter).
For Cisco Nexus 9300-X switches, the mac-move policy is supported since Cisco NX-OS Release 10.3(1)F, which is released August 19, 2022.
So there do have an option to disable/modify the mac policy. But switches can't update to this version will be affected, unless the user change the arp_interval to an large number.
As there is an workaround (either change the switch configure or arp_interval), I don't have a strong intend to change the bonding behavior. I will do it or ignore it based on your decision.
Thanks
Hangbin