bonding: 3ad: fix carrier state with no usable slaves
Subsystems (5)
v4 cover bonding: 3ad: fix carrier state with no usable slaves
Hi everyone,
This series addresses a blackholing issue and a subsequent link-flapping issue in the 802.3ad bonding driver when dealing with inactive slaves and the min_links parameter.
When an 802.3ad (LACP) bonding interface has no slaves in the collecting/distributing state, the bonding master still reports carrier as up as long as at least 'min_links' slaves have carrier.
In this situation, only one slave is effectively used for TX/RX, while traffic received on other slaves is dropped. Upper-layer daemons therefore consider the interface operational, even though traffic may be blackholed if the lack of LACP negotiation means the partner is not ready to deal with traffic.
This patchset introduces an optional behavior, widely adopted across the industry, to address this issue. It consists of bringing the bonding master interface down to signal to upper-layer processes that it is not usable.
This patchset depends on the following iproute2 change:
ip/bond: add lacp_strict support
Patch 1 introduces the lacp_strict configuration knob, which is applied in the subsequent patch. The default (off) mode preserves the existing behavior, while the strict mode (on) is intended to force the bonding master carrier down in this situation.
Patch 2 addresses the core issue when lacp_strict is set to strict. It ensures that carrier is asserted only when at least 'min_links' slaves are in the Collecting/Distributing state.
Patch 3 fixes a side effect of the second patch. Tightening the carrier logic exposes a state persistence bug: when a physical link goes down, the LACP collecting/distributing flags remain set. When the link returns, the interface briefly hallucinates that it is ready, bounces the carrier up, and then drops it again once LACP renegotiation starts. Fix by resetting Collecting and Distributing state as soon as the link goes down.
Patch 4 adds a test for bonding lacp_strict both modes.
Changelog:
v3 -> v4
- Rename the configuration knob to lacp_strict on/off instead of lacp_fallback legacy/strict.
- Patch 1: change the command documentation accordingly and wrap text at approximately 75 columns.
- Use "usable" wording instead "valid" for LACP Collecting / Distributing state in code and commit log.
- Patch 2: test collecting and distributing state regardless of coupled_control
- Patch 3: Reworked because removing the SELECTED flag was not compliant with 802.1AX. Instead, to transition to the WAITING state on port disabled, except when already in the DETACHED state. And remove Collecting and Distributing state in WAITING state.
- Patch 4 is removed. It was a fix for patch 3 but it is no more needed since patch 3 was reworked.
v2 -> v3
- Add an initial patch introducing the lacp_fallback configuration knob (no behavior change yet).
- Patch 2 (was patch 1 in v2): apply the new behavior only when lacp_fallback is set to strict, and re-evaluate the bonding master carrier when the setting changes.
v1 -> v2
- Patch 1: split a comment line that exceeded 80 characters.
- Move the change from patch 2 in __agg_ports_are_ready() into a third patch, as it is actually a side effect of the fix introduced in patch 2.
- Patch 2: Expand the commit message and add a code comment describing the change in ad_port_selection_logic().
- Patch 3: Check the READY_N flag only on ports in the WAITING state, rather than on all enabled ports. This more closely matches 802.3ad.
Louis Scalbert (4):
bonding: 3ad: add lacp_strict configuration knob
bonding: 3ad: fix carrier when no valid slaves
bonding: 3ad: fix mux port state on oper down
selftests: bonding: add test for lacp_strict mode
Documentation/networking/bonding.rst | 23 ++ drivers/net/bonding/bond_3ad.c | 28 +- drivers/net/bonding/bond_main.c | 1 + drivers/net/bonding/bond_netlink.c | 16 + drivers/net/bonding/bond_options.c | 27 ++ include/net/bond_options.h | 1 + include/net/bonding.h | 1 + include/uapi/linux/if_link.h | 1 + .../selftests/drivers/net/bonding/Makefile | 1 + .../drivers/net/bonding/bond_lacp_strict.sh | 299 ++++++++++++++++++ 10 files changed, 397 insertions(+), 1 deletion(-) create mode 100755 tools/testing/selftests/drivers/net/bonding/bond_lacp_strict.sh
Signature · 1 line
2.39.2
PATCH v4 1/4 bonding: 3ad: add lacp_strict configuration knob
When an 802.3ad (LACP) bonding interface has no slaves in the collecting/distributing state, the bonding master still reports carrier as up as long as at least 'min_links' slaves have carrier.
In this situation, only one slave is effectively used for TX/RX, while traffic received on other slaves is dropped. Upper-layer daemons therefore consider the interface operational, even though traffic may be blackholed if the lack of LACP negotiation means the partner is not ready to deal with traffic.
Introduce a configuration knob to control this behavior. It allows the bonding master to assert carrier only when at least 'min_links' slaves are in Collecting_Distributing state.
The default mode preserves the existing behavior. This patch only introduces the knob; its behavior is implemented in the subsequent commit.
--- Documentation/networking/bonding.rst | 23 +++++++++++++++++++++++ drivers/net/bonding/bond_main.c | 1 + drivers/net/bonding/bond_netlink.c | 16 ++++++++++++++++ drivers/net/bonding/bond_options.c | 26 ++++++++++++++++++++++++++ include/net/bond_options.h | 1 + include/net/bonding.h | 1 + include/uapi/linux/if_link.h | 1 + 7 files changed, 69 insertions(+)
Documentation/networking/bonding.rst +23 -0
@@ -619,6 +619,29 @@ min_links aggregator cannot be active without at least one available link, setting this option to 0 or to 1 has the exact same effect. +lacp_strict + + Specifies the fallback behavior of a bonding when LACP negotiation + fails on all slave links, i.e. when no slave is in the + Collecting_Distributing state, while at least `min_links` link still + reports carrier up. + + This option is only applicable to 802.3ad mode (mode 4). + + Valid values are: + + off or 0 + One interface of the bond is selected to be active, in order to + facilitate communication with peer devices that do not implement + LACP. + + on or 1 + Interfaces are only permitted to be made active if they have an + active LACP partner and have successfully reached + Collecting_Distributing state. + + The default value is 0 (off). + mode Specifies one of the bonding policies. The default is
drivers/net/bonding/bond_main.c +1 -0
@@ -6438,6 +6438,7 @@ static int __init bond_check_params(struct bond_params *params) params->ad_user_port_key = ad_user_port_key; params->coupled_control =1 ; params->broadcast_neighbor =0 ; + params->lacp_strict =0 ;if (packets_per_slave >0 ) { params->reciprocal_packets_per_slave =reciprocal_value (packets_per_slave);
drivers/net/bonding/bond_netlink.c +16 -0
@@ -139,6 +139,7 @@ static const struct nla_policy bond_policy[IFLA_BOND_MAX + 1] = { [IFLA_BOND_NS_IP6_TARGET] = { .type = NLA_NESTED }, [IFLA_BOND_COUPLED_CONTROL] = { .type = NLA_U8 }, [IFLA_BOND_BROADCAST_NEIGH] = { .type = NLA_U8 }, + [IFLA_BOND_LACP_STRICT] = { .type = NLA_U8 }, };static const struct nla_policy bond_slave_policy[IFLA_BOND_SLAVE_MAX +1 ] = {
@@ -595,6 +596,16 @@ static int bond_changelink(struct net_device *bond_dev, struct nlattr *tb[],return err; } +if (data[IFLA_BOND_LACP_STRICT ]) { +int fallback_mode =nla_get_u8 (data[IFLA_BOND_LACP_STRICT ]); + +bond_opt_initval (&newval, fallback_mode); + err =__bond_opt_set (bond,BOND_OPT_LACP_STRICT , &newval, + data[IFLA_BOND_LACP_STRICT ], extack); +if (err) +return err; + } +return 0 ; }
@@ -667,6 +678,7 @@ static size_t bond_get_size(const struct net_device *bond_dev)nla_total_size (sizeof (struct in6_addr )) *BOND_MAX_NS_TARGETS +nla_total_size (sizeof (u8)) +/* IFLA_BOND_COUPLED_CONTROL */ nla_total_size (sizeof (u8)) +/* IFLA_BOND_BROADCAST_NEIGH */ +nla_total_size (sizeof (u8)) +/* IFLA_BOND_LACP_STRICT */ 0 ; }
@@ -834,6 +846,10 @@ static int bond_fill_info(struct sk_buff *skb, bond->params.broadcast_neighbor)) goto nla_put_failure; +if (nla_put_u8 (skb,IFLA_BOND_LACP_STRICT , + bond->params.lacp_strict)) + goto nla_put_failure; +if (BOND_MODE (bond) ==BOND_MODE_8023AD ) {struct ad_info info;
drivers/net/bonding/bond_options.c +26 -0
@@ -68,6 +68,8 @@ static int bond_option_lacp_active_set(struct bonding *bond, const struct bond_opt_value *newval);static int bond_option_lacp_rate_set (struct bonding *bond,const struct bond_opt_value *newval); +static int bond_option_lacp_strict_set (struct bonding *bond, +const struct bond_opt_value *newval);static int bond_option_ad_select_set (struct bonding *bond,const struct bond_opt_value *newval); static int bond_option_queue_id_set(struct bonding *bond,
@@ -162,6 +164,12 @@ static const struct bond_opt_value bond_lacp_rate_tbl[] = { { NULL, -1, 0}, }; +static const struct bond_opt_value bond_lacp_strict_tbl[] = { + {"off" ,0 ,BOND_VALFLAG_DEFAULT }, + {"on" ,1 ,0 }, + {NULL ,-1 ,0 } +}; +static const struct bond_opt_value bond_ad_select_tbl[] = { {"stable" ,BOND_AD_STABLE ,BOND_VALFLAG_DEFAULT }, { "bandwidth", BOND_AD_BANDWIDTH, 0},
@@ -363,6 +371,14 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = { .values = bond_lacp_rate_tbl, .set = bond_option_lacp_rate_set }, + [BOND_OPT_LACP_STRICT] = { + .id = BOND_OPT_LACP_STRICT, + .name = "lacp_strict", + .desc = "Define the LACP fallback mode when no slaves have negotiated", + .unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)), + .values = bond_lacp_strict_tbl, + .set = bond_option_lacp_strict_set + }, [BOND_OPT_MINLINKS] = { .id = BOND_OPT_MINLINKS, .name = "min_links",
@@ -1684,6 +1700,16 @@ static int bond_option_lacp_rate_set(struct bonding *bond,return 0 ; } +static int bond_option_lacp_strict_set (struct bonding *bond, +const struct bond_opt_value *newval) +{ +netdev_dbg (bond->dev,"Setting LACP fallback to %s (%llu)\n" , + newval->string, newval->value); + bond->params.lacp_strict = newval->value; + +return 0 ; +} +static int bond_option_ad_select_set (struct bonding *bond,const struct bond_opt_value *newval) {
include/net/bond_options.h +1 -0
@@ -79,6 +79,7 @@ enum {BOND_OPT_COUPLED_CONTROL ,BOND_OPT_BROADCAST_NEIGH ,BOND_OPT_ACTOR_PORT_PRIO , +BOND_OPT_LACP_STRICT , BOND_OPT_LAST };
include/net/bonding.h +1 -0
@@ -129,6 +129,7 @@ struct bond_params {int peer_notif_delay;int lacp_active;int lacp_fast; +int lacp_strict;unsigned int min_links;int ad_select;char primary[IFNAMSIZ ];
include/uapi/linux/if_link.h +1 -0
@@ -1584,6 +1584,7 @@ enum {IFLA_BOND_NS_IP6_TARGET ,IFLA_BOND_COUPLED_CONTROL ,IFLA_BOND_BROADCAST_NEIGH , +IFLA_BOND_LACP_STRICT , __IFLA_BOND_MAX, };
Signature · 1 line
2.39.2
PATCH v4 2/4 bonding: 3ad: fix carrier when no usable slaves
In reply to Louis ScalbertApply the "lacp_strict" configuration from the previous commit.
"lacp_strict" mode "on" asserts that the bonding master carrier is up only when at least 'min_links' slaves are in the Collecting_Distributing state.
--- drivers/net/bonding/bond_3ad.c | 21 ++++++++++++++++++++- drivers/net/bonding/bond_options.c | 1 + 2 files changed, 21 insertions(+), 1 deletion(-)
drivers/net/bonding/bond_3ad.c +20 -1
@@ -745,6 +745,21 @@ static void __set_agg_ports_ready(struct aggregator *aggregator, int val) } } +static int __agg_usable_ports (struct aggregator *agg) +{ +struct port *port; +int valid =0 ; + +for (port = agg->lag_ports; port; + port = port->next_port_in_aggregator) { +if (port->actor_oper_port_state &LACP_STATE_COLLECTING && + port->actor_oper_port_state &LACP_STATE_DISTRIBUTING ) + valid++; + } + +return valid; +} +static int __agg_active_ports (struct aggregator *agg) {struct port *port;
@@ -2120,6 +2135,7 @@ static void ad_enable_collecting_distributing(struct port *port, port->actor_port_number, port->aggregator->aggregator_identifier);__enable_port (port); +bond_3ad_set_carrier (port->slave->bond);/* Slave array needs update */ *update_slave_arr = true;/* Should notify peers if possible */
@@ -2141,6 +2157,7 @@ static void ad_disable_collecting_distributing(struct port *port, port->actor_port_number, port->aggregator->aggregator_identifier);__disable_port (port); +bond_3ad_set_carrier (port->slave->bond);/* Slave array needs an update */ *update_slave_arr = true; }
@@ -2820,7 +2837,9 @@ int bond_3ad_set_carrier(struct bonding *bond) active =__get_active_agg (&(SLAVE_AD_INFO (first_slave)->aggregator));if (active) {/* are enough slaves available to consider link up? */ -if (__agg_active_ports (active) < bond->params.min_links) { +if ((bond->params.lacp_strict ?__agg_usable_ports (active) + :__agg_active_ports (active)) < + bond->params.min_links) {if (netif_carrier_ok (bond->dev)) {netif_carrier_off (bond->dev); goto out;
drivers/net/bonding/bond_options.c +1 -0
@@ -1706,6 +1706,7 @@ static int bond_option_lacp_strict_set(struct bonding *bond,netdev_dbg (bond->dev,"Setting LACP fallback to %s (%llu)\n" , newval->string, newval->value); bond->params.lacp_strict = newval->value; +bond_3ad_set_carrier (bond);return 0 ; }
Signature · 1 line
2.39.2
PATCH v4 3/4 bonding: 3ad: fix mux port state on oper down
In reply to Louis ScalbertWhen the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.
When lacp_strict mode is on, the interface should not transition to carrier up until LACP negotiation is complete.
This happens because the actor and partner port states remain in Collecting_Distributing when the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.
Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.
Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.
--- drivers/net/bonding/bond_3ad.c | 7 +++++++ 1 file changed, 7 insertions(+)
drivers/net/bonding/bond_3ad.c +7 -0
@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)if (port->sm_vars &AD_PORT_BEGIN ) { port->sm_mux_state =AD_MUX_DETACHED ; + }else if (!port->is_enabled && port->sm_mux_state !=AD_MUX_DETACHED ) { + port->sm_mux_state =AD_MUX_WAITING ; }else {switch (port->sm_mux_state) {case AD_MUX_DETACHED :
@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)break ;case AD_MUX_WAITING : port->sm_mux_timer_counter =__ad_timer_to_ticks (AD_WAIT_WHILE_TIMER ,0 ); + port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION ; +ad_disable_collecting_distributing (port, + update_slave_arr); + port->actor_oper_port_state &= ~LACP_STATE_COLLECTING ; + port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING ;break ;case AD_MUX_ATTACHED :if (port->aggregator->is_active)
Signature · 1 line
2.39.2
Jay Vosburgh
maintainerRe: [PATCH net v4 3/4] bonding: 3ad: fix mux port state on oper down
Quoted 32 lines · Louis Scalbert
When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.
When
lacp_strictmode is on, the interface should not transition to carrier up until LACP negotiation is complete.This happens because the actor and partner port states remain in
Collecting_Distributingwhen the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.
Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.
Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")Signed-off-by Louis Scalbert <[email protected]>---
drivers/net/bonding/bond_3ad.c| 7 +++++++1 file changed, 7 insertions(+)drivers/net/bonding/bond_3ad.c +0
-0
drivers/net/bonding/bond_3ad.c static void ad_mux_machine(struct port *port, bool *update_slave_arr) · line 1057
Quoted 5 lines
drivers/net/bonding/bond_3ad.c +2
-0@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)if (port->sm_vars &AD_PORT_BEGIN ) { port->sm_mux_state =AD_MUX_DETACHED ; + }else if (!port->is_enabled && port->sm_mux_state !=AD_MUX_DETACHED ) { + port->sm_mux_state =AD_MUX_WAITING ;
Technically, this is not exactly following the state machines.
The mux machine should transition to WAITING from DETACHED when
Selected == SELECTED or STANDBY, not for !is_enabled ("port_enabled" in the standard). The check for !is_enabled happens in the receive machine, and it would transition to PORT_DISABLED state (which clears Synchronization).
I'm not sure if this is actually an issue or not; I need to read
the relevant bits again to make sure I understand how it's supposed to work.
-J
Quoted 18 lines
drivers/net/bonding/bond_3ad.c +5
-0}else {switch (port->sm_mux_state) {case AD_MUX_DETACHED :@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)break ;case AD_MUX_WAITING : port->sm_mux_timer_counter =__ad_timer_to_ticks (AD_WAIT_WHILE_TIMER ,0 ); + port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION ; +ad_disable_collecting_distributing (port, + update_slave_arr); + port->actor_oper_port_state &= ~LACP_STATE_COLLECTING ; + port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING ;break ;case AD_MUX_ATTACHED :if (port->aggregator->is_active)Signature · 1 line
2.39.2
---
-Jay Vosburgh, [email protected]
Re: [PATCH net v4 3/4] bonding: 3ad: fix mux port state on oper down
Hello Jay,
Sorry for the late reply. I’ve been busy with another project these past few days.
Quoted 43 lines · Jay Vosburgh
Quoted 32 lines · Louis Scalbert
When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.
When
lacp_strictmode is on, the interface should not transition to carrier up until LACP negotiation is complete.This happens because the actor and partner port states remain in
Collecting_Distributingwhen the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.
Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.
Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")Signed-off-by Louis Scalbert <[email protected]>---
drivers/net/bonding/bond_3ad.c| 7 +++++++1 file changed, 7 insertions(+)drivers/net/bonding/bond_3ad.c +0
-0drivers/net/bonding/bond_3ad.c static void ad_mux_machine(struct port *port, bool *update_slave_arr) · line 1057
Quoted 5 lines
drivers/net/bonding/bond_3ad.c +2
-0@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)if (port->sm_vars &AD_PORT_BEGIN ) { port->sm_mux_state =AD_MUX_DETACHED ; + }else if (!port->is_enabled && port->sm_mux_state !=AD_MUX_DETACHED ) { + port->sm_mux_state =AD_MUX_WAITING ;Technically, this is not exactly following the state machines.
Quoted 3 lines
The mux machine should transition to WAITING from DETACHED when
Selected == SELECTED or STANDBY, not for !is_enabled ("port_enabled" in the standard).
The MUX machine still transitions from DETACHED to WAITING; this happens a few lines later and is unchanged.
The relevant code is:
} else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
port->sm_mux_state = AD_MUX_WAITING;
} else {
switch (port->sm_mux_state) {
case AD_MUX_DETACHED:
if ((port->sm_vars & AD_PORT_SELECTED)
|| (port->sm_vars & AD_PORT_STANDBY))
/* if SELECTED or STANDBY */
port->sm_mux_state = AD_MUX_WAITING;
break;My change is for NOT is_enabled AND NOT AD_MUX_DETACHED.
Quoted 3 lines
The check for !is_enabled happens in the receive
machine, and it would transition toPORT_DISABLEDstate (which clears Synchronization).
I agree with that statement. However, clearing Synchronization is not
sufficient: the purpose of the fix is to clear Collecting and Distributing.
I noticed that is_enabled is defined in the 802.3ad-2000 standard, while Port_Operational is defined in 802.1AX-2020. I am not sure about 802.1AX-2014, as I do not have access to that version.
The 802.1AX-2020 standard uses a different MUX state diagram. Compared to 802.3ad, the ATTACHED state has been split into ATTACH and ATTACHED. There is no longer a WAITING state; it has been replaced by ATTACHED_WTR.
The new standard says that the MUX machine should transition to ATTACHED_WTR when Port_Operational is FALSE and the current state is not DETACHED.
So, in my opinion, the change is correct, at least with respect to 802.1AX-2020.
Quoted 4 lines
I'm not sure if this is actually an issue or not; I need to read
the relevant bits again to make sure I understand how it's supposed to work.
Please confirm what you want me to do: should I keep the fix as it is ?
best regards,
Louis Scalbert
Quoted 24 lines
-JQuoted 16 lines
drivers/net/bonding/bond_3ad.c +5
-1}else {switch (port->sm_mux_state) {case AD_MUX_DETACHED :@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)break ;case AD_MUX_WAITING : port->sm_mux_timer_counter =__ad_timer_to_ticks (AD_WAIT_WHILE_TIMER ,0 ); + port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION ; +ad_disable_collecting_distributing (port, + update_slave_arr); + port->actor_oper_port_state &= ~LACP_STATE_COLLECTING ; + port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING ;break ;case AD_MUX_ATTACHED :if (port->aggregator->is_active) --Quoted 2 lines
2.39.2
---
-Jay Vosburgh, [email protected]
Jay Vosburgh
maintainerRe: [PATCH net v4 3/4] bonding: 3ad: fix mux port state on oper down
Quoted 75 lines · Louis Scalbert
Hello Jay,
Sorry for the late reply. I’ve been busy with another project these past few days.
Quoted 43 lines · Jay Vosburgh
Quoted 32 lines · Louis Scalbert
When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.
When
lacp_strictmode is on, the interface should not transition to carrier up until LACP negotiation is complete.This happens because the actor and partner port states remain in
Collecting_Distributingwhen the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.
Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.
Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")Signed-off-by Louis Scalbert <[email protected]>---
drivers/net/bonding/bond_3ad.c| 7 +++++++1 file changed, 7 insertions(+)drivers/net/bonding/bond_3ad.c +0
-0drivers/net/bonding/bond_3ad.c static void ad_mux_machine(struct port *port, bool *update_slave_arr) · line 1057
Quoted 5 lines
drivers/net/bonding/bond_3ad.c +2
-0@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)if (port->sm_vars &AD_PORT_BEGIN ) { port->sm_mux_state =AD_MUX_DETACHED ; + }else if (!port->is_enabled && port->sm_mux_state !=AD_MUX_DETACHED ) { + port->sm_mux_state =AD_MUX_WAITING ;Technically, this is not exactly following the state machines.
Quoted 3 lines
The mux machine should transition to WAITING from DETACHED when
Selected == SELECTED or STANDBY, not for !is_enabled ("port_enabled" in the standard).The MUX machine still transitions from DETACHED to WAITING; this happens a few lines later and is unchanged.
The relevant code is:
} else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) { port->sm_mux_state = AD_MUX_WAITING; } else { switch (port->sm_mux_state) { case AD_MUX_DETACHED: if ((port->sm_vars & AD_PORT_SELECTED) || (port->sm_vars & AD_PORT_STANDBY)) /* if SELECTED or STANDBY */ port->sm_mux_state = AD_MUX_WAITING; break;My change is for NOT
is_enabledAND NOTAD_MUX_DETACHED.Quoted 3 lines
The check for !is_enabled happens in the receive
machine, and it would transition toPORT_DISABLEDstate (which clears Synchronization).
Quoted 7 lines
I agree with that statement. However, clearing Synchronization is not
sufficient: the purpose of the fix is to clear Collecting and Distributing.I noticed that
is_enabledis defined in the 802.3ad-2000 standard, whilePort_Operationalis defined in 802.1AX-2020. I am not sure about 802.1AX-2014, as I do not have access to that version.
Did you mean 802.1AX-2014 instead of 802.3ad-2000 here?
Quoted 11 lines
The 802.1AX-2020 standard uses a different MUX state diagram. Compared to 802.3ad, the ATTACHED state has been split into ATTACH and ATTACHED. There is no longer a WAITING state; it has been replaced by
ATTACHED_WTR.The new standard says that the MUX machine should transition to
ATTACHED_WTRwhenPort_Operationalis FALSE and the current state is not DETACHED.So, in my opinion, the change is correct, at least with respect to 802.1AX-2020.
I'm concerned that mixing versions of the standard will cause
issues, precisely because the state machines vary between the 2014 and 2020 versions.
As the standards are not explicitly backwards compatible,
bonding should conform to one version, and in my opinion the version that makes the most sense is the 2014 edition. Bonding is already very close to the 2014 version, while compliance with the 2020 version would require significant changes.
The 2014 edition is freely available from the IEEE, for example at:
https://1.ieee802.org/tsn/802-1ax-2014/
Quoted 6 lines
Quoted 4 lines
I'm not sure if this is actually an issue or not; I need to read
the relevant bits again to make sure I understand how it's supposed to work.Please confirm what you want me to do: should I keep the fix as it is ?
I see that you've reposted the patch series, I'll reply there
later when I've had a chance to study the details.
-J
Quoted 27 lines
best regards,
Louis Scalbert
Quoted 24 lines
-JQuoted 16 lines
drivers/net/bonding/bond_3ad.c +5
-1}else {switch (port->sm_mux_state) {case AD_MUX_DETACHED :@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)break ;case AD_MUX_WAITING : port->sm_mux_timer_counter =__ad_timer_to_ticks (AD_WAIT_WHILE_TIMER ,0 ); + port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION ; +ad_disable_collecting_distributing (port, + update_slave_arr); + port->actor_oper_port_state &= ~LACP_STATE_COLLECTING ; + port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING ;break ;case AD_MUX_ATTACHED :if (port->aggregator->is_active) --Quoted 2 lines
2.39.2
---
-Jay Vosburgh, [email protected]
---
-Jay Vosburgh, [email protected]
PATCH v4 4/4 selftests: bonding: add test for lacp_strict mode
In reply to Louis ScalbertAdd a test for the bonding lacp_strict mode.
--- .../selftests/drivers/net/bonding/Makefile | 1 + .../drivers/net/bonding/bond_lacp_strict.sh | 299 ++++++++++++++++++ 2 files changed, 300 insertions(+) create mode 100755 tools/testing/selftests/drivers/net/bonding/bond_lacp_strict.sh
tools/testing/selftests/drivers/net/bonding/Makefile +1 -0
@@ -7,6 +7,7 @@ TEST_PROGS := \ bond-eth-type-change.sh \ bond-lladdr-target.sh \ bond_ipsec_offload.sh \ + bond_lacp_strict.sh \ bond_lacp_prio.sh \ bond_macvlan_ipvlan.sh \ bond_options.sh \
tools/testing/selftests/drivers/net/bonding/bond_lacp_strict.sh new file +299 -0
299 lines not on this page · raw message
Signature · 1 line
2.39.2
Jakub Kicinski
maintainerRe: [PATCH net v4 4/4] selftests: bonding: add test for lacp_strict mode
Quoted 1 line · On Fri, 17 Apr 2026 16:05:05 +0200 Louis Scalbert wrote:
+ip netns exec "${p_ns}" tc qdisc add dev eth0 root netem loss 100%
netem is not included the in the config for bonding tests, this fails in the CI.