Skip to contentSkip to conversation

bonding: 3ad: fix carrier state with no usable slaves

PATCH Superseded Louis Scalbert 3 participants 9 messages Apr 17 – May 7

Subsystems (5)

v4 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v4 cover bonding: 3ad: fix carrier state with no usable slaves

Hi everyone,

This series addresses a blackholing issue and a subsequent link-flapping issue in the 802.3ad bonding driver when dealing with inactive slaves and the min_links parameter.

When an 802.3ad (LACP) bonding interface has no slaves in the collecting/distributing state, the bonding master still reports carrier as up as long as at least 'min_links' slaves have carrier.

In this situation, only one slave is effectively used for TX/RX, while traffic received on other slaves is dropped. Upper-layer daemons therefore consider the interface operational, even though traffic may be blackholed if the lack of LACP negotiation means the partner is not ready to deal with traffic.

This patchset introduces an optional behavior, widely adopted across the industry, to address this issue. It consists of bringing the bonding master interface down to signal to upper-layer processes that it is not usable.

This patchset depends on the following iproute2 change:
ip/bond: add lacp_strict support

Patch 1 introduces the lacp_strict configuration knob, which is applied in the subsequent patch. The default (off) mode preserves the existing behavior, while the strict mode (on) is intended to force the bonding master carrier down in this situation.

Patch 2 addresses the core issue when lacp_strict is set to strict. It ensures that carrier is asserted only when at least 'min_links' slaves are in the Collecting/Distributing state.

Patch 3 fixes a side effect of the second patch. Tightening the carrier logic exposes a state persistence bug: when a physical link goes down, the LACP collecting/distributing flags remain set. When the link returns, the interface briefly hallucinates that it is ready, bounces the carrier up, and then drops it again once LACP renegotiation starts. Fix by resetting Collecting and Distributing state as soon as the link goes down.

Patch 4 adds a test for bonding lacp_strict both modes.

Changelog:

v3 -> v4
- Rename the configuration knob to lacp_strict on/off instead of lacp_fallback legacy/strict.
- Patch 1: change the command documentation accordingly and wrap text at approximately 75 columns.
- Use "usable" wording instead "valid" for LACP Collecting / Distributing state in code and commit log.
- Patch 2: test collecting and distributing state regardless of coupled_control
- Patch 3: Reworked because removing the SELECTED flag was not compliant with 802.1AX. Instead, to transition to the WAITING state on port disabled, except when already in the DETACHED state. And remove Collecting and Distributing state in WAITING state.
- Patch 4 is removed. It was a fix for patch 3 but it is no more needed since patch 3 was reworked.

v2 -> v3
- Add an initial patch introducing the lacp_fallback configuration knob (no behavior change yet).
- Patch 2 (was patch 1 in v2): apply the new behavior only when lacp_fallback is set to strict, and re-evaluate the bonding master carrier when the setting changes.

v1 -> v2
- Patch 1: split a comment line that exceeded 80 characters.
- Move the change from patch 2 in __agg_ports_are_ready() into a third patch, as it is actually a side effect of the fix introduced in patch 2.
- Patch 2: Expand the commit message and add a code comment describing the change in ad_port_selection_logic().
- Patch 3: Check the READY_N flag only on ports in the WAITING state, rather than on all enabled ports. This more closely matches 802.3ad.

Louis Scalbert (4):
bonding: 3ad: add lacp_strict configuration knob
bonding: 3ad: fix carrier when no valid slaves
bonding: 3ad: fix mux port state on oper down
selftests: bonding: add test for lacp_strict mode

 Documentation/networking/bonding.rst          |  23 ++
 drivers/net/bonding/bond_3ad.c                |  28 +-
 drivers/net/bonding/bond_main.c               |   1 +
 drivers/net/bonding/bond_netlink.c            |  16 +
 drivers/net/bonding/bond_options.c            |  27 ++
 include/net/bond_options.h                    |   1 +
 include/net/bonding.h                         |   1 +
 include/uapi/linux/if_link.h                  |   1 +
 .../selftests/drivers/net/bonding/Makefile    |   1 +
 .../drivers/net/bonding/bond_lacp_strict.sh   | 299 ++++++++++++++++++
 10 files changed, 397 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/drivers/net/bonding/bond_lacp_strict.sh
Signature · 1 line
2.39.2
#1 lore raw

PATCH v4 1/4 bonding: 3ad: add lacp_strict configuration knob

When an 802.3ad (LACP) bonding interface has no slaves in the collecting/distributing state, the bonding master still reports carrier as up as long as at least 'min_links' slaves have carrier.

In this situation, only one slave is effectively used for TX/RX, while traffic received on other slaves is dropped. Upper-layer daemons therefore consider the interface operational, even though traffic may be blackholed if the lack of LACP negotiation means the partner is not ready to deal with traffic.

Introduce a configuration knob to control this behavior. It allows the bonding master to assert carrier only when at least 'min_links' slaves are in Collecting_Distributing state.

The default mode preserves the existing behavior. This patch only introduces the knob; its behavior is implemented in the subsequent commit.

Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")
---
 Documentation/networking/bonding.rst | 23 +++++++++++++++++++++++
 drivers/net/bonding/bond_main.c      |  1 +
 drivers/net/bonding/bond_netlink.c   | 16 ++++++++++++++++
 drivers/net/bonding/bond_options.c   | 26 ++++++++++++++++++++++++++
 include/net/bond_options.h           |  1 +
 include/net/bonding.h                |  1 +
 include/uapi/linux/if_link.h         |  1 +
 7 files changed, 69 insertions(+)
Documentation/networking/bonding.rst +23 -0
index e700bf1d095c..33ca5afafdf6 100644
@@ -619,6 +619,29 @@ min_links
 	aggregator cannot be active without at least one available link,
 	setting this option to 0 or to 1 has the exact same effect.
 
+lacp_strict
+
+	Specifies the fallback behavior of a bonding when LACP negotiation
+	fails on all slave links, i.e. when no slave is in the
+	Collecting_Distributing state, while at least `min_links` link still
+	reports carrier up.
+
+	This option is only applicable to 802.3ad mode (mode 4).
+
+	Valid values are:
+
+	off or 0
+		One interface of the bond is selected to be active, in order to
+		facilitate communication with peer devices that do not implement
+		LACP.
+
+	on or 1
+		Interfaces are only permitted to be made active if they have an
+		active LACP partner and have successfully reached
+		Collecting_Distributing state.
+
+	The default value is 0 (off).
+
 mode
 
 	Specifies one of the bonding policies. The default is
drivers/net/bonding/bond_main.c +1 -0
index c7baa5c4bf40..b1a446630d1d 100644
@@ -6438,6 +6438,7 @@ static int __init bond_check_params(struct bond_params *params)
 	params->ad_user_port_key = ad_user_port_key;
 	params->coupled_control = 1;
 	params->broadcast_neighbor = 0;
+	params->lacp_strict = 0;
 	if (packets_per_slave > 0) {
 		params->reciprocal_packets_per_slave =
 			reciprocal_value(packets_per_slave);
drivers/net/bonding/bond_netlink.c +16 -0
index ea1a80e658ae..4b8207df4810 100644
@@ -139,6 +139,7 @@ static const struct nla_policy bond_policy[IFLA_BOND_MAX + 1] = {
 	[IFLA_BOND_NS_IP6_TARGET]	= { .type = NLA_NESTED },
 	[IFLA_BOND_COUPLED_CONTROL]	= { .type = NLA_U8 },
 	[IFLA_BOND_BROADCAST_NEIGH]	= { .type = NLA_U8 },
+	[IFLA_BOND_LACP_STRICT]		= { .type = NLA_U8 },
 };
 
 static const struct nla_policy bond_slave_policy[IFLA_BOND_SLAVE_MAX + 1] = {
@@ -595,6 +596,16 @@ static int bond_changelink(struct net_device *bond_dev, struct nlattr *tb[],
 			return err;
 	}
 
+	if (data[IFLA_BOND_LACP_STRICT]) {
+		int fallback_mode = nla_get_u8(data[IFLA_BOND_LACP_STRICT]);
+
+		bond_opt_initval(&newval, fallback_mode);
+		err = __bond_opt_set(bond, BOND_OPT_LACP_STRICT, &newval,
+				     data[IFLA_BOND_LACP_STRICT], extack);
+		if (err)
+			return err;
+	}
+
 	return 0;
 }
 
@@ -667,6 +678,7 @@ static size_t bond_get_size(const struct net_device *bond_dev)
 		nla_total_size(sizeof(struct in6_addr)) * BOND_MAX_NS_TARGETS +
 		nla_total_size(sizeof(u8)) +	/* IFLA_BOND_COUPLED_CONTROL */
 		nla_total_size(sizeof(u8)) +	/* IFLA_BOND_BROADCAST_NEIGH */
+		nla_total_size(sizeof(u8)) +	/* IFLA_BOND_LACP_STRICT */
 		0;
 }
 
@@ -834,6 +846,10 @@ static int bond_fill_info(struct sk_buff *skb,
 		       bond->params.broadcast_neighbor))
 		goto nla_put_failure;
 
+	if (nla_put_u8(skb, IFLA_BOND_LACP_STRICT,
+		       bond->params.lacp_strict))
+		goto nla_put_failure;
+
 	if (BOND_MODE(bond) == BOND_MODE_8023AD) {
 		struct ad_info info;
 
drivers/net/bonding/bond_options.c +26 -0
index 7380cc4ee75a..d358b831df77 100644
@@ -68,6 +68,8 @@ static int bond_option_lacp_active_set(struct bonding *bond,
 				       const struct bond_opt_value *newval);
 static int bond_option_lacp_rate_set(struct bonding *bond,
 				     const struct bond_opt_value *newval);
+static int bond_option_lacp_strict_set(struct bonding *bond,
+				       const struct bond_opt_value *newval);
 static int bond_option_ad_select_set(struct bonding *bond,
 				     const struct bond_opt_value *newval);
 static int bond_option_queue_id_set(struct bonding *bond,
@@ -162,6 +164,12 @@ static const struct bond_opt_value bond_lacp_rate_tbl[] = {
 	{ NULL,   -1,           0},
 };
 
+static const struct bond_opt_value bond_lacp_strict_tbl[] = {
+	{ "off", 0, BOND_VALFLAG_DEFAULT},
+	{ "on",  1, 0},
+	{ NULL, -1, 0 }
+};
+
 static const struct bond_opt_value bond_ad_select_tbl[] = {
 	{ "stable",          BOND_AD_STABLE,    BOND_VALFLAG_DEFAULT},
 	{ "bandwidth",       BOND_AD_BANDWIDTH, 0},
@@ -363,6 +371,14 @@ static const struct bond_option bond_opts[BOND_OPT_LAST] = {
 		.values = bond_lacp_rate_tbl,
 		.set = bond_option_lacp_rate_set
 	},
+	[BOND_OPT_LACP_STRICT] = {
+		.id = BOND_OPT_LACP_STRICT,
+		.name = "lacp_strict",
+		.desc = "Define the LACP fallback mode when no slaves have negotiated",
+		.unsuppmodes = BOND_MODE_ALL_EX(BIT(BOND_MODE_8023AD)),
+		.values = bond_lacp_strict_tbl,
+		.set = bond_option_lacp_strict_set
+	},
 	[BOND_OPT_MINLINKS] = {
 		.id = BOND_OPT_MINLINKS,
 		.name = "min_links",
@@ -1684,6 +1700,16 @@ static int bond_option_lacp_rate_set(struct bonding *bond,
 	return 0;
 }
 
+static int bond_option_lacp_strict_set(struct bonding *bond,
+				       const struct bond_opt_value *newval)
+{
+	netdev_dbg(bond->dev, "Setting LACP fallback to %s (%llu)\n",
+		   newval->string, newval->value);
+	bond->params.lacp_strict = newval->value;
+
+	return 0;
+}
+
 static int bond_option_ad_select_set(struct bonding *bond,
 				     const struct bond_opt_value *newval)
 {
include/net/bond_options.h +1 -0
index e6eedf23aea1..52b966e92793 100644
@@ -79,6 +79,7 @@ enum {
 	BOND_OPT_COUPLED_CONTROL,
 	BOND_OPT_BROADCAST_NEIGH,
 	BOND_OPT_ACTOR_PORT_PRIO,
+	BOND_OPT_LACP_STRICT,
 	BOND_OPT_LAST
 };
 
include/net/bonding.h +1 -0
index edd1942dcd73..2c54a36a8477 100644
@@ -129,6 +129,7 @@ struct bond_params {
 	int peer_notif_delay;
 	int lacp_active;
 	int lacp_fast;
+	int lacp_strict;
 	unsigned int min_links;
 	int ad_select;
 	char primary[IFNAMSIZ];
include/uapi/linux/if_link.h +1 -0
index 79ce4bc24cba..9ef5784e78e8 100644
@@ -1584,6 +1584,7 @@ enum {
 	IFLA_BOND_NS_IP6_TARGET,
 	IFLA_BOND_COUPLED_CONTROL,
 	IFLA_BOND_BROADCAST_NEIGH,
+	IFLA_BOND_LACP_STRICT,
 	__IFLA_BOND_MAX,
 };
 
Signature · 1 line
2.39.2
#2 lore raw

PATCH v4 2/4 bonding: 3ad: fix carrier when no usable slaves

In reply to Louis Scalbert

Apply the "lacp_strict" configuration from the previous commit.

"lacp_strict" mode "on" asserts that the bonding master carrier is up only when at least 'min_links' slaves are in the Collecting_Distributing state.

Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")
---
 drivers/net/bonding/bond_3ad.c     | 21 ++++++++++++++++++++-
 drivers/net/bonding/bond_options.c |  1 +
 2 files changed, 21 insertions(+), 1 deletion(-)
drivers/net/bonding/bond_3ad.c +20 -1
index af7f74cfdc08..9cf064243d58 100644
@@ -745,6 +745,21 @@ static void __set_agg_ports_ready(struct aggregator *aggregator, int val)
 	}
 }
 
+static int __agg_usable_ports(struct aggregator *agg)
+{
+	struct port *port;
+	int valid = 0;
+
+	for (port = agg->lag_ports; port;
+	     port = port->next_port_in_aggregator) {
+		if (port->actor_oper_port_state & LACP_STATE_COLLECTING &&
+		    port->actor_oper_port_state & LACP_STATE_DISTRIBUTING)
+			valid++;
+	}
+
+	return valid;
+}
+
 static int __agg_active_ports(struct aggregator *agg)
 {
 	struct port *port;
@@ -2120,6 +2135,7 @@ static void ad_enable_collecting_distributing(struct port *port,
 			  port->actor_port_number,
 			  port->aggregator->aggregator_identifier);
 		__enable_port(port);
+		bond_3ad_set_carrier(port->slave->bond);
 		/* Slave array needs update */
 		*update_slave_arr = true;
 		/* Should notify peers if possible */
@@ -2141,6 +2157,7 @@ static void ad_disable_collecting_distributing(struct port *port,
 			  port->actor_port_number,
 			  port->aggregator->aggregator_identifier);
 		__disable_port(port);
+		bond_3ad_set_carrier(port->slave->bond);
 		/* Slave array needs an update */
 		*update_slave_arr = true;
 	}
@@ -2820,7 +2837,9 @@ int bond_3ad_set_carrier(struct bonding *bond)
 	active = __get_active_agg(&(SLAVE_AD_INFO(first_slave)->aggregator));
 	if (active) {
 		/* are enough slaves available to consider link up? */
-		if (__agg_active_ports(active) < bond->params.min_links) {
+		if ((bond->params.lacp_strict ? __agg_usable_ports(active)
+					: __agg_active_ports(active)) <
+		    bond->params.min_links) {
 			if (netif_carrier_ok(bond->dev)) {
 				netif_carrier_off(bond->dev);
 				goto out;
drivers/net/bonding/bond_options.c +1 -0
index d358b831df77..94b7b0851f16 100644
@@ -1706,6 +1706,7 @@ static int bond_option_lacp_strict_set(struct bonding *bond,
 	netdev_dbg(bond->dev, "Setting LACP fallback to %s (%llu)\n",
 		   newval->string, newval->value);
 	bond->params.lacp_strict = newval->value;
+	bond_3ad_set_carrier(bond);
 
 	return 0;
 }
Signature · 1 line
2.39.2
#3 lore raw

PATCH v4 3/4 bonding: 3ad: fix mux port state on oper down

In reply to Louis Scalbert

When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.

When lacp_strict mode is on, the interface should not transition to carrier up until LACP negotiation is complete.

This happens because the actor and partner port states remain in Collecting_Distributing when the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.

Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.

Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.

Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")
---
 drivers/net/bonding/bond_3ad.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/net/bonding/bond_3ad.c +7 -0
index 9cf064243d58..bc2964ea11f5 100644
@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
 
 	if (port->sm_vars & AD_PORT_BEGIN) {
 		port->sm_mux_state = AD_MUX_DETACHED;
+	} else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
+		port->sm_mux_state = AD_MUX_WAITING;
 	} else {
 		switch (port->sm_mux_state) {
 		case AD_MUX_DETACHED:
@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
 			break;
 		case AD_MUX_WAITING:
 			port->sm_mux_timer_counter = __ad_timer_to_ticks(AD_WAIT_WHILE_TIMER, 0);
+			port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION;
+			ad_disable_collecting_distributing(port,
+							   update_slave_arr);
+			port->actor_oper_port_state &= ~LACP_STATE_COLLECTING;
+			port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING;
 			break;
 		case AD_MUX_ATTACHED:
 			if (port->aggregator->is_active)
Signature · 1 line
2.39.2
maintainer
#4 lore raw

Re: [PATCH net v4 3/4] bonding: 3ad: fix mux port state on oper down

Quoted 32 lines · Louis Scalbert

When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.

When lacp_strict mode is on, the interface should not transition to carrier up until LACP negotiation is complete.

This happens because the actor and partner port states remain in Collecting_Distributing when the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.

Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.

Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.

Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")

---
drivers/net/bonding/bond_3ad.c | 7 +++++++

1 file changed, 7 insertions(+)
drivers/net/bonding/bond_3ad.c +0 -0
index 9cf064243d58..bc2964ea11f5 100644

drivers/net/bonding/bond_3ad.c static void ad_mux_machine(struct port *port, bool *update_slave_arr) · line 1057

Quoted 5 lines
drivers/net/bonding/bond_3ad.c +2 -0
@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
 
 	if (port->sm_vars & AD_PORT_BEGIN) {
 		port->sm_mux_state = AD_MUX_DETACHED;
+	} else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
+		port->sm_mux_state = AD_MUX_WAITING;

Technically, this is not exactly following the state machines.

The mux machine should transition to WAITING from DETACHED when
Selected == SELECTED or STANDBY, not for !is_enabled ("port_enabled" in the standard). The check for !is_enabled happens in the receive machine, and it would transition to PORT_DISABLED state (which clears Synchronization).

I'm not sure if this is actually an issue or not; I need to read
the relevant bits again to make sure I understand how it's supposed to work.

	-J
Quoted 18 lines
drivers/net/bonding/bond_3ad.c +5 -0
 	} else {
 		switch (port->sm_mux_state) {
 		case AD_MUX_DETACHED:
@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
 			break;
 		case AD_MUX_WAITING:
 			port->sm_mux_timer_counter = __ad_timer_to_ticks(AD_WAIT_WHILE_TIMER, 0);
+			port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION;
+			ad_disable_collecting_distributing(port,
+							   update_slave_arr);
+			port->actor_oper_port_state &= ~LACP_STATE_COLLECTING;
+			port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING;
 			break;
 		case AD_MUX_ATTACHED:
 			if (port->aggregator->is_active)
Signature · 1 line
2.39.2

---
-Jay Vosburgh, [email protected]

#5 lore raw

Re: [PATCH net v4 3/4] bonding: 3ad: fix mux port state on oper down

Hello Jay,

Sorry for the late reply. I’ve been busy with another project these past few days.

Quoted 43 lines · Jay Vosburgh
Quoted 32 lines · Louis Scalbert

When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.

When lacp_strict mode is on, the interface should not transition to carrier up until LACP negotiation is complete.

This happens because the actor and partner port states remain in Collecting_Distributing when the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.

Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.

Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.

Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")

---
drivers/net/bonding/bond_3ad.c | 7 +++++++

1 file changed, 7 insertions(+)
drivers/net/bonding/bond_3ad.c +0 -0
index 9cf064243d58..bc2964ea11f5 100644

drivers/net/bonding/bond_3ad.c static void ad_mux_machine(struct port *port, bool *update_slave_arr) · line 1057

Quoted 5 lines
drivers/net/bonding/bond_3ad.c +2 -0
@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
 
      if (port->sm_vars & AD_PORT_BEGIN) {
              port->sm_mux_state = AD_MUX_DETACHED;
+      } else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
+              port->sm_mux_state = AD_MUX_WAITING;

Technically, this is not exactly following the state machines.

Quoted 3 lines

The mux machine should transition to WAITING from DETACHED when
Selected == SELECTED or STANDBY, not for !is_enabled ("port_enabled" in the standard).

The MUX machine still transitions from DETACHED to WAITING; this happens a few lines later and is unchanged.

The relevant code is:

} else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
    port->sm_mux_state = AD_MUX_WAITING;
} else {
     switch (port->sm_mux_state) {
     case AD_MUX_DETACHED:
          if ((port->sm_vars & AD_PORT_SELECTED)
              || (port->sm_vars & AD_PORT_STANDBY))
          /* if SELECTED or STANDBY */
          port->sm_mux_state = AD_MUX_WAITING;
          break;

My change is for NOT is_enabled AND NOT AD_MUX_DETACHED.

Quoted 3 lines

The check for !is_enabled happens in the receive
machine, and it would transition to PORT_DISABLED state (which clears Synchronization).

I agree with that statement. However, clearing Synchronization is not
sufficient: the purpose of the fix is to clear Collecting and Distributing.

I noticed that is_enabled is defined in the 802.3ad-2000 standard, while Port_Operational is defined in 802.1AX-2020. I am not sure about 802.1AX-2014, as I do not have access to that version.

The 802.1AX-2020 standard uses a different MUX state diagram. Compared to 802.3ad, the ATTACHED state has been split into ATTACH and ATTACHED. There is no longer a WAITING state; it has been replaced by ATTACHED_WTR.

The new standard says that the MUX machine should transition to ATTACHED_WTR when Port_Operational is FALSE and the current state is not DETACHED.

So, in my opinion, the change is correct, at least with respect to 802.1AX-2020.

Quoted 4 lines

I'm not sure if this is actually an issue or not; I need to read
the relevant bits again to make sure I understand how it's supposed to work.

Please confirm what you want me to do: should I keep the fix as it is ?

best regards,

Louis Scalbert

Quoted 24 lines
        -J
Quoted 16 lines
drivers/net/bonding/bond_3ad.c +5 -1
      } else {
              switch (port->sm_mux_state) {
              case AD_MUX_DETACHED:
@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
                      break;
              case AD_MUX_WAITING:
                      port->sm_mux_timer_counter = __ad_timer_to_ticks(AD_WAIT_WHILE_TIMER, 0);
+                      port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION;
+                      ad_disable_collecting_distributing(port,
+                                                         update_slave_arr);
+                      port->actor_oper_port_state &= ~LACP_STATE_COLLECTING;
+                      port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING;
                      break;
              case AD_MUX_ATTACHED:
                      if (port->aggregator->is_active)
--
Quoted 2 lines

2.39.2

---
-Jay Vosburgh, [email protected]

maintainer
#6 lore raw

Re: [PATCH net v4 3/4] bonding: 3ad: fix mux port state on oper down

Quoted 75 lines · Louis Scalbert

Hello Jay,

Sorry for the late reply. I’ve been busy with another project these past few days.

Quoted 43 lines · Jay Vosburgh
Quoted 32 lines · Louis Scalbert

When the bonding interface has carrier down due to the absence of usable slaves and a slave transitions from down to up, the bonding interface briefly goes carrier up, then down again, and finally up once LACP negotiates collecting and distributing on the port.

When lacp_strict mode is on, the interface should not transition to carrier up until LACP negotiation is complete.

This happens because the actor and partner port states remain in Collecting_Distributing when the port goes down. When the port comes back up, it temporarily remains in this state until LACP renegotiation occurs.

Previously this was mostly cosmetic, but since the bonding carrier state may depend on the LACP negotiation state, it causes the interface to flap.

Move an operationally down port to the Mux WAITING state and clear the Synchronization, Collecting, and Distributing states, in accordance with the 802.1AX Mux state machine diagram.

Fixes 655f8919d549 ("bonding: add min links parameter to 802.3ad")

---
drivers/net/bonding/bond_3ad.c | 7 +++++++

1 file changed, 7 insertions(+)
drivers/net/bonding/bond_3ad.c +0 -0
index 9cf064243d58..bc2964ea11f5 100644

drivers/net/bonding/bond_3ad.c static void ad_mux_machine(struct port *port, bool *update_slave_arr) · line 1057

Quoted 5 lines
drivers/net/bonding/bond_3ad.c +2 -0
@@ -1053,6 +1053,8 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
 
      if (port->sm_vars & AD_PORT_BEGIN) {
              port->sm_mux_state = AD_MUX_DETACHED;
+      } else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
+              port->sm_mux_state = AD_MUX_WAITING;

Technically, this is not exactly following the state machines.

Quoted 3 lines

The mux machine should transition to WAITING from DETACHED when
Selected == SELECTED or STANDBY, not for !is_enabled ("port_enabled" in the standard).

The MUX machine still transitions from DETACHED to WAITING; this happens a few lines later and is unchanged.

The relevant code is:

} else if (!port->is_enabled && port->sm_mux_state != AD_MUX_DETACHED) {
   port->sm_mux_state = AD_MUX_WAITING;
} else {
    switch (port->sm_mux_state) {
    case AD_MUX_DETACHED:
         if ((port->sm_vars & AD_PORT_SELECTED)
             || (port->sm_vars & AD_PORT_STANDBY))
         /* if SELECTED or STANDBY */
         port->sm_mux_state = AD_MUX_WAITING;
         break;

My change is for NOT is_enabled AND NOT AD_MUX_DETACHED.

Quoted 3 lines

The check for !is_enabled happens in the receive
machine, and it would transition to PORT_DISABLED state (which clears Synchronization).

Quoted 7 lines

I agree with that statement. However, clearing Synchronization is not
sufficient: the purpose of the fix is to clear Collecting and Distributing.

I noticed that is_enabled is defined in the 802.3ad-2000 standard, while Port_Operational is defined in 802.1AX-2020. I am not sure about 802.1AX-2014, as I do not have access to that version.

	Did you mean 802.1AX-2014 instead of 802.3ad-2000 here?
Quoted 11 lines

The 802.1AX-2020 standard uses a different MUX state diagram. Compared to 802.3ad, the ATTACHED state has been split into ATTACH and ATTACHED. There is no longer a WAITING state; it has been replaced by ATTACHED_WTR.

The new standard says that the MUX machine should transition to ATTACHED_WTR when Port_Operational is FALSE and the current state is not DETACHED.

So, in my opinion, the change is correct, at least with respect to 802.1AX-2020.

I'm concerned that mixing versions of the standard will cause
issues, precisely because the state machines vary between the 2014 and 2020 versions.

As the standards are not explicitly backwards compatible,
bonding should conform to one version, and in my opinion the version that makes the most sense is the 2014 edition. Bonding is already very close to the 2014 version, while compliance with the 2020 version would require significant changes.

The 2014 edition is freely available from the IEEE, for example at:

https://1.ieee802.org/tsn/802-1ax-2014/

Quoted 6 lines
Quoted 4 lines

I'm not sure if this is actually an issue or not; I need to read
the relevant bits again to make sure I understand how it's supposed to work.

Please confirm what you want me to do: should I keep the fix as it is ?

I see that you've reposted the patch series, I'll reply there
later when I've had a chance to study the details.

	-J
Quoted 27 lines

best regards,

Louis Scalbert

Quoted 24 lines
        -J
Quoted 16 lines
drivers/net/bonding/bond_3ad.c +5 -1
      } else {
              switch (port->sm_mux_state) {
              case AD_MUX_DETACHED:
@@ -1200,6 +1202,11 @@ static void ad_mux_machine(struct port *port, bool *update_slave_arr)
                      break;
              case AD_MUX_WAITING:
                      port->sm_mux_timer_counter = __ad_timer_to_ticks(AD_WAIT_WHILE_TIMER, 0);
+                      port->actor_oper_port_state &= ~LACP_STATE_SYNCHRONIZATION;
+                      ad_disable_collecting_distributing(port,
+                                                         update_slave_arr);
+                      port->actor_oper_port_state &= ~LACP_STATE_COLLECTING;
+                      port->actor_oper_port_state &= ~LACP_STATE_DISTRIBUTING;
                      break;
              case AD_MUX_ATTACHED:
                      if (port->aggregator->is_active)
--
Quoted 2 lines

2.39.2

---
-Jay Vosburgh, [email protected]

---
-Jay Vosburgh, [email protected]

#7 lore raw

PATCH v4 4/4 selftests: bonding: add test for lacp_strict mode

In reply to Louis Scalbert

Add a test for the bonding lacp_strict mode.

---
 .../selftests/drivers/net/bonding/Makefile    |   1 +
 .../drivers/net/bonding/bond_lacp_strict.sh   | 299 ++++++++++++++++++
 2 files changed, 300 insertions(+)
 create mode 100755 tools/testing/selftests/drivers/net/bonding/bond_lacp_strict.sh
tools/testing/selftests/drivers/net/bonding/Makefile +1 -0
index 9af5f84edd37..91269e7ceb63 100644
@@ -7,6 +7,7 @@ TEST_PROGS := \
 	bond-eth-type-change.sh \
 	bond-lladdr-target.sh \
 	bond_ipsec_offload.sh \
+	bond_lacp_strict.sh \
 	bond_lacp_prio.sh \
 	bond_macvlan_ipvlan.sh \
 	bond_options.sh \
tools/testing/selftests/drivers/net/bonding/bond_lacp_strict.sh new file +299 -0

299 lines not on this page · raw message

Signature · 1 line
2.39.2
maintainer
#8 lore raw

Re: [PATCH net v4 4/4] selftests: bonding: add test for lacp_strict mode

Quoted 1 line · On Fri, 17 Apr 2026 16:05:05 +0200 Louis Scalbert wrote:

+ip netns exec "${p_ns}" tc qdisc add dev eth0 root netem loss 100%

netem is not included the in the config for bonding tests, this fails in the CI.