Skip to contentSkip to conversation

ixgbe: force to synchronize reporting "link on" and getting speed and duplex

PATCH Superseded [email protected] 6 participants 39 messages Dec 23, 2015 – Jan 7, 2016

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |   16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
 	/* enable transmits */
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 54 lines · On Wed, 2015-12-23 at 14:46 +0800, zyjzyj2000 at gmail.com wrote:

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of
the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
?drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |???16
+++++++++++++++-
?1 file changed, 15 insertions(+), 1 deletion(-)

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)

? ???????(flow_rx ? "RX" :
? ???????(flow_tx ? "TX" : "None"))));
?

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
-	netif_carrier_on(netdev);
+	/*
+	?* In X540 NIC, there is a time span between reporting "link

on"

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	?* and getting the speed and duplex. To a bonding driver in

802.3ad

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	?* mode, this time span will make it not work well if the

time span

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	?* is big enough. To 82599_SFP NIC and other kinds of NICs,

this

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	?* problem does not exist. As such, it is better for X540 to

report

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	?* "link on" when the link speed is not

IXGBE_LINK_SPEED_UNKNOWN.

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8 -0
+	?*/
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	????(link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
Quoted 3 lines

? ixgbe_check_vf_rate_limit(adapter);
?
? /* enable transmits */

This patch only adds a needless test before
calling?netif_carrier_on(netdev), since the call happens no matter the
branch you take, it appears your running into a timing issue. ?So
adding a wait() before calling?netif_carrier_on(netdev) will accomplish
the same result and you do not have to add a useless test.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151223/f955f020/attachment.asc>

#2 lore raw

PATCH v2 1/1 ixgbe: force to synchronize reporting "link on" and

Hi, Jeff

Thanks for your reply.

Changes:
Since there is a time span between link_up and link_speed to X540 NIC, it is not appropriate to continue in the function ixgbe_watchdog_link_is_up if only link_up is ready.

Best Regards!
Zhu Yanjun

#3 lore raw

PATCH 1/1 ixgbe: force to synchronize reporting "link on" and getting speed

From: yzhu1 <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |   15 +++++++++++++++
 1 file changed, 15 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -0
index aed8d02..2b58a33 100644
@@ -6426,6 +6426,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	if (netif_carrier_ok(netdev))
 		return;
 
+	/* In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 * To other NICs, the link_up and link_speed are gotten at the same
+	 * time. To X540 NIC, there is a time span between link_up and
+	 * link_speed. As such, only continue if link_up and link_speed are
+	 * ready to X540 NIC.
+	 */
+	if (hw->mac.type == ixgbe_mac_X540)
+		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
+			return;
+
 	adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP;
 
 	switch (hw->mac.type) {
Signature · 1 line
1.7.9.5
author
#4 lore raw

Re: [V2 PATCH 1/1] ixgbe: force to synchronize reporting "link on" and

In reply to [email protected]
Quoted 7 lines · On 12/24/2015 11:12 AM, zyjzyj2000 at gmail.com wrote:

Hi, Jeff

Thanks for your reply.

Changes:
Since there is a time span between link_up and link_speed to X540 NIC, it is not appropriate to continue in the function ixgbe_watchdog_link_is_up if only link_up is ready.

Quoted 7 lines

Best Regards!
Zhu Yanjun
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi, Jeff

The following logs are from the customer. From these logs, there is a time span between link_up and link_speed.
This time span sometimes will make bonding in 802.3ad mode not work well.

What I have done in the patch is to prevent this time span in the ixgbe driver.

Best Regards!
Zhu Yanjun

Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151224/ca9d7c56/attachment.html>

#5 lore raw

RE: [Intel-wired-lan] [V2 PATCH 1/1] ixgbe: force to synchronize reporting "link on" and

Quoted 178 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zhuyj
Sent: Wednesday, December 23, 2015 9:11 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [V2 PATCH 1/1] ixgbe: force to synchronize reporting "link on" and

On 12/24/2015 11:12 AM, zyjzyj2000 at gmail.com <mailto:[email protected]> wrote:

Hi, Jeff

Thanks for your reply.

Changes:
Since there is a time span between link_up and link_speed to X540
NIC, it is not appropriate to continue in the function ixgbe_watchdog_link_is_up if only link_up is ready.

Best Regards!
Zhu Yanjun
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo at vger.kernel.org
<mailto:[email protected]>
More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi, Jeff

The following logs are from the customer. From these logs, there is a time span between link_up and link_speed.
This time span sometimes will make bonding in 802.3ad mode not work well.

What I have done in the patch is to prevent this time span in the ixgbe driver.

Best Regards!
Zhu Yanjun

Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full

                           1000baseT/Full
                           10000baseT/Full
   Supported pause frame use: No
   Supports auto-negotiation: Yes
   Advertised link modes:  100baseT/Full
                           1000baseT/Full
                           10000baseT/Full

Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full

                           1000baseT/Full
                           10000baseT/Full
   Supported pause frame use: No
   Supports auto-negotiation: Yes
   Advertised link modes:  100baseT/Full
                           1000baseT/Full
                           10000baseT/Full

Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full

                           1000baseT/Full
                           10000baseT/Full
   Supported pause frame use: No
   Supports auto-negotiation: Yes
   Advertised link modes:  100baseT/Full
                           1000baseT/Full
                           10000baseT/Full

Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full

                           1000baseT/Full
                           10000baseT/Full
   Supported pause frame use: No
   Supports auto-negotiation: Yes
   Advertised link modes:  100baseT/Full
                           1000baseT/Full
                           10000baseT/Full

Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full

                           1000baseT/Full
                           10000baseT/Full
   Supported pause frame use: No
   Supports auto-negotiation: Yes
   Advertised link modes:  100baseT/Full
                           1000baseT/Full
                           10000baseT/Full

Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full

                           1000baseT/Full
                           10000baseT/Full
   Supported pause frame use: No
   Supports auto-negotiation: Yes
   Advertised link modes:  100baseT/Full
                           1000baseT/Full
                           10000baseT/Full
Quoted 14 lines

Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes

Can you provide the related dmesg output? What did the driver report in the "Link Up" messages?

Thanks,
Emil

#6 lore raw

PATCH v3 1/1 ixgbe: force to synchronize link_up and speed as a slave

In reply to [email protected]

Hi, all

During the discussion with Emil, I think the time span between link_up and link_speed is not important when the X540 NIC acts as an independent interface. As such, a new patch is made to restrict synchronization between link_up and link_speed.

Any reply is appreciated.

Zhu Yanjun

#7 lore raw

PATCH 1/2 ixgbe: force to synchronize reporting "link on" and getting speed

From: yzhu1 <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |   15 +++++++++++++++
 1 file changed, 15 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -0
index aed8d02..ace21b9 100644
@@ -6426,6 +6426,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	if (netif_carrier_ok(netdev))
 		return;
 
+	/* In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 * To other NICs, the link_up and link_speed are gotten at the same
+	 * time. To X540 NIC, there is a time span between link_up and
+	 * link_speed. As such, only continue if link_up and link_speed are
+	 * ready to X540 NIC.
+	 */
+	if (hw->mac.type == ixgbe_mac_X540)
+		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
+			return;
+
 	adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP;
 
 	switch (hw->mac.type) {
Signature · 1 line
1.7.9.5
#8 lore raw

PATCH 2/2 ixgbe: restrict synchronization of link_up and speed

In reply to [email protected]

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8 -1
index ace21b9..1bb6056 100644
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	 * time. To X540 NIC, there is a time span between link_up and
 	 * link_speed. As such, only continue if link_up and link_speed are
 	 * ready to X540 NIC.
+	 * The time span between link_up and link_speed is very important
+	 * when the X540 NIC acts as a slave in some virtual NICs, such as
+	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
+	 * independent interface, it is not necessary to synchronize link_up
+	 * and link_speed.
+	 * In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN)
 	 */
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
 			return;
 
Signature · 1 line
1.7.9.5
#9 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 13 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 7 lines

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and link_speed"? Where is it you think the de-synchronization occurs?

Quoted 12 lines

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 9 ++++++++-

1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index ace21b9..1bb6056 100644
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6446

Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6 -0
 	 * time. To X540 NIC, there is a time span between link_up and
 	 * link_speed. As such, only continue if link_up and link_speed are
 	 * ready to X540 NIC.
+	 * The time span between link_up and link_speed is very important
+	 * when the X540 NIC acts as a slave in some virtual NICs, such as
+	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
+	 * independent interface, it is not necessary to synchronize link_up
+	 * and link_speed.
+	 * In the end, not continue if (X540 NIC && SLAVE && link_speed
 UNKNOWN)

This is a patch on top of your previous patch which I don't think was applied, so this is not going to apply cleanly.

Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
 	 */
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
			return;

If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"

by the interface you use in the bond?

Thanks,
Emil

#10 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 11 lines · Emil S
Quoted 6 lines

*/

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
			return;

If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"

Quoted 1 line

by the interface you use in the bond?

It seems odd to be checking the MAC type for this. Is this behavior perhaps more related to the copper phy? If so, then the check should be changed. Or would the check for unknown link speed be sufficient? It seems like an interface as a slave would not work with an unknown link speed, so it may as well wait in all cases, not just for X540.

--
Mark Rustad, Networking Division, Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151229/775e8321/attachment-0001.asc>

author
#11 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 12 lines · Mark D
Quoted 10 lines · Emil S
Quoted 6 lines

*/

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
			return;

If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"

Quoted 5 lines
Quoted 1 line

by the interface you use in the bond?

It seems odd to be checking the MAC type for this. Is this behavior perhaps more related to the copper phy? If so, then the check should be changed. Or would the check for unknown link speed be sufficient? It seems like an interface as a slave would not work with an unknown link speed, so it may as well wait in all cases, not just for X540.

--
Mark Rustad, Networking Division, Intel Corporation

Hi, Mark

Thanks for your suggestions.
The following is the feedback from the customer.
"
...
We observing this issue on x540T interfaces (8086:1528), but not on 82599_SFP (8086:10FB).
...
"
To narrow this problem, I restrict mac.type to ixgbe_mac_X540. I agree with you. Maybe this problem is
related with the copper phy. But I only have X540 NIC to test. So it is difficult for me to confirm whether this
problem occurs on other mac type or not, such as X550.

I will consider your suggestions in the latest patch.

Thanks again.

Zhu Yanjun

#12 lore raw

PATCH v4 1/1 ixgbe: synchronize the link_speed and link_up of a slave interface

In reply to Rustad, Mark D

Hi, all

According to Rustad, Mark D, maybe it is related with copper phy. To make fiber phy more robust, synchronize both the link_up and link_speed of a slave interface in ixgbe driver.

Best Regards!
Zhu Yanjun

#13 lore raw

PATCH 1/3 ixgbe: force to synchronize reporting "link on" and getting speed

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |   15 +++++++++++++++
 1 file changed, 15 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -0
index aed8d02..ace21b9 100644
@@ -6426,6 +6426,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	if (netif_carrier_ok(netdev))
 		return;
 
+	/* In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 * To other NICs, the link_up and link_speed are gotten at the same
+	 * time. To X540 NIC, there is a time span between link_up and
+	 * link_speed. As such, only continue if link_up and link_speed are
+	 * ready to X540 NIC.
+	 */
+	if (hw->mac.type == ixgbe_mac_X540)
+		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
+			return;
+
 	adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP;
 
 	switch (hw->mac.type) {
Signature · 1 line
1.7.9.5
#14 lore raw

PATCH 2/3 ixgbe: restrict synchronization of link_up and speed

In reply to [email protected]

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8 -1
index ace21b9..1bb6056 100644
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	 * time. To X540 NIC, there is a time span between link_up and
 	 * link_speed. As such, only continue if link_up and link_speed are
 	 * ready to X540 NIC.
+	 * The time span between link_up and link_speed is very important
+	 * when the X540 NIC acts as a slave in some virtual NICs, such as
+	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
+	 * independent interface, it is not necessary to synchronize link_up
+	 * and link_speed.
+	 * In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN)
 	 */
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
 			return;
 
Signature · 1 line
1.7.9.5
#15 lore raw

PATCH 3/3 ixgbe: synchronize the link_speed and link_up of a slave interface

In reply to [email protected]

From: Zhu Yanjun <[email protected]>

According to the suggestion from Rustad, Mark D, this behavior perhaps is more related to the copper phy. But to make fiber phy more robust, to all the interfaces as a slave interface, the link_speed and link_up is synchronized.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -3
index 1bb6056..ce47639 100644
@@ -6441,10 +6441,12 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
 	 * independent interface, it is not necessary to synchronize link_up
 	 * and link_speed.
-	 * In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN)
+	 * According to the suggestion from Rustad, Mark D, this behavior
+	 * perhaps is related to the copper phy. To make fiber phy more robust,
+	 * To all the interfaces as a slave, the link_speed is checked.
+	 * In the end, not continue if (SLAVE && link_speed UNKNOWN)
 	 */
-	if ((hw->mac.type == ixgbe_mac_X540) &&
-	    (netdev->flags & IFF_SLAVE))
+	if (netdev->flags & IFF_SLAVE)
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
 			return;
 
Signature · 1 line
1.7.9.5
#16 lore raw

Re: [PATCH 3/3] ixgbe: synchronize the link_speed and link_up of a slave interface

Quoted 17 lines · zyjzyj2000 at gmail.com wrote:

From: Zhu Yanjun <[email protected]>

According to the suggestion from Rustad, Mark D, this behavior perhaps is more related to the copper phy. But to make fiber phy more robust, to all the interfaces as a slave interface, the link_speed and link_up is synchronized.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 8 +++++---

1 file changed, 5 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index 1bb6056..ce47639 100644

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) · line 6447

Quoted 8 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +4 -1
@@ -6441,10 +6441,12 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
 	 * independent interface, it is not necessary to synchronize link_up
 	 * and link_speed.
-	 * In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN)
+	 * According to the suggestion from Rustad, Mark D, this behavior
+	 * perhaps is related to the copper phy. To make fiber phy more robust,
+	 * To all the interfaces as a slave, the link_speed is checked.
+	 * In the end, not continue if (SLAVE && link_speed UNKNOWN)

There is no need to make reference to my suggestion in the comment, especially since that is in the commit message. Please simplify your comment above to be something like:
* For all slave interfaces, wait for the link_speed to be known.

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) · line 6451

Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -2
 	 */
-	if ((hw->mac.type == ixgbe_mac_X540) &&
-	    (netdev->flags & IFF_SLAVE))
+	if (netdev->flags & IFF_SLAVE)
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
 			return;

The above would be better as:

	if ((netdev->flags & IFF_SLAVE) &&
	    link_speed == IXGBE_LINK_SPEED_UNKNOWN)
		return;

Please do not send a series of patches - it just adds needless confusion and is a bisect hazard. Just send a single patch with the desired change as a V5.

--
Mark Rustad, Networking Division, Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151230/5415a7a9/attachment.asc>

#17 lore raw

PATCH v5 1/1 ixgbe: synchronize link_up and link_speed of a slave

Hi, all

Thanks for the suggestions from Rustad, Mark D.
According to his suggestions, the logs and source code are simplified.

Zhu Yanjun

#18 lore raw

PATCH 1/1 ixgbe: synchronize link_up and link_speed of a slave interface

From: Zhu Yanjun <[email protected]>

According to the suggestions from Rustad, Mark D, to all the slave interfaces, the link_speed and link_up should be synchronized since the time span between link_up and link_speed will make some virtual NICs not work well, such as a bonding driver in 802.3ad mode.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -0
index aed8d02..fc461b9 100644
@@ -6426,6 +6426,11 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	if (netif_carrier_ok(netdev))
 		return;
 
+	/* For all slave interfaces, wait for the link_speed to be known. */
+	if ((netdev->flags & IFF_SLAVE) &&
+	    (link_speed == IXGBE_LINK_SPEED_UNKNOWN))
+		return;
+
 	adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP;
 
 	switch (hw->mac.type) {
Signature · 1 line
1.9.1
#19 lore raw

Re: [PATCH 1/1] ixgbe: synchronize link_up and link_speed of a slave interface

Quoted 9 lines · On Thu, 2015-12-31 at 13:04 +0800, zyjzyj2000 at gmail.com wrote:

From: Zhu Yanjun <[email protected]>

According to the suggestions from Rustad, Mark D, to all the slave? interfaces, the link_speed and link_up should be synchronized since the time span between link_up and link_speed will make some virtual NICs not work well, such as a bonding driver in 802.3ad mode.

---

Since this is version 5 of your original patch, you should be putting the change log of this patch here, so that we can follow how you got to this point. ?This helpful for the reviewers (and me, the maintainer). ?For example:
v2: dropped the "else" case of "if" statement, based on feedback from ? ? Jeff Kirsher
v3: changed the if statement to simply return when the link speed is ? ? unknown on X540 parts based on feedback from Emil Tantilov
v4: updated code comment and if statement to test for IFF_SLAVE flag
V5: simplified code comment and if statement to only test for IFF_SLAVE ? ? flag and unknown link speed.

Of course, you can expand on what I have above, I just did a quick summary of the changes as an example of a change log.?

Quoted 2 lines

?drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++++ ?1 file changed, 5 insertions(+)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151230/dd49f916/attachment-0001.asc>

#20 lore raw

PATCH v6 1/1 ixgbe: synchronize link_up and link_speed of a slave

Hi, all

Thanks for the reply from Jeff.

V2: Based on feedback from Jeff Kirsher, it is not appropriate to continue
in the function ixgbe_watchdog_link_is_up without link_speed since this will make some virtual NICs not work well.

V3: Based on feedback from Emil Tantilov, the time span between link_up
and link_speed is not important when the X540 NIC acts as an independent interface.

V4: According to Rustad, Mark D, maybe it is related with copper phy. To make
fiber phy more robust, synchronize both the link_up and link_speed of a slave interface in ixgbe driver.

V5: Based on feedback from Rustad, Mark D, simplify code comment and if statement
to only test for IFF_SLAVE flag and unknown link speed.

V6: Based on feedback from Jeff Kirsher, the patch format is adjusted
and change log of this patch are added.

Best Regards!
Zhu Yanjun

#21 lore raw

PATCH v6 1/1 ixgbe: synchronize link_up and link_speed of a slave interface

From: Zhu Yanjun <[email protected]>

According to the suggestions from Rustad, Mark D, to all the slave interfaces, the link_speed and link_up should be synchronized since the time span between link_up and link_speed will make some virtual NICs not work well, such as a bonding driver in 802.3ad mode.

---
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -0
index aed8d02..fc461b9 100644
@@ -6426,6 +6426,11 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 	if (netif_carrier_ok(netdev))
 		return;
 
+	/* For all slave interfaces, wait for the link_speed to be known. */
+	if ((netdev->flags & IFF_SLAVE) &&
+	    (link_speed == IXGBE_LINK_SPEED_UNKNOWN))
+		return;
+
 	adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP;
 
 	switch (hw->mac.type) {
Signature · 1 line
1.9.1
#22 lore raw

Re: [PATCH V5] ixgbe: synchronize link_up and link_speed of a slave

In reply to [email protected]
Quoted 3 lines · On Thu, 2015-12-31 at 13:04 +0800, zyjzyj2000 at gmail.com wrote:

Thanks for the suggestions from Rustad, Mark D. According to his suggestions, the logs and source code are simplified.

I find it funny that this email (no patch) is got the correct subject,
yet the updated patch you sent does not. ?The subject line for this
email should have been used for the updated patch you sent out.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151230/110ef944/attachment.asc>

author
#23 lore raw

Re: [PATCH V5] ixgbe: synchronize link_up and link_speed of a slave

Quoted 7 lines · Jeff Kirsher
Quoted 3 lines · On Thu, 2015-12-31 at 13:04 +0800, zyjzyj2000 at gmail.com wrote:

Thanks for the suggestions from Rustad, Mark D. According to his suggestions, the logs and source code are simplified.

I find it funny that this email (no patch) is got the correct subject, yet the updated patch you sent does not. The subject line for this email should have been used for the updated patch you sent out.

Thanks for your reply.
I will resend the patch soon.

Best Regards!
Zhu Yanjun

author
#24 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

In reply to Tantilov, Emil S
Quoted 13 lines · Emil S
Quoted 13 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 8 lines
Quoted 7 lines

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and link_speed"?

In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag is set, we continue to check
link_speed. If not, this function is executed whether this link_speed is unknown or not.

Quoted 36 lines
Quoted 13 lines

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 9 ++++++++-

1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index ace21b9..1bb6056 100644
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6446

Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6 -0
 	 * time. To X540 NIC, there is a time span between link_up and
 	 * link_speed. As such, only continue if link_up and link_speed are
 	 * ready to X540 NIC.
+	 * The time span between link_up and link_speed is very important
+	 * when the X540 NIC acts as a slave in some virtual NICs, such as
+	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
+	 * independent interface, it is not necessary to synchronize link_up
+	 * and link_speed.
+	 * In the end, not continue if (X540 NIC && SLAVE && link_speed
 UNKNOWN)

This is a patch on top of your previous patch which I don't think was applied, so this is not going to apply cleanly.

Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
 	 */
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
			return;

If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"

Quoted 1 line

by the interface you use in the bond?

Sure. There is a dmesg log from the customer.
"
...

2015-10-05T06:14:34.350 controller-0 kernel: info bonding: bond0: link 

status definitely up for interface eth0, 0 Mbps full duplex.
...
"

Thanks a lot.
Zhu Yanjun

Quoted 4 lines

Thanks,
Emil

#25 lore raw

RE: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 37 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization

of

Quoted 9 lines
Quoted 9 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

Quoted 7 lines
Quoted 1 line

What exactly do you mean by "time span between link_up and link_speed"?

In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Was this when reporting the link state from check_link() (reading the LINKS register) or reporting the adapter->link_speed?

Quoted 8 lines
Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag is set, we continue to check
link_speed. If not, this function is executed whether this link_speed is unknown or not.

I can already see this in your patch. I was asking about the reason why your change is needed.

Quoted 38 lines
Quoted 16 lines
Quoted 16 lines

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 9 ++++++++-

1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index ace21b9..1bb6056 100644
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	 * time. To X540 NIC, there is a time span between link_up and
 	 * link_speed. As such, only continue if link_up and link_speed are
 	 * ready to X540 NIC.
Quoted 8 lines

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)

Quoted 7 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6 -0
+	 * The time span between link_up and link_speed is very important
+	 * when the X540 NIC acts as a slave in some virtual NICs, such as
+	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
+	 * independent interface, it is not necessary to synchronize link_up
+	 * and link_speed.
+	 * In the end, not continue if (X540 NIC && SLAVE && link_speed
 UNKNOWN)

This is a patch on top of your previous patch which I don't think was

applied,

Quoted 9 lines

so this is not going to apply cleanly.

Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
 	 */
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
			return;

If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then

I would

Quoted 3 lines

assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"

Quoted 6 lines
Quoted 1 line

by the interface you use in the bond?

Sure. There is a dmesg log from the customer.
"
...

2015-10-05T06:14:34.350 controller-0 kernel: info bonding: bond0: link

status definitely up for interface eth0, 0 Mbps full duplex.

This message is from the bonding driver not from ixgbe.

In your patch you are adding a check for unknown link to ixgbe_watchdog_link_is_up() if that condition was true then you should also see "unknown link" being reported by ixgbe.

Thanks,
Emil

author
#26 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 47 lines · Emil S
Quoted 37 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization

of

Quoted 9 lines
Quoted 9 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

Quoted 6 lines
Quoted 1 line

What exactly do you mean by "time span between link_up and link_speed"?

In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Was this when reporting the link state from check_link() (reading the LINKS register) or reporting the adapter->link_speed?

Quoted 1 line
Quoted 1 line

Where is it you think the de-synchronization occurs?

Quoted 8 lines
Quoted 6 lines

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag is set, we continue to check
link_speed. If not, this function is executed whether this link_speed is unknown or not.

I can already see this in your patch. I was asking about the reason why your change is needed.

an extreme example, let us assume this scenario:

An ixgbe NIC directly connects to another NIC (let us call it NIC-a). And auto-negotiate is off while no static speed is set in the 2 NICs. These 2 NICs acts as 2 independent interfaces. As such, at this time, there is no speed in the both 2 NICs. That is, link_speed is unknown.

When the user run "ifconfig or ethtool", NIC-a will show "Link detected:
yes" while ixgbe NIC will show "Link detected: no" if the flag IFF_SLAVE is not set.

NIC-a stands for most NIC, such as e1000, e1000e and so on.

Best Regards!
Zhu Yanjun

Quoted 52 lines
Quoted 38 lines
Quoted 16 lines
Quoted 16 lines

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 9 ++++++++-

1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index ace21b9..1bb6056 100644
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	 * time. To X540 NIC, there is a time span between link_up and
 	 * link_speed. As such, only continue if link_up and link_speed are
 	 * ready to X540 NIC.
Quoted 8 lines

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)

Quoted 7 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6 -0
+	 * The time span between link_up and link_speed is very important
+	 * when the X540 NIC acts as a slave in some virtual NICs, such as
+	 * a bonding driver in 802.3ad mode. When X540 NIC acts as an
+	 * independent interface, it is not necessary to synchronize link_up
+	 * and link_speed.
+	 * In the end, not continue if (X540 NIC && SLAVE && link_speed
 UNKNOWN)

This is a patch on top of your previous patch which I don't think was

applied,

Quoted 9 lines

so this is not going to apply cleanly.

Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
 	 */
-	if (hw->mac.type == ixgbe_mac_X540)
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (netdev->flags & IFF_SLAVE))
 		if (link_speed == IXGBE_LINK_SPEED_UNKNOWN)
			return;

If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then

I would

Quoted 3 lines

assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"

Quoted 6 lines
Quoted 1 line

by the interface you use in the bond?

Sure. There is a dmesg log from the customer.
"
...

2015-10-05T06:14:34.350 controller-0 kernel: info bonding: bond0: link

status definitely up for interface eth0, 0 Mbps full duplex.

This message is from the bonding driver not from ixgbe.

In your patch you are adding a check for unknown link to ixgbe_watchdog_link_is_up() if that condition was true then you should also see "unknown link" being reported by ixgbe.

Thanks,
Emil

#27 lore raw

RE: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 71 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 8 lines · Emil S
Quoted 8 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 5 lines
Quoted 5 lines

of link_up and speed

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-

bounces at lists.osuosl.org]

Quoted 12 lines
Quoted 12 lines

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization

Quoted 8 lines
Quoted 8 lines

of

Quoted 7 lines
Quoted 7 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent

interface,

Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines

it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and link_speed"?

In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Was this when reporting the link state from check_link() (reading the

LINKS

Quoted 9 lines

register) or reporting the adapter->link_speed?

Quoted 7 lines
Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag is set, we continue to check
link_speed. If not, this function is executed whether this link_speed is unknown or not.

Quoted 4 lines
Quoted 2 lines

I can already see this in your patch. I was asking about the reason why your change is needed.

an extreme example, let us assume this scenario:

Is this the scenario you are trying to fix?

Quoted 2 lines

An ixgbe NIC directly connects to another NIC (let us call it NIC-a). And auto-negotiate is off while no static speed is set in the 2 NICs.

The ixgbe driver does not support disabling auto-negotiation directly. The only time this is true is when the advertised speed is restricted, so the above scenario is not possible (you either have autoneg or advertised speed set) with the current driver.

Is this example in theory or do you have your interface configured this way somehow?

Thanks,
Emil

author
#28 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 71 lines · Emil S
Quoted 71 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 8 lines · Emil S
Quoted 8 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 5 lines
Quoted 5 lines

of link_up and speed

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-

bounces at lists.osuosl.org]

Quoted 12 lines
Quoted 12 lines

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization

Quoted 8 lines
Quoted 8 lines

of

Quoted 7 lines
Quoted 7 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent

interface,

Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines

it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and link_speed"?

In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Was this when reporting the link state from check_link() (reading the

LINKS

Quoted 9 lines

register) or reporting the adapter->link_speed?

Quoted 7 lines
Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag is set, we continue to check
link_speed. If not, this function is executed whether this link_speed is unknown or not.

Quoted 4 lines
Quoted 3 lines
Quoted 2 lines

I can already see this in your patch. I was asking about the reason why your change is needed.

an extreme example, let us assume this scenario:

Is this the scenario you are trying to fix?

Sure. If IFF_SLAVE is checked, this scenario will not happen.

Zhu Yanjun

Quoted 15 lines
Quoted 3 lines

An ixgbe NIC directly connects to another NIC (let us call it NIC-a). And auto-negotiate is off while no static speed is set in the 2 NICs.

The ixgbe driver does not support disabling auto-negotiation directly. The only time this is true is when the advertised speed is restricted, so the above scenario is not possible (you either have autoneg or advertised speed set) with the current driver.

Is this example in theory or do you have your interface configured this way somehow?

Thanks,
Emil

#29 lore raw

RE: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 91 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, January 05, 2016 9:42 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 8 lines · Emil S
Quoted 8 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 8 lines
Quoted 8 lines

of link_up and speed

Quoted 5 lines · Emil S
Quoted 5 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W;

Ronciak,

Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines

John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 3 lines
Quoted 3 lines

of link_up and speed

On 12/30/2015 12:18 AM, Tantilov, Emil S wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-

bounces at lists.osuosl.org]

Quoted 5 lines
Quoted 5 lines

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John;

Williams,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

of

Quoted 4 lines
Quoted 4 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent

interface,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and

link_speed"?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

In the previous mail, I show you some ethtool logs. In these logs,

there

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

Was this when reporting the link state from check_link() (reading the

LINKS

Quoted 6 lines

register) or reporting the adapter->link_speed?

Quoted 4 lines
Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag

is

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

set, we continue to check
link_speed. If not, this function is executed whether this link_speed

Quoted 7 lines

is

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

unknown or not.

I can already see this in your patch. I was asking about the reason why your change is needed.

an extreme example, let us assume this scenario:

Is this the scenario you are trying to fix?

Sure. If IFF_SLAVE is checked, this scenario will not happen.

I already explained why this is not a valid scenario, but if you were able to set it up somehow I'd like to know how you did it

If we are to enter ixgbe_watchdog_link_is_up() with unknown link this would be an issue regardless of whether the interface is a part of a bond or not, but you haven't provided any proof that this is the case. Do you have a dmesg log that shows ixgbe reporting unknown speed?

Was your patch tested by the customer that reported this issue?

Thanks,
Emil

author
#30 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 92 lines · Emil S
Quoted 92 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, January 05, 2016 9:42 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 8 lines · Emil S
Quoted 8 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 8 lines
Quoted 8 lines

of link_up and speed

Quoted 5 lines · Emil S
Quoted 5 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W;

Ronciak,

Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines

John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 3 lines
Quoted 3 lines

of link_up and speed

On 12/30/2015 12:18 AM, Tantilov, Emil S wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-

bounces at lists.osuosl.org]

Quoted 5 lines
Quoted 5 lines

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John;

Williams,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

of

Quoted 4 lines
Quoted 4 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent

interface,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and

link_speed"?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

In the previous mail, I show you some ethtool logs. In these logs,

there

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

Was this when reporting the link state from check_link() (reading the

LINKS

Quoted 6 lines

register) or reporting the adapter->link_speed?

Quoted 4 lines
Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag

is

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

set, we continue to check
link_speed. If not, this function is executed whether this link_speed

is

Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

unknown or not.

I can already see this in your patch. I was asking about the reason why your change is needed.

an extreme example, let us assume this scenario:

Is this the scenario you are trying to fix?

Sure. If IFF_SLAVE is checked, this scenario will not happen.

I already explained why this is not a valid scenario, but if you were able to set it up somehow I'd like to know how you did it

If it is not a valid scenario, maybe there is something wrong with NIC driver/hardware.
We should pay attention to it.

Zhu Yanjun

Quoted 11 lines

If we are to enter ixgbe_watchdog_link_is_up() with unknown link this would be an issue regardless of whether the interface is a part of a bond or not, but you haven't provided any proof that this is the case. Do you have a dmesg log that shows ixgbe reporting unknown speed?

Was your patch tested by the customer that reported this issue?

Thanks,
Emil

author
#31 lore raw

Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

In reply to Tantilov, Emil S
Quoted 92 lines · Emil S
Quoted 92 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, January 05, 2016 9:42 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed

Quoted 8 lines · Emil S
Quoted 8 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 8 lines
Quoted 8 lines

of link_up and speed

Quoted 5 lines · Emil S
Quoted 5 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W;

Ronciak,

Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines

John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 3 lines
Quoted 3 lines

of link_up and speed

On 12/30/2015 12:18 AM, Tantilov, Emil S wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-

bounces at lists.osuosl.org]

Quoted 5 lines
Quoted 5 lines

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John;

Williams,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 2 lines
Quoted 2 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict

synchronization

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

of

Quoted 4 lines
Quoted 4 lines

link_up and speed

From: Zhu Yanjun <[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent

interface,

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.

What exactly do you mean by "time span between link_up and

link_speed"?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

In the previous mail, I show you some ethtool logs. In these logs,

there

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

is some
time with NIC up while speed is unknown. I think this "some time" is time span between
link_up and link_speed. Please see the previous mail for details.

Quoted 8 lines
Quoted 8 lines
Quoted 1 line

Was this when reporting the link state from check_link() (reading the

LINKS

Quoted 6 lines

register) or reporting the adapter->link_speed?

Quoted 4 lines
Quoted 1 line

Where is it you think the de-synchronization occurs?

When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag

is

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

set, we continue to check
link_speed. If not, this function is executed whether this link_speed

is

Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

unknown or not.

I can already see this in your patch. I was asking about the reason why your change is needed.

an extreme example, let us assume this scenario:

Is this the scenario you are trying to fix?

Sure. If IFF_SLAVE is checked, this scenario will not happen.

I already explained why this is not a valid scenario, but if you were able to set it up somehow I'd like to know how you did it

Sorry. Why can we not disable auto-negotiate on ixgbe NIC ? To be honest, I am interested in it.
This is related with hardware or driver?

Thanks a lot.
Zhu Yanjun

Quoted 12 lines

If we are to enter ixgbe_watchdog_link_is_up() with unknown link this would be an issue regardless of whether the interface is a part of a bond or not, but you haven't provided any proof that this is the case. Do you have a dmesg log that shows ixgbe reporting unknown speed?

Was your patch tested by the customer that reported this issue?

Thanks,
Emil

#32 lore raw
In reply to [email protected]

Hello.

Quoted 34 lines · On 12/23/2015 9:46 AM, zyjzyj2000 at gmail.com wrote:

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
  drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |   16 +++++++++++++++-
  1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
  	       (flow_rx ? "RX" :
  	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) · line 6496

Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +10 -0
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+

{} not needed here. And you do the same thing regardless of whether your
check succeeds or not, this doesn't make sense.

[...]

MBR, Sergei

#33 lore raw
In reply to [email protected]
Quoted 62 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++-

1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
	/* enable transmits */
Quoted 2 lines

--
1.7.9.5

NAK

I have already submitted a patch that will address the issue with bonding reporting unknown speed (in /proc/bonding/bondX) after the link is established due to link flaps:
http://patchwork.ozlabs.org/patch/552485/

The bonding driver gets the speed from ethtool and this is where the reporting needs to be fixed. The issue is that the bonding driver polls for netif_carrier_ok() at a certain rate and as such will not be able to detect rapid link changes.

If there is a case where link_speed is unknown when entering this function it's probably better to just bail rather than have this hack around the netif_carrier_on() especially after the driver already reported link status change. Rapid link changes can occur between link partners and not just for X540.

Thanks,
Emil

#34 lore raw
Quoted 67 lines · Emil S
Quoted 62 lines

-----Original Message-----
From: Intel-wired-lan [mailto:[email protected]] On Behalf Of [email protected]
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; [email protected]; [email protected]; e1000- [email protected]
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++-

1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
	/* enable transmits */
Quoted 2 lines

--
1.7.9.5

NAK

Quoted 7 lines

I have already submitted a patch that will address the issue with bonding reporting unknown speed (in /proc/bonding/bondX) after the link is established due to link flaps:
http://patchwork.ozlabs.org/patch/552485/

The bonding driver gets the speed from ethtool and this is where the reporting needs to be fixed. The issue is that the bonding driver polls for netif_carrier_ok() at a certain rate and as such will not be able to detect rapid link changes.

Kernel has netdev notifiers, these should really always be used rather than polling. The polling stuff was added back in ancient kernel days when netdev notifiers didn't work because most drivers were broken.

author
#35 lore raw
In reply to Tantilov, Emil S
Quoted 66 lines · Emil S
Quoted 62 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++-

1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
	/* enable transmits */
Quoted 2 lines

--
1.7.9.5

NAK

Quoted 7 lines

I have already submitted a patch that will address the issue with bonding reporting unknown speed (in /proc/bonding/bondX) after the link is established due to link flaps:
http://patchwork.ozlabs.org/patch/552485/

The bonding driver gets the speed from ethtool and this is where the reporting needs to be fixed. The issue is that the bonding driver polls for netif_carrier_ok() at a certain rate and as such will not be able to detect rapid link changes.

Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" while
the speed is unknown at the same time. We can run "ethtool ethx" to confirm it.

Any way, thanks a lot.

Zhu Yanjun

Quoted 9 lines

If there is a case where link_speed is unknown when entering this function it's probably better to just bail rather than have this hack around the netif_carrier_on() especially after the driver already reported link status change. Rapid link changes can occur between link partners and not just for X540.

Thanks,
Emil

#36 lore raw
Quoted 89 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 6:28 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 52 lines
Quoted 52 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++-

1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
Quoted 6 lines
Quoted 3 lines
	/* enable transmits */

--
1.7.9.5

NAK

I have already submitted a patch that will address the issue with bonding

reporting

Quoted 1 line

unknown speed (in /proc/bonding/bondX) after the link is established due

to link flaps:

Quoted 3 lines

http://patchwork.ozlabs.org/patch/552485/

The bonding driver gets the speed from ethtool and this is where the

Quoted 8 lines

reporting needs

Quoted 1 line

to be fixed. The issue is that the bonding driver polls for

netif_carrier_ok() at a

Quoted 1 line

certain rate and as such will not be able to detect rapid link changes.

Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" while

The "link up" event is a result of an LSC interrupt, the speed is determined as result of that interrupt by checking the LINKS register. If the LINKS register reports link as unknown then that is the actual state of the PHY - meaning the device is re-negotiating the speed for some reason.

Quoted 2 lines

the speed is unknown at the same time. We can run "ethtool ethx" to confirm it.

Prior to my patch the ethtool call will read the LINKS register which can show speed as unknown due to a link flap (for example). You are seeing the momentary state of the device.

If you are still seeing the bond reporting "unknown" speed after the patch I pointed out please file a bug either through e1000.sf.net or via Intel support and provide detailed information about the bonding setup, the type of the link partner (switch model etc) and full dmesg from the failed scenario along with the output from /proc/bonding/bond0

Thanks,
Emil

author
#37 lore raw
Quoted 91 lines · Emil S
Quoted 91 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 6:28 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 52 lines
Quoted 52 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++-

1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link on"
+	 * and getting the speed and duplex. To a bonding driver in 802.3ad
+	 * mode, this time span will make it not work well if the time span
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs, this
+	 * problem does not exist. As such, it is better for X540 to report
+	 * "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
Quoted 6 lines
Quoted 3 lines
	/* enable transmits */

--
1.7.9.5

NAK

I have already submitted a patch that will address the issue with bonding

reporting

Quoted 1 line

unknown speed (in /proc/bonding/bondX) after the link is established due

to link flaps:

Quoted 3 lines

http://patchwork.ozlabs.org/patch/552485/

The bonding driver gets the speed from ethtool and this is where the

reporting needs

Quoted 1 line

to be fixed. The issue is that the bonding driver polls for

Quoted 8 lines
Quoted 6 lines

netif_carrier_ok() at a

Quoted 1 line

certain rate and as such will not be able to detect rapid link changes.

Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" while

The "link up" event is a result of an LSC interrupt, the speed is determined as result of that interrupt by checking the LINKS register.

Hi,

Sorry. I do not agree with you. Please see the followings for details.

/**
* ixgbe_watchdog_update_link - update the link status
* @adapter: pointer to the device adapter structure
* @link_speed: pointer to a u32 to store the link_speed **/
static void ixgbe_watchdog_update_link(struct ixgbe_adapter *adapter)

From this function, link_up and link_speed is from watchdo poll.

Thanks for your reply.

Zhu Yanjun

Quoted 18 lines

If the LINKS register reports link as unknown then that is the actual state of the PHY - meaning the device is re-negotiating the speed for some reason.

Quoted 2 lines

the speed is unknown at the same time. We can run "ethtool ethx" to confirm it.

Prior to my patch the ethtool call will read the LINKS register which can show speed as unknown due to a link flap (for example). You are seeing the momentary state of the device.

If you are still seeing the bond reporting "unknown" speed after the patch I pointed out please file a bug either through e1000.sf.net or via Intel support and provide detailed information about the bonding setup, the type of the link partner (switch model etc) and full dmesg from the failed scenario along with the output from /proc/bonding/bond0

Thanks,
Emil

#38 lore raw
Quoted 133 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 10:24 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

Quoted 8 lines · Emil S
Quoted 8 lines

-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 6:28 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 7 lines
Quoted 7 lines

Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

Quoted 2 lines · Emil S
Quoted 2 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-

bounces at lists.osuosl.org]

Quoted 44 lines
Quoted 44 lines

On

Quoted 4 lines
Quoted 4 lines

Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,

Mitch

Quoted 3 lines
Quoted 3 lines

A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);

Bourg,

Quoted 34 lines
Quoted 34 lines

Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplex

From: Zhu Yanjun <[email protected]>

In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.

To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.

---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++-

1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0 -0
index aed8d02..cb9d310 100644
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct
 ixgbe_adapter *adapter)
 	       (flow_rx ? "RX" :
 	       (flow_tx ? "TX" : "None"))));
 
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)

Quoted 3 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2 -1
-	netif_carrier_on(netdev);
+	/*
+	 * In X540 NIC, there is a time span between reporting "link

on"

Quoted 1 line
Quoted 1 line
Quoted 1 line

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6485

Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	 * and getting the speed and duplex. To a bonding driver in

802.3ad

Quoted 1 line
Quoted 1 line
Quoted 1 line

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6486

Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	 * mode, this time span will make it not work well if the time

span

Quoted 1 line
Quoted 1 line
Quoted 1 line

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6487

Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	 * is big enough. To 82599_SFP NIC and other kinds of NICs,

this

Quoted 1 line
Quoted 1 line
Quoted 1 line

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6488

Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	 * problem does not exist. As such, it is better for X540 to

report

Quoted 1 line
Quoted 1 line
Quoted 1 line

drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6489

Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1 -0
+	 * "link on" when the link speed is not

IXGBE_LINK_SPEED_UNKNOWN.

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8 -0
+	 */
+	if ((hw->mac.type == ixgbe_mac_X540) &&
+	    (link_speed != IXGBE_LINK_SPEED_UNKNOWN)) {
+		netif_carrier_on(netdev);
+	} else {
+		netif_carrier_on(netdev);
+	}
+
 	ixgbe_check_vf_rate_limit(adapter);
 
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
	/* enable transmits */

--
1.7.9.5

NAK

I have already submitted a patch that will address the issue with

bonding

Quoted 2 lines
Quoted 2 lines

reporting

Quoted 1 line

unknown speed (in /proc/bonding/bondX) after the link is established

due

Quoted 6 lines
Quoted 6 lines

to link flaps:

Quoted 3 lines

http://patchwork.ozlabs.org/patch/552485/

The bonding driver gets the speed from ethtool and this is where the

reporting needs

Quoted 1 line

to be fixed. The issue is that the bonding driver polls for

Quoted 8 lines
Quoted 6 lines

netif_carrier_ok() at a

Quoted 1 line

certain rate and as such will not be able to detect rapid link changes.

Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" while

The "link up" event is a result of an LSC interrupt, the speed is determined as result of that interrupt by checking the LINKS register.

Hi,

Sorry. I do not agree with you. Please see the followings for details.

/**
* ixgbe_watchdog_update_link - update the link status
* @adapter: pointer to the device adapter structure
* @link_speed: pointer to a u32 to store the link_speed **/
static void ixgbe_watchdog_update_link(struct ixgbe_adapter *adapter)

Quoted 3 lines

From this function, link_up and link_speed is from watchdo poll.

Thanks for your reply.

ixgbe_watchdog_update_link() only runs when the IXGBE_FLAG_NEED_LINK_UPDATE is set which can happen in 2 situations - LSC is received or the interface is brought up. At that point the driver will call hw->mac.ops.check_link() which sets adapter->link_up based on the LINKS register and if we have link_up then we update the link status:

6669 if (adapter->link_up)
6670 ixgbe_watchdog_link_is_up(adapter);

The thing is if the LINKS.LINK_UP bit was set then at that point we also have a speed which is contrary to what you are suggesting. If you want to debug your issue you have to monitor the LINKS register.

That is why I asked for the dmesg output - if your claim is correct then the "Link Up" event in dmesg will be reporting "unknown speed" and if not then we are not dealing with delayed speed, but a simple link flap.

Thanks,
Emil