ixgbe: force to synchronize reporting "link on" and getting speed and duplex
Subsystems (3)
PATCH 1/1 ixgbe: force to synchronize reporting "link on" and getting speed and duplex
From: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -1
@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ?"RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);/* enable transmits */
Signature · 1 line
1.7.9.5
Quoted 54 lines · On Wed, 2015-12-23 at 14:46 +0800, zyjzyj2000 at gmail.com wrote:
From: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of
the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
?drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |???16
+++++++++++++++-
?1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)? ???????(flow_rx ? "RX" :
? ???????(flow_tx ? "TX" : "None"))));
?drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1-netif_carrier_on (netdev); +/* +?* In X540 NIC, there is a time span between reporting "link on"
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+ ?* and getting the speed and duplex. To a bonding driver in802.3ad
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+ ?* mode, this time span will make it not work well if thetime span
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+ ?* is big enough. To 82599_SFP NIC and other kinds of NICs,this
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+ ?* problem does not exist. As such, it is better for X540 toreport
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+ ?* "link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8
-0+?*/ + if ((hw->mac.type == ixgbe_mac_X540) && + ????(link_speed != IXGBE_LINK_SPEED_UNKNOWN)) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +
Quoted 3 lines
?
ixgbe_check_vf_rate_limit(adapter);
?
? /* enable transmits */
This patch only adds a needless test before
calling?netif_carrier_on(netdev), since the call happens no matter the
branch you take, it appears your running into a timing issue. ?So
adding a wait() before calling?netif_carrier_on(netdev) will accomplish
the same result and you do not have to add a useless test.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151223/f955f020/attachment.asc>
PATCH v2 1/1 ixgbe: force to synchronize reporting "link on" and
Hi, Jeff
Thanks for your reply.
Changes:
Since there is a time span between link_up and link_speed to X540 NIC, it is not appropriate to continue in the function ixgbe_watchdog_link_is_up if only link_up is ready.
Best Regards!
Zhu Yanjun
PATCH 1/1 ixgbe: force to synchronize reporting "link on" and getting speed
From: yzhu1 <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -0
@@ -6426,6 +6426,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)if (netif_carrier_ok (netdev))return ; +/* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +* To other NICs, the link_up and link_speed are gotten at the same +* time. To X540 NIC, there is a time span between link_up and +* link_speed. As such, only continue if link_up and link_speed are +* ready to X540 NIC. +*/ +if (hw->mac.type == ixgbe_mac_X540) +if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN ) +return ; + adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP ;switch (hw->mac.type) {
Signature · 1 line
1.7.9.5
Re: [V2 PATCH 1/1] ixgbe: force to synchronize reporting "link on" and
In reply to [email protected]Quoted 7 lines · On 12/24/2015 11:12 AM, zyjzyj2000 at gmail.com wrote:
Hi, Jeff
Thanks for your reply.
Changes:
Since there is a time span betweenlink_upandlink_speedto X540 NIC, it is not appropriate to continue in the functionixgbe_watchdog_link_is_upif onlylink_upis ready.
Quoted 7 lines
Best Regards!
Zhu Yanjun
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi, Jeff
The following logs are from the customer. From these logs, there is a time span between link_up and link_speed.
This time span sometimes will make bonding in 802.3ad mode not work well.
What I have done in the patch is to prevent this time span in the ixgbe driver.
Best Regards!
Zhu Yanjun
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: no
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151224/ca9d7c56/attachment.html>
RE: [Intel-wired-lan] [V2 PATCH 1/1] ixgbe: force to synchronize reporting "link on" and
Quoted 178 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zhuyj
Sent: Wednesday, December 23, 2015 9:11 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [V2 PATCH 1/1] ixgbe: force to synchronize reporting "link on" andOn 12/24/2015 11:12 AM, zyjzyj2000 at gmail.com <mailto:[email protected]> wrote:
Hi, Jeff
Thanks for your reply.
Changes:
Since there is a time span betweenlink_upandlink_speedto X540
NIC, it is not appropriate to continue in the functionixgbe_watchdog_link_is_upif onlylink_upis ready.Best Regards!
Zhu Yanjun
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo at vger.kernel.org
<mailto:[email protected]>
More majordomo info at http://vger.kernel.org/majordomo-info.htmlHi, Jeff
The following logs are from the customer. From these logs, there is a time span between
link_upandlink_speed.
This time span sometimes will make bonding in 802.3ad mode not work well.What I have done in the patch is to prevent this time span in the ixgbe driver.
Best Regards!
Zhu YanjunSettings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/FullAdvertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level:0x00000007(7)
drv probe link
Link detected: no
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/FullAdvertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level:0x00000007(7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/FullAdvertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level:0x00000007(7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/FullAdvertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level:0x00000007(7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/FullAdvertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: Unknown!
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level:0x00000007(7)
drv probe link
Link detected: yes
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full1000baseT/Full 10000baseT/Full Supported pause frame use: No Supports auto-negotiation: Yes Advertised link modes: 100baseT/Full 1000baseT/Full 10000baseT/Full
Quoted 14 lines
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Current message level:0x00000007(7)
drv probe link
Link detected: yes
Can you provide the related dmesg output? What did the driver report in the "Link Up" messages?
Thanks,
Emil
PATCH v3 1/1 ixgbe: force to synchronize link_up and speed as a slave
In reply to [email protected]Hi, all
During the discussion with Emil, I think the time span between link_up and link_speed is not important when the X540 NIC acts as an independent interface. As such, a new patch is made to restrict synchronization between link_up and link_speed.
Any reply is appreciated.
Zhu Yanjun
PATCH 1/2 ixgbe: force to synchronize reporting "link on" and getting speed
From: yzhu1 <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -0
@@ -6426,6 +6426,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)if (netif_carrier_ok (netdev))return ; +/* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +* To other NICs, the link_up and link_speed are gotten at the same +* time. To X540 NIC, there is a time span between link_up and +* link_speed. As such, only continue if link_up and link_speed are +* ready to X540 NIC. +*/ +if (hw->mac.type == ixgbe_mac_X540) +if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN ) +return ; + adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP ;switch (hw->mac.type) {
Signature · 1 line
1.7.9.5
PATCH 2/2 ixgbe: restrict synchronization of link_up and speed
In reply to [email protected]From: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8 -1
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)* time. To X540 NIC, there is a time span between link_up and * link_speed. As such, only continue if link_up and link_speed are * ready to X540 NIC. +* The time span between link_up and link_speed is very important +* when the X540 NIC acts as a slave in some virtual NICs, such as +* a bonding driver in 802.3ad mode. When X540 NIC acts as an +* independent interface, it is not necessary to synchronize link_up +* and link_speed. +* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) */ -if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return ;
Signature · 1 line
1.7.9.5
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 13 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speed
Quoted 7 lines
From: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronizelink_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.
What exactly do you mean by "time span between link_up and link_speed"? Where is it you think the de-synchronization occurs?
Quoted 12 lines
Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 9 ++++++++-1 file changed, 8 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6446
Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6
-0* time. To X540 NIC, there is a time span between link_up and * link_speed. As such, only continue if link_up and link_speed are * ready to X540 NIC. +* The time span between link_up and link_speed is very important +* when the X540 NIC acts as a slave in some virtual NICs, such as +* a bonding driver in 802.3ad mode. When X540 NIC acts as an +* independent interface, it is not necessary to synchronize link_up +* and link_speed. +* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN)
This is a patch on top of your previous patch which I don't think was applied, so this is not going to apply cleanly.
Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1*/ -if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return;
If you were to enter ixgbe_watchdog_link_is_up() with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"
by the interface you use in the bond?
Thanks,
Emil
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 11 lines · Emil S
Quoted 6 lines
*/
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1-if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed == IXGBE_LINK_SPEED_UNKNOWN) return;If you were to enter
ixgbe_watchdog_link_is_up()with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"
Quoted 1 line
by the interface you use in the bond?
It seems odd to be checking the MAC type for this. Is this behavior perhaps more related to the copper phy? If so, then the check should be changed. Or would the check for unknown link speed be sufficient? It seems like an interface as a slave would not work with an unknown link speed, so it may as well wait in all cases, not just for X540.
--
Mark Rustad, Networking Division, Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151229/775e8321/attachment-0001.asc>
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 12 lines · Mark D
Quoted 10 lines · Emil S
Quoted 6 lines
*/
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1-if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed == IXGBE_LINK_SPEED_UNKNOWN) return;If you were to enter
ixgbe_watchdog_link_is_up()with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"
Quoted 5 lines
Quoted 1 line
by the interface you use in the bond?
It seems odd to be checking the MAC type for this. Is this behavior perhaps more related to the copper phy? If so, then the check should be changed. Or would the check for unknown link speed be sufficient? It seems like an interface as a slave would not work with an unknown link speed, so it may as well wait in all cases, not just for X540.
--
Mark Rustad, Networking Division, Intel Corporation
Hi, Mark
Thanks for your suggestions.
The following is the feedback from the customer.
"
...
We observing this issue on x540T interfaces (8086:1528), but not on 82599_SFP (8086:10FB).
...
"
To narrow this problem, I restrict mac.type to ixgbe_mac_X540. I agree with you. Maybe this problem is
related with the copper phy. But I only have X540 NIC to test. So it is difficult for me to confirm whether this
problem occurs on other mac type or not, such as X550.
I will consider your suggestions in the latest patch.
Thanks again.
Zhu Yanjun
PATCH v4 1/1 ixgbe: synchronize the link_speed and link_up of a slave interface
In reply to Rustad, Mark DHi, all
According to Rustad, Mark D, maybe it is related with copper phy. To make fiber phy more robust, synchronize both the link_up and link_speed of a slave interface in ixgbe driver.
Best Regards!
Zhu Yanjun
PATCH 1/3 ixgbe: force to synchronize reporting "link on" and getting speed
From: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15 -0
@@ -6426,6 +6426,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)if (netif_carrier_ok (netdev))return ; +/* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +* To other NICs, the link_up and link_speed are gotten at the same +* time. To X540 NIC, there is a time span between link_up and +* link_speed. As such, only continue if link_up and link_speed are +* ready to X540 NIC. +*/ +if (hw->mac.type == ixgbe_mac_X540) +if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN ) +return ; + adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP ;switch (hw->mac.type) {
Signature · 1 line
1.7.9.5
PATCH 2/3 ixgbe: restrict synchronization of link_up and speed
In reply to [email protected]From: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize link_up and link_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronize link_up and link_speed. That is, the time span between link_up and link_speed is acceptable.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8 -1
@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)* time. To X540 NIC, there is a time span between link_up and * link_speed. As such, only continue if link_up and link_speed are * ready to X540 NIC. +* The time span between link_up and link_speed is very important +* when the X540 NIC acts as a slave in some virtual NICs, such as +* a bonding driver in 802.3ad mode. When X540 NIC acts as an +* independent interface, it is not necessary to synchronize link_up +* and link_speed. +* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) */ -if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return ;
Signature · 1 line
1.7.9.5
PATCH 3/3 ixgbe: synchronize the link_speed and link_up of a slave interface
In reply to [email protected]From: Zhu Yanjun <[email protected]>
According to the suggestion from Rustad, Mark D, this behavior perhaps is more related to the copper phy. But to make fiber phy more robust, to all the interfaces as a slave interface, the link_speed and link_up is synchronized.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -3
@@ -6441,10 +6441,12 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)* a bonding driver in 802.3ad mode. When X540 NIC acts as an * independent interface, it is not necessary to synchronize link_up * and link_speed. -* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) +* According to the suggestion from Rustad, Mark D, this behavior +* perhaps is related to the copper phy. To make fiber phy more robust, +* To all the interfaces as a slave, the link_speed is checked. +* In the end, not continue if (SLAVE && link_speed UNKNOWN) */ -if ((hw->mac.type == ixgbe_mac_X540) && - (netdev->flags &IFF_SLAVE )) +if (netdev->flags &IFF_SLAVE )if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return ;
Signature · 1 line
1.7.9.5
Re: [PATCH 3/3] ixgbe: synchronize the link_speed and link_up of a slave interface
Quoted 17 lines · zyjzyj2000 at gmail.com wrote:
From: Zhu Yanjun <[email protected]>
According to the suggestion from Rustad, Mark D, this behavior perhaps is more related to the copper phy. But to make fiber phy more robust, to all the interfaces as a slave interface, the
link_speedandlink_upis synchronized.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 8 +++++---1 file changed, 5 insertions(+), 3 deletions(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) · line 6447
Quoted 8 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +4
-1@@ -6441,10 +6441,12 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)* a bonding driver in 802.3ad mode. When X540 NIC acts as an * independent interface, it is not necessary to synchronize link_up * and link_speed. -* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) +* According to the suggestion from Rustad, Mark D, this behavior +* perhaps is related to the copper phy. To make fiber phy more robust, +* To all the interfaces as a slave, the link_speed is checked. +* In the end, not continue if (SLAVE && link_speed UNKNOWN)
There is no need to make reference to my suggestion in the comment, especially since that is in the commit message. Please simplify your comment above to be something like:
* For all slave interfaces, wait for the link_speed to be known.
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) · line 6451
Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-2*/ -if ((hw->mac.type == ixgbe_mac_X540) && - (netdev->flags &IFF_SLAVE )) +if (netdev->flags &IFF_SLAVE )if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return ;
The above would be better as:
if ((netdev->flags & IFF_SLAVE) && link_speed == IXGBE_LINK_SPEED_UNKNOWN) return;
Please do not send a series of patches - it just adds needless confusion and is a bisect hazard. Just send a single patch with the desired change as a V5.
--
Mark Rustad, Networking Division, Intel Corporation
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151230/5415a7a9/attachment.asc>
PATCH v5 1/1 ixgbe: synchronize link_up and link_speed of a slave
Hi, all
Thanks for the suggestions from Rustad, Mark D.
According to his suggestions, the logs and source code are simplified.
Zhu Yanjun
PATCH 1/1 ixgbe: synchronize link_up and link_speed of a slave interface
From: Zhu Yanjun <[email protected]>
According to the suggestions from Rustad, Mark D, to all the slave interfaces, the link_speed and link_up should be synchronized since the time span between link_up and link_speed will make some virtual NICs not work well, such as a bonding driver in 802.3ad mode.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -0
@@ -6426,6 +6426,11 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)if (netif_carrier_ok (netdev))return ; +/* For all slave interfaces, wait for the link_speed to be known. */ +if ((netdev->flags &IFF_SLAVE ) && + (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )) +return ; + adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP ;switch (hw->mac.type) {
Signature · 1 line
1.9.1
Re: [PATCH 1/1] ixgbe: synchronize link_up and link_speed of a slave interface
Quoted 9 lines · On Thu, 2015-12-31 at 13:04 +0800, zyjzyj2000 at gmail.com wrote:
From: Zhu Yanjun <[email protected]>
According to the suggestions from Rustad, Mark D, to all the slave? interfaces, the
link_speedandlink_upshould be synchronized since the time span betweenlink_upandlink_speedwill make some virtual NICs not work well, such as a bonding driver in 802.3ad mode.Signed-off-by Zhu Yanjun <[email protected]>---
Since this is version 5 of your original patch, you should be putting the change log of this patch here, so that we can follow how you got to this point. ?This helpful for the reviewers (and me, the maintainer). ?For example:
v2: dropped the "else" case of "if" statement, based on feedback from ? ? Jeff Kirsher
v3: changed the if statement to simply return when the link speed is ? ? unknown on X540 parts based on feedback from Emil Tantilov
v4: updated code comment and if statement to test for IFF_SLAVE flag
V5: simplified code comment and if statement to only test for IFF_SLAVE ? ? flag and unknown link speed.
Of course, you can expand on what I have above, I just did a quick summary of the changes as an example of a change log.?
Quoted 2 lines
?drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++++ ?1 file changed, 5 insertions(+)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151230/dd49f916/attachment-0001.asc>
PATCH v6 1/1 ixgbe: synchronize link_up and link_speed of a slave
Hi, all
Thanks for the reply from Jeff.
V2: Based on feedback from Jeff Kirsher, it is not appropriate to continue
in the function ixgbe_watchdog_link_is_up without link_speed since this will make some virtual NICs not work well.
V3: Based on feedback from Emil Tantilov, the time span between link_up
and link_speed is not important when the X540 NIC acts as an independent interface.
V4: According to Rustad, Mark D, maybe it is related with copper phy. To make
fiber phy more robust, synchronize both the link_up and link_speed of a slave interface in ixgbe driver.
V5: Based on feedback from Rustad, Mark D, simplify code comment and if statement
to only test for IFF_SLAVE flag and unknown link speed.
V6: Based on feedback from Jeff Kirsher, the patch format is adjusted
and change log of this patch are added.
Best Regards!
Zhu Yanjun
PATCH v6 1/1 ixgbe: synchronize link_up and link_speed of a slave interface
From: Zhu Yanjun <[email protected]>
According to the suggestions from Rustad, Mark D, to all the slave interfaces, the link_speed and link_up should be synchronized since the time span between link_up and link_speed will make some virtual NICs not work well, such as a bonding driver in 802.3ad mode.
--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5 -0
@@ -6426,6 +6426,11 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)if (netif_carrier_ok (netdev))return ; +/* For all slave interfaces, wait for the link_speed to be known. */ +if ((netdev->flags &IFF_SLAVE ) && + (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )) +return ; + adapter->flags2 &= ~IXGBE_FLAG2_SEARCH_FOR_SFP ;switch (hw->mac.type) {
Signature · 1 line
1.9.1
Re: [PATCH V5] ixgbe: synchronize link_up and link_speed of a slave
In reply to [email protected]Quoted 3 lines · On Thu, 2015-12-31 at 13:04 +0800, zyjzyj2000 at gmail.com wrote:
Thanks for the suggestions from Rustad, Mark D. According to his suggestions, the logs and source code are simplified.
I find it funny that this email (no patch) is got the correct subject,
yet the updated patch you sent does not. ?The subject line for this
email should have been used for the updated patch you sent out.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.osuosl.org/pipermail/intel-wired-lan/attachments/20151230/110ef944/attachment.asc>
Re: [PATCH V5] ixgbe: synchronize link_up and link_speed of a slave
Quoted 7 lines · Jeff Kirsher
Quoted 3 lines · On Thu, 2015-12-31 at 13:04 +0800, zyjzyj2000 at gmail.com wrote:
Thanks for the suggestions from Rustad, Mark D. According to his suggestions, the logs and source code are simplified.
I find it funny that this email (no patch) is got the correct subject, yet the updated patch you sent does not. The subject line for this email should have been used for the updated patch you sent out.
Thanks for your reply.
I will resend the patch soon.
Best Regards!
Zhu Yanjun
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
In reply to Tantilov, Emil SQuoted 13 lines · Emil S
Quoted 13 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speed
Quoted 8 lines
Quoted 7 lines
From: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronizelink_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.What exactly do you mean by "time span between
link_upandlink_speed"?
In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_up and link_speed. Please see the previous mail for details.
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we check IFF_SLAVE flag. If this flag is set, we continue to checklink_speed. If not, this function is executed whether this link_speed is unknown or not.
Quoted 36 lines
Quoted 13 lines
Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 9 ++++++++-1 file changed, 8 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6446
Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6
-0* time. To X540 NIC, there is a time span between link_up and * link_speed. As such, only continue if link_up and link_speed are * ready to X540 NIC. +* The time span between link_up and link_speed is very important +* when the X540 NIC acts as a slave in some virtual NICs, such as +* a bonding driver in 802.3ad mode. When X540 NIC acts as an +* independent interface, it is not necessary to synchronize link_up +* and link_speed. +* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) This is a patch on top of your previous patch which I don't think was applied, so this is not going to apply cleanly.
Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1*/ -if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return;If you were to enter
ixgbe_watchdog_link_is_up()with unknown speed, then I would assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"
Quoted 1 line
by the interface you use in the bond?
Sure. There is a dmesg log from the customer.
"
...
2015-10-05T06:14:34.350 controller-0 kernel: info bonding: bond0: link
status definitely up for interface eth0, 0 Mbps full duplex.
...
"
Thanks a lot.
Zhu Yanjun
Quoted 4 lines
Thanks,
Emil
RE: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 37 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronizationof
Quoted 9 lines
Quoted 9 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronizelink_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.
Quoted 7 lines
Quoted 1 line
What exactly do you mean by "time span between
link_upandlink_speed"?In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.
Was this when reporting the link state from check_link() (reading the LINKS register) or reporting the adapter->link_speed?
Quoted 8 lines
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flag is set, we continue to checklink_speed. If not, this function is executed whether thislink_speedis unknown or not.
I can already see this in your patch. I was asking about the reason why your change is needed.
Quoted 38 lines
Quoted 16 lines
Quoted 16 lines
Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 9 ++++++++-1 file changed, 8 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) * time. To X540 NIC, there is a time span between link_up and * link_speed. As such, only continue if link_up and link_speed are * ready to X540 NIC.Quoted 8 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
Quoted 7 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6
-0+* The time span between link_up and link_speed is very important +* when the X540 NIC acts as a slave in some virtual NICs, such as +* a bonding driver in 802.3ad mode. When X540 NIC acts as an +* independent interface, it is not necessary to synchronize link_up +* and link_speed. +* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) This is a patch on top of your previous patch which I don't think was
applied,
Quoted 9 lines
so this is not going to apply cleanly.
Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1*/ -if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return;If you were to enter
ixgbe_watchdog_link_is_up()with unknown speed, thenI would
Quoted 3 lines
assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"
Quoted 6 lines
Quoted 1 line
by the interface you use in the bond?
Sure. There is a dmesg log from the customer.
"
...2015-10-05T06:14:34.350 controller-0 kernel: info bonding: bond0: linkstatus definitely up for interface eth0, 0 Mbps full duplex.
This message is from the bonding driver not from ixgbe.
In your patch you are adding a check for unknown link to ixgbe_watchdog_link_is_up() if that condition was true then you should also see "unknown link" being reported by ixgbe.
Thanks,
Emil
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 47 lines · Emil S
Quoted 37 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronizationof
Quoted 9 lines
Quoted 9 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independent interface, it is not necessary to synchronizelink_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.Quoted 6 lines
Quoted 1 line
What exactly do you mean by "time span between
link_upandlink_speed"?In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.Was this when reporting the link state from
check_link()(reading the LINKS register) or reporting theadapter->link_speed?Quoted 1 line
Quoted 1 line
Where is it you think the de-synchronization occurs?
Quoted 8 lines
Quoted 6 lines
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flag is set, we continue to checklink_speed. If not, this function is executed whether thislink_speedis unknown or not.I can already see this in your patch. I was asking about the reason why your change is needed.
an extreme example, let us assume this scenario:
An ixgbe NIC directly connects to another NIC (let us call it NIC-a). And auto-negotiate is off while no static speed is set in the 2 NICs. These 2 NICs acts as 2 independent interfaces. As such, at this time, there is no speed in the both 2 NICs. That is, link_speed is unknown.
When the user run "ifconfig or ethtool", NIC-a will show "Link detected:
yes" while ixgbe NIC will show "Link detected: no" if the flag IFF_SLAVE is not set.
NIC-a stands for most NIC, such as e1000, e1000e and so on.
Best Regards!
Zhu Yanjun
Quoted 52 lines
Quoted 38 lines
Quoted 16 lines
Quoted 16 lines
Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 9 ++++++++-1 file changed, 8 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -6436,8 +6436,15 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) * time. To X540 NIC, there is a time span between link_up and * link_speed. As such, only continue if link_up and link_speed are * ready to X540 NIC.Quoted 8 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
Quoted 7 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +6
-0+* The time span between link_up and link_speed is very important +* when the X540 NIC acts as a slave in some virtual NICs, such as +* a bonding driver in 802.3ad mode. When X540 NIC acts as an +* independent interface, it is not necessary to synchronize link_up +* and link_speed. +* In the end, not continue if (X540 NIC && SLAVE && link_speed UNKNOWN) This is a patch on top of your previous patch which I don't think was
applied,
Quoted 9 lines
so this is not going to apply cleanly.
Quoted 6 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1*/ -if (hw->mac.type == ixgbe_mac_X540) +if ((hw->mac.type == ixgbe_mac_X540) && + (netdev->flags &IFF_SLAVE ))if (link_speed ==IXGBE_LINK_SPEED_UNKNOWN )return;If you were to enter
ixgbe_watchdog_link_is_up()with unknown speed, thenI would
Quoted 3 lines
assume that you also have a dmesg that shows:
"NIC Link is Up unknown speed"Quoted 6 lines
Quoted 1 line
by the interface you use in the bond?
Sure. There is a dmesg log from the customer.
"
...2015-10-05T06:14:34.350 controller-0 kernel: info bonding: bond0: linkstatus definitely up for interface eth0, 0 Mbps full duplex.
This message is from the bonding driver not from ixgbe.
In your patch you are adding a check for unknown link to
ixgbe_watchdog_link_is_up()if that condition was true then you should also see "unknown link" being reported by ixgbe.Thanks,
Emil
RE: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 71 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 8 lines · Emil S
Quoted 8 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 5 lines
Quoted 5 lines
of
link_upand speedQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
Quoted 12 lines
Quoted 12 lines
On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronizationQuoted 8 lines
Quoted 8 lines
of
Quoted 7 lines
Quoted 7 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independentinterface,
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
it is not necessary to synchronize
link_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.What exactly do you mean by "time span between
link_upandlink_speed"?In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.Was this when reporting the link state from
check_link()(reading theLINKS
Quoted 9 lines
register) or reporting the
adapter->link_speed?Quoted 7 lines
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flag is set, we continue to checklink_speed. If not, this function is executed whether thislink_speedis unknown or not.
Quoted 4 lines
Quoted 2 lines
I can already see this in your patch. I was asking about the reason why your change is needed.
an extreme example, let us assume this scenario:
Is this the scenario you are trying to fix?
Quoted 2 lines
An ixgbe NIC directly connects to another NIC (let us call it NIC-a). And auto-negotiate is off while no static speed is set in the 2 NICs.
The ixgbe driver does not support disabling auto-negotiation directly. The only time this is true is when the advertised speed is restricted, so the above scenario is not possible (you either have autoneg or advertised speed set) with the current driver.
Is this example in theory or do you have your interface configured this way somehow?
Thanks,
Emil
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 71 lines · Emil S
Quoted 71 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 8 lines · Emil S
Quoted 8 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 5 lines
Quoted 5 lines
of
link_upand speedQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
Quoted 12 lines
Quoted 12 lines
On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronizationQuoted 8 lines
Quoted 8 lines
of
Quoted 7 lines
Quoted 7 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independentinterface,
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
it is not necessary to synchronize
link_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.What exactly do you mean by "time span between
link_upandlink_speed"?In the previous mail, I show you some ethtool logs. In these logs, there is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.Was this when reporting the link state from
check_link()(reading theLINKS
Quoted 9 lines
register) or reporting the
adapter->link_speed?Quoted 7 lines
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flag is set, we continue to checklink_speed. If not, this function is executed whether thislink_speedis unknown or not.
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
I can already see this in your patch. I was asking about the reason why your change is needed.
an extreme example, let us assume this scenario:
Is this the scenario you are trying to fix?
Sure. If IFF_SLAVE is checked, this scenario will not happen.
Zhu Yanjun
Quoted 15 lines
Quoted 3 lines
An ixgbe NIC directly connects to another NIC (let us call it NIC-a). And auto-negotiate is off while no static speed is set in the 2 NICs.
The ixgbe driver does not support disabling auto-negotiation directly. The only time this is true is when the advertised speed is restricted, so the above scenario is not possible (you either have autoneg or advertised speed set) with the current driver.
Is this example in theory or do you have your interface configured this way somehow?
Thanks,
Emil
RE: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 91 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, January 05, 2016 9:42 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 8 lines · Emil S
Quoted 8 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 8 lines
Quoted 8 lines
of
link_upand speedQuoted 5 lines · Emil S
Quoted 5 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W;Ronciak,
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 3 lines
Quoted 3 lines
of
link_upand speedOn 12/30/2015 12:18 AM, Tantilov, Emil S wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
Quoted 5 lines
Quoted 5 lines
On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John;Williams,
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
of
Quoted 4 lines
Quoted 4 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independentinterface,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
it is not necessary to synchronize
link_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.What exactly do you mean by "time span between
link_upand
link_speed"?Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
In the previous mail, I show you some ethtool logs. In these logs,
there
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Was this when reporting the link state from
check_link()(reading theLINKS
Quoted 6 lines
register) or reporting the
adapter->link_speed?Quoted 4 lines
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flagis
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
set, we continue to check
link_speed. If not, this function is executed whether thislink_speed
Quoted 7 lines
is
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
unknown or not.
I can already see this in your patch. I was asking about the reason why your change is needed.
an extreme example, let us assume this scenario:
Is this the scenario you are trying to fix?
Sure. If
IFF_SLAVEis checked, this scenario will not happen.
I already explained why this is not a valid scenario, but if you were able to set it up somehow I'd like to know how you did it
If we are to enter ixgbe_watchdog_link_is_up() with unknown link this would be an issue regardless of whether the interface is a part of a bond or not, but you haven't provided any proof that this is the case. Do you have a dmesg log that shows ixgbe reporting unknown speed?
Was your patch tested by the customer that reported this issue?
Thanks,
Emil
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
Quoted 92 lines · Emil S
Quoted 92 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, January 05, 2016 9:42 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 8 lines · Emil S
Quoted 8 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 8 lines
Quoted 8 lines
of
link_upand speedQuoted 5 lines · Emil S
Quoted 5 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W;Ronciak,
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 3 lines
Quoted 3 lines
of
link_upand speedOn 12/30/2015 12:18 AM, Tantilov, Emil S wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
Quoted 5 lines
Quoted 5 lines
On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John;Williams,
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
of
Quoted 4 lines
Quoted 4 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independentinterface,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
it is not necessary to synchronize
link_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.What exactly do you mean by "time span between
link_upand
link_speed"?Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
In the previous mail, I show you some ethtool logs. In these logs,
there
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Was this when reporting the link state from
check_link()(reading theLINKS
Quoted 6 lines
register) or reporting the
adapter->link_speed?Quoted 4 lines
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flagis
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
set, we continue to check
link_speed. If not, this function is executed whether thislink_speedis
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
unknown or not.
I can already see this in your patch. I was asking about the reason why your change is needed.
an extreme example, let us assume this scenario:
Is this the scenario you are trying to fix?
Sure. If
IFF_SLAVEis checked, this scenario will not happen.I already explained why this is not a valid scenario, but if you were able to set it up somehow I'd like to know how you did it
If it is not a valid scenario, maybe there is something wrong with NIC driver/hardware.
We should pay attention to it.
Zhu Yanjun
Quoted 11 lines
If we are to enter
ixgbe_watchdog_link_is_up()with unknown link this would be an issue regardless of whether the interface is a part of a bond or not, but you haven't provided any proof that this is the case. Do you have a dmesg log that shows ixgbe reporting unknown speed?Was your patch tested by the customer that reported this issue?
Thanks,
Emil
Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization of link_up and speed
In reply to Tantilov, Emil SQuoted 92 lines · Emil S
Quoted 92 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, January 05, 2016 9:42 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrict synchronization oflink_upand speedQuoted 8 lines · Emil S
Quoted 8 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 30, 2015 12:20 AM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 8 lines
Quoted 8 lines
of
link_upand speedQuoted 5 lines · Emil S
Quoted 5 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Tuesday, December 29, 2015 6:49 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W;Ronciak,
Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 3 lines
Quoted 3 lines
of
link_upand speedOn 12/30/2015 12:18 AM, Tantilov, Emil S wrote:
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
Quoted 5 lines
Quoted 5 lines
On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Monday, December 28, 2015 6:32 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John;Williams,
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 2 lines
Quoted 2 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 2/2] ixgbe: restrictsynchronization
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
of
Quoted 4 lines
Quoted 4 lines
link_upand speedFrom: Zhu Yanjun <[email protected]>
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
When the X540 NIC acts as a slave of some virtual NICs, it is very important to synchronize
link_upandlink_speed, such as a bonding driver in 802.3ad mode. When X540 NIC acts as an independentinterface,
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
it is not necessary to synchronize
link_upandlink_speed. That is, the time span betweenlink_upandlink_speedis acceptable.What exactly do you mean by "time span between
link_upand
link_speed"?Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
In the previous mail, I show you some ethtool logs. In these logs,
there
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
is some
time with NIC up while speed is unknown. I think this "some time" is time span betweenlink_upandlink_speed. Please see the previous mail for details.Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Was this when reporting the link state from
check_link()(reading theLINKS
Quoted 6 lines
register) or reporting the
adapter->link_speed?Quoted 4 lines
Quoted 1 line
Where is it you think the de-synchronization occurs?
When a NIC interface acts as a slave, a flag "
IFF_SLAVE" is set in netdevice struct.
Before we enter this function, we checkIFF_SLAVEflag. If this flagis
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
set, we continue to check
link_speed. If not, this function is executed whether thislink_speedis
Quoted 8 lines
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
unknown or not.
I can already see this in your patch. I was asking about the reason why your change is needed.
an extreme example, let us assume this scenario:
Is this the scenario you are trying to fix?
Sure. If
IFF_SLAVEis checked, this scenario will not happen.I already explained why this is not a valid scenario, but if you were able to set it up somehow I'd like to know how you did it
Sorry. Why can we not disable auto-negotiate on ixgbe NIC ? To be honest, I am interested in it.
This is related with hardware or driver?
Thanks a lot.
Zhu Yanjun
Quoted 12 lines
If we are to enter
ixgbe_watchdog_link_is_up()with unknown link this would be an issue regardless of whether the interface is a part of a bond or not, but you haven't provided any proof that this is the case. Do you have a dmesg log that shows ixgbe reporting unknown speed?Was your patch tested by the customer that reported this issue?
Thanks,
Emil
Hello.
Quoted 34 lines · On 12/23/2015 9:46 AM, zyjzyj2000 at gmail.com wrote:
From: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>--- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +5
-1@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ?"RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) · line 6496
Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +10
-0+* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +
{} not needed here. And you do the same thing regardless of whether your
check succeeds or not, this doesn't make sense.
[...]
MBR, Sergei
Tantilov, Emil S
NAKQuoted 62 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexFrom: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15
-1@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ? "RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);/* enable transmits */
Quoted 2 lines
--
1.7.9.5
NAK
I have already submitted a patch that will address the issue with bonding reporting unknown speed (in /proc/bonding/bondX) after the link is established due to link flaps:
http://patchwork.ozlabs.org/patch/552485/
The bonding driver gets the speed from ethtool and this is where the reporting needs to be fixed. The issue is that the bonding driver polls for netif_carrier_ok() at a certain rate and as such will not be able to detect rapid link changes.
If there is a case where link_speed is unknown when entering this function it's probably better to just bail rather than have this hack around the netif_carrier_on() especially after the driver already reported link status change. Rapid link changes can occur between link partners and not just for X540.
Thanks,
Emil
Quoted 67 lines · Emil S
Quoted 62 lines
-----Original Message-----
From: Intel-wired-lan [mailto:[email protected]] On Behalf Of [email protected]
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; [email protected]; [email protected]; e1000- [email protected]
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexFrom: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15
-1@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ? "RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);/* enable transmits */Quoted 2 lines
--
1.7.9.5NAK
Quoted 7 lines
I have already submitted a patch that will address the issue with bonding reporting unknown speed (in
/proc/bonding/bondX) after the link is established due to link flaps:
http://patchwork.ozlabs.org/patch/552485/The bonding driver gets the speed from ethtool and this is where the reporting needs to be fixed. The issue is that the bonding driver polls for
netif_carrier_ok()at a certain rate and as such will not be able to detect rapid link changes.
Kernel has netdev notifiers, these should really always be used rather than polling. The polling stuff was added back in ancient kernel days when netdev notifiers didn't work because most drivers were broken.
Quoted 66 lines · Emil S
Quoted 62 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org] On Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexFrom: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15
-1@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ? "RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);/* enable transmits */Quoted 2 lines
--
1.7.9.5NAK
Quoted 7 lines
I have already submitted a patch that will address the issue with bonding reporting unknown speed (in
/proc/bonding/bondX) after the link is established due to link flaps:
http://patchwork.ozlabs.org/patch/552485/The bonding driver gets the speed from ethtool and this is where the reporting needs to be fixed. The issue is that the bonding driver polls for
netif_carrier_ok()at a certain rate and as such will not be able to detect rapid link changes.
Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" while
the speed is unknown at the same time. We can run "ethtool ethx" to confirm it.
Any way, thanks a lot.
Zhu Yanjun
Quoted 9 lines
If there is a case where
link_speedis unknown when entering this function it's probably better to just bail rather than have this hack around thenetif_carrier_on()especially after the driver already reported link status change. Rapid link changes can occur between link partners and not just for X540.Thanks,
Emil
Quoted 89 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 6:28 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 52 lines
Quoted 52 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexFrom: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15
-1@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ? "RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);Quoted 6 lines
Quoted 3 lines
/* enable transmits */--
1.7.9.5NAK
I have already submitted a patch that will address the issue with bonding
reporting
Quoted 1 line
unknown speed (in
/proc/bonding/bondX) after the link is established dueto link flaps:
Quoted 3 lines
http://patchwork.ozlabs.org/patch/552485/
The bonding driver gets the speed from ethtool and this is where the
Quoted 8 lines
reporting needs
Quoted 1 line
to be fixed. The issue is that the bonding driver polls for
netif_carrier_ok()at aQuoted 1 line
certain rate and as such will not be able to detect rapid link changes.
Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" while
The "link up" event is a result of an LSC interrupt, the speed is determined as result of that interrupt by checking the LINKS register. If the LINKS register reports link as unknown then that is the actual state of the PHY - meaning the device is re-negotiating the speed for some reason.
Quoted 2 lines
the speed is unknown at the same time. We can run "ethtool ethx" to confirm it.
Prior to my patch the ethtool call will read the LINKS register which can show speed as unknown due to a link flap (for example). You are seeing the momentary state of the device.
If you are still seeing the bond reporting "unknown" speed after the patch I pointed out please file a bug either through e1000.sf.net or via Intel support and provide detailed information about the bonding setup, the type of the link partner (switch model etc) and full dmesg from the failed scenario along with the output from /proc/bonding/bond0
Thanks,
Emil
Quoted 91 lines · Emil S
Quoted 91 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 6:28 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 52 lines
Quoted 52 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexFrom: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +15
-1@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ? "RX" : (flow_tx ? "TX" : "None")))); -netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on" +* and getting the speed and duplex. To a bonding driver in 802.3ad +* mode, this time span will make it not work well if the time span +* is big enough. To 82599_SFP NIC and other kinds of NICs, this +* problem does not exist. As such, it is better for X540 to report +* "link on" when the link speed is not IXGBE_LINK_SPEED_UNKNOWN. +*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);Quoted 6 lines
Quoted 3 lines
/* enable transmits */--
1.7.9.5NAK
I have already submitted a patch that will address the issue with bonding
reporting
Quoted 1 line
unknown speed (in
/proc/bonding/bondX) after the link is established dueto link flaps:
Quoted 3 lines
http://patchwork.ozlabs.org/patch/552485/
The bonding driver gets the speed from ethtool and this is where the
reporting needs
Quoted 1 line
to be fixed. The issue is that the bonding driver polls for
Quoted 8 lines
Quoted 6 lines
netif_carrier_ok()at aQuoted 1 line
certain rate and as such will not be able to detect rapid link changes.
Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" whileThe "link up" event is a result of an LSC interrupt, the speed is determined as result of that interrupt by checking the LINKS register.
Hi,
Sorry. I do not agree with you. Please see the followings for details.
/**
* ixgbe_watchdog_update_link - update the link status
* @adapter: pointer to the device adapter structure
* @link_speed: pointer to a u32 to store the link_speed **/
static void ixgbe_watchdog_update_link(struct ixgbe_adapter *adapter)
From this function, link_up and link_speed is from watchdo poll.
Thanks for your reply.
Zhu Yanjun
Quoted 18 lines
If the LINKS register reports link as unknown then that is the actual state of the PHY - meaning the device is re-negotiating the speed for some reason.
Quoted 2 lines
the speed is unknown at the same time. We can run "ethtool ethx" to confirm it.
Prior to my patch the ethtool call will read the LINKS register which can show speed as unknown due to a link flap (for example). You are seeing the momentary state of the device.
If you are still seeing the bond reporting "unknown" speed after the patch I pointed out please file a bug either through e1000.sf.net or via Intel support and provide detailed information about the bonding setup, the type of the link partner (switch model etc) and full dmesg from the failed scenario along with the output from
/proc/bonding/bond0Thanks,
Emil
Quoted 133 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 10:24 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River); Bourg, Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexQuoted 8 lines · Emil S
Quoted 8 lines
-----Original Message-----
From: zhuyj [mailto:zyjzyj2000 at gmail.com]
Sent: Wednesday, December 23, 2015 6:28 PM
To: Tantilov, Emil S; Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams, Mitch A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000-devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 7 lines
Quoted 7 lines
Vincent (Wind River)
Subject: Re: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexQuoted 2 lines · Emil S
Quoted 2 lines
-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at lists.osuosl.org]
Quoted 44 lines
Quoted 44 lines
On
Quoted 4 lines
Quoted 4 lines
Behalf Of zyjzyj2000 at gmail.com
Sent: Tuesday, December 22, 2015 10:47 PM
To: Kirsher, Jeffrey T; Brandeburg, Jesse; Nelson, Shannon; Wyborny, Carolyn; Skidmore, Donald C; Allan, Bruce W; Ronciak, John; Williams,Mitch
Quoted 3 lines
Quoted 3 lines
A; intel-wired-lan at lists.osuosl.org; netdev at vger.kernel.org; e1000- devel at lists.sourceforge.net
Cc: Viswanathan, Ven (Wind River); Shteinbock, Boris (Wind River);Bourg,
Quoted 34 lines
Quoted 34 lines
Vincent (Wind River)
Subject: [Intel-wired-lan] [PATCH 1/1] ixgbe: force to synchronize reporting "link on" and getting speed and duplexFrom: Zhu Yanjun <[email protected]>
In X540 NIC, there is a time span between reporting "link on" and getting the speed and duplex. To a bonding driver in 802.3ad mode, this time span will make it not work well if the time span is big enough. The big time span will make bonding driver change the state of the slave device to up while the speed and duplex of the slave device can not be gotten. Later the bonding driver will not have change to get the speed and duplex of the slave device. The speed and duplex of the slave device are important to a bonding driver in 802.3ad mode.
To 82599_SFP NIC and other kinds of NICs, this problem does not exist. As such, it is necessary for X540 to report"link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Signed-off-by Zhu Yanjun <[email protected]>---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c| 16 +++++++++++++++-1 file changed, 15 insertions(+), 1 deletion(-)drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +0
-0@@ -6479,7 +6479,21 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter) (flow_rx ? "RX" : (flow_tx ? "TX" : "None"))));Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
Quoted 3 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +2
-1-netif_carrier_on (netdev); +/* +* In X540 NIC, there is a time span between reporting "link on"
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6485
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+* and getting the speed and duplex. To a bonding driver in 802.3ad
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6486
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+* mode, this time span will make it not work well if the time span
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6487
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+* is big enough. To 82599_SFP NIC and other kinds of NICs, this
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6488
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+* problem does not exist. As such, it is better for X540 to report
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c static void ixgbe_watchdog_link_is_up(struct · line 6489
Quoted 1 line
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +1
-0+* "link on" when the link speed is not
IXGBE_LINK_SPEED_UNKNOWN.Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +8
-0+*/ +if ((hw->mac.type == ixgbe_mac_X540) && + (link_speed !=IXGBE_LINK_SPEED_UNKNOWN )) { +netif_carrier_on (netdev); + }else { +netif_carrier_on (netdev); + } +ixgbe_check_vf_rate_limit (adapter);Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
/* enable transmits */--
1.7.9.5NAK
I have already submitted a patch that will address the issue with
bonding
Quoted 2 lines
Quoted 2 lines
reporting
Quoted 1 line
unknown speed (in
/proc/bonding/bondX) after the link is establisheddue
Quoted 6 lines
Quoted 6 lines
to link flaps:
Quoted 3 lines
http://patchwork.ozlabs.org/patch/552485/
The bonding driver gets the speed from ethtool and this is where the
reporting needs
Quoted 1 line
to be fixed. The issue is that the bonding driver polls for
Quoted 8 lines
Quoted 6 lines
netif_carrier_ok()at aQuoted 1 line
certain rate and as such will not be able to detect rapid link changes.
Thanks for your reply. The root cause is different from my problem. My problem is that
"link up" is prior to "speed and duplex". That is, the physical NIC reports "link up" whileThe "link up" event is a result of an LSC interrupt, the speed is determined as result of that interrupt by checking the LINKS register.
Hi,
Sorry. I do not agree with you. Please see the followings for details.
/**
*ixgbe_watchdog_update_link- update the link status
* @adapter: pointer to the device adapter structure
* @link_speed: pointer to a u32 to store thelink_speed**/
static voidixgbe_watchdog_update_link(struct ixgbe_adapter *adapter)
Quoted 3 lines
From this function,
link_upandlink_speedis from watchdo poll.Thanks for your reply.
ixgbe_watchdog_update_link() only runs when the IXGBE_FLAG_NEED_LINK_UPDATE is set which can happen in 2 situations - LSC is received or the interface is brought up. At that point the driver will call hw->mac.ops.check_link() which sets adapter->link_up based on the LINKS register and if we have link_up then we update the link status:
6669 if (adapter->link_up)
6670 ixgbe_watchdog_link_is_up(adapter);
The thing is if the LINKS.LINK_UP bit was set then at that point we also have a speed which is contrary to what you are suggesting. If you want to debug your issue you have to monitor the LINKS register.
That is why I asked for the dmesg output - if your claim is correct then the "Link Up" event in dmesg will be reporting "unknown speed" and if not then we are not dealing with delayed speed, but a simple link flap.
Thanks,
Emil