Skip to contentSkip to conversation

ixgbe: X540T + BMC - speed stuck at 100Mbit

Intel-wired-lan Daniel Axtens 2 participants 2 messages Jul 25, 2017

lore
Share & tools
lore
#0 lore raw

Hi,

I'm looking at a Dell system that has an ixgbe X540T card that is shared with a BMC.

When the system boots, the card negotiates a speed of 100Mbit, even though the other end is capable of 10Gbit and autonegotiation is on.

Unplugging the network cable from the card and plugging it back in again causes the speed to go to 10Gbit, but if the system is shut down and then booted again, the speed drops back to 100Mbit.

Asking for a renegotiation with 'ethtool -r' doesn't cause the speed to change, nor does bringing it down and up with 'ip link'.

Our theory is that this is related to the sharing with the BMC. It was first observed when the userspace on the system changed from Ubuntu 14.04 to Ubuntu 16.04, but it doesn't seem to depend on the kernel version - it seems to be some sort of weird race condition.

Is there any way to force a harder reset of the card to make it renegotiate? I thought that probe() would call reset_hw() and that ixgbe_reset_hw_X540() would be sufficient to reset the link but it doesn't seem to be taking.

(This has been observed on a stock Xenial kernel, which is based on v4.4 but with some of the more recent fixes to the driver backported. I haven't seen anything in the git history that looks like it would fix the issue, but I can try a more recent kernel version if that would be helpful.)

Thanks in advance!

Regards,
Daniel

#1 lore raw
Quoted 4 lines

-----Original Message-----
From: Intel-wired-lan [mailto:intel-wired-lan-bounces at osuosl.org] On Behalf Of Daniel Axtens
Sent: Monday, July 24, 2017 10:23 PM

To: [email protected]; Kirsher, Jeffrey T

Quoted 15 lines

<[email protected]>
Subject: [Intel-wired-lan] ixgbe: X540T + BMC - speed stuck at 100Mbit

Hi,

I'm looking at a Dell system that has an ixgbe X540T card that is shared with a BMC.

When the system boots, the card negotiates a speed of 100Mbit, even though the other end is capable of 10Gbit and autonegotiation is on.

Unplugging the network cable from the card and plugging it back in again causes the speed to go to 10Gbit, but if the system is shut down and then booted again, the speed drops back to 100Mbit.

Quoted 7 lines

Asking for a renegotiation with 'ethtool -r' doesn't cause the speed to change, nor does bringing it down and up with 'ip link'.

Our theory is that this is related to the sharing with the BMC. It was first observed when the userspace on the system changed from Ubuntu 14.04 to Ubuntu 16.04, but it doesn't seem to depend on the kernel version - it seems to be some sort of weird race condition.

The only time the driver cares about the existence of a BMC is when deciding whether to shut down the PHY. The supported/negotiated speeds are configured the same. If you suspect the BMC you can try and disable it. Also you can try with different link partners, or simply limit the advertised speed to 10gig:

Quoted 6 lines

ethtool -s ethX advertise 0x1000

Is there any way to force a harder reset of the card to make it renegotiate? I thought that probe() would call reset_hw() and that ixgbe_reset_hw_X540() would be sufficient to reset the link but it doesn't seem to be taking.

Any reset that results in a link drop (as observed in dmesg as link down/up events should also result in the link being re-negotiated).

Quoted 5 lines

(This has been observed on a stock Xenial kernel, which is based on v4.4 but with some of the more recent fixes to the driver backported. I haven't seen anything in the git history that looks like it would fix the issue, but I can try a more recent kernel version if that would be helpful.)

You can try the out-of-tree driver from e1000.s.net.

Thanks,
Emil