tg3: link is permanently down after ifdown and ifup

Hi,

I have a problem with tg3 driver on a custom MPC8536 based board running linux-2.6.31, with tg3 and Broadcom phy drivers taken from linux-2.6.32-rc7. Broadcom NIC is BCM57760, phy is BCM57780.

The problem I'm seeing is that the downing and interface leads to permanent link loss, even after interface is upped again. E.g, to reproduce the problem it is sufficient to run:

modprobe tg3
ifconfig eth2 up
ifconfig eth2 down
ifconfig eth2 up

After ifdown PHY LEDs also go down and do not come back after ifup. Ethtool reports that no link is detected. After reloading the driver the link comes back.

Am I the only one seeing this problem ?
Any help on fixing this is appreciated.

Thanks a lot.

Felix.

_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev

Hi,

The problem goes away if I remove the call to

tg3_set_power_state(tp, PCI_D3hot);

from tg3_close().

Quoted 4 lines · Felix Radensky

Hi,

The problem goes away if I remove the call to

Quoted 4 lines

tg3_set_power_state(tp, PCI_D3hot);

from tg3_close().

Added Matt to CC. He is on vacation and may not be able to look into this right away. Thanks.

Quoted 8 lines · Felix Radensky

Hi,

The problem goes away if I remove the call to

tg3_set_power_state(tp, PCI_D3hot);

from tg3_close().

Added Matt to CC. He is on vacation and may not be able to look into this right away. Thanks.

_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev

Hi, Matt

Did you have a chance to look into this problem ?

Thanks a lot.

Felix.

Quoted 13 lines · Michael Chan

Quoted 8 lines · Felix Radensky

Hi,

The problem goes away if I remove the call to

tg3_set_power_state(tp, PCI_D3hot);

from tg3_close().

Added Matt to CC. He is on vacation and may not be able to look into this right away. Thanks.

I understand the problem now. The problem is that tg3_set_power_state() puts the phy into a low-power mode when it releases the device. When the phy is reacquired, a phy reset is missing to take the device back out of the low-power mode.

The patch at the bottom of this email is the fix I'm currently testing. If you wish, you can try it out.

Hi, Matt

Quoted 9 lines · Matt Carlson

I understand the problem now. The problem is that tg3_set_power_state() puts the phy into a low-power mode when it releases the device. When the phy is reacquired, a phy reset is missing to take the device back out of the low-power mode.

The patch at the bottom of this email is the fix I'm currently testing. If you wish, you can try it out.

Thanks a lot for the fix, I'll ask the guys with access to hardware to test it.

The problem with lost link happened to me when link was synchronized at 1000Mbit/sec. At 100Mbit/sec, ifdown would result in the following soft lockup:

BUG: soft lockup - CPU#0 stuck for 61s! [events/0:5]
Modules linked in: ehci_hcd usbcore
NIP: c0191f0c LR: c018774c CTR: c0191f04
REGS: ef83fe60 TRAP: 0901 Not tainted (2.6.31)
MSR: 00029000 <EE,ME,CE> CR: 24022084 XER: 00000000

TASK = ef8312c0[5] 'events/0' THREAD: ef83e000
GPR00: f1040000 ef83ff10 ef8312c0 ef8292c0 00000400 00c04808 00000000 00000000
GPR08: 00000000 00000000 00000006 00c04800 44024084
NIP [c0191f0c] tg3_read32+0x8/0x18
LR [c018774c] _tw32_flush+0x94/0xa4

Call Trace:

[ef83ff10] [ef8314b8] 0xef8314b8 (unreliable) [ef83ff30] [c0192074] tg3_adjust_link+0x98/0x24c [ef83ff50] [c017eab0] phy_state_machine+0xd4/0x5b8 [ef83ff70] [c0037570] worker_thread+0x124/0x1b8 [ef83ffc0] [c003b5fc] kthread+0x78/0x7c [ef83fff0] [c000f584] kernel_thread+0x4c/0x68

Can you reproduce this ?

Thanks a lot for your help.

Quoted 38 lines · Felix Radensky

Hi, Matt

Quoted 10 lines · Matt Carlson

I understand the problem now. The problem is that tg3_set_power_state() puts the phy into a low-power mode when it releases the device. When the phy is reacquired, a phy reset is missing to take the device back out of the low-power mode.

The patch at the bottom of this email is the fix I'm currently testing. If you wish, you can try it out.

Thanks a lot for the fix, I'll ask the guys with access to hardware to test it.

The problem with lost link happened to me when link was synchronized at 1000Mbit/sec. At 100Mbit/sec, ifdown would result in the following soft lockup:

BUG: soft lockup - CPU#0 stuck for 61s! [events/0:5]
Modules linked in: ehci_hcd usbcore
NIP: c0191f0c LR: c018774c CTR: c0191f04
REGS: ef83fe60 TRAP: 0901 Not tainted (2.6.31)
MSR: 00029000 <EE,ME,CE> CR: 24022084 XER: 00000000

TASK = ef8312c0[5] 'events/0' THREAD: ef83e000
GPR00: f1040000 ef83ff10 ef8312c0 ef8292c0 00000400 00c04808 00000000 00000000
GPR08: 00000000 00000000 00000006 00c04800 44024084
NIP [c0191f0c] tg3_read32+0x8/0x18
LR [c018774c] _tw32_flush+0x94/0xa4

Call Trace:

[ef83ff10] [ef8314b8] 0xef8314b8 (unreliable) [ef83ff30] [c0192074] tg3_adjust_link+0x98/0x24c [ef83ff50] [c017eab0] phy_state_machine+0xd4/0x5b8 [ef83ff70] [c0037570] worker_thread+0x124/0x1b8 [ef83ffc0] [c003b5fc] kthread+0x78/0x7c [ef83fff0] [c000f584] kernel_thread+0x4c/0x68

Quoted 4 lines

Can you reproduce this ?

Thanks a lot for your help.

Is this with the "Fix phylib locking strategy" patch in place?

I understand the problem now. The problem is that tg3_set_power_state() puts the phy into a low-power mode when it releases the device. When the phy is reacquired, a phy reset is missing to take the device back out of the low-power mode.

The patch at the bottom of this email is the fix I'm currently testing. If you wish, you can try it out.

[PATCH] tg3: Fix 57780 connectivity problems

When management firmware is not present, and WOL is disabled, the driver will put the phy into a low-power mode when shutting down. To get out of low-power mode, the operation must be reversed or the phy must be reset.

In the past, the phylib reset the phy from either phy_start() or phy_start_aneg(). This phy reset has been removed from more recent kernels. The tg3 driver already has phy reset code in place which is called when it assumes control of the device. It is bypassed if phylib is enabled though.

This patch removes the phylib exception.

---
 drivers/net/tg3.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

drivers/net/tg3.c +0 -0

index 3a74d21..c0a3080 100644

Hi, Matt

Quoted 33 lines · Matt Carlson

I understand the problem now. The problem is that tg3_set_power_state() puts the phy into a low-power mode when it releases the device. When the phy is reacquired, a phy reset is missing to take the device back out of the low-power mode.

The patch at the bottom of this email is the fix I'm currently testing. If you wish, you can try it out.

[PATCH] tg3: Fix 57780 connectivity problems

When management firmware is not present, and WOL is disabled, the driver will put the phy into a low-power mode when shutting down. To get out of low-power mode, the operation must be reversed or the phy must be reset.

In the past, the phylib reset the phy from either phy_start() or phy_start_aneg(). This phy reset has been removed from more recent kernels. The tg3 driver already has phy reset code in place which is called when it assumes control of the device. It is bypassed if phylib is enabled though.

This patch removes the phylib exception.

---
 drivers/net/tg3.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

drivers/net/tg3.c +0 -0

index 3a74d21..c0a3080 100644

drivers/net/tg3.c static int tg3_reset_hw(struct tg3 *tp, int reset_phy) · line 7529

Quoted 9 lines

drivers/net/tg3.c +1 -2

@@ -7523,8 +7523,7 @@ static int tg3_reset_hw(struct tg3 *tp, int reset_phy)
 		tg3_abort_hw(tp, 1);
 	}
 
-	if (reset_phy &&
-	    !(tp->tg3_flags3 & TG3_FLG3_USE_PHYLIB))
+	if (reset_phy)
 		tg3_phy_reset(tp);
 
 	err = tg3_chip_reset(tp);

I can confirm that this patch fixes the problem I had. Thanks a lot for the fix.

There was another problem in 2.6.32 that I reported, namely kernel soft lockup
when link is synchronized at 100Mbit/sec and tg3 with 57780 goes down . Were you able to reproduce it ?

Quoted 5 lines

There was another problem in 2.6.32 that I reported, namely kernel soft lockup when link is synchronized at 100Mbit/sec and tg3 with 57780 goes down . Were you able to reproduce it ?

No, but perhaps I don't understand the problem well enough. Are you saying that you encounter the problem when you run 'ifconfig bcm57780 down', or is it that losing link itself causes the problem?

Hi, Matt

Quoted 6 lines · Matt Carlson

Quoted 5 lines

There was another problem in 2.6.32 that I reported, namely kernel soft lockup when link is synchronized at 100Mbit/sec and tg3 with 57780 goes down . Were you able to reproduce it ?

Quoted 6 lines

No, but perhaps I don't understand the problem well enough. Are you saying that you encounter the problem when you run 'ifconfig bcm57780 down', or is it that losing link itself causes the problem?

It is enough to connect the NIC to 10/100 switch and run 'ifconfig ethX down' to reproduce the problem.