Skip to contentSkip to conversation

XLlTemac soft lockup BUG

DISCUSSION khollan 4 participants 8 messages Apr 15, 2008 – Apr 23, 2008

lore
Share & tools
lore

khollan

author
#0 lore raw

Hi all,

I am trying to switch my design from the PLB Temac to the local link temac, and i have a problem when trying to run ifconfig in my init script. It gives me a BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:208] Here is the rest of the dump, thanks for your help in advance.

[  233.114960] eth0: XLlTemac: Options: 0x3fa
[  233.118449] eth0: XLlTemac: allocating interrupt 8 for fifo mode.
[  237.135029] eth0: XLlTemac: We renegotiated the speed to: 100
[  237.150339] eth0: XLlTemac: speed set to 100Mb/s
[  244.792140] BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:204]
[  244.797813] NIP: c0188a5c LR: c012d3d0 CTR: c016c3f8
[  244.803533] REGS: ceb1fbf0 TRAP: 1001   Not tainted  (2.6.24-rc8-xlnx)
[  244.811039] MSR: 00029030 <EE,ME,IR,DR>  CR: 22000444  XER: 20000000
[  244.818363] TASK = cf82f7f0[204] 'ifconfig' THREAD: ceb1e000
[  244.824673] GPR00: cebca800 ceb1fca0 cf82f7f0 cebc84a0 cf86c000 00000010

cebc8520 00000000

[  244.834297] GPR08: 00000000 00000040 cebc84a0 00000001 c0248780 
[  244.841416] NIP [c0188a5c] eth_type_trans+0x10/0x114
[  244.847132] LR [c012d3d0] FifoRecvHandler+0x154/0x1dc
[  244.852945] Call Trace:
[  244.855758] [ceb1fca0] [c012d368] FifoRecvHandler+0xec/0x1dc (unreliable)
[  244.863579] [ceb1fcc0] [c0023b4c] tasklet_action+0x88/0xe4
[  244.869894] [ceb1fcd0] [c0023c20] __do_softirq+0x64/0xd4
[  244.876010] [ceb1fcf0] [c00064f4] do_softirq+0x40/0x58
[  244.881925] [ceb1fd00] [c0023a50] irq_exit+0x38/0x48
[  244.887640] [ceb1fd10] [c000673c] do_IRQ+0x88/0xa4
[  244.893154] [ceb1fd20] [c000325c] ret_from_except+0x0/0x18
[  244.899470] [ceb1fde0] [c0130804] xenet_open+0x24c/0x41c
[  244.905586] [ceb1fe10] [c017aa60] dev_open+0x88/0xf0
[  244.911300] [ceb1fe20] [c017a904] dev_change_flags+0xcc/0x1a0
[  244.917917] [ceb1fe40] [c01b9110] devinet_ioctl+0x298/0x6c0
[  244.924334] [ceb1fea0] [c01b965c] inet_ioctl+0xd0/0x104
[  244.930349] [ceb1feb0] [c016d12c] sock_ioctl+0x22c/0x268
[  244.936465] [ceb1fed0] [c006eb78] do_ioctl+0x38/0x84
[  244.942179] [ceb1fee0] [c006ef48] vfs_ioctl+0x384/0x3b8
[  244.948195] [ceb1ff10] [c006efbc] sys_ioctl+0x40/0x70
[  244.954010] [ceb1ff40] [c0002c10] ret_from_syscall+0x0/0x3c
[  244.960423] Instruction dump:
[  244.963832] 7d005379 3863012c 41820010 7d2534aa 7d2335aa 38c00000

7cc33378 4e800020

[  244.972752] 81230054 80030090 2b89000d 7c6a1b78 <90030080> 90830018

409d002c 80030058

Signature · 2 lines
View this message in context: http://www.nabble.com/XLlTemac--soft-lockup-BUG-tp16711066p16711066.html
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
#1 lore raw

linuxppc-embedded-bounces+hugo.villeneuve=3Dlyrtech.com@ozlabs.org =

Quoted 8 lines · wrote:

Hi all,
=20
I am trying to switch my design from the PLB Temac to the local link temac, and i have a problem when trying to run ifconfig in my init script. It gives me a BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:208] Here is the rest of the dump, thanks for your help in advance. [ 233.114960] eth0: XLlTemac: Options: 0x3fa =20 =20

Quoted 7 lines
[  233.118449] eth0: XLlTemac: allocating interrupt 8 for fifo mode.
=20
[  237.135029] eth0: XLlTemac: We renegotiated the speed to: 100
=20
[  237.150339] eth0: XLlTemac: speed set to 100Mb/s
=20
[  244.792140] BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:204]

Hi,
we had a similar error message, which was caused by us selecting the = wrong PHY type in the kernel configuration menu (latest = linux-2.6-xlnx-git tree). In fact, we had to modify the lltemac driver = to support our PHY (BCM5466). Once we did that, the error message went = away.

Hugo V.

Hugo Villeneuve
Hardware developer | Concepteur mat=E9riel
Lyrtech
Phone/T=E9l. : (1) (418) 877-4644 #2395 Toll-free/Sans frais - Canada & USA : (1) (888) 922-4644 #2395 Fax/T=E9l=E9c. : (1) (418) 877-7710
www.lyrtech.com
Infinite possibilities...TM

THIS MESSAGE AND ALL ATTACHED DOCUMENTS ARE EXCLUSIVELY INTENDED TO THE INDICATED RECIPIENTS AND ITS CONTENTS MAY BE CONFIDENTIAL. IT IS STRICTLY FORBIDDEN TO ANYONE TO TAKE COGNIZANCE, USE, OR DIVULGE THE INFORMATION CONTAINED HEREIN. IF YOU MISTAKENLY RECEIVE THIS MESSAGE, IMMEDIATELY INFORM LYRTECH AND DESTROY THE MESSAGE AND ATTACHMENTS FORTHWITH.THANK YOU.

LE PRESENT MESSAGE ET LES DOCUMENTS QUI Y SONT JOINTS S'ADRESSENT EXCLUSIVEMENT AU(X)DESTINATAIRE(S) INDIQUE(S) ET LEUR TENEUR PEUT ETRE CONFIDENTIELLE. IL EST STRICTEMENT INTERDIT A QUICONQUE D'EN PRENDRE CONNAISSANCE, DE LES UTILISER OU DE LES DIVULGUER. SI VOUS RECEVEZ LE PRESENT MESSAGE PAR ERREUR, VEUILLEZ EN AVISER LYRTECH IMMEDIATEMENT ET DETRUIRE LE MESSAGE SEANCE TENANTE, AINSI QUE LES DOCUMENTS QUI Y SONT JOINTS.

author
#2 lore raw
Quoted 9 lines · Hugo Villeneuve-3 wrote:

Hi,
we had a similar error message, which was caused by us selecting the wrong PHY type in the kernel configuration menu (latest linux-2.6-xlnx-git tree). In fact, we had to modify the lltemac driver to support our PHY (BCM5466). Once we did that, the error message went away.

Hugo V.

I tried out all 3 configurations and it didn't seem to help. We are using the Marvell Phy on our prototype. The weird thing is it seems to work perfectly every once in awhile, but not very often. I seem to have tracked the problem to the FifoRecvHandler tasklet function in xlltemac_main.c it gets stuck in this while loop:
while (XLlFifo_RxOccupancy(&lp->Fifo) != 0) {
the XLlFifo_RxOccupancy always reads the same value.
Any ideas?

Thanks
Kevin

Signature · 2 lines
View this message in context: http://www.nabble.com/XLlTemac--soft-lockup-BUG-tp16711066p16734685.html
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
#3 lore raw

Hi Kevin,

Does xlltemac_main.c have

    #define MARVELL_88E1111_PHY

in it? If not, try setting that.

It's odd that sometimes it works. Does your system meet timing?

We've found that when the PHY is being poked, it can take a while before the PHY settles down and starts working. If the code tries to start using the PHY too soon, then the PHY never seems to get to a good state. So, you might experiement with different values in the udelay() call in _XLlTemac_SetOperatingSpeed().

If you really think there's something wrong in the fifo code, you can enable debug (#define DEBUG) in the following files:
drivers/xilinx_common/xdebug.h
drivers/xilinx_common/xstreamer.h

Perhaps that will shed some light on the problem.

- John

Quoted 32 lines · khollan
Quoted 9 lines · Hugo Villeneuve-3 wrote:

Hi,
we had a similar error message, which was caused by us selecting the wrong PHY type in the kernel configuration menu (latest linux-2.6-xlnx-git tree). In fact, we had to modify the lltemac driver to support our PHY (BCM5466). Once we did that, the error message went away.

Hugo V.

I tried out all 3 configurations and it didn't seem to help. We are using the Marvell Phy on our prototype. The weird thing is it seems to work perfectly every once in awhile, but not very often. I seem to have tracked the problem to the FifoRecvHandler tasklet function in xlltemac_main.c it gets stuck in this while loop:
while (XLlFifo_RxOccupancy(&lp->Fifo) != 0) {
the XLlFifo_RxOccupancy always reads the same value.
Any ideas?

Thanks
Kevin

Signature · 7 lines
View this message in context: http://www.nabble.com/XLlTemac--soft-lockup-BUG-tp16711066p16734685.html
Sent from the linuxppc-embedded mailing list archive at Nabble.com.

_______________________________________________
Linuxppc-embedded mailing list
[email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-embedded
author
#4 lore raw

Hi,

I have done some more digging and it seems like whenever the system works the command XLlFifo_IsRxEmpty(&lp->Fifo)) returns a 1 meaning that the RxFifo is empty. I stuck this test in the xenet_open function right before the fifo interupt gets enabled.

When ever this test comes back as 0 the ifconfig will get into the soft lockup I described above.

Is there a function that empties out the fifo, I tried XLlFifo_RxReset but that doesn't seem to work.

Thanks

Kevin

Signature · 2 lines
View this message in context: http://www.nabble.com/XLlTemac--soft-lockup-BUG-tp16711066p16834738.html
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
#5 lore raw

Hi Kevin,

When the system locks up, does it get stuck in the FifoRecvHandler() routine?

How many bytes are returned from XLlFifo_RxGetLen(&lp->Fifo) in that routine?

You can try calling XLlFifo_Reset(&lp->Fifo); to see if that clears the fifo. Though, I suspect that this wont fix the root cause of the problem you're seeing.

- John

Quoted 16 lines · khollan

Hi,

I have done some more digging and it seems like whenever the system works the command XLlFifo_IsRxEmpty(&lp->Fifo)) returns a 1 meaning that the RxFifo is empty. I stuck this test in the xenet_open function right before the fifo interupt gets enabled.

When ever this test comes back as 0 the ifconfig will get into the soft lockup I described above.

Is there a function that empties out the fifo, I tried XLlFifo_RxReset but that doesn't seem to work.

Thanks

Kevin

#6 lore raw
In reply to Hugo Villeneuve

Hugo,

We have a design with the BCM5482, and will be trying it out shortly. Any chance you could post a patch of your BCM5466 mods?

-bri

Quoted 54 lines · Hugo Villeneuve
Quoted 8 lines · [email protected] wrote:

Hi all,

I am trying to switch my design from the PLB Temac to the local link temac, and i have a problem when trying to run ifconfig in my init script. It gives me a BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:208] Here is the rest of the dump, thanks for your help in advance. [ 233.114960] eth0: XLlTemac: Options: 0x3fa

Quoted 8 lines
[  233.118449] eth0: XLlTemac: allocating interrupt 8 for fifo mode.
[  237.135029] eth0: XLlTemac: We renegotiated the speed to: 100
[  237.150339] eth0: XLlTemac: speed set to 100Mb/s
[  244.792140] BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:204]

Hi,
we had a similar error message, which was caused by us selecting the wrong PHY type in the kernel configuration menu (latest linux-2.6-xlnx-git tree). In fact, we had to modify the lltemac driver to support our PHY (BCM5466). Once we did that, the error message went away.

Hugo V.

Hugo Villeneuve
Hardware developer | Concepteur matériel
Lyrtech
Phone/Tél. : (1) (418) 877-4644 #2395
Toll-free/Sans frais - Canada & USA : (1) (888) 922-4644 #2395 Fax/Téléc. : (1) (418) 877-7710
www.lyrtech.com
Infinite possibilities...TM

THIS MESSAGE AND ALL ATTACHED DOCUMENTS ARE EXCLUSIVELY INTENDED TO THE INDICATED RECIPIENTS AND ITS CONTENTS MAY BE CONFIDENTIAL. IT IS STRICTLY FORBIDDEN TO ANYONE TO TAKE COGNIZANCE, USE, OR DIVULGE THE INFORMATION CONTAINED HEREIN. IF YOU MISTAKENLY RECEIVE THIS MESSAGE, IMMEDIATELY INFORM LYRTECH AND DESTROY THE MESSAGE AND ATTACHMENTS FORTHWITH.THANK YOU.

LE PRESENT MESSAGE ET LES DOCUMENTS QUI Y SONT JOINTS S'ADRESSENT EXCLUSIVEMENT AU(X)DESTINATAIRE(S) INDIQUE(S) ET LEUR TENEUR PEUT ETRE CONFIDENTIELLE. IL EST STRICTEMENT INTERDIT A QUICONQUE D'EN PRENDRE CONNAISSANCE, DE LES UTILISER OU DE LES DIVULGUER. SI VOUS RECEVEZ LE PRESENT MESSAGE PAR ERREUR, VEUILLEZ EN AVISER LYRTECH IMMEDIATEMENT ET DETRUIRE LE MESSAGE SEANCE TENANTE, AINSI QUE LES DOCUMENTS QUI Y SONT JOINTS.
_______________________________________________
Linuxppc-embedded mailing list [email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-embedded

Signature · 2 lines
Brian Silverman
Concept X, LLC
#7 lore raw
Quoted 3 lines · Brian Silverman

Hugo,
We have a design with the BCM5482, and will be trying it out shortly. Any chance you could post a patch of your BCM5466 mods?=20

Hi,
The main issue here is that you need to tell the LLTEMAC interface about = the speed of your PHY. You can do this by adding this code example in = the function=20
set_mac_speed():

---
#define BROADCOM_BCM5466_RGMII_AUX_STATUS 0x19

#define BCM5466_SPEED_1000BASE_T_FULL_DUPLEX 7
#define BCM5466_SPEED_1000BASE_T_HALF_DUPLEX 6

#define BCM5466_SPEED_100BASE_T_FULL_DUPLEX  5
#define BCM5466_SPEED_100BASE_T4             4
#define BCM5466_SPEED_100BASE_T_HALF_DUPLEX  3
#define BCM5466_SPEED_10BASE_T_FULL_DUPLEX   2
#define BCM5466_SPEED_10BASE_T_HALF_DUPLEX   1
#define BCM5466_SPEED_AN_NOT_COMPLETE        0
        u16 RegValue;

        _XLlTemac_PhyRead(&lp->Emac, lp->gmii_addr, =
BROADCOM_BCM5466_RGMII_AUX_STATUS, &RegValue);
        phylinkspeed =3D ( RegValue & 0x0700 ) >> 8; /* Keeping only =

bits 10-8 */

        /* Update TEMAC speed accordingly */
        switch (phylinkspeed) {
        case BCM5466_SPEED_1000BASE_T_FULL_DUPLEX:
        case BCM5466_SPEED_1000BASE_T_HALF_DUPLEX:
                lp->cur_speed =3D 1000;
                break;
        case BCM5466_SPEED_100BASE_T_FULL_DUPLEX:
        case BCM5466_SPEED_100BASE_T_HALF_DUPLEX:
        case BCM5466_SPEED_100BASE_T4:
                lp->cur_speed =3D 100;
                break;
        case BCM5466_SPEED_10BASE_T_FULL_DUPLEX:
        case BCM5466_SPEED_10BASE_T_HALF_DUPLEX:
                lp->cur_speed =3D 10;
                break;
        default:
                lp->cur_speed =3D 1000;
                break;
        }

        _XLlTemac_SetOperatingSpeed(&lp->Emac, lp->cur_speed);

---

Hope it helps, Hugo.

Quoted 54 lines

Hugo Villeneuve wrote:
=20
linuxppc-embedded-bounces+hugo.villeneuve=3Dlyrtech.com@ozlabs.org
wrote:=20
=20
=20
Hi all,
=20
I am trying to switch my design from the PLB Temac to the local link temac, and i have a problem when trying to run ifconfig in my init script. It gives me a BUG: soft lockup - CPU#0 stuck for 11s! [ifconfig:208] Here is the rest of the dump, thanks for your help in advance. [ 233.114960] eth0: XLlTemac: Options: 0x3fa
=20

		[  233.118449] eth0: XLlTemac: allocating interrupt 8 for fifo mode.
=20
		[  237.135029] eth0: XLlTemac: We renegotiated the speed to: 100
=20
		[  237.150339] eth0: XLlTemac: speed set to 100Mb/s
=20
		[  244.792140] BUG: soft lockup - CPU#0 stuck for 11s!

[ifconfig:204]=20
=20
=20
=20

	Hi,
	we had a similar error message, which was caused by us selecting the

wrong PHY type in the kernel configuration menu (latest linux-2.6-xlnx-git tree). In fact, we had to modify the lltemac driver to support our PHY (BCM5466). Once we did that, the error
message went away. =20
=20
Hugo V.
=20
=20

	Hugo Villeneuve
	Hardware developer | Concepteur mat=E9riel
	Lyrtech
	Phone/T=E9l. : (1) (418) 877-4644 #2395
	Toll-free/Sans frais - Canada & USA : (1) (888) 922-4644 #2395
	Fax/T=E9l=E9c. : (1) (418) 877-7710
	www.lyrtech.com
	Infinite possibilities...TM

=20
THIS MESSAGE AND ALL ATTACHED DOCUMENTS ARE EXCLUSIVELY INTENDED TO THE INDICATED RECIPIENTS AND ITS CONTENTS MAY BE CONFIDENTIAL. IT IS STRICTLY FORBIDDEN TO ANYONE TO TAKE COGNIZANCE, USE, OR DIVULGE THE INFORMATION CONTAINED HEREIN. IF YOU MISTAKENLY RECEIVE THIS MESSAGE, IMMEDIATELY INFORM LYRTECH AND DESTROY THE MESSAGE AND ATTACHMENTS FORTHWITH.THANK YOU.
=20
LE PRESENT MESSAGE ET LES DOCUMENTS QUI Y SONT JOINTS S'ADRESSENT EXCLUSIVEMENT AU(X)DESTINATAIRE(S) INDIQUE(S) ET LEUR TENEUR PEUT ETRE CONFIDENTIELLE. IL EST STRICTEMENT INTERDIT A QUICONQUE D'EN

Quoted 8 lines

PRENDRE CONNAISSANCE, DE LES UTILISER OU DE LES DIVULGUER. SI VOUS RECEVEZ LE PRESENT MESSAGE PAR ERREUR, VEUILLEZ EN AVISER LYRTECH IMMEDIATEMENT ET DETRUIRE LE MESSAGE SEANCE TENANTE, AINSI QUE LES DOCUMENTS QUI Y SONT JOINTS.
_______________________________________________
Linuxppc-embedded mailing list [email protected]
https://ozlabs.org/mailman/listinfo/linuxppc-embedded

Hugo Villeneuve
Hardware developer | Concepteur mat=E9riel
Lyrtech
Phone/T=E9l. : (1) (418) 877-4644 #2395 Toll-free/Sans frais - Canada & USA : (1) (888) 922-4644 #2395 Fax/T=E9l=E9c. : (1) (418) 877-7710
www.lyrtech.com
Infinite possibilities...TM

THIS MESSAGE AND ALL ATTACHED DOCUMENTS ARE EXCLUSIVELY INTENDED TO THE INDICATED RECIPIENTS AND ITS CONTENTS MAY BE CONFIDENTIAL. IT IS STRICTLY FORBIDDEN TO ANYONE TO TAKE COGNIZANCE, USE, OR DIVULGE THE INFORMATION CONTAINED HEREIN. IF YOU MISTAKENLY RECEIVE THIS MESSAGE, IMMEDIATELY INFORM LYRTECH AND DESTROY THE MESSAGE AND ATTACHMENTS FORTHWITH.THANK YOU.

LE PRESENT MESSAGE ET LES DOCUMENTS QUI Y SONT JOINTS S'ADRESSENT EXCLUSIVEMENT AU(X)DESTINATAIRE(S) INDIQUE(S) ET LEUR TENEUR PEUT ETRE CONFIDENTIELLE. IL EST STRICTEMENT INTERDIT A QUICONQUE D'EN PRENDRE CONNAISSANCE, DE LES UTILISER OU DE LES DIVULGUER. SI VOUS RECEVEZ LE PRESENT MESSAGE PAR ERREUR, VEUILLEZ EN AVISER LYRTECH IMMEDIATEMENT ET DETRUIRE LE MESSAGE SEANCE TENANTE, AINSI QUE LES DOCUMENTS QUI Y SONT JOINTS.