Skip to contentSkip to conversation

TSEC ethernet controller problems (crc errors / corruption)

U-Boot Ira Snyder 7 participants 33 messages Jun 2, 2009 – Jun 9, 2009 Network drivers

lore
Share & tools
lore

Ira Snyder

author
#0 lore raw

Hello U-Boot users,

I've been working on a custom board, based heavily on the Freescale MPC8349EMDS board. The only major difference is that the board has some FPGAs connected to the local bus.

I've found that the TSEC / gianfar ethernet does not work for me in 1000mbit mode. I constantly get "got error 4" from U-Boot, which means that a CRC error was detected by the TSEC controller.

I have tried running the TSEC in 100mbit mode, without any problems. I can tftp as much data as I want in U-Boot, and I've transferred many, many gigabytes without issue in Linux. The problem is only with 1000mbit mode.

I initially suspected problems with the PHY, a Marvell 88E1111. To diganose the problem, I set the PHY into loopback mode, then connected my logic analyzer to the PHY's RX lines. This removes the external network from the equation: I don't even have an ethernet cable connected.

The logic analyzer traces show the correct, expected packet data coming from the PHY. By the time the TSEC copies the data into a buffer descriptor, the packet data has been corrupted. The problem is extra or missing bytes in the packet data. I have added some code to dump the packet contents on CRC error, and what I see in memory is not what the PHY sent.

The corruption is not at a consistent offset in the packet. The number of extra or missing bytes is not always the same.

The extra bytes that show up in the packet data seem like a FIFO issue to me. It is as if the internal FIFO reported that more data was available before the data was actually present. Something like that.

For reference, here is what a memory dump of a good packet looks like. It has a length of 64 bytes. I have dumped 16 bytes past the end of the packet data.

0fffcbe0: ff ff ff ff ff ff ae dc 30 45 e8 3a 08 06 00 01    ........0E.:....
0fffcbf0: 08 00 06 04 00 01 ae dc 30 45 e8 3a c0 a8 11 45    ........0E.:...E
0fffcc00: 00 00 00 00 00 00 c0 a8 11 39 00 00 00 00 00 00    .........9......
0fffcc10: 00 00 00 00 00 00 00 00 00 00 00 00 2c d1 57 03    ............,.W.
0fffcc20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................

And this is what a bad packet looks like in memory. The length was reported by the TSEC as 64 bytes, but it was flagged as having a CRC error. Notice the extra bytes starting at 0x0fffcc0c, "copied" from 16 bytes earlier in the packet. The CRC is shifted 4 bytes in the packet due to the extra bytes.

0fffcbe0: ff ff ff ff ff ff ae dc 30 45 e8 3a 08 06 00 01    ........0E.:....
0fffcbf0: 08 00 06 04 00 01 ae dc 30 45 e8 3a c0 a8 11 45    ........0E.:...E
0fffcc00: 00 00 00 00 00 00 c0 a8 11 39 00 00 c0 a8 11 45    .........9.....E
0fffcc10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
0fffcc20: 2c d1 57 03 00 00 00 00 00 00 00 00 00 00 00 00    ,.W.............

I have been running U-Boot and Linux on this board for >6 months. I've run plenty of stress tests, and I've been doing lots of work with the DMA controller. I've stressed the memory pretty hard, and I'm fairly confident that it is not a problem with RAM. The board has been perfectly stable.

Other solutions I have tried, without any effect:
* slowing down the core frequency from 528 -> 264 MHz
* slowing down the RAM frequency from 266 -> 200 MHz

I am unable to reproduce the corruption on my MPC8349EMDS eval board, but I'm running out of ideas to try and find the source of this problem.

Has anyone seen anything like this before?

Thanks for your help,
Ira

PS - Below is the patch of the debugging code I've added.

drivers/net/tsec.c +27 -3
index 399116f..2f8f1bc 100644
@@ -214,7 +214,9 @@ int tsec_init(struct eth_device *dev, bd_t * bd)
 	startup_tsec(dev);
 
 	/* If there's no link, fail */
-	return (priv->link ? 0 : -1);
+	printf("LINK: %d\n", priv->link);
+	return 0;
+	//return (priv->link ? 0 : -1);
 }
 
 /* Writes the given phy's reg with value, using the specified MDIO regs */
@@ -928,22 +930,44 @@ static int tsec_send(struct eth_device *dev, volatile void *packet, int length)
 	return result;
 }
 
+static unsigned char MYPACKET[4096];
+
 static int tsec_recv(struct eth_device *dev)
 {
 	int length;
 	struct tsec_private *priv = (struct tsec_private *)dev->priv;
 	volatile tsec_t *regs = priv->regs;
+	u32 hwcrc, swcrc, cpcrc, rxaddr;
 
 	while (!(rtx.rxbd[rxIdx].status & RXBD_EMPTY)) {
 
 		length = rtx.rxbd[rxIdx].length;
 
+		hwcrc = *(u32 *)(NetRxPackets[rxIdx] + length - 4);
+		memcpy(MYPACKET, NetRxPackets[rxIdx], length + 16);
+		swcrc = crc32(0, MYPACKET, length - 4);
+		swcrc = swab32(swcrc);
+		cpcrc = *(u32 *)(MYPACKET + length - 4);
+
+		printf("CRC HW 0x%.8x SW 0x%.8x CP 0x%.8x IEVENT 0x%.8x MACCFG1 0x%.8x\n",
+				hwcrc, swcrc, cpcrc, regs->ievent, regs->maccfg1);
+
 		/* Send the packet up if there were no errors */
 		if (!(rtx.rxbd[rxIdx].status & RXBD_STATS)) {
 			NetReceive(NetRxPackets[rxIdx], length - 4);
 		} else {
-			printf("Got error %x\n",
-			       (rtx.rxbd[rxIdx].status & RXBD_STATS));
+			rxaddr = (u32)NetRxPackets[rxIdx];
+			printf("Got error %x\n", (rtx.rxbd[rxIdx].status & RXBD_STATS));
+			printf("PACKET DATA START 0x%.8x END 0x%.8x LEN %d\n", rxaddr, rxaddr + length, length);
+			printf("RXIDX %d STAT 0x%.8x LEN %d\n", rxIdx, rtx.rxbd[rxIdx].status, length);
+			/* Print the buffer:
+			 * $1: start address (for display purposes)
+			 * $2: memory buffer to print
+			 * $3: access mode (byte, short, word)
+			 * $4: length to print
+			 * $5: line length
+			 */
+			print_buffer(rxaddr, MYPACKET, 1, length + 16, 0);
 		}
 
 		rtx.rxbd[rxIdx].length = 0;
include/configs/MPC8349EMDS.h +4 -0
index 3c57403..1287829 100644
@@ -53,6 +53,10 @@
 #define CONFIG_83XX_PCICLK	66666666	/* in Hz */
 #endif /* CONFIG_PCISLAVE */
 
+#define CONFIG_SYS_RX_ETH_BUFFER	16
+#define CONFIG_ARP_TIMEOUT		750UL
+#define CONFIG_NET_RETRY_COUNT		64
+
 #ifndef CONFIG_SYS_CLK_FREQ
 #ifdef PCI_66M
 #define CONFIG_SYS_CLK_FREQ	66000000
#1 lore raw
Quoted 3 lines

I am unable to reproduce the corruption on my MPC8349EMDS eval board, but I'm running out of ideas to try and find the source of this problem.

One more piece of info: the MPC8349E-MDS board contains the MPC8349E processor, whereas our board contains the MPC8349EA (silicon v 3.1) processor. We did have a couple of MPC8349EA-MDS boards, however, the Xilinx CPLD on both boards spontaneously destructed (the CPLD sticky labels melted!).

We can try putting the MPC8349EA processor from one of the dead boards into the MPC8349E-MDS board socket - I believe this is ok - feel free to warn us that its not :)

Cheers,
Dave

#2 lore raw
Quoted 9 lines · David Hawkins
Quoted 2 lines

I am unable to reproduce the corruption on my MPC8349EMDS eval board, but I'm running out of ideas to try and find the source of this problem.

One more piece of info: the MPC8349E-MDS board contains the MPC8349E processor, whereas our board contains the MPC8349EA (silicon v 3.1) processor. We did have a couple of MPC8349EA-MDS boards, however, the Xilinx CPLD on both boards spontaneously destructed (the CPLD sticky labels melted!).

Quoted 3 lines

We can try putting the MPC8349EA processor from one of the dead boards into the MPC8349E-MDS board socket - I believe this is ok - feel free to warn us that its not :)

What interface are you using between the MAC and PHY? I saw somewhat similar weirdness at gigabit on a board when using RGMII and the clock delays in the PHY and PCB didn't match up correctly (Section 5.12.2 from the 88e1111 manual). If you aren't using RGMII, this wouldn't be your issue though...

Best,
Peter

#3 lore raw

Hi Peter,

Quoted 12 lines
Quoted 11 lines
Quoted 2 lines

I am unable to reproduce the corruption on my MPC8349EMDS eval board, but I'm running out of ideas to try and find the source of this problem.

One more piece of info: the MPC8349E-MDS board contains the MPC8349E processor, whereas our board contains the MPC8349EA (silicon v 3.1) processor. We did have a couple of MPC8349EA-MDS boards, however, the Xilinx CPLD on both boards spontaneously destructed (the CPLD sticky labels melted!).

We can try putting the MPC8349EA processor from one of the dead boards into the MPC8349E-MDS board socket - I believe this is ok - feel free to warn us that its not :)

Quoted 5 lines

What interface are you using between the MAC and PHY? I saw somewhat similar weirdness at gigabit on a board when using RGMII and the clock delays in the PHY and PCB didn't match up correctly (Section 5.12.2 from the 88e1111 manual). If you aren't using RGMII, this wouldn't be your issue though...

The default is to use GMII. Ira reconfigured the interface to RGMII after boot, and saw similar errors to those he was getting in GMII mode.

Probing the PHY-to-MAC path shows the data we expect, so the error seems to be internal to the processor ...

The signal integrity on the board is good, we compared the timing between our board and the Freescale MDS board, and it all looks very similar.

Cheers,
Dave

author
#4 lore raw
In reply to Peter Tyser
Quoted 14 lines · Peter Tyser
Quoted 9 lines · David Hawkins
Quoted 2 lines

I am unable to reproduce the corruption on my MPC8349EMDS eval board, but I'm running out of ideas to try and find the source of this problem.

One more piece of info: the MPC8349E-MDS board contains the MPC8349E processor, whereas our board contains the MPC8349EA (silicon v 3.1) processor. We did have a couple of MPC8349EA-MDS boards, however, the Xilinx CPLD on both boards spontaneously destructed (the CPLD sticky labels melted!).

Quoted 3 lines

We can try putting the MPC8349EA processor from one of the dead boards into the MPC8349E-MDS board socket - I believe this is ok - feel free to warn us that its not :)

Quoted 6 lines

What interface are you using between the MAC and PHY? I saw somewhat similar weirdness at gigabit on a board when using RGMII and the clock delays in the PHY and PCB didn't match up correctly (Section 5.12.2 from the 88e1111 manual). If you aren't using RGMII, this wouldn't be your issue though...

We're using GMII, exactly the same as the Freescale eval board. I did a very quick test with the board in RGMII mode as well, and the board behaved in exactly the same way it does in GMII mode.

Thanks,
Ira

#5 lore raw
Quoted 11 lines · On Tue, 2 Jun 2009 11:17:26 -0700 Ira Snyder <[email protected]> wrote:
Quoted 10 lines · Peter Tyser
Quoted 9 lines · David Hawkins
Quoted 2 lines

I am unable to reproduce the corruption on my MPC8349EMDS eval board, but I'm running out of ideas to try and find the source of this problem.

One more piece of info: the MPC8349E-MDS board contains the MPC8349E processor, whereas our board contains the MPC8349EA (silicon v 3.1) processor. We did have a couple of MPC8349EA-MDS boards, however, the Xilinx CPLD on both boards spontaneously destructed (the CPLD sticky labels melted!).

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

We can try putting the MPC8349EA processor from one of the dead boards into the MPC8349E-MDS board socket - I believe this is ok - feel free to warn us that its not :)

the 8349E rev.1.x had a TSEC FIFO erratum, and I've put an 8360EA chip on an 8360E-MDS board and it came up fine (apart from the PHY timing issues that plagued the 8360), so it would be good to try.

Quoted 10 lines
Quoted 6 lines

What interface are you using between the MAC and PHY? I saw somewhat similar weirdness at gigabit on a board when using RGMII and the clock delays in the PHY and PCB didn't match up correctly (Section 5.12.2 from the 88e1111 manual). If you aren't using RGMII, this wouldn't be your issue though...

We're using GMII, exactly the same as the Freescale eval board. I did a very quick test with the board in RGMII mode as well, and the board behaved in exactly the same way it does in GMII mode.

all I can say is git grep R_TSEC for more TSEC frequency & timing knobs...

hth,

Kim

#6 lore raw
In reply to Ira Snyder

Dear Ira Snyder,

Quoted 5 lines · In message <[email protected]> you wrote:

I've been working on a custom board, based heavily on the Freescale MPC8349EMDS board. The only major difference is that the board has some FPGAs connected to the local bus.

Quoted 8 lines

I've found that the TSEC / gianfar ethernet does not work for me in 1000mbit mode. I constantly get "got error 4" from U-Boot, which means that a CRC error was detected by the TSEC controller.

I have tried running the TSEC in 100mbit mode, without any problems. I can tftp as much data as I want in U-Boot, and I've transferred many, many gigabytes without issue in Linux. The problem is only with 1000mbit mode.

I read that the same problem happens in Linux when running at 1000mbit. Is this understanding correct?

Does running with root file system mounted over NFS work on this board? Does it work at 1000mbit, too?

Did you try and re-check the memory controller initialization?

Best regards,

Wolfgang Denk

Signature · 5 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"Spock, did you see the looks on their faces?"
"Yes, Captain, a sort of vacant contentment."
#7 lore raw

Hi Wolfgang,

Quoted 13 lines
Quoted 4 lines

I've been working on a custom board, based heavily on the Freescale MPC8349EMDS board. The only major difference is that the board has some FPGAs connected to the local bus.

Quoted 8 lines

I've found that the TSEC / gianfar ethernet does not work for me in 1000mbit mode. I constantly get "got error 4" from U-Boot, which means that a CRC error was detected by the TSEC controller.

I have tried running the TSEC in 100mbit mode, without any problems. I can tftp as much data as I want in U-Boot, and I've transferred many, many gigabytes without issue in Linux. The problem is only with 1000mbit mode.

Quoted 5 lines

I read that the same problem happens in Linux when running at 1000mbit. Is this understanding correct?

Does running with root file system mounted over NFS work on this board? Does it work at 1000mbit, too?

I'll let Ira answer those questions.

Quoted 1 line

Did you try and re-check the memory controller initialization?

The memory is DDR1 (2.5V) configured with ECC enabled. The DDR control clock was moved to various settings to see how much timing margin there was, and the central setting was selected (there appears to be good margin).

I'm not sure that a TSEC induced memory burst would be any different than a DMA induced memory, so if we were going to see errors, I would have expected to see them during DMA testing.

The strange thing about the patterns Ira is seeing is that the erroneous data sometimes matches data that occurred much earlier in a packet, not the last data on the DDR bus. This has a symptom like a FIFO address wrap-around where the FIFO erroneously indicates it has valid data, so the FIFO reader reads the stale data, and writes it to DDR.

Ira will play with some of the TSEC settings and see if he can change the symptom.

Thanks for the suggestions!

Cheers,
Dave

author
#8 lore raw
In reply to Wolfgang Denk
Quoted 12 lines · Wolfgang Denk

Dear Ira Snyder,

Quoted 9 lines · In message <[email protected]> you wrote:

I've been working on a custom board, based heavily on the Freescale MPC8349EMDS board. The only major difference is that the board has some FPGAs connected to the local bus.

I've found that the TSEC / gianfar ethernet does not work for me in 1000mbit mode. I constantly get "got error 4" from U-Boot, which means that a CRC error was detected by the TSEC controller.

Quoted 8 lines
Quoted 4 lines

I have tried running the TSEC in 100mbit mode, without any problems. I can tftp as much data as I want in U-Boot, and I've transferred many, many gigabytes without issue in Linux. The problem is only with 1000mbit mode.

I read that the same problem happens in Linux when running at 1000mbit. Is this understanding correct?

Yes. Both U-Boot and Linux exhibit problems at 1000mbit. Both work without errors at 100mbit.

Quoted 3 lines

Does running with root file system mounted over NFS work on this board? Does it work at 1000mbit, too?

100mbit: yes
1000mbit: no

If I boot up the board with the ethernet cable in a 100mbit port, then move the cable to a 1000mbit port, the NFS code says "server not responding" in <10 seconds.

Quoted 2 lines

Did you try and re-check the memory controller initialization?

See Dave's comments in reply your message. I've just re-checked that the memory configuration I expect is in the correct registers.

Thanks,
Ira

#9 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

What is the ACR register settings?
What is the freq of core/csb/DDR and TSEC block?

Thanks, Dave

#10 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

In reply to Ira Snyder
Quoted 2 lines

What is the ACR register settings?
What is the freq of core/csb/DDR and TSEC block?

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

author
#11 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 1 line · On Wed, Jun 03, 2009 at 04:44:54AM +0800, Liu Dave-R63238 wrote:
Quoted 1 line

What is the ACR register settings?

=> md e0000800 1
e0000800: 00030300 ....

Quoted 1 line
Quoted 1 line

What is the freq of core/csb/DDR and TSEC block?

According to U-Boot's clocks command:
=> clocks
Clock configuration:
Core: 533.333 MHz
Coherent System Bus: 266.667 MHz
Local Bus Controller:266.667 MHz

  Local Bus:           33.333 MHz
  DDR:                 266.667 MHz
  SEC:                 88.889 MHz
  I2C1:                266.667 MHz
  I2C2:                266.667 MHz
  TSEC1:               266.667 MHz
  TSEC2:               266.667 MHz
  USB DR:              88.889 MHz
  USB MPH:             88.889 MHz
Quoted 4 lines

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same setting in that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

I'll go find out why this is set in the MPC8349EMDS configuration as well. It shouldn't be.

Thanks,
Ira

#12 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/corruption)

Did you check the GMII interface (H/W)?
Like EC_GTX_CLK125, this signal is input for TSEC block for 1000Mbps case.
When you used the 100Mbps, the GTX_CLK125 is not used. Because you are Using the MII interface for 100Mbps case.

Thanks, Dave

author
#13 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

In reply to Ira Snyder
Quoted 5 lines · Ira Snyder
Quoted 4 lines

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

Quoted 7 lines

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same setting in that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

#define CONFIG_SYS_SICRH SICRH_TSOBI1

This seems wrong for the MPC8349EMDS board. I tried setting the register value to 0x0 by hand on my MPC8349EMDS eval board, and the network still works as expected.

Is this a bug in the MPC8349EMDS config?

Thanks,
Ira

#14 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/corruption)

Quoted 15 lines
Quoted 4 lines

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same

setting in

Quoted 3 lines

that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

#define CONFIG_SYS_SICRH SICRH_TSOBI1

Quoted 7 lines

This seems wrong for the MPC8349EMDS board. I tried setting the register
value to 0x0 by hand on my MPC8349EMDS eval board, and the network still
works as expected.

Is this a bug in the MPC8349EMDS config?

I will roll back the history to have a check it.

#15 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

In reply to Ira Snyder
Quoted 18 lines · On Tue, 2 Jun 2009 15:19:18 -0700 Ira Snyder <[email protected]> wrote:
Quoted 5 lines · Ira Snyder
Quoted 4 lines

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

Quoted 7 lines

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same setting in that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

Quoted 5 lines

#define CONFIG_SYS_SICRH SICRH_TSOBI1

This seems wrong for the MPC8349EMDS board. I tried setting the register value to 0x0 by hand on my MPC8349EMDS eval board, and the network still works as expected.

still works or does not work? 100mbit or 1000mbit??

Quoted 1 line

Is this a bug in the MPC8349EMDS config?

It might be. Another thing that changed wrt that area was commit 846f1574. Assuming this is all still clear to me, perhaps what that patch does should be made 8349-revision dependent?

Kim

author
#16 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 21 lines · Kim Phillips
Quoted 18 lines · On Tue, 2 Jun 2009 15:19:18 -0700 Ira Snyder <[email protected]> wrote:
Quoted 5 lines · Ira Snyder
Quoted 4 lines

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

Quoted 7 lines

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same setting in that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

Quoted 8 lines
Quoted 5 lines

#define CONFIG_SYS_SICRH SICRH_TSOBI1

This seems wrong for the MPC8349EMDS board. I tried setting the register value to 0x0 by hand on my MPC8349EMDS eval board, and the network still works as expected.

still works or does not work? 100mbit or 1000mbit??

Network works, both 100 and 1000 mbit.

Quoted 6 lines
Quoted 1 line

Is this a bug in the MPC8349EMDS config?

It might be. Another thing that changed wrt that area was commit 846f1574. Assuming this is all still clear to me, perhaps what that patch does should be made 8349-revision dependent?

My MPC8349EMDS reference manual says that bits 28-29 should be preserved, which is exactly what the commit message states.

I don't see a problem with commit 846f1574 at all. The bug is that SICRH is set to SICRH_TSOBI1 (0x2), which changes the TSEC1 output buffer parameters to RGMII/RTBI mode.

The MPC8349EMDS's PHY's are both in GMII mode, the do not use RGMII/RTBI or any of the other reduced pin count interfaces. Therefore, I think the bit should be cleared, and SICRH should be set to 0x0.

Even if I'm wrong, then the bits for both TSEC1 and TSEC2 should be the same. Right now, TSEC1 has the RGMII/RTBI output parameters, and TSEC2 has the GMII output parameters.

In practice, this doesn't seem to make a difference on the MPC8349EMDS eval board. Both settings work without any errors. For people like me, who are copying an existing board port to a similar board, it would be nice if it was correct.

Thanks,
Ira

#17 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 10 lines · On Wed, 3 Jun 2009 10:50:25 -0700 Ira Snyder <[email protected]> wrote:
Quoted 9 lines · Kim Phillips
Quoted 6 lines · On Tue, 2 Jun 2009 15:19:18 -0700 Ira Snyder <[email protected]> wrote:
Quoted 5 lines · Ira Snyder
Quoted 4 lines

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same setting in that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

ok this...

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

#define CONFIG_SYS_SICRH SICRH_TSOBI1

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

This seems wrong for the MPC8349EMDS board. I tried setting the register value to 0x0 by hand on my MPC8349EMDS eval board, and the network still works as expected.

still works or does not work? 100mbit or 1000mbit??

Network works, both 100 and 1000 mbit.

and this seeming contradiction lead to my source of confusion.

What I think is going on is that you're setting SICRH to 0 on your custom board and it still has its network problems, and setting it to 0 on the original MPC8349E-MDS board makes no difference - networking works with it set to both 0 and 2. ok.

Quoted 7 lines
Quoted 6 lines
Quoted 1 line

Is this a bug in the MPC8349EMDS config?

It might be. Another thing that changed wrt that area was commit 846f1574. Assuming this is all still clear to me, perhaps what that patch does should be made 8349-revision dependent?

Quoted 6 lines

My MPC8349EMDS reference manual says that bits 28-29 should be preserved, which is exactly what the commit message states.

I don't see a problem with commit 846f1574 at all. The bug is that SICRH is set to SICRH_TSOBI1 (0x2), which changes the TSEC1 output buffer parameters to RGMII/RTBI mode.

yeah my bad, I miscalculated which bits that commit was masking.

Quoted 3 lines

The MPC8349EMDS's PHY's are both in GMII mode, the do not use RGMII/RTBI or any of the other reduced pin count interfaces. Therefore, I think the bit should be cleared, and SICRH should be set to 0x0.

technically BCSR5 holds the reduced mode MII setting (set via SW4). The default is GMII, but, as in the case of the 8360 MDS, networking didn't work out of the box with the default switch settings.

Quoted 8 lines

Even if I'm wrong, then the bits for both TSEC1 and TSEC2 should be the same. Right now, TSEC1 has the RGMII/RTBI output parameters, and TSEC2 has the GMII output parameters.

In practice, this doesn't seem to make a difference on the MPC8349EMDS eval board. Both settings work without any errors. For people like me, who are copying an existing board port to a similar board, it would be nice if it was correct.

granted. I'm betting the sbc8349 and tqm834x just copied the setting from the MDS code, so those should get 0 too. Board maintainers, please test:

include/configs/MPC8349EMDS.h +1 -1
index 3c57403..2d2799e 100644
@@ -598,7 +598,7 @@
 #define CONFIG_SYS_SCCR_TSEC2CM        1       /* TSEC2 & I2C0 clock mode (0-3)
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT   0x000000000
include/configs/TQM834x.h +1 -1
index 5ca8720..5510730 100644
@@ -393,7 +393,7 @@ extern int tqm834x_num_flash_banks;
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH       SICRH_TSOBI1
+#define CONFIG_SYS_SICRH       0
 #define CONFIG_SYS_SICRL       SICRL_LDP_A
 
 /* i-cache and d-cache disabled */
include/configs/sbc8349.h +1 -1
index d0338f1..edd928d 100644
@@ -519,7 +519,7 @@
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT   0x000000000

Thanks,

Kim

author
#18 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 37 lines · Kim Phillips
Quoted 10 lines · On Wed, 3 Jun 2009 10:50:25 -0700 Ira Snyder <[email protected]> wrote:
Quoted 9 lines · Kim Phillips
Quoted 6 lines · On Tue, 2 Jun 2009 15:19:18 -0700 Ira Snyder <[email protected]> wrote:
Quoted 5 lines · Ira Snyder
Quoted 4 lines

And what is the SICRH[30-31]?
Did you have the matching settings for GMII with 3.3V?

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

=> md e0000118 1
e0000118: 00000002 ....

This looks wrong. The MPC8349EMDS board has the exact same setting in that register. Writing 0x0 to the SICRH register did not cause the problem to go away.

ok this...

Quoted 15 lines
Quoted 14 lines
Quoted 11 lines
Quoted 1 line

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

#define CONFIG_SYS_SICRH SICRH_TSOBI1

This seems wrong for the MPC8349EMDS board. I tried setting the register value to 0x0 by hand on my MPC8349EMDS eval board, and the network still works as expected.

still works or does not work? 100mbit or 1000mbit??

Quoted 9 lines
Quoted 1 line

Network works, both 100 and 1000 mbit.

and this seeming contradiction lead to my source of confusion.

What I think is going on is that you're setting SICRH to 0 on your custom board and it still has its network problems, and setting it to 0 on the original MPC8349E-MDS board makes no difference - networking works with it set to both 0 and 2. ok.

Yes, that is exactly correct. Sorry for my poor choice of words earlier.

Quoted 42 lines
Quoted 7 lines
Quoted 6 lines
Quoted 1 line

Is this a bug in the MPC8349EMDS config?

It might be. Another thing that changed wrt that area was commit 846f1574. Assuming this is all still clear to me, perhaps what that patch does should be made 8349-revision dependent?

Quoted 6 lines

My MPC8349EMDS reference manual says that bits 28-29 should be preserved, which is exactly what the commit message states.

I don't see a problem with commit 846f1574 at all. The bug is that SICRH is set to SICRH_TSOBI1 (0x2), which changes the TSEC1 output buffer parameters to RGMII/RTBI mode.

yeah my bad, I miscalculated which bits that commit was masking.

Quoted 3 lines

The MPC8349EMDS's PHY's are both in GMII mode, the do not use RGMII/RTBI or any of the other reduced pin count interfaces. Therefore, I think the bit should be cleared, and SICRH should be set to 0x0.

technically BCSR5 holds the reduced mode MII setting (set via SW4). The default is GMII, but, as in the case of the 8360 MDS, networking didn't work out of the box with the default switch settings.

Quoted 8 lines

Even if I'm wrong, then the bits for both TSEC1 and TSEC2 should be the same. Right now, TSEC1 has the RGMII/RTBI output parameters, and TSEC2 has the GMII output parameters.

In practice, this doesn't seem to make a difference on the MPC8349EMDS eval board. Both settings work without any errors. For people like me, who are copying an existing board port to a similar board, it would be nice if it was correct.

granted. I'm betting the sbc8349 and tqm834x just copied the setting from the MDS code, so those should get 0 too. Board maintainers, please test:

include/configs/MPC8349EMDS.h +0 -0
index 3c57403..2d2799e 100644

include/configs/MPC8349EMDS.h line 604

Quoted 8 lines
include/configs/MPC8349EMDS.h +1 -1
@@ -598,7 +598,7 @@
 #define CONFIG_SYS_SCCR_TSEC2CM        1       /* TSEC2 & I2C0 clock mode (0-3)
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT   0x000000000

I can ACK this board. I've run the following combinations on my MPC8349EMDS eval board. All of the configurations worked perfectly.

U-Boot, PCI Host mode, 1000 mbit
U-Boot, PCI Host mode, 100 mbit
U-Boot, PCI Agent mode, 1000 mbit
U-Boot, PCI Agent mode, 100 mbit

Linux,  PCI Host mode,  1000 mbit
Linux,  PCI Host mode,  100  mbit

Thanks for looking at this!
Ira

#19 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

In reply to Kim Phillips
Quoted 2 lines · Kim Phillips

On Wed, 3 Jun 2009 10:50:25 -0700
Ira Snyder <[email protected]> wrote:

...

Quoted 7 lines
Quoted 5 lines

In practice, this doesn't seem to make a difference on the MPC8349EMDS eval board. Both settings work without any errors. For people like me, who are copying an existing board port to a similar board, it would be nice if it was correct.

granted. I'm betting the sbc8349 and tqm834x just copied the setting

You would be correct (at least for the sbc).

Quoted 2 lines

from the MDS code, so those should get 0 too. Board maintainers, please test:

I won't be able to test this on an sbc until later this week, but don't let that stop you from queueing it up; I suspect it will just be a formality.

Thanks,
Paul.

Quoted 17 lines
include/configs/sbc8349.h +1 -1
index d0338f1..edd928d 100644
@@ -519,7 +519,7 @@
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT   0x000000000

Thanks,

Kim

Tested-by
#20 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 14 lines · Paul Gortmaker
Quoted 2 lines · Kim Phillips

On Wed, 3 Jun 2009 10:50:25 -0700
Ira Snyder <[email protected]> wrote:

...

Quoted 7 lines
Quoted 5 lines

In practice, this doesn't seem to make a difference on the MPC8349EMDS eval board. Both settings work without any errors. For people like me, who are copying an existing board port to a similar board, it would be nice if it was correct.

granted. I'm betting the sbc8349 and tqm834x just copied the setting

Quoted 8 lines

You would be correct (at least for the sbc).

Quoted 2 lines

from the MDS code, so those should get 0 too. Board maintainers, please test:

I won't be able to test this on an sbc until later this week, but don't let that stop you from queueing it up; I suspect it will just be a formality.

I applied the chunk below onto a checkout of v2009.06-rc2 and it works fine on sbc8349; tested both TSEC and at both 100 and 1GB.

-------
U-Boot 2009.06-rc2-dirty (Jun 05 2009 - 14:11:18) MPC83XX

Reset Status: Software Hard, External/Internal Soft, External/Internal Hard

CPU:   e300c1, MPC8349E, Rev: 1.1 at 396 MHz, CSB: 264 MHz                      
Board: Wind River SBC834x                                                       
I2C:   ready                                                                    
DRAM:     SDRAM on Local Bus: Disabled in config                                
256 MB (DDR1, 64-bit, ECC off, 264 MHz)                                         
FLASH:  8 MB                                                                    
In:    serial                                                                   
Out:   serial                                                                   
Err:   serial                                                                   
Net:   TSEC0, TSEC1                                                             

=>
-------

Quoted 24 lines

Thanks,
Paul.

Quoted 17 lines
include/configs/sbc8349.h +1 -1
index d0338f1..edd928d 100644
@@ -519,7 +519,7 @@
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT   0x000000000

Thanks,

Kim

#21 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 4 lines · On Fri, 05 Jun 2009 14:45:38 -0400 Paul Gortmaker…

I applied the chunk below onto a checkout of v2009.06-rc2 and it works fine on sbc8349; tested both TSEC and at both 100 and 1GB.

Thanks Paul, Ira, and Dave.

I've applied the following on u-boot-mpc83xx master:

mpc83xx: fix tsec "Got error 4" errors - don't set SICRH_TSOBI1 to RMII/RTBI operation

While not making a difference on the MPC8349EMDS, changing the output buffer impedance on TSEC1 to 2.5V has negative effect on other mpc83xx based boards - they start dropping frame data consequently prompting TSEC errors and a general loss of 1000mbit networking.

U-Boot always operated this PHY interface in GMII mode. It is assumed this was missed in the clean up by the original board porters, and copied along to the TQM and sbc boards.

---
 include/configs/MPC8349EMDS.h |    2 +-
 include/configs/TQM834x.h     |    2 +-
 include/configs/sbc8349.h     |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
include/configs/MPC8349EMDS.h +1 -1
index 3c57403..2d2799e 100644
@@ -598,7 +598,7 @@
 #define CONFIG_SYS_SCCR_TSEC2CM	1	/* TSEC2 & I2C0 clock mode (0-3) */
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT	0x000000000
include/configs/TQM834x.h +1 -1
index 5ca8720..5510730 100644
@@ -393,7 +393,7 @@ extern int tqm834x_num_flash_banks;
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH	SICRH_TSOBI1
+#define CONFIG_SYS_SICRH	0
 #define CONFIG_SYS_SICRL	SICRL_LDP_A
 
 /* i-cache and d-cache disabled */
include/configs/sbc8349.h +1 -1
index d0338f1..edd928d 100644
@@ -519,7 +519,7 @@
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT	0x000000000
Signature · 1 line
1.6.3.1
#22 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Hi Kim,

The bit setting did not help with our errors at 1Gbit, so the comment:

Quoted 6 lines

While not making a difference on the MPC8349EMDS, changing the output buffer impedance on TSEC1 to 2.5V has negative effect on other mpc83xx based boards - they start dropping frame data consequently prompting TSEC errors and a general loss of 1000mbit networking.

May not actually be relevant, this was more of a cleanup to make the source consistent with the user manual.

Perhaps something like the following comment:

In GMII mode (which operates at 3.3V) both SICRH TSEC1/2 output buffer impedance bits should be clear, i.e., SICRH[TSIOB1] = 0 and SICRH[TSIOB2] = 0. SICRH[TSIOB1] was erroneously being set high.

As far as debugging our error goes, we're going to try a different clock source on the PHY. Currently its 125MHz, but the PHY can use 25MHz. We'll put a synthesizer on the pin, adjust the operating frequency around nominal, and see if we can make our problem with 1Gbit mode worse, so we can figure our what our real problem is.

Cheers,
Dave

#23 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 2 lines · On Fri, 5 Jun 2009 19:31:06 -0700 (PDT) dwh at ovro.caltech.edu wrote:

The bit setting did not help with our errors at
1Gbit, so the comment:

fixed and pushed onto mpc83xx master. regenerated patch below.

thanks for the clarification David.

Kim

mpc83xx: don't set SICRH_TSOBI1 to RMII/RTBI operation

In GMII mode (which operates at 3.3V) both SICRH TSEC1/2 output buffer impedance bits should be clear, i.e., SICRH[TSIOB1] = 0 and SICRH[TSIOB2] = 0. SICRH[TSIOB1] was erroneously being set high.

U-Boot always operated this PHY interface in GMII mode. It is assumed this was missed in the clean up by the original board porters, and copied along to the TQM and sbc boards.

---
 include/configs/MPC8349EMDS.h |    2 +-
 include/configs/TQM834x.h     |    2 +-
 include/configs/sbc8349.h     |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
include/configs/MPC8349EMDS.h +1 -1
index 3c57403..2d2799e 100644
@@ -598,7 +598,7 @@
 #define CONFIG_SYS_SCCR_TSEC2CM	1	/* TSEC2 & I2C0 clock mode (0-3) */
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT	0x000000000
include/configs/TQM834x.h +1 -1
index 5ca8720..5510730 100644
@@ -393,7 +393,7 @@ extern int tqm834x_num_flash_banks;
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH	SICRH_TSOBI1
+#define CONFIG_SYS_SICRH	0
 #define CONFIG_SYS_SICRL	SICRL_LDP_A
 
 /* i-cache and d-cache disabled */
include/configs/sbc8349.h +1 -1
index d0338f1..edd928d 100644
@@ -519,7 +519,7 @@
 #endif
 
 /* System IO Config */
-#define CONFIG_SYS_SICRH SICRH_TSOBI1
+#define CONFIG_SYS_SICRH 0
 #define CONFIG_SYS_SICRL SICRL_LDP_A
 
 #define CONFIG_SYS_HID0_INIT	0x000000000
Signature · 1 line
1.6.3.1
#24 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Hi all,

Over the weekend we performed some hardware modifications on our boards with their troublesome gigabit interfaces.

We performed a couple of tests, one of which may be the solution to our problem.

1) Modified the value of the series termination of EC_GTX_CLK125

The PHY drives a 125MHz clock signal to the PowerPC (EC_GTX_CLK125 signal) when operating in gigabit ethernet mode.

Our schematic used a 33-Ohm series termination on this 125MHz signal, but we had used 22-Ohms for all other series terminations. The MDS board also uses 22-Ohms series terminations.

We replaced the 33-ohm termination with 22-ohms ... and things started working. We looked at the 125MHz signal at the destination pin on the BGA of the PowerPC using a high-bandwidth scope (2GHz LeCroy) for both 22-Ohms termination and 33-Ohms, and there is a slight difference in waveform shape, but no obvious ringing, or steps in the clock edges that might suggest transmission line reflections.

The fact that we only ever observed errors in gigabit mode, that the PHY never reports any errors, that EC_GTX_CLK125 is only used in gigabit mode, and that the data errors looked like a FIFO clocking issue, is consistent with the error being related to the quanlity of the EC_GTX_CLK125 signal.

I'm not ready to claim success, as I need to make the change on other boards ... but its very encouraging!

2) Frequency accuracy testing

We replaced the PHY 125MHz clock source (an onboard oscillator) with a frequency synthesizer driven into a buffer (to create a 3.3V logic level clock).

The frequency was adjusted to 124.90MHz, 124.95MHz, 125.00MHz, 125.05MHz, and 125.10MHz. Errors were generated for the +/-0.1MHz cases, but things worked fine for the +/-0.05MHz offsets.

So the PHY is tolerant of frequency errors in the oscillator.

(This test was performed after the series termination
had been changed).

Ira is going to boot the working board with Linux and perform some network stress tests.

I'll modify some other boards, and we'll tests those too.

Cheers,
Dave

#25 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Quoted 33 lines

Over the weekend we performed some hardware modifications on our boards with their troublesome gigabit interfaces.

We performed a couple of tests, one of which may be the solution to our problem.

1) Modified the value of the series termination of EC_GTX_CLK125

The PHY drives a 125MHz clock signal to the PowerPC (EC_GTX_CLK125 signal) when operating in gigabit ethernet mode.

Our schematic used a 33-Ohm series termination on this 125MHz signal, but we had used 22-Ohms for all other series terminations. The MDS board also uses 22-Ohms series terminations.

We replaced the 33-ohm termination with 22-ohms ... and things started working. We looked at the 125MHz signal at the destination pin on the BGA of the PowerPC using a high-bandwidth scope (2GHz LeCroy) for both 22-Ohms termination and 33-Ohms, and there is a slight difference in waveform shape, but no obvious ringing, or steps in the clock edges that might suggest transmission line reflections.

The fact that we only ever observed errors in gigabit mode, that the PHY never reports any errors, that EC_GTX_CLK125 is only used in gigabit mode, and that the data errors looked like a FIFO clocking issue, is consistent with the error being related to the quanlity of the EC_GTX_CLK125 signal.

Quoted 2 lines

I'm not ready to claim success, as I need to make the change on other boards ... but its very encouraging!

Good news, Dave.
Sure, we need to keep the imdependence matching. change the series resister.
Anyway, I strong recommand you to check if the signal of GMII interface meet the H/W spec of 8349EA.

Quoted 29 lines

2) Frequency accuracy testing

We replaced the PHY 125MHz clock source (an onboard oscillator) with a frequency synthesizer driven into a buffer (to create a 3.3V logic level clock).

The frequency was adjusted to 124.90MHz, 124.95MHz, 125.00MHz, 125.05MHz, and 125.10MHz. Errors were generated for the +/-0.1MHz cases, but things worked fine for the +/-0.05MHz offsets.

So the PHY is tolerant of frequency errors in the oscillator.

(This test was performed after the series termination
had been changed).

Ira is going to boot the working board with Linux and perform some network stress tests.

I'll modify some other boards, and we'll tests those too.

Cheers,
Dave

#26 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Hi Dave,

Quoted 8 lines
Quoted 2 lines

I'm not ready to claim success, as I need to make the change on other boards ... but its very encouraging!

Good news, Dave.
Sure, we need to keep the impedance matching. change the series resister.
Anyway, I strong recommand you to check if the signal of GMII interface meet the H/W spec of 8349EA.

I modified a second board, and it wasn't quite as successful.

The signal levels from the PHY all appear to be within spec for the MPC8349EA.

Unfortunately we can not probe the pins on the PowerPC BGA on the MDS board to compare with our board. The MDS board has the processor socket housing covering the BGA pins.

We'll continue modifying and testing ... we'll track it down eventually :)

Cheers,
Dave

#27 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Hi all,

I modified the MPC8349E-MDS board and can create the same problem we were experiencing on our boards; CRC errors and data packets with received bytes repeated within the receive packets.

So I am certain we have tracked the root cause of the problem we were having with our boards.

The problem was related to the EC_GTX_CLK125MHZ signal being sent from the PHY to the PowerPC processor. The termination resistance on our board was too large (33-Ohms). Reducing the termination resistor value increased the signal swing, and produced a better looking clock at the PowerPC BGA pin; a 10-Ohm source resistor was required in our case. Linux network stress tests now pass.

The modification to break the MDS board was simple; I increased the CLK125MHz source termination to 50-Ohms (from its nominal 22-Ohms).
The MDS board had another error; there is a jumper on the board for selecting the supply rail for 2.5V GETH (JS2 and JS3 on the MPC8349E-MDS-PB schematic) as either 3.3V for GMII, or 2.5V for RGMII. The jumper on our board was in the 2.5V location, whereas it should be on 3.3V. Changing this jumper made no real difference.

This testing revealed some interesting observations;

1) The Marvell 88E1111 PHY generates a 125MHz output clock that is used as the PowerPC EC_GTX_CLK125MHZ clock source on the MDS board.

The MDS board has to use the PHY output as the 125MHz clock source to the PowerPC, as the PHY is clocked using a 25MHz oscillator, so there is no alternative source of 125MHz on the board.

However, the PHY 125MHz output has a *huge* amount of duty cycle variation depending on whether the PHY has negotiated as a *master* (clock looks good), or as a *slave* (horrible looking clock).

When the PHY on the MDS board, or our board, negotiates the 1Gbit link as a *slave*, observing the 125MHz output clock with an oscilloscope triggered on the rising edge of the clock, there is about 1ns of variation in the timing of the falling edge.

Yikes!!!

Has anyone else been disturbed by this???

I know its not just on our board, as I can see it on the MDS board too.

2) The MPC8349EA hardware specification indicates a 45% to 55% duty cycle requirement on the EC_GTX_CLK125MHZ clock signal. The duty cycle is measured at LVDD/2 = 1.65V for GMII operating at LVDD = 3.3V.

With the PHY generating the 125MHz clock, and the PHY operated at 2.5V, the duty cycle specification is *not* met by the MDS board - especially when the PHY has negotiated slave mode and has a huge amount of jitter on it.

This seems like a minor design error,
in that a 3.3V buffer could have been used on the PHY output clock to ensure 3.3V levels at the PowerPC.

Anyone out there care to comment how this has been implemented on other boards?

Our board has a 3.3V 125MHz oscillator as the PHY source, and clock source for other devices on the board. I need to make some changes to our PCB, so I'll also route an option for a 3.3V 125MHz clock to the PowerPC pin (I'll put a 3.3V buffer on the board that can be driven by the oscillator directly, or by the PHY output).

I'd be interested in hearing anyone else's experience with the 125MHz Gigabit PHY clock.

And for those finding this email in the archive, I hope this helps you :)

Cheers,
Dave

#28 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Hi Dave,

Good news, Good summary!

Quoted 44 lines

I modified the MPC8349E-MDS board and can create the same problem we were experiencing on our boards; CRC errors and data packets with received bytes repeated within the receive packets.

So I am certain we have tracked the root cause of the problem we were having with our boards.

The problem was related to the EC_GTX_CLK125MHZ signal being sent from the PHY to the PowerPC processor. The termination resistance on our board was too large (33-Ohms). Reducing the termination resistor value increased the signal swing, and produced a better looking clock at the PowerPC BGA pin; a 10-Ohm source resistor was required in our case. Linux network stress tests now pass.

The modification to break the MDS board was simple; I increased the CLK125MHz source termination to 50-Ohms (from its nominal 22-Ohms).
The MDS board had another error; there is a jumper on the board for selecting the supply rail for 2.5V GETH (JS2 and JS3 on the MPC8349E-MDS-PB schematic) as either 3.3V for GMII, or 2.5V for RGMII. The jumper on our board was in the 2.5V location, whereas it should be on 3.3V. Changing this jumper made no real difference.

This testing revealed some interesting observations;

1) The Marvell 88E1111 PHY generates a 125MHz output clock that is used as the PowerPC EC_GTX_CLK125MHZ clock source on the MDS board.

The MDS board has to use the PHY output as the 125MHz clock source to the PowerPC, as the PHY is clocked using a 25MHz oscillator, so there is no alternative source of 125MHz on the board.

However, the PHY 125MHz output has a *huge* amount of duty cycle variation depending on whether the PHY has negotiated as a *master* (clock looks good), or as a *slave* (horrible looking clock).

Quoted 6 lines

When the PHY on the MDS board, or our board, negotiates the 1Gbit link as a *slave*, observing the 125MHz output clock with an oscilloscope triggered on the rising edge of the clock, there is about 1ns of variation in the timing of the falling edge.

IIRC, The FPGA of MPC8349EA-MDS can control if we use the PHY as master. We were aware of this.

Quoted 52 lines

Yikes!!!

Has anyone else been disturbed by this???

I know its not just on our board, as I can see it on the MDS board too.

2) The MPC8349EA hardware specification indicates a 45% to 55% duty cycle requirement on the EC_GTX_CLK125MHZ clock signal. The duty cycle is measured at LVDD/2 = 1.65V for GMII operating at LVDD = 3.3V.

With the PHY generating the 125MHz clock, and the PHY operated at 2.5V, the duty cycle specification is *not* met by the MDS board - especially when the PHY has negotiated slave mode and has a huge amount of jitter on it.

This seems like a minor design error,
in that a 3.3V buffer could have been used on the PHY output clock to ensure 3.3V levels at the PowerPC.

Anyone out there care to comment how this has been implemented on other boards?

Our board has a 3.3V 125MHz oscillator as the PHY source, and clock source for other devices on the board. I need to make some changes to our PCB, so I'll also route an option for a 3.3V 125MHz clock to the PowerPC pin (I'll put a 3.3V buffer on the board that can be driven by the oscillator directly, or by the PHY output).

I'd be interested in hearing anyone else's experience with the 125MHz Gigabit PHY clock.

And for those finding this email in the archive, I hope this helps you :)

Cheers,
Dave

#29 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/ corruption)

Hi Dave,

Quoted 1 line

Good news, Good summary!

Thanks!

Quoted 23 lines
Quoted 16 lines

This testing revealed some interesting observations;

1) The Marvell 88E1111 PHY generates a 125MHz output clock that is used as the PowerPC EC_GTX_CLK125MHZ clock source on the MDS board.

The MDS board has to use the PHY output as the 125MHz clock source to the PowerPC, as the PHY is clocked using a 25MHz oscillator, so there is no alternative source of 125MHz on the board.

However, the PHY 125MHz output has a *huge* amount of duty cycle variation depending on whether the PHY has negotiated as a *master* (clock looks good), or as a *slave* (horrible looking clock).

Quoted 6 lines

When the PHY on the MDS board, or our board, negotiates the 1Gbit link as a *slave*, observing the 125MHz output clock with an oscilloscope triggered on the rising edge of the clock, there is about 1ns of variation in the timing of the falling edge.

Quoted 2 lines

IIRC, The FPGA of MPC8349EA-MDS can control if we use the PHY as master. We were aware of this.

Ok at least someone else has seen it!

Of course if Freescale had seen this, its a shame they did not put a warning in the MDS documentation. However, its really the Marvell data sheet that should have information on this feature!

Our board layout is such that it'll be a fairly easy fix to add both a 3.3V buffer and the option to use a 125MHz oscillator directly, or the PHY 125MHz output.

I'm just glad to figure out what was happening.

Cheers,
Dave

#30 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/corruption)

In reply to Ira Snyder
Quoted 6 lines

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

#define CONFIG_SYS_SICRH SICRH_TSOBI1

Quoted 7 lines

This seems wrong for the MPC8349EMDS board. I tried setting the register
value to 0x0 by hand on my MPC8349EMDS eval board, and the network still
works as expected.

Is this a bug in the MPC8349EMDS config?

IIRC, the MPC8349EMDS has one configuration LVDD for TSEC1/2 MAC-PHY interface,
That is jumpers to choose 2.5v or 3.3v . It is possible we leave the mess in the code
at that time.

Please fix the bug according to real fact.

Thanks, Dave

#31 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/corruption)

Hi Dave,

Quoted 14 lines
Quoted 6 lines

The MPC8349EMDS config has had that setting since it was imported into U-Boot. I've copied the relevant part of include/configs/MPC8349EMDS.h below.

#define CONFIG_SYS_SICRH SICRH_TSOBI1

Quoted 7 lines

This seems wrong for the MPC8349EMDS board. I tried setting the register
value to 0x0 by hand on my MPC8349EMDS eval board, and the network still
works as expected.

Is this a bug in the MPC8349EMDS config?

Quoted 7 lines

IIRC, the MPC8349EMDS has one configuration LVDD for TSEC1/2 MAC-PHY interface,
That is jumpers to choose 2.5v or 3.3v . It is possible we leave the mess in the code
at that time.

Please fix the bug according to real fact.

The MPC8349EA LVDD power is supposed to be 3.3V for GMII and 2.5V for RGMII. As you comment, the evaluation board has a jumper to select the power rail. By default the board is setup for GMII with LVDD at 3.3V, so the TSEC impedance bits should be left low.

The patch by Kim is correct for the MPC8349EA-MDS nominal board configuration.

Cheers,
Dave

#32 lore raw

[U-Boot] TSEC ethernet controller problems (crc errors/corruption)

In reply to Ira Snyder
Quoted 5 lines

Did you check the GMII interface (H/W)?
Like EC_GTX_CLK125, this signal is input for TSEC block for 1000Mbps case.
When you used the 100Mbps, the GTX_CLK125 is not used. Because you are Using the MII interface for 100Mbps case.

IIRC, the MV88E1111 has some h/w config option to configure something such
like interface mode, enable/disable 125MHz clock...if you are using the 125MHz of PHY acts as source of GTX_CLK125. you need care it.