Add CONFIG_GMAC_TX_DELAY=4 for OlinuXino Lime2

Quoted 8 lines · Michael Haas

This change is required to get GBIT Ethernet to work reliably on my board. Without CONFIG_GMAC_TX_DELAY=4, the connection suffers severe packet loss and SSH becomes unusable.

--- 192.168.1.1 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 19842ms rtt min/avg/max/mdev = 0.260/0.334/0.527/0.038 ms

Quoted 2 lines

I have also tried CONFIG_GMAC_TX_DELAY=3, which still yielded 11-13% packet loss.

Hello,

unfortunately this change doesn't solve the gigabit issues on my LIME2 - it even seems to make them worse on my board.

Quoted 10 lines · Michael Haas

This change is required to get GBIT Ethernet to work reliably on my board. Without CONFIG_GMAC_TX_DELAY=4, the connection suffers severe packet loss and SSH becomes unusable.

--- 192.168.1.1 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 19842ms rtt min/avg/max/mdev = 0.260/0.334/0.527/0.038 ms

I have also tried CONFIG_GMAC_TX_DELAY=3, which still yielded 11-13% packet loss.

Hello,

unfortunately this change doesn't solve the gigabit issues on my LIME2 - it even seems to make them worse on my board.

Could you perhaps try netbooting a kernel with your LIME2? I would be interested in knowing whether that works on your board (with CONFIG_GMAC_TX_DELAY=0/4).

Quoted 14 lines · Karsten Merker

Quoted 10 lines · Michael Haas

This change is required to get GBIT Ethernet to work reliably on my board. Without CONFIG_GMAC_TX_DELAY=4, the connection suffers severe packet loss and SSH becomes unusable.

--- 192.168.1.1 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 19842ms rtt min/avg/max/mdev = 0.260/0.334/0.527/0.038 ms

I have also tried CONFIG_GMAC_TX_DELAY=3, which still yielded 11-13% packet loss.

Hello,

Quoted 9 lines

unfortunately this change doesn't solve the gigabit issues on my LIME2 - it even seems to make them worse on my board.

Could you perhaps try netbooting a kernel with your LIME2? I would be interested in knowing whether that works on your board (with CONFIG_GMAC_TX_DELAY=0/4).

Hello Karsten,

I just tried netbooting the debian installer: it didn't work, just as you said.

Even worse: I noticed I had not installed my patched u-boot at all when I ran the ping test above! With my patch, the kernel would not download at all.

So the patch is wrong and I'm sorry for having wasted everyone's time here.

I won't give up yet, however. The fact still stands that at some point, I was able to use GBit Ethernet
without any issues in Linux, possibly after a warm reboot.

For (my) future reference, I used these commands:

--
setenv bootargs console=ttyS0,115200 rootwait panic=10 setenv autoload no
dhcp
setenv serverip 192.168.1.170
tftpboot ${scriptaddr} /debian-installer/armhf/tftpboot.scr source ${scriptaddr}
--

I'll be playing around with the value a bit more to find the "best" value for netbooting from a cold state, then try to get it to work within Linux itself. Perhaps there are also some clock adjusts to be done from within the DTS.

Sorry again!

Michael

Hi,

Quoted 33 lines · Michael Haas

Quoted 14 lines · Karsten Merker

Quoted 10 lines · Michael Haas

This change is required to get GBIT Ethernet to work reliably on my board. Without CONFIG_GMAC_TX_DELAY=4, the connection suffers severe packet loss and SSH becomes unusable.

--- 192.168.1.1 ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 19842ms rtt min/avg/max/mdev = 0.260/0.334/0.527/0.038 ms

I have also tried CONFIG_GMAC_TX_DELAY=3, which still yielded 11-13% packet loss.

Hello,

Quoted 9 lines

unfortunately this change doesn't solve the gigabit issues on my LIME2 - it even seems to make them worse on my board.

Could you perhaps try netbooting a kernel with your LIME2? I would be interested in knowing whether that works on your board (with CONFIG_GMAC_TX_DELAY=0/4).

Hello Karsten,

I just tried netbooting the debian installer: it didn't work, just as you said.

Even worse: I noticed I had not installed my patched u-boot at all when I ran the ping test above! With my patch, the kernel would not download at all.

Quoted 5 lines

So the patch is wrong and I'm sorry for having wasted everyone's time here.

I won't give up yet, however. The fact still stands that at some point, I was able to use GBit Ethernet
without any issues in Linux, possibly after a warm reboot.

Interesting, if you can reproduce things working better after a warm reboot, try dumping the reg which gets modified by modifying ONFIG_GMAC_TX_DELAY, note that reg has RX_DELAY bits too, maybe we need to tweak those.

Although a warm reboot typically only preserves settings outside of the soc, so if a warm reboot fixes things int may be that whatever you're warm rebooting from (I would assume some vendor image) is setting some things differently in the phy.

Quoted 23 lines

For (my) future reference, I used these commands:

--
setenv bootargs console=ttyS0,115200 rootwait panic=10 setenv autoload no
dhcp
setenv serverip 192.168.1.170
tftpboot ${scriptaddr} /debian-installer/armhf/tftpboot.scr source ${scriptaddr}
--

I'll be playing around with the value a bit more to find the "best" value for netbooting from a cold state, then try to get it to work within Linux itself. Perhaps there are also some clock adjusts to be done from within the DTS.

Sorry again!

Michael

Michael, if you would like to try testing higher RX delay chain values on the Lime2, I have attached a small patch that adds an appropriate Kconfig option. Regards, Karsten

Hello all,

Quoted 3 lines · Karsten Merker

Michael, if you would like to try testing higher RX delay chain values on the Lime2, I have attached a small patch that adds an appropriate Kconfig option. Regards, Karsten

I will be testing your patch along with Hans' patch. In fact, I have already compiled it - I'm just waiting for a new SD card reader to arrive tomorrow. Sorry for the delay.

In the mean time, as I was trying to get the device to boot at all (not related to u-boot, just my broken sdcard setup), I did try my old SD card again to load the debian netinstaller. This should be the version with the TX_DELAY set to 4.

Now I'm sitting in an ash shell and I can wget files from my desktop at around 100MB/s.

~ # wget http://192.168.1.170:8000/debian-8.3.0-i386-netinst.iso -O - | md5sum
Connecting to 192.168.1.170:8000 (192.168.1.170:8000)
- 100% |*******************************| 315M
0:00:00 ETA
37f329724a1c072cbe0ebda211fdeb88 -
~ # wget http://192.168.1.170:8000/debian-8.3.0-i386-netinst.iso -O - | md5sum
Connecting to 192.168.1.170:8000 (192.168.1.170:8000)
- 100% |*******************************| 315M
0:00:00 ETA
37f329724a1c072cbe0ebda211fdeb88 -

~ # date +%s; wget
http://192.168.1.170:8000/debian-8.3.0-i386-netinst.iso -O /d ev/null; date +%s
1258
Connecting to 192.168.1.170:8000 (192.168.1.170:8000)
null 100% |*******************************| 315M
0:00:00 ETA
1261

~ # dmesg | grep -i link

[    1.504918] audit: initializing netlink subsys (disabled)
[   88.561822] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   90.547368] sun7i-dwmac 1c50000.ethernet eth0: Link is Up -
1Gbps/Full - flow control rx/tx
[   90.547424] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

~ # cat /etc/lsb-release
DISTRIB_ID=Debian
DISTRIB_DESCRIPTION="Debian GNU/Linux installer" DISTRIB_RELEASE="9 (stretch) - installer build 20160316-00:13" X_INSTALLATION_MEDIUM=netboot
~ # uname -a
Linux debian 4.4.0-1-armmp #1 SMP Debian 4.4.4-2 (2016-03-09) armv7l GNU/Linux

It takes around three seconds (date +%s; time not set) to transfer the 315MB. The md5sum matches.

Now, with the device sitting on my desk in a debian installer session and working gbit.. how do I proceed from here? I don't dare rebooting..

Best

Michael

Hi,

There are 3 things which would be interesting to save and compare with a boot which does have the problem:

1) The gmc clk register, to do this download mmio_dump from:

2) The pmic settings for various ldo-s, etc. as root run:

i2cdump -f -y 0 0x34
And record the output. You may need to
install i2c-tools (or the deb. alternative) and
do "modprobe i2c_dev"

3) The phy settings, unfortunately I do not know of a way to dump these, so lets just forget about these and focus on the other 2.

Quoted 8 lines · Hans de Goede

Hi,

There are 3 things which would be interesting to save and compare with a boot which does have the problem:

1) The gmc clk register, to do this download mmio_dump from:

Working GBit:
01c20164: 00000c06

Broken GBit:
tomorrow :)

Quoted 8 lines

2) The pmic settings for various ldo-s, etc. as root run:

i2cdump -f -y 0 0x34
And record the output. You may need to
install i2c-tools (or the deb. alternative) and
do "modprobe i2c_dev"

I had to download the Debian kernel package on another box, extract the .ko, transfer and insmod it. Now I'm getting:

~ # /mnt/usr/sbin/i2cdump -f -y 0 0x34
No size specified (using byte-data access)
Error: Could not open file `/dev/i2c-0': No such device

I did mknod the devices, but I'm getting the same error msg:

~ # ls -al /dev/i2c-*

crw-r--r--    1 root     root       89,   0 Mar 17 21:31 /dev/i2c-0
crw-r--r--    1 root     root       89,   1 Mar 17 21:31 /dev/i2c-1

~ #

Module                  Size  Used by
i2c_dev                 7087  0
i2c_sun6i_p2wi          3664  0
dm_mod                 98419  0
md_mod                121938  0
jfs                   175313  0
crc32c_generic          1862  1
btrfs                1068047  0
xor                     4846  1 btrfs
zlib_deflate           20354  1 btrfs
raid6_pq               87885  1 btrfs
vfat                   10249  0
fat                    55086  1 vfat
ext4                  552524  2
crc16                   1274  1 ext4
mbcache                 9623  1 ext4
jbd2                   96157  1 ext4
usb_storage            45523  0
ahci_sunxi              2851  0
libahci_platform        6430  1 ahci_sunxi
libahci                23066  2 libahci_platform,ahci_sunxi
libata                182871  3 libahci,libahci_platform,ahci_sunxi
scsi_mod              188073  2 usb_storage,libata
realtek                 2711  0
dwmac_sunxi             2431  0
stmmac_platform         3302  1 dwmac_sunxi
stmmac                 74687  3 stmmac_platform,dwmac_sunxi
ohci_platform           4658  0
ohci_hcd               38045  1 ohci_platform
ehci_platform           5398  0
ehci_hcd               63999  1 ehci_platform
usbcore               190581  5
usb_storage,ohci_platform,ohci_hcd,ehci_hcd,ehci_platform
usb_common              3195  1 usbcore
phy_sun4i_usb           7858  8
extcon                 10436  1 phy_sun4i_usb
sunxi_mmc              11288  0
leds_gpio               3606  0

I also loaded i2c_sun6i_p2wi for good measure. Any idea what's missing here?

Quoted 5 lines

3) The phy settings, unfortunately I do not know of a way to dump these, so lets just forget about these and focus on the other 2.

I might even look at the data sheet tomorrow if time permits.

Michael

The installer supports a number of systems that can only load images of limited size, therefore the installer includes only an installation-related subset of all the kernel modules and i2c-mv64xxx currently isn't among those. This hasn't been a practical problem up to now, but being able to talk to the pmic is indeed something that should be possible even in the installer. I'll try to address that in the next days. Regards, Karsten

Quoted 7 lines · Karsten Merker

The installer supports a number of systems that can only load images of limited size, therefore the installer includes only an installation-related subset of all the kernel modules and i2c-mv64xxx currently isn't among those. This hasn't been a practical problem up to now, but being able to talk to the pmic is indeed something that should be possible even in the installer. I'll try to address that in the next days. Regards, Karsten

Just an anecdote: I loaded i2c-mv64xx.ko which I had extracted from the deb and the olinuxino locked up.

So now I'm trying to reproduce broken GBit, which is harder than expected. I managed to do it once right after the lockup in u-boot: tftp was slow and ran into timeouts. Then I typed 'reset' in the u-boot shell. u-boot started again and GBit was working again.

Weird.
_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

Quoted 7 lines · Karsten Merker

The installer supports a number of systems that can only load images of limited size, therefore the installer includes only an installation-related subset of all the kernel modules and i2c-mv64xxx currently isn't among those. This hasn't been a practical problem up to now, but being able to talk to the pmic is indeed something that should be possible even in the installer. I'll try to address that in the next days. Regards, Karsten

Just an anecdote: I loaded i2c-mv64xx.ko which I had extracted from the deb and the olinuxino locked up.

So now I'm trying to reproduce broken GBit, which is harder than expected. I managed to do it once right after the lockup in u-boot: tftp was slow and ran into timeouts. Then I typed 'reset' in the u-boot shell. u-boot started again and GBit was working again.

Weird.
_______________________________________________
U-Boot mailing list
U-Boot at lists.denx.de
http://lists.denx.de/mailman/listinfo/u-boot

There are 3 things which would be interesting to save and compare with a boot which does have the problem:

./mmio-dump 0x1c20164

2) The pmic settings for various ldo-s, etc. as root run:

i2cdump -f -y 0 0x34

3) The phy settings, unfortunately I do not know of a way to dump these, so lets just forget about these and focus on the other 2.

Regards,

Hans

Quoted 2 lines

3) The phy settings, unfortunately I do not know of a way to dump these, so lets just forget about these and focus on the other 2.

Hi,

As I?m banging my head on this too, please allow me to provide some details I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Hi,

Quoted 7 lines · Fran?ois-David Collin

Hi,

As I?m banging my head on this too, please allow me to provide some details
I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Michael Haas' work to debug this by looking at the phy registers seems to be the most promising so-far, the clk reg and axp209 registers all seem to be identical between good and bad setups.

Can you try to:

1) Stop the boot in u-boot (press a key on the serial console)
2) Bring up the network, e.g. type "dhcp" then ctrl+c when it tries to tftp
3) Do: "mii read 1 0x11" in u-boot and record the output ?

Regards,

Hans

Hi,

Quoted 7 lines · Fran?ois-David Collin

Hi,

As I?m banging my head on this too, please allow me to provide some details
I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Michael Haas' work to debug this by looking at the phy registers seems to be the most promising so-far, the clk reg and axp209 registers all seem to be identical between good and bad setups.

Can you try to:

1) Stop the boot in u-boot (press a key on the serial console)
2) Bring up the network, e.g. type "dhcp" then ctrl+c when it tries to tftp
3) Do: "mii read 1 0x11" in u-boot and record the output ?

Regards,

Hans

Hi,

Quoted 7 lines · Fran?ois-David Collin

Hi,

As I?m banging my head on this too, please allow me to provide some details
I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Michael Haas' work to debug this by looking at the phy registers seems to be the most promising so-far, the clk reg and axp209 registers all seem to be identical between good and bad setups.

Can you try to:

1) Stop the boot in u-boot (press a key on the serial console)
2) Bring up the network, e.g. type "dhcp" then ctrl+c when it tries to tftp
3) Do: "mii read 1 0x11" in u-boot and record the output ?

Regards,

Hans

Quoted 17 lines · Hans de Goede

Hi,

Quoted 7 lines · Fran?ois-David Collin

Hi,

As I?m banging my head on this too, please allow me to provide some details
I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Michael Haas' work to debug this by looking at the phy registers seems to be the most promising so-far, the clk reg and axp209 registers all seem to be identical between good and bad setups.

Can you try to:

Quoted 8 lines

1) Stop the boot in u-boot (press a key on the serial console)
2) Bring up the network, e.g. type "dhcp" then ctrl+c when it tries to tftp
3) Do: "mii read 1 0x11" in u-boot and record the output ?

Regards,

Hans

Hi all,

i have diffed and cross-compared logs of several working and broken tftp downloads. The most significant
difference between working and broken was registers 0xa, 0x1c and 0x2a in the first MII page.

These registers started making sense when I looked at the datasheet for the RTL8211CL. Previously, I was looking at the RTL8211E.

0xA was set to 7800 for working runs and 3800 for broken runs. The difference is bit 14 in the GBSR: 'MASTER/SLAVE Configuration Resolution'. In the broken runs, the autonegotation configured the OlinuXino to 'slave'. I have whipped up a quick patch which disable the master/slave part of the autonegotiation and force master mode.

The patch itself is quite terrible as it's forcing the master mode bits for every phy, not just for the RTL8211CL. It's good enough for testing, however, and I seem to be getting consistent download speeds. There are occasional hangs when booting the downloaded kernel, but that is probably a different issue.

drivers/net/phy/phy.c +1 -0

index 51b5746..484b2be 100644
@@ -170,6 +170,7 @@ int genphy_restart_aneg(struct phy_device *phydev)
  int genphy_config_aneg(struct phy_device *phydev)
  {
         int result;
+       phy_write(phydev, MDIO_DEVAD_NONE, 0x09, 0x1A00);
 
         if (AUTONEG_ENABLE != phydev->autoneg)
                 return genphy_setup_forced(phydev);

Please test this change and let me know. If it's successful, I will submit a proper version.

Hi,

Quoted 62 lines · Michael Haas

Quoted 17 lines · Hans de Goede

Hi,

Quoted 7 lines · Fran?ois-David Collin

Hi,

As I?m banging my head on this too, please allow me to provide some details
I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Michael Haas' work to debug this by looking at the phy registers seems to be the most promising so-far, the clk reg and axp209 registers all seem to be identical between good and bad setups.

Can you try to:

Quoted 8 lines

1) Stop the boot in u-boot (press a key on the serial console)
2) Bring up the network, e.g. type "dhcp" then ctrl+c when it tries to tftp
3) Do: "mii read 1 0x11" in u-boot and record the output ?

Regards,

Hans

Hi all,

i have diffed and cross-compared logs of several working and broken tftp downloads. The most significant
difference between working and broken was registers 0xa, 0x1c and 0x2a in the first MII page.

These registers started making sense when I looked at the datasheet for the RTL8211CL. Previously, I was looking at the RTL8211E.

0xA was set to 7800 for working runs and 3800 for broken runs. The difference is bit 14 in the GBSR: 'MASTER/SLAVE Configuration Resolution'. In the broken runs, the autonegotation configured the OlinuXino to 'slave'. I have whipped up a quick patch which disable the master/slave part of the autonegotiation and force master mode.

The patch itself is quite terrible as it's forcing the master mode bits for every phy, not just for the RTL8211CL. It's good enough for testing, however, and I seem to be getting consistent download speeds. There are occasional hangs when booting the downloaded kernel, but that is probably a different issue.

drivers/net/phy/phy.c +1 -0

index 51b5746..484b2be 100644
@@ -170,6 +170,7 @@ int genphy_restart_aneg(struct phy_device *phydev)
  int genphy_config_aneg(struct phy_device *phydev)
  {
         int result;
+       phy_write(phydev, MDIO_DEVAD_NONE, 0x09, 0x1A00);
 
         if (AUTONEG_ENABLE != phydev->autoneg)
                 return genphy_setup_forced(phydev);

Quoted 2 lines

Please test this change and let me know. If it's successful, I will submit a proper version.

Good catch, I wonder why we need this. I believe that the proper version should probably be wrapped in a #ifdef CONFIG_REALTEK_PHY_FORCE_MASTER and then add a Kconfig option for this (and enable it in the lime2 defconfig), forcing this on all rtl8211cl phy-s seems wrong, unless someone can dig up an errata from realtek which said we should.

Are any other sunxi boards impacted by the same problem that you know ?

Quoted 65 lines · Hans de Goede

Hi,

Quoted 62 lines · Michael Haas

Quoted 17 lines · Hans de Goede

Hi,

Quoted 7 lines · Fran?ois-David Collin

Hi,

As I?m banging my head on this too, please allow me to provide some details
I got two stable situations :
The Lime2 is connected directly to the Gbit interface of my laptop, speed are OK:

Michael Haas' work to debug this by looking at the phy registers seems to be the most promising so-far, the clk reg and axp209 registers all seem to be identical between good and bad setups.

Can you try to:

Quoted 8 lines

1) Stop the boot in u-boot (press a key on the serial console)
2) Bring up the network, e.g. type "dhcp" then ctrl+c when it tries to tftp
3) Do: "mii read 1 0x11" in u-boot and record the output ?

Regards,

Hans

Hi all,

i have diffed and cross-compared logs of several working and broken tftp downloads. The most significant
difference between working and broken was registers 0xa, 0x1c and 0x2a in the first MII page.

These registers started making sense when I looked at the datasheet for the RTL8211CL. Previously, I was looking at the RTL8211E.

0xA was set to 7800 for working runs and 3800 for broken runs. The difference is bit 14 in the GBSR: 'MASTER/SLAVE Configuration Resolution'. In the broken runs, the autonegotation configured the OlinuXino to 'slave'. I have whipped up a quick patch which disable the master/slave part of the autonegotiation and force master mode.

The patch itself is quite terrible as it's forcing the master mode bits for every phy, not just for the RTL8211CL. It's good enough for testing, however, and I seem to be getting consistent download speeds. There are occasional hangs when booting the downloaded kernel, but that is probably a different issue.

drivers/net/phy/phy.c +1 -0

index 51b5746..484b2be 100644
@@ -170,6 +170,7 @@ int genphy_restart_aneg(struct phy_device *phydev)
   int genphy_config_aneg(struct phy_device *phydev)
   {
          int result;
+       phy_write(phydev, MDIO_DEVAD_NONE, 0x09, 0x1A00);
 
          if (AUTONEG_ENABLE != phydev->autoneg)
                  return genphy_setup_forced(phydev);

Quoted 8 lines

Quoted 2 lines

Please test this change and let me know. If it's successful, I will submit a proper version.

Good catch, I wonder why we need this. I believe that the proper version should probably be wrapped in a #ifdef CONFIG_REALTEK_PHY_FORCE_MASTER and then add a Kconfig option for this (and enable it in the lime2 defconfig), forcing this on all rtl8211cl phy-s seems wrong, unless someone can dig up an errata from realtek which said we should.

I was going to add that in realtek.c, not in phy.c - as soon as I figure out which section there is applicable.

I have started a new thread
asking about that.

Quoted 2 lines

Are any other sunxi boards impacted by the same problem that you know ?

No, I don't know of any other boards, but I have not looked very hard :)

Quoted 2 lines

Quoted 2 lines

Quoted 1 line

Are any other sunxi boards impacted by the same problem that you know ?

No, I don't know of any other boards, but I have not looked very hard :)

At least the Olimex A20-SOM-EVB (which uses the same RTL8211CL PHY) has the same issue and forcing master mode helps there as well. With the above change added on top of v2016.01, I can successfully boot via tftp on a gigabit link on both the Lime2 and the A20-SOM-EVB, which was impossible before.

Some quick iperf measurements on the A20-SOM-EVB with the patch applied show network throughput comparable to what I get with other A20-based systems.

All my other A20-based boards - which don't have problems on gigabit links - use an RTL8211E instead of the RTL8211CL. I had been looking into the source of the gigabit problems on the A20-SOM-EVB for a while already and talked with Olimex about them. Olimex has also been trying to debug the issue and told me that the problem doesn't seem to occur on all boards, and that during their tests different RTL8211CL chips (i.e. same chip model, but with different production date codes) behaved differently. This would fit into the hypothesis that there might be some kind of erratum in at least some series of the RTL8211CL.

Quoted 7 lines · Karsten Merker

Quoted 2 lines

Quoted 2 lines

Quoted 1 line

Are any other sunxi boards impacted by the same problem that you know ?

No, I don't know of any other boards, but I have not looked very hard :)

At least the Olimex A20-SOM-EVB (which uses the same RTL8211CL PHY) has the same issue and forcing master mode helps there as well. With the above change added on top of v2016.01, I can successfully boot via tftp on a gigabit link on both the Lime2 and the A20-SOM-EVB, which was impossible before.

Great news!

Quoted 5 lines

Some quick iperf measurements on the A20-SOM-EVB with the patch applied show network throughput comparable to what I get with other A20-based systems.

Quoted 12 lines

All my other A20-based boards - which don't have problems on gigabit links - use an RTL8211E instead of the RTL8211CL. I had been looking into the source of the gigabit problems on the A20-SOM-EVB for a while already and talked with Olimex about them. Olimex has also been trying to debug the issue and told me that the problem doesn't seem to occur on all boards, and that during their tests different RTL8211CL chips (i.e. same chip model, but with different production date codes) behaved differently. This would fit into the hypothesis that there might be some kind of erratum in at least some series of the RTL8211CL.

So, Olimex knows about these issues? :/ Perhaps you can CC your contact there once we have a final patch.

By the way, I just searched the web for "rtl8211c master mode" and found this [0]:

---%--

// The EVK board with RTL8211C requires this sequence per information

	// Realtek.
	// 1. Register 0x1F, write 0005 -> Page 5
	// 2. Register 0x0C, write 0000
	// 3. Register 0x01, change bit [7]  to 1
	// 4. Register 0x1F, write 0000 -> Page 0

	// However, please note that it is not recommended to use the RTL8211CL 125MHz clock output because
	// it is not stable when link at Slave mode. If you need the clock output,
	// please consider using RTL8211E instead.
	// 5. Force manual "master" mode in auto-negotiation.

	#if defined(CONFIG_DIGICOLOR_MAINBOARD_THUNDERBOLT_REV1)
	if (!RTL8211C_hack) {
	  // Enable 125 Mhz clock output
	  gmac_mdio_write(ei, phy_id, 0x1f, 5);
	  gmac_mdio_write(ei, phy_id, 0x0c, 0);
   	  gmac_mdio_read (ei, phy_id, 0x01, &data);
	  data |= 0x80;
	  gmac_mdio_write(ei, phy_id, 0x01, data);
	  gmac_mdio_write(ei, phy_id, 0x1f, 0);
	  // Set Manual MASTER Mode
   	  gmac_mdio_read (ei, phy_id, 0x09, &data);
	  data |= 0x1800;
	  gmac_mdio_write(ei, phy_id, 0x09, data);
	  // Restart Autonegotiation
   	  gmac_mdio_read (ei, phy_id, 0x00, &data);
	  data |= 0x0200;
	  gmac_mdio_write(ei, phy_id, 0x00, data);
	  // All done now.
	  RTL8211C_hack = 1;
	}
	#endif

%---

I guess we are not the first to find this out the hard way. If the issue is related to an internal clock of the RTL8211C, is there perhaps an external clock we can use?

Hi,

Quoted 70 lines · Michael Haas

Quoted 7 lines · Karsten Merker

Quoted 2 lines

Quoted 2 lines

Quoted 1 line

Are any other sunxi boards impacted by the same problem that you know ?

No, I don't know of any other boards, but I have not looked very hard :)

At least the Olimex A20-SOM-EVB (which uses the same RTL8211CL PHY) has the same issue and forcing master mode helps there as well. With the above change added on top of v2016.01, I can successfully boot via tftp on a gigabit link on both the Lime2 and the A20-SOM-EVB, which was impossible before.

Great news!

Quoted 5 lines

Some quick iperf measurements on the A20-SOM-EVB with the patch applied show network throughput comparable to what I get with other A20-based systems.

Quoted 12 lines

All my other A20-based boards - which don't have problems on gigabit links - use an RTL8211E instead of the RTL8211CL. I had been looking into the source of the gigabit problems on the A20-SOM-EVB for a while already and talked with Olimex about them. Olimex has also been trying to debug the issue and told me that the problem doesn't seem to occur on all boards, and that during their tests different RTL8211CL chips (i.e. same chip model, but with different production date codes) behaved differently. This would fit into the hypothesis that there might be some kind of erratum in at least some series of the RTL8211CL.

So, Olimex knows about these issues? :/ Perhaps you can CC your contact there once we have a final patch.

By the way, I just searched the web for "rtl8211c master mode" and found this [0]:

---%--

// The EVK board with RTL8211C requires this sequence per information

    // Realtek.
    // 1. Register 0x1F, write 0005 -> Page 5
    // 2. Register 0x0C, write 0000
    // 3. Register 0x01, change bit [7]  to 1
    // 4. Register 0x1F, write 0000 -> Page 0

    // However, please note that it is not recommended to use the RTL8211CL 125MHz clock output because
    // it is not stable when link at Slave mode. If you need the clock output,
    // please consider using RTL8211E instead.
    // 5. Force manual "master" mode in auto-negotiation.

    #if defined(CONFIG_DIGICOLOR_MAINBOARD_THUNDERBOLT_REV1)
    if (!RTL8211C_hack) {
      // Enable 125 Mhz clock output
      gmac_mdio_write(ei, phy_id, 0x1f, 5);
      gmac_mdio_write(ei, phy_id, 0x0c, 0);
         gmac_mdio_read (ei, phy_id, 0x01, &data);
      data |= 0x80;
      gmac_mdio_write(ei, phy_id, 0x01, data);
      gmac_mdio_write(ei, phy_id, 0x1f, 0);
      // Set Manual MASTER Mode
         gmac_mdio_read (ei, phy_id, 0x09, &data);
      data |= 0x1800;
      gmac_mdio_write(ei, phy_id, 0x09, data);
      // Restart Autonegotiation
         gmac_mdio_read (ei, phy_id, 0x00, &data);
      data |= 0x0200;
      gmac_mdio_write(ei, phy_id, 0x00, data);
      // All done now.
      RTL8211C_hack = 1;
    }
    #endif

Quoted 6 lines

%---

I guess we are not the first to find this out the hard way. If the issue is related to an internal clock of the RTL8211C, is there perhaps an external clock we can use?

Oh good catch, so I believe that we should submit a patch to the u-boot realtek phy driver which:

1) Adds a seperate init function for the RTL8211C
2) In that function set the force master mode bit and then call the init function currently used for RTL8211C and others, this assumes that the more generic init function will not reset the force master mode bit.

Good work! I believe we're quite close to a fix now :)

Hmm, I just realized / read that RTL8211B and RTL8211C have the same

Otherwise we are back to having a #ifdef for the fix ...

Regards,

Quoted 1 line

Hmm, I just realized / read that RTL8211B and RTL8211C have the same

phyid, maybe there is some other reg which we can use to differentiate between them ?

Quoted 1 line

Otherwise we are back to having a #ifdef for the fix ...

Quoted 1 line

Regards,

Hans

The BSD people seem to key off a revision field, which has value 3 for the rtl8211c:

https://lists.freebsd.org/pipermail/svn-src-stable-9/2015-March/007669.html

Hi,

Quoted 8 lines · Michael Haas

Quoted 1 line

Hmm, I just realized / read that RTL8211B and RTL8211C have the same

phyid, maybe there is some other reg which we can use to differentiate between them ?

Quoted 3 lines

Otherwise we are back to having a #ifdef for the fix ...

Regards,

Quoted 8 lines

Hans

The BSD people seem to key off a revision field, which has value 3 for the rtl8211c:

https://lists.freebsd.org/pipermail/svn-src-stable-9/2015-March/007669.html

And another good catch, yes that looks like it should work.

Now all we need to do is turn all this info into a proper patch.

Regards,

Hans

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Hans" == Hans de Goede <[email protected]> writes:

Oh good catch, so I believe that we should submit a patch to the u-boot realtek phy driver which:

1) Adds a seperate init function for the RTL8211C
2) In that function set the force master mode bit and then call the init function currently used for RTL8211C and others, this assumes that the more generic init function will not reset the force master mode bit.

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Hans" == Hans de Goede <[email protected]> writes:

Hi,

Quoted 2 lines

Oh good catch, so I believe that we should submit a patch to the u-boot realtek phy driver which:

Quoted 5 lines

1) Adds a seperate init function for the RTL8211C
2) In that function set the force master mode bit and then call the init function currently used for RTL8211C and others, this assumes that the more generic init function will not reset the force master mode bit.

So what happens if you connect two of these boards together without a switch in between? Will they be able to autonegotiate?

Hi,

Quoted 5 lines · Peter Korsgaard

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Hans" == Hans de Goede <[email protected]> writes:

Hi,

Quoted 1 line

Oh good catch, so I believe that we should submit a patch to the

u-boot

Quoted 6 lines

Quoted 6 lines

realtek phy driver which:

1) Adds a seperate init function for the RTL8211C
2) In that function set the force master mode bit and then call the init function currently used for RTL8211C and others, this assumes that the more generic init function will not

reset

Quoted 4 lines

Quoted 1 line

the force master mode bit.

So what happens if you connect two of these boards together without a switch in between? Will they be able to autonegotiate?

Likely not, but without this fix, they would and they would have an unreliable connection. Arguably no connection is better then an unreliable one.

Regards,

Hans

Quoted 4 lines

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Hans" == Hans de Goede <[email protected]> writes:

So what happens if you connect two of these boards together without a switch in between? Will they be able to autonegotiate?

Likely not, but without this fix, they would and they would have an unreliable connection. Arguably no connection is better then an unreliable one.

Quoted 21 lines · Karsten Merker

Quoted 20 lines · Hans de Goede

Quoted 19 lines · Michael Haas

Quoted 18 lines · Hans de Goede

Quoted 16 lines · Michael Haas

0xA was set to 7800 for working runs and 3800 for broken runs. The difference is bit 14 in the GBSR: 'MASTER/SLAVE Configuration Resolution'. In the broken runs, the autonegotation configured the OlinuXino to 'slave'. I have whipped up a quick patch which disable the master/slave part of the autonegotiation and force master mode.

drivers/net/phy/phy.c +1 -0

index 51b5746..484b2be 100644
@@ -170,6 +170,7 @@ int genphy_restart_aneg(struct phy_device *phydev)
   int genphy_config_aneg(struct phy_device *phydev)
   {
          int result;
+       phy_write(phydev, MDIO_DEVAD_NONE, 0x09, 0x1A00);
 

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Quoted 3 lines

drivers/net/phy/phy.c int genphy_restart_aneg(struct phy_device *phydev) · line 176

Quoted 3 lines

drivers/net/phy/phy.c +0 -0

          if (AUTONEG_ENABLE != phydev->autoneg)
                  return genphy_setup_forced(phydev);

I was going to add that in realtek.c, not in phy.c - as soon as I figure out which section there is applicable.

Unfortunately the issue is now split over two threads - this one and the "Which entry in drivers/net/phy/realtek.c matches my device?" thread.

It looks like we have three related but separate issues here:

a) The sunxi config doesn't actually build the phy drivers:

include/configs/sunxi-common.h contains:

302 /* Ethernet support */
303 #ifdef CONFIG_SUNXI_EMAC
304 #define CONFIG_PHY_ADDR 1
305 #define CONFIG_MII /* MII PHY management */
306 #define CONFIG_PHYLIB
307 #endif
308
309 #ifdef CONFIG_SUNXI_GMAC
310 #define CONFIG_PHY_GIGE /* GMAC can use gigabit PHY */
311 #define CONFIG_PHY_ADDR 1
312 #define CONFIG_MII /* MII PHY management */
313 #endif

For the sunxi EMAC case (which is AFAIK only used on A10/A13) CONFIG_PHYLIB is enabled, but for the CONFIG_SUNXI_GMAC case (all other sunxi-based systems) we don't enable any PHY driver.

So we
either need to enable CONFIG_PHYLIB for CONFIG_SUNXI_GMAC as well or enable the relevant PHY drivers (do we actually have anything besides realtek on sunxi-based systems?) separately, i.e. something like

include/configs/sunxi-common.h +1 -0

index b26363d..ea15e37 100644
@@ -310,6 +310,7 @@ extern int soft_i2c_gpio_scl;
  #define CONFIG_PHY_GIGE			/* GMAC can use gigabit PHY	*/
  #define CONFIG_PHY_ADDR		1
  #define CONFIG_MII			/* MII PHY management		*/
+#define CONFIG_PHY_REALTEK
  #endif
 
  #ifdef CONFIG_USB_EHCI_HCD

b) The realtek PHY driver (drivers/net/phy/realtek.c) doesn't match on the RTL8211CL PHY ID, so even if it is compiled in, it still doesn't work. For more details on this issue, see the "Which entry in drivers/net/phy/realtek.c matches my device?" thread.

c) The master/slave mode problem that can be worked around by forcing master mode on the RTL8211CL.