Skip to contentSkip to conversation

arm64: dts: rockchip: fix rk3328-roc-cc gmac2io tx/rx_delay

PATCH In review Leonidas P. Papadakos 3 participants 16 messages Mar 6, 2019 – Mar 13, 2019 12 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 arm64: dts: rockchip: fix rk3328-roc-cc gmac2io tx/rx_delay

The rk3328-roc-cc board exhibits tx stability issues with large packets, as does the rock64 board, which was fixed with this patch
https://patchwork.kernel.org/patch/10178969/

A similar patch was merged for the rk3328-roc-cc here
https://patchwork.kernel.org/patch/10804863/
but it doesn't include the tx/rx_delay tweaks, and I find that they help with an issue where large transfers would bring the ethernet link down, causing a link reset regularly.

These values are better for the Renegade specifically, borrowed from the armbian rk3328-roc-cc.dts:
https://github.com/armbian/build/blob/master/patch/kernel/rockchip64-default/Add_dts_rk3328-roc-cc.patch

---
 arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +2 -2
index 33c44e857..ec5454028 100644
@@ -108,8 +108,8 @@
 	snps,reset-gpio = <&gpio1 RK_PC2 GPIO_ACTIVE_LOW>;
 	snps,reset-active-low;
 	snps,reset-delays-us = <0 10000 50000>;
-	tx_delay = <0x25>;
-	rx_delay = <0x11>;
+	tx_delay = <0x28>;
+	rx_delay = <0x16>;
 	status = "okay";
 };
 
Signature · 1 line
2.21.0

A message that is not in the archive (sent off-list or not yet received): 20190306223454.2959-1-papadakospan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#2 lore raw

I think this should be included in the 5.1 release cycle if possible.

It goes hand-in-hand with the previous fix ("Enable thresh dma mode to disable checksuming.") when it comes to network stability

#3 lore raw
In reply to a message not in the archive
Quoted 25 lines · Leonidas P. Papadakos

The rk3328-roc-cc board exhibits tx stability issues with large packets, as does the rock64 board, which was fixed with this patch
https://patchwork.kernel.org/patch/10178969/

A similar patch was merged for the rk3328-roc-cc here
https://patchwork.kernel.org/patch/10804863/
but it doesn't include the tx/rx_delay tweaks, and I find that they help with an issue where large transfers would bring the ethernet link down, causing a link reset regularly.

These values are better for the Renegade specifically, borrowed from the armbian rk3328-roc-cc.dts:
https://github.com/armbian/build/blob/master/patch/kernel/rockchip64-default/Add_dts_rk3328-roc-cc.patch

---
  arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0 -0
index 33c44e857..ec5454028 100644

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 115

Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +2 -2
@@ -108,8 +108,8 @@
  	snps,reset-gpio = <&gpio1 RK_PC2 GPIO_ACTIVE_LOW>;
  	snps,reset-active-low;
  	snps,reset-delays-us = <0 10000 50000>;
-	tx_delay = <0x25>;
-	rx_delay = <0x11>;
+	tx_delay = <0x28>;
+	rx_delay = <0x16>;
  	status = "okay";
  };
 

When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.

I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?

A message that is not in the archive (sent off-list or not yet received): 24c5ffaa-644d-4623-a635-6842f67efe8a-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#5 lore raw
Quoted 7 lines

When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.

I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?

I feared as much. The delays on the tx side do seem to be more of a mitigation. I you have the know-how for it, it would be pretty cool

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw
Quoted 11 lines
Quoted 8 lines

When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.

I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?

I feared as much. The delays on the tx side do seem to be more of a mitigation. I you have the know-how for it, it would be pretty cool

Interesting approach to this by ayufan:
https://github.com/ayufan-rock64/linux-mainline-kernel/commit/41eeb7cd789ea7cac0cbf4b35b53055f354da757

I hope he sends it upstream as well

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 9 lines · Leonidas P. Papadakos
Quoted 9 lines
Quoted 8 lines

When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.

I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?

Quoted 8 lines
Quoted 2 lines

I feared as much. The delays on the tx side do seem to be more of a mitigation. I you have the know-how for it, it would be pretty cool

Interesting approach to this by ayufan:
https://github.com/ayufan-rock64/linux-mainline-kernel/commit/41eeb7cd789ea7cac0cbf4b35b53055f354da757

I hope he sends it upstream as well

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
 			rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
 		};
 	};
+
+	gmac-1 {
+		rgmiim1_pins: rgmiim1-pins {
+			rockchip,pins =
+				/* mac_txclk */
+				<0 RK_PB0 1 &pcfg_pull_none_12ma>,
+				/* mac_txen */
+				<0 RK_PB4 1 &pcfg_pull_none_12ma>,
+				/* mac_clk */
+				<0 RK_PD0 1 &pcfg_pull_none_12ma>,
+				/* mac_txd1 */
+				<0 RK_PC0 1 &pcfg_pull_none_12ma>,
+				/* mac_txd0 */
+				<0 RK_PC1 1 &pcfg_pull_none_12ma>,
+				/* mac_txd3 */
+				<0 RK_PC7 1 &pcfg_pull_none_12ma>,
+				/* mac_txd2 */
+				<0 RK_PC6 1 &pcfg_pull_none_12ma>;
+		};
+	};
 };
 
 &sdmmc {

A message that is not in the archive (sent off-list or not yet received): ec810aad-dd04-84e4-bfb0-98b005cf3f03-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#11 lore raw
Quoted 33 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +13 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
 			rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
 		};
 	};
+
+	gmac-1 {
+		rgmiim1_pins: rgmiim1-pins {
+			rockchip,pins =
+				/* mac_txclk */
+				<0 RK_PB0 1 &pcfg_pull_none_12ma>,
+				/* mac_txen */
+				<0 RK_PB4 1 &pcfg_pull_none_12ma>,
+				/* mac_clk */
+				<0 RK_PD0 1 &pcfg_pull_none_12ma>,
+				/* mac_txd1 */
+				<0 RK_PC0 1 &pcfg_pull_none_12ma>,
+				/* mac_txd0 */

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305

Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +7 -0
+				<0 RK_PC1 1 &pcfg_pull_none_12ma>,
+				/* mac_txd3 */
+				<0 RK_PC7 1 &pcfg_pull_none_12ma>,
+				/* mac_txd2 */
+				<0 RK_PC6 1 &pcfg_pull_none_12ma>;
+		};
+	};
 };
 
 &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw
Quoted 42 lines · Leonidas P. Papadakos
Quoted 42 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
              rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
          };
      };
+
+    gmac-1 {
+        rgmiim1_pins: rgmiim1-pins {
+            rockchip,pins =
+                /* mac_txclk */
+                <0 RK_PB0 1 &pcfg_pull_none_12ma>,
+                /* mac_txen */
+                <0 RK_PB4 1 &pcfg_pull_none_12ma>,
+                /* mac_clk */
+                <0 RK_PD0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd1 */
+                <0 RK_PC0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd0 */
+                <0 RK_PC1 1 &pcfg_pull_none_12ma>,
+                /* mac_txd3 */
+                <0 RK_PC7 1 &pcfg_pull_none_12ma>,
+                /* mac_txd2 */
+                <0 RK_PC6 1 &pcfg_pull_none_12ma>;
+        };
+    };
  };
 
Quoted 4 lines

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305

Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0 -0
  &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

With Ayufan's patch offload patch and my pull patch, I now have the following results:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec  885             sender
[  5]   0.00-10.01  sec  1022 MBytes   856 Mbits/sec receiver

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

        inet6 fe80::e8b5:5eff:fe49:d353  prefixlen 64  scopeid 0x20<link>
        ether ea:b5:5e:49:d3:53  txqueuelen 1000  (Ethernet)
        RX packets 39822  bytes 2668618 (2.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1005649  bytes 1516698158 (1.5 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 29

Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

A message that is not in the archive (sent off-list or not yet received): e9c539ce-f1cb-06f8-1083-ea64c9fd9e7b-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#15 lore raw

Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:

Quoted 67 lines · Leonidas P. Papadakos
Quoted 44 lines
Quoted 44 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
             rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
         };
     };
+
+    gmac-1 {
+        rgmiim1_pins: rgmiim1-pins {
+            rockchip,pins =
+                /* mac_txclk */
+                <0 RK_PB0 1 &pcfg_pull_none_12ma>,
+                /* mac_txen */
+                <0 RK_PB4 1 &pcfg_pull_none_12ma>,
+                /* mac_clk */
+                <0 RK_PD0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd1 */
+                <0 RK_PC0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd0 */
+                <0 RK_PC1 1 &pcfg_pull_none_12ma>,
+                /* mac_txd3 */
+                <0 RK_PC7 1 &pcfg_pull_none_12ma>,
+                /* mac_txd2 */
+                <0 RK_PC6 1 &pcfg_pull_none_12ma>;
+        };
+    };
 };
 
Quoted 5 lines

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305

Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0 -0
 &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

With Ayufan's patch offload patch and my pull patch, I now have the following results:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec  885             sender
[  5]   0.00-10.01  sec  1022 MBytes   856 Mbits/sec 

receiver

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

         inet6 fe80::e8b5:5eff:fe49:d353  prefixlen 64  scopeid 0x20<link>
         ether ea:b5:5e:49:d3:53  txqueuelen 1000  (Ethernet)
         RX packets 39822  bytes 2668618 (2.6 MB)
         RX errors 0  dropped 0  overruns 0  frame 0
         TX packets 1005649  bytes 1516698158 (1.5 GB)
         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
         device interrupt 29
Quoted 2 lines

Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.

also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.

Heiko

#16 lore raw
Quoted 71 lines · Heiko Stuebner

Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:

Quoted 67 lines · Leonidas P. Papadakos
Quoted 44 lines
Quoted 44 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
              rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
          };
      };
+
+    gmac-1 {
+        rgmiim1_pins: rgmiim1-pins {
+            rockchip,pins =
+                /* mac_txclk */
+                <0 RK_PB0 1 &pcfg_pull_none_12ma>,
+                /* mac_txen */
+                <0 RK_PB4 1 &pcfg_pull_none_12ma>,
+                /* mac_clk */
+                <0 RK_PD0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd1 */
+                <0 RK_PC0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd0 */
+                <0 RK_PC1 1 &pcfg_pull_none_12ma>,
+                /* mac_txd3 */
+                <0 RK_PC7 1 &pcfg_pull_none_12ma>,
+                /* mac_txd2 */
+                <0 RK_PC6 1 &pcfg_pull_none_12ma>;
+        };
+    };
  };
 
Quoted 5 lines

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305

Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0 -0
  &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

With Ayufan's patch offload patch and my pull patch, I now have the following results:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec  885             sender
[  5]   0.00-10.01  sec  1022 MBytes   856 Mbits/sec

receiver

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

          inet6 fe80::e8b5:5eff:fe49:d353  prefixlen 64  scopeid 0x20<link>
          ether ea:b5:5e:49:d3:53  txqueuelen 1000  (Ethernet)
          RX packets 39822  bytes 2668618 (2.6 MB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1005649  bytes 1516698158 (1.5 GB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
          device interrupt 29
Quoted 2 lines

Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.

Quoted 7 lines

also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.

Heiko

Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.

I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

A message that is not in the archive (sent off-list or not yet received): 4f17a798-ed28-8403-ed75-30fdf4bfc44b-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#18 lore raw

Στις Σαβ, 9 Μαρ, 2019 at 4:21 ΜΜ, ο/η Peter Geis <[email protected]> έγραψε:

Quoted 85 lines · Heiko Stuebner
Quoted 74 lines

Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:

Quoted 70 lines · Leonidas P. Papadakos
Quoted 45 lines
Quoted 45 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
              rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO 
 &pcfg_pull_none>;
          };
      };
+
+    gmac-1 {
+        rgmiim1_pins: rgmiim1-pins {
+            rockchip,pins =
+                /* mac_txclk */
+                <0 RK_PB0 1 &pcfg_pull_none_12ma>,
+                /* mac_txen */
+                <0 RK_PB4 1 &pcfg_pull_none_12ma>,
+                /* mac_clk */
+                <0 RK_PD0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd1 */
+                <0 RK_PC0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd0 */
+                <0 RK_PC1 1 &pcfg_pull_none_12ma>,
+                /* mac_txd3 */
+                <0 RK_PC7 1 &pcfg_pull_none_12ma>,
+                /* mac_txd2 */
+                <0 RK_PC6 1 &pcfg_pull_none_12ma>;
+        };
+    };
  };
 
Quoted 5 lines
Quoted 1 line
  &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

With Ayufan's patch offload patch and my pull patch, I now have the following results:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec  885           

sender
[ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/sec
receiver

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

          inet6 fe80::e8b5:5eff:fe49:d353  prefixlen 64  scopeid 

0x20<link>
ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet) RX packets 39822 bytes 2668618 (2.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1005649 bytes 1516698158 (1.5 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 29

Quoted 2 lines

Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.

Quoted 7 lines

also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.

Heiko

Quoted 6 lines

Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.

I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!

For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <laptop_ip> -t 30.

I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity

2) just this patch by itself
the ssh lag hit me, but it does seem better. The link reset still
happens

3) the force-thresh-dma-mode patch with this one:
No shs lag, and as before, the tx activity seems to hit less? (Maybe
placebo) but the link reset does happen regularly.

As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

A message that is not in the archive (sent off-list or not yet received): [email protected]

#20 lore raw
Quoted 103 lines · Leonidas P. Papadakos

Στις Σαβ, 9 Μαρ, 2019 at 4:21 ΜΜ, ο/η Peter Geis <[email protected]> έγραψε:

Quoted 82 lines · Heiko Stuebner
Quoted 71 lines

Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:

Quoted 67 lines · Leonidas P. Papadakos
Quoted 44 lines
Quoted 44 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
               rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>;
           };
       };
+
+    gmac-1 {
+        rgmiim1_pins: rgmiim1-pins {
+            rockchip,pins =
+                /* mac_txclk */
+                <0 RK_PB0 1 &pcfg_pull_none_12ma>,
+                /* mac_txen */
+                <0 RK_PB4 1 &pcfg_pull_none_12ma>,
+                /* mac_clk */
+                <0 RK_PD0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd1 */
+                <0 RK_PC0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd0 */
+                <0 RK_PC1 1 &pcfg_pull_none_12ma>,
+                /* mac_txd3 */
+                <0 RK_PC7 1 &pcfg_pull_none_12ma>,
+                /* mac_txd2 */
+                <0 RK_PC6 1 &pcfg_pull_none_12ma>;
+        };
+    };
   };
 
Quoted 5 lines

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305

Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0 -0
   &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval           Transfer     Bitrate         Retr [  5]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec  885   sender [  5]   0.00-10.01  sec  1022 MBytes   856 Mbits/sec receiver

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500 inet6 fe80::e8b5:5eff:fe49:d353  prefixlen 64  scopeid 0x20<link> ether ea:b5:5e:49:d3:53  txqueuelen 1000  (Ethernet) RX packets 39822  bytes 2668618 (2.6 MB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1005649  bytes 1516698158 (1.5 GB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0 device interrupt 29

Quoted 2 lines

Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.

Quoted 7 lines

also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.

Heiko

Quoted 6 lines

Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.

I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!

For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <laptop_ip> -t 30.

I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity

2) just this patch by itself
  the ssh lag hit me, but it does seem better. The link reset still
happens

Quoted 8 lines

3) the force-thresh-dma-mode patch with this one:
  No shs lag, and as before, the tx activity seems to hit less? (Maybe
placebo) but the link reset does happen regularly.

As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better

We have several issues on the rk3328.

The first issue, which you describe as the ssh lag, is the tx-offload issue. Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled. To fix this, you must somehow disable tx-offload.
force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498. You can also disable tx-offload from userspace with ethtool.

The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.

The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.

I hope this clears things up a little for you.

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

A message that is not in the archive (sent off-list or not yet received): 1bab9d4c-5e9d-968d-026d-98e65af6e7ee-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#22 lore raw

Στις Κυρ, 10 Μαρ, 2019 at 1:16 ΠΜ, ο/η Peter Geis <[email protected]> έγραψε:

Quoted 127 lines · Leonidas P. Papadakos
Quoted 106 lines

Στις Σαβ, 9 Μαρ, 2019 at 4:21 ΜΜ, ο/η Peter Geis <[email protected]> έγραψε:

Quoted 85 lines · Heiko Stuebner
Quoted 74 lines

Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:

Quoted 67 lines · Leonidas P. Papadakos
Quoted 45 lines
Quoted 45 lines

So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.

I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.

I am also going to test his adaptive patch once it's done building.

Thoughts?

arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
index c0519bc5884e..696627bf3331 100644
@@ -280,6 +280,26 @@
              rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO 
 &pcfg_pull_none>;
          };
      };
+
+    gmac-1 {
+        rgmiim1_pins: rgmiim1-pins {
+            rockchip,pins =
+                /* mac_txclk */
+                <0 RK_PB0 1 &pcfg_pull_none_12ma>,
+                /* mac_txen */
+                <0 RK_PB4 1 &pcfg_pull_none_12ma>,
+                /* mac_clk */
+                <0 RK_PD0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd1 */
+                <0 RK_PC0 1 &pcfg_pull_none_12ma>,
+                /* mac_txd0 */
+                <0 RK_PC1 1 &pcfg_pull_none_12ma>,
+                /* mac_txd3 */
+                <0 RK_PC7 1 &pcfg_pull_none_12ma>,
+                /* mac_txd2 */
+                <0 RK_PC6 1 &pcfg_pull_none_12ma>;
+        };
+    };
  };
 
Quoted 5 lines
Quoted 1 line
  &sdmmc {

Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it

With Ayufan's patch offload patch and my pull patch, I now have the following results:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1022 MBytes   857 Mbits/sec  885   sender
[  5]   0.00-10.01  sec  1022 MBytes   856 Mbits/sec

receiver

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

          inet6 fe80::e8b5:5eff:fe49:d353  prefixlen 64  scopeid 
0x20<link>
          ether ea:b5:5e:49:d3:53  txqueuelen 1000  (Ethernet)
          RX packets 39822  bytes 2668618 (2.6 MB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 1005649  bytes 1516698158 (1.5 GB)
          TX errors 0  dropped 0 overruns 0  carrier 0  
Quoted 5 lines

collisions 0
device interrupt 29

Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.

Quoted 7 lines

also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.

Heiko

Quoted 6 lines

Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.

I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!

For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <laptop_ip> -t 30.

I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity

2) just this patch by itself
the ssh lag hit me, but it does seem better. The link reset still
happens

Quoted 8 lines

3) the force-thresh-dma-mode patch with this one:
No shs lag, and as before, the tx activity seems to hit less?
(Maybe placebo) but the link reset does happen regularly.

As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better

We have several issues on the rk3328.

The first issue, which you describe as the ssh lag, is the tx-offload issue. Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled. To fix this, you must somehow disable tx-offload.
force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498. You can also disable tx-offload from userspace with ethtool.

Quoted 5 lines

The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.

The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.

I hope this clears things up a little for you.

Yup, that's what I have in my mind already. My previous comment wasn't clear, I know. I pieced it together after your dma-mode patch, because that one fixed the ssh lag. My previous email was just what I saw after those changes.

In my case the link reset still happens with your patch.

You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work for the network instability problem?

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

A message that is not in the archive (sent off-list or not yet received): [email protected]

#24 lore raw

[clip]

Quoted 34 lines
Quoted 28 lines
Quoted 11 lines

For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <laptop_ip> -t 30.

I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity

2) just this patch by itself
   the ssh lag hit me, but it does seem better. The link reset still happens

Quoted 7 lines

3) the force-thresh-dma-mode patch with this one:
   No shs lag, and as before, the tx activity seems to hit less? (Maybe placebo) but the link reset does happen regularly.

As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better

We have several issues on the rk3328.

The first issue, which you describe as the ssh lag, is the tx-offload issue. Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled. To fix this, you must somehow disable tx-offload.
force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498. You can also disable tx-offload from userspace with ethtool.

Quoted 5 lines

The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.

The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.

I hope this clears things up a little for you.

Quoted 8 lines

Yup, that's what I have in my mind already. My previous comment wasn't clear, I know. I pieced it together after your dma-mode patch, because that one fixed the ssh lag. My previous email was just what I saw after those changes.

In my case the link reset still happens with your patch.

You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work for the network instability problem?

Correct, the link-reset tends to happen when the delays are at the edge of working.
There is also the possibility that board level manufacturing tolerances are causing issues where the delay can change between boards. If this is the case, we will have to make an auto-tune function, probably based off the mmc version.

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

A message that is not in the archive (sent off-list or not yet received): 6d6a700a-39c9-25b6-dfff-296c0757603d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#26 lore raw
Quoted 33 lines
Quoted 26 lines
Quoted 12 lines

We have several issues on the rk3328.

The first issue, which you describe as the ssh lag, is the tx-offload issue.
Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled.
To fix this, you must somehow disable tx-offload. force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498.
You can also disable tx-offload from userspace with ethtool.

Quoted 7 lines

The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.

The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.

I hope this clears things up a little for you.

Yup, that's what I have in my mind already. My previous comment wasn't clear, I know.
I pieced it together after your dma-mode patch, because that one fixed the ssh lag.
My previous email was just what I saw after those changes.

Quoted 6 lines

In my case the link reset still happens with your patch.

You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work for the network instability problem?

Quoted 6 lines

Correct, the link-reset tends to happen when the delays are at the edge of working.
There is also the possibility that board level manufacturing tolerances are causing issues where the delay can change between boards.
If this is the case, we will have to make an auto-tune function, probably based off the mmc version.

So, I tried just your patch with archlinuxARM linux 5.0 (meaning the tx/rx_delays are 0x25/0x11 respectively if I'm not mistaken) and disabled tx offload through ethtool.

After 3 30-second iperf3 tests, no link resets! I get about the same speed you mentioned before, 800-something to 700-something lowest.

Seems like the delays that are already there in the dts are better than the ones I picked up from the Armbian dts.

It's worth noting that I've never noticed a link reset in my ~3 months of using the 4.4 Rockchip kernel that Armbian packages using ayufan's linux-kernel repo. Maybe something there?

All in all, it does seem to be much better with your patch.

#27 lore raw

Στις Κυρ, 10 Μαρ, 2019 at 2:26 ΠΜ, ο/η "Leonidas P. Papadakos" <[email protected]> έγραψε:

Quoted 57 lines
Quoted 40 lines
Quoted 5 lines
Quoted 5 lines

We have several issues on the rk3328.

The first issue, which you describe as the ssh lag, is the tx-offload issue.
Packets that are larger than the MTU and will be fragmented

break if

Quoted 3 lines
Quoted 3 lines

tx-offload is enabled.
To fix this, you must somehow disable tx-offload. force-thresh-dma-mode disabled tx-offload, but had other side

effects.

Quoted 1 line
Quoted 1 line

Ayufan's patch disables tx-offload only if the packet is larger

than

Quoted 7 lines
Quoted 7 lines

1498.
You can also disable tx-offload from userspace with ethtool.

The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.

The third issue is the tx and rx delays need to be tuned now

that

Quoted 5 lines
Quoted 3 lines

the tx packet corruption issue has been fixed.

I hope this clears things up a little for you.

Yup, that's what I have in my mind already. My previous comment

wasn't

Quoted 2 lines

clear, I know.
I pieced it together after your dma-mode patch, because that one

fixed

Quoted 7 lines

the ssh lag.
My previous email was just what I saw after those changes.

In my case the link reset still happens with your patch.

You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work

for

Quoted 2 lines

the network instability problem?

Quoted 9 lines

Correct, the link-reset tends to happen when the delays are at the
edge of
working.
There is also the possibility that board level manufacturing
tolerances are
causing issues where the delay can change between boards. If this is the case, we will have to make an auto-tune function,
probably
based off the mmc version.

So, I tried just your patch with archlinuxARM linux 5.0 (meaning the tx/rx_delays are 0x25/0x11 respectively if I'm not mistaken) and disabled tx offload through ethtool.

After 3 30-second iperf3 tests, no link resets! I get about the same speed you mentioned before, 800-something to 700-something lowest.

Quoted 6 lines

Seems like the delays that are already there in the dts are better than the ones I picked up from the Armbian dts.

It's worth noting that I've never noticed a link reset in my ~3 months of using the 4.4 Rockchip kernel that Armbian packages using ayufan's linux-kernel repo. Maybe something there?

All in all, it does seem to be much better with your patch.

I've been trying to reach ayufan about his patch, getting is into mainline:
https://github.com/ayufan-rock64/linux-mainline-kernel/commit/41eeb7cd789ea7cac0cbf4b35b53055f354da757

but it seems he's not responding. I'll cc him.
His patch would be a really cool fix if it could be in 5.1, or even 5.2, as a replacement to force_thresh_dma_mode

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip