arm64: dts: rockchip: fix rk3328-roc-cc gmac2io tx/rx_delay
Subsystems (2)
PATCH v2 1/1 arm64: dts: rockchip: fix rk3328-roc-cc gmac2io tx/rx_delay
The rk3328-roc-cc board exhibits tx stability issues with large packets, as does the rock64 board, which was fixed with this patch
https://patchwork.kernel.org/patch/10178969/
A similar patch was merged for the rk3328-roc-cc here
https://patchwork.kernel.org/patch/10804863/
but it doesn't include the tx/rx_delay tweaks, and I find that they help with an issue where large transfers would bring the ethernet link down, causing a link reset regularly.
These values are better for the Renegade specifically, borrowed from the armbian rk3328-roc-cc.dts:
https://github.com/armbian/build/blob/master/patch/kernel/rockchip64-default/Add_dts_rk3328-roc-cc.patch
--- arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +2 -2
@@ -108,8 +108,8 @@ snps,reset-gpio = <&gpio1 RK_PC2 GPIO_ACTIVE_LOW>; snps,reset-active-low; snps,reset-delays-us = <0 10000 50000>; - tx_delay = <0x25>; - rx_delay = <0x11>; + tx_delay = <0x28>; + rx_delay = <0x16>; status = "okay"; };
Signature · 1 line
2.21.0
A message that is not in the archive (sent off-list or not yet received): 20190306223454.2959-1-papadakospan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
I think this should be included in the 5.1 release cycle if possible.
It goes hand-in-hand with the previous fix ("Enable thresh dma mode to disable checksuming.") when it comes to network stability
Quoted 25 lines · Leonidas P. Papadakos
The rk3328-roc-cc board exhibits tx stability issues with large packets, as does the rock64 board, which was fixed with this patch
https://patchwork.kernel.org/patch/10178969/A similar patch was merged for the rk3328-roc-cc here
https://patchwork.kernel.org/patch/10804863/
but it doesn't include the tx/rx_delay tweaks, and I find that they help with an issue where large transfers would bring the ethernet link down, causing a link reset regularly.These values are better for the Renegade specifically, borrowed from the armbian
rk3328-roc-cc.dts:
https://github.com/armbian/build/blob/master/patch/kernel/rockchip64-default/Add_dts_rk3328-roc-cc.patchSigned-off-by Leonidas P. Papadakos <[email protected]>--- arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0
-0
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 115
Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +2
-2@@ -108,8 +108,8 @@ snps,reset-gpio = <&gpio1 RK_PC2 GPIO_ACTIVE_LOW>; snps,reset-active-low; snps,reset-delays-us = <0 10000 50000>; - tx_delay = <0x25>; - rx_delay = <0x11>; + tx_delay = <0x28>; + rx_delay = <0x16>; status = "okay"; };
When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.
I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?
A message that is not in the archive (sent off-list or not yet received): 24c5ffaa-644d-4623-a635-6842f67efe8a-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Quoted 7 lines
When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.
I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?
I feared as much. The delays on the tx side do seem to be more of a mitigation. I you have the know-how for it, it would be pretty cool
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines
Quoted 8 lines
When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.
I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?
I feared as much. The delays on the tx side do seem to be more of a mitigation. I you have the know-how for it, it would be pretty cool
Interesting approach to this by ayufan:
https://github.com/ayufan-rock64/linux-mainline-kernel/commit/41eeb7cd789ea7cac0cbf4b35b53055f354da757
I hope he sends it upstream as well
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Leonidas P. Papadakos
Quoted 9 lines
Quoted 8 lines
When I submitted the patch to fix the TX stability issue, I intended to follow up with another patch for delay updates. Unfortunately, I was unable to locate a delay that was acceptable, as I was never able to get below ~700 retry attempts on the TX side in testing.
I am unsure if this is due to manufacturing defects or if I'm missing something else. It may be worth considering implementing some sort of delay auto tuning, similar to the mmc driver. Thoughts?
Quoted 8 lines
Quoted 2 lines
I feared as much. The delays on the tx side do seem to be more of a mitigation. I you have the know-how for it, it would be pretty cool
Interesting approach to this by ayufan:
https://github.com/ayufan-rock64/linux-mainline-kernel/commit/41eeb7cd789ea7cac0cbf4b35b53055f354da757I hope he sends it upstream as well
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The rk3328.dtsi is missing several rgmii tx pull strength values. This is causing them to default to 0ma.
I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20 -0
@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; }; &sdmmc {
A message that is not in the archive (sent off-list or not yet received): ec810aad-dd04-84e4-bfb0-98b005cf3f03-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Quoted 33 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +13
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305
Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +7
-0+ <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; }; &sdmmc {
Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 42 lines · Leonidas P. Papadakos
Quoted 42 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; };
Quoted 4 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305
Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0
-0&sdmmc {Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1022 MBytes 857 Mbits/sec 885 sender [ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/sec receiver
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e8b5:5eff:fe49:d353 prefixlen 64 scopeid 0x20<link>
ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet)
RX packets 39822 bytes 2668618 (2.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1005649 bytes 1516698158 (1.5 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 29Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
A message that is not in the archive (sent off-list or not yet received): e9c539ce-f1cb-06f8-1083-ea64c9fd9e7b-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:
Quoted 67 lines · Leonidas P. Papadakos
Quoted 44 lines
Quoted 44 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; };Quoted 5 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305
Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0
-0&sdmmc {Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1022 MBytes 857 Mbits/sec 885 sender [ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/secreceiver
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e8b5:5eff:fe49:d353 prefixlen 64 scopeid 0x20<link> ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet) RX packets 39822 bytes 2668618 (2.6 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1005649 bytes 1516698158 (1.5 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 29
Quoted 2 lines
Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.
also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.
Heiko
Quoted 71 lines · Heiko Stuebner
Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:
Quoted 67 lines · Leonidas P. Papadakos
Quoted 44 lines
Quoted 44 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; };Quoted 5 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305
Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0
-0&sdmmc {Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1022 MBytes 857 Mbits/sec 885 sender [ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/secreceiver
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e8b5:5eff:fe49:d353 prefixlen 64 scopeid 0x20<link> ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet) RX packets 39822 bytes 2668618 (2.6 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1005649 bytes 1516698158 (1.5 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 29Quoted 2 lines
Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.
Quoted 7 lines
also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.
Heiko
Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.
I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
A message that is not in the archive (sent off-list or not yet received): 4f17a798-ed28-8403-ed75-30fdf4bfc44b-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Στις Σαβ, 9 Μαρ, 2019 at 4:21 ΜΜ, ο/η Peter Geis <[email protected]> έγραψε:
Quoted 85 lines · Heiko Stuebner
Quoted 74 lines
Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:
Quoted 70 lines · Leonidas P. Papadakos
Quoted 45 lines
Quoted 45 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; };Quoted 5 lines
Quoted 1 line
&sdmmc {Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1022 MBytes 857 Mbits/sec 885sender
[ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/sec
receivereth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e8b5:5eff:fe49:d353 prefixlen 64 scopeid0x20<link>
ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet) RX packets 39822 bytes 2668618 (2.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1005649 bytes 1516698158 (1.5 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 29Quoted 2 lines
Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.
Quoted 7 lines
also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.
Heiko
Quoted 6 lines
Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!
For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <laptop_ip> -t 30.
I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity
2) just this patch by itself
the ssh lag hit me, but it does seem better. The link reset still
happens
3) the force-thresh-dma-mode patch with this one:
No shs lag, and as before, the tx activity seems to hit less? (Maybe
placebo) but the link reset does happen regularly.
As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 103 lines · Leonidas P. Papadakos
Στις Σαβ, 9 Μαρ, 2019 at 4:21 ΜΜ, ο/η Peter Geis <[email protected]> έγραψε:
Quoted 82 lines · Heiko Stuebner
Quoted 71 lines
Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:
Quoted 67 lines · Leonidas P. Papadakos
Quoted 44 lines
Quoted 44 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; };Quoted 5 lines
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts line 305
Quoted 1 line
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +0
-0&sdmmc {Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1022 MBytes 857 Mbits/sec 885 sender [ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/sec receivereth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 fe80::e8b5:5eff:fe49:d353 prefixlen 64 scopeid 0x20<link> ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet) RX packets 39822 bytes 2668618 (2.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1005649 bytes 1516698158 (1.5 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 29Quoted 2 lines
Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.
Quoted 7 lines
also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.
Heiko
Quoted 6 lines
Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!
For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <
laptop_ip> -t 30.I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity2) just this patch by itself
the ssh lag hit me, but it does seem better. The link reset still
happens
Quoted 8 lines
3) the force-thresh-dma-mode patch with this one:
No shs lag, and as before, the tx activity seems to hit less? (Maybe
placebo) but the link reset does happen regularly.As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better
We have several issues on the rk3328.
The first issue, which you describe as the ssh lag, is the tx-offload issue. Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled. To fix this, you must somehow disable tx-offload.
force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498. You can also disable tx-offload from userspace with ethtool.
The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.
The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.
I hope this clears things up a little for you.
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
A message that is not in the archive (sent off-list or not yet received): 1bab9d4c-5e9d-968d-026d-98e65af6e7ee-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Στις Κυρ, 10 Μαρ, 2019 at 1:16 ΠΜ, ο/η Peter Geis <[email protected]> έγραψε:
Quoted 127 lines · Leonidas P. Papadakos
Quoted 106 lines
Στις Σαβ, 9 Μαρ, 2019 at 4:21 ΜΜ, ο/η Peter Geis <[email protected]> έγραψε:
Quoted 85 lines · Heiko Stuebner
Quoted 74 lines
Am Samstag, 9. März 2019, 13:45:21 CET schrieb Peter Geis:
Quoted 67 lines · Leonidas P. Papadakos
Quoted 45 lines
Quoted 45 lines
So I discovered the cause of my TX issues, at least on my rk3328-roc-cc. The
rk3328.dtsiis missing several rgmii tx pull strength values. This is causing them to default to 0ma.I've pushed them to 12ma, same as the other tx values, and I'm getting 600 mbps tx, with no TX failures.
I am also going to test his adaptive patch once it's done building.
Thoughts?
arch/arm64/boot/dts/rockchip/rk3328-roc-cc.dts +20
-0@@ -280,6 +280,26 @@ rockchip,pins = <0 RK_PA0 RK_FUNC_GPIO &pcfg_pull_none>; }; }; + + gmac-1 { + rgmiim1_pins: rgmiim1-pins { + rockchip,pins = + /* mac_txclk */ + <0 RK_PB0 1 &pcfg_pull_none_12ma>, + /* mac_txen */ + <0 RK_PB4 1 &pcfg_pull_none_12ma>, + /* mac_clk */ + <0 RK_PD0 1 &pcfg_pull_none_12ma>, + /* mac_txd1 */ + <0 RK_PC0 1 &pcfg_pull_none_12ma>, + /* mac_txd0 */ + <0 RK_PC1 1 &pcfg_pull_none_12ma>, + /* mac_txd3 */ + <0 RK_PC7 1 &pcfg_pull_none_12ma>, + /* mac_txd2 */ + <0 RK_PC6 1 &pcfg_pull_none_12ma>; + }; + }; };Quoted 5 lines
Quoted 1 line
&sdmmc {Good find! It seems like they're there in the dtsi, but configured as none when it comes to ma. I'll test it
With Ayufan's patch offload patch and my pull patch, I now have the following results:
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 1022 MBytes 857 Mbits/sec 885 sender [ 5] 0.00-10.01 sec 1022 MBytes 856 Mbits/secreceiver
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::e8b5:5eff:fe49:d353 prefixlen 64 scopeid 0x20<link> ether ea:b5:5e:49:d3:53 txqueuelen 1000 (Ethernet) RX packets 39822 bytes 2668618 (2.6 MB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1005649 bytes 1516698158 (1.5 GB) TX errors 0 dropped 0 overruns 0 carrier 0Quoted 5 lines
collisions 0
device interrupt 29Much better overall, still not perfect, but I think it might be worth re-running the delay tests now.
Quoted 7 lines
also you could check if the thres-dma-flag is still necessary or can be removed - might be interesting to know if fixing the drive strength magically also fixes that issue.
Heiko
Quoted 6 lines
Part of my test was removing that flag.
I actually haven't run with it for a while, since I chose to disable offload in userspace instead. With threshold DMA I was having serious performance issues, but it was better than networking being completely broken.I'll write up a patch for the .dtsi and submit it today. If we can contact ayufan about his patch, that would be great!
For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <
laptop_ip> -t 30.I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity2) just this patch by itself
the ssh lag hit me, but it does seem better. The link reset still
happensQuoted 8 lines
3) the force-thresh-dma-mode patch with this one:
No shs lag, and as before, the tx activity seems to hit less?
(Maybe placebo) but the link reset does happen regularly.As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better
We have several issues on the rk3328.
The first issue, which you describe as the ssh lag, is the tx-offload issue. Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled. To fix this, you must somehow disable tx-offload.
force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498. You can also disable tx-offload from userspace with ethtool.
Quoted 5 lines
The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.
The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.
I hope this clears things up a little for you.
Yup, that's what I have in my mind already. My previous comment wasn't clear, I know. I pieced it together after your dma-mode patch, because that one fixed the ssh lag. My previous email was just what I saw after those changes.
In my case the link reset still happens with your patch.
You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work for the network instability problem?
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
A message that is not in the archive (sent off-list or not yet received): [email protected]
[clip]
Quoted 34 lines
Quoted 28 lines
Quoted 11 lines
For the test I use iperf3 -s on my laptop (Gigabit Ethernet) and on the renegade: iperf3 -c <
laptop_ip> -t 30.I tried using:
1) the force-thresh-dma-mode patch by itself with the changed tx/rx_delay. (Maybe the default one is better? It needs testing nayway) No ssh lag but I do get the Link reset on tx activity2) just this patch by itself
the ssh lag hit me, but it does seem better. The link reset still happensQuoted 7 lines
3) the force-thresh-dma-mode patch with this one:
No shs lag, and as before, the tx activity seems to hit less? (Maybe placebo) but the link reset does happen regularly.As far as I understand ayufan's offload patch removes the need for the force-thresh-dma-mode flag. I'll try ayufan's patch (adding the option to the rk3328-roc-cc as well) together with this patch. Maybe it'll be better
We have several issues on the rk3328.
The first issue, which you describe as the ssh lag, is the tx-offload issue. Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled. To fix this, you must somehow disable tx-offload.
force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498. You can also disable tx-offload from userspace with ethtool.Quoted 5 lines
The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.
The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.
I hope this clears things up a little for you.
Quoted 8 lines
Yup, that's what I have in my mind already. My previous comment wasn't clear, I know. I pieced it together after your dma-mode patch, because that one fixed the ssh lag. My previous email was just what I saw after those changes.
In my case the link reset still happens with your patch.
You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work for the network instability problem?
Correct, the link-reset tends to happen when the delays are at the edge of working.
There is also the possibility that board level manufacturing tolerances are causing issues where the delay can change between boards. If this is the case, we will have to make an auto-tune function, probably based off the mmc version.
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip
A message that is not in the archive (sent off-list or not yet received): 6d6a700a-39c9-25b6-dfff-296c0757603d-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Quoted 33 lines
Quoted 26 lines
Quoted 12 lines
We have several issues on the rk3328.
The first issue, which you describe as the ssh lag, is the tx-offload issue.
Packets that are larger than the MTU and will be fragmented break if tx-offload is enabled.
To fix this, you must somehow disable tx-offload. force-thresh-dma-mode disabled tx-offload, but had other side effects. Ayufan's patch disables tx-offload only if the packet is larger than 1498.
You can also disable tx-offload from userspace with ethtool.Quoted 7 lines
The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.
The third issue is the tx and rx delays need to be tuned now that the tx packet corruption issue has been fixed.
I hope this clears things up a little for you.
Yup, that's what I have in my mind already. My previous comment wasn't clear, I know.
I pieced it together after your dma-mode patch, because that one fixed the ssh lag.
My previous email was just what I saw after those changes.Quoted 6 lines
In my case the link reset still happens with your patch.
You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work for the network instability problem?
Quoted 6 lines
Correct, the link-reset tends to happen when the delays are at the edge of working.
There is also the possibility that board level manufacturing tolerances are causing issues where the delay can change between boards.
If this is the case, we will have to make an auto-tune function, probably based off the mmc version.
So, I tried just your patch with archlinuxARM linux 5.0 (meaning the tx/rx_delays are 0x25/0x11 respectively if I'm not mistaken) and disabled tx offload through ethtool.
After 3 30-second iperf3 tests, no link resets! I get about the same speed you mentioned before, 800-something to 700-something lowest.
Seems like the delays that are already there in the dts are better than the ones I picked up from the Armbian dts.
It's worth noting that I've never noticed a link reset in my ~3 months of using the 4.4 Rockchip kernel that Armbian packages using ayufan's linux-kernel repo. Maybe something there?
All in all, it does seem to be much better with your patch.
Στις Κυρ, 10 Μαρ, 2019 at 2:26 ΠΜ, ο/η "Leonidas P. Papadakos" <[email protected]> έγραψε:
Quoted 57 lines
Quoted 40 lines
Quoted 5 lines
Quoted 5 lines
We have several issues on the rk3328.
The first issue, which you describe as the ssh lag, is the tx-offload issue.
Packets that are larger than the MTU and will be fragmentedbreak if
Quoted 3 lines
Quoted 3 lines
tx-offload is enabled.
To fix this, you must somehow disable tx-offload. force-thresh-dma-mode disabled tx-offload, but had other sideeffects.
Quoted 1 line
Quoted 1 line
Ayufan's patch disables tx-offload only if the packet is larger
than
Quoted 7 lines
Quoted 7 lines
1498.
You can also disable tx-offload from userspace with ethtool.The second issue is the high rate of tx packets being corrupted, which my patch to set the tx pull values fixes.
The third issue is the tx and rx delays need to be tuned now
that
Quoted 5 lines
Quoted 3 lines
the tx packet corruption issue has been fixed.
I hope this clears things up a little for you.
Yup, that's what I have in my mind already. My previous comment
wasn't
Quoted 2 lines
clear, I know.
I pieced it together after your dma-mode patch, because that onefixed
Quoted 7 lines
the ssh lag.
My previous email was just what I saw after those changes.In my case the link reset still happens with your patch.
You're thinking that the combination of your patch, ayufan's "conditional" patch and a specific tweak in the delays would work
for
Quoted 2 lines
the network instability problem?
Quoted 9 lines
Correct, the link-reset tends to happen when the delays are at the
edge of
working.
There is also the possibility that board level manufacturing
tolerances are
causing issues where the delay can change between boards. If this is the case, we will have to make an auto-tune function,
probably
based off the mmc version.So, I tried just your patch with archlinuxARM linux 5.0 (meaning the tx/rx_delays are 0x25/0x11 respectively if I'm not mistaken) and disabled tx offload through ethtool.
After 3 30-second iperf3 tests, no link resets! I get about the same speed you mentioned before, 800-something to 700-something lowest.
Quoted 6 lines
Seems like the delays that are already there in the dts are better than the ones I picked up from the Armbian dts.
It's worth noting that I've never noticed a link reset in my ~3 months of using the 4.4 Rockchip kernel that Armbian packages using ayufan's linux-kernel repo. Maybe something there?
All in all, it does seem to be much better with your patch.
I've been trying to reach ayufan about his patch, getting is into mainline:
https://github.com/ayufan-rock64/linux-mainline-kernel/commit/41eeb7cd789ea7cac0cbf4b35b53055f354da757
but it seems he's not responding. I'll cc him.
His patch would be a really cool fix if it could be in 5.1, or even 5.2, as a replacement to force_thresh_dma_mode
_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip