Skip to contentSkip to conversation

arm64: dts: rockchip: Fix rk3328 rgmii high tx error rate

PATCH In review Peter Geis 4 participants 7 messages Mar 9, 2019 – Mar 13, 2019 5 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Peter Geis

author
#0 lore raw

PATCH 1/1 arm64: dts: rockchip: Fix rk3328 rgmii high tx error rate

Several rk3328 based boards experience high rgmii tx error rates. This is due to several pins in the rk3328.dtsi rgmii pinmux that are missing a pull level setting.
This causes the pinmux driver to default to 0ma.

Fix this by setting those pins to 12ma, consistent with the other tx pins. This allows much higher data rates with much fewer retries and no recorded tx errors.

Tested on the rk3328-roc-cc board.

---
 arch/arm64/boot/dts/rockchip/rk3328.dtsi | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
arch/arm64/boot/dts/rockchip/rk3328.dtsi +7 -7
index 84f14b132e8f..48a4477ebe58 100644
@@ -1673,19 +1673,19 @@
 					<1 RK_PC1 2 &pcfg_pull_none_12ma>,
 
 					/* mac_txclk */
-					<0 RK_PB0 1 &pcfg_pull_none>,
+					<0 RK_PB0 1 &pcfg_pull_none_12ma>,
 					/* mac_txen */
-					<0 RK_PB4 1 &pcfg_pull_none>,
+					<0 RK_PB4 1 &pcfg_pull_none_12ma>,
 					/* mac_clk */
-					<0 RK_PD0 1 &pcfg_pull_none>,
+					<0 RK_PD0 1 &pcfg_pull_none_12ma>,
 					/* mac_txd1 */
-					<0 RK_PC0 1 &pcfg_pull_none>,
+					<0 RK_PC0 1 &pcfg_pull_none_12ma>,
 					/* mac_txd0 */
-					<0 RK_PC1 1 &pcfg_pull_none>,
+					<0 RK_PC1 1 &pcfg_pull_none_12ma>,
 					/* mac_txd3 */
-					<0 RK_PC7 1 &pcfg_pull_none>,
+					<0 RK_PC7 1 &pcfg_pull_none_12ma>,
 					/* mac_txd2 */
-					<0 RK_PC6 1 &pcfg_pull_none>;
+					<0 RK_PC6 1 &pcfg_pull_none_12ma>;
 			};
 
 			rmiim1_pins: rmiim1-pins {
Signature · 1 line
2.20.1

A message that is not in the archive (sent off-list or not yet received): 20190309182013.22162-1-pgwipeout-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#2 lore raw

In the rk3328.dtsi, the previous /* mac_clk */ commented value is using 2ma, maybe the one you patch here should be 2ma as well?

A message that is not in the archive (sent off-list or not yet received): 20190310004722.5140-1-papadakospan-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#4 lore raw

I do have to say, This patch does fix my Link Reset issue reliably enough, so it does *something*

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw
Quoted 3 lines

I do have to say, This patch does fix my Link Reset issue reliably enough, so it does *something*

Well, I just tried the same things I did yesterday (iperf3 testts, GbE) and it seems to work even without the patch. (?) Using the defailt tx/rx delays though. (0x25,0x11)

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw
Quoted 10 lines · Leonidas P. Papadakos
Quoted 5 lines

I do have to say, This patch does fix my Link Reset issue reliably enough, so it does *something*

Well, I just tried the same things I did yesterday (iperf3 testts, GbE) and it seems to work even without the patch. (?)
Using the defailt tx/rx delays though. (0x25,0x11)

Good Morning,

So after getting Robin's response and the follow on responses, I've dug further into the RK3328 TRM, Specifications, and the rk3328-roc-cc schematics.

I've made the following discoveries:
The labels in the rk3328.dtsi for those pins are inaccurate. For instance the first of the pins missing a drive setting RK_PB0 is labeled "mac_txclk", but it is in fact "gmac_txd1".

What I thought was 0 ma is in fact 2 ma, which is the result of both bits being 0, apologies for the incorrect information.
This is the resulting state following the pinmux driver initialization. This is still very low for the function performed, and is what is causing our tx issues.
Essentially what we have is some of the tx lines are running at 12ma and some running at 2ma, which can cause imbalance issues.

This also calls into question if 12ma is the correct drive strength, or if we are driving too hard, leading to ringing on the interface. The highest default drive strength is the sd interfaces at 8ma. I've actually been running the sd interface at this level for a while, since the 4ma setting in the current dtsi was causing data corruption.

I will conduct further tests, and resubmit the patch with the labels corrected, as well as other fixes.
This may become a patch series, since I will be going through all of the pins and their expected values in the TRM.

A message that is not in the archive (sent off-list or not yet received): CAMdYzYrn2eEr=JzNFe3wp-VjoVHwxcQkf98zc0MSMh4j=0ZDvg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#10 lore raw
Quoted 38 lines · Peter Geis
Quoted 10 lines · Leonidas P. Papadakos
Quoted 5 lines

I do have to say, This patch does fix my Link Reset issue reliably enough, so it does *something*

Well, I just tried the same things I did yesterday (iperf3 testts, GbE) and it seems to work even without the patch. (?)
Using the defailt tx/rx delays though. (0x25,0x11)

Good Morning,

So after getting Robin's response and the follow on responses, I've dug further into the RK3328 TRM, Specifications, and the rk3328-roc-cc schematics.

I've made the following discoveries:
The labels in the rk3328.dtsi for those pins are inaccurate. For instance the first of the pins missing a drive setting RK_PB0 is labeled "mac_txclk", but it is in fact "gmac_txd1".

What I thought was 0 ma is in fact 2 ma, which is the result of both bits being 0, apologies for the incorrect information.
This is the resulting state following the pinmux driver initialization. This is still very low for the function performed, and is what is causing our tx issues.
Essentially what we have is some of the tx lines are running at 12ma and some running at 2ma, which can cause imbalance issues.

This also calls into question if 12ma is the correct drive strength, or if we are driving too hard, leading to ringing on the interface. The highest default drive strength is the sd interfaces at 8ma. I've actually been running the sd interface at this level for a while, since the 4ma setting in the current dtsi was causing data corruption.

Quoted 5 lines

I will conduct further tests, and resubmit the patch with the labels corrected, as well as other fixes.
This may become a patch series, since I will be going through all of the pins and their expected values in the TRM.

I need to apologize again, I was incorrect about the pins missing drive controls.
I misread them as gpio1 and not gpio0, but now I am even more confused.
The whole group of pins on gpio0, B0, B4, C0, C1, C6, C7, and D0, as far as the rk3328 TRM V1.1, Datasheet Rev 1.2, and the rk3328-roc-cc schematics are concerned do not exist. That whole block of gpio config registers in the TRM are simply "reserved", and they are referenced nowhere else.

So here's the kicker, if you get rid of them, the rgmii interface fails to work. It enumerates, and the mac is detected, but no data passes successfully.

I've also conducted some testing with the following patch, and I now get almost full speed with absolutely zero retries or packet errors. I think it's safe to say that the interface was likely ringing, but I'd like some more tests to be sure.

Please give me your feedback, one and all!

arch/arm64/boot/dts/rockchip/rk3328.dtsi +22 -22
index 84f14b132e8f..c55a3f1a87ff 100644
@@ -1642,50 +1642,50 @@
 			rgmiim1_pins: rgmiim1-pins {
 				rockchip,pins =
 					/* mac_txclk */
-					<1 RK_PB4 2 &pcfg_pull_none_12ma>,
+					<1 RK_PB4 2 &pcfg_pull_none_8ma>,
 					/* mac_rxclk */
-					<1 RK_PB5 2 &pcfg_pull_none_2ma>,
+					<1 RK_PB5 2 &pcfg_pull_none_4ma>,
 					/* mac_mdio */
-					<1 RK_PC3 2 &pcfg_pull_none_2ma>,
+					<1 RK_PC3 2 &pcfg_pull_none_4ma>,
 					/* mac_txen */
-					<1 RK_PD1 2 &pcfg_pull_none_12ma>,
+					<1 RK_PD1 2 &pcfg_pull_none_8ma>,
 					/* mac_clk */
-					<1 RK_PC5 2 &pcfg_pull_none_2ma>,
+					<1 RK_PC5 2 &pcfg_pull_none_4ma>,
 					/* mac_rxdv */
-					<1 RK_PC6 2 &pcfg_pull_none_2ma>,
+					<1 RK_PC6 2 &pcfg_pull_none_4ma>,
 					/* mac_mdc */
-					<1 RK_PC7 2 &pcfg_pull_none_2ma>,
+					<1 RK_PC7 2 &pcfg_pull_none_4ma>,
 					/* mac_rxd1 */
-					<1 RK_PB2 2 &pcfg_pull_none_2ma>,
+					<1 RK_PB2 2 &pcfg_pull_none_4ma>,
 					/* mac_rxd0 */
-					<1 RK_PB3 2 &pcfg_pull_none_2ma>,
+					<1 RK_PB3 2 &pcfg_pull_none_4ma>,
 					/* mac_txd1 */
-					<1 RK_PB0 2 &pcfg_pull_none_12ma>,
+					<1 RK_PB0 2 &pcfg_pull_none_8ma>,
 					/* mac_txd0 */
-					<1 RK_PB1 2 &pcfg_pull_none_12ma>,
+					<1 RK_PB1 2 &pcfg_pull_none_8ma>,
 					/* mac_rxd3 */
-					<1 RK_PB6 2 &pcfg_pull_none_2ma>,
+					<1 RK_PB6 2 &pcfg_pull_none_4ma>,
 					/* mac_rxd2 */
-					<1 RK_PB7 2 &pcfg_pull_none_2ma>,
+					<1 RK_PB7 2 &pcfg_pull_none_4ma>,
 					/* mac_txd3 */
-					<1 RK_PC0 2 &pcfg_pull_none_12ma>,
+					<1 RK_PC0 2 &pcfg_pull_none_8ma>,
 					/* mac_txd2 */
-					<1 RK_PC1 2 &pcfg_pull_none_12ma>,
+					<1 RK_PC1 2 &pcfg_pull_none_8ma>,
 
 					/* mac_txclk */
-					<0 RK_PB0 1 &pcfg_pull_none>,
+					<0 RK_PB0 1 &pcfg_pull_none_8ma>,
 					/* mac_txen */
-					<0 RK_PB4 1 &pcfg_pull_none>,
+					<0 RK_PB4 1 &pcfg_pull_none_8ma>,
 					/* mac_clk */
-					<0 RK_PD0 1 &pcfg_pull_none>,
+					<0 RK_PD0 1 &pcfg_pull_none_4ma>,
 					/* mac_txd1 */
-					<0 RK_PC0 1 &pcfg_pull_none>,
+					<0 RK_PC0 1 &pcfg_pull_none_8ma>,
 					/* mac_txd0 */
-					<0 RK_PC1 1 &pcfg_pull_none>,
+					<0 RK_PC1 1 &pcfg_pull_none_8ma>,
 					/* mac_txd3 */
-					<0 RK_PC7 1 &pcfg_pull_none>,
+					<0 RK_PC7 1 &pcfg_pull_none_8ma>,
 					/* mac_txd2 */
-					<0 RK_PC6 1 &pcfg_pull_none>;
+					<0 RK_PC6 1 &pcfg_pull_none_8ma>;
 			};
 
 			rmiim1_pins: rmiim1-pins {
#11 lore raw
In reply to Peter Geis
Quoted 4 lines · Peter Geis

Several rk3328 based boards experience high rgmii tx error rates. This is due to several pins in the rk3328.dtsi rgmii pinmux that are missing a pull level setting.
This causes the pinmux driver to default to 0ma.

Hmm, according to the TRM, there is no 0ma setting; only 2, 4, 8, or 12...

Quoted 29 lines

Fix this by setting those pins to 12ma, consistent with the other tx pins. This allows much higher data rates with much fewer retries and no recorded tx errors.

Tested on the rk3328-roc-cc board.

---
  arch/arm64/boot/dts/rockchip/rk3328.dtsi | 14 +++++++-------
  1 file changed, 7 insertions(+), 7 deletions(-)
arch/arm64/boot/dts/rockchip/rk3328.dtsi +3 -4
index 84f14b132e8f..48a4477ebe58 100644
@@ -1673,19 +1673,19 @@
  					<1 RK_PC1 2 &pcfg_pull_none_12ma>,
 
  					/* mac_txclk */
-					<0 RK_PB0 1 &pcfg_pull_none>,
+					<0 RK_PB0 1 &pcfg_pull_none_12ma>,
  					/* mac_txen */
-					<0 RK_PB4 1 &pcfg_pull_none>,
+					<0 RK_PB4 1 &pcfg_pull_none_12ma>,
  					/* mac_clk */
-					<0 RK_PD0 1 &pcfg_pull_none>,
+					<0 RK_PD0 1 &pcfg_pull_none_12ma>,
  					/* mac_txd1 */
-					<0 RK_PC0 1 &pcfg_pull_none>,

arch/arm64/boot/dts/rockchip/rk3328.dtsi line 1688

Quoted 10 lines
arch/arm64/boot/dts/rockchip/rk3328.dtsi +4 -3
+					<0 RK_PC0 1 &pcfg_pull_none_12ma>,
  					/* mac_txd0 */
-					<0 RK_PC1 1 &pcfg_pull_none>,
+					<0 RK_PC1 1 &pcfg_pull_none_12ma>,
  					/* mac_txd3 */
-					<0 RK_PC7 1 &pcfg_pull_none>,
+					<0 RK_PC7 1 &pcfg_pull_none_12ma>,
  					/* mac_txd2 */
-					<0 RK_PC6 1 &pcfg_pull_none>;
+					<0 RK_PC6 1 &pcfg_pull_none_12ma>;

...but weirder than that, according to the datasheet none of these are actual pins anyway - those are the GPIO1B/C/D entries listed above this set - so it's not really clear what might be going on here, but it doesn't seem as straightforward as the commit message implies.

The TRM lists the entire IOMUX registers for GPIO0B and GPOIO0C as reserved, and doesn't even list drive strength registers corresponding to these banks at all. What's interesting is that Rockchip's 4.4 BSP kernel implies that these pins might have actually existed at some point during the chip's development[1], and digging right back into the 3.10 BSP suggests there is some non-obvious interaction between the two configs[2][3]. So I guess there's a question of whether this patch is really doing what we think it's doing, and whether what it does do is truly appropriate to apply as the SoC-level default.

Robin.

[1] https://github.com/rockchip-linux/kernel/commit/8aceffc885c7d4f3f35d09d56d22ba47e64aad5b
[2] https://github.com/rockchip-linux/kernel/commit/727de6da39ee62bb5b1252141a53ed027d5609f8
[3] https://github.com/rockchip-linux/kernel/commit/8e6b7f85dd5b451e57f220922a0ca1caecd71a94

Quoted 4 lines
arch/arm64/boot/dts/rockchip/rk3328.dtsi +0 -0
  			};
 
  			rmiim1_pins: rmiim1-pins {