Skip to contentSkip to conversation

net: stmmac: dwmac-rk: fix unbalanced pm_runtime_enable warnings

PATCH Applied Michael Riesch 7 participants 14 messages Aug 23, 2021 – Sep 17, 2021

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net: stmmac: dwmac-rk: fix unbalanced pm_runtime_enable warnings

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

---
 drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c | 9 ---------
 1 file changed, 9 deletions(-)
drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c +0 -9
index 280ac0129572..ed817011a94a 100644
@@ -21,7 +21,6 @@
 #include <linux/delay.h>
 #include <linux/mfd/syscon.h>
 #include <linux/regmap.h>
-#include <linux/pm_runtime.h>
 
 #include "stmmac_platform.h"
 
@@ -1529,9 +1528,6 @@ static int rk_gmac_powerup(struct rk_priv_data *bsp_priv)
 		return ret;
 	}
 
-	pm_runtime_enable(dev);
-	pm_runtime_get_sync(dev);
-
 	if (bsp_priv->integrated_phy)
 		rk_gmac_integrated_phy_powerup(bsp_priv);
 
@@ -1540,14 +1536,9 @@ static int rk_gmac_powerup(struct rk_priv_data *bsp_priv)
 
 static void rk_gmac_powerdown(struct rk_priv_data *gmac)
 {
-	struct device *dev = &gmac->pdev->dev;
-
 	if (gmac->integrated_phy)
 		rk_gmac_integrated_phy_powerdown(gmac);
 
-	pm_runtime_put_sync(dev);
-	pm_runtime_disable(dev);
-
 	phy_power_on(gmac, false);
 	gmac_clk_enable(gmac, false);
 }
Signature · 1 line
2.17.1
maintainer
#1 lore raw
Quoted 4 lines · On Mon, 23 Aug 2021 16:37:54 +0200 Michael Riesch wrote:

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

Can we get a Fixes tag? I.e. reference to the earliest commit where the warning can be triggered?

Quoted 4 lines

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

#2 lore raw

Hi Jakub,

Quoted 8 lines · Jakub Kicinski
Quoted 4 lines · On Mon, 23 Aug 2021 16:37:54 +0200 Michael Riesch wrote:

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

Can we get a Fixes tag? I.e. reference to the earliest commit where the warning can be triggered?

Of course :-)

Fixes 5ec55823438e850c91c6b92aec93fb04ebde29e2 ("net: stmmac: add

clocks management for gmac driver")

This commit introduces power management support to the stmmac core (stmmac_{dvr_probe, dvr_remove, suspend, resume}).

Best regards,
Michael

Quoted 4 lines
Quoted 4 lines

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

#3 lore raw
In reply to Michael Riesch
This patch was applied to netdev/net.git (refs/heads/master):

Hello:

This patch was applied to netdev/net.git (refs/heads/master):

Quoted 10 lines · On Mon, 23 Aug 2021 16:37:54 +0200 you wrote:

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

[...]

Here is the summary with links:
- net: stmmac: dwmac-rk: fix unbalanced pm_runtime_enable warnings

    https://git.kernel.org/netdev/net/c/2d26f6e39afb

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

#4 lore raw
In reply to Michael Riesch

Hi,

Quoted 10 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

Quoted 6 lines

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

ChenYu

#5 lore raw

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

Quoted 1 line

ChenYu

Best regards,
Michael

#6 lore raw

Hi Michael,

Quoted 22 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

I am seeing the same failure symptoms reported by ChenYu on my RockPro64 with v5.14. Reverting the revert i.e., 2d26f6e39afb ("net: stmmac:
dwmac-rk: fix unbalanced pm_runtime_enable warnings") brings back the network.

Quoted 10 lines
Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

Not sure about the other RK3399 boards but RockPro64 is easily available.

Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

I tried to look for the differences between RK3568 and RK3399 but the upstream device tree doesn't seem to carry a gmac node in the device tree for EK3568 EVB1. Do you have a pointer for the dts you're using?

Also, other than the warning "Unbalanced pm_runtime_enable!" do you notice any other ill-effects without your patch?

If this affects all RK3399 boards as ChenYu suggests quite a few users are going to miss the network once they upgrade.

Punit

[...]

#7 lore raw

Hi Punit,

Quoted 45 lines · Punit Agrawal

Hi Michael,

Quoted 22 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

I am seeing the same failure symptoms reported by ChenYu on my RockPro64 with v5.14. Reverting the revert i.e., 2d26f6e39afb ("net: stmmac:
dwmac-rk: fix unbalanced pm_runtime_enable warnings") brings back the network.

Quoted 6 lines
Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Quoted 7 lines
Quoted 4 lines

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

Not sure about the other RK3399 boards but RockPro64 is easily available.

I was thinking to get one of those anyway ;-)

Quoted 8 lines
Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

I tried to look for the differences between RK3568 and RK3399 but the upstream device tree doesn't seem to carry a gmac node in the device tree for EK3568 EVB1. Do you have a pointer for the dts you're using?

The gmac nodes have been added recently and should enter 5.15-rc1. Until then, you can check out the dts from linux-rockchip/for-next [0].

Quoted 2 lines

Also, other than the warning "Unbalanced pm_runtime_enable!" do you notice any other ill-effects without your patch?

No, not as such.

Best regards,
Michael

Quoted 6 lines

If this affects all RK3399 boards as ChenYu suggests quite a few users are going to miss the network once they upgrade.>
Punit

[...]

[0]
https://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip.git/log/?h=for-next

#8 lore raw
Quoted 63 lines · Michael Riesch

Hi Punit,

Quoted 45 lines · Punit Agrawal

Hi Michael,

Quoted 22 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

I am seeing the same failure symptoms reported by ChenYu on my RockPro64 with v5.14. Reverting the revert i.e., 2d26f6e39afb ("net: stmmac:
dwmac-rk: fix unbalanced pm_runtime_enable warnings") brings back the network.

Quoted 6 lines
Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Quoted 7 lines
Quoted 4 lines

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

Not sure about the other RK3399 boards but RockPro64 is easily available.

I was thinking to get one of those anyway ;-)

Quoted 5 lines
Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

Quoted 6 lines
Quoted 3 lines

I tried to look for the differences between RK3568 and RK3399 but the upstream device tree doesn't seem to carry a gmac node in the device tree for EK3568 EVB1. Do you have a pointer for the dts you're using?

The gmac nodes have been added recently and should enter 5.15-rc1. Until then, you can check out the dts from linux-rockchip/for-next [0].

Do you have the upstream commit?

As I compiled v5.15-rc1 and still can't get the ethernet work.

Not sure if it's my Uboot->systemd-boot->customer kernel setup not passing the device tree correctly or something else...

Thanks,
Qu

Quoted 24 lines
Quoted 3 lines

Also, other than the warning "Unbalanced pm_runtime_enable!" do you notice any other ill-effects without your patch?

No, not as such.

Best regards,
Michael

Quoted 6 lines

If this affects all RK3399 boards as ChenYu suggests quite a few users are going to miss the network once they upgrade.>
Punit

[...]

[0]
https://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip.git/log/?h=for-next

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#9 lore raw

Hi Qu,

Quoted 69 lines · Qu Wenruo
Quoted 61 lines · Michael Riesch

Hi Punit,

Quoted 45 lines · Punit Agrawal

Hi Michael,

Quoted 22 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

I am seeing the same failure symptoms reported by ChenYu on my RockPro64 with v5.14. Reverting the revert i.e., 2d26f6e39afb ("net: stmmac:
dwmac-rk: fix unbalanced pm_runtime_enable warnings") brings back the network.

Quoted 6 lines
Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Quoted 7 lines
Quoted 4 lines

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

Not sure about the other RK3399 boards but RockPro64 is easily available.

I was thinking to get one of those anyway ;-)

Quoted 5 lines
Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

Quoted 6 lines
Quoted 3 lines

I tried to look for the differences between RK3568 and RK3399 but the upstream device tree doesn't seem to carry a gmac node in the device tree for EK3568 EVB1. Do you have a pointer for the dts you're using?

The gmac nodes have been added recently and should enter 5.15-rc1. Until
then, you can check out the dts from linux-rockchip/for-next [0].

Quoted 6 lines

Do you have the upstream commit?

As I compiled v5.15-rc1 and still can't get the ethernet work.

Not sure if it's my Uboot->systemd-boot->customer kernel setup not passing the device tree correctly or something else...

For the RK3568 device tree changes, I think the pull request got delayed to the next cycle. So likely to land in v5.16.

In case you're after ethernet on RK3399, there's no solution yet. Reverting 2d26f6e39afb ("net: stmmac: dwmac-rk: fix unbalanced pm_runtime_enable warnings") gets you there in the meanwhile.

[...]

#10 lore raw
Quoted 80 lines · Punit Agrawal

Hi Qu,

Quoted 69 lines · Qu Wenruo
Quoted 61 lines · Michael Riesch

Hi Punit,

Quoted 45 lines · Punit Agrawal

Hi Michael,

Quoted 22 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

I am seeing the same failure symptoms reported by ChenYu on my RockPro64 with v5.14. Reverting the revert i.e., 2d26f6e39afb ("net: stmmac:
dwmac-rk: fix unbalanced pm_runtime_enable warnings") brings back the network.

Quoted 6 lines
Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Quoted 7 lines
Quoted 4 lines

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

Not sure about the other RK3399 boards but RockPro64 is easily available.

I was thinking to get one of those anyway ;-)

Quoted 5 lines
Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

Quoted 6 lines
Quoted 3 lines

I tried to look for the differences between RK3568 and RK3399 but the upstream device tree doesn't seem to carry a gmac node in the device tree for EK3568 EVB1. Do you have a pointer for the dts you're using?

The gmac nodes have been added recently and should enter 5.15-rc1. Until
then, you can check out the dts from linux-rockchip/for-next [0].

Quoted 6 lines

Do you have the upstream commit?

As I compiled v5.15-rc1 and still can't get the ethernet work.

Not sure if it's my Uboot->systemd-boot->customer kernel setup not passing the device tree correctly or something else...

Quoted 6 lines

For the RK3568 device tree changes, I think the pull request got delayed to the next cycle. So likely to land in v5.16.

In case you're after ethernet on RK3399, there's no solution yet. Reverting 2d26f6e39afb ("net: stmmac: dwmac-rk: fix unbalanced pm_runtime_enable warnings") gets you there in the meanwhile.

Thanks, currently I have seen other distros like ManjaroARM is already reverting that commit.

But even with that commit reverted, I still get some other strange network behavior.

The most weird one is distcc, when the RK3399 board is the client and x86_64 desktop acts as a volunteer, after compiling hundreds of files, it suddenly no longer work.

All work can no longer be distributed to the same volunteer.

But on RPI CM4 board, the same kernel (both upstream 5.14.2, even the binary is the same), the same distro (Manjaro ARM), the same distcc setup, the setup works flawless.

Not sure if this is related, but it looks like a network related problem, and considering both boards are using the same kernel, just different ethernet driver, I guess there is something more problematic here in recent RK3399 code.

Thanks,
Qu

Quoted 3 lines

[...]

_______________________________________________
Linux-rockchip mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-rockchip

#11 lore raw
Quoted 89 lines · Qu Wenruo
Quoted 76 lines · Punit Agrawal

Hi Qu,

Quoted 73 lines · Qu Wenruo
Quoted 61 lines · Michael Riesch

Hi Punit,

Quoted 45 lines · Punit Agrawal

Hi Michael,

Quoted 22 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

I am seeing the same failure symptoms reported by ChenYu on my RockPro64 with v5.14. Reverting the revert i.e., 2d26f6e39afb ("net: stmmac:
dwmac-rk: fix unbalanced pm_runtime_enable warnings") brings back the network.

Quoted 6 lines
Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Quoted 7 lines
Quoted 4 lines

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

Not sure about the other RK3399 boards but RockPro64 is easily available.

I was thinking to get one of those anyway ;-)

Quoted 5 lines
Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

Quoted 6 lines
Quoted 3 lines

I tried to look for the differences between RK3568 and RK3399 but the upstream device tree doesn't seem to carry a gmac node in the device tree for EK3568 EVB1. Do you have a pointer for the dts you're using?

The gmac nodes have been added recently and should enter 5.15-rc1. Until
then, you can check out the dts from linux-rockchip/for-next [0].

Do you have the upstream commit?

As I compiled v5.15-rc1 and still can't get the ethernet work.

Quoted 8 lines
Quoted 2 lines

Not sure if it's my Uboot->systemd-boot->customer kernel setup not passing the device tree correctly or something else...

For the RK3568 device tree changes, I think the pull request got delayed
to the next cycle. So likely to land in v5.16.
In case you're after ethernet on RK3399, there's no solution yet. Reverting 2d26f6e39afb ("net: stmmac: dwmac-rk: fix unbalanced pm_runtime_enable warnings") gets you there in the meanwhile.

Thanks, currently I have seen other distros like ManjaroARM is already reverting that commit.

Quoted 6 lines

But even with that commit reverted, I still get some other strange network behavior.

The most weird one is distcc, when the RK3399 board is the client and x86_64 desktop acts as a volunteer, after compiling hundreds of files, it suddenly no longer work.

I haven't seen something like this - but then I am not a heavy user of the network on the board.

Is it just the network that dies or the whole system freezes? Sometimes I've seen the board lock up if it's under heavy load.

Quoted 8 lines

All work can no longer be distributed to the same volunteer.

But on RPI CM4 board, the same kernel (both upstream 5.14.2, even the binary is the same), the same distro (Manjaro ARM), the same distcc setup, the setup works flawless.

Quoted 4 lines

Not sure if this is related, but it looks like a network related problem, and considering both boards are using the same kernel, just different ethernet driver, I guess there is something more problematic here in recent RK3399 code.

If you are only seeing the problem with recent kernels, maybe a bisect might help narrow things down.

[...]

#12 lore raw
In reply to Michael Riesch

Hi,

Quoted 33 lines · Michael Riesch

Hi ChenYu,

Quoted 19 lines · Chen-Yu Tsai

Hi,

Quoted 15 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 4 lines
Quoted 1 line

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

Cc Sasha as this patch has just been applied to 5.13-stable.
Quoted 8 lines
Quoted 3 lines

The symptom I see is no DHCP responses, either because the request isn't getting sent over the wire, or the response isn't getting received. The PHY seems to be working correctly.

Unfortunately I don't have any RK3399 hardware. Is this a custom board/special hardware or something that is readily available in the shops? Maybe this is a good reason to buy a RK3399 based single-board computer :-)

I hit this on an ROC-RK3399-PC first, then bisected it on a NanoPi M4V2. They both should be easy to get from their respective vendors. AFAIK all the RK3399 devices have pretty much the same setup for Ethernet.

BTW, Don't get the Nanopi in the M4V2 variant. Get the original M4, or the M4B (but the WiFi on the M4B doesn't seem to work lately).

Quoted 4 lines

I am working on the RK3568 EVB1 and have not encountered faulty behavior. DHCP works fine and I can boot via NFS. Therefore, not sure whether I can be much of help in this matter, but in case you want to discuss this further please do not hesitate to contact me off-list.

My gut is telling me (without looking at the code) that maybe the GRF access was not going through, and so the RGMII delays and stuff weren't set properly, hence no traffic getting to/from the PHY. Otherwise there were no error messages. I don't think I'll be able to squeeze out more cycles to track down what actually went wrong though.

Regards
ChenYu

#13 lore raw
In reply to Michael Riesch
Quoted 24 lines · Michael Riesch

Hi ChenYu,

Quoted 21 lines · Chen-Yu Tsai

Hi,

Quoted 16 lines · Michael Riesch

This reverts commit 2c896fb02e7f65299646f295a007bda043e0f382 "net: stmmac: dwmac-rk: add pd_gmac support for rk3399" and fixes unbalanced pm_runtime_enable warnings.

In the commit to be reverted, support for power management was introduced to the Rockchip glue code. Later, power management support was introduced to the stmmac core code, resulting in multiple invocations of pm_runtime_{enable,disable,get_sync,put_sync}.

The multiple invocations happen in rk_gmac_powerup and stmmac_{dvr_probe, resume} as well as in rk_gmac_powerdown and stmmac_{dvr_remove, suspend}, respectively, which are always called in conjunction.

Quoted 7 lines
Quoted 2 lines

I just found that Ethernet stopped working on my RK3399 devices, and I bisected it down to this patch.

Oh dear. First patch in a kernel release for a while and I already break things.

Cc Sasha as this patch has just been applied to 5.13-stable.

I'll drop it for now, thanks for letting me know!

Signature · 2 lines
Thanks,
Sasha