Skip to contentSkip to conversation

net: stmmac: resume PHY before reopening the interface on MTU change

PATCH In review Stefan Agner 3 participants 5 messages Jul 7

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net: stmmac: resume PHY before reopening the interface on MTU change

Since the referenced commit, changing the MTU on a running interface no longer disconnects and reconnects the PHY; __stmmac_release() merely stops phylink, which also suspends the PHY (BMCR power-down) when WoL is not enabled. __stmmac_open() then performs the DMA software reset in stmmac_hw_setup() before phylink_start() resumes the PHY again.

IEEE 802.3 22.2.4.1.5 allows a PHY to stop its receive clock while powered down, and stmmac requires a running receive clock for the DMA software reset to complete (the phylink config sets mac_requires_rxc). On such setups, e.g. the RK3566-based Home Assistant Green with an RTL8211F-VD PHY in RGMII mode, any runtime MTU change now times out and leaves the interface dead:

rk_gmac-dwmac fe010000.ethernet end0: Failed to reset the dma rk_gmac-dwmac fe010000.ethernet end0: stmmac_hw_setup: DMA engine initialization failed rk_gmac-dwmac fe010000.ethernet end0: __stmmac_open: Hw setup failed rk_gmac-dwmac fe010000.ethernet end0: failed reopening the interface after MTU change

In the field this is triggered by NetworkManager applying an MTU while activating the connection, breaking networking entirely.

Resume the PHY before reopening the interface, like stmmac_resume() does, to ensure the receive clock is running for the DMA software reset.

Fixes db299a0c09e9 ("net: stmmac: move PHY handling out of __stmmac_open()/release()")
Assisted-by Claude:claude-fable-5

---
Note: phylink_prepare_resume()'s kernel-doc says it is to be called prior to phylink_resume(); here it is paired with phylink_start() (called from __stmmac_open()) instead, which phylink_resume() itself uses to restart the machinery. If preferred, I can extend the kernel-doc or introduce a more generically named helper.

 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +9 -0
@@ -5884,6 +5884,15 @@
 
 		__stmmac_release(dev);
 
+		/* phylink_stop() in __stmmac_release() suspends the PHY.
+		 * IEEE 802.3 allows PHYs to stop their receive clock while
+		 * powered down, but the DMA software reset performed by
+		 * stmmac_hw_setup() requires a running receive clock.
+		 * Resume the PHY, as on system resume, to ensure its clocks
+		 * are running before reopening the interface.
+		 */
+		phylink_prepare_resume(priv->phylink);
+
 		ret = __stmmac_open(dev, dma_conf);
 		if (ret) {
 			free_dma_desc_resources(priv, dma_conf);
Signature · 1 line
2.49.0
#1 lore raw
Quoted 5 lines · Stefan Agner
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +0 -0
@@ -5884,6 +5884,15 @@
 

drivers/net/ethernet/stmicro/stmmac/stmmac_main.c line 5894

Quoted 10 lines
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +8 -0
 		__stmmac_release(dev);
 
+		/* phylink_stop() in __stmmac_release() suspends the PHY.
+		 * IEEE 802.3 allows PHYs to stop their receive clock while
+		 * powered down, but the DMA software reset performed by
+		 * stmmac_hw_setup() requires a running receive clock.
+		 * Resume the PHY, as on system resume, to ensure its clocks
+		 * are running before reopening the interface.
+		 */
+		phylink_prepare_resume(priv->phylink);

Does it work without warnings? Nothing in dmesg? phylink_prepare_resume() does have ASSERT_RTNL() which is not called anywhere.

BR
Jakub Raczynski

#2 lore raw
Quoted 8 lines · Jakub Raczynski
Quoted 7 lines · Stefan Agner
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +0 -0
@@ -5884,6 +5884,15 @@
 
 		__stmmac_release(dev);
 
Quoted 10 lines

drivers/net/ethernet/stmicro/stmmac/stmmac_main.c line 5894

Quoted 8 lines
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +8 -0
+		/* phylink_stop() in __stmmac_release() suspends the PHY.
+		 * IEEE 802.3 allows PHYs to stop their receive clock while
+		 * powered down, but the DMA software reset performed by
+		 * stmmac_hw_setup() requires a running receive clock.
+		 * Resume the PHY, as on system resume, to ensure its clocks
+		 * are running before reopening the interface.
+		 */
+		phylink_prepare_resume(priv->phylink);

Does it work without warnings? Nothing in dmesg?

With this patch applied, this is the dmesg log on MTU change:

[   56.761175] rk_gmac-dwmac fe010000.ethernet end0: Register
MEM_TYPE_PAGE_POOL RxQ-0
[   56.762769] rk_gmac-dwmac fe010000.ethernet end0: Link is Down
[   56.801327] dwmac4: Master AXI performs any burst length
[   56.801368] rk_gmac-dwmac fe010000.ethernet end0: No Safety Features
support found
[   56.801409] rk_gmac-dwmac fe010000.ethernet end0: IEEE 1588-2008
Advanced Timestamp supported
[   56.801780] rk_gmac-dwmac fe010000.ethernet end0: registered PTP
clock
[   56.801804] rk_gmac-dwmac fe010000.ethernet end0: configuring for
phy/rgmii link mode
[   61.032985] rk_gmac-dwmac fe010000.ethernet end0: Link is Up -

1Gbps/Full - flow control off

Quoted 1 line

phylink_prepare_resume() does have ASSERT_RTNL() which is not called anywhere.

I am not very familiar with the codebase, but Fable states:

Quoted 4 lines

RTNL is held here: ndo_change_mtu is invoked from netif_set_mtu_ext(), which calls netdev_ops_assert_locked() — for a driver without instance locking such as stmmac that is ASSERT_RTNL() — right before calling the driver op.

What I can definitely confirm is that without this patch applied, a MTU change does not succeed, with it applied MTU change works and the network interface remains functional.

--
Stefan

#3 lore raw
In reply to Stefan Agner
Quoted 49 lines · Stefan Agner

Since the referenced commit, changing the MTU on a running interface no longer disconnects and reconnects the PHY; __stmmac_release() merely stops phylink, which also suspends the PHY (BMCR power-down) when WoL is not enabled. __stmmac_open() then performs the DMA software reset in stmmac_hw_setup() before phylink_start() resumes the PHY again.

IEEE 802.3 22.2.4.1.5 allows a PHY to stop its receive clock while powered down, and stmmac requires a running receive clock for the DMA software reset to complete (the phylink config sets mac_requires_rxc). On such setups, e.g. the RK3566-based Home Assistant Green with an RTL8211F-VD PHY in RGMII mode, any runtime MTU change now times out and leaves the interface dead:

rk_gmac-dwmac fe010000.ethernet end0: Failed to reset the dma rk_gmac-dwmac fe010000.ethernet end0: stmmac_hw_setup: DMA engine initialization failed rk_gmac-dwmac fe010000.ethernet end0: __stmmac_open: Hw setup failed rk_gmac-dwmac fe010000.ethernet end0: failed reopening the interface after MTU change

In the field this is triggered by NetworkManager applying an MTU while activating the connection, breaking networking entirely.

Resume the PHY before reopening the interface, like stmmac_resume() does, to ensure the receive clock is running for the DMA software reset.

Fixes db299a0c09e9 ("net: stmmac: move PHY handling out of __stmmac_open()/release()")
Assisted-by Claude:claude-fable-5

---
Note: phylink_prepare_resume()'s kernel-doc says it is to be called prior to phylink_resume(); here it is paired with phylink_start() (called from __stmmac_open()) instead, which phylink_resume() itself uses to restart the machinery. If preferred, I can extend the kernel-doc or introduce a more generically named helper.

 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +2 -0
@@ -5884,6 +5884,15 @@
 
 		__stmmac_release(dev);
 
+		/* phylink_stop() in __stmmac_release() suspends the PHY.
+		 * IEEE 802.3 allows PHYs to stop their receive clock while

drivers/net/ethernet/stmicro/stmmac/stmmac_main.c line 5898

Quoted 10 lines
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +7 -0
+		 * powered down, but the DMA software reset performed by
+		 * stmmac_hw_setup() requires a running receive clock.
+		 * Resume the PHY, as on system resume, to ensure its clocks
+		 * are running before reopening the interface.
+		 */
+		phylink_prepare_resume(priv->phylink);
+
 		ret = __stmmac_open(dev, dma_conf);
 		if (ret) {
 			free_dma_desc_resources(priv, dma_conf);

I'm not convinced.

__stmmac_open() and __stmmac_release() should be opposites of each other. If __stmmac_release() stops the clock, __stmmac_open() should start the clock.

Andrew

#4 lore raw
Quoted 63 lines · Andrew Lunn
Quoted 49 lines · Stefan Agner

Since the referenced commit, changing the MTU on a running interface no longer disconnects and reconnects the PHY; __stmmac_release() merely stops phylink, which also suspends the PHY (BMCR power-down) when WoL is not enabled. __stmmac_open() then performs the DMA software reset in stmmac_hw_setup() before phylink_start() resumes the PHY again.

IEEE 802.3 22.2.4.1.5 allows a PHY to stop its receive clock while powered down, and stmmac requires a running receive clock for the DMA software reset to complete (the phylink config sets mac_requires_rxc). On such setups, e.g. the RK3566-based Home Assistant Green with an RTL8211F-VD PHY in RGMII mode, any runtime MTU change now times out and leaves the interface dead:

rk_gmac-dwmac fe010000.ethernet end0: Failed to reset the dma rk_gmac-dwmac fe010000.ethernet end0: stmmac_hw_setup: DMA engine initialization failed rk_gmac-dwmac fe010000.ethernet end0: __stmmac_open: Hw setup failed rk_gmac-dwmac fe010000.ethernet end0: failed reopening the interface after MTU change

In the field this is triggered by NetworkManager applying an MTU while activating the connection, breaking networking entirely.

Resume the PHY before reopening the interface, like stmmac_resume() does, to ensure the receive clock is running for the DMA software reset.

Fixes db299a0c09e9 ("net: stmmac: move PHY handling out of __stmmac_open()/release()")
Assisted-by Claude:claude-fable-5

---
Note: phylink_prepare_resume()'s kernel-doc says it is to be called prior to phylink_resume(); here it is paired with phylink_start() (called from __stmmac_open()) instead, which phylink_resume() itself uses to restart the machinery. If preferred, I can extend the kernel-doc or introduce a more generically named helper.

 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +2 -0
@@ -5884,6 +5884,15 @@
 
 		__stmmac_release(dev);
 
+		/* phylink_stop() in __stmmac_release() suspends the PHY.
+		 * IEEE 802.3 allows PHYs to stop their receive clock while

drivers/net/ethernet/stmicro/stmmac/stmmac_main.c line 5898

Quoted 10 lines
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +7 -0
+		 * powered down, but the DMA software reset performed by
+		 * stmmac_hw_setup() requires a running receive clock.
+		 * Resume the PHY, as on system resume, to ensure its clocks
+		 * are running before reopening the interface.
+		 */
+		phylink_prepare_resume(priv->phylink);
+
 		ret = __stmmac_open(dev, dma_conf);
 		if (ret) {
 			free_dma_desc_resources(priv, dma_conf);

I'm not convinced.

Quoted 3 lines

__stmmac_open() and __stmmac_release() should be opposites of each other. If __stmmac_release() stops the clock, __stmmac_open() should start the clock.

Hm, I see. __stmmac_release() calls phylink_stop(). But from what I can tell we can't simply move phylink_start() in __stmmac_open() since it does too much. So we need to use phylink_prepare_resume() in __stmmac_open(), so there is still some asymmetry. I'll send a v2.

--
Stefan