Skip to contentSkip to conversation

net: improve stmmac resume rx clocking

PATCH Applied Russell King (Oracle) 3 participants 7 messages Mar 20, 2025 – Mar 25, 2025

Subsystems (7)
lore
Share & tools
lore

Russell King (Oracle)

author maintainer
#0 lore raw

cover net: improve stmmac resume rx clocking

Hi,

stmmac has had a long history of problems with resuming, illustrated by reset failure due to the receive clock not running.

Several attempts have been attempted over the years to address this issue, such as moving phylink_start() (now phylink_resume()) super early in stmmac_resume() in commit 90702dcd19c0 ("net: stmmac: fix MAC not working when system resume back with WoL a ctive.") However, this has the downside that stmmac_mac_link_up() can (and demonstrably is) called before or during the driver initialisation in another thread. This can cause issues as packets could begin to be queued, and the transmit/receive enable bits will be set before any initialisation has been done.

Another attempt is used by dwmac-socfpga.c in commit 2d871aa07136 ("net:
stmmac: add platform init/exit for Altera's ARM socfpga") which pre-dates the above commit.

Neither of these two approaches consider the effect of EEE with a PHY that supports receive clock-stop and has that feature enabled (which the stmmac driver does enable). If the link is up, then there is the possibility for the receive path to be in low-power mode, and the PHY may stop its receive clock.

This series addresses these issues by (each is not necessarily a separate patch):

1) introducing phylink_prepare_resume(), which can be used by MAC drivers to ensure that the PHY is resumed prior to doing any re-initialisation work. This call is added to stmmac_resume().

2) moving phylink_resume() after all re-initialisation has completed, thereby ensuring that the hardware is ready to be enabled for packet reception/transmission.

3) with (1) and (2) addressed, the need for socfpga to have a private work-around is no longer necessary, so it is removed.

4) introducing phylink functions to block/unblock the receive clock- stop at the PHY. As these require PHY access over the MDIO bus, they can sleep, so are not suitable for atomic access.

5) the stmmac hardware requires the receive clock to be running for reset to complete. Depending on synthesis options, this requirement may also extend to writing various registers as well, e.g. setting the MAC address, writing some of the vlan registers, etc. Full details are in the databook.

We add blocking/unblocking of the PHY receive clock-stop around parts of the main stmmac driver where we have a context that we can sleep. These are wrapped with the new phylink functions.

However, depending on synthesis options, there could be other places where the net core calls the driver with a BH-disabled context where we can't sleep, and thus can't block the PHY from disabling its receive clock. These are documented with FIXME comments.

Given the last paragraph above, I am wondering whether a better approach would be to ensure that receive clock-stop is always disabled at the PHY with stmmac. From what I can see, implementations do not document to this level of detail, which makes it difficult to tell which registers require the receive clock to be running to behave correctly.

This patch series has been tested on the Tegra194 Jetson Xavier NX board kindly donated by NVidia, with two additional patches that are pending in patchwork - the first is required to have EEE's LPI mode passed through to the MAC on this platform to allow testing under PHY clock-stop scenarios. The second is a bug fix for PHYLIB and makes "eee off" functional, but should not affect this series.

All patches on top of net-next commit f749448ce9f1 ("Merge branch 'net-mlx5-hw-steering-cleanups'")

https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/
https://patchwork.kernel.org/project/netdevbpf/patch/[email protected]/

 .../net/ethernet/stmicro/stmmac/dwmac-socfpga.c    | 18 -----
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c  | 46 ++++++++++--
 drivers/net/phy/phylink.c                          | 84 ++++++++++++++++++++++
 include/linux/phylink.h                            |  4 ++
 4 files changed, 129 insertions(+), 23 deletions(-)
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!
#1 lore raw

PATCH 1/5 net: phylink: add phylink_prepare_resume()

When the system is suspended, the PHY may be placed in low-power mode by setting the BMCR 0.11 Power down bit. IEEE 802.3 states that the behaviour of the PHY in this state is implementation specific, and the PHY is not required to meet the RX_CLK and TX_CLK requirements. Essentially, this means that a PHY may stop the clocks that it is generating while in power down state.

However, MACs exist which require the clocks from the PHY to be running in order to properly resume. phylink_prepare_resume() provides them with a way to clear the Power down bit early.

Note, however, that IEEE 802.3 gives PHYs up to 500ms grace before the transmit and receive clocks meet the requirements after clearing the power down bit.

Add a resume preparation function, which will ensure that the receive clock from the PHY is appropriately configured while resuming.

---
 drivers/net/phy/phylink.c | 25 +++++++++++++++++++++++++
 include/linux/phylink.h   |  1 +
 2 files changed, 26 insertions(+)
drivers/net/phy/phylink.c +25 -0
index 0f70a7f3dfcc..1b76ddc286fe 100644
@@ -2481,6 +2481,31 @@ void phylink_suspend(struct phylink *pl, bool mac_wol)
 }
 EXPORT_SYMBOL_GPL(phylink_suspend);
 
+/**
+ * phylink_prepare_resume() - prepare to resume a network device
+ * @pl: a pointer to a &struct phylink returned from phylink_create()
+ *
+ * Optional, but if called must be called prior to phylink_resume().
+ *
+ * Prepare to resume a network device, preparing the PHY as necessary.
+ */
+void phylink_prepare_resume(struct phylink *pl)
+{
+	struct phy_device *phydev = pl->phydev;
+
+	ASSERT_RTNL();
+
+	/* IEEE 802.3 22.2.4.1.5 allows PHYs to stop their receive clock
+	 * when PDOWN is set. However, some MACs require RXC to be running
+	 * in order to resume. If the MAC requires RXC, and we have a PHY,
+	 * then resume the PHY. Note that 802.3 allows PHYs 500ms before
+	 * the clock meets requirements. We do not implement this delay.
+	 */
+	if (pl->config->mac_requires_rxc && phydev && phydev->suspended)
+		phy_resume(phydev);
+}
+EXPORT_SYMBOL_GPL(phylink_prepare_resume);
+
 /**
  * phylink_resume() - handle a network device resume event
  * @pl: a pointer to a &struct phylink returned from phylink_create()
include/linux/phylink.h +1 -0
index 79876c84ae81..06f1b649f173 100644
@@ -707,6 +707,7 @@ void phylink_start(struct phylink *);
 void phylink_stop(struct phylink *);
 
 void phylink_suspend(struct phylink *pl, bool mac_wol);
+void phylink_prepare_resume(struct phylink *pl);
 void phylink_resume(struct phylink *pl);
 
 void phylink_ethtool_get_wol(struct phylink *, struct ethtool_wolinfo *);
Signature · 1 line
2.30.2
#2 lore raw

PATCH 2/5 net: stmmac: address non-LPI resume failures properly

In reply to Russell King (Oracle)

The Synopsys Designware GMAC core databook requires all clocks to be active in order to complete software reset, which we perform during resume.

However, IEEE 802.3 allows a PHY to stop its clocks when placed in low-power mode, which happens when the system is suspended and WoL is not enabled.

As an attempt to work around this, commit 36d18b5664ef ("net: stmmac:
start phylink instance before stmmac_hw_setup()") started phylink early, but this has the side effect that the mac_link_up() method may be called before or during the initialisation of GMAC hardware.

We also have the socfpga glue driver directly calling phy_resume() also as an attempt to work around this.

In a previous commit, phylink_prepare_resume() has been introduced to give MAC drivers a way to ensure that the PHY is resumed prior to their initialisation of their MAC hardware. This commit adds the call, and moves the phylink_resume() call back to where it should be before the aforementioned commit.

---
 .../net/ethernet/stmicro/stmmac/stmmac_main.c | 19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +14 -5
index a6a533d8a45b..860f800cd014 100644
@@ -7936,12 +7936,12 @@ int stmmac_resume(struct device *dev)
 	}
 
 	rtnl_lock();
-	phylink_resume(priv->phylink);
-	if (device_may_wakeup(priv->device) && !priv->plat->pmt)
-		phylink_speed_up(priv->phylink);
-	rtnl_unlock();
 
-	rtnl_lock();
+	/* Prepare the PHY to resume, ensuring that its clocks which are
+	 * necessary for the MAC DMA reset to complete are running
+	 */
+	phylink_prepare_resume(priv->phylink);
+
 	mutex_lock(&priv->lock);
 
 	stmmac_reset_queues_param(priv);
@@ -7959,6 +7959,15 @@ int stmmac_resume(struct device *dev)
 	stmmac_enable_all_dma_irq(priv);
 
 	mutex_unlock(&priv->lock);
+
+	/* phylink_resume() must be called after the hardware has been
+	 * initialised because it may bring the link up immediately in a
+	 * workqueue thread, which will race with initialisation.
+	 */
+	phylink_resume(priv->phylink);
+	if (device_may_wakeup(priv->device) && !priv->plat->pmt)
+		phylink_speed_up(priv->phylink);
+
 	rtnl_unlock();
 
 	netif_device_attach(ndev);
Signature · 1 line
2.30.2
#3 lore raw

PATCH 3/5 net: stmmac: socfpga: remove phy_resume() call

In reply to Russell King (Oracle)

As the previous commit addressed DWGMAC resuming with a PHY in suspended state, there is now no need for socfpga to work around this. Remove this code.

---
 .../ethernet/stmicro/stmmac/dwmac-socfpga.c    | 18 ------------------
 1 file changed, 18 deletions(-)
drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c +0 -18
index 6b78ae730466..116855658559 100644
@@ -523,24 +523,6 @@ static int socfpga_dwmac_resume(struct device *dev)
 
 	dwmac_priv->ops->set_phy_mode(priv->plat->bsp_priv);
 
-	/* Before the enet controller is suspended, the phy is suspended.
-	 * This causes the phy clock to be gated. The enet controller is
-	 * resumed before the phy, so the clock is still gated "off" when
-	 * the enet controller is resumed. This code makes sure the phy
-	 * is "resumed" before reinitializing the enet controller since
-	 * the enet controller depends on an active phy clock to complete
-	 * a DMA reset. A DMA reset will "time out" if executed
-	 * with no phy clock input on the Synopsys enet controller.
-	 * Verified through Synopsys Case #8000711656.
-	 *
-	 * Note that the phy clock is also gated when the phy is isolated.
-	 * Phy "suspend" and "isolate" controls are located in phy basic
-	 * control register 0, and can be modified by the phy driver
-	 * framework.
-	 */
-	if (ndev->phydev)
-		phy_resume(ndev->phydev);
-
 	return stmmac_resume(dev);
 }
 #endif /* CONFIG_PM_SLEEP */
Signature · 1 line
2.30.2
#4 lore raw

PATCH 4/5 net: phylink: add functions to block/unblock rx clock stop

In reply to Russell King (Oracle)

Some MACs require the PHY receive clock to be running to complete setup actions. This may fail if the PHY has negotiated EEE, the MAC supports receive clock stop, and the link has entered LPI state. Provide a pair of APIs that MAC drivers can use to temporarily block the PHY disabling the receive clock.

---
 drivers/net/phy/phylink.c | 59 +++++++++++++++++++++++++++++++++++++++
 include/linux/phylink.h   |  3 ++
 2 files changed, 62 insertions(+)
drivers/net/phy/phylink.c +59 -0
index 1b76ddc286fe..2cf389c68913 100644
@@ -89,6 +89,7 @@ struct phylink {
 	bool mac_enable_tx_lpi;
 	bool mac_tx_clk_stop;
 	u32 mac_tx_lpi_timer;
+	u8 mac_rx_clk_stop_blocked;
 
 	struct sfp_bus *sfp_bus;
 	bool sfp_may_have_phy;
@@ -2437,6 +2438,64 @@ void phylink_stop(struct phylink *pl)
 }
 EXPORT_SYMBOL_GPL(phylink_stop);
 
+/**
+ * phylink_rx_clk_stop_block() - block PHY ability to stop receive clock in LPI
+ * @pl: a pointer to a &struct phylink returned from phylink_create()
+ *
+ * Disable the PHY's ability to stop the receive clock while the receive path
+ * is in EEE LPI state, until the number of calls to phylink_rx_clk_stop_block()
+ * are balanced by calls to phylink_rx_clk_stop_unblock().
+ */
+void phylink_rx_clk_stop_block(struct phylink *pl)
+{
+	ASSERT_RTNL();
+
+	if (pl->mac_rx_clk_stop_blocked == U8_MAX) {
+		phylink_warn(pl, "%s called too many times - ignoring\n",
+			     __func__);
+		dump_stack();
+		return;
+	}
+
+	/* Disable PHY receive clock stop if this is the first time this
+	 * function has been called and clock-stop was previously enabled.
+	 */
+	if (pl->mac_rx_clk_stop_blocked++ == 0 &&
+	    pl->mac_supports_eee_ops && pl->phydev &&
+	    pl->config->eee_rx_clk_stop_enable)
+		phy_eee_rx_clock_stop(pl->phydev, false);
+}
+EXPORT_SYMBOL_GPL(phylink_rx_clk_stop_block);
+
+/**
+ * phylink_rx_clk_stop_unblock() - unblock PHY ability to stop receive clock
+ * @pl: a pointer to a &struct phylink returned from phylink_create()
+ *
+ * All calls to phylink_rx_clk_stop_block() must be balanced with a
+ * corresponding call to phylink_rx_clk_stop_unblock() to restore the PHYs
+ * ability to stop the receive clock when the receive path is in EEE LPI mode.
+ */
+void phylink_rx_clk_stop_unblock(struct phylink *pl)
+{
+	ASSERT_RTNL();
+
+	if (pl->mac_rx_clk_stop_blocked == 0) {
+		phylink_warn(pl, "%s called too many times - ignoring\n",
+			     __func__);
+		dump_stack();
+		return;
+	}
+
+	/* Re-enable PHY receive clock stop if the number of unblocks matches
+	 * the number of calls to the block function above.
+	 */
+	if (--pl->mac_rx_clk_stop_blocked == 0 &&
+	    pl->mac_supports_eee_ops && pl->phydev &&
+	    pl->config->eee_rx_clk_stop_enable)
+		phy_eee_rx_clock_stop(pl->phydev, true);
+}
+EXPORT_SYMBOL_GPL(phylink_rx_clk_stop_unblock);
+
 /**
  * phylink_suspend() - handle a network device suspend event
  * @pl: a pointer to a &struct phylink returned from phylink_create()
include/linux/phylink.h +3 -0
index 06f1b649f173..1f5773ab5660 100644
@@ -706,6 +706,9 @@ int phylink_pcs_pre_init(struct phylink *pl, struct phylink_pcs *pcs);
 void phylink_start(struct phylink *);
 void phylink_stop(struct phylink *);
 
+void phylink_rx_clk_stop_block(struct phylink *);
+void phylink_rx_clk_stop_unblock(struct phylink *);
+
 void phylink_suspend(struct phylink *pl, bool mac_wol);
 void phylink_prepare_resume(struct phylink *pl);
 void phylink_resume(struct phylink *pl);
Signature · 1 line
2.30.2
#5 lore raw

PATCH 5/5 net: stmmac: block PHY RXC clock-stop

In reply to Russell King (Oracle)

The DesignWare core requires the receive clock to be running during certain operations. Ensure that we block PHY RXC clock-stop during these operations.

This is a best-efforts change - not everywhere can be covered by this because of net's core locking, which means we can't access the MDIO bus to configure the PHY to disable RXC clock-stop in certain areas. These are marked with FIXME comments.

---
 .../net/ethernet/stmicro/stmmac/stmmac_main.c | 27 +++++++++++++++++++
 1 file changed, 27 insertions(+)
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +27 -0
index 860f800cd014..a4f2c98e851a 100644
@@ -3484,9 +3484,18 @@ static int stmmac_hw_setup(struct net_device *dev, bool ptp_register)
 	if (priv->hw->phylink_pcs)
 		phylink_pcs_pre_init(priv->phylink, priv->hw->phylink_pcs);
 
+	/* Note that clk_rx_i must be running for reset to complete. This
+	 * clock may also be required when setting the MAC address.
+	 *
+	 * Block the receive clock stop for LPI mode at the PHY in case
+	 * the link is established with EEE mode active.
+	 */
+	phylink_rx_clk_stop_block(priv->phylink);
+
 	/* DMA initialization and SW reset */
 	ret = stmmac_init_dma_engine(priv);
 	if (ret < 0) {
+		phylink_rx_clk_stop_unblock(priv->phylink);
 		netdev_err(priv->dev, "%s: DMA engine initialization failed\n",
 			   __func__);
 		return ret;
@@ -3494,6 +3503,7 @@ static int stmmac_hw_setup(struct net_device *dev, bool ptp_register)
 
 	/* Copy the MAC addr into the HW  */
 	stmmac_set_umac_addr(priv, priv->hw, dev->dev_addr, 0);
+	phylink_rx_clk_stop_unblock(priv->phylink);
 
 	/* PS and related bits will be programmed according to the speed */
 	if (priv->hw->pcs) {
@@ -3604,7 +3614,9 @@ static int stmmac_hw_setup(struct net_device *dev, bool ptp_register)
 	/* Start the ball rolling... */
 	stmmac_start_all_dma(priv);
 
+	phylink_rx_clk_stop_block(priv->phylink);
 	stmmac_set_hw_vlan_mode(priv, priv->hw);
+	phylink_rx_clk_stop_unblock(priv->phylink);
 
 	return 0;
 }
@@ -5888,6 +5900,9 @@ static void stmmac_tx_timeout(struct net_device *dev, unsigned int txqueue)
  *  whenever multicast addresses must be enabled/disabled.
  *  Return value:
  *  void.
+ *
+ *  FIXME: This may need RXC to be running, but it may be called with BH
+ *  disabled, which means we can't call phylink_rx_clk_stop*().
  */
 static void stmmac_set_rx_mode(struct net_device *dev)
 {
@@ -6020,7 +6035,9 @@ static int stmmac_set_features(struct net_device *netdev,
 	else
 		priv->hw->hw_vlan_en = false;
 
+	phylink_rx_clk_stop_block(priv->phylink);
 	stmmac_set_hw_vlan_mode(priv, priv->hw);
+	phylink_rx_clk_stop_unblock(priv->phylink);
 
 	return 0;
 }
@@ -6304,7 +6321,9 @@ static int stmmac_set_mac_address(struct net_device *ndev, void *addr)
 	if (ret)
 		goto set_mac_error;
 
+	phylink_rx_clk_stop_block(priv->phylink);
 	stmmac_set_umac_addr(priv, priv->hw, ndev->dev_addr, 0);
+	phylink_rx_clk_stop_unblock(priv->phylink);
 
 set_mac_error:
 	pm_runtime_put(priv->device);
@@ -6660,6 +6679,9 @@ static int stmmac_vlan_update(struct stmmac_priv *priv, bool is_double)
 	return stmmac_update_vlan_hash(priv, priv->hw, hash, pmatch, is_double);
 }
 
+/* FIXME: This may need RXC to be running, but it may be called with BH
+ * disabled, which means we can't call phylink_rx_clk_stop*().
+ */
 static int stmmac_vlan_rx_add_vid(struct net_device *ndev, __be16 proto, u16 vid)
 {
 	struct stmmac_priv *priv = netdev_priv(ndev);
@@ -6691,6 +6713,9 @@ static int stmmac_vlan_rx_add_vid(struct net_device *ndev, __be16 proto, u16 vid
 	return ret;
 }
 
+/* FIXME: This may need RXC to be running, but it may be called with BH
+ * disabled, which means we can't call phylink_rx_clk_stop*().
+ */
 static int stmmac_vlan_rx_kill_vid(struct net_device *ndev, __be16 proto, u16 vid)
 {
 	struct stmmac_priv *priv = netdev_priv(ndev);
@@ -7951,9 +7976,11 @@ int stmmac_resume(struct device *dev)
 
 	stmmac_hw_setup(ndev, false);
 	stmmac_init_coalesce(priv);
+	phylink_rx_clk_stop_block(priv->phylink);
 	stmmac_set_rx_mode(ndev);
 
 	stmmac_restore_hw_vlan_rx_fltr(priv, ndev, priv->hw);
+	phylink_rx_clk_stop_unblock(priv->phylink);
 
 	stmmac_enable_all_queues(priv);
 	stmmac_enable_all_dma_irq(priv);
Signature · 1 line
2.30.2
#6 lore raw
In reply to Russell King (Oracle)
This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (main) by Jakub Kicinski <[email protected]>:

Quoted 15 lines · On Thu, 20 Mar 2025 22:11:06 +0000 you wrote:

Hi,

stmmac has had a long history of problems with resuming, illustrated by reset failure due to the receive clock not running.

Several attempts have been attempted over the years to address this issue, such as moving phylink_start() (now phylink_resume()) super early in stmmac_resume() in commit 90702dcd19c0 ("net: stmmac: fix MAC not working when system resume back with WoL a ctive.") However, this has the downside that stmmac_mac_link_up() can (and demonstrably is) called before or during the driver initialisation in another thread. This can cause issues as packets could begin to be queued, and the transmit/receive enable bits will be set before any initialisation has been done.

Quoted 1 line

[...]

Here is the summary with links:
- [net-next,1/5] net: phylink: add phylink_prepare_resume()

    https://git.kernel.org/netdev/net-next/c/367f1854d442
  - [net-next,2/5] net: stmmac: address non-LPI resume failures properly
    https://git.kernel.org/netdev/net-next/c/ef43e5132895
  - [net-next,3/5] net: stmmac: socfpga: remove phy_resume() call
    https://git.kernel.org/netdev/net-next/c/366aeeba7908
  - [net-next,4/5] net: phylink: add functions to block/unblock rx clock stop
    https://git.kernel.org/netdev/net-next/c/ddf4bd3f7384
  - [net-next,5/5] net: stmmac: block PHY RXC clock-stop
    https://git.kernel.org/netdev/net-next/c/dd557266cf5f

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html