net: pse-pd: decouple controller lookup from MDIO probe
Subsystems (6)
v5 cover net: pse-pd: decouple controller lookup from MDIO probe
This is v5 of Corey's series [1]. It takes the PSE controller lookup out of the MDIO probe path, so a modular PSE driver no longer makes the PHY/DSA probe spin on -EPROBE_DEFER until the PSE module loads.
Patches 1-3 are the same three notifier patches as v4 [4], unchanged, with Jonas's Tested-by. Patches 4 and 5 are new and fix two problems the v4 review surfaced.
Patch 4: Aleksander reported [5] that v4 deadlocks on probe for an MDIO bus registered from ndo_init (lantiq_etop, sni_ave, netsec): those already hold rtnl via register_netdevice(), and v4's phy attach took rtnl again underneath. Patch 4 swaps that rtnl for a dedicated mutex, so the register path no longer recurses. The ethtool PSE paths take the same mutex, so the use-after-free that rtnl used to close stays closed. Aleksander confirmed it fixes his deadlock.
Patch 5: Paolo's review [6] pointed out that patch 3 defers the pse_control_put() to phy_device_release(). A phy that is device_del()'d but still pinned (an attached netdev) is off the mdio_bus_type klist, so the PSE_UNREGISTERED notifier walk never clears its phydev->psec, and the deferred put later touches a pcdev->pi[] the controller has already freed. Patch 5 puts phydev->psec back in phy_device_remove(), which the mutex from patch 4 now makes safe (the rtnl recursion that motivated the deferral is gone), so the detach is synchronous and cannot outlive the controller.
How it works: pse_core gets a notifier chain (REGISTERED / UNREGISTERED). The phy layer subscribes, owns phydev->psec, and attaches the PSE handle when the controller shows up instead of during probe. fwnode_mdio loses its PSE awareness, so no -EPROBE_DEFER leaves it and the probe-retry loop is gone.
Tested on a Realtek rtl93xx PoE switch with two HS104 PSE controllers on i2c:
- clean boot, no probe-retry loop, no watchdog reset
- 10G SFP+ port: module hotplug works, no deadlock
- ethtool --set-pse enable/disable cuts and restores power to a PD
- i2c unbind -> rmmod -> modprobe: PSE detaches on unbind and re-attaches on reload with power restored, no reboot. No lockdep splats.
Jonas confirmed the RTL8214FC deadlock he reported is gone. Aleksander confirmed the lantiq_etop probe deadlock is gone.
Changes in v5:
- Add patch 4: replace rtnl with a dedicated mutex in the PSE attach path, fixing the ndo_init probe deadlock Aleksander reported [5]. Tested-by Aleksander.
- Add patch 5: put phydev->psec back in phy_device_remove(), closing the off-klist use-after-free Paolo raised [6].
- Patches 1-3 are unchanged.
Changes in v4:
- Add Tested-by from Jonas Jelonek. No code changes.
- Repost now that net-next has reopened (v3 was deferred during the merge window).
Changes in v3:
- Drop patch 1 (regulator handle fix); it goes to net separately [2].
- Rebase on net-next. No code changes to the three patches.
v1 was an RFC by Corey [3].
[1] https://lore.kernel.org/netdev/[email protected]/
[2] https://lore.kernel.org/netdev/[email protected]/
[3] https://lore.kernel.org/netdev/[email protected]/
[4] https://lore.kernel.org/netdev/[email protected]/
[5] https://lore.kernel.org/netdev/[email protected]/
[6] https://lore.kernel.org/netdev/[email protected]/
Carlo Szelinsky (2):
net: phy: use a dedicated mutex instead of rtnl for PSE control attach
net: phy: release phydev->psec from phy_device_remove() again
Corey Leavitt (3):
net: pse-pd: add notifier chain for controller lifecycle events
net: pse-pd: fire lifecycle events on controller register/unregister
net: phy: own phydev->psec via PSE notifier and remove fwnode_mdio
hook
drivers/net/mdio/fwnode_mdio.c | 34 ------ drivers/net/phy/phy_device.c | 186 ++++++++++++++++++++++++++++++++- drivers/net/phy/sfp.c | 2 +- drivers/net/pse-pd/pse_core.c | 54 ++++++++++ include/linux/phy.h | 4 + include/linux/pse-pd/pse.h | 41 ++++++++ net/ethtool/pse-pd.c | 15 ++- 7 files changed, 292 insertions(+), 44 deletions(-)
base-commit: cef9d6804030793cf8b8796fd6936197d065dd3e
Signature · 1 line
2.43.0
PATCH v5 1/5 net: pse-pd: add notifier chain for controller lifecycle events
From: Corey Leavitt <[email protected]>
Introduce a blocking notifier chain that allows other subsystems to be informed when a PSE controller is registered or unregistered, and provide pse_register_notifier() / pse_unregister_notifier() as the subscriber interface.
Subsequent patches will use this to let the phy subsystem own the phydev->psec lifecycle directly, decoupling PSE lookup from fwnode_mdiobus_register_phy() and removing the probe-time -EPROBE_DEFER coupling that currently exists between mdio, phy and pse-pd when the PSE controller driver is modular.
A blocking chain (rather than atomic) is used because callbacks will take rtnl_lock and call back into pse_core via of_pse_control_get().
The enum pse_controller_event is placed outside the IS_ENABLED(CONFIG_PSE_CONTROLLER) guard so that subscribers compiled into a kernel without PSE support can still reference the event values in dead-code paths without breaking the build.
This patch is pure infrastructure: nothing fires events yet, and nothing subscribes. No observable behavior change.
--- drivers/net/pse-pd/pse_core.c | 34 ++++++++++++++++++++++++++++++++++ include/linux/pse-pd/pse.h | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+)
drivers/net/pse-pd/pse_core.c +34 -0
@@ -8,6 +8,7 @@#include <linux/device.h> #include <linux/ethtool.h> #include <linux/ethtool_netlink.h> +#include <linux/notifier.h> #include <linux/of.h> #include <linux/phy.h> #include <linux/pse-pd/pse.h>
@@ -23,6 +24,39 @@ static LIST_HEAD(pse_controller_list);static DEFINE_XARRAY_ALLOC (pse_pw_d_map);static DEFINE_MUTEX (pse_pw_d_mutex); +static BLOCKING_NOTIFIER_HEAD (pse_controller_notifier); + +/** +* pse_register_notifier - register a callback for PSE controller events +* @nb: notifier block to register +* +* See enum pse_controller_event for events fired and their subscriber +* contract. Callbacks run in process context; they may sleep, take +* rtnl, and call of_pse_control_get(). The chain fires synchronously, +* so a PSE controller driver's probe/unbind path must not hold any +* such lock when calling pse_controller_register() or +* pse_controller_unregister(). +* +* Return: 0 on success, negative error code otherwise. +*/ +int pse_register_notifier (struct notifier_block *nb) +{ +return blocking_notifier_chain_register (&pse_controller_notifier, nb); +} +EXPORT_SYMBOL_GPL (pse_register_notifier); + +/** +* pse_unregister_notifier - unregister a previously registered callback +* @nb: notifier block previously passed to pse_register_notifier() +* +* Return: 0 on success, negative error code otherwise. +*/ +int pse_unregister_notifier (struct notifier_block *nb) +{ +return blocking_notifier_chain_unregister (&pse_controller_notifier, nb); +} +EXPORT_SYMBOL_GPL (pse_unregister_notifier); +/** * struct pse_control - a PSE control * @pcdev: a pointer to the PSE controller device
include/linux/pse-pd/pse.h +32 -0
@@ -21,6 +21,7 @@ struct net_device;struct phy_device ;struct pse_controller_dev ;struct netlink_ext_ack ; +struct notifier_block ;/* C33 PSE extended state and substate. */ struct ethtool_c33_pse_ext_state_info {
@@ -337,6 +338,24 @@ enum pse_budget_eval_strategies { PSE_BUDGET_EVAL_STRAT_DYNAMIC = 1 << 2, }; +/** +* enum pse_controller_event - PSE controller lifecycle events +* +* Event data in callbacks is always a pointer to the struct +* pse_controller_dev firing the event. +* +* @PSE_REGISTERED: controller added to pse_controller_list and +* resolvable by of_pse_control_get(). +* @PSE_UNREGISTERED: controller about to be removed from +* pse_controller_list. Subscribers holding pse_control references +* targeting it must drop them before returning and must not +* acquire new references for it. +*/ +enum pse_controller_event { +PSE_REGISTERED , +PSE_UNREGISTERED , +}; +#if IS_ENABLED (CONFIG_PSE_CONTROLLER )int pse_controller_register (struct pse_controller_dev *pcdev);void pse_controller_unregister (struct pse_controller_dev *pcdev);
@@ -366,6 +385,9 @@ int pse_ethtool_set_prio(struct pse_control *psec,bool pse_has_podl (struct pse_control *psec);bool pse_has_c33 (struct pse_control *psec); +int pse_register_notifier (struct notifier_block *nb); +int pse_unregister_notifier (struct notifier_block *nb); +#else static inline struct pse_control *of_pse_control_get(struct device_node *node,
@@ -416,6 +438,16 @@ static inline bool pse_has_c33(struct pse_control *psec)return false; } +static inline int pse_register_notifier (struct notifier_block *nb) +{ +return 0 ; +} + +static inline int pse_unregister_notifier (struct notifier_block *nb) +{ +return 0 ; +} +#endif #endif
Signature · 1 line
2.43.0
PATCH v5 2/5 net: pse-pd: fire lifecycle events on controller register/unregister
In reply to Carlo SzelinskyFrom: Corey Leavitt <[email protected]>
Hook the newly-introduced pse_controller_notifier chain so that pse_controller_register() fires PSE_REGISTERED after the controller has been added to pse_controller_list (i.e. is now resolvable by of_pse_control_get()), and pse_controller_unregister() fires PSE_UNREGISTERED before the controller is removed from the list (while it is still valid to dereference from a subscriber's pse_control pointer targeting it).
With no subscribers yet, this is observably a no-op. A later change wires the phy subsystem in as the first subscriber.
--- drivers/net/pse-pd/pse_core.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/net/pse-pd/pse_core.c +6 -0
@@ -1138,6 +1138,9 @@ int pse_controller_register(struct pse_controller_dev *pcdev)list_add (&pcdev->list, &pse_controller_list);mutex_unlock (&pse_list_mutex); +blocking_notifier_call_chain (&pse_controller_notifier, +PSE_REGISTERED , pcdev); +return 0 ; }EXPORT_SYMBOL_GPL (pse_controller_register);
@@ -1148,6 +1151,9 @@ EXPORT_SYMBOL_GPL(pse_controller_register);*/ void pse_controller_unregister (struct pse_controller_dev *pcdev) { +blocking_notifier_call_chain (&pse_controller_notifier, +PSE_UNREGISTERED , pcdev); +pse_flush_pw_ds (pcdev);pse_release_pis (pcdev);if (pcdev->irq)
Signature · 1 line
2.43.0
PATCH v5 3/5 net: phy: own phydev->psec via PSE notifier and remove fwnode_mdio hook
In reply to Carlo SzelinskyFrom: Corey Leavitt <[email protected]>
Transfer ownership of phydev->psec from fwnode_mdio to the phy subsystem itself. The phy subsystem now subscribes to the pse-pd notifier chain and manages psec attach/detach in response to PSE controller lifecycle events, while fwnode_mdio loses its PSE awareness entirely.
phydev->psec is attached after device_add() has made the phy visible on mdio_bus_type, under a narrow rtnl_lock() that covers only phy_try_attach_pse(). Ordering the attach after registration closes the race that would otherwise leave a phy unattached: a PSE_REGISTERED event firing during registration walks mdio_bus_type and either finds the phy already added (and attaches it) or runs before device_add(), in which case the post-add attach resolves it. The phydev->psec check in phy_try_attach_pse() makes the two paths idempotent. Holding rtnl across of_pse_control_get() is safe because pse_list_mutex is never taken in the opposite order.
device_add() is deliberately left outside rtnl. Binding a phy that itself provides an SFP cage reaches sfp_bus_add_upstream() through phy_probe() -> phy_setup_ports() -> phy_sfp_probe(), and sfp_bus_add_upstream() takes rtnl_lock(); holding rtnl across device_add() would deadlock such phys (reported on RTL8214FC).
phy_device_register() is split into the public form, which takes the narrow rtnl_lock() around the attach, and a phy_device_register_locked() form for callers that already hold rtnl (the SFP module state machine via __sfp_sm_event). This pair mirrors the register_netdevice() / register_netdev() split convention already established in the core networking stack. The _locked form runs device_add() under the caller's rtnl, which is safe because a phy resident on an SFP module does not itself provide a downstream cage, so phy_sfp_probe() is a no-op there.
- On PSE_REGISTERED: an rtnl-guarded bus walk retries the attach for every registered phy whose psec is still NULL. This is the "phy was enumerated before the PSE controller loaded" case, the root cause of the boot-time probe-retry storm on systems with a modular PSE controller driver.
- On PSE_UNREGISTERED: an rtnl-guarded bus walk releases every
phydev->psec that targets the departing controller before
pse_release_pis() frees pcdev->pi. Without this, a phy still
holding a pse_control reference would cause a use-after-free in
__pse_control_release()'s pcdev->pi[psec->id] access, and the PSE
driver module could not finish unloading while any phy still held a
reference.A bad pses binding -- an error from of_pse_control_get() other than -ENOENT (no phandle) or -EPROBE_DEFER (controller not yet registered) -- is reported with phydev_warn() rather than silently dropped, preserving the diagnostic that the removed fwnode_mdio lookup used to provide.
The final pse_control_put() of phydev->psec moves from phy_device_remove() to phy_device_release(), so it runs only after every reference on the device -- including the bus-iterator references taken by bus_for_each_dev() in the notifier walk -- has been dropped.
Finally, delete fwnode_find_pse_control() and its call site in fwnode_mdiobus_register_phy(), and drop the PSE header from fwnode_mdio.c. The MDIO/DSA probe no longer sees any PSE-originated -EPROBE_DEFER, so the probe-retry storm is gone and fwnode_mdio is now PSE-agnostic.
--- drivers/net/mdio/fwnode_mdio.c | 34 ------- drivers/net/phy/phy_device.c | 168 +++++++++++++++++++++++++++++++-- drivers/net/phy/sfp.c | 2 +- drivers/net/pse-pd/pse_core.c | 14 +++ include/linux/phy.h | 2 + include/linux/pse-pd/pse.h | 9 ++ 6 files changed, 186 insertions(+), 43 deletions(-)
drivers/net/mdio/fwnode_mdio.c +0 -34
@@ -11,33 +11,11 @@#include <linux/fwnode_mdio.h> #include <linux/of.h> #include <linux/phy.h> -#include <linux/pse-pd/pse.h> MODULE_AUTHOR ("Calvin Johnson <[email protected]>" );MODULE_LICENSE ("GPL" );MODULE_DESCRIPTION ("FWNODE MDIO bus (Ethernet PHY) accessors" ); -static struct pse_control * -fwnode_find_pse_control (struct fwnode_handle *fwnode, -struct phy_device *phydev) -{ -struct pse_control *psec; -struct device_node *np; - -if (!IS_ENABLED (CONFIG_PSE_CONTROLLER )) -return NULL ; - - np =to_of_node (fwnode); -if (!np) -return NULL ; - - psec =of_pse_control_get (np, phydev); -if (PTR_ERR (psec) == -ENOENT ) -return NULL ; - -return psec; -} -static struct mii_timestamper *fwnode_find_mii_timestamper (struct fwnode_handle *fwnode) {
@@ -118,7 +96,6 @@ int fwnode_mdiobus_register_phy(struct mii_bus *bus, struct fwnode_handle *child, u32 addr) {struct mii_timestamper *mii_ts =NULL ; -struct pse_control *psec =NULL ;struct phy_device *phy;bool is_c45;u32 phy_id;
@@ -159,14 +136,6 @@ int fwnode_mdiobus_register_phy(struct mii_bus *bus, goto clean_phy; } - psec =fwnode_find_pse_control (child, phy); -if (IS_ERR (psec)) { - rc =PTR_ERR (psec); - goto unregister_phy; - } - - phy->psec = psec; -/* phy->mii_ts may already be defined by the PHY driver. A * mii_timestamper probed via the device tree will still have * precedence.
@@ -176,9 +145,6 @@ int fwnode_mdiobus_register_phy(struct mii_bus *bus,return 0 ; -unregister_phy: -if (is_acpi_node (child) ||is_of_node (child)) -phy_device_remove (phy); clean_phy:phy_device_free (phy); clean_mii_ts:
drivers/net/phy/phy_device.c +160 -8
211 lines not on this page · raw message
drivers/net/phy/sfp.c +1 -1
@@ -2083,7 +2083,7 @@ static int sfp_sm_probe_phy(struct sfp *sfp, int addr, bool is_c45)/* Mark this PHY as being on a SFP module */ phy->is_on_sfp_module = true; - err =phy_device_register (phy); + err =phy_device_register_locked (phy);if (err) {phy_device_free (phy); dev_err(sfp->dev, "phy_device_register failed: %pe\n",
drivers/net/pse-pd/pse_core.c +14 -0
@@ -2021,3 +2021,17 @@ bool pse_has_c33(struct pse_control *psec)return psec->pcdev->types ÐTOOL_PSE_C33 ; }EXPORT_SYMBOL_GPL (pse_has_c33); + +/** +* pse_control_matches_pcdev - Test whether a pse_control targets a controller +* @psec: pse_control obtained from of_pse_control_get() +* @pcdev: PSE controller to compare against +* +* Return: %true if @psec was obtained from @pcdev, %false otherwise. +*/ +bool pse_control_matches_pcdev (struct pse_control *psec, +struct pse_controller_dev *pcdev) +{ +return psec->pcdev == pcdev; +} +EXPORT_SYMBOL_GPL (pse_control_matches_pcdev);
include/linux/phy.h +2 -0
@@ -2158,6 +2158,8 @@ struct phy_device *fwnode_phy_find_device(struct fwnode_handle *phy_fwnode);struct fwnode_handle *fwnode_get_phy_node (const struct fwnode_handle *fwnode);struct phy_device *get_phy_device (struct mii_bus *bus,int addr,bool is_c45);int phy_device_register (struct phy_device *phy); +/* Caller must hold rtnl_lock(); see phy_device_register() for the public form. */ +int phy_device_register_locked (struct phy_device *phy);void phy_device_free (struct phy_device *phydev);void phy_device_remove (struct phy_device *phydev);int phy_get_c45_ids (struct phy_device *phydev);
include/linux/pse-pd/pse.h +9 -0
@@ -385,6 +385,9 @@ int pse_ethtool_set_prio(struct pse_control *psec,bool pse_has_podl (struct pse_control *psec);bool pse_has_c33 (struct pse_control *psec); +bool pse_control_matches_pcdev (struct pse_control *psec, +struct pse_controller_dev *pcdev); +int pse_register_notifier (struct notifier_block *nb);int pse_unregister_notifier (struct notifier_block *nb);
@@ -438,6 +441,12 @@ static inline bool pse_has_c33(struct pse_control *psec)return false; } +static inline bool pse_control_matches_pcdev (struct pse_control *psec, +struct pse_controller_dev *pcdev) +{ +return false; +} +static inline int pse_register_notifier (struct notifier_block *nb) {return 0 ;
Signature · 1 line
2.43.0
PATCH v5 4/5 net: phy: use a dedicated mutex instead of rtnl for PSE control attach
In reply to Carlo Szelinskyphy_device_register() took rtnl_lock() around phy_try_attach_pse() to serialise phydev->psec against the PSE controller notifier walk. But an MDIO bus registered from ndo_init() runs with rtnl already held:
register_netdevice() # holds rtnl
ndo_init() == ltq_etop_init()
ltq_etop_mdio_init()
mdiobus_register()
mdiobus_scan()
phy_device_register()
rtnl_lock() # deadlockso any such driver (lantiq_etop, sni_ave, netsec) deadlocks on probe.
Replace rtnl with a dedicated phy_pse_lock mutex for the attach, for the notifier attach/detach walks, and for the ethtool PSE paths that dereference phydev->psec. A private lock cannot recurse against the caller's rtnl, so the register path no longer deadlocks, while attach vs notifier and detach vs ethtool stay mutually excluded.
rtnl also kept the ethtool PSE reads from racing the PSE_UNREGISTERED detach that frees phydev->psec, so net/ethtool/pse-pd.c takes the same lock across its phydev->psec accesses; guarding only the phy side would reopen a use-after-free there. The lock order is phy_pse_lock -> pse_list_mutex -> pcdev->lock, and the notifier walks enter at phy_pse_lock and never take rtnl.
--- drivers/net/phy/phy_device.c | 53 +++++++++++++++++++++++++++--------- include/linux/phy.h | 2 ++ net/ethtool/pse-pd.c | 15 ++++++---- 3 files changed, 52 insertions(+), 18 deletions(-)
drivers/net/phy/phy_device.c +40 -13
@@ -1113,18 +1113,21 @@ struct phy_device *get_phy_device(struct mii_bus *bus, int addr, bool is_c45) }EXPORT_SYMBOL (get_phy_device); +/* Serialises phydev->psec against the PSE notifier and ethtool, not rtnl. */ +static DEFINE_MUTEX (phy_pse_lock); +/* Best-effort attach of phydev->psec from a DT `pses = <&...>` phandle. -* Caller must hold rtnl. A missing phandle (-ENOENT) or a not-yet-registered -* controller (-EPROBE_DEFER) is silent; the notifier retries the latter at -* PSE_REGISTERED time. Any other error means a broken binding and is warned -* about, but left non-fatal so the phy still registers. +* Caller must hold phy_pse_lock. A missing phandle (-ENOENT) or a +* not-yet-registered controller (-EPROBE_DEFER) is silent; the notifier +* retries the latter at PSE_REGISTERED time. Any other error means a broken +* binding and is warned about, but left non-fatal so the phy still registers. */ static void phy_try_attach_pse (struct phy_device *phydev) {struct pse_control *psec;struct device_node *np; -ASSERT_RTNL (); +lockdep_assert_held (&phy_pse_lock); np = phydev->mdio.dev.of_node;if (!np)
@@ -1146,7 +1149,7 @@ static void phy_try_attach_pse(struct phy_device *phydev)static int phy_pse_attach_one (struct device *dev,void *data__maybe_unused ) { -ASSERT_RTNL (); +lockdep_assert_held (&phy_pse_lock);if (dev->type != &mdio_bus_phy_type)return 0 ;
@@ -1161,7 +1164,7 @@ static int phy_pse_detach_one(struct device *dev, void *data)struct phy_device *phydev;struct pse_control *psec; -ASSERT_RTNL (); +lockdep_assert_held (&phy_pse_lock);if (dev->type != &mdio_bus_phy_type)return 0 ;
@@ -1181,16 +1184,16 @@ static int phy_pse_notifier_event(struct notifier_block *nb, {switch (event) {case PSE_REGISTERED : -rtnl_lock (); +mutex_lock (&phy_pse_lock);bus_for_each_dev (&mdio_bus_type,NULL ,NULL , phy_pse_attach_one); -rtnl_unlock (); +mutex_unlock (&phy_pse_lock);return NOTIFY_OK ;case PSE_UNREGISTERED : -rtnl_lock (); +mutex_lock (&phy_pse_lock);bus_for_each_dev (&mdio_bus_type,NULL , data, phy_pse_detach_one); -rtnl_unlock (); +mutex_unlock (&phy_pse_lock);return NOTIFY_OK ;default :return NOTIFY_DONE ;
@@ -1201,6 +1204,28 @@ static struct notifier_block phy_pse_notifier __read_mostly = { .notifier_call = phy_pse_notifier_event, }; +/** +* phy_pse_control_lock - hold phydev->psec stable against PSE controller teardown +* +* The PSE_UNREGISTERED notifier detaches phydev->psec and drops its last +* reference. Callers that dereference phydev->psec (the ethtool PSE paths) must +* hold this lock across the access so the detach cannot run underneath them. +*/ +void phy_pse_control_lock (void ) +{ +mutex_lock (&phy_pse_lock); +} +EXPORT_SYMBOL_GPL (phy_pse_control_lock); + +/** +* phy_pse_control_unlock - release the lock taken by phy_pse_control_lock() +*/ +void phy_pse_control_unlock (void ) +{ +mutex_unlock (&phy_pse_lock); +} +EXPORT_SYMBOL_GPL (phy_pse_control_unlock); +/* Core registration: add the phy to the MDIO bus. Does not touch rtnl or * PSE. phydev->psec is attached by the callers below, after device_add() * has made the phy visible on mdio_bus_type, so that a concurrent PSE
@@ -1260,7 +1285,9 @@ int phy_device_register_locked(struct phy_device *phydev)if (err)return err; +mutex_lock (&phy_pse_lock);phy_try_attach_pse (phydev); +mutex_unlock (&phy_pse_lock);return 0 ; }
@@ -1280,9 +1307,9 @@ int phy_device_register(struct phy_device *phydev)if (err)return err; -rtnl_lock (); +mutex_lock (&phy_pse_lock);phy_try_attach_pse (phydev); -rtnl_unlock (); +mutex_unlock (&phy_pse_lock);return 0 ; }
include/linux/phy.h +2 -0
@@ -2162,6 +2162,8 @@ int phy_device_register(struct phy_device *phy);int phy_device_register_locked (struct phy_device *phy);void phy_device_free (struct phy_device *phydev);void phy_device_remove (struct phy_device *phydev); +void phy_pse_control_lock (void ); +void phy_pse_control_unlock (void );int phy_get_c45_ids (struct phy_device *phydev);int phy_init_hw (struct phy_device *phydev);int phy_suspend (struct phy_device *phydev);
net/ethtool/pse-pd.c +10 -5
@@ -71,7 +71,9 @@ static int pse_prepare_data(const struct ethnl_req_info *req_base,if (ret <0 )return ret; +phy_pse_control_lock (); ret =pse_get_pse_attributes (phydev, info->extack, data); +phy_pse_control_unlock ();ethnl_ops_complete (dev);
@@ -281,9 +283,12 @@ ethnl_set_pse(struct ethnl_req_info *req_info, struct genl_info *info) phydev =ethnl_req_get_phydev (req_info, tb,ETHTOOL_A_PSE_HEADER , info->extack); + +phy_pse_control_lock (); + ret =ethnl_set_pse_validate (phydev, info);if (ret) -return ret; + goto out;if (tb[ETHTOOL_A_PSE_PRIO ]) {unsigned int prio;
@@ -291,7 +296,7 @@ ethnl_set_pse(struct ethnl_req_info *req_info, struct genl_info *info) prio =nla_get_u32 (tb[ETHTOOL_A_PSE_PRIO ]); ret =pse_ethtool_set_prio (phydev->psec, info->extack, prio);if (ret) -return ret; + goto out; }if (tb[ETHTOOL_A_C33_PSE_AVAIL_PW_LIMIT ]) {
@@ -301,7 +306,7 @@ ethnl_set_pse(struct ethnl_req_info *req_info, struct genl_info *info) ret =pse_ethtool_set_pw_limit (phydev->psec, info->extack, pw_limit);if (ret) -return ret; + goto out; }/* These values are already validated by the ethnl_pse_set_policy */
@@ -319,11 +324,11 @@ ethnl_set_pse(struct ethnl_req_info *req_info, struct genl_info *info)*/ ret =pse_ethtool_set_config (phydev->psec, info->extack, &config); -if (ret) -return ret; } +out:/* Return errno or zero - PSE has no notification */ +phy_pse_control_unlock ();return ret; }
Signature · 1 line
2.43.0
PATCH v5 5/5 net: phy: release phydev->psec from phy_device_remove() again
In reply to Carlo Szelinsky"net: phy: own phydev->psec via PSE notifier and remove fwnode_mdio hook" deferred the final pse_control_put() of phydev->psec from phy_device_remove() to phy_device_release(), so it would run only after the PSE_UNREGISTERED notifier walk had dropped its bus-iterator reference on the phy. But bus_for_each_dev() only reaches phys still on the mdio_bus_type klist: a phy that has been device_del()'d yet is still pinned (e.g. by an attached netdev) is invisible to the walk, so phy_pse_detach_one() never clears its phydev->psec. Its deferred put then runs after pse_controller_unregister() -> pse_release_pis() has freed pcdev->pi[], and __pse_control_release() dereferences the freed array:
use-after-free.
Put phydev->psec back in phy_device_remove(), before device_del(), so the detach is synchronous and ordered ahead of the phy leaving the bus; it can no longer outlive the PSE controller. "net: phy: use a dedicated mutex instead of rtnl for PSE control attach" replaced rtnl with phy_pse_lock for the attach/detach, so this put can take that same lock without the rtnl recursion that originally motivated the deferral, and it serialises against the notifier walk: whichever runs first clears phydev->psec, the other sees NULL.
--- drivers/net/phy/phy_device.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-)
drivers/net/phy/phy_device.c +11 -12
@@ -223,19 +223,8 @@ static void phy_mdio_device_free(struct mdio_device *mdiodev)static void phy_device_release (struct device *dev) { -struct phy_device *phydev =to_phy_device (dev); - -/* bus_for_each_dev() holds get_device() across each iteration -* step, deferring this release callback until any in-flight PSE -* notifier walk has advanced past this phy. pse_control_put() -* takes pse_list_mutex, so this path must run in sleepable -* context. -*/ -might_sleep (); -pse_control_put (phydev->psec); -fwnode_handle_put (dev->fwnode); -kfree (phydev); +kfree (to_phy_device (dev)); }static void phy_mdio_device_remove (struct mdio_device *mdiodev)
@@ -1326,6 +1315,16 @@ EXPORT_SYMBOL(phy_device_register);void phy_device_remove (struct phy_device *phydev) {unregister_mii_timestamper (phydev->mii_ts); + +/* Detach synchronously, before the phy leaves the bus, so the put cannot +* outlive the PSE controller (an off-bus but still-pinned phy is missed by +* the PSE_UNREGISTERED walk). phy_pse_lock serialises against that walk. +*/ +mutex_lock (&phy_pse_lock); +pse_control_put (phydev->psec); + phydev->psec =NULL ; +mutex_unlock (&phy_pse_lock); +device_del (&phydev->mdio.dev);/* Assert the reset signal */
Signature · 1 line
2.43.0
Paolo Abeni
maintainerQuoted 41 lines · Carlo Szelinsky
This is v5 of Corey's series [1]. It takes the PSE controller lookup out of the MDIO probe path, so a modular PSE driver no longer makes the PHY/DSA probe spin on -EPROBE_DEFER until the PSE module loads.
Patches 1-3 are the same three notifier patches as v4 [4], unchanged, with Jonas's Tested-by. Patches 4 and 5 are new and fix two problems the v4 review surfaced.
Patch 4: Aleksander reported [5] that v4 deadlocks on probe for an MDIO bus registered from
ndo_init(lantiq_etop,sni_ave, netsec): those already hold rtnl viaregister_netdevice(), and v4's phy attach took rtnl again underneath. Patch 4 swaps that rtnl for a dedicated mutex, so the register path no longer recurses. The ethtool PSE paths take the same mutex, so the use-after-free that rtnl used to close stays closed. Aleksander confirmed it fixes his deadlock.Patch 5: Paolo's review [6] pointed out that patch 3 defers the
pse_control_put()tophy_device_release(). A phy that is device_del()'d but still pinned (an attached netdev) is off themdio_bus_typeklist, so thePSE_UNREGISTEREDnotifier walk never clears itsphydev->psec, and the deferred put later touches a pcdev->pi[] the controller has already freed. Patch 5 putsphydev->psecback inphy_device_remove(), which the mutex from patch 4 now makes safe (the rtnl recursion that motivated the deferral is gone), so the detach is synchronous and cannot outlive the controller.How it works:
pse_coregets a notifier chain (REGISTERED / UNREGISTERED). The phy layer subscribes, ownsphydev->psec, and attaches the PSE handle when the controller shows up instead of during probe.fwnode_mdioloses its PSE awareness, so no -EPROBE_DEFER leaves it and the probe-retry loop is gone.Tested on a Realtek rtl93xx PoE switch with two HS104 PSE controllers on i2c:
- clean boot, no probe-retry loop, no watchdog reset
- 10G SFP+ port: module hotplug works, no deadlock
- ethtool --set-pse enable/disable cuts and restores power to a PD
- i2c unbind -> rmmod -> modprobe: PSE detaches on unbind and re-attaches on reload with power restored, no reboot. No lockdep splats.
Quoted 4 lines
Jonas confirmed the RTL8214FC deadlock he reported is gone. Aleksander confirmed the
lantiq_etopprobe deadlock is gone.Tested-by Carlo Szelinsky <[email protected]>
## Form letter - net-next-closed
net-next pull request for v7.3 has already been merged, and therefore the net-next tree is closed for new drivers, features, code refactoring and optimizations. We are currently accepting bug fixes only.
Please repost when net-next reopens after Aug 31st.
RFC patches sent for review only are obviously welcome at any time.
See:
https://www.kernel.org/doc/html/next/process/maintainer-netdev.html#development-cycle
Signature · 2 lines
pw-bot: defer pv-bot: closed