Phy, mdiobus, and netdev struct device fixes
Subsystems (10)
v3 cover Phy, mdiobus, and netdev struct device fixes
Hi,
The third version of this series fixes the build error which David identified, and drops the broken changes for the Cavium Thunger BGX ethernet driver as this driver requires some complex changes to resolve the leakage - and this is best done by people who can test the driver.
Compared to v2, the only patch which has changed is patch 6
"net: fix phy refcounting in a bunch of drivers"
I _think_ I've been able to build-test all the drivers touched by that patch to some degree now, though several of them needed the Kconfig hacked to allow it (not all had || COMPILE_TEST clause on their dependencies.)
Previous cover letters below:
This is the second version of the series, with the comments David had on the first patch fixed up. Original series description with updated diffstat below.
While looking at the DSA code, I noticed we have a of_find_net_device_by_node(), and it looks like users of that are similarly buggy - it looks like net/dsa/dsa.c is the only user. Fix that too.
Hi,
While looking at the phy code, I identified a number of weaknesses where refcounting on device structures was being leaked, where modules could be removed while in-use, and where the fixed-phy could end up having unintended consequences caused by incorrect calls to fixed_phy_update_state().
This patch series resolves those issues, some of which were discovered with testing on an Armada 388 board. Not all patches are fully tested, particularly the one which touches several network drivers.
When resolving the struct device refcounting problems, several different solutions were considered before settling on the implementation here - one of the considerations was to avoid touching many network drivers. The solution here is:
phy_attach*() - takes a refcount phy_detach*() - drops the phy_attach refcount
Provided drivers always attach and detach their phys, which they should already be doing, this should change nothing, even if they leak a refcount.
of_phy_find_device() and of_* functions which use that take a refcount. Arrange for this refcount to be dropped once the phy is attached.
This is the reason why the previous change is important - we can't drop this refcount taken by of_phy_find_device() until something else holds a reference on the device. This resolves the leaked refcount caused by using of_phy_connect() or of_phy_attach().
Even without the above changes, these drivers are leaking by calling of_phy_find_device(). These drivers are addressed by adding the appropriate release of that refcount.
The mdiobus code also suffered from the same kind of leak, but thankfully this only happened in one place - the mdio-mux code.
I also found that the try_module_get() in the phy layer code was utterly
useless: phydev->dev.driver was guaranteed to always be NULL, so try_module_get() was always being called with a NULL argument. I proved this with my SFP code, which declares its own MDIO bus - the module use count was never incremented irrespective of how I set the MDIO bus up. This allowed the MDIO bus code to be removed from the kernel while there were still PHYs attached to it.
One other bug was discovered: while using in-band-status with mvneta, it was found that if a real phy is attached with in-band-status enabled, and another ethernet interface is using the fixed-phy infrastructure, the interface using the fixed-phy infrastructure is configured according to the other interface using the in-band-status - which is caused by the fixed-phy code not verifying that the phy_device passed in is actually a fixed-phy device, rather than a real MDIO phy.
Lastly, having mdio_bus reversing phy_device_register() internals seems like a layering violation - it's trivial to move that code to the phy device layer.
drivers/net/ethernet/apm/xgene/xgene_enet_hw.c | 24 ++++++---- drivers/net/ethernet/freescale/gianfar.c | 6 ++- drivers/net/ethernet/freescale/ucc_geth.c | 8 +++- drivers/net/ethernet/marvell/mvneta.c | 2 + drivers/net/ethernet/xilinx/xilinx_emaclite.c | 2 + drivers/net/phy/fixed_phy.c | 2 +- drivers/net/phy/mdio-mux.c | 19 +++++--- drivers/net/phy/mdio_bus.c | 24 ++++++---- drivers/net/phy/phy_device.c | 62 ++++++++++++++++++++------ drivers/of/of_mdio.c | 27 +++++++++-- include/linux/phy.h | 6 ++- net/core/net-sysfs.c | 9 ++++ net/dsa/dsa.c | 41 ++++++++++++++--- 13 files changed, 181 insertions(+), 51 deletions(-)
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
PATCH 1/9 phy: fix of_mdio_find_bus() device refcount leak
of_mdio_find_bus() leaks a struct device refcount, caused by using class_find_device() and not realising that the device reference has its refcount incremented:
* Note, you will need to drop the reference with put_device() after use.
...
while ((dev = class_dev_iter_next(&iter))) {
if (match(dev, data)) {
get_device(dev);
break;
}Update the comment, and arrange for the phy code to drop this refcount when disposing of a reference to it.
--- drivers/net/phy/mdio-mux.c | 19 +++++++++++++------ drivers/net/phy/mdio_bus.c | 4 +++- 2 files changed, 16 insertions(+), 7 deletions(-)
drivers/net/phy/mdio-mux.c +13 -6
@@ -113,18 +113,18 @@ int mdio_mux_init(struct device *dev,if (!parent_bus_node)return -ENODEV ; - parent_bus =of_mdio_find_bus (parent_bus_node); -if (parent_bus ==NULL ) { - ret_val = -EPROBE_DEFER ; - goto err_parent_bus; - } - pb =devm_kzalloc (dev,sizeof (*pb),GFP_KERNEL );if (pb ==NULL ) { ret_val = -ENOMEM ; goto err_parent_bus; } + parent_bus =of_mdio_find_bus (parent_bus_node); +if (parent_bus ==NULL ) { + ret_val = -EPROBE_DEFER ; + goto err_parent_bus; + } + pb->switch_data = data; pb->switch_fn = switch_fn; pb->current_child =-1 ;
@@ -173,6 +173,10 @@ int mdio_mux_init(struct device *dev,dev_info (dev,"Version " DRV_VERSION "\n" );return 0 ; } + +/* balance the reference of_mdio_find_bus() took */ +put_device (&pb->mii_bus->dev); + err_parent_bus:of_node_put (parent_bus_node);return ret_val;
@@ -189,6 +193,9 @@ void mdio_mux_uninit(void *mux_handle)mdiobus_free (cb->mii_bus); cb = cb->next; } + +/* balance the reference of_mdio_find_bus() in mdio_mux_init() took */ +put_device (&pb->mii_bus->dev); }EXPORT_SYMBOL_GPL (mdio_mux_uninit);
drivers/net/phy/mdio_bus.c +3 -1
@@ -167,7 +167,9 @@ static int of_mdio_bus_match(struct device *dev, const void *mdio_bus_np)* of_mdio_find_bus - Given an mii_bus node, find the mii_bus. * @mdio_bus_np: Pointer to the mii_bus. * -* Returns a pointer to the mii_bus, or NULL if none found. +* Returns a reference to the mii_bus, or NULL if none found. The +* embedded struct device will have its reference count incremented, +* and this must be put once the bus is finished with. * * Because the association of a device_node and mii_bus is made via * of_mdiobus_register(), the mii_bus cannot be found before it is
Signature · 1 line
2.1.0
PATCH 2/9 net: dsa: fix of_mdio_find_bus() device refcount leak
In reply to Russell King - ARM LinuxCurrent users of of_mdio_find_bus() leak a struct device refcount, as they fail to clean up the reference obtained inside class_find_device().
Fix the DSA code to properly refcount the returned MDIO bus by:
1. taking a reference on the struct device whenever we assign it to pd->chip[x].host_dev.
2. dropping the reference when we overwrite the existing reference.
3. dropping the reference when we free the data structure.
4. dropping the initial reference we obtained after setting up the platform data structure, or on failure.
In step 2 above, where we obtain a new MDIO bus, there is no need to take a reference on it as we would only have to drop it immediately after assignment again, iow:
put_device(cd->host_dev); /* drop original assignment ref */ cd->host_dev = get_device(&mdio_bus_switch->dev); /* get our ref */ put_device(&mdio_bus_switch->dev); /* drop of_mdio_find_bus ref */
--- net/dsa/dsa.c | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-)
net/dsa/dsa.c +31 -7
@@ -634,6 +634,10 @@ static void dsa_of_free_platform_data(struct dsa_platform_data *pd) port_index++; }kfree (pd->chip[i].rtable); + +/* Drop our reference to the MDIO bus device */ +if (pd->chip[i].host_dev) +put_device (pd->chip[i].host_dev); }kfree (pd->chip); }
@@ -661,16 +665,22 @@ static int dsa_of_probe(struct device *dev)return -EPROBE_DEFER ; ethernet =of_parse_phandle (np,"dsa,ethernet" ,0 ); -if (!ethernet) -return -EINVAL ; +if (!ethernet) { + ret = -EINVAL ; + goto out_put_mdio; + } ethernet_dev =of_find_net_device_by_node (ethernet); -if (!ethernet_dev) -return -EPROBE_DEFER ; +if (!ethernet_dev) { + ret = -EPROBE_DEFER ; + goto out_put_mdio; + } pd =kzalloc (sizeof (*pd),GFP_KERNEL ); -if (!pd) -return -ENOMEM ; +if (!pd) { + ret = -ENOMEM ; + goto out_put_mdio; + } dev->platform_data = pd; pd->of_netdev = ethernet_dev;
@@ -691,7 +701,9 @@ static int dsa_of_probe(struct device *dev) cd = &pd->chip[chip_index]; cd->of_node = child; - cd->host_dev = &mdio_bus->dev; + +/* When assigning the host device, increment its refcount */ + cd->host_dev =get_device (&mdio_bus->dev); sw_addr =of_get_property (child,"reg" ,NULL );if (!sw_addr)
@@ -711,6 +723,12 @@ static int dsa_of_probe(struct device *dev) ret = -EPROBE_DEFER ; goto out_free_chip; } + +/* Drop the mdio_bus device ref, replacing the host +* device with the mdio_bus_switch device, keeping +* the refcount from of_mdio_find_bus() above. +*/ +put_device (cd->host_dev); cd->host_dev = &mdio_bus_switch->dev; }
@@ -744,6 +762,10 @@ static int dsa_of_probe(struct device *dev) } } +/* The individual chips hold their own refcount on the mdio bus, +* so drop ours */ +put_device (&mdio_bus->dev); +return 0 ; out_free_chip:
@@ -751,6 +773,8 @@ static int dsa_of_probe(struct device *dev) out_free:kfree (pd); dev->platform_data =NULL ; +out_put_mdio: +put_device (&mdio_bus->dev);return ret; }
Signature · 1 line
2.1.0
PATCH 3/9 phy: fix mdiobus module safety
In reply to Russell King - ARM LinuxRe-implement the mdiobus module refcounting to ensure that we actually ensure that the mdiobus module code does not go away while we might call into it.
The old scheme using bus->dev.driver was buggy, because bus->dev is a class device which never has a struct device_driver associated with it, and hence the associated code trying to obtain a refcount did nothing useful.
Instead, take the approach that other subsystems do: pass the module when calling mdiobus_register(), and record that in the mii_bus struct. When we need to increment the module use count in the phy code, use this stored pointer. When the phy is deteched, drop the module refcount, remembering that the phy device might go away at that point.
This doesn't stop the mii_bus going away while there are in-use phys - it merely stops the underlying code vanishing.
--- drivers/net/phy/mdio_bus.c | 5 +++-- drivers/net/phy/phy_device.c | 32 ++++++++++++++++++-------------- include/linux/phy.h | 5 ++++- 3 files changed, 25 insertions(+), 17 deletions(-)
drivers/net/phy/mdio_bus.c +3 -2
@@ -244,7 +244,7 @@ static inline void of_mdiobus_link_phydev(struct mii_bus *mdio,* * Returns 0 on success or < 0 on error. */ -int mdiobus_register (struct mii_bus *bus) +int __mdiobus_register (struct mii_bus *bus,struct module *owner) {int i, err;
@@ -255,6 +255,7 @@ int mdiobus_register(struct mii_bus *bus)BUG_ON (bus->state !=MDIOBUS_ALLOCATED && bus->state !=MDIOBUS_UNREGISTERED ); + bus->owner = owner; bus->dev.parent = bus->parent; bus->dev.class = &mdio_bus_class; bus->dev.groups =NULL ;
@@ -296,7 +297,7 @@ int mdiobus_register(struct mii_bus *bus)device_del (&bus->dev);return err; } -EXPORT_SYMBOL (mdiobus_register); +EXPORT_SYMBOL (__mdiobus_register);void mdiobus_unregister (struct mii_bus *bus) {
drivers/net/phy/phy_device.c +18 -14
@@ -582,10 +582,15 @@ EXPORT_SYMBOL(phy_init_hw);int phy_attach_direct (struct net_device *dev,struct phy_device *phydev,u32 flags,phy_interface_t interface) { +struct mii_bus *bus = phydev->bus;struct device *d = &phydev->dev; -struct module *bus_module;int err; +if (!try_module_get (bus->owner)) { +dev_err (&dev->dev,"failed to get the bus module\n" ); +return -EIO ; + } +/* Assume that if there is no driver, that it doesn't * exist, and we should use the genphy driver. */
@@ -600,20 +605,13 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev, err =device_bind_driver (d);if (err) -return err; + goto error; }if (phydev->attached_dev) {dev_err (&dev->dev,"PHY already attached\n" ); -return -EBUSY ; - } - -/* Increment the bus module reference count */ - bus_module = phydev->bus->dev.driver ? - phydev->bus->dev.driver->owner :NULL ; -if (!try_module_get (bus_module)) { -dev_err (&dev->dev,"failed to get the bus module\n" ); -return -EIO ; + err = -EBUSY ; + goto error; } phydev->attached_dev = dev;
@@ -636,6 +634,10 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev,phy_resume (phydev);return err; + +error: +module_put (bus->owner); +return err; }EXPORT_SYMBOL (phy_attach_direct);
@@ -680,11 +682,9 @@ EXPORT_SYMBOL(phy_attach);*/ void phy_detach (struct phy_device *phydev) { +struct mii_bus *bus;int i; -if (phydev->bus->dev.driver) -module_put (phydev->bus->dev.driver->owner); - phydev->attached_dev->phydev =NULL ; phydev->attached_dev =NULL ;phy_suspend (phydev);
@@ -700,6 +700,10 @@ void phy_detach(struct phy_device *phydev)break ; } } + + bus = phydev->bus; + +module_put (bus->owner); }EXPORT_SYMBOL (phy_detach);
include/linux/phy.h +4 -1
@@ -19,6 +19,7 @@#include <linux/spinlock.h> #include <linux/ethtool.h> #include <linux/mii.h> +#include <linux/module.h> #include <linux/timer.h> #include <linux/workqueue.h> #include <linux/mod_devicetable.h>
@@ -153,6 +154,7 @@ struct sk_buff;* PHYs should register using this structure */ struct mii_bus { +struct module *owner;const char *name;char id[MII_BUS_ID_SIZE ];void *priv;
@@ -198,7 +200,8 @@ static inline struct mii_bus *mdiobus_alloc(void)return mdiobus_alloc_size (0 ); } -int mdiobus_register (struct mii_bus *bus); +int __mdiobus_register (struct mii_bus *bus,struct module *owner); +#define mdiobus_register (bus) __mdiobus_register(bus, THIS_MODULE)void mdiobus_unregister (struct mii_bus *bus);void mdiobus_free (struct mii_bus *bus);struct mii_bus *devm_mdiobus_alloc_size (struct device *dev,int sizeof_priv);
Signature · 1 line
2.1.0
PATCH 4/9 phy: add proper phy struct device refcounting
In reply to Russell King - ARM LinuxTake a refcount on the phy struct device when the phy device is attached to a network device, and drop it after it's detached. This ensures that a refcount is held on the phy device while the device is being used by a network device, thereby preventing the phy_device from being unexpectedly kfree()'d by phy_device_release().
--- drivers/net/phy/phy_device.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
drivers/net/phy/phy_device.c +12 -0
@@ -578,6 +578,7 @@ EXPORT_SYMBOL(phy_init_hw);* generic driver is used. The phy_device is given a ptr to * the attaching device, and given a callback for link status * change. The phy_device is returned to the attaching driver. +* This function takes a reference on the phy device. */ int phy_attach_direct (struct net_device *dev,struct phy_device *phydev,u32 flags,phy_interface_t interface)
@@ -591,6 +592,8 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev,return -EIO ; } +get_device (d); +/* Assume that if there is no driver, that it doesn't * exist, and we should use the genphy driver. */
@@ -636,6 +639,7 @@ int phy_attach_direct(struct net_device *dev, struct phy_device *phydev,return err; error: +put_device (d);module_put (bus->owner);return err; }
@@ -679,6 +683,9 @@ EXPORT_SYMBOL(phy_attach);/** * phy_detach - detach a PHY device from its network device * @phydev: target phy_device struct +* +* This detaches the phy device from its network device and the phy +* driver, and drops the reference count taken in phy_attach_direct(). */ void phy_detach (struct phy_device *phydev) {
@@ -701,8 +708,13 @@ void phy_detach(struct phy_device *phydev) } } +/* +* The phydev might go away on the put_device() below, so avoid +* a use-after-free bug by reading the underlying bus first. +*/ bus = phydev->bus; +put_device (&phydev->dev);module_put (bus->owner); }EXPORT_SYMBOL (phy_detach);
Signature · 1 line
2.1.0
PATCH 5/9 of_mdio: fix MDIO phy device refcounting
In reply to Russell King - ARM Linuxbus_find_device() is defined as:
* This is similar to the bus_for_each_dev() function above, but it
* returns a reference to a device that is 'found' for later use, as
* determined by the @match callback.
and it does indeed return a reference-counted pointer to the device:
while ((dev = next_device(&i)))
if (match(dev, data) && get_device(dev))
^^^^^^^^^^^^^^^
break;
klist_iter_exit(&i);
return dev;What that means is that when we're done with the struct device, we must drop that reference. Neither of_phy_connect() nor of_phy_attach() did this when phy_connect_direct() or phy_attach_direct() failed.
With our previous patch, phy_connect_direct() and phy_attach_direct() take a new refcount on the phy device when successful, so we can drop our local reference immediatley after these functions, whether or not they succeeded.
--- drivers/of/of_mdio.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)
drivers/of/of_mdio.c +23 -4
@@ -197,7 +197,8 @@ static int of_phy_match(struct device *dev, void *phy_np)* of_phy_find_device - Give a PHY node, find the phy_device * @phy_np: Pointer to the phy's device tree node * -* Returns a pointer to the phy_device. +* If successful, returns a pointer to the phy_device with the embedded +* struct device refcount incremented by one, or NULL on failure. */ struct phy_device *of_phy_find_device (struct device_node *phy_np) {
@@ -217,7 +218,9 @@ EXPORT_SYMBOL(of_phy_find_device);* @hndlr: Link state callback for the network device * @iface: PHY data interface type * -* Returns a pointer to the phy_device if successful. NULL otherwise +* If successful, returns a pointer to the phy_device with the embedded +* struct device refcount incremented by one, or NULL on failure. The +* refcount must be dropped by calling phy_disconnect() or phy_detach(). */ struct phy_device *of_phy_connect(struct net_device *dev, struct device_node *phy_np,
@@ -225,13 +228,19 @@ struct phy_device *of_phy_connect(struct net_device *dev, phy_interface_t iface) {struct phy_device *phy =of_phy_find_device (phy_np); +int ret;if (!phy)return NULL ; phy->dev_flags = flags; -return phy_connect_direct (dev, phy, hndlr, iface) ?NULL : phy; + ret =phy_connect_direct (dev, phy, hndlr, iface); + +/* refcount is held by phy_connect_direct() on success */ +put_device (&phy->dev); + +return ret ?NULL : phy; }EXPORT_SYMBOL (of_phy_connect);
@@ -241,17 +250,27 @@ EXPORT_SYMBOL(of_phy_connect);* @phy_np: Node pointer for the PHY * @flags: flags to pass to the PHY * @iface: PHY data interface type +* +* If successful, returns a pointer to the phy_device with the embedded +* struct device refcount incremented by one, or NULL on failure. The +* refcount must be dropped by calling phy_disconnect() or phy_detach(). */ struct phy_device *of_phy_attach (struct net_device *dev,struct device_node *phy_np,u32 flags,phy_interface_t iface) {struct phy_device *phy =of_phy_find_device (phy_np); +int ret;if (!phy)return NULL ; -return phy_attach_direct (dev, phy, flags, iface) ?NULL : phy; + ret =phy_attach_direct (dev, phy, flags, iface); + +/* refcount is held by phy_attach_direct() on success */ +put_device (&phy->dev); + +return ret ?NULL : phy; }EXPORT_SYMBOL (of_phy_attach);
Signature · 1 line
2.1.0
Re: [PATCH 5/9] of_mdio: fix MDIO phy device refcounting
Quoted 107 lines · Russell King
bus_find_device()is defined as:* This is similar to the
bus_for_each_dev()function above, but it
* returns a reference to a device that is 'found' for later use, as
* determined by the @match callback.and it does indeed return a reference-counted pointer to the device:
while ((dev = next_device(&i))) if (match(dev, data) && get_device(dev)) ^^^^^^^^^^^^^^^ break; klist_iter_exit(&i); return dev;What that means is that when we're done with the struct device, we must drop that reference. Neither
of_phy_connect()norof_phy_attach()did this whenphy_connect_direct()orphy_attach_direct()failed.With our previous patch,
phy_connect_direct()andphy_attach_direct()take a new refcount on the phy device when successful, so we can drop our local reference immediatley after these functions, whether or not they succeeded.Signed-off-by Russell King <[email protected]>--- drivers/of/of_mdio.c | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-)drivers/of/of_mdio.c +23
-5@@ -197,7 +197,8 @@ static int of_phy_match(struct device *dev, void *phy_np)* of_phy_find_device - Give a PHY node, find the phy_device * @phy_np: Pointer to the phy's device tree node * -* Returns a pointer to the phy_device. +* If successful, returns a pointer to the phy_device with the embedded +* struct device refcount incremented by one, or NULL on failure. */ struct phy_device *of_phy_find_device (struct device_node *phy_np) {@@ -217,7 +218,9 @@ EXPORT_SYMBOL(of_phy_find_device);* @hndlr: Link state callback for the network device * @iface: PHY data interface type * -* Returns a pointer to the phy_device if successful. NULL otherwise +* If successful, returns a pointer to the phy_device with the embedded +* struct device refcount incremented by one, or NULL on failure. The +* refcount must be dropped by calling phy_disconnect() or phy_detach(). */ struct phy_device *of_phy_connect(struct net_device *dev, struct device_node *phy_np,@@ -225,13 +228,19 @@ struct phy_device *of_phy_connect(struct net_device *dev, phy_interface_t iface) {struct phy_device *phy =of_phy_find_device (phy_np); +int ret;if (!phy)return NULL ; phy->dev_flags = flags; -return phy_connect_direct (dev, phy, hndlr, iface) ?NULL : phy; + ret =phy_connect_direct (dev, phy, hndlr, iface); + +/* refcount is held by phy_connect_direct() on success */ +put_device (&phy->dev); + +return ret ?NULL : phy; }EXPORT_SYMBOL (of_phy_connect);@@ -241,17 +250,27 @@ EXPORT_SYMBOL(of_phy_connect);* @phy_np: Node pointer for the PHY * @flags: flags to pass to the PHY * @iface: PHY data interface type +* +* If successful, returns a pointer to the phy_device with the embedded +* struct device refcount incremented by one, or NULL on failure. The +* refcount must be dropped by calling phy_disconnect() or phy_detach(). */ struct phy_device *of_phy_attach (struct net_device *dev,struct device_node *phy_np,u32 flags,phy_interface_t iface) {struct phy_device *phy =of_phy_find_device (phy_np); +int ret;if (!phy)return NULL ; -return phy_attach_direct (dev, phy, flags, iface) ?NULL : phy; + ret =phy_attach_direct (dev, phy, flags, iface); + +/* refcount is held by phy_attach_direct() on success */ +put_device (&phy->dev); + +return ret ?NULL : phy; }EXPORT_SYMBOL (of_phy_attach); --2.1.0
PATCH 6/9 net: fix phy refcounting in a bunch of drivers
In reply to Russell King - ARM Linuxof_phy_find_device() increments the phy struct device refcount, which we need to properly balance. Add code to network drivers using this function to ensure that the struct device refcount is correctly balanced.
For xgene, looking back in the history, we should be able to use of_phy_connect() with a zero flags argument for the DT case as this is how the driver used to operate prior to de7b5b3d790a ("net: eth: xgene:
change APM X-Gene SoC platform ethernet to support ACPI").
This leaves the Cavium Thunder BGX unfixed; fixing this driver is a complicated task, one which the maintainers need to be involved with.
--- drivers/net/ethernet/apm/xgene/xgene_enet_hw.c | 24 ++++++++++++++++-------- drivers/net/ethernet/freescale/gianfar.c | 3 +++ drivers/net/ethernet/freescale/ucc_geth.c | 8 +++++++- drivers/net/ethernet/marvell/mvneta.c | 2 ++ drivers/net/ethernet/xilinx/xilinx_emaclite.c | 2 ++ 5 files changed, 30 insertions(+), 9 deletions(-)
drivers/net/ethernet/apm/xgene/xgene_enet_hw.c +16 -8
@@ -689,16 +689,24 @@ static int xgene_enet_phy_connect(struct net_device *ndev)netdev_dbg (ndev,"No phy-handle found in DT\n" );return -ENODEV ; } - pdata->phy_dev =of_phy_find_device (phy_np); - } - phy_dev = pdata->phy_dev; + phy_dev =of_phy_connect (ndev, phy_np, &xgene_enet_adjust_link, +0 , pdata->phy_mode); +if (!phy_dev) { +netdev_err (ndev,"Could not connect to PHY\n" ); +return -ENODEV ; + } + + pdata->phy_dev = phy_dev; + }else { + phy_dev = pdata->phy_dev; -if (!phy_dev || -phy_connect_direct (ndev, phy_dev, &xgene_enet_adjust_link, - pdata->phy_mode)) { -netdev_err (ndev,"Could not connect to PHY\n" ); -return -ENODEV ; +if (!phy_dev || +phy_connect_direct (ndev, phy_dev, &xgene_enet_adjust_link, + pdata->phy_mode)) { +netdev_err (ndev,"Could not connect to PHY\n" ); +return -ENODEV ; + } } pdata->phy_speed =SPEED_UNKNOWN ;
drivers/net/ethernet/freescale/gianfar.c +3 -0
@@ -1702,6 +1702,7 @@ static void gfar_configure_serdes(struct net_device *dev) tbiphy =of_phy_find_device (priv->tbi_node);if (!tbiphy) {dev_err (&dev->dev,"error: Could not get TBI device\n" ); +put_device (&tbiphy->dev);return ; }
@@ -1723,6 +1724,8 @@ static void gfar_configure_serdes(struct net_device *dev)phy_write (tbiphy,MII_BMCR ,BMCR_ANENABLE |BMCR_ANRESTART |BMCR_FULLDPLX |BMCR_SPEED1000 ); + +put_device (&tbiphy->dev); }static int __gfar_is_rx_idle (struct gfar_private *priv)
drivers/net/ethernet/freescale/ucc_geth.c +7 -1
@@ -1384,6 +1384,8 @@ static int adjust_enet_interface(struct ucc_geth_private *ugeth) value =phy_read (tbiphy,ENET_TBI_MII_CR ); value &= ~0x1000 ;/* Turn off autonegotiation */ phy_write (tbiphy,ENET_TBI_MII_CR , value); + +put_device (&tbiphy->dev); }init_check_frame_length_mode (ug_info->lengthCheckRx, &ug_regs->maccfg2);
@@ -1702,8 +1704,10 @@ static void uec_configure_serdes(struct net_device *dev)* everything for us? Resetting it takes the link down and requires * several seconds for it to come back. */ -if (phy_read (tbiphy,ENET_TBI_MII_SR ) &TBISR_LSTATUS ) +if (phy_read (tbiphy,ENET_TBI_MII_SR ) &TBISR_LSTATUS ) { +put_device (&tbiphy->dev);return ; + }/* Single clk mode, mii mode off(for serdes communication) */ phy_write (tbiphy,ENET_TBI_MII_ANA ,TBIANA_SETTINGS );
@@ -1711,6 +1715,8 @@ static void uec_configure_serdes(struct net_device *dev)phy_write (tbiphy,ENET_TBI_MII_TBICON ,TBICON_CLK_SELECT );phy_write (tbiphy,ENET_TBI_MII_CR ,TBICR_SETTINGS ); + +put_device (&tbiphy->dev); }/* Configure the PHY for dev.
drivers/net/ethernet/marvell/mvneta.c +2 -0
@@ -3173,6 +3173,8 @@ static int mvneta_probe(struct platform_device *pdev)struct phy_device *phy =of_phy_find_device (dn);mvneta_fixed_link_update (pp, phy); + +put_device (&phy->dev); }return 0 ;
drivers/net/ethernet/xilinx/xilinx_emaclite.c +2 -0
@@ -828,6 +828,8 @@ static int xemaclite_mdio_setup(struct net_local *lp, struct device *dev)if (!phydev)dev_info (dev,"MDIO of the phy is not registered yet\n" ); +else +put_device (&phydev->dev);return 0 ; }
Signature · 1 line
2.1.0
PATCH 7/9 phy: fixed-phy: properly validate phy in fixed_phy_update_state()
In reply to Russell King - ARM LinuxValidate that the phy_device passed into fixed_phy_update_state() is a fixed-phy device before walking the list of phys for a fixed phy at the same address.
--- drivers/net/phy/fixed_phy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/phy/fixed_phy.c +1 -1
@@ -220,7 +220,7 @@ int fixed_phy_update_state(struct phy_device *phydev,struct fixed_mdio_bus *fmb = &platform_fmb;struct fixed_phy *fp; -if (!phydev || !phydev->bus) +if (!phydev || phydev->bus != fmb->mii_bus)return -EINVAL ;list_for_each_entry (fp, &fmb->phys, node) {
Signature · 1 line
2.1.0
PATCH 8/9 phy: add phy_device_remove()
In reply to Russell King - ARM LinuxAdd a phy_device_remove() function to complement phy_device_register(), which undoes the effects of phy_device_register() by removing the phy device from visibility, but not freeing it.
This allows these details to be moved out of the mdio bus code into the phy code where this action belongs.
--- drivers/net/ethernet/freescale/gianfar.c | 5 +++-- drivers/net/phy/mdio_bus.c | 15 ++++++++++----- drivers/net/phy/phy_device.c | 18 ++++++++++++++++++ include/linux/phy.h | 1 + 4 files changed, 32 insertions(+), 7 deletions(-)
drivers/net/ethernet/freescale/gianfar.c +3 -2
@@ -1702,7 +1702,6 @@ static void gfar_configure_serdes(struct net_device *dev) tbiphy =of_phy_find_device (priv->tbi_node);if (!tbiphy) {dev_err (&dev->dev,"error: Could not get TBI device\n" ); -put_device (&tbiphy->dev);return ; }
@@ -1711,8 +1710,10 @@ static void gfar_configure_serdes(struct net_device *dev)* everything for us? Resetting it takes the link down and requires * several seconds for it to come back. */ -if (phy_read (tbiphy,MII_BMSR ) &BMSR_LSTATUS ) +if (phy_read (tbiphy,MII_BMSR ) &BMSR_LSTATUS ) { +put_device (&tbiphy->dev);return ; + }/* Single clk mode, mii mode off(for serdes communication) */ phy_write (tbiphy,MII_TBICON ,TBICON_CLK_SELECT );
drivers/net/phy/mdio_bus.c +10 -5
@@ -291,8 +291,11 @@ int __mdiobus_register(struct mii_bus *bus, struct module *owner) error:while (--i >=0 ) { -if (bus->phy_map[i]) -device_unregister (&bus->phy_map[i]->dev); +struct phy_device *phydev = bus->phy_map[i]; +if (phydev) { +phy_device_remove (phydev); +phy_device_free (phydev); + } }device_del (&bus->dev);return err;
@@ -307,9 +310,11 @@ void mdiobus_unregister(struct mii_bus *bus) bus->state =MDIOBUS_UNREGISTERED ;for (i =0 ; i <PHY_MAX_ADDR ; i++) { -if (bus->phy_map[i]) -device_unregister (&bus->phy_map[i]->dev); - bus->phy_map[i] =NULL ; +struct phy_device *phydev = bus->phy_map[i]; +if (phydev) { +phy_device_remove (phydev); +phy_device_free (phydev); + } }device_del (&bus->dev); }
drivers/net/phy/phy_device.c +18 -0
@@ -384,6 +384,24 @@ int phy_device_register(struct phy_device *phydev)EXPORT_SYMBOL (phy_device_register);/** +* phy_device_remove - Remove a previously registered phy device from the MDIO bus +* @phydev: phy_device structure to remove +* +* This doesn't free the phy_device itself, it merely reverses the effects +* of phy_device_register(). Use phy_device_free() to free the device +* after calling this function. +*/ +void phy_device_remove (struct phy_device *phydev) +{ +struct mii_bus *bus = phydev->bus; +int addr = phydev->addr; + +device_del (&phydev->dev); + bus->phy_map[addr] =NULL ; +} +EXPORT_SYMBOL (phy_device_remove); + +/** * phy_find_first - finds the first PHY device on the bus * @bus: the target MII bus */
include/linux/phy.h +1 -0
@@ -745,6 +745,7 @@ struct phy_device *phy_device_create(struct mii_bus *bus, int addr, int phy_id, struct phy_c45_device_ids *c45_ids);struct phy_device *get_phy_device (struct mii_bus *bus,int addr,bool is_c45);int phy_device_register (struct phy_device *phy); +void phy_device_remove (struct phy_device *phydev);int phy_init_hw (struct phy_device *phydev);int phy_suspend (struct phy_device *phydev);int phy_resume (struct phy_device *phydev);
Signature · 1 line
2.1.0
PATCH 9/9 net: fix net_device refcounting
In reply to Russell King - ARM Linuxof_find_net_device_by_node() uses class_find_device() internally to lookup the corresponding network device. class_find_device() returns a reference to the embedded struct device, with its refcount incremented.
Add a comment to the definition in net/core/net-sysfs.c indicating the need to drop this refcount, and fix the DSA code to drop this refcount when the OF-generated platform data is cleaned up and freed. Also arrange for the ref to be dropped when handling errors.
--- net/core/net-sysfs.c | 9 +++++++++ net/dsa/dsa.c | 5 ++++- 2 files changed, 13 insertions(+), 1 deletion(-)
net/core/net-sysfs.c +9 -0
@@ -1481,6 +1481,15 @@ static int of_dev_node_match(struct device *dev, const void *data)return ret ==0 ? dev->of_node == data : ret; } +/* +* of_find_net_device_by_node - lookup the net device for the device node +* @np: OF device node +* +* Looks up the net_device structure corresponding with the device node. +* If successful, returns a pointer to the net_device with the embedded +* struct device refcount incremented by one, or NULL on failure. The +* refcount must be dropped when done with the net_device. +*/ struct net_device *of_find_net_device_by_node (struct device_node *np) {struct device *dev;
net/dsa/dsa.c +4 -1
@@ -679,7 +679,7 @@ static int dsa_of_probe(struct device *dev) pd =kzalloc (sizeof (*pd),GFP_KERNEL );if (!pd) { ret = -ENOMEM ; - goto out_put_mdio; + goto out_put_ethernet; } dev->platform_data = pd;
@@ -773,6 +773,8 @@ static int dsa_of_probe(struct device *dev) out_free:kfree (pd); dev->platform_data =NULL ; +out_put_ethernet: +put_device (ðernet_dev->dev); out_put_mdio:put_device (&mdio_bus->dev);return ret;
@@ -786,6 +788,7 @@ static void dsa_of_remove(struct device *dev)return ;dsa_of_free_platform_data (pd); +put_device (&pd->of_netdev->dev); kfree(pd); }#else
Signature · 1 line
2.1.0
...
Quoted 4 lines
While looking at the DSA code, I noticed we have a
of_find_net_device_by_node(), and it looks like users of that are similarly buggy - it looks likenet/dsa/dsa.cis the only user. Fix that too.
...
Quoted 2 lines
The mdiobus code also suffered from the same kind of leak, but thankfully this only happened in one place - the mdio-mux code.
Hi Russell
I tested both of these with my board. It is a Freescale Vybrid, using the FEC ethernet driver, and i have three switches attached, using mdio-mux to give three mdio busses.
No obvious regressions, my board boots, the switches are all present and correct. I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1unregister_netdevice: waiting for eth1 to become free. Usage count = 1
i assume because DSA holds a reference. I've not tried a fully module build, DSA has issues with that.
Thanks
Andrew
Quoted 9 lines · Andrew Lunn
Hi Russell
I tested both of these with my board. It is a Freescale Vybrid, using the FEC ethernet driver, and i have three switches attached, using mdio-mux to give three mdio busses.
No obvious regressions, my board boots, the switches are all present and correct. I built the FEC driver as a module, and it won't unload:
Quoted 7 lines
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1i assume because DSA holds a reference. I've not tried a fully module build, DSA has issues with that.
Tested-by Andrew Lunn <[email protected]>
Thanks for testing. Please could you confirm whether the same behaviour is observed without the patches, just to make absolutely sure that isn't a regression.
However, I think you are correct - I'm unable to locate where in the DSA code:
- dst->master_dev's dev_hold() is undone (hence a reference left)
- dst is freed - dsa_probe() allocates it using kzalloc(), but dsa_remove() and it's children don't free this structure.
There's no notifier which detects whether the underlying device has gone away - it registers a netdev notifier (dsa_slave_netdevice_event) but this only deals with slave devices, not the master device.
Thanks.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
Andrew Lunn
maintainerQuoted 3 lines
Thanks for testing. Please could you confirm whether the same behaviour is observed without the patches, just to make absolutely sure that isn't a regression.
So i tested this now.
I have two FEC interfaces. One i my main access interface, and the second is used by DSA to access switches. With your patches, the module Used by count is equal to the number of interfaces which are up.
Without your patches, the count is always 0.
When i try to remove the fec module, without your patches, but DSA still using the interface, i get the same
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
as with your patch. So this is not a regression.
Andrew
Quoted 4 lines · Andrew Lunn
Quoted 3 lines
Thanks for testing. Please could you confirm whether the same behaviour is observed without the patches, just to make absolutely sure that isn't a regression.
Quoted 8 lines
So i tested this now.
I have two FEC interfaces. One i my main access interface, and the second is used by DSA to access switches. With your patches, the module Used by count is equal to the number of interfaces which are up.
Without your patches, the count is always 0.
That will be as a result of the MDIO bus module refcounting patch - "phy: fix mdiobus module safety". The code prior to that patch was totally useless and ineffectual - it might as well not even have been present, because it would never have any effect. bus_module would always be NULL in phy_attach_direct().
While my change makes the code start to work as originally intended, it's still unsafe: there's nothing to stop you manually unbinding the driver providing the MDIO bus from the struct device. The driver will then free the resources it claimed in its probe function, which may include the MMIO mapping for the MDIO bus accessor functions.
If the accessors are then called, despite keeping the mdio bus, phy, etc data structures properly refcounted, the kernel will oops when the (many) MDIO bus drivers hit the free'd MMIO mapping. This is, unfortunately, just another pre-existing bug in this code.
To stop that, we need some way to say "this MDIO bus has been removed, prevent further access" and that needs to be done in a race free way. Right now, that doesn't exist.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
David Miller
maintainerFrom: Andrew Lunn <[email protected]>
Date: Thu, 24 Sep 2015 23:57:31 +0200
Quoted 9 lines
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1i assume because DSA holds a reference. I've not tried a fully module build, DSA has issues with that.
Tested-by Andrew Lunn <[email protected]>
So, is this a regression?
Please don't provide a "Tested-by: " tag is you encounter a new problem which could have been introduced by the changes in question. That _REALLY_ screws everything up for me.
Thank.
Andrew Lunn
maintainerQuoted 8 lines · David Miller
From: Andrew Lunn <[email protected]>
Date: Thu, 24 Sep 2015 23:57:31 +0200Quoted 5 lines
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1
Quoted 6 lines
Quoted 4 lines
i assume because DSA holds a reference. I've not tried a fully module build, DSA has issues with that.
Tested-by Andrew Lunn <[email protected]>So, is this a regression?
Sorry, worded that badly. Since DSA is still active, it should not be possible to unload the FEC driver. DSA should have a reference to it, and mdio-mux also should have a reference to the mdio bus of the FEC driver.
As Russell requested, i will re-test without his patches, just to make sure.
Andrew
David Miller
maintainerFrom: Andrew Lunn <[email protected]>
Date: Fri, 25 Sep 2015 00:26:54 +0200
Quoted 16 lines · David Miller
Quoted 8 lines
From: Andrew Lunn <[email protected]>
Date: Thu, 24 Sep 2015 23:57:31 +0200Quoted 5 lines
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1Quoted 6 lines
Quoted 4 lines
i assume because DSA holds a reference. I've not tried a fully module build, DSA has issues with that.
Tested-by Andrew Lunn <[email protected]>So, is this a regression?
Quoted 7 lines
Sorry, worded that badly. Since DSA is still active, it should not be possible to unload the FEC driver. DSA should have a reference to it, and mdio-mux also should have a reference to the mdio bus of the FEC driver.
As Russell requested, i will re-test without his patches, just to make sure.
Something needs to hold into the underlying device refcount of a DSA blob so that an unload can't even be attempted in that state.
Quoted 24 lines · David Miller
From: Andrew Lunn <[email protected]>
Date: Fri, 25 Sep 2015 00:26:54 +0200Quoted 21 lines · David Miller
Quoted 8 lines
From: Andrew Lunn <[email protected]>
Date: Thu, 24 Sep 2015 23:57:31 +0200Quoted 5 lines
I built the FEC driver as a module, and it won't unload:
kernel:unregister_netdevice: waiting for eth1 to become free. Usage count = 1
unregister_netdevice: waiting for eth1 to become free. Usage count = 1Quoted 6 lines
Quoted 4 lines
i assume because DSA holds a reference. I've not tried a fully module build, DSA has issues with that.
Tested-by Andrew Lunn <[email protected]>So, is this a regression?
Sorry, worded that badly. Since DSA is still active, it should not be possible to unload the FEC driver. DSA should have a reference to it, and mdio-mux also should have a reference to the mdio bus of the FEC driver.
Quoted 5 lines
Quoted 2 lines
As Russell requested, i will re-test without his patches, just to make sure.
Something needs to hold into the underlying device refcount of a DSA blob so that an unload can't even be attempted in that state.
Holding a reference on a struct device does _not_ stop the device being unbound or the module driving it being removed. It merely stops the struct device from being freed before all references have been dropped.
Devices are always free to be unbound from their bound drivers irrespective of the struct device refcount. Even taking a reference on the module (via try_module_get()) does not stop this.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.
Quoted 8 lines · Russell King - ARM Linux
Hi,
The third version of this series fixes the build error which David identified, and drops the broken changes for the Cavium Thunger BGX ethernet driver as this driver requires some complex changes to resolve the leakage - and this is best done by people who can test the driver.
Quoted 7 lines
Compared to v2, the only patch which has changed is patch 6
"net: fix phy refcounting in a bunch of drivers"I _think_ I've been able to build-test all the drivers touched by that patch to some degree now, though several of them needed the
Kconfighacked to allow it (not all had ||COMPILE_TESTclause on their dependencies.)
Thanks for fixing that.
Quoted 86 lines
Previous cover letters below:
This is the second version of the series, with the comments David had on the first patch fixed up. Original series description with updated diffstat below.
While looking at the DSA code, I noticed we have a
of_find_net_device_by_node(), and it looks like users of that are similarly buggy - it looks likenet/dsa/dsa.cis the only user. Fix that too.Hi,
While looking at the phy code, I identified a number of weaknesses where refcounting on device structures was being leaked, where modules could be removed while in-use, and where the fixed-phy could end up having unintended consequences caused by incorrect calls to
fixed_phy_update_state().This patch series resolves those issues, some of which were discovered with testing on an Armada 388 board. Not all patches are fully tested, particularly the one which touches several network drivers.
When resolving the struct device refcounting problems, several different solutions were considered before settling on the implementation here - one of the considerations was to avoid touching many network drivers. The solution here is:
phy_attach*() - takes a refcount phy_detach*() - drops the
phy_attachrefcountProvided drivers always attach and detach their phys, which they should already be doing, this should change nothing, even if they leak a refcount.
of_phy_find_device()and of_* functions which use that take a refcount. Arrange for this refcount to be dropped once the phy is attached.This is the reason why the previous change is important - we can't drop this refcount taken by
of_phy_find_device()until something else holds a reference on the device. This resolves the leaked refcount caused by usingof_phy_connect()orof_phy_attach().Even without the above changes, these drivers are leaking by calling
of_phy_find_device(). These drivers are addressed by adding the appropriate release of that refcount.The mdiobus code also suffered from the same kind of leak, but thankfully this only happened in one place - the mdio-mux code.
I also found that the
try_module_get()in the phy layer code was utterly
useless:phydev->dev.driverwas guaranteed to always be NULL, sotry_module_get()was always being called with a NULL argument. I proved this with my SFP code, which declares its own MDIO bus - the module use count was never incremented irrespective of how I set the MDIO bus up. This allowed the MDIO bus code to be removed from the kernel while there were still PHYs attached to it.One other bug was discovered: while using in-band-status with mvneta, it was found that if a real phy is attached with in-band-status enabled, and another ethernet interface is using the fixed-phy infrastructure, the interface using the fixed-phy infrastructure is configured according to the other interface using the in-band-status - which is caused by the fixed-phy code not verifying that the
phy_devicepassed in is actually a fixed-phy device, rather than a real MDIO phy.Lastly, having
mdio_busreversingphy_device_register()internals seems like a layering violation - it's trivial to move that code to the phy device layer.drivers/net/ethernet/apm/xgene/xgene_enet_hw.c | 24 ++++++---- drivers/net/ethernet/freescale/gianfar.c | 6 ++- drivers/net/ethernet/freescale/ucc_geth.c | 8 +++- drivers/net/ethernet/marvell/mvneta.c | 2 + drivers/net/ethernet/xilinx/xilinx_emaclite.c | 2 + drivers/net/phy/fixed_phy.c | 2 +- drivers/net/phy/mdio-mux.c | 19 +++++--- drivers/net/phy/mdio_bus.c | 24 ++++++---- drivers/net/phy/phy_device.c | 62 ++++++++++++++++++++------ drivers/of/of_mdio.c | 27 +++++++++-- include/linux/phy.h | 6 ++- net/core/net-sysfs.c | 9 ++++ net/dsa/dsa.c | 41 ++++++++++++++--- 13 files changed, 181 insertions(+), 51 deletions(-)
Signature · 1 line
Florian
David Miller
maintainerFrom: Florian Fainelli <[email protected]>
Date: Thu, 24 Sep 2015 18:39:56 -0700
Quoted 17 lines · Russell King - ARM Linux
Quoted 8 lines
Hi,
The third version of this series fixes the build error which David identified, and drops the broken changes for the Cavium Thunger BGX ethernet driver as this driver requires some complex changes to resolve the leakage - and this is best done by people who can test the driver.
Quoted 7 lines
Compared to v2, the only patch which has changed is patch 6
"net: fix phy refcounting in a bunch of drivers"I _think_ I've been able to build-test all the drivers touched by that patch to some degree now, though several of them needed the
Kconfighacked to allow it (not all had ||COMPILE_TESTclause on their dependencies.)
Quoted 4 lines
Tested-by Florian Fainelli <[email protected]>Reviewed-by Florian Fainelli <[email protected]>Thanks for fixing that.
Series applied, thanks.