Skip to contentSkip to conversation

Module use count must be updated as bridges are created/destroyed

PATCH In review Jan Beulich 7 participants 18 messages Sep 26, 2007 – Sep 29, 2007 2 messages not in the archive

Subsystems (3)
Networking IPv4/IPv6 Ethernet bridge
lore
Share & tools
lore

Jan Beulich

author
#0 lore raw

PATCH 1/1 Module use count must be updated as bridges are created/destroyed

Otherwise 'modprobe -r' on a module having a dependency on bridge will implicitly unload bridge, bringing down all connectivity that was using bridges.

Signed-off-by Jan Beulich <[email protected]>
 net/bridge/br_if.c |    9 +++++++++
 1 file changed, 9 insertions(+)
net/bridge/br_if.c +9 -0
@@ -276,6 +276,11 @@ int br_add_bridge(const char *name)
 	if (!dev)
 		return -ENOMEM;
 
+	if (!try_module_get(THIS_MODULE)) {
+		free_netdev(dev);
+		return -ENOENT;
+	}
+
 	rtnl_lock();
 	if (strchr(dev->name, '%')) {
 		ret = dev_alloc_name(dev, dev->name);

@@ -294,6 +299,8 @@ int br_add_bridge(const char *name)
 		unregister_netdevice(dev);
  out:
 	rtnl_unlock();
+	if (ret)
+		module_put(THIS_MODULE);
 	return ret;
 }
 

@@ -321,6 +328,8 @@ int br_del_bridge(const char *name)
 		del_br(netdev_priv(dev));
 
 	rtnl_unlock();
+	if (ret == 0)
+		module_put(THIS_MODULE);
 	return ret;
 }
#1 lore raw
Quoted 6 lines · On Wed, 26 Sep 2007 08:53:27 +0100 "Jan Beulich" <[email protected]> wrote:

Otherwise 'modprobe -r' on a module having a dependency on bridge will implicitly unload bridge, bringing down all connectivity that was using bridges.

Signed-off-by Jan Beulich <[email protected]>

No, network devices don't do reference counting.
What is the dependency? Where is the source of the module interacting with the bridge?

#2 lore raw

why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

* Wed, 26 Sep 2007 08:37:05 -0700
* Organization: Linux Foundation

Quoted 4 lines

On Wed, 26 Sep 2007 08:53:27 +0100
"Jan Beulich" <[email protected]> wrote:

Quoted 8 lines
Quoted 6 lines

Otherwise 'modprobe -r' on a module having a dependency on bridge will implicitly unload bridge, bringing down all connectivity that was using bridges.

Signed-off-by Jan Beulich <[email protected]>

No, network devices don't do reference counting.

Could you explain why, please?

After udevd on boot loads lots of unused crap, i surrendered, and use $(rmmod lsmod | just first column). Networing bravely wipes away. OK, there are lots of configs: udev, hotplug, modprobe, that somebody might like to fix. But it came to the end with me. I just don't care. So, please answer :)
____

#3 lore raw

Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

Quoted 15 lines · On Wed, 26 Sep 2007 23:06:53 +0200 Oleg Verych <[email protected]> wrote:

* Wed, 26 Sep 2007 08:37:05 -0700
* Organization: Linux Foundation

Quoted 4 lines

On Wed, 26 Sep 2007 08:53:27 +0100
"Jan Beulich" <[email protected]> wrote:

Quoted 8 lines
Quoted 6 lines

Otherwise 'modprobe -r' on a module having a dependency on bridge will implicitly unload bridge, bringing down all connectivity that was using bridges.

Signed-off-by Jan Beulich <[email protected]>

No, network devices don't do reference counting.

Quoted 8 lines

Could you explain why, please?

After udevd on boot loads lots of unused crap, i surrendered, and use $(rmmod lsmod | just first column). Networing bravely wipes away. OK, there are lots of configs: udev, hotplug, modprobe, that somebody might like to fix. But it came to the end with me. I just don't care. So, please answer :)
____

For hotplug and other reasons, the network developers decided that being able to remove a network module at any time was a good thing. It works.

Signature · 1 line
Stephen Hemminger <[email protected]>
#4 lore raw

Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

Quoted 5 lines · Stephen Hemminger
Quoted 5 lines
Quoted 2 lines

No, network devices don't do reference counting.

Could you explain why, please?

Quoted 8 lines
Quoted 5 lines

After udevd on boot loads lots of unused crap, i surrendered, and use $(rmmod lsmod | just first column). Networing bravely wipes away. OK, there are lots of configs: udev, hotplug, modprobe, that somebody might like to fix. But it came to the end with me. I just don't care. So, please answer :)

For hotplug and other reasons, the network developers decided that being able to remove a network module at any time was a good thing. It works.

Except that for ipv6.ko, it's all opposite. After modprobe, it already got a refcount like 8 and you're wondering how to get rid of that.

#5 lore raw

Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

Quoted 13 lines · On Thu, 27 Sep 2007 00:18:55 +0200 (CEST) Jan Engelhardt…
Quoted 11 lines · Stephen Hemminger
Quoted 10 lines
Quoted 2 lines

No, network devices don't do reference counting.

Could you explain why, please?

After udevd on boot loads lots of unused crap, i surrendered, and use $(rmmod lsmod | just first column). Networing bravely wipes away. OK, there are lots of configs: udev, hotplug, modprobe, that somebody might like to fix. But it came to the end with me. I just don't care. So, please answer :)

Quoted 6 lines
Quoted 2 lines

For hotplug and other reasons, the network developers decided that being able to remove a network module at any time was a good thing. It works.

Except that for ipv6.ko, it's all opposite. After modprobe, it already got a refcount like 8 and you're wondering how to get rid of that.

ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap all the routes and applications, ...

Signature · 1 line
Stephen Hemminger <[email protected]>
maintainer
#6 lore raw

Re: why network devices don't do reference counting?

From: Stephen Hemminger <[email protected]>
Date: Wed, 26 Sep 2007 15:33:30 -0700

Quoted 2 lines

ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap all the routes and applications, ...

It is purposefully set to have a permanent elevated reference count because it is not designed to be unloaded safely.

It has been unloadable forever.

#7 lore raw

Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

In reply to Stephen Hemminger
Quoted 19 lines · Stephen Hemminger
Quoted 15 lines · On Thu, 27 Sep 2007 00:18:55 +0200 (CEST) Jan Engelhardt…
Quoted 13 lines · Stephen Hemminger
Quoted 10 lines
Quoted 2 lines

No, network devices don't do reference counting.

Could you explain why, please?

After udevd on boot loads lots of unused crap, i surrendered, and use $(rmmod lsmod | just first column). Networing bravely wipes away. OK, there are lots of configs: udev, hotplug, modprobe, that somebody might like to fix. But it came to the end with me. I just don't care. So, please answer :)

For hotplug and other reasons, the network developers decided that being able to remove a network module at any time was a good thing. It works.

Quoted 8 lines
Quoted 4 lines

Except that for ipv6.ko, it's all opposite. After modprobe, it already got a refcount like 8 and you're wondering how to get rid of that.

ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap all the routes and applications, ...

Wouldn't it be enough to down all the interfaces and close all the sockets? No need to bring down every app.

Helge Hafting

#8 lore raw

Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

Quoted 25 lines · On Thu, 27 Sep 2007 13:54:23 +0200 Helge Hafting <[email protected]>…
Quoted 24 lines · Stephen Hemminger
Quoted 19 lines · On Thu, 27 Sep 2007 00:18:55 +0200 (CEST) Jan Engelhardt…
Quoted 4 lines · Stephen Hemminger
Quoted 4 lines
Quoted 2 lines

No, network devices don't do reference counting.

Could you explain why, please?

Quoted 9 lines
Quoted 6 lines

After udevd on boot loads lots of unused crap, i surrendered, and use $(rmmod lsmod | just first column). Networing bravely wipes away. OK, there are lots of configs: udev, hotplug, modprobe, that somebody might like to fix. But it came to the end with me. I just don't care. So, please answer :)

For hotplug and other reasons, the network developers decided that being able to remove a network module at any time was a good thing. It works.

Except that for ipv6.ko, it's all opposite. After modprobe, it already got a refcount like 8 and you're wondering how to get rid of that.

Quoted 5 lines
Quoted 3 lines

ipv6 is not a network driver, it is a protocol. You might be able to remove it if you zap all the routes and applications, ...

Wouldn't it be enough to down all the interfaces and close all the sockets? No need to bring down every app.

You need every socket to close and all routes to go away including the routes through loopback device, and still there probably are control sockets buried inside ipv6 that hold ref count.

IMHO the kernel should just admit that IPV6 can't be removed.

net/ipv6/af_inet6.c +4 -0
@@ -914,6 +914,9 @@ out_unregister_tcp_proto:
 }
 module_init(inet6_init);
 
+/* Disabled at present because it is impossible to remove all references */
+#ifdef IPV6_UNLOAD
+
 static void __exit inet6_exit(void)
 {
 	/* First of all disallow new sockets creation. */

@@ -952,5 +955,6 @@ static void __exit inet6_exit(void)
 	proto_unregister(&tcpv6_prot);
 }
 module_exit(inet6_exit);
+#endif
 
 MODULE_ALIAS_NETPROTO(PF_INET6);
net/ipv6/addrconf.c +2 -0
@@ -4255,6 +4255,7 @@ errout:
 	return err;
 }
 
+#ifdef IPV6_UNLOAD
 void __exit addrconf_cleanup(void)
 {
 	struct net_device *dev;

@@ -4308,3 +4309,4 @@ void __exit addrconf_cleanup(void)
 	proc_net_remove(&init_net, "if_inet6");
 #endif
 }
+#endif
net/ipv6/ipv6_sockglue.c +3 -1
@@ -1132,7 +1132,9 @@ void __init ipv6_packet_init(void)
 	dev_add_pack(&ipv6_packet_type);
 }
 
-void ipv6_packet_cleanup(void)
+#ifdef IPV6_UNLOAD
+void __exit ipv6_packet_cleanup(void)
 {
 	dev_remove_pack(&ipv6_packet_type);
 }
+#endif
#9 lore raw

Re: why network devices don't do reference counting? (Re: [PATCH] Module use count must be updated as bridges are created/destroyed)

Quoted 6 lines · Stephen Hemminger

You need every socket to close and all routes to go away including the routes through loopback device, and still there probably are control sockets buried inside ipv6 that hold ref count.

IMHO the kernel should just admit that IPV6 can't be removed.

I cannot accept that. If ipv6.ko has a way to tack ipv6 structs onto all sockets, interfaces and addresses, it should also be able to untack it again.

maintainer
#10 lore raw

Re: why network devices don't do reference counting?

From: Jan Engelhardt <[email protected]>
Date: Thu, 27 Sep 2007 16:55:36 +0200 (CEST)

Quoted 7 lines · Stephen Hemminger
Quoted 5 lines

You need every socket to close and all routes to go away including the routes through loopback device, and still there probably are control sockets buried inside ipv6 that hold ref count.

Quoted 5 lines
Quoted 1 line

IMHO the kernel should just admit that IPV6 can't be removed.

I cannot accept that. If ipv6.ko has a way to tack ipv6 structs onto all sockets, interfaces and addresses, it should also be able to untack it again.

Then you do the work and see how incredibly difficult the implementation is, others have tried. It's not trivial and at best it's a very time consuming piece of work to embark on.

Until then it's un-removable, plain as that :-)

I don't know why we're wasting our fingers discussing this at all to be honest with you.

maintainer
#11 lore raw

Re: why network devices don't do reference counting?

In reply to Helge Hafting

From: Helge Hafting <[email protected]>
Date: Thu, 27 Sep 2007 13:54:23 +0200

Quoted 2 lines

Wouldn't it be enough to down all the interfaces and close all the sockets? No need to bring down every app.

And there are routes, and neighbour cache entries, and all sorts of external references to the stack. For example, if a packet gets stuck in a device because the link just went down, that can hold references to the ipv6 module from several angles.

But you have to add code to actually keep track of all of these references and there is no such code in the ipv6 module at all and it's a nontrivial time consuming job to implement it.

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#14 lore raw
Quoted 25 lines
Quoted 1 line
Quoted 1 line

Stephen Hemminger <[email protected]> 26.09.07 19:12 >>>

Quoted 21 lines · On Wed, 26 Sep 2007 17:08:19 +0100 "Jan Beulich" <[email protected]> wrote:
Quoted 8 lines
Quoted 1 line
Quoted 1 line

Stephen Hemminger <[email protected]> 26.09.07 17:37 >>>

Quoted 4 lines · On Wed, 26 Sep 2007 08:53:27 +0100 "Jan Beulich" <[email protected]> wrote:

Otherwise 'modprobe -r' on a module having a dependency on bridge will implicitly unload bridge, bringing down all connectivity that was using bridges.

Quoted 6 lines
Quoted 2 lines
Signed-off-by Jan Beulich <[email protected]>

No, network devices don't do reference counting.
What is the dependency? Where is the source of the module interacting with the bridge?

On a Xen system, I loaded and then unloaded ebtable_broute. The unload implicitly unloaded bridge, destroying the network. The only way I could see to avoid the implicit unload was to bump the reference count on bridge creation. Otherwise I would have to ask why bridge has a zero reference count despite a bridge being configured.

Quoted 4 lines
Quoted 1 line

Jan

Sounds like a module utilities problem since unloading one module doesn't normally unload others.

I have to disagree here - 'modprobe -r' is specifically unloading all modules the specified one references as long as they have a use count of zero. The difference to other networking modules is that the latter normally don't export symbols, and hence don't have dependent modules (and thus cannot be subject of implicit unloading). Bridge does have dependents, and hence must avoid implicit unloading by managing its use count.

Jan

#15 lore raw
Quoted 30 lines · On Thu, 27 Sep 2007 08:40:10 +0100 "Jan Beulich" <[email protected]> wrote:
Quoted 25 lines
Quoted 1 line
Quoted 1 line

Stephen Hemminger <[email protected]> 26.09.07 19:12 >>>

Quoted 21 lines · On Wed, 26 Sep 2007 17:08:19 +0100 "Jan Beulich" <[email protected]> wrote:
Quoted 8 lines
Quoted 1 line
Quoted 1 line

Stephen Hemminger <[email protected]> 26.09.07 17:37 >>>

Quoted 4 lines · On Wed, 26 Sep 2007 08:53:27 +0100 "Jan Beulich" <[email protected]> wrote:

Otherwise 'modprobe -r' on a module having a dependency on bridge will implicitly unload bridge, bringing down all connectivity that was using bridges.

Quoted 6 lines
Quoted 2 lines
Signed-off-by Jan Beulich <[email protected]>

No, network devices don't do reference counting.
What is the dependency? Where is the source of the module interacting with the bridge?

On a Xen system, I loaded and then unloaded ebtable_broute. The unload implicitly unloaded bridge, destroying the network. The only way I could see to avoid the implicit unload was to bump the reference count on bridge creation. Otherwise I would have to ask why bridge has a zero reference count despite a bridge being configured.

Quoted 4 lines
Quoted 1 line

Jan

Sounds like a module utilities problem since unloading one module doesn't normally unload others.

Quoted 6 lines

I have to disagree here - 'modprobe -r' is specifically unloading all modules the specified one references as long as they have a use count of zero. The difference to other networking modules is that the latter normally don't export symbols, and hence don't have dependent modules (and thus cannot be subject of implicit unloading). Bridge does have dependents, and hence must avoid implicit unloading by managing its use count.

I want keep the behavior that:
modprobe -r bridge
removes all bridges. It is too useful and may already be in some user scripts.

Signature · 1 line
Stephen Hemminger <[email protected]>
author
#16 lore raw
Quoted 10 lines
Quoted 9 lines
Quoted 2 lines

Sounds like a module utilities problem since unloading one module doesn't normally unload others.

I have to disagree here - 'modprobe -r' is specifically unloading all modules the specified one references as long as they have a use count of zero. The difference to other networking modules is that the latter normally don't export symbols, and hence don't have dependent modules (and thus cannot be subject of implicit unloading). Bridge does have dependents, and hence must avoid implicit unloading by managing its use count.

Quoted 3 lines

I want keep the behavior that:
modprobe -r bridge
removes all bridges. It is too useful and may already be in some user scripts.

So we have an unsolvable problem here then, unless infrastructure gets added that allows a module to declare itself as not-implicit-unload-safe, forcing modprobe -r to keep its hands off it. Ugly.

Jan

#17 lore raw
Quoted 4 lines · Jan Beulich

So we have an unsolvable problem here then, unless infrastructure gets added that allows a module to declare itself as not-implicit-unload-safe, forcing modprobe -r to keep its hands off it. Ugly.

Yes I've always wanted to have a separate count that indicates a module is in use but does not prevent its immediate removal.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#18 lore raw
Quoted 10 lines · On Fri, 28 Sep 2007 13:37:39 +0800 Herbert Xu <[email protected]>…
Quoted 4 lines · Jan Beulich

So we have an unsolvable problem here then, unless infrastructure gets added that allows a module to declare itself as not-implicit-unload-safe, forcing modprobe -r to keep its hands off it. Ugly.

Yes I've always wanted to have a separate count that indicates a module is in use but does not prevent its immediate removal.

Cheers,

It is only the nested removal problem.

Signature · 1 line
Stephen Hemminger <[email protected]>
#19 lore raw
Quoted 2 lines · Stephen Hemminger

It is only the nested removal problem.

Well such a count could apply to direct removals too. In other words, you could ask modprobe to remove all modules for which this count is zero.

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt