/sbin/hotplug multiplexor
Hi all,
With the advent of a lot of people wanting to use /sbin/hotplug to add their own different types of functions, I want to propose the following replacement for the current /sbin/hotplug:
-----
#!/bin/sh
DIR="/etc/hotplug.d"
for I in "${DIR}/"* ; do
$I $1 &
done
exit 1
-----
Then all scripts/programs/whatever that wants to get called when /sbin/hotplug goes off can add themselves to the /etc/hotplug.d directory.
This should help solve the recent devlabel issue with the current hotplug scripts, and allow things like udev to also watch all hotplug actions.
Any objections or comments? If not, I'll make the changes in the linux-hotplug project and release a new version based on this.
Thanks to Martin Schwenke for giving me this idea (even if he doesn't realize it :)
Note, this is only for the "big" hotplug versions that live on everyone's disk. I'm still advocating something small like a combination of udev and dietHotplug for the initramfs image.
thanks,
greg k-h
Quoted 3 lines
Any objections or comments? If not, I'll make the changes in the linux-hotplug project and release a new version based on this.
Yes, consider what this does if you connect to a FibreChannel grid. You are pushing system load by at least an order of magnitude.
Regards
Oliver
Quoted 6 lines · Oliver Neukum
Quoted 3 lines
Any objections or comments? If not, I'll make the changes in the linux-hotplug project and release a new version based on this.
Yes, consider what this does if you connect to a FibreChannel grid. You are pushing system load by at least an order of magnitude.
When you add a FibreChannel grid, the devices are discovered in sequential order, with SCSI IO happening between each device discovered. In talking to the SCSI people, that should be about 30ms per device found at the quickest. So there's no /sbin/hotplug process storm :)
And even if there is, we have to be able to handle such a load under normal situations anyway :)
thanks,
greg k-h
Am Montag, 14. April 2003 21:54 schrieb Greg KH:
Quoted 7 lines · Oliver Neukum
Quoted 5 lines
Quoted 2 lines
Any objections or comments? If not, I'll make the changes in the linux-hotplug project and release a new version based on this.
Yes, consider what this does if you connect to a FibreChannel grid. You are pushing system load by at least an order of magnitude.
Quoted 7 lines
When you add a FibreChannel grid, the devices are discovered in sequential order, with SCSI IO happening between each device discovered. In talking to the SCSI people, that should be about 30ms per device found at the quickest. So there's no
/sbin/hotplugprocess storm :)And even if there is, we have to be able to handle such a load under normal situations anyway :)
Well, plugging them in is one case. But what is plugged in, will eventually be unplugged as well. That will not require probing.
Now let's be conservative and assume 16KB unswappable memory per task. Now we take the famous 4000 disk case. 64MB. A lot but probably not deadly. But multiply this by 15 and the machine is absolutely dead.
Regards
Oliver
Quoted 16 lines · Oliver Neukum
Am Montag, 14. April 2003 21:54 schrieb Greg KH:
Quoted 7 lines · Oliver Neukum
Quoted 5 lines
Quoted 2 lines
Any objections or comments? If not, I'll make the changes in the linux-hotplug project and release a new version based on this.
Yes, consider what this does if you connect to a FibreChannel grid. You are pushing system load by at least an order of magnitude.
Quoted 7 lines
When you add a FibreChannel grid, the devices are discovered in sequential order, with SCSI IO happening between each device discovered. In talking to the SCSI people, that should be about 30ms per device found at the quickest. So there's no
/sbin/hotplugprocess storm :)And even if there is, we have to be able to handle such a load under normal situations anyway :)
Quoted 7 lines
Well, plugging them in is one case. But what is plugged in, will eventually be unplugged as well. That will not require probing.
Now let's be conservative and assume 16KB unswappable memory per task. Now we take the famous 4000 disk case. 64MB. A lot but probably not deadly. But multiply this by 15 and the machine is absolutely dead.
Ok, then the "Enterprise Edition" of the distros that expect to handle 4000 disks will have to add the following patch to their version of the hotplug package.
In the meantime, the other 99% of current Linux users will exist just fine :)
greg k-h
hotplug renamed from hotplug.orig +1 -1
@@ -2,7 +2,7 @@ DIR="/etc/hotplug.d" for I in "${DIR}/"* ; do - $I $1 & + $I $1 done exit 1
Quoted 6 lines
Quoted 5 lines
Now let's be conservative and assume 16KB unswappable memory per task. Now we take the famous 4000 disk case. 64MB. A lot but probably not deadly. But multiply this by 15 and the machine is absolutely dead.
Quoted 6 lines
Ok, then the "Enterprise Edition" of the distros that expect to handle 4000 disks will have to add the following patch to their version of the hotplug package.
In the meantime, the other 99% of current Linux users will exist just fine :)
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
Regards
Oliver
Quoted 9 lines · Oliver Neukum
Quoted 9 lines
Quoted 4 lines
Now let's be conservative and assume 16KB unswappable memory per task. Now we take the famous 4000 disk case. 64MB. A lot but probably not deadly. But multiply this by 15 and the machine is absolutely dead.
Ok, then the "Enterprise Edition" of the distros that expect to handle 4000 disks will have to add the following patch to their version of the hotplug package.
Quoted 7 lines
Quoted 2 lines
In the meantime, the other 99% of current Linux users will exist just fine :)
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
Personally, this is one reason why I'd much rather see a daemon-based model where each interested daemon can "subscribe" to the messages it is interested in. It's very possible (and likely, i.e. udev) that the steps involved for the daemon to respond to the hotplug event are so lightweight that creating a subprocess to handle them would be very wasteful.
Also, this lends itself to multiple levels of messaging: say, for example, userspace partitioning. How would the proposed scheme manage to invoke the userspace partition reading _after_ udev has done its job? If udev itself generated a message into the queue after the device node had been created, the partitioning code could listen for that instead of the native hotplug event.
Quoted 6 lines · Kevin P. Fleming
Personally, this is one reason why I'd much rather see a daemon-based model where each interested daemon can "subscribe" to the messages it is interested in. It's very possible (and likely, i.e. udev) that the steps involved for the daemon to respond to the hotplug event are so lightweight that creating a subprocess to handle them would be very wasteful.
One of the hotplug programs will create a D-BUS message that anyone can subscribe to. But I don't want to add that to one of the existing hotplug programs, as it's a separate task. That's one reason for the proposed change.
thanks,
greg k-h
Quoted 5 lines · Kevin P. Fleming
Personally, this is one reason why I'd much rather see a daemon-based model where each interested daemon can "subscribe" to the messages it is interested in. It's very possible (and likely, i.e. udev) that the steps involved for the daemon to respond to the hotplug event are so lightweight that creating a subprocess to handle them would be very wasteful.
This screams for d-bus.
I spent the weekend reading about it and I spoke with some of the d-bus hackers.
It is really neat and certainly something we should look into.
See http://www.freedesktop.org/software/dbus/
Robert Love
Quoted 6 lines · Robert Love
I spent the weekend reading about it and I spoke with some of the d-bus hackers.
It is really neat and certainly something we should look into.
This is way too complicated.
It's a message protocol that resembles XDR. The messages are also binary instead of ASCII. Bah. If one would want to subscibe to a small selection of events then the usual filesystem provides a much better concept. For example, a /proc/kernel/event tree.
Signature · 1 line
Frank
Quoted 2 lines · Kevin P. Fleming
Quoted 1 line
Personally, this is one reason why I'd much rather see a daemon-based model where each interested daemon can "subscribe" to the messages it is interested in. It's very possible (and likely, i.e. udev) that the steps involved for the daemon to respond to the hotplug event are so lightweight that creating a subprocess to handle them would be very wasteful.
Historically, one of the motivations for the /sbin/hotplug approach was to avoid the costs of running Yet Another Daemon that's idle almost all the time ... and all it'd do when it learns about a new device is fork an easily-customized shell script, which normally loads driver modules and runs device setup scripts. (Modprobe does per-driver scripts, which are no good for per-device actions, and needs a config file.)
Cheaper all around to have the kernel do that; plus, it had information that was not otherwise available to daemons. Much easier to adopt and evolve shell scripts, too.
That is, the design center was for medium-weight events that always involve starting new processes, and which moreover tend not to run often. How often do you connect a new USB or PCI device? If it takes a full second to react, that's OK; and Linux is usually faster than that. (Windows isn't! :)
I'd certainly agree that some hotplug agents need to be talking with daemons. But I've always thought of them as being application-specific ... tell the print server about a new printer, for example. And what you need to tell that server seems unlikely to be what you'd tell something that's sampling your collection of video or audio streams.
Quoted 1 line · Robert Love
Looks interesting.
- Dave
Quoted 10 lines · Oliver Neukum
Quoted 10 lines
Quoted 5 lines
Now let's be conservative and assume 16KB unswappable memory per task. Now we take the famous 4000 disk case. 64MB. A lot but probably not deadly. But multiply this by 15 and the machine is absolutely dead.
Ok, then the "Enterprise Edition" of the distros that expect to handle 4000 disks will have to add the following patch to their version of the hotplug package.
Quoted 5 lines
Quoted 2 lines
In the meantime, the other 99% of current Linux users will exist just fine :)
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
No, that's for the individual scripts/programs to decide. For example, that's what the current hotplug scripts do, but that's not at all what the udev program wants to do.
thanks,
greg k-h
Quoted 7 lines
Quoted 3 lines
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
No, that's for the individual
scripts/programsto decide. For example, that's what the current hotplug scripts do, but that's not at all what the udev program wants to do.
So have them put a symlink into each subdirectory. This is the way it's done for init since times immemorial.
Regards
Oliver
Quoted 10 lines · Oliver Neukum
Quoted 7 lines
Quoted 3 lines
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
No, that's for the individual
scripts/programsto decide. For example, that's what the current hotplug scripts do, but that's not at all what the udev program wants to do.So have them put a symlink into each subdirectory. This is the way it's done for init since times immemorial.
But the number of different "types" keeps growing. For some programs (like udev) they really don't care about the type, and if you add a new type, it still works just fine. Other programs do care about the type, so they can look at it and make a judgement based on it.
thanks,
greg k-h
Am Montag, 14. April 2003 23:52 schrieb Greg KH:
Quoted 8 lines · Oliver Neukum
Quoted 7 lines
Quoted 6 lines
Quoted 2 lines
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
No, that's for the individual
scripts/programsto decide. For example, that's what the current hotplug scripts do, but that's not at all what the udev program wants to do.
Quoted 7 lines
Quoted 2 lines
So have them put a symlink into each subdirectory. This is the way it's done for init since times immemorial.
But the number of different "types" keeps growing. For some programs (like udev) they really don't care about the type, and if you add a new type, it still works just fine. Other programs do care about the type, so they can look at it and make a judgement based on it.
How can that be? What kind of thing should care about both device addition and routing changes in the same way?
Not needlessly exposing scripts to event types they are not written to handle is an advantage.
Regards
Oliver
Quoted 12 lines · Oliver Neukum
Am Montag, 14. April 2003 23:52 schrieb Greg KH:
Quoted 11 lines · Oliver Neukum
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
Well, for a little elegance you might introduce subdirectories for each type of hotplug event and use only them.
No, that's for the individual
scripts/programsto decide. For example, that's what the current hotplug scripts do, but that's not at all what the udev program wants to do.So have them put a symlink into each subdirectory. This is the way it's done for init since times immemorial.
Quoted 7 lines
Quoted 4 lines
But the number of different "types" keeps growing. For some programs (like udev) they really don't care about the type, and if you add a new type, it still works just fine. Other programs do care about the type, so they can look at it and make a judgement based on it.
How can that be? What kind of thing should care about both device addition and routing changes in the same way?
No, udev doesn't care about routing changes. But there isn't enough hardcoded logic in it to care about the different subsystems, so it wants to figure out that it shouldn't care about an event all on its own.
Quoted 2 lines
Not needlessly exposing scripts to event types they are not written to handle is an advantage.
Ok, I like Arnd's proposal of a "default" directory. Maybe I should change that to "all" as no one better create a subsystem or driver class in the kernel called "all" :)
thanks,
greg k-h
[RFC] /sbin/hotplug multiplexor - take 2
In reply to Greg KHOk, based on the comments so far, how about this proposed version of /sbin/hotplug to provide a multiplexor?
thanks,
greg k-h
----------
#!/bin/sh
DIR="/etc/hotplug.d"
for I in "${DIR}/$1/"* "${DIR}/"all/* ; do
test -x $I && $I $1 ;
done
exit 1
----------
Re: [RFC] /sbin/hotplug multiplexor - take 2
Quoted 2 lines · Greg KH
Ok, based on the comments so far, how about this proposed version of
/sbin/hotplugto provide a multiplexor?
It'd be a reduction in functionality. I could no longer just type "/sbin/hotplug" to see what agents disabled or missing ... or notice that since hotplugging was on, the problem had to be RH9 storing /bin/true into /proc/sys/kernel/hotplug again! :P
If the idea is just to loosen today's "one agent per event" rule (I've had that idea too), then wouldn't it be simpler to just (a1) pay an extra process context, not using "exec" to run /etc/hotplug/$1.agent, and when it returns (a2) THEN try other programs? Or even (b) just modify appropriate agent scripts to do so, instead of /sbin/hotplug?
I'd think better about this problem if I had a handful of examples showing why category-specific or event-specific dispatch wouldn't be preferable.
- Dave
Quoted 10 lines
----------
#!/bin/sh
DIR="/etc/hotplug.d"for I in "${DIR}/$1/"* "${DIR}/"all/* ; do
test -x $I && $I $1 ;done
exit 1
----------
Re: [RFC] /sbin/hotplug multiplexor - take 2
Quoted 8 lines · David Brownell
Quoted 2 lines · Greg KH
Ok, based on the comments so far, how about this proposed version of
/sbin/hotplugto provide a multiplexor?It'd be a reduction in functionality. I could no longer just type "
/sbin/hotplug" to see what agents disabled or missing ... or notice that since hotplugging was on, the problem had to be RH9 storing/bin/trueinto/proc/sys/kernel/hotplugagain! :P
Well you can still do that, the current scripts will get run that way too.
Quoted 10 lines
If the idea is just to loosen today's "one agent per event" rule (I've had that idea too), then wouldn't it be simpler to just (a1) pay an extra process context, not using "exec" to run
/etc/hotplug/$1.agent, and when it returns (a2) THEN try other programs? Or even (b) just modify appropriate agent scripts to do so, instead of/sbin/hotplug?I'd think better about this problem if I had a handful of examples showing why category-specific or event-specific dispatch wouldn't be preferable.
udev is such an example. I want it to run for every hotplug event. To do that with the current scripts, I have to either modify the current scripts to always call it (not a big deal, I have CVS access :) or add a handler for every type of device, and every type of those devices.
devlabel is also another type of example. The author of that had to persuade us to modify the scripts in order to get support for his program. I don't want to be a gating function, with this simple proposal, he could just dump the devlabel script into the /etc/hotplug.d directory in the proper place and everything would just work.
I've also been hearing from lots of other people (interestingly, not publicly, I guess they like to stay hidden for some reason) that also hook the hotplug scripts. They have usually been either recommending that their users patch the current script, or just replace the current ones all together (thereby loosing the module load functionality) in order to support their devices. This proposal would also help them, and their users.
thanks,
greg k-h
Re: [RFC] /sbin/hotplug multiplexor - take 2
In reply to Greg KHQuoted 2 lines · Greg KH
Ok, based on the comments so far, how about this proposed version of
/sbin/hotplugto provide a multiplexor?
I think it would be good to launch only the files ending by a special extension to avoid running backup files. Something like that:
#!/bin/sh
DIR="/etc/hotplug.d"
for I in "${DIR}/$1/"*.hotplug "${DIR}/"all/*.hotplug ; do
test -x $I && $I $1 ;
done
Signature · 1 line
Fred - May the source be with you
Re: [RFC] /sbin/hotplug multiplexor - take 2
Quoted 7 lines · Frederic Lepied
Quoted 2 lines · Greg KH
Ok, based on the comments so far, how about this proposed version of
/sbin/hotplugto provide a multiplexor?I think it would be good to launch only the files ending by a special extension to avoid running backup files. Something like that:
Ah, good idea, will do.
thanks,
greg k-h