udev is too slow creating devices
Hello people!
When I load a module (with modprobe) the relative device is too slowly created with udev, so modprobe return before the device is really created. Because of this my init.d script will fail with modular microcode + udev
test case:
udev + modular microcode:
$ modprobe -r microcode $ modprobe microcode ; microcode_ctl -u => microcode_ctl does NOT find the device
$ modprobe -r microcode $ modprobe microcode ; sleep 3; microcode_ctl -u => microcode_ctl FIND the device
[without udev it is OK, so I assume no errors
in modprobe]
Is it a bug of udev?
Else what workaround I can use? (sleep is to slow for an already to sloow system initialitation)
ciao
cate
Quoted 18 lines · Giacomo A. Catenazzi
Hello people!
When I load a module (with modprobe) the relative device is too slowly created with udev, so modprobe return before the device is really created. Because of this my init.d script will fail with modular microcode + udev
test case:
udev + modular microcode:
$ modprobe -r microcode $ modprobe microcode ; microcode_ctl -u => microcode_ctl does NOT find the device$ modprobe -r microcode $ modprobe microcode ; sleep 3; microcode_ctl -u => microcode_ctl FIND the device
Quoted 6 lines
[without udev it is OK, so I assume no errors
in modprobe]Is it a bug of udev?
Else what workaround I can use? (sleep is to slow for an already to sloow system initialitation)
I don't know of a proper fix, but a better work around would be to use...
while [ ! -c /dev/cpu/microcode ]; do sleep 0; done
...in place of your "sleep 3". Replacing the /dev/cpu/microcode with the actual character device that microcode_ctl is looking for.
Signature · 1 line
Chris
Quoted 4 lines · Giacomo A. Catenazzi
udev + modular microcode:
$ modprobe -r microcode $ modprobe microcode ; microcode_ctl -u => microcode_ctl does NOT find the device
The loading of the module triggers udev to run. There is no guarantee that udev runs before microcode_ctl.
One workaround would be to have microcode_ctl use dnotify to get woken up whenever /dev changes.
There may be other ways to do this as well, but Greg would be the guy for that.
Chris
Quoted 7 lines · Chris Friesen
Quoted 4 lines · Giacomo A. Catenazzi
udev + modular microcode:
$ modprobe -r microcode $ modprobe microcode ; microcode_ctl -u => microcode_ctl does NOT find the device
Quoted 5 lines
The loading of the module triggers udev to run. There is no guarantee that udev runs before
microcode_ctl.One workaround would be to have
microcode_ctluse dnotify to get woken up whenever /dev changes.
Ick, no. Use the /etc/dev.d/ notify method I described. That is what it is there for.
thanks,
greg k-h
Quoted 7 lines · Greg KH
Quoted 1 line · Chris Friesen
One workaround would be to have
microcode_ctluse dnotify to get woken up whenever /dev changes.Ick, no. Use the
/etc/dev.d/notify method I described. That is what it is there for.
Right. Sorry, it's been a while since I looked at udev.
Chris
Quoted 10 lines · Greg KH
Quoted 8 lines · Chris Friesen
Quoted 4 lines · Giacomo A. Catenazzi
udev + modular microcode:
$ modprobe -r microcode $ modprobe microcode ; microcode_ctl -u => microcode_ctl does NOT find the device
Quoted 8 lines
Quoted 3 lines
The loading of the module triggers udev to run. There is no guarantee that udev runs before
microcode_ctl.One workaround would be to have
microcode_ctluse dnotify to get woken up whenever /dev changes.Ick, no. Use the
/etc/dev.d/notify method I described. That is what it is there for.
After a brief discussion with debian udev maintainer, I've an other proposal/opinion.
The "bug" appear only in two places: at udev start and after a modprobe, so IMHO we should correct these two place, so that:
- from a user side perspective it is the right thing! (after a successful modprobe, I expect module and devices are created sussesfully)
- there are not many special case:
with udev use dev.d, else do actions now!
Else every distribution should create a script for every init.d script that would eventually use (also indirectly) a kernel module.
ciao
cate
Quoted 14 lines · Giacomo A. Catenazzi
Hello people!
When I load a module (with modprobe) the relative device is too slowly created with udev, so modprobe return before the device is really created. Because of this my init.d script will fail with modular microcode + udev
test case:
udev + modular microcode:
$ modprobe -r microcode $ modprobe microcode ; microcode_ctl -u => microcode_ctl does NOT find the device
Quoted 8 lines
$ modprobe -r microcode $ modprobe microcode ; sleep 3; microcode_ctl -u => microcode_ctl FIND the device[without udev it is OK, so I assume no errors
in modprobe]Is it a bug of udev?
No, it's just the way it now works.
Quoted 2 lines
Else what workaround I can use? (sleep is to slow for an already to sloow system initialitation)
Run microcode_ctl from a script in /etc/dev.d/microcode/ See the file:
http://kernel.org/pub/linux/utils/kernel/hotplug/RFC-dev.d
for more information on now /etc/dev.d works.
thanks,
greg k-h
Quoted 5 lines · Giacomo A. Catenazzi
The "bug" appear only in two places: at udev start and after a modprobe, so IMHO we should correct these two place, so that:
- from a user side perspective it is the right thing! (after a successful modprobe, I expect module and devices are created sussesfully)
That "expectation" is incorrect. The device node will show up any time after modprobe has started, just because modprobe returns does not mean the device node is present.
Quoted 2 lines
- there are not many special case:
with udev use dev.d, else do actions now!
I don't understand what you are proposing.
Quoted 3 lines
Else every distribution should create a script for every init.d script that would eventually use (also indirectly) a kernel module.
What's wrong with the /etc/dev.d/ location for any type of script that you want to run after a device node has appeared? This is an application specific issue, not a kernel issue.
thanks,
greg k-h
Quoted 3 lines · Greg KH
What's wrong with the
/etc/dev.d/location for any type of script that you want to run after a device node has appeared? This is an application specific issue, not a kernel issue.
The problem is that since most distributions cannot make udev usage mandatory, this would require duplicating in the init script and in the dev.d script whatever needs to be done with the device.
Then there are the issues of scripts needing programs in /usr, which may not be mounted when the module is loaded, or which are interactive and need console access (think fsck).
Basically asyncronous creation of devices requires a ground up redesign of the init scripts of a distribution. I'm not claiming that this is not possible, but it's not going to happen soon.
Signature · 2 lines
ciao, | Marco | [8002 aveHSeIFcnkpI]
Quoted 6 lines · On Tue, 14 Sep 2004 22:43:41 +0200 "Giacomo A. Catenazzi" <[email protected]>…
After a brief discussion with debian udev maintainer, I've an other proposal/opinion.
The "bug" appear only in two places: at udev start and after a modprobe, so IMHO we should correct these two place, so that:
This cannot be fixed easily and there would be little gain but great risks. For example, I have a device where reading partition tables can take up to two seconds. Other devices could be broken and modprobe would never return. This would effectively freeze the system when it happens in boot scripts.
Quoted 3 lines
- from a user side perspective it is the right thing! (after a successful modprobe, I expect module and devices are created sussesfully)
This assumption is wrong, even with a completely static /dev. Common example:
modprobe usb-storage / ppa / whatever
Now, for any reason the kernel is unable to read the partition table. The script tries to access /dev/sda1 => boom
In fact, udev is even an improvement here. Instead of checking the device node repeatedly you simply wait for udev to call you. When it does you can be sure that the device node exists and is valid.
Quoted 3 lines
Else every distribution should create a script for every init.d script that would eventually use (also indirectly) a kernel module.
Why not? This is the best way to achieve true and reliable hotplug support.
Regards
Quoted 8 lines · Marco d'Itri
Quoted 3 lines · Greg KH
What's wrong with the
/etc/dev.d/location for any type of script that you want to run after a device node has appeared? This is an application specific issue, not a kernel issue.The problem is that since most distributions cannot make udev usage mandatory, this would require duplicating in the init script and in the dev.d script whatever needs to be done with the device.
Well, that sounds like a distro problem then, not a kernel or udev one :)
Quoted 3 lines
Then there are the issues of scripts needing programs in /usr, which may not be mounted when the module is loaded, or which are interactive and need console access (think fsck).
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
Quoted 3 lines
Basically asyncronous creation of devices requires a ground up redesign of the init scripts of a distribution. I'm not claiming that this is not possible, but it's not going to happen soon.
Well, it needs to go that way, as the kernel moves toward not providing the device node right when modprobe returns with the advent of multi-threading device discovery and such. That's why the /etc/dev.d heirachy was created, it's not dependant on udev to provide such a solution. I suggest that distros that do not use udev (of which Debian is slowly becoming a miniority) create a tool that causes /etc/dev.d events for a static /dev tree.
thanks,
greg k-h
Quoted 2 lines · Greg KH
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
that's more a band-aid than a fix (I can imagine a userspace hang if the device isn't created for whatever reason), if there's no way to do better than this if you've to run fsck (or if it's not the best to run the fsck inside the dev.d scripts), then probably this needs better fixing. is such a big problem to execute a sys_wait4 to wait the udev userspace to return before returning from the insmod syscall?
Quoted 10 lines · Andrea Arcangeli
Quoted 2 lines · Greg KH
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
that's more a band-aid than a fix (I can imagine a userspace hang if the device isn't created for whatever reason), if there's no way to do better than this if you've to run fsck (or if it's not the best to run the fsck inside the dev.d scripts), then probably this needs better fixing. is such a big problem to execute a
sys_wait4to wait the udev userspace to return before returning from the insmod syscall?
But how do you know what to wait for?
If you modprobe a usb-storage driver, and the usb bus is not done discovering devices, the insmod will instantly return, and only some time later will the device node be created after the device is discovered by the bus and then passed to the module you already loaded.
Sitting and waiting is a band-aid, I agree. That's why we created the /etc/dev.d/ notifier system to fix this issue (that is there for systems that don't even use udev.)
thanks,
greg k-h
Quoted 5 lines · Greg KH
Quoted 4 lines · Andrea Arcangeli
Quoted 2 lines · Greg KH
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
Quoted 8 lines
Quoted 6 lines
that's more a band-aid than a fix (I can imagine a userspace hang if the device isn't created for whatever reason), if there's no way to do better than this if you've to run fsck (or if it's not the best to run the fsck inside the dev.d scripts), then probably this needs better fixing. is such a big problem to execute a
sys_wait4to wait the udev userspace to return before returning from the insmod syscall?But how do you know what to wait for?
the kernel sure can know about it, by passing a waitqueue into the registration routine and calling wake_up once the discovery is over.
Quoted 3 lines
Sitting and waiting is a band-aid, I agree. That's why we created the
/etc/dev.d/notifier system to fix this issue (that is there for systems that don't even use udev.)
if the fsck can run from there ok, if for some reason it's not feasible to run it from there (like if it would create a bus congestion by running all the fsck in parallel if you attach multiple devices at the same time), and/or you need some other seriaization, then probably having a way to run things serially without a spin-and-sleep-and-risk-to-hang could be needed. I guess in the worst case one could serialize things by using file locking in /var/run and by creating an API between the dev.d and the init.d scripts, is that how the long term is supposed to work?
So it's more a question if the current interface is complete for all usages, and if the fsck spin-and-sleep is just a temporary band-aid, or if the spin-and-sleep is supposed to last longer than a few month release cycle.
(and I'm not a udev guru, just in listen mode ;)
Quoted 32 lines · Andrea Arcangeli
Quoted 5 lines · Greg KH
Quoted 4 lines · Andrea Arcangeli
Quoted 2 lines · Greg KH
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
Quoted 8 lines
Quoted 6 lines
that's more a band-aid than a fix (I can imagine a userspace hang if the device isn't created for whatever reason), if there's no way to do better than this if you've to run fsck (or if it's not the best to run the fsck inside the dev.d scripts), then probably this needs better fixing. is such a big problem to execute a
sys_wait4to wait the udev userspace to return before returning from the insmod syscall?But how do you know what to wait for?
the kernel sure can know about it, by passing a waitqueue into the registration routine and calling
wake_uponce the discovery is over.Quoted 3 lines
Sitting and waiting is a band-aid, I agree. That's why we created the
/etc/dev.d/notifier system to fix this issue (that is there for systems that don't even use udev.)if the fsck can run from there ok, if for some reason it's not feasible to run it from there (like if it would create a bus congestion by running all the fsck in parallel if you attach multiple devices at the same time), and/or you need some other seriaization, then probably having a way to run things serially without a spin-and-sleep-and-risk-to-hang could be needed. I guess in the worst case one could serialize things by using file locking in
/var/runand by creating an API between the dev.d and the init.d scripts, is that how the long term is supposed to work?
Quoted 4 lines
So it's more a question if the current interface is complete for all usages, and if the fsck spin-and-sleep is just a temporary band-aid, or if the spin-and-sleep is supposed to last longer than a few month release cycle.
Surely fsck can be done on top of /etc/dev.d/ by just writing to a FIFO? That'll serialize up the fsck caller...
Signature · 3 lines
// Gianni Tedesco (gianni at scaramanga dot co dot uk) lynx --source www.scaramanga.co.uk/scaramanga.asc | gpg --import 8646BE7D: 6D9F 2287 870E A2C9 8F60 3A3C 91B5 7669 8646 BE7D
Quoted 2 lines · Gianni Tedesco
Surely fsck can be done on top of
/etc/dev.d/by just writing to a FIFO? That'll serialize up the fsck caller...
with file locking in /var/run I didn't necessairly meant to call flock(2), the fifo in your example would act like a mutex lock, but more than the fifo, we need a bit more of API, the dev.d script will have to look for a certain file in /var/run and block on it to serialize. I just hope the spin loops will go away soon from the userspace ;)
Quoted 3 lines · Greg KH
Sitting and waiting is a band-aid, I agree. That's why we created the
/etc/dev.d/notifier system to fix this issue (that is there for systems that don't even use udev.)
Hello Greg!
[note that I really appreciate udev].
You said me to use dev.d, but really I don't see it as a working solution. Maybe because I missinterpreted the documentation, so maybe you can give me some hints.
Real case: distribution boot script.
It should work with non udev kernel, udev non modular kernel and udev very modular kernel. The script load (directly or not) a module and need the device impelmented by module.
Old behaviour (modprobe "waits" for the creation of device):
normal init.d script, with normal boot priorities.
New behaviour (dev.d). What I should do?
My init.d script is loaded with priority XX, so
I require that dev.d on my device is executed after boot priority XX (else I don't have the needed functionalities), also in case of non-udev or non modular kernel. How should I implement script in dev.d/?
I see some design problems in dev.d/, or am I wrong?
ciao
cate
PS: - What are the best (on topic) mailing list?
- What do other distributions?
Quoted 1 line · Giacomo A. Catenazzi
Old behaviour (modprobe "waits" for the creation of device):
I wonder if it would be feasible for modprobe (or some other utility) to have a new option: --wait-for=/dev/something which would wait for the device node to appear. Perhaps by:
- some mechanism based on HAL, DBUS, whatever
- dnotify on /dev/?
- falling back to spinning and waiting.
The first two sound nice since they avoid spinning and sleeping for a second at a time, which just adds to the delay.
In addition a timeout of some sort to give an upper bound would seem like a good idea.
Ian.
Signature · 4 lines
Ian Campbell, Senior Design Engineer
Web: http://www.arcom.com
Arcom, Clifton Road, Direct: +44 (0)1223 403 465
Cambridge CB1 7EA, United Kingdom Phone: +44 (0)1223 411 200Salut,
Quoted 6 lines · Ian Campbell
I wonder if it would be feasible for modprobe (or some other utility) to have a new option: --wait-for=/dev/something which would wait for the device node to appear. Perhaps by:
- some mechanism based on HAL, DBUS, whatever
- dnotify on/dev/?
- falling back to spinning and waiting.
This would end up as hideous misfeature as you can't guarantee the device to show up *at* *all*.
The reason udev is there is that we can dynamically respond to created device nodes and devices that show up. They might have changed since the last boot. Maybe they don't show up at all.
Thus you should trigger your actions from /etc/dev.d.
Tonnerre
Quoted 11 lines · Tonnerre
Quoted 6 lines · Ian Campbell
I wonder if it would be feasible for modprobe (or some other utility) to have a new option: --wait-for=/dev/something which would wait for the device node to appear. Perhaps by:
- some mechanism based on HAL, DBUS, whatever
- dnotify on/dev/?
- falling back to spinning and waiting.
Quoted 8 lines
This would end up as hideous misfeature as you can't guarantee the device to show up *at* *all*.
The reason udev is there is that we can dynamically respond to created device nodes and devices that show up. They might have changed since the last boot. Maybe they don't show up at all.
Thus you should trigger your actions from
/etc/dev.d.
It is right.
But an option --wait would be sufficient.
This option will require modprobe to wait (with a timeout of x seconds) that hotplug event finish (so if device is created or not is no more a problem).
Ideally this should be done modifing only hotplug and IMHO should be enabled by default.
ciao
cate
Quoted 6 lines · Ian Campbell
Quoted 1 line · Giacomo A. Catenazzi
Old behaviour (modprobe "waits" for the creation of device):
I wonder if it would be feasible for modprobe (or some other utility) to have a new option: --wait-for=/dev/something which would wait for the device node to appear.
Why? What's the point? You are asking to paper over the way the kernel works (remember storage devices are on lots of different kinds of busses today...)
The fact that a static /dev keeps these race conditions from showing up as often as they really are there is no reason to keep trying to rely on old, undefined behaviour :)
thanks,
greg k-h
Quoted 4 lines · Giacomo A. Catenazzi
Real case: distribution boot script.
It should work with non udev kernel, udev non modular kernel and udev very modular kernel.
That's a lot to ask for a simple script :)
Why not just pick one of those options and go with it?
Quoted 2 lines
The script load (directly or not) a module
and need the device impelmented by module.
Then do the 'sit and spin' method. That works for all of the above options, right?
Quoted 2 lines
Old behaviour (modprobe "waits" for the creation of device):
normal init.d script, with normal boot priorities.
Please realize that this "behaviour" is not guaranteed at all. And in fact will be changing in the future so that it is _not_ the way it will work.
Quoted 6 lines
New behaviour (dev.d). What I should do?
My init.d script is loaded with priority XX, so
I require that dev.d on my device is executed after boot priority XX (else I don't have the needed functionalities), also in case of non-udev or non modular kernel. How should I implement script in dev.d/?
dev.d is called by the kernel, not your script. You don't have to set a priority of when dev.d runs at all, it's not necessary.
Quoted 1 line
I see some design problems in dev.d/, or am I wrong?
What problems?
Quoted 1 line
PS: - What are the best (on topic) mailing list?
linux-hotplug-devel
Quoted 1 line
- What do other distributions?
What do they do? They just work :)
thanks,
greg k-h
Quoted 13 lines · Andrea Arcangeli
Quoted 12 lines · Greg KH
Quoted 10 lines · Andrea Arcangeli
Quoted 2 lines · Greg KH
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
that's more a band-aid than a fix (I can imagine a userspace hang if the device isn't created for whatever reason), if there's no way to do better than this if you've to run fsck (or if it's not the best to run the fsck inside the dev.d scripts), then probably this needs better fixing. is such a big problem to execute a
sys_wait4to wait the udev userspace to return before returning from the insmod syscall?
Quoted 4 lines
Quoted 1 line
But how do you know what to wait for?
the kernel sure can know about it, by passing a waitqueue into the registration routine and calling
wake_uponce the discovery is over.
But the low level driver (like a USB driver for example), has no way of knowing when the "device discovery" process is over. Actually the USB core never knows this either, as devices come and go all the time.
That's why we had to move to the "probe" and "release" way of writing drivers. The bus cores notify the driver when they find something, as the driver never knows when a device is found.
So the kernel can not know what or when to wait for something it doesn't know is going to ever happen in the future.
Does that make more sense now?
Remember, this isn't your old "static device tree" unix-like kernel that people grew up with, anymore. :)
thanks,
greg k-h
Quoted 23 lines · Giacomo A. Catenazzi
Quoted 11 lines · Tonnerre
Quoted 6 lines · Ian Campbell
I wonder if it would be feasible for modprobe (or some other utility) to have a new option: --wait-for=/dev/something which would wait for the device node to appear. Perhaps by:
- some mechanism based on HAL, DBUS, whatever
- dnotify on/dev/?
- falling back to spinning and waiting.Quoted 8 lines
This would end up as hideous misfeature as you can't guarantee the device to show up *at* *all*.
The reason udev is there is that we can dynamically respond to created device nodes and devices that show up. They might have changed since the last boot. Maybe they don't show up at all.
Thus you should trigger your actions from
/etc/dev.d.
Quoted 7 lines
It is right.
But an option --wait would be sufficient.
This option will require modprobe to wait (with a timeout of x seconds) that hotplug event finish (so if device is created or not is no more a problem).
Ideally this should be done modifing only hotplug and IMHO should be enabled by default.
Um, I don't think this is posible to do at all. But hey, go ahead and implement it to prove me wrong :)
good luck,
greg k-h
Quoted 7 lines · On Wed, 15 Sep 2004 17:45:03 +0200 "Giacomo A. Catenazzi" <[email protected]>…
It is right.
But an option --wait would be sufficient.
This option will require modprobe to wait (with a timeout of x seconds) that hotplug event finish (so if device is created or not is no more a problem).
Ideally this should be done modifing only hotplug and IMHO should be enabled by default.
At the moment th hotplug event finishes nothing is guaranteed. In fact, the device node is never created at this point. All you know is that udev will now *begin* to create the node. You don't know how long it will take or if it will succeed at all.
As i understand, udev definitely has to be involved in this process. It would need a way to inform modprobe of its state.
Maybe something like an udev state could be added. The script would pass a cookie to modprobe, which would in turn pass it to the kernel (or to udevd?), which would add it to the hotplug event. udev would then place the cookie in a defined file that is checked by modprobe. If that cookie's state is set to "done" modprobe would return and the script would continue.
Example:
modprobe blah -c cookie-123
"cookie-123" is passed to the kernel and returned by all hotplug events this modprobe triggers.
udev will then place something like "cookie-123=>processing" in /dev/udev-state.
modprobe is still running and will poll this file until it contains "cookie-123=>finished". When that happens modprobe will tell udevd to remove this entry and return succesfully. If the timeout is reached modprobe will return an error code instead.
(Of course, modprobe could handle the cookie generation internally.)
This sound complicated and requires changes in many places. Maybe there is an easier solution.
Regards
Quoted 25 lines · Marc Ballarin
Quoted 7 lines · On Wed, 15 Sep 2004 17:45:03 +0200 "Giacomo A. Catenazzi" <[email protected]>…
It is right.
But an option --wait would be sufficient.
This option will require modprobe to wait (with a timeout of x seconds) that hotplug event finish (so if device is created or not is no more a problem).
Ideally this should be done modifing only hotplug and IMHO should be enabled by default.At the moment th hotplug event finishes nothing is guaranteed. In fact, the device node is never created at this point. All you know is that udev will now *begin* to create the node. You don't know how long it will take or if it will succeed at all.
As i understand, udev definitely has to be involved in this process. It would need a way to inform modprobe of its state.Maybe something like an udev state could be added. The script would pass a cookie to modprobe, which would in turn pass it to the kernel (or to udevd?), which would add it to the hotplug event. udev would then place the cookie in a defined file that is checked by modprobe. If that cookie's state is set to "done" modprobe would return and the script would continue.
Quoted 4 lines
Example:
modprobe blah -c cookie-123
"cookie-123" is passed to the kernel and returned by all hotplug events this modprobe triggers.
Again, such tracking is pretty much impossible.
Quoted 11 lines
udev will then place something like "cookie-123=>processing" in
/dev/udev-state.
modprobe is still running and will poll this file until it contains "cookie-123=>finished". When that happens modprobe will tell udevd to remove this entry and return succesfully. If the timeout is reached modprobe will return an error code instead.(Of course, modprobe could handle the cookie generation internally.)
This sound complicated and requires changes in many places. Maybe there is an easier solution.
There is, just run your stuff off of /etc/dev.d/ and stop relying on a device node to be present after modprobe returns.
thanks,
greg k-h
Quoted 3 lines · Greg KH
But the low level driver (like a USB driver for example), has no way of knowing when the "device discovery" process is over. Actually the USB core never knows this either, as devices come and go all the time.
eventually the discovery will end with a call into userspace, it'd be enough to wait4 and wakeup a waitqueue when the wait4 returns.
Quoted 2 lines
So the kernel can not know what or when to wait for something it doesn't know is going to ever happen in the future.
the kernel certainly knows when it's about time to fork an userspace process to create the device, doesn't it? then just wait4 while the process is running.
Quoted 2 lines
Remember, this isn't your old "static device tree" unix-like kernel that people grew up with, anymore. :)
if you intentionally don't want to provide serialization and force into /var/run locking, that's fine with me, but I don't buy your claim that it's not feasible.
Quoted 3 lines · Andrea Arcangeli
the kernel certainly knows when it's about time to fork an userspace process to create the device, doesn't it? then just wait4 while the process is running.
Last I checked, if using udevsend the process that gets forked is not the same process that actually creates the device node.
Chris
Quoted 8 lines · Chris Friesen
Quoted 3 lines · Andrea Arcangeli
the kernel certainly knows when it's about time to fork an userspace process to create the device, doesn't it? then just wait4 while the process is running.
Last I checked, if using udevsend the process that gets forked is not the same process that actually creates the device node.
Good point, so wait4 wouldn't be enough, you'd need a bit more of API on the udev side. But be sure, if dev.d can make it with a FIFO lock in /var/run, then the kernel can make it too.
Also note the maybe usb is just completely async and when you load the device driver you've no idea of when you'll get the signal and it's impossible to make it sync, fine. I'm really not talking about hotplug, hotplug basically has to be async by design.
But the microcode driver has nothing to do with hotplug or usb, and waiting for the device creation in the insmod syscall is sure natural and doable. Now if we don't want to create a little bit of API in udev to let insmod provide a sync behaviour and force people into the dev.d scripts and file locking in /var/run that it's one issue (you can aruge about higher perf), but there is a whole large class of devices that are not hot-pluggable and where the discovery is definitely synchronous, even common pci is fully sync in its discovery when you call pci_find_device/get_device.
Quoted 7 lines · Andrea Arcangeli
Quoted 3 lines · Greg KH
But the low level driver (like a USB driver for example), has no way of knowing when the "device discovery" process is over. Actually the USB core never knows this either, as devices come and go all the time.
eventually the discovery will end with a call into userspace, it'd be enough to wait4 and wakeup a waitqueue when the wait4 returns.
But to wait for what? That's the point. The individual driver doesn't know what it should be waiting for, if anything at all. That's why we have the "probe" and "release" model for device drivers now.
Quoted 6 lines
Quoted 2 lines
So the kernel can not know what or when to wait for something it doesn't know is going to ever happen in the future.
the kernel certainly knows when it's about time to fork an userspace process to create the device, doesn't it? then just wait4 while the process is running.
Yes and no. You see there's a lot of hotplug events that get kicked off when a device (such as a USB device) is discovered. We have the "here's a device", "here's an interface on that device", "here's a scsi-host controller that got bound to the interface", "here's a scsi device on that host controller that got added", "here's a sg interface bound to that scsi device", "here's a sd interface bound to that device", "here's a block device attached to that sd interface", "here's a partition belonging to that block device".
And all of those get kicked off by different kernel threads at different times, all for one possible modprobe. And odds are that modprobe has long returned by the time that last "block device" and "partition" hotplug events get emitted by the kernel.
And different modprobe events all caused that end result (modprobing of the bus, host controller, usb driver, scsi core, scsi disk, and scsi generic modules.) Which modprobe will you want to wait for?
Quoted 6 lines
Quoted 2 lines
Remember, this isn't your old "static device tree" unix-like kernel that people grew up with, anymore. :)
if you intentionally don't want to provide serialization and force into
/var/runlocking, that's fine with me, but I don't buy your claim that it's not feasible.
I think it's not feasible. But feel free to dig through the usb/scsi/driver core/block code to prove me wrong :)
In the meantime, the rest of us over here will be using the /etc/dev.d interface...
thanks,
greg k-h
Quoted 1 line · Greg KH
generic modules.) Which modprobe will you want to wait for?
all the ones triggered by the modprobe? It's really a matter of sync and async. if the discovery can be synchronous, then waiting for it is doable. It's like when you call find_pci_device, you will sure know that you will get either a definitive answer when the function returns, this is not the case for USB it seems.
For scsi as well after the timeout triggers you know no device is supposed to showup (and that's why boot has to be so slow, of course we really should wait only for the root device and to scan all hosts in parallel, but that's quite orthogonal, the issue is to get a synchronous answer after a definite number of operations). if we hit an async path, then I agree it doesn't even worth to wait and we should wakeup immediatly.
Quoted 2 lines
In the meantime, the rest of us over here will be using the
/etc/dev.dinterface...
I did aruge just because you said some of us is running a spin-and-sleep instead of the /etc/dev.d interface, which contraddicts your statement that everyone is using /etc/dev.d, which make it quite obvious using the dev.d is more tricky than the old way. I have scripts myself insmodding a device and then doing some I/O on it in /etc/init.d/boot.local, mixed with other stuff, which is quite simple and works right now.
But I do certainly agree about your point in the other email that sync-discovery could become a thing of the past, even if so far usb and pci hotplug has been the exception ;). So sounds like we may just go complicated for everything.
Quoted 10 lines · Andrea Arcangeli
But the microcode driver has nothing to do with hotplug or usb, and waiting for the device creation in the insmod syscall is sure natural and doable. Now if we don't want to create a little bit of API in udev to let insmod provide a sync behaviour and force people into the dev.d scripts and file locking in
/var/runthat it's one issue (you can aruge about higher perf), but there is a whole large class of devices that are not hot-pluggable and where the discovery is definitely synchronous, even common pci is fully sync in its discovery when you call pci_find_device/get_device.
That will be soon going away with my multi-threaded device discovery work. And I run pci hotplug boxes (and so do all of the PCMCIA/CardBus users), so don't discard PCI from being a async bus type :)
Async is now the norm, and drivers like the microcode module are the exception.
thanks,
greg k-h
Quoted 17 lines · Greg KH
Quoted 10 lines · Marco d'Itri
Quoted 3 lines · Greg KH
What's wrong with the
/etc/dev.d/location for any type of script that you want to run after a device node has appeared? This is an application specific issue, not a kernel issue.The problem is that since most distributions cannot make udev usage mandatory, this would require duplicating in the init script and in the dev.d script whatever needs to be done with the device.
Well, that sounds like a distro problem then, not a kernel or udev one :)
Quoted 7 lines
Quoted 3 lines
Then there are the issues of scripts needing programs in /usr, which may not be mounted when the module is loaded, or which are interactive and need console access (think fsck).
True, so sit and spin and sleep until you see the device node. That's how a number of distros have fixed the fsck startup issue.
Hm, why should _I_ sleep and spin after modprobe, without even knowing if the node will appear at all, when you can include the "modprobe" wrapper script with udev source package.
This wrapper should call modprobe.orig with original arguments, and then call udev for /sys entries that appeared (or just run udevstart), and only then return. Yes, this will result in duplicate hotplug events (synthetic + real) being delivered to udev, but it solves the problem with modprobe once, for all programs, and in compatible way.
Signature · 1 line
Alexander E. Patrakov
Quoted 38 lines · Marc Ballarin
Quoted 7 lines · On Wed, 15 Sep 2004 17:45:03 +0200 "Giacomo A. Catenazzi" <[email protected]>…
It is right.
But an option --wait would be sufficient.
This option will require modprobe to wait (with a timeout of x seconds) that hotplug event finish (so if device is created or not is no more a problem).
Ideally this should be done modifing only hotplug and IMHO should be enabled by default.At the moment th hotplug event finishes nothing is guaranteed. In fact, the device node is never created at this point. All you know is that udev will now *begin* to create the node. You don't know how long it will take or if it will succeed at all.
As i understand, udev definitely has to be involved in this process. It would need a way to inform modprobe of its state.Maybe something like an udev state could be added. The script would pass a cookie to modprobe, which would in turn pass it to the kernel (or to udevd?), which would add it to the hotplug event. udev would then place the cookie in a defined file that is checked by modprobe. If that cookie's state is set to "done" modprobe would return and the script would continue.
Example:
modprobe blah -c cookie-123
"cookie-123" is passed to the kernel and returned by all hotplug events this modprobe triggers.
udev will then place something like "cookie-123=>processing" in/dev/udev-state.
modprobe is still running and will poll this file until it contains "cookie-123=>finished". When that happens modprobe will tell udevd to remove this entry and return succesfully. If the timeout is reached modprobe will return an error code instead.
Quoted 4 lines
(Of course, modprobe could handle the cookie generation internally.)
This sound complicated and requires changes in many places. Maybe there is an easier solution.
Yes, there is, and it is a purely userspace one. Instead of waiting for those hotplug events, synthetize their duplicates in the "modprobe" binary, pass directly to udev, and wait for these duplicates.
Signature · 1 line
Alexander E. Patrakov
Quoted 7 lines · Greg KH
Quoted 2 lines · Marc Ballarin
This sound complicated and requires changes in many places. Maybe there is an easier solution.
There is, just run your stuff off of
/etc/dev.d/and stop relying on a device node to be present after modprobe returns.
Indeed.
I don't see what is so far for distributions to overcome here - if you want to avoid having two sets of alternative init scripts, why not have a generic set of /etc/dev.d entries that are either called by udev or if udev is not available run them in some predetermined order on a static device tree. What's the hassle here? I can't see anything world shattering.
Cheers,
Jon.
Quoted 2 lines · Greg KH
There is, just run your stuff off of
/etc/dev.d/and stop relying on a device node to be present after modprobe returns.
But installation scripts will need to sleep/loop after fdisk, till the devices are created, right?
And I'm currently mknod'ing /dev/md* devices before creating the kernel structures with mkraid (which needs the device inode). Is there any other way to do this?
sure, in the long run designs like dm with a special control device are a better than this hack for md.
Also I wonder how dm works: will dmsetup create the /dev inode itself, or use udev to do that? would I need the sleep/loop in a script creating device mappings to wait for the inode?
Regards, Andreas
Quoted 2 lines · Andreas Jellinghaus
Also I wonder how dm works: will dmsetup create the /dev inode
itself, or use udev to do that? would I need the sleep/loop in a script creating device mappings to wait for the inode?
You can safely tell udev to ignore dm devices, if you use LVM or EVMS. It almost always creates them under the wrong directory and with wrong name anyway. Correcting this means just duplicating LVM/EVMS configuration in udev rules. Better leave this task of dm node creation to dmsetup and related tools.
Signature · 1 line
Alexander E. Patrakov