Unloading drivers, start-up, shut-down and a rewrite (a problem)
Quoted 25 lines · Keith Owens
Quoted 8 lines · stamit
Now, the final thing that remains for unloading to be complete is reference counting. I tried to do it with bash (by storing counts in *.ref files in the driver directories) and guess what I stumbled to... a race condition! Multiple hotplug events can be processed by the scripts at the same time. So I took it back. Isn't there a way to use modutils programs to play with the module's real refrence counts? It would make things a whole lot easier (and safer).
Without knowing exactly what you are trying to do I am guessing but it sounds like you need this:
*
kernel/module.cregisters/proc/sys/kernel/mod_use_count, mode 600.
*module.ccreates a dummy module entry calledmod_use_countthe first time that/proc/sys/kernel/mod_use_countis written to.
*/proc/sys/kernel/mod_use_countrecognises commands like up module-name
down module-name
and adjusts the use counts and reference chain accordingly. Hotplug scripts usemod_use_countto bump the reference count before allowing a start event to continue and after completing a stop event.
Quoted 4 lines
If the module does not exist (already unloaded) then write fails with some return code. There is no specific code for "module does not exist" so pick something similar, like ESRCH or ENXIO.
I'm trying to implement module unloading in the hotplug scripts. Aside that, I'm also doing a rewrite, a slight redesign and "beautification" of the scripts generally because thay lacked such things as readability and uniformity. I'm trying to make them beautiful because, hey, vendors might choose to release drivers with scripts for our hotplug project!
So I'm making them pretty so they'll be attractive. :-)
Of course, because, besides making scripts and framework pretty, I'm also implementing the ability to unload modules, I needed to have a method to play with the modules' reference counts(which is the only thing still missing from my rewrite). And since you pointed me to the right way, I'll try to add it and post a patch when it's ready. I didn't really know about that because I'm not very into the kernel's internals. I'm doing mostly user-space stuff so far (thus my interest for the scripts).
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines · Stamatis Mitrofanis
Of course, because, besides making scripts and framework pretty, I'm also implementing the ability to unload modules, I needed to have a method to play with the modules' reference counts(which is the only thing still missing from my rewrite).
More details please. Why do you need to externally adjust a module's use count? All other modules handle their use counts internally, what is different about hotplug?
Quoted 3 lines
And since you pointed me to the
right way, I'll try to add it and post a patch when it's ready. I didn't really know about that because I'm not very into the kernel's internals.
I will do the patch to kernel/module.c, once I understand exactly what the requirements are.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 9 lines
More details please. Why do you need to externally adjust a module's use count? All other modules handle their use counts internally, what is different about hotplug?
I will do the patch to
kernel/module.c, once I understand exactly what the requirements are.
It's when you have many hotpluggable devices of the same type (handled by the same driver). Managing a reference (device) count for each driver from the hotpug scripts is necessary for unloading the driver _only_ after the last such device is unplugged.
It might be a bad idea to play with the modules' real reference counts though, but there must be at least some form of mutual exclusion for accessing this driver-specific device count. Therefore, it was wrong for me to try to implement this device count by making the scripts read/write"*.ref" files in the hotplug directory (possible race condition which would mess up the count).
If it should be implemented in the kernel then the functionality should be exposed through modprobe. I'm sorry, but I have no idea about the actual details of (un)loading modules so you'll have to do everything behind modprobe.
The requirement is just that modprobe gets a new command-line paremeter which, when given in a call to load/unload a module, will also increment/decrement the module's reference count. I.e. if the parameter is "--ref" and the module ABC is not loaded, then the following should result in module ABC being in memory with a reference count of 1:
# modprobe --ref ABC # modprobe --ref ABC # modprobe --ref -r ABC The first command will load the module and set its reference count to 1. The second command will increment the ABC's reference count to 2 and the third should decrement the reference count to 1 without, of course, unloading the module.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines · Stamatis Mitrofanis
It's when you have many hotpluggable devices of the same type (handled by the same driver). Managing a reference (device) count for each driver from the hotpug scripts is necessary for unloading the driver _only_ after the last such device is unplugged.
That means we are changing the meaning of the module use count on drivers from "number of devices opened" to "number of devices present and activated". Not such a bad idea, there has long been a problem with network drivers that the use count can be zero but the driver code can be active, rmmod gets very nasty then.
Before I write the code, what needs to change in user space to match the new semantics? Every device activation and inactivation must adjust the use count of the supporting module. That is easy for hotplug but what about PCMCIA and fixed devices? In particular ifconfig eth0 up must adjust the use count of the module controlling eth0. A partial solution that only works for hotplug is not a good idea. Can we get a consistent user space framework for the new semantics?
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 9 lines · Keith Owens
Before I write the code, what needs to change in user space to match the new semantics? Every device activation and inactivation must adjust the use count of the supporting module. That is easy for hotplug but what about PCMCIA and fixed devices? In particular ifconfig eth0 up must adjust the use count of the module controlling eth0. A partial solution that only works for hotplug is not a good idea. Can we get a consistent user space framework for the new semantics?
Just so you know, and not saying whether it is a better or worse solution, the way PCMCIA drivers handle this, is that there's a separate interface, /proc/bus/pccard/drivers, that has "use counts" for PCMCIA drivers that indicate how many physical devices are currently associated with each driver. These use counts are independent of the regular module use counts.
-- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoting David Hinds:
Quoted 6 lines
Just so you know, and not saying whether it is a better or worse solution, the way PCMCIA drivers handle this, is that there's a separate interface,
/proc/bus/pccard/drivers, that has "use counts" for PCMCIA drivers that indicate how many physical devices are currently associated with each driver. These use counts are independent of the regular module use counts.
And FWIW, when I've thought of how to solve this "driver unload" problem in a clean way that improves user-friendliness, that's just what I end up wanting: a devices-per-driver count that's distinct from the existing uses-per-driver refcount.
It's late, so don't hold me to this, but I think adding that is probably sufficient for user-mode policy decisions. Hotplugging would by default consider the device-per-driver count when seeing if it's OK to unload a driver module.
Likely "rmmod" needs a flag that says whether the device count is ignored, as (effectively) done today. Driver developers will want the flexibility to say "remove this driver even though there's hardware attached, I've got a bugfix". USB developers work in that mode today, it's quite handy.
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines
The requirement is just that modprobe gets a new command-line paremeter which, when given in a call to load/unload a module, will also increment/decrement the module's reference count. I.e. if the parameter is "--ref" and the module ABC is not loaded, then the following should result in module ABC being in memory with a reference count of 1:
Modulo the issue about making a new "devices per driver" counter, I'd also like to raise the issue of whether this counter shouldn't be driven entirely by the kernel code binding drivers to devices.
That is, code the USB or PCI layers would increment/decrement the counter when the binding was successfully achieved. No changes to the device drivers needed, but having the kernel maintain those counters would be safer than doing it in user mode.
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 8 lines · David Brownell
Quoting David Hinds:
Quoted 6 lines
Just so you know, and not saying whether it is a better or worse solution, the way PCMCIA drivers handle this, is that there's a separate interface,
/proc/bus/pccard/drivers, that has "use counts" for PCMCIA drivers that indicate how many physical devices are currently associated with each driver. These use counts are independent of the regular module use counts.
Quoted 4 lines
And FWIW, when I've thought of how to solve this "driver unload" problem in a clean way that improves user-friendliness, that's just what I end up wanting: a devices-per-driver count that's distinct from the existing uses-per-driver refcount.
I am not convinced that we need a separate count, at least not at the bottom layer. The module use count is a pure reference count on the module code, rmmod does not care what the types of reference are, rmmod just cares if it is safe to unload the code.
Quoted 5 lines
Likely "rmmod" needs a flag that says whether the device count is ignored, as (effectively) done today. Driver developers will want the flexibility to say "remove this driver even though there's hardware attached, I've got a bugfix". USB developers work in that mode today, it's quite handy.
See above, I don't see any need to distinguish between references. If the device is up then the module cannot be unloaded. That does assume that all devices can be deactivated which is not currently true, to do this properly every device driver would need an up and a down routine.
I can add a use count up/down interface to kernel/module.c without waiting for every driver to be converted to an up/down interface. However we have to agree on a common design for drivers, I would hate to see some modules supporting up/down and others doing it a different way.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Am Freitag, 5. Oktober 2001 09:28 schrieb Keith Owens:
Quoted 9 lines
On Fri, 05 Oct 2001 00:13:15 -0700,
Quoted 4 lines · David Brownell
And FWIW, when I've thought of how to solve this "driver unload" problem in a clean way that improves user-friendliness, that's just what I end up wanting: a devices-per-driver count that's distinct from the existing uses-per-driver refcount.
Quoted 4 lines
I am not convinced that we need a separate count, at least not at the bottom layer. The module use count is a pure reference count on the module code, rmmod does not care what the types of reference are, rmmod just cares if it is safe to unload the code.
Yes. In last consequence only the kernel cares about reference counts. Yet it is not a pure reference counter. It's possible for a module, though possibly not a good idea to bump the counter while eg running a timer. In fact you need to bump it while handling probe() and disconnect().
Quoted 10 lines
Quoted 5 lines
Likely "rmmod" needs a flag that says whether the device count is ignored, as (effectively) done today. Driver developers will want the flexibility to say "remove this driver even though there's hardware attached, I've got a bugfix". USB developers work in that mode today, it's quite handy.
See above, I don't see any need to distinguish between references. If the device is up then the module cannot be unloaded. That does assume that all devices can be deactivated which is not currently true, to do this properly every device driver would need an up and a down routine.
The problem is not whether the driver can be unloaded without endangering system integrity. It's about whether unloading a driver breaks the desired function. Else a perfectly usable device becomes driverless all of a sudden. With the current design in unloading you need a seperate counter.
Quoted 5 lines
I can add a use count up/down interface to
kernel/module.cwithout waiting for every driver to be converted to an up/down interface. However we have to agree on a common design for drivers, I would hate to see some modules supporting up/down and others doing it a different way.
That would be helpful. And doing this in kernel makes handling errors in binding drivers to devices trivial.
For USB the core should handle it evaluating the return value of probe(). That needs a module* in the usb_device struct. On the other hand the conventional usage counters would be simplified by this.
Regards
Oliver
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Hi Keith,
Quoted 4 lines
I am not convinced that we need a separate count, at least not at the bottom layer. The module use count is a pure reference count on the module code, rmmod does not care what the types of reference are, rmmod just cares if it is safe to unload the code.
While "rmmod" may not care, there are two distinct user communities that IMO need to be supported:
- "real users", 95+% of the computer users in the world, who want things to "just work" and never need root access or a sysadmin;
- "sysadmin" or "developer" ... who does troubleshooting/upgrading/... and so needs extra system capabilities.
I think "one use count" might work for a "real user", but not for anyone who wants safe ways to replace drivers on running systems. If there's just one use count, adjusted in userland, it'd be too easy for userland bugs to corrupt things. (What do you mean by "need"? :)
Quoted 7 lines
Quoted 5 lines
Likely "rmmod" needs a flag that says whether the device count is ignored, as (effectively) done today. Driver developers will want the flexibility to say "remove this driver even though there's hardware attached, I've got a bugfix". USB developers work in that mode today, it's quite handy.
See above, I don't see any need to distinguish between references.
How would someone SSH into a system and upgrade some of its drivers? With two counts, one can shutdown all code using the drivers (use count goes to zero), then say "ignore any hardware, just remove the driver". And then start things up again; perfectly safe, even a sleep-deprived and seriously aggravated sysadmin can't make mistakes (run some command too many times :) and thereby crash the server (consequences left to your imagination).
Quoted 3 lines
If the device is up then the module cannot be unloaded. That does assume
that all devices can be deactivated which is not currently true, to do this properly every device driver would need an up and a down routine.
If "up" is a software notion, that's what I see the current use count as being used for. (I can't see it being a hardware notion...) But I'm not sure what you're saying "is not currently true".
Hotpluggable drivers get told when the hardware comes and goes, by USB or PCI etc. With USB, that doesn't modify any use counter, ensuring that drivers can safely be reloaded so long as they're not actually in use. If the hardware is accessed through the file system device nodes, open/release routines apply; network drivers have up/down analogues. Those all (should) update the current use count, keeping the driver from being unloaded until the device becomes inactive.
Quoted 2 lines
I can add a use count up/down interface to
kernel/module.cwithout waiting for every driver to be converted to an up/down interface.
The same with MOD_{INC,DEC}_DEV_COUNT() support;
it's not drivers that'd need converting, but subsystems. Two lines (each) in the right places of usb.c, pci.c (or wherever). Changing every single driver would be wrong, I'd agree.
Quoted 3 lines
However we have to agree on a common design for drivers, I would hate to see some modules supporting up/down and others doing it a different way.
We have USB pretty well in hand, as I described above. It could again serve as a guinea pig ... :)
- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines · David Brownell
That is, code the USB or PCI layers would increment/decrement the counter when the binding was successfully achieved. No changes to the device drivers needed, but having the kernel maintain those counters would be safer than doing it in user mode.
This is what PCMCIA does.
-- Dave
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel