Roadmap to restoring working usb module autoloading?
I am concerned that the situation with usb module autoloading
seems to be deadlocked and so I want to propose a roadmap for getting to the point where usb module autoloading works when a user installs the latest versions of everything and does not make any additional modifications.
Here is my proposal:
0. We agree that the endpoint of roadmap is not necessarily the
final destination that we are going to settle on, but rather a point where everything works again and one that is closer to where we want to be than before.
1. I have just posted changes to usbmodules to recognize
both the old and new versions of modules.usbmap. If the patches work for everyone, then I would appreciate it if Thomas would make a new release of usbutils that includes these changes. There should be no compatability problem with this.
2. As I understand the situation, Linus and Alan have not
integrated Keith's module table version number patches after a number of 2.4.1-pre releases. The big advantage of using version numbers for each ID structure instead of having depmod decide based on the kernel version string is that you can update different components of the kernel if yours is somewhat customized (e.g., imagine a 2.5.x USB backport to 2.4). However, this is currently defeated by an error check in depmod which compalins "Modules have a mixture of
version ___ and version ___". Also, integer version numbers that
increase by one create an ID allocation problem if somebody wants to ship a modified device ID structure and a modutils that will recognize it. These are issues that have occurred to me while wondering why the changes have not been integrated yet. They are just second guessing, though. Someone should corner one of them at LinuxWorld about it in any case. I'm not sure what the other advantages of the structure version numbers are supposed to be, but I realize I'm probably not doing it justice in this paragraph.
Although my understanding of the situtation is obviously
incomplete, I think the idea behind these version numbers was to avoid version skew ("you have to upgrade the kernel if you upgrade modutils"). However, that is preferable to nonfunction ("if you upgrade the kernel, your modules won't work"). I'm also beginning to think that there may be...maybe...some method to the madness of not going with version numbers for the device ID tables, at least enough so that I'm not sufficiently confident that Linus is wrong that would I want to support an intransagence contest between the standard kernels and the standard userland facilities for USB hot plugging. So, I propose that, for the meantime that modutils be changed to accomodate Linus's kernels for now (I am willing to write these patches):
2a. Back out the device ID structure version number code for now
and instead have a new modutils that supports only the new format for usb_device_id. module_device_id tables are not in 2.2 and the old format was used only in 2.4.0-pre 2.4.0-test kernels, so everyone is supposed to upgrade anyhow, especially if they're upgrading other components on their system.
2b. depmod wants usb_device_id.match_flags toward the end of
the structure and Linus's kernels put it toward the beginning. I know we had talked about rearranging the usb_device_id structure this way to improve compatability. Again, I propose having depmod accomodate Linus's kernels.
3. Likewise, the rest of USB userland support scripts should
be updated. I do not use these. Perhaps david-b could comment on this.
If there are people in the midst of negotiating some other fix
with Linus and those who have his ear on this matter, I don't mean to preempt their efforts. But, I don't think that is what is going on. I think everybody is waiting for some other event to break this impasse and, in the meantime, USB hot plugging remains out of commission and Linux will make a bad impression in this area with people who are trying Linux because of the 2.4 hoopla.
This does not mean that device ID version number are wrong
or should not get into Linus's kernels. I just want the software to continue to work with the current kernels in the meantime.
Comments? (Flames?)
Adam J. Richter __ ______________ 4880 Stevens Creek Blvd, Suite 104 [email protected] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of America
fax +1 408 261-6631 "Free Software For The Rest Of Us."
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines
From: Adam J. Richter <[email protected]>
Sent: Saturday, January 20, 2001 5:32 AMI am concerned that the situation with usb module autoloading seems to be deadlocked
With regards to getting kernel and modutils talking again, in production releases of both.
Quoted 6 lines
Here is my proposal:
0. We agree that the endpoint of roadmap is not necessarily the final destination that we are going to settle on, but rather a point where everything works again and one that is closer to where we want to be than before.
Yep. Milestones of note include "works as well as the prerelease", and "bugs noted in prerelease are now fixed".
Quoted 5 lines
1. I have just posted changes to usbmodules to recognize both the old and new versions of modules.usbmap. If the patches work for everyone, then I would appreciate it if Thomas would make a new release of usbutils that includes these changes. There should be no compatability problem with this.
Thanks. This should get rid of the "doesn't work with usbutils" set of problems.
Quoted 5 lines
2. As I understand the situation, Linus and Alan have not integrated Keith's module table version number patches after a number of 2.4.1-pre releases. The big advantage of using version numbers for each ID structure instead of having depmod decide based on the kernel version string
It was using structure size before, which was clearly not going to work in cases like this (same size, fields rearranged). I don't know that the (optional?) kernel version string was being considered.
Quoted 8 lines
is that you can update different components
of the kernel if yours is somewhat customized (e.g., imagine a 2.5.x USB backport to 2.4). However, this is currently defeated by an error check in depmod which compalins "Modules have a mixture of
version ___ and version ___". Also, integer version numbers that
increase by one create an ID allocation problem if somebody wants to ship a modified device ID structure and a modutils that will recognize it.
I'm not sure we'd want to encourage or support that. It'd effectively be a kernel fork.
Quoted 3 lines
These are issues that have occurred to me while
wondering why the changes have not been integrated yet. They are just second guessing, though.
I'm concerned that's most of what we have to go by.
Another thought is that it was a recognition that the failure modes related to the netdevice stuff (see Andrew's recent post) were a bit too severe to push on other parts of hotplug without
having that fixed first. Complete system hangs are not polite
in "production" kernels.
Quoted 5 lines
Someone should corner one of
them at LinuxWorld about it in any case. I'm not sure what the other advantages of the structure version numbers are supposed to be, but I realize I'm probably not doing it justice in this paragraph.
Quoted 12 lines
Although my understanding of the situtation is obviously incomplete, I think the idea behind these version numbers was to avoid version skew ("you have to upgrade the kernel if you upgrade modutils"). However, that is preferable to nonfunction ("if you upgrade the kernel, your modules won't work"). I'm also beginning to think that there may be...maybe...some method to the madness of not going with version numbers for the device ID tables, at least enough so that I'm not sufficiently confident that Linus is wrong that would I want to support an intransagence contest between the standard kernels and the standard userland facilities for USB hot plugging. So, I propose that, for the meantime that modutils be changed to accomodate Linus's kernels for now (I am willing to write these patches):
I'm really concerned about that second-guessing issue. That's not the way to achieve consistent forward motion, at least in my book, and likely contributed to the original 2.4.0 problem.
Quoted 2 lines
3. Likewise, the rest of USB userland support scripts should be updated. I do not use these. Perhaps david-b could comment on this.
They've been pretty much up-to-date for a few weeks, and won't care so long as the "match_flags" shows up in the first field of the text output.
I will note those scripts currently ignore the match_flags field, since bitfields are just nasty to deal with in shell scripts (fixable). And they're expecting only "new style" output from modutils (also fixable). Anyone wanting to hotplug with prerelease or 2.4.0-test kernels can use "usbmodules", or older scripts ... anyone wanting a different USB agent (in C?), feel free to provide one.
Quoted 4 lines
If there are people in the midst of negotiating some other fix with Linus and those who have his ear on this matter, I don't mean to preempt their efforts. But, I don't think that is what is going on. I think everybody is waiting for some other event to break this impasse
For my part, I'm wanting to hear something from Linus. We're now in decent shape (matching functionality in the prerelease, unfortunately including that netdevice problem) if he'd merge that patch from Keith.
- Dave
Quoted 8 lines
and, in the meantime, USB hot plugging remains out of commission and Linux will make a bad impression in this area with people who are trying Linux because of the 2.4 hoopla.
This does not mean that device ID version number are wrong or should not get into Linus's kernels. I just want the software to continue to work with the current kernels in the meantime.
Quoted 7 lines
Comments? (Flames?)
Adam J. Richter __ ______________ 4880 Stevens Creek Blvd, Suite 104 [email protected] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of Americafax +1 408 261-6631 "Free Software For The Rest Of Us."
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 15 lines
Quoted 1 line
= Adam Richter
= David Brownell
Quoted 11 lines
[...] The big advantage of using version numbers
for each ID structure instead of having depmod decide based on the kernel version string
is that you can update different components
of the kernel if yours is somewhat customized (e.g., imagine a 2.5.x USB backport to 2.4). However, this is currently defeated by an error check in depmod which compalins "Modules have a mixture of
version ___ and version ___". Also, integer version numbers that
increase by one create an ID allocation problem if somebody wants to ship a modified device ID structure and a modutils that will recognize it.
Quoted 2 lines
I'm not sure we'd want to encourage or support that. It'd effectively be a kernel fork.
It's free software, so "forks" that work tend to weave back
into the standard versions and are in integral part of the process. In practice, I believe just about every Linux distribution has had to modify their kernels, and this often helps get those changes into Linus's kernels. I believe at least one Linux distribution shipped the 2.4 "back port" of the USB code in their 2.2 kernel, which, was probably a net benefit to the world.
Of course, it's also easy enough to stip out the code from
depmod that complains about the device ID version number differences without changing anything else if device ID version numbers do get integrated.
Adam J. Richter __ ______________ 4880 Stevens Creek Blvd, Suite 104 [email protected] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of America
fax +1 408 261-6631 "Free Software For The Rest Of Us."
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines · Adam J. Richter
Of course, it's also easy enough to stip out the code from
depmod that complains about the device ID version number differences without changing anything else if device ID version numbers do get integrated.
And break everything downstream from depmod. Most (all?) of the utilities that read the depmod generated maps rely on the header line to tell them which format of table they are reading. One header line -> one supported table format in any map -> all input device ids must be the same for a map.
If you want a generated map format that can cope with different input formats in the same map then each line needs a tag that says which fields it contains. Before you even think about that approach, be absolutely certain that the utilities which read the text can cope with missing or extra fields. Under all circumstances. Without ever making any errors. Assumptions about the side effects of missing or new fields must be valid for all fields, in all tables, forever.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines
Quoted 1 line
= Adam Richter
= Keith Owens
"Adam J. Richter" <[email protected]> wrote:
Quoted 8 lines
Quoted 4 lines
Of course, it's also easy enough to stip out the code from
depmod that complains about the device ID version number differences without changing anything else if device ID version numbers do get integrated.And break everything downstream from depmod. Most (all?) of the utilities that read the depmod generated maps rely on the header line to tell them which format of table they are reading. [...]
Whoa! I did not say strip out the header lines. I said allow
device ID structure version numbers to be different. For example, I might have version 2 usb_device_id's and version 1 parport_device_id's.
Adam J. Richter __ ______________ 4880 Stevens Creek Blvd, Suite 104 [email protected] \ / San Jose, California 95129-1034 +1 408 261-6630 | g g d r a s i l United States of America
fax +1 408 261-6631 "Free Software For The Rest Of Us."
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines · Adam J. Richter
Whoa! I did not say strip out the header lines. I said allow
device ID structure version numbers to be different. For example, I might have version 2 usb_device_id's and version 1 parport_device_id's.
depmod already allows that. The table version number is per table type, it is not a global kernel -> depmod version number. With my kernel patch we already have usb at version 2 and everything else at version 1. If that does not work then it is a depmod bug.
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
http://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel