2.5.47bk2 + current modutils == broken hotplug

The module-init-tools-0.6.tar.gz utilities (or something related -- kbuild changes?) break hotplug since they no longer produce the /lib/modules/$(uname -r)/modules.*map files as output ... so the hotplug agents don't have the pre-built database mapping device info to drivers.

What's the plan for getting that back? (And module.conf params etc.) "Changes" says version 2.4.2 is fine, which appears to be wrong...

- Dave

Last I heard, Rusty's still working on this. He's also going to be changing the format so we don't expose kernel structures to userspace, which would be a good thing.

In short, he knows this is a requirement, and shouldn't be broken for long.

thanks,

greg k-h

So long as the _information_ in those structures stays available, good. And it'd be handy if the text format for that information didn't change; how it's stored in object modules doesn't matter.

Good, that's important.

- Dave

Agreed.

Correction -- the tools that read the text format are buggy if they do not transparently support changes to the text format. (Corollary: the text format is buggy if it does not support a method of noticing format changes)

I am planning on adding PCI revision id to the information exported via MODULE_DEVICE_TABLE(pci,...). Tools which correctly read the first-line-format-definition will continue to function as before, regardless of additional fields I want to add. Tools which make silly assumptions will have those assumptions come back to bite them ;-)

(tangent warning!)
Another long term idea I would eventually like to realize is the removal of device ids from the C source code. I don't care where they go -- drivers/net/pci_ids [per directory ids?], drivers/net/3c59x.meta, whereever. Anywhere but the C source code. It's quite silly to require a driver rebuild just to add a single PCI id, and further, embedding metadata in C source is rarely a good idea in the long term. [reference some of Linus's counter-arguments when it was mentioned that Donald Becker's method of including Config.{in,help} data in C source might be useful]

Jeff

True, this would be nice, but how would the driver know to bind to a new device, if it isn't rebuilt, and doesn't know about the new id that was just added? In the current scheme of driver matching to devices, I don't see how this could be done.

Not to say I would not want to see this changed to allow this to happen, I'm very tired of telling USB Palm users to get a new kernel version just because a single device id was added which their new device has.

thanks,

greg k-h

That's not a correction, it's a tangent! Either way, it's still
"handy" ... since supporting new formats requires new tools, and it's clearly "handy" not to need to write, debug, and deploy new tools (including teaching, documenting, etc).

The "silly assumption" seems to be that the current "modules.*map" format can reasonably be extended ... when no rules for extending those file formats were really defined, and the _only_ precedent for even changing them involved incompatibility between "versions".

I'd far rather have a decent file format for that data, and move away from that ungainly syntax, than try to retrofit rules about how to extend that syntax. Such a format should:

- allow comments
- not have as much garbage (zeroes, 0xffffffff, etc)
- still be easily parsable from shell scripts (*)
- be easier for site admins to edit (insert vi/emacs flamewar)
- have an explicit version code, and rules about evolution

Alternatively, the tried'n'true approach of coupling the format to the filename works: don't reuse "modules.*map", use "*.modules" or something.
- Dave

(*) This is the annoying requirement. Maybe worth relaxing.
For example, an XML format could easily support all the other requirements ... but not (AFAIK) that one.

I think that truly seamless rebinding is doable but would require too much additional complexity in the kernel. Rebinding to a new id table between unregister() ... register() pairs, or between mod unload and mod load, should be enough to be useable for 98% of the cases.

It should be noted that David Hinds' pcmcia-cs package stores id in an external database. There are both positive and negative lessons that have been learned from that experience (WRT external id tables only! :):)) too.

Indeed :/ There are also issues like vendors who want to GPG-sign drivers, and updating the PCI id table makes the GnuPG signature and the certification that goes along with it invalid, requiring a vendor update.

Jeff

Hmm, maybe Linux should use Microsoft's ".inf" file syntax? %-)

That's one thing that it achieves, at the cost of serious chaos when the files with the device IDs get out of sync with the drivers they supposedly work with.

It'd be good if we had ways that user mode tools can request drivers be bound and un-bound. "usbfs" has some support for that, not exactly packaged in the way I'd most like (and "usbfs" is problematic too).

I'd rather not try swapping ID tables ... likely better to keep some of that information compiled in to the drivers, but also ADD ways that user mode tools can modify the bindings that the kernel does (or doesn't) establish. Unless someone wants to get radical and insist that ONLY the user mode tools can define such policies (after bootstrapping is done).

Of course Greg's example of a Palm could be addressed using current infrastructure and module parameters, with "wildcard" binding (to make sure the driver can see if the device matches the parameters).

- Dave

In message <[email protected]> you write:

Sorry, I've been feeding Linus (and lkml) slowly. It's not permenant, I promise 8) I discussed the issue with Greg Kroah-Hartman and have sent him a patch for examination. In the new method, external code doesn't know about kernel datastructures.

The question is whether to force an /sbin/hotplug change to use the module alias mechanism, or generate the modules.*map files at "make modules_install" time to avoid breakage. I'm leaning towards #2, but I haven't written it yet (should be simple).

0.7 has preliminary /etc/modprobe.conf support, but it only does primitive aliases not options as yet. That's next on my list for userspace, along with "modprobe -r".

Thanks for the feedback,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Yes, please. The kmod style aliasing presumes the answer to "what do I do with this device?" is "load these modules", which makes answers like "mount the disk" or "start this daemon/driver" needlessly hard to achieve.

Hotplugging may well change more in 2.5, but I'd rather have it do so on a more flexible schedule than "quick, before 2.5.48 ships"! :)

Thanks for the info ... it didn't seem very visible, so I wanted to know what the story was! Heck, I'm just glad to see forward motion in the module load/unload area.

Is it true that the infrastructure newly in place can easily be made to provide (from user-space) the policy of "driver remains loaded until the devices it's bound to are all unplugged"?

That'd be a user-friendly policy, but we'd still need to handle today's developer-oriented "sysadmin can always remove module" policy. (Me, I'd run with the "user friendly" policy except when hacking a driver. Then I'd debug/rmmod/update/modprobe.)

- Dave

In message <[email protected]> you write:

Hey, I want to get humpty dumpty back together ASAP, too. But I'm in Spain without an x86 box (my test box is available though a modem to IBM's intranet, two ssh's away), so it's a little difficult for me, too. And I'm still catching up on mail.

That should always be true, unless I'm missing something. What kind of devices?

rmmod -f is about as unfriendly as you can get, really 8)

Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

This is about another of the limitations/flaws of the current module framework, which I'm hoping the new one could fix...

Any device whose driver gets modprobed through hotplug ... it'd be good to have the module unload automatically when the hardware is gone. The pcmcia tools do this already, with a more limited problem domain, by tracking device-to-driver associations in the kernel. But other hotpluggable frameworks can't do it today, and it seems to be the module framework which gets in the way.

Consider two instances of such a device. Hotplug one, the driver is loaded, refcount zero since it's not opened. Then rmmod on unplug would be appropriate: the hardware is gone.

But instead of unplugging it, plug in a second. Now rmmod is no longer an appropriate default policy on unplug, even though the module "refcount" is still zero, since the user could still try access the other device. (Maybe they were plugging in devices until they found the one with the data they were after, and just hadn't looked at that the other one yet.)

It's a case where the current modprobe/rmmod scheme is counting the wrong kinds of things. Changing module refcounts to track the device bindings would be a "hard" policy, interfering with driver developer and sysadmins (gotta reboot or unplug to change drivers). Tracking such device-to-driver bindings in userland is fragile. Today's workaround is not to rmmod ... an undesirable answer.

So I keep thinking the right answer is to have a separate count to track hardware bindings, or at any rate some "soft" policy hook is needed to prevent rmmod in such cases. Sysadmins and driver developers would be able to override the policy, but Joe Tux-user would just stick to the default.

- Dave

In message <[email protected]> you write:

Hmmm, interesting problem. Perhaps your idea of having drivers hold a refcount for every device they control makes sense in this case, but as you point out, it's a significant departure from current policy.

With "rmmod -f" it's not quite reboot time, though.

Anyway, I don't think this is a battle I want to fight 8)

Cheers,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Doing that at modules_install time doesn't work for external drivers (which are _not_ in the kernel tree) ...

Gerd

You can't please everybody.  And usually if you _try_ to please
everybody, the end result is one big mess.
				-- Linus Torvalds, 2002-04-20