Modules and the Interfaces who Love Them (Take I)

Feedback appreciated. It's aimed at driver writers.
================
Documentation/module-writing.txt

Writing Modules and the Interfaces To Be Used By Them: A Gentle Guide. Copyright 2002, Rusty Russell IBM Corportation

Modules are running parts of the kernel which can be added, and sometimes removed, while the kernel is operational.

There are several delicate issues involved in this procedure which indicate special care should be taken.

There are three cases you need to be careful:

1) Any code which creates an interface for callbacks (ie. almost any function called register_*)

	=> See Rule #1

2) Any modules which use (old) interfaces which do not obey Rule #1

	=> See Rule #2

Rule #1: Module-safe Interfaces. Any interface which allows
registration of callbacks, must also allow registration of a "struct module *owner", either in the structure or as a function parameter, and it must use them to protect the callbacks. See "MAKING INTERFACES SAFE".

Exception #1: As an optimization, you may skip this protection if you
*know* that the callbacks are non-preemtible and never sleep (eg. registration of interrupt handlers).

Rule #2: Modules using unsafe interfaces. If your module is using any
interface which does not obey rule number 1, that means your module functions may be called from the rest of the kernel without the caller first doing a successful try_module_get().

You must not register a "module_cleanup" handler, and your module cannot be unloaded except by force. You must be especially careful in this case with initialization: see "INITIALIZING MODULES WHICH USE UNSAFE INTERFACES".

MAKING INTERFACES SAFE

A caller must always call "try_module_get()" on a function pointers's owner before calling through that function pointer. If "try_module_get()" returns 0 (false), the function pointer must *not* be called, and the caller should pretend that registration does not
exist: this means the (module) owner is closing down and doesn't want any more calls, or in the process of starting up and isn't ready yet.

For many interfaces, this can be optimized by assuming that a structure containing function pointers has the same owner, and knowing that one function is always called before the others, such as the filesystem code which knows a mount must succeed before any other methods can be accessed.

You must call "module_put()" on the owner sometime after you have called the function(s).

If you cannot make your interface module-safe in this way, you can at least split registration into a "reserve" stage and an "activate" stage, so that modules can use the interface, even if they cannot (easily) unload.

INITIALIZING MODULES WHICH USE UNSAFE INTERFACES

Safe interfaces will never enter your module before module_init() has successfully finished, but unsafe interfaces may. The rule is simple:
your init_module() function *must* succeed (by returning 0) if it has successfully used any unsafe interfaces.

So, if you are only using ONE unsafe interface, simply use that interface last. Otherwise you will have to use printk() to report failure and leave the module initialized (but possibly useless).

If you have questions about how to apply this document to your own modules, please ask [email protected] or [email protected].

Thankyou,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Hi,

If that's your audience, I expect a very confused audience. You can make it very simple: There are safe interfaces and there are broken interfaces and you shall never write or use broken interfaces. For the majority of driver writers that's good enough.
Any documentation about module writing should also include/point to a chapter about resource management. The user has to understand anyway, by whom the module resources are used, otherwise he has more problems than just module unloading. Module management is just a small part of this whole picture or could be part of it, but right now the current module code is desperate attempt at keeping it out of it.
I would prefer if the user would be teached about proper resource management at kernel level. As soon as the user gets that right he will also have no problems to get module unloading right _if_ that would follow the same rules, but currently it involves lots of black magic instead. Rusty, I'm not impressed by the new module code, maybe I'm missing something, but it doesn't fix anything and only encourages to write more broken interfaces.

bye, Roman

In message <Pine.LNX.4.44.0211131430190.2109-100000@serv> you write:

Yes, maybe I should.

Yes, it'd be nice to have good documentation on this.

Look, your implementation was slow, confusing, invasive, inflexible *and* buggy.

Slow: Your approach where every interface has to do reference counts even though they're only useful for modules makes every interface slow, whether they are using modules or not. You can't make them fast, because that would make every interface NR_CPUS * sizeof(cacheline) larger.

Confusing: Your "deregistration can fail if busy but must succeed otherwise" is guaranteed to confuse authors who will wonder why. Currently some fail if it wasn't registered in the first place. To understand why this has to change, they have to understand your interface. You require a massive style change of not handling failure within the failing function. Each module has to implement four (or was it five?) functions for managing it.

Invasive: Every unregister interface to change, although many are very happy with try_inc_modcount(). Worse, every module in the kernel has to change to implement your method.

Inflexible: There is no way your scheme could allow "rmmod --wait" or "rmmod --force" without adding yet another interface to *every* module.

Buggy: You did not solve the "module used before init failed" problem, but left the module loaded and uninitialized in this case.

I seriously question your taste in this matter. You obviously hold a
personal dislike for my code. Fine.

Then you don't understand the problem.

Sorry, but I'm wasting my time sending you mail. 8( Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

(tangent alert)

Objects controlled by interfaces typically should be ref-counted... life in the kernel is slowly getting better WRT this, but we're not there yet. In any case, the ref counts help, not hurt.

Jeff

Hi,

My patch was meant as demonstration, you reduce it to a single aspect - the module - driver interface.
The patch was intended to demonstrate more. First of all, how to fix the module mess without breaking everything. It demonstrated a way to introduce a new interface without breaking compability. The new driver interface on top of it was optional, it could also have been the old interface or your monster refs.

Doing to the linking in the kernel is just plain wrong. Most of the module code could perfectly live in user space and it could be as simple as your kernel loader.

Maybe you could explain, what problems it fixes, that justifies complete breakage?

bye, Roman

Hi,

BTW while talking about taste: I'd rather make certain modules not unloadable, than have code like stop_refcounts() in the kernel. Module loading _and_ unloading could be so simple and fast, but it seems people prefer to keep the module stuff black magic. If nobody else cares I won't care either, alsa was the last reason I needed modules, now I can live without it.

bye, Roman