Userspace Test Framework for module loader porting

The userspace test framework I used to develop module loading on different archs is up at:

http://www.kernel.org/pub/linux/kernel/people/rusty/modules/module-test-framework.tar.gz

I found it much easier to use for each arch than doing the crash/reboot cycle (and you can use a real debugger).

BTW, the change to use shared objects for modules is going to be a 2.7
thing: after 10 architectures, MIPS toolchain issues made it non-trivial. So the current stuff is what is going to be there for 2.6, so no point waiting 8)

Please report any problems!
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Rusty> BTW, the change to use shared objects for modules is going to Rusty> be a 2.7 thing: after 10 architectures, MIPS toolchain issues Rusty> made it non-trivial. So the current stuff is what is going Rusty> to be there for 2.6, so no point waiting 8)

What about all the problems that Richard Henderson pointed out with the original in-kernel module loader? Were those solved? My gut feeling is that we really want shared objects for kernel modules on ia64 (and probably alpha?).

	--david

Yes. The most important one for correctness was

[email protected], 2003-01-01 19:02:38-08:00, [email protected]
[PATCH] Modules 3/3: Sort sections

RTH's final complaint (so far 8) was that we should sort the module
sections: archs might require some sections to be adjacent, so they can all be reached by a relative pointer (ie. GOT pointer). This implements that reordering, and simplfies the module interface for architectures as well.

Previously an arch could specify it wanted extra space, but not where that space would be. The new method (used only by PPC so far) is to allocate an empty section (in asm/module.h or by setting LDFLAGS_MODULE to use an arch specific linker script), and expand that to the desired size in "module_frob_arch_sections()".

Well, most everyone wants it. Except that MIPS is terminally broken. They need a rewrite of bfd/elfxx-mips.c in order to be able to do non-pic ET_DYN images. Which leaves the rest of us out in the cold.

r~

Rich> On Mon, Jan 06, 2003 at 11:38:20AM -0800, David Mosberger

Rich> Yes.

OK, that's good.

Rusty, have you maintained the ia64 support of your in-kernel loader? To be honest, I have less than zero interest in maintaining such code. I'd rather prefer the old (user-level loader) or the new shared-object loader. (Of course, if someone else wants to volunteer, that would be fine, too... ;-)

	--david

In message <[email protected]> you write:

No, but I can update it next week when I'm back in the office. Should be trivial to do; I'll pass it by Mike Stephens if he's willing, though, since he wrote the original code I cribbed off.

Really? Because it already exists (and is maintained by someone else) or for some other reason?

I thought about letting archs choose which one they wanted to use, but it would really mess up the core code. Of course, the transition won't break userspace (kind of the whole point of the in-kernel module loader).

Cheers!
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Rusty> Really? Because it already exists (and is maintained by Rusty> someone else) or for some other reason?

Yeah, I'm lazy: I don't really want to have to deal with two new module loaders: one for 2.6, soon to be followed by one for 2.7. But if someone volunteers to do and _maintain_ an interim kernel loader, that's fine with me.

Rusty> I thought about letting archs choose which one they wanted to Rusty> use, but it would really mess up the core code. Of course, Rusty> the transition won't break userspace (kind of the whole point Rusty> of the in-kernel module loader).

But it would be more in keeping with the Linux philosophy: do the Right Thing, fix up "broken" stuff by doing whatever is necessary. I'm also a bit worried about changing module loaders so often. Yeah, once you switch to a kernel-loader, presumably users won't be affected, but (kernel-module) developers will be.

	--david

In message <[email protected]> you write:

Well, "soon" here is > 12 months away, of course. And most of it involves removing, rather than adding, code.

I think you missed the "work around what we can't change" (eg. always initializing per-cpu variables because Sparc's toolchain is broken, or adding that crazy restart stuff so we didn't have to create a one-arg re-enterable nanosleep then make glibc use it).

And, of course, the other Golden Rule: "if it's not x86, it doesn't matter" 8)

While ET_DYN modules are a reasonably serious win for ia64 (and probably hppa) (ie. -300 lines or so), they're a minor win for alpha and ppc64 (-100 lines or so), and no real change for arm, i386, ppc, sparc, and sparc64. It's a lose for x86_64 (toolchain fixes, unless they want to use -fPIC for modules), mips and mips64 (major toolchain fixes, unless they want to use -fPIC for modules and stop using r28 for current inside modules).

So, if I were ia64 maintainer, I'd be lobbying for ET_DYN modules now, too, but I don't it's a big enough general win to outweigh the other problems.

Sorry,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Hmm, what's the gain from using shared objects as opposed to relocatables
and why is there any for non-PIC objects at all? After all, the reason for using shared objects is to handle shared PIC code, for which relocation is indeed easier to perform -- you only need to relocate the GOT and data sections, the latter being rare and with a limited number of relocation types. For non-PIC shared objects the GOT and data sections still need to be relocated, but text sections do as well, introducting all the relocation types back. The end result is almost no different from operating on relocatables.

From the MIPS pov, adding non-PIC support to tools would mean defining
R_MIPS_26 (with some effort these could mostly be avoided for modules less than 128kB in size), R_MIPS_32, R_MIPS_HI16 and R_MIPS_LO16 for shared objects. Additionally R_MIPS_64, R_MIPS_HIGHEST and R_MIPS_HIGHER for MIPS64. A few ABI issues should be resolved, specifically how to create the GOT and the dynamic section, possibly others, as the MIPS ABI supplement doesn't define handling of non-PIC shared objects. An implementation for the BFD, though likely tiresome, shouldn't be difficult in principle, but tricky issues may arrise due to BFD's subtleties. And the overall processing gain would be questionable, IMO.

Maciej

+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +

In message <[email protected]> you write:

[ Other maintainers dropped from CC, they're probably not that interested ].

Basically it comes down to ET_DYNs being designed to be easy to load. For most archs, new sections don't have to be allocated (eg. plt, GOT). Sections don't have to be sorted, or layed out. As you point out, relocations are bundled nicely together, etc.

On some architectures, making the linker do more of the work makes a significant difference (eg. in ia64 it generates the stub code to jump out of modules). The module being layed out continuously, however, means we need a "vmalloc_truncate" to free init sections (except for archs which use their own alloc functions anyway).

Like most things, it's a tradeoff. 19 archs makes it even more complex. Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Hi Rusty,

Could you please change the return-type of search_extable() to something that allows a bit more flexibility? The value returned by this function is "something that lets architecture-specific code recover from a memory-managment-fault". This may or may not be the same as an exception_table_entry. For example, on ia64, I want to return an already-relocated fixup-word. Perhaps the cleanest way to fix this would be to have:

	exception_fixup_t search_extable (...);

By default, you could then use

	typedef struct exception_table_entry *exception_fixup_t;

and on ia64 I could use:

typedef long exception_fixup_t.

The only restriction on exception_fixup_t would be that it's a type that can be tested for being equal to zero and, if it is zero, it would mean that there is no exception-table entry.

Alternatively, we could make search_extable() just always return a "void *" or a "long", but that's less clear and less type-safe.

	--david

In message <[email protected]> you write:

Sure. Of course, you could just cast the return value to a long inside ia64, but that's pretty unclear.

Hmm, I actually just tried to do this, and like all typedefs, it's icky. search_exception_table() belongs in kernel.h, not module.h where it is now. But that means kernel.h has to include asm/uaccess.h to get exception_fixup_t.

Maybe the cleanest way really is to simply document that your search_extable() returns the fixup word... 8(

Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Rusty> Hmm, I actually just tried to do this, and like all typedefs, Rusty> it's icky. search_exception_table() belongs in kernel.h, not Rusty> module.h where it is now. But that means kernel.h has to Rusty> include asm/uaccess.h to get exception_fixup_t.

Yeah, that is a bit ugly.

Rusty> Maybe the cleanest way really is to simply document that your Rusty> search_extable() returns the fixup word... 8(

It turns out that on Dec 18th laster year, HJ Lu fixed the gas bug that prevented me from using the place-relative relocations. I ended up biting the bullet and making the bleeding edge assembler a pre-requisite for building 2.5.59. I try to avoid that, but in this case, it was better than the alternatives.

Anyhow, it means that we don't need a separate type anymore for ia64.

Thanks,

	--david

In message <[email protected]> you write:

Yes. Richard withdrew the one about allocation (we used a special per-arch allocator, he missed it), the one about common section ordering was fixed by RTH noticing we use -fno-common, so all that logic was dropped anyway, and his section ordering query was fixed by his section ordering patch (which changed the arch interface slightly).

It's certainly simpler. My ia64 current implementation is around 500 lines, vs 225 lines for the shared version.

But I'm not prepared to shaft an arch over it this late in the cycle, and the current scheme puts less pressure on toolchains (several other binutils "issues" were discovered in implementing modules-as-shared-objects).

Hope that clarifies,
Rusty.
--
Anyone who quotes me in their sig is an idiot. -- Rusty Russell.