Problem with call across kernel modules

The attached files (in a shell archive) illustrate a problem with calls between kernel modules on ia64 linux.

The problem was verified on two ia64 systems: insmod version 2.4.22 on RedHat's 2.4.18-e.12smp, and insmod version 2.4.21 on a 2.4.21-pre3 kernel.

The Makefile makes two modules: caller.o and helper.o. The caller module calls the function helper from its init_module routine. The helper function is in the helper module. Making the call from init_module is a simplification for the example. The original problem made the bad call after the module was successfully loaded.

If you insmod caller.o first, it will fail with an unresolved symbol for helper, as expected.

After insmod'ing helper, the insmod of caller is expected to succeed, but the call to helper from init_module causes an Oops. The oops message seems to indicate that the procedure call to helper lands at the wrong address. The ISR 4294967296 (or 0x1 0000 0000) indicates the x bit (Execute exception - Interruption is associated with an instruction fetch).

The ip register is:
ip : [<0880210040004000>]

The two cases that discovered the problem reported 0000058014494800 and 000005800e293010 for their ip registers (same ISR). My test case does work on a 2.4.18 ia32 box.

# insmod caller.o
caller.o: unresolved symbol helper
# insmod helper.o
# dmesg
...
helper initialized
# insmod caller.o
Segmentation fault
# dmesg | tail -40
...
helper initialized
caller in init_module, about to call helper...
Unable to handle kernel paging request at virtual address 0880210040004000
insmod[1400]: Oops 4294967296
--> Using_Versions [] 0x40003fff <-- 

Pid: 1400, comm: insmod
psr : 0000101008026018 ifs : 8000000000000002 ip : [<0880210040004000>] Not tainted
unat: 0000000000000000 pfs : 0000000000000185 rsc : 0000000000000003
rnat: e0000040f77c7d50 bsps: e0000040f77c0000 pr : 000000000000a559
ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c0270033f b0 : a000000000354110 b6 : 088021004000400a b7 : 0000000000000000 f6 : 1003e000000000000000a f7 : 1003e0a3d70a3d70a3eb8
f8 : 1003ee0000000049f0400 f9 : 10003c000000000000000
r1 : 0004000000420020 r2 : 0000000000000000 r3 : 0000000000000000 r8 : 0000000000000032 r9 : 0000000000000018 r10 : 0000000000000000 r11 : 0000000000000a17 r12 : e0000040f77c7d90 r13 : e0000040f77c0000 r14 : 0000000000004000 r15 : a00000000034c0c8 r16 : 088021004000400a r17 : e0000040fc790048 r18 : 0000000000080000 r19 : e0000000049f0d30 r20 : e0000040fc790050 r21 : e0000000049f0408 r22 : e0000040f77c0000 r23 : e0000000049f0d28 r24 : e0000040fc790040 r25 : e0000040fc790058 r26 : 200000000029fd58 r27 : ffffffffffffffff r28 : 0000000000000000 r29 : 0000000000000001 r30 : 0000000000000018 r31 : 0000000000000c9e

Call Trace: [<e000000004414870>] sp=0xe0000040f77c7980

bsp=0xe0000040f77c1218
decoded to show_stack [kernel] 0x50 [<e0000000044150a0>] sp=0xe0000040f77c7b40 bsp=0xe0000040f77c11b8 decoded to show_regs [kernel] 0x7c0 [<e00000000442e490>] sp=0xe0000040f77c7b60 bsp=0xe0000040f77c1190 decoded to die [kernel] 0x190
[<e000000004450180>] sp=0xe0000040f77c7b60 bsp=0xe0000040f77c1130 decoded to ia64_do_page_fault [kernel] 0x780 [<e00000000440dec0>] sp=0xe0000040f77c7bf0 bsp=0xe0000040f77c1130 decoded to ia64_leave_kernel [kernel] 0x0

I wrote last week:

...

After some more looking:

The problem is that the symbol exported by default for the helper function (as visible from /proc/ksyms) is the address of the function's entry point, and not the address of a function descriptor.

So when insmod connects the second module to it, the call stub code that thinks it's loading the function address and the gp value is really loading the first two words of the code of the function itself. These work really badly as a function address and a gp value and show up in the oops message as bizarre values for IP and r1.

One solution:
EXPORT_SYMBOL(helper)
and add -DEXPORT_SYMTAB to the makefile.

If you don't use EXPORT_SYMBOL, but rely on the default "export everything", even -DEXPORT_SYMTAB doesn't seem to help. The symbol that gets exported will be wrong as described above.

So it looks like the question has become: Why does the default exported symbol from an insmod'd module not correctly point to a function descriptor as required on IA64? Or, if it isn't right, why make it visible?

Hopefully this is getting closer to the real problem.

Bob Montgomery, HP

I had not realised that you were relying on the "export everything by default" behaviour. That does not work for ia64 and I don't see any way of making it work. As you say, the global symbol for a function must point to the function descriptor, not the address. When you EXPORT_SYMBOL(function_name),

__attribute__((section("__ksymtab"))) = \
{ (unsigned long)&sym, __kstrtab_##sym }

uses &function_name so gcc generates the function descriptor in the modules export table. However if you rely on default exporting, gcc has not been told that the function will be exported so there is no function descriptor to use. That means there is nothing for insmod to point to, except the function address.

Bottom line - do not rely on default export behaviour for ia64.

Nobody should rely on this feature anyway, the default export is compatibility code that dates back to 2.0 kernels, current code should explicitly export what it needs. In 2.5 kernels that is exactly what happens, default exporting is off.

modutils 2.4.23 will issue a big warning for all architectures that have function descriptors when you load a module that relies on default export behaviour. "Functions must be explicitly exported on this architecture, attempts to use functions in module <foo> will probably kill your system".