Lazy linking issues

So,

It turns out that our ideas of how lazy linking would work are not quite right. The problem is that fixup() has to be called with the struct link_map stored in GOT[1] of the object we are trying to relocate, which means the strategy of looking at _GLOBAL_OFFSET_TABLE_ obviously won't work (in ld.so it would always get us the link_map for ld.so, and it won't compile anyway since the code has to get linked into libc.a ... I don't actually know why it worked before, but probably because I forgot a */ on the comment at the top of TRAMPOLINE_TEMPLATE).

In fact the code in elf_machine_runtime_setup() is wrong as well since it is storing the link_map in the GOT of ld.so rather than the dynamic object that is being loaded lazily :)

I think the strategy of (ab-)using the LTP part of the PLT slot to pass extra info to the trampoline code is basically sound. The only problem is that we need more information than we currently have.

What the trampoline needs to know is:

1) Offset of the PLT relocation we are going to perform in the DT_JMPREL array. Note that this is often the same as the index in the PLT, but for us it currently isn't because our IPLT relocations are not sorted (due to _init weirdness).

2) The struct link_map * that was saved in elf_machine_runtime_setup().

Both of these are hard. The reason they are hard is because we need to find them for the object we are coming *from*, and since there is no code in the PLT we cannot easily arrange for this to be so. Actually we would need more help from the linker to do that, anyway, since the addresses of the GOT and PLT are not easily accessible from any object (since the LTP doesn't really point to either of them).

Here is what I tried (until I realized it was doomed to failure):

First, I added an extra machine-specific dynamic tag (I would have used DT_HP_LOAD_MAP but glibc does not support the 'OS-specific' dynamic tags, and in fact has the wrong value for DT_LOOS) pointing to the reserved area of each object and fixed up this object's IPLT to point to it. The larger reserved area contained:

0 - link-time address of _DYNAMIC (same as before) 4 - pointer to this object's struct link_map (also same) 8 - address of this object's PLT
12 - address of the LTP value for the fixup function

Having the extra dynamic tag is a good way to flag that an object can do lazy linking, so the changes would not affect older binaries.

The problem was that I was working under the false assumption that $$dyncall and import stubs would (or could) load the address of the PLT slot in %r1 (not sure why I thought that ... didn't look close enough at the import stub code I guess) and that we could get the offset of the PLT relocation this way. This doesn't work for two
reasons: (1) obviously, import stubs don't load %r1 with the address of the PLT slot, and (2) the PLT relocations aren't sorted so the value would be wrong anyway.

This means that my old idea of storing the index of the IPLT relocation in each PLT slot in elf_machine_runtime_setup() was essentially correct, but then we have the problem of how to get at the reserved area of the GOT. There is basically no way to do this that I can think of without adding another word to the PLT slot (which might not be a bad idea, actually).

Anyway I am going home, I'll leave you to think about this I guess...

[email protected], http://www.linuxcare.com/
Linuxcare. Support for the revolution.

The easiest thing would be to make PLTGOT always point to the start of .got, which is what most other architectures do. Having PLTGOT (and hence the linkage table pointer) point elsewhere is just an optimisation of dubious value. At the moment we don't try to remove the addil's from import stubs anyway, and in any case, positioning the LTP at the start of the .got (which is also the end of the .plt) is a fairly good choice for addressing both .plt and .got with 14-bit offsets.

Is this enough for lazy linking? No, because each jump through a plt entry needs to pass it's .got (struct link_map really) as well as pltrel offset to the fixup routine. I think the dynamic linker needs some help from the linker. Again stealing ideas from other architectures, let's modify the linker to set up the .plt function pointers to point back to the start of the plt. We'll reserve some space at the start of the plt for code to load any necessary pointers into registers. elf_machine_runtime_setup will need to relocate the .plt function pointers, and also the code at the start of the plt unless I can make it PIC.

Alan Modra

Linuxcare.  Support for the Revolution.

OK, here's what I've come up with.

First four entries of the .got are
0x0 - pointer to dynamic section (as before)
0x4 - pointer to this object's struct link_map (as before) 0x8 - address of the fixup function (normal or profile) 0xc - the LTP value for the fixup function

Code at the start of the .plt looks like:

	b,l	1f,%r26
	depi	0,31,2,%r26
1:	addil	LD'_GLOBAL_OFFSET_TABLE_ + 8 - ($PIC_pcrel$0 - 8),%r26
	ldwm	RD'_GLOBAL_OFFSET_TABLE_ + 8 - ($PIC_pcrel$0 - 12)(%r1),%r26
	copy	%r19,%r25
	ldw	4(%r1),%r19
	bv	%r0(%r26)
	ldw	-4(%r1),%r26

which puts struct link_map * in %r26, reloc offset in %r25, fixup LTP in %r19, &got[2] in %r1, and branches to the address in got[2]. Funny enough, this code exposed the fact that gas currently doesn't handle relocs using the LD and RD field selectors.

A non-pic version saving 2 instructions, but meaning the dynamic linker needs to relocate some insns:

	ldil	LD'_GLOBAL_OFFSET_TABLE_ + 8,%r1
	ldwm	RD'_GLOBAL_OFFSET_TABLE_ + 8(%r1),%r26
	copy	%r19,%r25
	ldw	4(%r1),%r19
	bv	%r0(%r26)
	ldw	-4(%r1),%r26

We still need some means for elf_machine_runtime_setup to figure out where the .got starts. One way is to make PLTGOT do this for us, which is my preferred solution. Alternatives would be a) read the elf section headers, or b) look at the first iplt reloc to find an entry in the .plt, read the function pointer which give us the above code, extract the .got pointer from the addil/ldil and ldwm insns. Was that retching I heard?

Hmm, one variation on this theme that occurs to me, is to put the above code at the _end_ of the plt - the advantage is that the .got is right next-door, so we don't need the addil, and case b) above becomes a little simpler.

Alan Modra

Linuxcare.  Support for the Revolution.

That's all a bit silly. How come no-one noticed and told me how foolish I am? We can't trash the arg registers willy-nilly.

I'm going for a stub at the end of the .plt, with .plt and .got guaranteed to be glued together. The fixup fptr will be immediately after the stub at the end of the .plt, ie. at got[-2] istead of got[2].

Alan Modra

Linuxcare.  Support for the Revolution.

Probably because I was asleep :)

Okay. I suppose I will take the same approach as the Alpha code does, and look for a particular instruction at the end of the GOT in elf_machine_runtime_setup() as an indication of whether lazy linking can be done or not.

[email protected], http://www.linuxcare.com/
Linuxcare. Support for the revolution.

Yep. I'll check in my mods to dl-machine.h in a few minutes after I fix this silly plt stub.

The plt stub looks like

1: ldw 0(%r20),%r22

	bv	%r0(%r22)
	ldw	4(%r20),%r21
2:	b,l	1b,%r20
	depi	0,31,2,%r20
9:	.word 0xc0ffee
	.word 0xdeadbeef

with entry at label 2. The idea here being that the linker can change the plt stub without magic numbers in the dynamic linker needing to be changed. So we jump into the trampoline with r19 = reloc offset, r20 = got-8, r21 = fixup ltp.

Alan Modra

Linuxcare.  Support for the Revolution.