Skip to contentSkip to conversation

Emit OPD reloc for all global symbols and then some.

RFC John David Anglin 5 participants 15 messages Jun 20, 2005 – Jun 22, 2005 11 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 9 lines
Quoted 7 lines

Under hppa-linux:
------------------
We want to generate a PLABEL32 (OPD reloc) against all global symbols, and then some. For example it should be possible for an application to call dlsym asking for the address of main, and when compared to main it should match.

Does it work on Linux/ia32? If not, why should it work on Linux/hppa?

The linux manpage for dlsym indicates that symbols in the main program can be looked up with dlsym:

If filename is a NULL pointer, then the returned handle is for the main program. When given to dlsym(), this handle causes a search for a symbol in the main program, followed by all shared libraries loaded at program startup, and then all shared libraries loaded by dlopen() with the flag RTLD_GLOBAL.

So, if it doesn't work, that would seem to be a bug in Linux/ia32.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw
Quoted 13 lines · John David Anglin
Quoted 9 lines
Quoted 7 lines

Under hppa-linux:
------------------
We want to generate a PLABEL32 (OPD reloc) against all global symbols, and then some. For example it should be possible for an application to call dlsym asking for the address of main, and when compared to main it should match.

Does it work on Linux/ia32? If not, why should it work on Linux/hppa?

The linux manpage for dlsym indicates that symbols in the main program can be looked up with dlsym:

Quoted 8 lines

If filename is a NULL pointer, then the returned handle is for the main program. When given to dlsym(), this handle causes a search for a symbol in the main program, followed by all shared libraries loaded at program startup, and then all shared libraries loaded by dlopen() with the flag RTLD_GLOBAL.

So, if it doesn't work, that would seem to be a bug in Linux/ia32.

It works if the global symbol is exported. But not all global symbols in main are exported by default. You can use --export-dynamic to export all global symbols.

H.J.
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw
Quoted 21 lines · H. J. Lu
Quoted 20 lines · John David Anglin
Quoted 9 lines
Quoted 7 lines

Under hppa-linux:
------------------
We want to generate a PLABEL32 (OPD reloc) against all global symbols, and then some. For example it should be possible for an application to call dlsym asking for the address of main, and when compared to main it should match.

Does it work on Linux/ia32? If not, why should it work on Linux/hppa?

The linux manpage for dlsym indicates that symbols in the main program can be looked up with dlsym:

If filename is a NULL pointer, then the returned handle is for the main program. When given to dlsym(), this handle causes a search for a symbol in the main program, followed by all shared libraries loaded at program startup, and then all shared libraries loaded by dlopen() with the flag RTLD_GLOBAL.

Quoted 5 lines
Quoted 1 line

So, if it doesn't work, that would seem to be a bug in Linux/ia32.

It works if the global symbol is exported. But not all global symbols in main are exported by default. You can use --export-dynamic to export all global symbols.

Regardless of wether or not I export all global symbols in main, I must still emit OPD relocs in the executable for all dynamic symbols. In the past the OPD reloc was "fudged" and pointed at the PLT, and this is not a satisfactory situation. I want ld.so.1 to generate the OPD so we no longer need gcc to emit the __cffc helper function for comparisons.

I owe you a big thank you for writing the generic function descriptor code in glibc, since hppa is also using that code :)

c.

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw
Quoted 22 lines · Carlos O'Donell
Quoted 21 lines · H. J. Lu
Quoted 20 lines · John David Anglin
Quoted 9 lines
Quoted 7 lines

Under hppa-linux:
------------------
We want to generate a PLABEL32 (OPD reloc) against all global symbols, and then some. For example it should be possible for an application to call dlsym asking for the address of main, and when compared to main it should match.

Does it work on Linux/ia32? If not, why should it work on Linux/hppa?

The linux manpage for dlsym indicates that symbols in the main program can be looked up with dlsym:

If filename is a NULL pointer, then the returned handle is for the main program. When given to dlsym(), this handle causes a search for a symbol in the main program, followed by all shared libraries loaded at program startup, and then all shared libraries loaded by dlopen() with the flag RTLD_GLOBAL.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

So, if it doesn't work, that would seem to be a bug in Linux/ia32.

It works if the global symbol is exported. But not all global symbols in main are exported by default. You can use --export-dynamic to export all global symbols.

Regardless of wether or not I export all global symbols in main, I must still emit OPD relocs in the executable for all dynamic symbols. In the

						^^^^^^^^^^

Did you mean dynamic or global? There is a big difference. In your proposal, you said "We want to generate a PLABEL32 (OPD reloc) against all global symbols,". I am not familiar with HPPA. I just want to know what you have in mind.

Quoted 3 lines

past the OPD reloc was "fudged" and pointed at the PLT, and this is not a satisfactory situation. I want ld.so.1 to generate the OPD so we no longer need gcc to emit the __cffc helper function for comparisons.

H.J.
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 8 lines · H. J. Lu
Quoted 2 lines

Regardless of wether or not I export all global symbols in main, I must still emit OPD relocs in the executable for all dynamic symbols. In the

						^^^^^^^^^^

Did you mean dynamic or global? There is a big difference. In your proposal, you said "We want to generate a PLABEL32 (OPD reloc) against all global symbols,". I am not familiar with HPPA. I just want to know what you have in mind.

Sorry for being sloppy, there is a qualifier in my mind that I keep forgetting to write, read the following:

ld must emit an OPD reloc in the executable for each global symbol that had an OPD reloc to begin with. This way the dynamic loader can assign an OPD at runtime for use in a comparison. Dynamic symbols that had an OPD reloc should also emit the reloc in the executable.

Previously an executable just pointed the OPD data word for global and dynamic symbols at their associated .plt entries (since .plt entries like on ia64 are function descriptors). No OPD reloc emitted. Thus the address of the OPD was really the address of the .plt entry.

This meant that if you passed the address of a dynamic symbol to a shared library function, from which the symbol came from, the two addresses would be different. One would point to the executables .plt entry for that dynamic symbol, while the shared library would have an OPD for that symbol. The ip/gp of both would be the same though (after resolution of the .plt that is).

The shared case in hppa was always different, and a DSO has always created OPD relocs against symbols that started with OPD relocs.

Quoted 3 lines
Quoted 3 lines

past the OPD reloc was "fudged" and pointed at the PLT, and this is not a satisfactory situation. I want ld.so.1 to generate the OPD so we no longer need gcc to emit the __cffc helper function for comparisons.

Calling through the .plt is still faster than calling through an OPD. You can still call through an OPD using $dyncall in hppa. Thus we probably want to keep the .plt entry even though we are creating an OPD that could be used to call through.

---

I'm still a bit wet behind the ears, and I fumble the terminology, so please accept my apologies.

c.

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#11 lore raw
Quoted 10 lines · H. J. Lu
Quoted 8 lines
Quoted 2 lines

Regardless of wether or not I export all global symbols in main, I must still emit OPD relocs in the executable for all dynamic symbols. In the

						^^^^^^^^^^

Did you mean dynamic or global? There is a big difference. In your proposal, you said "We want to generate a PLABEL32 (OPD reloc) against all global symbols,". I am not familiar with HPPA. I just want to know what you have in mind.

Quoted 7 lines

Sorry for being sloppy, there is a qualifier in my mind that I keep forgetting to write, read the following:

ld must emit an OPD reloc in the executable for each global symbol that had an OPD reloc to begin with. This way the dynamic loader can assign an OPD at runtime for use in a comparison. Dynamic symbols that had an OPD reloc should also emit the reloc in the executable.

There is a difference between ia32 and hppa. As you note, global symbols are not automatically exported on ia32, even when the symbol is used in a function-pointer comparison. This is possible because ia32 doesn't use function descriptors.

However, on hppa, GCC generates a PLABEL32 relocation when a function symbol is used in a comparison. If this relocation resolves to a non-unique function descriptor in the PLT, then we get comparison errors when symbols get passed between shared and main for comparison. As a result, hpux and linux currently have magic functions to canonicalize function pointers. Aside from this being inefficient, it also causes issues for building kernel software (i.e., canonicalization is only applicable to user space).

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

Under hpux, in a 32-bit link or a +compat mode 64-bit link, ld marks only those symbols symbols that are actually referenced by a shared library seen at link time for export. In a 64-bit +std link, all symbols are exported by default. However, explicitly exporting symbols has the side effect that the symbols will not be removed when using dead code elimination.

Quoted 1 line
Quoted 1 line

From my standpoint, I would like to see the same behavior for both

32 and 64-bit links when hppa linux supports both 32 and 64-bit user spaces. The original linux implementation was modelled on hpux but over the years we have found that hpux compatibility isn't important.

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw
Quoted 3 lines · John David Anglin

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

hppa needs to be fixed to implement correct variants of the macros in dl-lookupcfg.h. It probably just has to copy the definitions of DL_FIXUP_* from sysdeps/ia64/dl-lookupcfg.h, and _dl_runtime_{resolve,profile} has to be adapted accordingly.

Andreas.

Signature · 8 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
#14 lore raw
In reply to a message not in the archive
Quoted 3 lines · John David Anglin
Quoted 1 line

From my standpoint, I would like to see the same behavior for both

32 and 64-bit links when hppa linux supports both 32 and 64-bit user spaces.

Yeah, that would make sense.

Quoted 3 lines

The original linux implementation was modelled on hpux
but over the years we have found that hpux compatibility isn't important.

When I first got involved in parisc-linux in 1999, there was an interest in commercializing parisc-linux. That would mean investing several million dollars per year on a team to support/test parisc-linux *and* compatibility (where possible) with hpux so customers could run smaller apps on both. HP chose not to do that (it was a very rational decision though personally disappointing). parisc-linux did not get commercialized and the developer community around parisc-linux is not interested in HPUX compatibility.

If someone is interested in HPUX application compatibility, they better start providing parisc-linux kernel patches this year. An ABI compatibility event in the tool chain would be sufficient reason for me to rip out the broken HPUX compatibility support currently in the parisc-linux source tree.
So far, I've lobbied to keep HPUX app support in the tree with the hope we could someday run the HPUX X11 server (3D HW Accel!). But the right people haven't volunteered to make that work.

grant
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#16 lore raw

Re: [parisc-linux] Re: [RFC] Emit OPD reloc for all global symbols

Quoted 10 lines
Quoted 3 lines · John David Anglin

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

hppa needs to be fixed to implement correct variants of the macros in dl-lookupcfg.h. It probably just has to copy the definitions of DL_FIXUP_* from sysdeps/ia64/dl-lookupcfg.h, and _dl_runtime_{resolve,profile} has to be adapted accordingly.

There could be problems here but it seems that hppa is exporting all global symbols in .dynsym.

Regarding the patch, I'm not sure why but it seems to result in more R_PARISC_NONE dynamic relocation records (one for each real dynamic relocation?).

Dave

Signature · 6 lines
J. David Anglin                                  [email protected]
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#18 lore raw

Re: [parisc-linux] Re: [RFC] Emit OPD reloc for all global symbols

Quoted 6 lines · John David Anglin
Quoted 6 lines
Quoted 3 lines · John David Anglin

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

Quoted 7 lines
Quoted 4 lines

hppa needs to be fixed to implement correct variants of the macros in dl-lookupcfg.h. It probably just has to copy the definitions of DL_FIXUP_* from sysdeps/ia64/dl-lookupcfg.h, and _dl_runtime_{resolve,profile} has to be adapted accordingly.

There could be problems here but it seems that hppa is exporting all global symbols in .dynsym.

Why does hppa do that? A global symbol in executable should be exported only if it is referenced by a DSO or --export-dynamic is used.

Quoted 5 lines

Regarding the patch, I'm not sure why but it seems to result in more R_PARISC_NONE dynamic relocation records (one for each real dynamic relocation?).

IA64 also uses function descriptors. Everything works fine.

H.J.
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

#19 lore raw

Re: [parisc-linux] Re: [RFC] Emit OPD reloc for all global symbols

In reply to a message not in the archive
Quoted 11 lines · John David Anglin
Quoted 10 lines
Quoted 3 lines · John David Anglin

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

hppa needs to be fixed to implement correct variants of the macros in dl-lookupcfg.h. It probably just has to copy the definitions of DL_FIXUP_* from sysdeps/ia64/dl-lookupcfg.h, and _dl_runtime_{resolve,profile} has to be adapted accordingly.

Quoted 6 lines

There could be problems here but it seems that hppa is exporting all global symbols in .dynsym.

Regarding the patch, I'm not sure why but it seems to result in more R_PARISC_NONE dynamic relocation records (one for each real dynamic relocation?).

I'm not sure why there are extra R_PARISC_NONE relocs, I was sure I had fixed this problem. There was a ref counting issue before, but I should review this.

c.

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw

Re: [parisc-linux] Re: [RFC] Emit OPD reloc for all global symbols

Quoted 12 lines · H. J. Lu
Quoted 11 lines · John David Anglin
Quoted 10 lines
Quoted 3 lines · John David Anglin

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

hppa needs to be fixed to implement correct variants of the macros in dl-lookupcfg.h. It probably just has to copy the definitions of DL_FIXUP_* from sysdeps/ia64/dl-lookupcfg.h, and _dl_runtime_{resolve,profile} has to be adapted accordingly.

Quoted 5 lines
Quoted 2 lines

There could be problems here but it seems that hppa is exporting all global symbols in .dynsym.

Why does hppa do that? A global symbol in executable should be exported only if it is referenced by a DSO or --export-dynamic is used.

Legacy behaviour. Does an ia64 hpux system also export main in .dynsms?

Quoted 7 lines
Quoted 5 lines

Regarding the patch, I'm not sure why but it seems to result in more R_PARISC_NONE dynamic relocation records (one for each real dynamic relocation?).

IA64 also uses function descriptors. Everything works fine.

This sort of behaviour is architecture dependant. We have problems in our implementation, and we are trying to fix them :)

c.

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

#22 lore raw
In reply to a message not in the archive
Quoted 10 lines · Andreas Schwab
Quoted 3 lines · John David Anglin

Having the PLABEL32 relocation resolve to an OPD appears to require that the symbol be exported. As a result, we have a different behaviors for dlsym on hppa and ia32. What's ia64 do?

hppa needs to be fixed to implement correct variants of the macros in dl-lookupcfg.h. It probably just has to copy the definitions of DL_FIXUP_* from sysdeps/ia64/dl-lookupcfg.h, and _dl_runtime_{resolve,profile} has to be adapted accordingly.

We have the correct variants implemented.

http://cvs.parisc-linux.org/glibc/sysdeps/hppa/dl-lookupcfg.h?rev=1.3&view=markup

Upstream glibc is not a good indicator of the status of our port. I am trying to merge as much as I can upstream.

c.

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

A message that is not in the archive (sent off-list or not yet received): [email protected]

#24 lore raw

[parisc-linux] [OT] Emit OPD reloc (HPUX compatibility)

Quoted 21 lines · Grant Grundler
Quoted 3 lines · John David Anglin
Quoted 1 line

From my standpoint, I would like to see the same behavior for both

32 and 64-bit links when hppa linux supports both 32 and 64-bit user spaces.

Yeah, that would make sense.

Quoted 3 lines

The original linux implementation was modelled on hpux
but over the years we have found that hpux compatibility isn't important.

When I first got involved in parisc-linux in 1999, there was an interest in commercializing parisc-linux. That would mean investing several million dollars per year on a team to support/test parisc-linux *and* compatibility (where possible) with hpux so customers could run smaller apps on both. HP chose not to do that (it was a very rational decision though personally disappointing). parisc-linux did not get commercialized and the developer community around parisc-linux is not interested in HPUX compatibility.

Quoted 8 lines

If someone is interested in HPUX application compatibility, they better start providing parisc-linux kernel patches this year. An ABI compatibility event in the tool chain would be sufficient reason for me to rip out the broken HPUX compatibility support currently in the parisc-linux source tree.
So far, I've lobbied to keep HPUX app support in the tree with the hope we could someday run the HPUX X11 server (3D HW Accel!). But the right people haven't volunteered to make that work.

This is completely orthogonal, but I think that supporting the HPUX X11 server is a pipe dream.

Simple HPUX applications will run. Anything more than that is going to require a *lot* of work. And a lot of documentation that will never be made available.

My personal goal is to be as compatible with other linux ports as possible, offering everything that they also offer.

e.g. TLS, NPTL ... and more.

c.

_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

#25 lore raw
In reply to a message not in the archive
Quoted 33 lines · Carlos O'Donell
Quoted 8 lines · H. J. Lu
Quoted 2 lines

Regardless of wether or not I export all global symbols in main, I must still emit OPD relocs in the executable for all dynamic symbols. In the

						^^^^^^^^^^

Did you mean dynamic or global? There is a big difference. In your proposal, you said "We want to generate a PLABEL32 (OPD reloc) against all global symbols,". I am not familiar with HPPA. I just want to know what you have in mind.

Sorry for being sloppy, there is a qualifier in my mind that I keep forgetting to write, read the following:

ld must emit an OPD reloc in the executable for each global symbol that had an OPD reloc to begin with. This way the dynamic loader can assign an OPD at runtime for use in a comparison. Dynamic symbols that had an OPD reloc should also emit the reloc in the executable.

Previously an executable just pointed the OPD data word for global and dynamic symbols at their associated .plt entries (since .plt entries like on ia64 are function descriptors). No OPD reloc emitted. Thus the address of the OPD was really the address of the .plt entry.

This meant that if you passed the address of a dynamic symbol to a shared library function, from which the symbol came from, the two addresses would be different. One would point to the executables .plt entry for that dynamic symbol, while the shared library would have an OPD for that symbol. The ip/gp of both would be the same though (after resolution of the .plt that is).

The shared case in hppa was always different, and a DSO has always created OPD relocs against symbols that started with OPD relocs.

Quoted 9 lines
Quoted 3 lines
Quoted 3 lines

past the OPD reloc was "fudged" and pointed at the PLT, and this is not a satisfactory situation. I want ld.so.1 to generate the OPD so we no longer need gcc to emit the __cffc helper function for comparisons.

Calling through the .plt is still faster than calling through an OPD. You can still call through an OPD using $dyncall in hppa. Thus we probably want to keep the .plt entry even though we are creating an OPD that could be used to call through.

Check out SYMBOL_REFERENCES_LOCAL, SYMBOL_CALLS_LOCAL, elfNN_ia64_dynamic_symbol_p and _bfd_elf_dynamic_symbol_p.

H.J.
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux