forestalling GNU incompatibility - proposal for binary relative dynamic linking
hey all,
Forgive the crosspost in advance, but I had an idea that touched many areas, and would need input from multiple groups associated with the gnu build chain, and perhaps the kernel itself.
After spending *two weeks* on various ways of building glibc, I'm convinced that the gnu/linux toolchain is in great danger of losing interoperability.
The main problem is that the glibc's supplied with each commercial system are *heavily* patched. My Suse 9.2 system has a rpm for glibc with fifty patches. Fifty patches!
Fifty patches which make the SuSE glibc binarily incompatible with the redhat, and so on. And everything is incompatible with the vanilla flavor.
All this makes me very skeptical that I'm ever going to get to a 'standard' out of the box glibc build. I build a standard glibc, and then all the supplied programs that come with SuSE break. And each vendor has the same problem - new that they are stuck with a heavily patched glibc, what chance do *they* have of getting back to a standard with the need to have old programs still work with the upgrades?
What IMO is desperately needed is a backwards compatibility hack or hacks. And of course the will of the different linux distribution providers to migrate back to the standard gnu toolchain.
Below is such a hack that lets providers do this, and solves quite a few problems in the process - the basic problem being that one currently can't get from a nonstandard glibc to a standard one without quite a bit of pain - for migrating back breaks all the legacy binaries out there.
What is needed is the ability to reference multiple versions of the glibc WITHOUT changing my environment to do so.
Ie: If I set up my LD_LIBRARY_PATH to reference:
setenv LD_LIBRARY_PATH /system/path/to/libc:.....
then the SuSE executables work fine but my new executables break, and if I set up my LD_LIBRARY_PATH to reference:
setenv LD_LIBRARY_PATH /new/path/to/libc:.....
then my new executables work, but my *old* executables break.
What I'd like to do is be able to set up my LD_LIBRARY_PATH so that I can reference it from the point of view of the *executable*:
setenv LD_LIBRARY_PATH "*/../lib:....."
Here, read "* == full path of dirname of executable".
So if gcc was installed in say, "/opt/tools/bin/gcc", LD_LIBRARY_PATH would become at runtime "/opt/tools/bin/../lib" or "/opt/tools/lib". And hence if I had a libc.so installed there, it would pick it up and use it.
Anyways, I have no idea whether or not this idea is being considered or has been considered in the past, but AFAICT it would save me the trouble of having hundreds of wrapper scripts. And I would like to get an idea of what adding something like this to the gnu toolset would require. Discussion is welcome..
Ed
Quoted 7 lines · Edward Peschko
What I'd like to do is be able to set up my
LD_LIBRARY_PATHso that I can reference it from the point of view of the *executable*:setenv LD_LIBRARY_PATH "*/../lib:....."Here, read "* == full path of dirname of executable".
See -Wl,-rpath,'$ORIGIN/../lib/'
r~
Quoted 5 lines · Richard Henderson
Quoted 4 lines · Edward Peschko
What I'd like to do is be able to set up my
LD_LIBRARY_PATHso that I can reference it from the point of view of the *executable*:
Quoted 8 lines
Quoted 3 lines
setenv LD_LIBRARY_PATH "*/../lib:....."Here, read "* == full path of dirname of executable".
See -Wl,-rpath,'$ORIGIN/../lib/'
r~
cool.. any chance for some syntactic sugar so me (and other users/vendors) wouldn't need to change any of their build scripts and compilation processes?
The only thing I would see as a drawback would be backwards compatibility, but how often do people have directories named '*'?
Ed
Quoted 3 lines · Edward Peschko
cool.. any chance for some syntactic sugar so me (and other users/vendors) wouldn't need to change any of their build scripts and compilation processes?
Uh, like what? That's about as simple as you can get.
r~
Quoted 9 lines · Richard Henderson
Quoted 3 lines · Edward Peschko
cool.. any chance for some syntactic sugar so me (and other users/vendors) wouldn't need to change any of their build scripts and compilation processes?
Uh, like what? That's about as simple as you can get.
r~
I don't understand.
Which is simpler, changing an environmental variable, or adding extra CFLAGS to every single compile and recompiling?
In addition, in your --rpath example, the relative pathing is hardcoded into the executable, wheras with "*" you could modify the runtime behavior of the executable at runtime. I suppose you could change this with chrpath, but why bother? What if you want to test out two versions of relative libraries side by side?
And in any case, I'm not even sure if you can change the runtime path to something longer than what currently exists in the executable using chrunpath.
And finally, certain programs (glibc, for example) seem to get into to trouble (ie: not compile) when you use --rpath flags. So, what's the issue with "*"?
Ed
Quoted 13 lines · Edward Peschko
Quoted 9 lines · Richard Henderson
Quoted 3 lines · Edward Peschko
cool.. any chance for some syntactic sugar so me (and other users/vendors) wouldn't need to change any of their build scripts and compilation processes?
Uh, like what? That's about as simple as you can get.
r~
I don't understand.
Quoted 8 lines
Which is simpler, changing an environmental variable, or adding extra CFLAGS to every single compile and recompiling?
In addition, in your --rpath example, the relative pathing is hardcoded into the executable, wheras with "*" you could modify the runtime behavior of the executable at runtime. I suppose you could change this with chrpath, but why bother? What if you want to test out two versions of relative libraries side by side?
You might want to take a look at Richard's suggestion again. The string '$ORIGIN' gets hardcoded into the binary and handled by the dynamic linker.
But really, RPATH is a good solution to almost no problems.
Signature · 1 line
Daniel Jacobowitz
Quoted 3 lines · Edward Peschko
After spending *two weeks* on various ways of building glibc, I'm convinced that the gnu/linux toolchain is in great danger of losing interoperability.
sounds like what you want is already being tackled by OSDL and their Binary Regression Testing group ...
http://groups.osdl.org/apps/group_public/workgroup.php?wg_abbrev=binary_sig
http://www.osdl.org/docs/isv_issues_and_binary_testing.pdf
-mike
Quoted 10 lines · Mike Frysinger
Quoted 3 lines · Edward Peschko
After spending *two weeks* on various ways of building glibc, I'm convinced that the gnu/linux toolchain is in great danger of losing interoperability.
sounds like what you want is already being tackled by OSDL and their Binary Regression Testing group ...
http://groups.osdl.org/apps/group_public/workgroup.php?wg_abbrev=binary_sig
http://www.osdl.org/docs/isv_issues_and_binary_testing.pdf
-mike
well of course the osdl is going to focus on this, but they need tools and functionality to do it correctly..
In particular, the statement 'Vendor lock-in (at any level) is not desirable' is false - there *is* vendor lock-in, and the suggestion of relative pathing for LD_LIBRARY_PATH is just one way to migrate back to doing things the right way.
Distributions are basically hoist by their own petard - they need to support old legacy executables, which have nonstandard glibc's. And since they need to support legacy executables in the past, they need to support them in the future.
What I'd envision is that the distributions split basically into two:
executables using the old style glibc/libraries, and executables using the new, standard glibcs. There would be two paths,
/usr/bin
and
/usr/standard/bin
The first directory would contain old executables which haven't been ported to the standard glibc. The second would contain executables that have. The relative pathing in LD_LIBRARY_PATH would insure that each tree used the correct libraries.
In the process of making /usr/standard/bin, linux vendors would need to make their rpms both path-neutral and build clean. However, they would *not* need to hold up their release process until everything is ported - they could pick and choose which applications were the most important to port. Ultimately, /usr/standard/bin would go away, and be moved back to /usr/bin, and perhaps the cycle could begin again, with upgrades going into a new /usr/standard/bin.
Ed
(
ps -
just to forestall a particular objection that was pointed out to me
- simply use chroot - I don't want a chroot environment. I want to use both old binaries and new binaries seamlessly.
Ed
)
On Monday 24 January 2005 23.24, Edward Peschko wrote:
[ Linux interoperability in danger - libc != libc ]
Hi,
[this may be just ignorance - I only use one distro (Debian), and don't try to mix binaries]
Before discussing solutions: Is there a survey on how bad the problem really is? IIRC some minor API changes were done between some gcc versions, but mostly on the C++ side. I know Debian, too, ships with a quite heavily patched toolchain, but the Debian libc/binutils/gcc packagers work together with the upstream developers, and many Debian specific patches end up in the official releases sooner or later. (And most of Debian's patches concern architectures like ARM, m68k, HPPA etc., which the big distributions don't support anyway.)
So, I guess, the survey would need to compare only C programs (not C++), since you explicitly are talking about libc - C++ has always been more difficult. And, it would be important to differentiate between incompatibilites caused by gcc versions from incompatibilities really caused by vendor specific modifications.
cheers
-- vbi
P.S.: looking at Debian's libc6 glibc package (2.3.2.ds1-20):
- 106 patches total
Going by the name of the patch files only:
- 8 hurd specific
- 4 arm specific
- 9 hppa specific
- 4 m68k specific
- 3 alpha specific
- 1 amd64 specific
- 1 x86 specific
- 4 sparc specific
- 5 mips specific
- 3 ppc specific
- 1 ia64 specific
- 3 s390 specific
- 4 correcting paths/build system only (wouldn't affect the libc API as
such)
- 15 locale specific (dito)
So, over half of the patches will likely not affect 95% of Linux users. Of the rest, a good number clearly affect multithreaded programs only (which, admittedly, are much more widespread now than a few years back.) I didn't look at any patches at all, so I can't judge how much the patches do really change the libc. Also, I can't say how intrusive the patches of other distributions are.
Signature · 3 lines
We are not loved by our friends for what we are; rather, we are loved in spite of what we are. -- Victor Hugo