Generating external toolchains with Buildroot
Hello,
In order to make the autobuilders toolchain available to everyone, I had to generate tarballs of Buildroot toolchains. Basically, I just did BR2_HOST_DIR=/opt/<some-toolchain-name> and built just the toolchain. This has raised two problems.
Problem 1: everything is in usr/
================================
In the host directory, Buildroot installs everything under a usr/ subdirectory. So the toolchain binaries are available under usr/bin/<foo>-gcc and al.
However, all other external toolchains (from Sourcery CodeBench, Crosstool-NG, Blackfin, Linaro, etc.) all have the toolchain binaries directly under bin/.
So the external toolchain code extracts the toolchain tarball into host/opt/ext-toolchain/ and excepts to find the binaries in host/opt/external-toolchain/bin/. Which is not the case with Buildroot toolchain if you simply create a tarball of the host/ directory contents.
For now, I've simply adjusted how I create the tarball so that the tarball has everything without the usr/ sub-directory.
Question: should we be changing our host directory policy and use --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we keep things as it is and leave with this little drawback ?
Problem 2: relocalability
=========================
The main problem though is the relocalability of the toolchain. We have discussed in length the relocability of the Buildroot SDK (i.e, the host/ directory) during previous Buildroot Developer meetings. For the specific case of a SDK that contains only the toolchain, there is only one problem I had: we build gmp, mpfr and gmp as shared libraries, and the gcc binaries have absolute rpath in their binaries to reference the host/usr/lib directory in which these shared libraries are installed.
We thought of solving this problem by using a relative rpath:
-Wl,-rpath,$ORIGIN/../lib. Unfortunately, this probably works fine for binaries installed in host/usr/bin, but not for the cc1 and collect2 binaries installed in host/usr/libexec/gcc/<foo>/<gcc-version>/. Those binaries would need a different relative rpath to work properly.
For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.
Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this problem? Another solution?
Thanks!
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Underwhelming number of replies in this thread...
Quoted 10 lines · Thomas Petazzoni
Hello,
In order to make the autobuilders toolchain available to everyone, I had to generate tarballs of Buildroot toolchains. Basically, I just did BR2_HOST_DIR=/opt/<some-toolchain-name> and built just the toolchain. This has raised two problems.
Problem 1: everything is in
usr/
================================
[snip]
Quoted 4 lines
Question: should we be changing our host directory policy and use --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we keep things as it is and leave with this little drawback ?
Yes! I've always been annoyed with the redundant usr part, especially
since two or three packages install in $(HOST_DIR)/bin...
It's a big change but can probably easily be automated with sed.
Quoted 17 lines
Problem 2: relocalability
=========================The main problem though is the relocalability of the toolchain. We have discussed in length the relocability of the Buildroot SDK (i.e, the host/ directory) during previous Buildroot Developer meetings. For the specific case of a SDK that contains only the toolchain, there is only one problem I had: we build gmp, mpfr and gmp as shared libraries, and the gcc binaries have absolute rpath in their binaries to reference the host/usr/lib directory in which these shared libraries are installed.
We thought of solving this problem by using a relative rpath:
-Wl,-rpath,$ORIGIN/../lib. Unfortunately, this probably works fine for binaries installed in host/usr/bin, but not for the cc1 and collect2 binaries installed in host/usr/libexec/gcc/<foo>/<gcc-version>/. Those binaries would need a different relative rpath to work properly.
Quoted 6 lines
For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.
I'm not sure if linking mpfr, gmp and mpc statically is so bad... How
much larger do the binaries become with static linking? If it's not much larger, the startup of cc1 may actually become faster with static linking!
Quick comparison: buildroot-built cc1 for gcc-4.5.3 is 11M; statically-linked
cc1 from Sourcery's gcc-4.5.2 is 12M.
I think crosstool-NG links statically as well. Yann?
Quoted 3 lines
Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this problem? Another solution?
Not very nice. Perhaps a specific rpath for linking cc1 and friends?
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286540 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
Quoted 6 lines
For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.
What's the draw back there?
Signature · 8 lines
_______________________________________________________________________ 'Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface' (Doug McIlroy) _______________________________________________________________________
Le Tue, 28 Aug 2012 09:36:40 +0200,
Arnout Vandecappelle <[email protected]> a ?crit :
Quoted 11 lines
Quoted 3 lines
Problem 1: everything is in
usr/
================================[snip]
Quoted 4 lines
Question: should we be changing our host directory policy and use --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we keep things as it is and leave with this little drawback ?
Yes! I've always been annoyed with the redundant usr part,
especially since two or three packages install in $(HOST_DIR)/bin...
Ah, really? Which ones?
Quoted 1 line
It's a big change but can probably easily be automated with sed.
Well, most of the host packages are probably using the host-autotools-package infrastructure, so those are easy to change:
only one place. Of course, all the host-generic-package would need manual intervention.
Peter, what's your feeling about this?
Quoted 8 lines
Quoted 6 lines
For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.I'm not sure if linking mpfr, gmp and mpc statically is so bad...
No, it's not so bad. It's just that from a "beauty" point of view, I would have preferred to be able to link dynamically against them. Just because we _should_ be able to do it :-)
Quoted 6 lines
How much larger do the binaries become with static linking? If it's not much larger, the startup of cc1 may actually become faster with static linking!
Quick comparison: buildroot-built cc1 for gcc-4.5.3 is 11M;
statically-linked cc1 from Sourcery's gcc-4.5.2 is 12M.
Yes, the impact is not that significant, you're right.
Quoted 1 line
I think crosstool-NG links statically as well. Yann?
Yes, crosstool-NG links statically.
Quoted 6 lines
Quoted 3 lines
Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this problem? Another solution?
Not very nice. Perhaps a specific rpath for linking cc1 and
friends?
That means diving into the internal gcc build process. You, crazy guy? :-)
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com