Skip to contentSkip to conversation

Generating external toolchains with Buildroot

Buildroot Thomas Petazzoni 3 participants 4 messages Aug 25, 2012 – Aug 28, 2012

lore
Share & tools
lore
#0 lore raw

Hello,

In order to make the autobuilders toolchain available to everyone, I had to generate tarballs of Buildroot toolchains. Basically, I just did BR2_HOST_DIR=/opt/<some-toolchain-name> and built just the toolchain. This has raised two problems.

Problem 1: everything is in usr/
================================

In the host directory, Buildroot installs everything under a usr/ subdirectory. So the toolchain binaries are available under usr/bin/<foo>-gcc and al.

However, all other external toolchains (from Sourcery CodeBench, Crosstool-NG, Blackfin, Linaro, etc.) all have the toolchain binaries directly under bin/.

So the external toolchain code extracts the toolchain tarball into host/opt/ext-toolchain/ and excepts to find the binaries in host/opt/external-toolchain/bin/. Which is not the case with Buildroot toolchain if you simply create a tarball of the host/ directory contents.

For now, I've simply adjusted how I create the tarball so that the tarball has everything without the usr/ sub-directory.

Question: should we be changing our host directory policy and use --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we keep things as it is and leave with this little drawback ?

Problem 2: relocalability
=========================

The main problem though is the relocalability of the toolchain. We have discussed in length the relocability of the Buildroot SDK (i.e, the host/ directory) during previous Buildroot Developer meetings. For the specific case of a SDK that contains only the toolchain, there is only one problem I had: we build gmp, mpfr and gmp as shared libraries, and the gcc binaries have absolute rpath in their binaries to reference the host/usr/lib directory in which these shared libraries are installed.

We thought of solving this problem by using a relative rpath:
-Wl,-rpath,$ORIGIN/../lib. Unfortunately, this probably works fine for binaries installed in host/usr/bin, but not for the cc1 and collect2 binaries installed in host/usr/libexec/gcc/<foo>/<gcc-version>/. Those binaries would need a different relative rpath to work properly.

For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.

Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this problem? Another solution?

Thanks!

Thomas

Signature · 4 lines
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#1 lore raw

Underwhelming number of replies in this thread...

Quoted 10 lines · Thomas Petazzoni

Hello,

In order to make the autobuilders toolchain available to everyone, I had to generate tarballs of Buildroot toolchains. Basically, I just did BR2_HOST_DIR=/opt/<some-toolchain-name> and built just the toolchain. This has raised two problems.

Problem 1: everything is in usr/
================================

[snip]

Quoted 4 lines

Question: should we be changing our host directory policy and use --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we keep things as it is and leave with this little drawback ?

Yes! I've always been annoyed with the redundant usr part, especially
since two or three packages install in $(HOST_DIR)/bin...

It's a big change but can probably easily be automated with sed.

Quoted 17 lines

Problem 2: relocalability
=========================

The main problem though is the relocalability of the toolchain. We have discussed in length the relocability of the Buildroot SDK (i.e, the host/ directory) during previous Buildroot Developer meetings. For the specific case of a SDK that contains only the toolchain, there is only one problem I had: we build gmp, mpfr and gmp as shared libraries, and the gcc binaries have absolute rpath in their binaries to reference the host/usr/lib directory in which these shared libraries are installed.

We thought of solving this problem by using a relative rpath:
-Wl,-rpath,$ORIGIN/../lib. Unfortunately, this probably works fine for binaries installed in host/usr/bin, but not for the cc1 and collect2 binaries installed in host/usr/libexec/gcc/<foo>/<gcc-version>/. Those binaries would need a different relative rpath to work properly.

Quoted 6 lines

For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.

I'm not sure if linking mpfr, gmp and mpc statically is so bad... How
much larger do the binaries become with static linking? If it's not much larger, the startup of cc1 may actually become faster with static linking!

Quick comparison: buildroot-built cc1 for gcc-4.5.3 is 11M; statically-linked
cc1 from Sourcery's gcc-4.5.2 is 12M.

I think crosstool-NG links statically as well. Yann?

Quoted 3 lines

Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this problem? Another solution?

Not very nice. Perhaps a specific rpath for linking cc1 and friends?

Regards,
Arnout

Signature · 6 lines
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#3 lore raw
In reply to Thomas Petazzoni
Quoted 6 lines

For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.

What's the draw back there?

Signature · 8 lines
_______________________________________________________________________

'Write programs that do one thing and do it well. Write programs to work
together. Write programs to handle text streams, because that is a
universal interface'

(Doug McIlroy)
_______________________________________________________________________
#2 lore raw
In reply to Arnout Vandecappelle

Le Tue, 28 Aug 2012 09:36:40 +0200,
Arnout Vandecappelle <[email protected]> a ?crit :

Quoted 11 lines
Quoted 3 lines

Problem 1: everything is in usr/
================================

[snip]

Quoted 4 lines

Question: should we be changing our host directory policy and use --prefix=$(HOST_DIR) instead of --prefix=$(HOST_DIR)/usr ? Or should we keep things as it is and leave with this little drawback ?

Yes! I've always been annoyed with the redundant usr part,
especially since two or three packages install in $(HOST_DIR)/bin...

Ah, really? Which ones?

Quoted 1 line

It's a big change but can probably easily be automated with sed.

Well, most of the host packages are probably using the host-autotools-package infrastructure, so those are easy to change:
only one place. Of course, all the host-generic-package would need manual intervention.

Peter, what's your feeling about this?

Quoted 8 lines
Quoted 6 lines

For the toolchains that I've put online at
http://autobuild.buildroot.org/toolchains/tarballs/, I've solved this problem by modifying Buildroot so that it builds mpfr, gmp and mpc as static libraries, so that the gcc binaries do not depend to any shared libraries besides the C library. And this seems to work fine, though it is not a completely satisfying solution.

I'm not sure if linking mpfr, gmp and mpc statically is so bad...

No, it's not so bad. It's just that from a "beauty" point of view, I would have preferred to be able to link dynamically against them. Just because we _should_ be able to do it :-)

Quoted 6 lines

How much larger do the binaries become with static linking? If it's not much larger, the startup of cc1 may actually become faster with static linking!

Quick comparison: buildroot-built cc1 for gcc-4.5.3 is 11M;
statically-linked cc1 from Sourcery's gcc-4.5.2 is 12M.

Yes, the impact is not that significant, you're right.

Quoted 1 line

I think crosstool-NG links statically as well. Yann?

Yes, crosstool-NG links statically.

Quoted 6 lines
Quoted 3 lines

Any ideas on this one? Should we add both -Wl,-rpath,$ORIGIN/../lib and -Wl,-rpath,$ORIGIN/../../../../lib when building gcc to solve this problem? Another solution?

Not very nice. Perhaps a specific rpath for linking cc1 and
friends?

That means diving into the internal gcc build process. You, crazy guy? :-)

Thomas

Signature · 4 lines
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com