external toolchains and sysroot
Hello,
There is a partial copy of toolchain's sysroot in staging directory, and the ext-wrapper will force this directory with --sysroot.
Which problem this procedure is supposed to fix?
Thanks,
Jean-Mickael
Quoted 4 lines · Jean-Mickael Guerin
There is a partial copy of toolchain's sysroot in staging directory, and the ext-wrapper will force this directory with --sysroot.
Which problem this procedure is supposed to fix?
It tells the compiler where to find headers and libraries. The original toolchain directory is fine for the standard libraries, but not for those installed afterwards.
Signature · 1 line
Richard Braun
Quoted 10 lines · Richard Braun
Quoted 4 lines · Jean-Mickael Guerin
There is a partial copy of toolchain's sysroot in staging directory, and the ext-wrapper will force this directory with --sysroot.
Which problem this procedure is supposed to fix?
It tells the compiler where to find headers and libraries. The original toolchain directory is fine for the standard libraries, but not for those installed afterwards.
I would expect the wrapper the use -I and -L to help the lookup of headers and libraries installed afterwards. Why --sysroot?
Quoted 2 lines · Jean-Mickael Guerin
I would expect the wrapper the use -I and -L to help the lookup of headers and libraries installed afterwards. Why --sysroot?
-I and -L extend the default list of search directories whereas --sysroot replaces them. This completely avoids using native libraries by mistake.
Signature · 1 line
Richard Braun
Quoted 7 lines · Richard Braun
Quoted 2 lines · Jean-Mickael Guerin
I would expect the wrapper the use -I and -L to help the lookup of headers and libraries installed afterwards. Why --sysroot?
-I and -L extend the default list of search directories whereas --sysroot replaces them. This completely avoids using native libraries by mistake.
I might have answered a bit fast. I suppose what you have in mind is let the toolchain use its own sysroot, defined at compile time (when the toolchain was built) and then extend it with -I and -L. The problem with this is that it would force everyone to use the very same path for one toolchain, which can be tedious for external toolchains (you might have several with the same built-in sysroot).
Signature · 1 line
Richard Braun
Quoted 9 lines · Richard Braun
Quoted 7 lines · Richard Braun
Quoted 2 lines · Jean-Mickael Guerin
I would expect the wrapper the use -I and -L to help the lookup of headers and libraries installed afterwards. Why --sysroot?
-I and -L extend the default list of search directories whereas --sysroot replaces them. This completely avoids using native libraries by mistake.
Quoted 7 lines
I might have answered a bit fast. I suppose what you have in mind is let the toolchain use its own sysroot, defined at compile time (when the toolchain was built) and then extend it with -I and -L. The problem with this is that it would force everyone to use the very same path for one toolchain, which can be tedious for external toolchains (you might have several with the same built-in sysroot).
Not exactly: let the toolchain use its own sysroot, and use -I /-L to look up in the staging directory. Original sysroot is read-only & shared, staging belongs to your build directory.
In another words my point is that the copy of sysroot is useless, using --sysroot=<my copy of built-in sysroot path, computed from the toolchain's compiler> cannot be safer than let it use the built-in syroot. Removing this copy will save time, space, and compatible whatever the directory layout of the toolchain.
Quoted 10 lines · Jean-Mickael Guerin
Not exactly: let the toolchain use its own sysroot, and use -I /-L to look up in the staging directory. Original sysroot is read-only & shared, staging belongs to your build directory.
In another words my point is that the copy of sysroot is useless, using --sysroot=<my copy of built-in sysroot path, computed from the toolchain's compiler> cannot be safer than let it use the built-in syroot.
Removing this copy will save time, space, and compatible whatever the directory layout of the toolchain.
Right, this should work just fine. I understand your point now, but then I can't answer you.
Signature · 1 line
Richard Braun
Quoted 9 lines · Jean-Mickael Guerin
Not exactly: let the toolchain use its own sysroot, and use -I /-L to look up in the staging directory. Original sysroot is read-only & shared, staging belongs to your build directory.
In another words my point is that the copy of sysroot is useless, using --sysroot=<my copy of built-in sysroot path, computed from the toolchain's compiler> cannot be safer than let it use the built-in syroot. Removing this copy will save time, space, and compatible whatever the directory layout of the toolchain.
I am the one who wrote the external toolchain support as it is now. Initially, things were working as you suggested, i.e keep the sysroot in its original place, and use -I/-L to point to the headers and libraries that Buildroot adds. It /mostly/ works, but causes for some packages horrible problems with libtool.
I don't have the time right now to dig into the archives, but this is a problem we worked on around December 2008, and it has allowed the external toolchain support to work nicely and properly since then.
If you have problems with external toolchain support, don't hesitate to point me to the toolchain you're using, and how to reproduce your problem. I am fairly sure it is not related to the fact that we're copying the sysroot.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Quoted 12 lines · Thomas Petazzoni
I am the one who wrote the external toolchain support as it is now. Initially, things were working as you suggested, i.e keep the sysroot in its original place, and use -I/-L to point to the headers and libraries that Buildroot adds. It/mostly/ works, but causes for some packages horrible problems with libtool.
I don't have the time right now to dig into the archives, but this is a problem we worked on around December 2008, and it has allowed the external toolchain support to work nicely and properly since then.
Quoted 4 lines
If you have problems with external toolchain support, don't hesitate to point me to the toolchain you're using, and how to reproduce your problem. I am fairly sure it is not related to the fact that we're copying the sysroot.
Well, the issue *is* this large copy.
We can improve the share of toolchains among multiple builds, and simplify this code for ext toolchain support (btw there is hack to avoid too big usr/locale..).
I can propose the patch, but which are the options to get confident? Do we have ext-toolchain in the serie of daily auto-build?
I could find in archive some discussion about Qtopia, DirectFB, but nothing explicitly related to sysroot or library path. Someone else may remember other problem? a specific toolchain?
Regards,
Jean-Mickael
Jean-Mickael,
Quoted 6 lines · Jean-Mickael Guerin
Quoted 4 lines
If you have problems with external toolchain support, don't hesitate to point me to the toolchain you're using, and how to reproduce your problem. I am fairly sure it is not related to the fact that we're copying the sysroot.
Well, the issue *is* this large copy.
Which toolchain are you using that is causing such a large copy?
Here, using the CodeSourcery 2012.03 toolchain, it copies 63 MB into host/usr/<tuple>/sysroot, and it takes a second when the toolchain tarball has just been uncompressed before, due to the page cache already having the contents of the toolchain.
With an already installed toolchain, and therefore no tarball uncompression, causing the page cache to be cold with regard to toolchain files, it takes 4 seconds here to copy the sysroot of a CS 2010.09 toolchain, which weights 59 MB.
How big is your sysroot?
Quoted 6 lines
We can improve the share of toolchains among multiple builds, and simplify this code for ext toolchain support (btw there is hack to avoid too big usr/locale..).
I can propose the patch, but which are the options to get confident? Do we have ext-toolchain in the serie of daily auto-build?
Yes, many external toolchains configuration are tested by the auto-builders. Since I was originally the primary person interested in external toolchain support, I added a lot of external toolchain configurations to the autobuilders.
Quoted 3 lines
I could find in archive some discussion about Qtopia, DirectFB, but nothing explicitly related to sysroot or library path. Someone else may remember other problem? a specific toolchain?
I remember having horrible problem with libtool, and also many hand-craft build system not properly passing -I/-L arguments.
Also, I believe that the current mechanism is cleaner: what we are actually doing is building a second sysroot for the toolchain, that we extend with our own libraries.
Unless proven otherwise, I believe that the stability we have achieved in external toolchain support using this solution is well worth the ~60 MB copy at the beginning of a build. It takes less than 5 seconds, with is in the statistical noise even compared to a simple Busybox-only build.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Quoted 5 lines · Thomas Petazzoni
Unless proven otherwise, I believe that the stability we have achieved in external toolchain support using this solution is well worth the ~60 MB copy at the beginning of a build. It takes less than 5 seconds, with is in the statistical noise even compared to a simple Busybox-only build.
FWIW, I agree. The copying of the sysroot is a bit tricky, but at least
after the copy is done, things are nice and consistent. Also, some trickery is required anyway to make sure the right things are copied into the target directory.
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286540 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F