Libtool work: a tentative summary
Hello,
There has recently been quite some work done around libtool handling in Buildroot, in the form of two proposals done by Martin Banky and Lionel Landwerlin. Those two proposals are quite different and we need to make a choice between them. This e-mail tries to summarize the libtool problem, how it is currently handled in Buildroot, the technical proposals made by Martin and Lionel with their respective advantages and drawbacks. Martin and Lionel are obviously invited to comment on those (and the rest of this e-mail as well).
Libtool problems
================
Libtool problems in cross-compilation mode are well-known. The famous http://www.metastatic.org/text/libtool.html lists most of the problems, and some hacks to solve them.
Basically, Libtool does not understand the fact that we may be building a root filesystem in a different place than "/", and keeps looking in /usr/lib for libraries.
It is worth noting that libtool 2.4 has finally introduced a --sysroot argument which should finally solve those problems. However, until libtool 2.4 is picked up by most packages, we will have to live with the libtool problems for quite some time. See
http://lists.gnu.org/archive/html/libtool/2010-09/msg00082.html.
Current Buildroot solution
==========================
The current Buildroot solution consists in two mechanisms :
*) Apply the package/buildroot-libtool.patch to packages when
<pkg>_LIBTOOL_PATCH is YES. This is the default value for autotools-based packages.
This patch directly modifies the ltmain.sh script, which will be used to generate the libtool script in the package as configure time.
This patch only works with libtool 1.5.x, but a number of packages are now using libtool 2.2.x, so more and more packages are doing <pkg>_LIBTOOL_PATCH=NO to disable the patch. This works for most packages but not for others (example: I'm bumping GTK to 2.20 and it needs a patch for the ltmain.sh script which has been generated by libtool 2.2.x)
*) Modify the .la files installed by Buildroot in order to add
$(STAGING_DIR) at the beginning of the "libdir" paths.
Martin Banky proposal
=====================
Martin Banky has sent a proposal to improve libtool handling in Buildroot on September 16th. See
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
and the following 4 e-mails from Martin.
Basically, Martin's proposal is to continue with the current solution to the libtool problem, but to clean it up a bit and to extend it to libtool 2.2.
This patch set has 3 main components :
*) Simplify the existing package/buildroot-libtool.patch by removing
useless chunks
*) Rename package/buildroot-libtool.patch to
package/buildroot-libtool-v1.5.patch, add a new package/buildroot-libtool-v2.2.patch. Those two patches respectively modify the ltmain.sh generated by libtool 1.5 and 2.2. Then the Autotools infrastructure is modified to apply one patch or the other depending on which libtool version the package is using.
*) The remaining modifications remove libtool-related patches in
various packages that have become useless thanks to the previous patches.
Advantages of this approach :
*) We're keeping the current solution and only cleanup/extend it to
work with recent libtool versions. This is good from a "conservative" point of view.
*) As today, we don't need to autoreconf all autotools-based packages
just to fix the libtool issue.
Drawbacks of this approach :
*) We may have similar issues with future versions of libtool, even if
the addition of --sysroot to libtool 2.4 probably means that libtool 1.5 and 2.2 will remain the only two versions of libtool causing problems in widespread use.
*) It keeps the current solution, which is a bit ugly: modify scripts
that are generated, and hack the .la files.
*) It modifies the .la by adding informations relative to
$(STAGING_DIR), which may cause problems if we want to generate packages (.ipkg) or if we would like to allow users to relocate the Buildroot staging directory.
Lionel Landwerlin proposal
==========================
Lionel Landwerlin has sent a proposal to improve libtool handling in Buildroot on September, 18th. See
http://lists.busybox.net/pipermail/buildroot/2010-September/037663.html
for the mailing list post.
Basically, Lionel's proposal is to patch libtool itself to make it sysroot-capable and then autoreconfigure all autotools-based packages to make them use the fixed libtool version.
Lionel's proposal is relatively complicated because in order for the patch on libtool to take effect, libtool itself needs to be autoreconfigured, which in turn requires libtool. So Lionel has introduced a host-libtool-host package, which is an unmodified libtool, installed in a new $(HOST_HOST_DIR) directory. He later uses that host-libtool-host to autoreconfigure the host-libtool (which has been previously patched to understand sysroot related things). host-libtool-host will be kept for the autoreconfiguration of host packages, while host-libtool is used for the autoreconfiguration of target packages. (If your head hasn't exploded yet, keep reading, otherwise, take a break and come back in 10 minutes). This is what the first patch in Lionel patch set does. The other patches are only minor related changes.
Advantages of this approach :
*) We no longer arbitrarly patch ltmain.sh in every package, with the
need to carry libtool-version-specific patches.
*) We no longer need to modify the .la files.
*) The approach seems cleaner.
*) It is closer to what we could have with libtool 2.4, except that
libtool 2.4 wouldn't need to be patch as it already supports --sysroot by default. Therefore, a large part of the complexity of this patch set could be removed (since we wouldn't have to autoreconfigure libtool itself). However, we'd still need to autoreconfigure all autotools-based packages (see below). I know Lionel has started experimenting with libtool 2.4, as can be seen on
http://lists.gnu.org/archive/html/bug-libtool/2010-09/msg00064.html
Drawbacks of this approach :
*) We need to autoreconfigure *ALL* autotools-based packages. This may
be an issue for two reasons: 1/ speed, 2/ packages for which autoreconfiguration does not work properly (yes those packages should be fixed, but anyway, this may add some work to us). Obviously, as packages will start using libtool 2.4 by default, this shouldn't be a problem, but we all know that it's going to take years before everybody moves to that newer libtool version.
*) The HOST_HOST_DIR thing and the host-libtool-host as it is
implemented today is pretty ugly. I'd rather have a either single libtool installed in $(HOST_DIR), which work for both the autoreconfiguration of host and target packages, or have two libtools installed in $(HOST_DIR), on for host packages (just named 'libtool') and another for target packages (named 'cross-libtool' or something similar).
Conclusion
==========
Therefore, the choice seems to be between :
* A conservative solution, only extending what we have today, with limited changes and impact, but keeping the existing hacks in place.
* A radically different solution, with wider changes, more in line with libtool future, but requiring an autoreconfiguration of all packages that will not have libtool 2.4 by default.
In my opinion, we should settle on a solution before mid-october, in order to merge it before the end of october and give us enough time for testing before the release at the end of november. Considering how much time Martin and Lionel have dedicated to this issue, I think that their respective work deserve some attention.
Thoughts ? Comments ?
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Thanks a lot to Thomas, for having made this summary.
Le mardi 28 septembre 2010 ? 22:36 +0200, Thomas Petazzoni a ?crit :
Quoted 10 lines
Drawbacks of this approach :
*) We need to autoreconfigure *ALL* autotools-based packages. This may
be an issue for two reasons: 1/ speed, 2/ packages for which autoreconfiguration does not work properly (yes those packages should be fixed, but anyway, this may add some work to us). Obviously, as packages will start using libtool 2.4 by default, this shouldn't be a problem, but we all know that it's going to take years before everybody moves to that newer libtool version.
To be correct, autoreconfigure would not apply to all autotools-based packages, but just to autotools-based ones using libtool. For example, the at package which uses autotools, wouldn't require to be autoreconfigured to work properly.
I confess that packages using libtool represent the majority of the packages, and surely the most complex ones to autoreconfigure.
Regards,
--
Lionel Landwerlin
On Tue, Sep 28, 2010 at 11:36 PM, Thomas Petazzoni <[email protected]> wrote:
[...]
Quoted 12 lines
Conclusion
==========Therefore, the choice seems to be between :
?* A conservative solution, only extending what we have today, with ? limited changes and impact, but keeping the existing hacks in place.
?* A radically different solution, with wider changes, more in line ? with libtool future, but requiring an autoreconfiguration of all ? packages that will not have libtool 2.4 by default.
Quoted 7 lines
In my opinion, we should settle on a solution before mid-october, in order to merge it before the end of october and give us enough time for testing before the release at the end of november. Considering how much time Martin and Lionel have dedicated to this issue, I think that their respective work deserve some attention.
Thoughts ? Comments ?
I would go for a conservative solution.
The main reason here is compile speed.
Currently my configuration takes ~1 hour to compile on my 4x3.4GHz machine with DDR3 RAM...
The worst part here is that autoreconfigure and ./configure scripts doesn't scale on multicore CPU :(
So adding a lot more autoreconfigure usage would drastically reduce build time.
Hello Paulius,
Quoted 4 lines · On Wed, 29 Sep 2010 11:32:29 +0300 Paulius Zaleckas…
I would go for a conservative solution.
The main reason here is compile speed.
Currently my configuration takes ~1 hour to compile on my 4x3.4GHz machine with DDR3 RAM...
Are you using internal toolchain or external toolchain ?
Quoted 4 lines
The worst part here is that autoreconfigure and ./configure scripts doesn't scale on multicore CPU :(
So adding a lot more autoreconfigure usage would drastically reduce build time.
Thanks for your feedback.
For what it's worth, I've started to experiment the modification of our package infrastructure in order to use real file targets instead of phony targets for the steps sequencing (download -> extract -> patch -> configure, etc.). This is one of the requirements to be able to do top-level parallel builds, which I think, definitely becomes necessary to parallelize the autoreconf/configure steps of packages. This is however a lot of work, and I don't expect this to be ready anytime soon.
Regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
Quoted 11 lines · Thomas Petazzoni
Hello Paulius,
Quoted 4 lines · On Wed, 29 Sep 2010 11:32:29 +0300 Paulius Zaleckas…
I would go for a conservative solution.
The main reason here is compile speed.
Currently my configuration takes ~1 hour to compile on my 4x3.4GHz machine with DDR3 RAM...Are you using internal toolchain or external toolchain ?
internal
Quoted 28 lines
Quoted 4 lines
The worst part here is that autoreconfigure and ./configure scripts doesn't scale on multicore CPU :(
So adding a lot more autoreconfigure usage would drastically reduce build time.Thanks for your feedback.
For what it's worth, I've started to experiment the modification of our package infrastructure in order to use real file targets instead of phony targets for the steps sequencing (download -> extract -> patch -> configure, etc.). This is one of the requirements to be able to do top-level parallel builds, which I think, definitely becomes necessary to parallelize the autoreconf/configure steps of packages. This is however a lot of work, and I don't expect this to be ready anytime soon.
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
[Buildroot] On internal vs. external toolchains
Quoted 3 lines · On Wed, 29 Sep 2010 14:01:23 +0300 Paulius Zaleckas…
Quoted 1 line
Are you using internal toolchain or external toolchain ?
internal
Then you could probably save ~20-30% of your build time by using an external toolchain (generated either by Buildroot itself, Crosstool-NG or using a ready-to-use CodeSourcery toolchain).
Regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
In the libtool heads up patch that I had sent, I didn't include some of the preliminary work that I've been doing with libtool 2.4. I've been taking the approach of both our methods. I'm autoreconfiguring the packages that don't have issues, and patching the others with a modified Makefile.autotools.inthat recognizes both 1.5 and 2.2 ltmain.sh files and ignores the 2.4 ones.
If I remember correctly, the packages that I've had issues with autoreconfiguring are expat, libusb, and libvorbis. I've not done a full package build yet. I feel that Lionel has spent far more time on his solution, than I have on mine. For me, I discovered the buildroot-libtool.patch issue when I started upgrading some of the packages, and didn't want to keep including a v2.2 patch for every package that needed it. So, I just came up with this fast and easy method to solve this issue. Currently, I'm working on converting the rest of the packages to either autotargets or gentargets, and was planning on looking at incorporating libtool 2.4 after this. That is, if Lionel hasn't already done something by then.
Martin
Quoted 45 lines · Paulius Zaleckas
Quoted 11 lines · Thomas Petazzoni
Hello Paulius,
Quoted 4 lines · On Wed, 29 Sep 2010 11:32:29 +0300 Paulius Zaleckas…
I would go for a conservative solution.
The main reason here is compile speed.
Currently my configuration takes ~1 hour to compile on my 4x3.4GHz machine with DDR3 RAM...Are you using internal toolchain or external toolchain ?
internal
Quoted 1 line
Quoted 1 line
The worst part here is that autoreconfigure and ./configure scripts
doesn't
Quoted 16 lines
Quoted 3 lines
scale on multicore CPU :(
So adding a lot more autoreconfigure usage would drastically reduce build time.Thanks for your feedback.
For what it's worth, I've started to experiment the modification of our package infrastructure in order to use real file targets instead of phony targets for the steps sequencing (download -> extract -> patch -> configure, etc.). This is one of the requirements to be able to do top-level parallel builds, which I think, definitely becomes necessary to parallelize the autoreconf/configure steps of packages. This is however a lot of work, and I don't expect this to be ready anytime soon.
Regards,
Quoted 11 lines
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Quoted 5 lines
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20100929/ccb86115/attachment.html>
Hello,
Here is what I would like to us to do for the next releases :
1) Bump libtool package to 2.2, more and more packages require libtool 2.2, and we're stuck to not autoreconfigure them without 2.2. This is already creating problems to Thomas when trying to bump host libglib/libgtk packages.
2) Eventually integrate some patches to libtool 2.2 to be able to cross compile autoreconfigured packages.
3) When the libtool 2.4 sysroot issue is sorted out, bump to libtool 2.4 and get rid of the patches integrated in 2).
I think 1) is mandatory for 2010.11.
Regards,
--
Lionel Landwerlin
Le mardi 28 septembre 2010 ? 22:36 +0200, Thomas Petazzoni a ?crit :
Quoted 187 lines
Hello,
There has recently been quite some work done around libtool handling in Buildroot, in the form of two proposals done by Martin Banky and Lionel Landwerlin. Those two proposals are quite different and we need to make a choice between them. This e-mail tries to summarize the libtool problem, how it is currently handled in Buildroot, the technical proposals made by Martin and Lionel with their respective advantages and drawbacks. Martin and Lionel are obviously invited to comment on those (and the rest of this e-mail as well).
Libtool problems
================Libtool problems in cross-compilation mode are well-known. The famous http://www.metastatic.org/text/libtool.html lists most of the problems, and some hacks to solve them.
Basically, Libtool does not understand the fact that we may be building a root filesystem in a different place than "/", and keeps looking in
/usr/libfor libraries.It is worth noting that libtool 2.4 has finally introduced a --sysroot argument which should finally solve those problems. However, until libtool 2.4 is picked up by most packages, we will have to live with the libtool problems for quite some time. See
http://lists.gnu.org/archive/html/libtool/2010-09/msg00082.html.Current Buildroot solution
==========================The current Buildroot solution consists in two mechanisms :
*) Apply the
package/buildroot-libtool.patchto packages when
<pkg>_LIBTOOL_PATCH is YES. This is the default value for autotools-based packages.This patch directly modifies the
ltmain.shscript, which will be used to generate the libtool script in the package as configure time.This patch only works with libtool 1.5.x, but a number of packages are now using libtool 2.2.x, so more and more packages are doing <pkg>_LIBTOOL_PATCH=NO to disable the patch. This works for most packages but not for others (example: I'm bumping GTK to 2.20 and it needs a patch for the
ltmain.shscript which has been generated by libtool 2.2.x)*) Modify the .la files installed by Buildroot in order to add
$(STAGING_DIR) at the beginning of the "libdir" paths.Martin Banky proposal
=====================Martin Banky has sent a proposal to improve libtool handling in Buildroot on September 16th. See
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
and the following 4 e-mails from Martin.Basically, Martin's proposal is to continue with the current solution to the libtool problem, but to clean it up a bit and to extend it to libtool 2.2.
This patch set has 3 main components :
*) Simplify the existing
package/buildroot-libtool.patchby removing
useless chunks*) Rename
package/buildroot-libtool.patchto
package/buildroot-libtool-v1.5.patch, add a newpackage/buildroot-libtool-v2.2.patch. Those two patches respectively modify theltmain.shgenerated by libtool 1.5 and 2.2. Then the Autotools infrastructure is modified to apply one patch or the other depending on which libtool version the package is using.*) The remaining modifications remove libtool-related patches in
various packages that have become useless thanks to the previous patches.Advantages of this approach :
*) We're keeping the current solution and only cleanup/extend it to
work with recent libtool versions. This is good from a "conservative" point of view.*) As today, we don't need to autoreconf all autotools-based packages
just to fix the libtool issue.Drawbacks of this approach :
*) We may have similar issues with future versions of libtool, even if
the addition of --sysroot to libtool 2.4 probably means that libtool 1.5 and 2.2 will remain the only two versions of libtool causing problems in widespread use.*) It keeps the current solution, which is a bit ugly: modify scripts
that are generated, and hack the .la files.*) It modifies the .la by adding informations relative to
$(STAGING_DIR), which may cause problems if we want to generate packages (.ipkg) or if we would like to allow users to relocate the Buildroot staging directory.Lionel Landwerlin proposal
==========================Lionel Landwerlin has sent a proposal to improve libtool handling in Buildroot on September, 18th. See
http://lists.busybox.net/pipermail/buildroot/2010-September/037663.html
for the mailing list post.Basically, Lionel's proposal is to patch libtool itself to make it sysroot-capable and then autoreconfigure all autotools-based packages to make them use the fixed libtool version.
Lionel's proposal is relatively complicated because in order for the patch on libtool to take effect, libtool itself needs to be autoreconfigured, which in turn requires libtool. So Lionel has introduced a host-libtool-host package, which is an unmodified libtool, installed in a new $(HOST_HOST_DIR) directory. He later uses that host-libtool-host to autoreconfigure the host-libtool (which has been previously patched to understand sysroot related things). host-libtool-host will be kept for the autoreconfiguration of host packages, while host-libtool is used for the autoreconfiguration of target packages. (If your head hasn't exploded yet, keep reading, otherwise, take a break and come back in 10 minutes). This is what the first patch in Lionel patch set does. The other patches are only minor related changes.
Advantages of this approach :
*) We no longer arbitrarly patch
ltmain.shin every package, with the
need to carry libtool-version-specific patches.*) We no longer need to modify the .la files.
*) The approach seems cleaner.
*) It is closer to what we could have with libtool 2.4, except that
libtool 2.4 wouldn't need to be patch as it already supports --sysroot by default. Therefore, a large part of the complexity of this patch set could be removed (since we wouldn't have to autoreconfigure libtool itself). However, we'd still need to autoreconfigure all autotools-based packages (see below). I know Lionel has started experimenting with libtool 2.4, as can be seen on
http://lists.gnu.org/archive/html/bug-libtool/2010-09/msg00064.htmlDrawbacks of this approach :
*) We need to autoreconfigure *ALL* autotools-based packages. This may
be an issue for two reasons: 1/ speed, 2/ packages for which autoreconfiguration does not work properly (yes those packages should be fixed, but anyway, this may add some work to us). Obviously, as packages will start using libtool 2.4 by default, this shouldn't be a problem, but we all know that it's going to take years before everybody moves to that newer libtool version.*) The
HOST_HOST_DIRthing and the host-libtool-host as it is
implemented today is pretty ugly. I'd rather have a either single libtool installed in $(HOST_DIR), which work for both the autoreconfiguration of host and target packages, or have two libtools installed in $(HOST_DIR), on for host packages (just named 'libtool') and another for target packages (named 'cross-libtool' or something similar).Conclusion
==========Therefore, the choice seems to be between :
* A conservative solution, only extending what we have today, with limited changes and impact, but keeping the existing hacks in place.
* A radically different solution, with wider changes, more in line with libtool future, but requiring an autoreconfiguration of all packages that will not have libtool 2.4 by default.
In my opinion, we should settle on a solution before mid-october, in order to merge it before the end of october and give us enough time for testing before the release at the end of november. Considering how much time Martin and Lionel have dedicated to this issue, I think that their respective work deserve some attention.
Thoughts ? Comments ?
Thomas
Hello,
Quoted 6 lines · On Sun, 03 Oct 2010 15:22:26 +0200 Lionel Landwerlin <[email protected]>…
Here is what I would like to us to do for the next releases :
1) Bump libtool package to 2.2, more and more packages require libtool 2.2, and we're stuck to not autoreconfigure them without 2.2. This is already creating problems to Thomas when trying to bump host libglib/libgtk packages.
1a) Integrate Martin Banky's proposal so that packages using libtool
2.2 can work without the need to autoreconfigure them.
Quoted 7 lines
2) Eventually integrate some patches to libtool 2.2 to be able to cross compile autoreconfigured packages.
3) When the libtool 2.4 sysroot issue is sorted out, bump to libtool 2.4 and get rid of the patches integrated in 2).
I think 1) is mandatory for 2010.11.
And 1a).
So, I would suggest :
*) Peter merges Martin Banky's set of patches on libtool
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
*) Lionel, could you propose a patch that just bumps libtool to 2.2 ?
Lionel, Martin, Peter, what do you think ?
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
If you want to integrate my patches, I would like to submit a new set of patches with some changes in preparation for libtool 2.4, mainly with Makefile.autotools.in. I have it setup now to distinguish between 1.5.x, 2.2.x, and 2.4. Also, in working on converting the packages to either gentargets or autotargets, I've been noticing a lot of packages have their own libtool patch. I would like to go through, and clean them up. I have a question, why can't we upgrade to libtool 2.4? I've been running it now since I first posted the heads up patch, and have used it with the sysroot option turned on without any obvious issues. If the sysroot option is an issue, we could turn it off for now and use the libtool patches until we integrate the sysroot option properly, right? The imagemagick patch set that I posted, was first done with libtool 2.4 and autoreconfigure turned on. I wanted to make sure that it would work properly in this configuration, in preparation of the coming changes. It was then actually posted with autoreconfigure turned off and using the libtool patch. As a side note, I just realized that imagemagick is using libtool 2.2.x, which means that it's incorporation is blocked until either I add it's own libtool patch or we commit my libtool patch set. Sorry about that. If anyone would like me to add the libtool patch to imagemagick, and repost, let me know.
Martin
Quoted 37 lines · Thomas Petazzoni
Hello,
Quoted 6 lines · On Sun, 03 Oct 2010 15:22:26 +0200 Lionel Landwerlin <[email protected]>…
Here is what I would like to us to do for the next releases :
1) Bump libtool package to 2.2, more and more packages require libtool 2.2, and we're stuck to not autoreconfigure them without 2.2. This is already creating problems to Thomas when trying to bump host libglib/libgtk packages.
1a) Integrate Martin Banky's proposal so that packages using libtool
2.2 can work without the need to autoreconfigure them.Quoted 7 lines
2) Eventually integrate some patches to libtool 2.2 to be able to cross compile autoreconfigured packages.
3) When the libtool 2.4 sysroot issue is sorted out, bump to libtool 2.4 and get rid of the patches integrated in 2).
I think 1) is mandatory for 2010.11.
And 1a).
So, I would suggest :
*) Peter merges Martin Banky's set of patches on libtool
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
*) Lionel, could you propose a patch that just bumps libtool to 2.2 ?
Lionel, Martin, Peter, what do you think ?
Thanks,
Quoted 11 lines
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20101003/d697e920/attachment.html>
As long as we keep the following things, we can bump to 2.2 or 2.4 :
* sed pass on .la files
* Add -L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib to LDFLAGS
* patches for encountered libtool versions
I always took the hypothesis that we wanted to get rid of the first 2, and wanted to keep the third only for special cases.
I encountered problem with libtool 2.4 for several packages under the following conditions :
* run autoreconf on the package
* do not apply the libtool patch
* configure the package with --with-sysroot=$(STAGING_DIR) option
Martin, did you have the same setup ?
I guess we're back to the current situation if we don't pass the --with-sysroot option and we apply a libtool patch for the correct libtool version.
Regards,
--
Lionel Landwerlin
Le dimanche 03 octobre 2010 ? 15:36 -0700, Martin Banky a ?crit :
Quoted 91 lines
If you want to integrate my patches, I would like to submit a new set of patches with some changes in preparation for libtool 2.4, mainly with Makefile.autotools.in. I have it setup now to distinguish between 1.5.x, 2.2.x, and 2.4. Also, in working on converting the packages to either gentargets or autotargets, I've been noticing a lot of packages have their own libtool patch. I would like to go through, and clean them up. I have a question, why can't we upgrade to libtool 2.4? I've been running it now since I first posted the heads up patch, and have used it with the sysroot option turned on without any obvious issues. If the sysroot option is an issue, we could turn it off for now and use the libtool patches until we integrate the sysroot option properly, right? The imagemagick patch set that I posted, was first done with libtool 2.4 and autoreconfigure turned on. I wanted to make sure that it would work properly in this configuration, in preparation of the coming changes. It was then actually posted with autoreconfigure turned off and using the libtool patch. As a side note, I just realized that imagemagick is using libtool 2.2.x, which means that it's incorporation is blocked until either I add it's own libtool patch or we commit my libtool patch set. Sorry about that. If anyone would like me to add the libtool patch to imagemagick, and repost, let me know.
Martin
On Sun, Oct 3, 2010 at 7:22 AM, Thomas Petazzoni
<[email protected]> wrote:
Hello,On Sun, 03 Oct 2010 15:22:26 +0200 Lionel Landwerlin <[email protected]> wrote:
> Here is what I would like to us to do for the next releases :
>
> 1) Bump libtool package to 2.2, more and more packages require libtool
> 2.2, and we're stuck to not autoreconfigure them without 2.2. This is
> already creating problems to Thomas when trying to bump host > libglib/libgtk packages.1a) Integrate Martin Banky's proposal so that packages using
libtool
2.2 can work without the need to autoreconfigure them.> 2) Eventually integrate some patches to libtool 2.2 to be able to
> cross compile autoreconfigured packages.
>
> 3) When the libtool 2.4 sysroot issue is sorted out, bump to libtool
> 2.4 and get rid of the patches integrated in 2). >
> I think 1) is mandatory for 2010.11.And 1a).
So, I would suggest :
*) Peter merges Martin Banky's set of patches on libtool
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
*) Lionel, could you propose a patch that just bumps libtool
to 2.2 ?Lionel, Martin, Peter, what do you think ?
Thanks,
Thomas
--
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support.
http://free-electrons.com
_______________________________________________buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Lionel,
On the first three items, won't we always have to keep these until all
of the packages either support autoreconfiguring or start using libtool 2.4? Like I said before, I ran into three packages that wouldn't autoreconfigure, so they would still need the traditional approach. Also, I haven't had a chance to really investigate this, but isn't TARGET_LDFLAGS+=-L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib also needed by gentargets to link to the correct libraries?
As for my setup, the only difference is, I compiled libtool with
--with-sysroot (no directory given), and I don't configure the package with it. libtool gets the sysroot directory from gcc's sysroot, if none is specified. Everything else is the same. What kind of problems did you encounter? Was it just the problem with autoreconfiguring packages?
Martin
Quoted 83 lines · Lionel Landwerlin
As long as we keep the following things, we can bump to 2.2 or 2.4 :
* sed pass on .la files
* Add -L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib to LDFLAGS
* patches for encountered libtool versionsI always took the hypothesis that we wanted to get rid of the first 2, and wanted to keep the third only for special cases.
I encountered problem with libtool 2.4 for several packages under the following conditions :
* run autoreconf on the package
* do not apply the libtool patch
* configure the package with --with-sysroot=$(STAGING_DIR) optionMartin, did you have the same setup ?
I guess we're back to the current situation if we don't pass the --with-sysroot option and we apply a libtool patch for the correct libtool version.
Regards,
--
Lionel LandwerlinLe dimanche 03 octobre 2010 ? 15:36 -0700, Martin Banky a ?crit :
Quoted 57 lines
If you want to integrate my patches, I would like to submit a new set of patches with some changes in preparation for libtool 2.4, mainly with Makefile.autotools.in. I have it setup now to distinguish between 1.5.x, 2.2.x, and 2.4. Also, in working on converting the packages to either gentargets or autotargets, I've been noticing a lot of packages have their own libtool patch. I would like to go through, and clean them up. I have a question, why can't we upgrade to libtool 2.4? I've been running it now since I first posted the heads up patch, and have used it with the sysroot option turned on without any obvious issues. If the sysroot option is an issue, we could turn it off for now and use the libtool patches until we integrate the sysroot option properly, right? The imagemagick patch set that I posted, was first done with libtool 2.4 and autoreconfigure turned on. I wanted to make sure that it would work properly in this configuration, in preparation of the coming changes. It was then actually posted with autoreconfigure turned off and using the libtool patch. As a side note, I just realized that imagemagick is using libtool 2.2.x, which means that it's incorporation is blocked until either I add it's own libtool patch or we commit my libtool patch set. Sorry about that. If anyone would like me to add the libtool patch to imagemagick, and repost, let me know.
Martin
On Sun, Oct 3, 2010 at 7:22 AM, Thomas Petazzoni
<[email protected]> wrote:
Hello,On Sun, 03 Oct 2010 15:22:26 +0200 Lionel Landwerlin <[email protected]> wrote:
> Here is what I would like to us to do for the next releases :
>
> 1) Bump libtool package to 2.2, more and more packages require libtool
> 2.2, and we're stuck to not autoreconfigure them without 2.2. This is
> already creating problems to Thomas when trying to bump host > libglib/libgtk packages.1a) Integrate Martin Banky's proposal so that packages using
libtool
2.2 can work without the need to autoreconfigure them.> 2) Eventually integrate some patches to libtool 2.2 to be able to
> cross compile autoreconfigured packages.
>
> 3) When the libtool 2.4 sysroot issue is sorted out, bump to libtool
> 2.4 and get rid of the patches integrated in 2). >
> I think 1) is mandatory for 2010.11.
Quoted 38 lines
Quoted 7 lines
And 1a).
So, I would suggest :
*) Peter merges Martin Banky's set of patches on libtool
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
Quoted 27 lines
*) Lionel, could you propose a patch that just bumps libtool
to 2.2 ?Lionel, Martin, Peter, what do you think ?
Thanks,
Thomas
--
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support.
http://free-electrons.com
_______________________________________________buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20101003/a3480adb/attachment.html>
The libtool script patch prevents libtool to give a direct path an host system shared lib path.
The -L$(STAGING_DIR)... is added to the top of link arguments to prevent the linker to use the -L/usr/lib outputted by libtool. Finally the sed pass avoids libtool to open the .la files from host system.
So I'm afraid all those items are required even with libtool 2.4 (without sysroot support).
From what I remember, we added the -L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib for a libtool problem (commit efb1d8d3f40281645c178c150d992601c8042c1a).
I'm going to check with your libtool 2.4 sysroot setup.
Regards,
--
Lionel Landwerlin
Quoted 146 lines · Martin Banky
Lionel,
???? On the first three items, won't we always have to keep these until all of the packages either support autoreconfiguring or start using libtool 2.4? Like I said before, I ran into three packages that wouldn't autoreconfigure, so they would still need the traditional approach. Also, I haven't had a chance to really investigate this, but isn't TARGET_LDFLAGS+=-L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib also needed by gentargets to link to the correct libraries?
???? As for my setup, the only difference is, I compiled libtool with --with-sysroot (no directory given), and I don't configure the package with it. libtool gets the sysroot directory from gcc's sysroot, if none is specified. Everything else is the same. What kind of problems did you encounter? Was it just the problem with autoreconfiguring packages?Martin
Quoted 21 lines · Lionel Landwerlin
As long as we keep the following things, we can bump to 2.2 or 2.4 :
? ? ? ?* sed pass on .la files
? ? ? ?* Add -L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib to LDFLAGS ? ? ? ?* patches for encountered libtool versionsI always took the hypothesis that we wanted to get rid of the first 2, and wanted to keep the third only for special cases.
I encountered problem with libtool 2.4 for several packages under the following conditions :
? ? ? ?* run autoreconf on the package
? ? ? ?* do not apply the libtool patch
? ? ? ?* configure the package with --with-sysroot=$(STAGING_DIR) optionMartin, did you have the same setup ?
I guess we're back to the current situation if we don't pass the --with-sysroot option and we apply a libtool patch for the correct libtool version.
Quoted 100 lines
Regards,
--
Lionel LandwerlinLe dimanche 03 octobre 2010 ? 15:36 -0700, Martin Banky a ?crit :
Quoted 92 lines
If you want to integrate my patches, I would like to submit a new set of patches with some changes in preparation for libtool 2.4, mainly with Makefile.autotools.in. I have it setup now to distinguish between 1.5.x, 2.2.x, and 2.4. Also, in working on converting the packages to either gentargets or autotargets, I've been noticing a lot of packages have their own libtool patch. I would like to go through, and clean them up. I have a question, why can't we upgrade to libtool 2.4? I've been running it now since I first posted the heads up patch, and have used it with the sysroot option turned on without any obvious issues. If the sysroot option is an issue, we could turn it off for now and use the libtool patches until we integrate the sysroot option properly, right? The imagemagick patch set that I posted, was first done with libtool 2.4 and autoreconfigure turned on. I wanted to make sure that it would work properly in this configuration, in preparation of the coming changes. It was then actually posted with autoreconfigure turned off and using the libtool patch. As a side note, I just realized that imagemagick is using libtool 2.2.x, which means that it's incorporation is blocked until either I add it's own libtool patch or we commit my libtool patch set. Sorry about that. If anyone would like me to add the libtool patch to imagemagick, and repost, let me know.
Martin
On Sun, Oct 3, 2010 at 7:22 AM, Thomas Petazzoni
<[email protected]> wrote:
? ? ? ? Hello,? ? ? ? On Sun, 03 Oct 2010 15:22:26 +0200
? ? ? ? Lionel Landwerlin <[email protected]> wrote:? ? ? ? > Here is what I would like to us to do for the next ? ? ? ? releases :
? ? ? ? >
? ? ? ? > 1) Bump libtool package to 2.2, more and more packages ? ? ? ? require libtool
? ? ? ? > 2.2, and we're stuck to not autoreconfigure them without ? ? ? ? 2.2. This is
? ? ? ? > already creating problems to Thomas when trying to bump host ? ? ? ? > libglib/libgtk packages.? ? ? ? ?1a) Integrate Martin Banky's proposal so that packages using ? ? ? ? libtool
? ? ? ? ? ? ?2.2 can work without the need to autoreconfigure them.? ? ? ? > 2) Eventually integrate some patches to libtool 2.2 to be ? ? ? ? able to
? ? ? ? > cross compile autoreconfigured packages.
? ? ? ? >
? ? ? ? > 3) When the libtool 2.4 sysroot issue is sorted out, bump to ? ? ? ? libtool
? ? ? ? > 2.4 and get rid of the patches integrated in 2). ? ? ? ? >
? ? ? ? > I think 1) is mandatory for 2010.11.? ? ? ? And 1a).
? ? ? ? So, I would suggest :
? ? ? ? ?*) Peter merges Martin Banky's set of patches on libtool
?http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
? ? ? ? ?*) Lionel, could you propose a patch that just bumps libtool ? ? ? ? to 2.2 ?
? ? ? ? Lionel, Martin, Peter, what do you think ?
? ? ? ? Thanks,
? ? ? ? Thomas
? ? ? ? --
? ? ? ? Thomas Petazzoni, Free Electrons
? ? ? ? Kernel, drivers, real-time and embedded Linux ? ? ? ? development, consulting, training and support. ? ? ? ? http://free-electrons.com
? ? ? ? _______________________________________________? ? ? ? buildroot mailing list
? ? ? ? buildroot at busybox.net
? ? ? ? http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Ok, it didn't work.
As it's probably going to work just as well as what we have now, we could integrate Martin's patches, thus avoiding to block others bumps.
Regards,
--
Lionel Landwerlin
Le lundi 04 octobre 2010 ? 10:09 +0200, Lionel Landwerlin a ?crit :
Quoted 168 lines
The libtool script patch prevents libtool to give a direct path an host system shared lib path.
The -L$(STAGING_DIR)... is added to the top of link arguments to prevent the linker to use the -L/usr/lib outputted by libtool. Finally the sed pass avoids libtool to open the .la files from host system.So I'm afraid all those items are required even with libtool 2.4 (without sysroot support).
From what I remember, we added the -L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib for a libtool problem (commit
efb1d8d3f40281645c178c150d992601c8042c1a).I'm going to check with your libtool 2.4 sysroot setup.
Regards,
--
Lionel LandwerlinQuoted 146 lines · Martin Banky
Lionel,
On the first three items, won't we always have to keep these until all
of the packages either support autoreconfiguring or start using libtool 2.4? Like I said before, I ran into three packages that wouldn't autoreconfigure, so they would still need the traditional approach. Also, I haven't had a chance to really investigate this, but isn't TARGET_LDFLAGS+=-L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib also needed by gentargets to link to the correct libraries?
As for my setup, the only difference is, I compiled libtool with
--with-sysroot (no directory given), and I don't configure the package with it. libtool gets the sysroot directory from gcc's sysroot, if none is specified. Everything else is the same. What kind of problems did you encounter? Was it just the problem with autoreconfiguring packages?Martin
Quoted 21 lines · Lionel Landwerlin
As long as we keep the following things, we can bump to 2.2 or 2.4 :
* sed pass on .la files
* Add -L$(STAGING_DIR)/lib -L$(STAGING_DIR)/usr/lib to LDFLAGS
* patches for encountered libtool versionsI always took the hypothesis that we wanted to get rid of the first 2, and wanted to keep the third only for special cases.
I encountered problem with libtool 2.4 for several packages under the following conditions :
* run autoreconf on the package
* do not apply the libtool patch
* configure the package with --with-sysroot=$(STAGING_DIR) optionMartin, did you have the same setup ?
I guess we're back to the current situation if we don't pass the --with-sysroot option and we apply a libtool patch for the correct libtool version.
Quoted 100 lines
Regards,
--
Lionel LandwerlinLe dimanche 03 octobre 2010 ? 15:36 -0700, Martin Banky a ?crit :
Quoted 92 lines
If you want to integrate my patches, I would like to submit a new set of patches with some changes in preparation for libtool 2.4, mainly with Makefile.autotools.in. I have it setup now to distinguish between 1.5.x, 2.2.x, and 2.4. Also, in working on converting the packages to either gentargets or autotargets, I've been noticing a lot of packages have their own libtool patch. I would like to go through, and clean them up. I have a question, why can't we upgrade to libtool 2.4? I've been running it now since I first posted the heads up patch, and have used it with the sysroot option turned on without any obvious issues. If the sysroot option is an issue, we could turn it off for now and use the libtool patches until we integrate the sysroot option properly, right? The imagemagick patch set that I posted, was first done with libtool 2.4 and autoreconfigure turned on. I wanted to make sure that it would work properly in this configuration, in preparation of the coming changes. It was then actually posted with autoreconfigure turned off and using the libtool patch. As a side note, I just realized that imagemagick is using libtool 2.2.x, which means that it's incorporation is blocked until either I add it's own libtool patch or we commit my libtool patch set. Sorry about that. If anyone would like me to add the libtool patch to imagemagick, and repost, let me know.
Martin
On Sun, Oct 3, 2010 at 7:22 AM, Thomas Petazzoni
<[email protected]> wrote:
Hello,On Sun, 03 Oct 2010 15:22:26 +0200 Lionel Landwerlin <[email protected]> wrote:
> Here is what I would like to us to do for the next releases :
>
> 1) Bump libtool package to 2.2, more and more packages require libtool
> 2.2, and we're stuck to not autoreconfigure them without 2.2. This is
> already creating problems to Thomas when trying to bump host > libglib/libgtk packages.1a) Integrate Martin Banky's proposal so that packages using
libtool
2.2 can work without the need to autoreconfigure them.> 2) Eventually integrate some patches to libtool 2.2 to be able to
> cross compile autoreconfigured packages.
>
> 3) When the libtool 2.4 sysroot issue is sorted out, bump to libtool
> 2.4 and get rid of the patches integrated in 2). >
> I think 1) is mandatory for 2010.11.And 1a).
So, I would suggest :
*) Peter merges Martin Banky's set of patches on libtool
http://lists.busybox.net/pipermail/buildroot/2010-September/037505.html
*) Lionel, could you propose a patch that just bumps libtool
to 2.2 ?Lionel, Martin, Peter, what do you think ?
Thanks,
Thomas
--
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support.
http://free-electrons.com
_______________________________________________buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
[Buildroot] Tentative conclusion on the libtool work
In reply to Thomas PetazzoniQuoted 5 lines · On Tue, 28 Sep 2010 22:36:52 +0200 Thomas Petazzoni…
In my opinion, we should settle on a solution before mid-october, in order to merge it before the end of october and give us enough time for testing before the release at the end of november. Considering how much time Martin and Lionel have dedicated to this issue, I think that their respective work deserve some attention.
From the various comments on this list and discussions on IRC, it seems that the consensus is that :
*) We should merge Martin Banky's work, as it keeps the existing
known-working approach and just extends it to support packages that have used libtool 2.2 to generate their libtool script.
Peter, if you're ok, I can give some build tests to Martin's work and send you a pull request for it.
*) We should upgrade our own libtool package to libtool 2.2, so that
packages such as libgtk2 that needs to be autoreconf'ed *and* need libtool >= 2.2 can actually be autoreconf'ed.
Lionel, would you mind preparing such a patch set ?
Do we agree on this solution ?
Ideally, it'd be nice to merge it within the next few weeks, at least before the end of October, so that we can bump libgtk2 and have all this in 2010.11.
Regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
[Buildroot] Tentative conclusion on the libtool work
Quoted 19 lines · Thomas Petazzoni
Quoted 7 lines · On Tue, 28 Sep 2010 22:36:52 +0200 Thomas Petazzoni…
In my opinion, we should settle on a solution before mid-october, in order to merge it before the end of october and give us enough time for testing before the release at the end of november. Considering how much time Martin and Lionel have dedicated to this issue, I think that their respective work deserve some attention.
From the various comments on this list and discussions on IRC, it seems
that the consensus is that :
?*) We should merge Martin Banky's work, as it keeps the existing ? ?known-working approach and just extends it to support packages that ? ?have used libtool 2.2 to generate their libtool script.
? ?Peter, if you're ok, I can give some build tests to Martin's work ? ?and send you a pull request for it.
Quoted 7 lines
?*) We should upgrade our own libtool package to libtool 2.2, so that ? ?packages such as libgtk2 that needs to be autoreconf'ed *and* need ? ?libtool >= 2.2 can actually be autoreconf'ed.
? ?Lionel, would you mind preparing such a patch set ?
Do we agree on this solution ?
Yep, agree. I will send it tonight.
Quoted 18 lines
Ideally, it'd be nice to merge it within the next few weeks, at least before the end of October, so that we can bump libgtk2 and have all this in 2010.11.
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot