Fix TARGET_PATH variable with external toolchain
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Some time ago, an external toolchain improvement patch added $(HOST_DIR)/bin and $(HOST_DIR)/usr/bin to the TARGET_PATH variable. This patch also removed $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin to TARGET_PATH which breaks the build of the WebKit package for example.
WebKit depends on the icu package, and when configuring the package, there is a need to run icu-config which is installed inside the staging directory.
--- package/Makefile.in | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
package/Makefile.in +1 -1
@@ -119,7 +119,7 @@ else TOOLCHAIN_EXTERNAL_PREFIX:=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PREFIX)) TOOLCHAIN_EXTERNAL_PATH:=$(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_PATH)) TOOLCHAIN_DIR=$(BASE_DIR)/toolchain -TARGET_PATH="$(HOST_DIR)/bin:$(HOST_DIR)/usr/bin:$(TOOLCHAIN_DIR)/bin:$(TOOLCHAIN_EXTERNAL_PATH)/bin:$(PATH)" +TARGET_PATH="$(HOST_DIR)/bin:$(HOST_DIR)/usr/bin:$(TOOLCHAIN_DIR)/bin:$(TOOLCHAIN_EXTERNAL_PATH)/bin:$(STAGING_DIR)/bin:$(STAGING_DIR)/usr/bin:$(PATH)" #IMAGE:=$(BINARIES_DIR)/$(BR2_ROOTFS_PREFIX).$(TOOLCHAIN_EXTERNAL_PREFIX)$(ROOTFS_SUFFIX) IMAGE:=$(BINARIES_DIR)/$(BR2_ROOTFS_PREFIX).$(ARCH)$(ROOTFS_SUFFIX)
Signature · 1 line
1.6.4.3
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Lionel" == Lionel Landwerlin <[email protected]> writes:
Lionel> Some time ago, an external toolchain improvement patch added Lionel> $(HOST_DIR)/bin and $(HOST_DIR)/usr/bin to the TARGET_PATH variable. Lionel> This patch also removed $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin Lionel> to TARGET_PATH which breaks the build of the WebKit package for Lionel> example.
Lionel> WebKit depends on the icu package, and when configuring the package, Lionel> there is a need to run icu-config which is installed inside the Lionel> staging directory.
I don't like this, as it mixes binaries/scripts for the host and the target - I think it's better for the icu package to install the host tool into host/usr/bin as well instead.
Signature · 1 line
Bye, Peter Korsgaard
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Le samedi 24 octobre 2009 ? 20:37 +0200, Peter Korsgaard a ?crit :
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Lionel" == Lionel Landwerlin <[email protected]> writes:
Lionel> Some time ago, an external toolchain improvement patch added Lionel> $(HOST_DIR)/bin and $(HOST_DIR)/usr/bin to the
TARGET_PATHvariable. Lionel> This patch also removed $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin Lionel> toTARGET_PATHwhich breaks the build of the WebKit package for Lionel> example.
Quoted 8 lines
Lionel> WebKit depends on the icu package, and when configuring the package, Lionel> there is a need to run icu-config which is installed inside the Lionel> staging directory.
I don't like this, as it mixes binaries/scripts for the host and the target - I think it's better for the icu package to install the host tool into host/usr/bin as well instead.
First let me argue that when buildroot is not using an external toolchain, the $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin paths are a part of the TARGET_PATH variable.
But anyway, you're right, mixing binaries sucks.
Then, in this case, I don't think we can use the host script.
Let's right icu-config on my system :
djdeath at coalu:~$ icu-config --ldflags
-lm -L/usr/lib -licui18n -licuuc -licudata -lm
Same thing inside the staging directory :
djdeath at coalu:~/src/buildroot/buildroot_rebase$ ./output/staging/usr/bin/icu-config --ldflags -lpthread -lm -L/home/djdeath/src/buildroot/buildroot_rebase/output/staging/usr/lib -licui18n -licuuc -licudata -lpthread -lm
The script returns paths relative to the host. This is going to be a problem when linking the webkit binaries because the linker will have to deal with host binaries... That's the same kind of problem we are fixing with the libtool patch.
I'm wondering whether adding the following to package/webkit/webkit.mk should fix the problem :
WEBKIT_CONF_ENV = icu_config=$(STAGING_DIR)/usr/bin/icu-config
This way the configure script should use the pre-filled variable and not try to detect the icu-config's path. And we don't need to add the staging bin directories to the TARGET_PATH.
A long term fix could be to use pkg-config instead of self made tool. Patching/contributing to libicu is required.
Signature · 1 line
Lionel Landwerlin <[email protected]>
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Lionel" == Lionel Landwerlin <[email protected]> writes:
Lionel> First let me argue that when buildroot is not using an external Lionel> toolchain, the $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin Lionel> paths are a part of the TARGET_PATH variable.
Hmm, you're right - what a mess. There's no good reason why these should be different in the regard.
Lionel> The script returns paths relative to the host. This is going to Lionel> be a problem when linking the webkit binaries because the Lionel> linker will have to deal with host binaries... That's the same Lionel> kind of problem we are fixing with the libtool patch.
Yes, but copying the script from staging to host shouldn't change that.
Lionel> I'm wondering whether adding the following to package/webkit/webkit.mk Lionel> should fix the problem :
Lionel> WEBKIT_CONF_ENV = icu_config=$(STAGING_DIR)/usr/bin/icu-config
If the webkit configure script handles that (didn't check), then that would certainly be the easiest/cleanest solution.
Signature · 1 line
Bye, Peter Korsgaard
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Le samedi 24 octobre 2009 ? 22:22 +0200, Peter Korsgaard a ?crit :
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Lionel" == Lionel Landwerlin <[email protected]> writes:
Lionel> First let me argue that when buildroot is not using an external Lionel> toolchain, the $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin Lionel> paths are a part of the
TARGET_PATHvariable.Hmm, you're right - what a mess. There's no good reason why these should be different in the regard.
Quoted 6 lines
Lionel> The script returns paths relative to the host. This is going to Lionel> be a problem when linking the webkit binaries because the Lionel> linker will have to deal with host binaries... That's the same Lionel> kind of problem we are fixing with the libtool patch.
Yes, but copying the script from staging to host shouldn't change that.
Yeah, that an idea. I don't which one is the cleanest.
Quoted 9 lines
Lionel> I'm wondering whether adding the following to
package/webkit/webkit.mkLionel> should fix the problem :Lionel>
WEBKIT_CONF_ENV= icu_config=$(STAGING_DIR)/usr/bin/icu-configIf the webkit configure script handles that (didn't check), then that would certainly be the easiest/cleanest solution.
Ok, I just tested it and it works.
So the next step would probably be to remove $(STAGING_DIR)/bin and $(STAGING_DIR)/usr/bin from the TARGET_PATH with source built toolchains, but I'm pretty sure it's going to break some other packages.
Please find a patch as attachment.
Regards,
Signature · 9 lines
Lionel Landwerlin <[email protected]> -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-package-Set-icu-config-path.patch Type: text/x-patch Size: 748 bytes Desc: not available URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20091025/02848f92/attachment.bin>
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
In reply to Peter KorsgaardLe Sat, 24 Oct 2009 20:37:57 +0200,
Peter Korsgaard <[email protected]> a ?crit :
Quoted 6 lines
Lionel> Some time ago, an external toolchain improvement patch added Lionel> $(HOST_DIR)/bin and $(HOST_DIR)/usr/bin to the
TARGET_PATHLionel> variable. This patch also removed $(STAGING_DIR)/bin and Lionel> $(STAGING_DIR)/usr/bin toTARGET_PATHwhich breaks the build Lionel> of the WebKit package for example.
Quoted 7 lines
Lionel> WebKit depends on the icu package, and when configuring the Lionel> package, there is a need to run icu-config which is Lionel> installed inside the staging directory.
I don't like this, as it mixes binaries/scripts for the host and the target - I think it's better for the icu package to install the host tool into host/usr/bin as well instead.
As part of my work on the package infrastructure, I converted icu over to the new format.
At first, I'd agree with you that icu-config, being executed on the host, should theorically be installed in $(HOST_DIR). However, this raises an issue : icu is compiled twice, once for the host, once for the target.
In the current package/icu/icu.mk, the host icu is not installed at all, but my package infrastructure work converts it to a more normal package where the host icu is installed in $(HOST_DIR). Therefore, the host icu installs its own icu-config in $(HOST_DIR)/usr/bin.
Then, we build the target icu, which also installs its own icu-config, in $(STAGING_DIR)/usr/bin. It is not possible to install this icu-config into $(HOST_DIR)/usr/bin, otherwise it would overwrite the host-icu one.
Therefore, I think we have no other choice than leaving the *-config stuff into $(STAGING_DIR)/usr/bin and then finding some trick to get them executed even if $(STAGING_DIR)/usr/bin is not in the PATH.
What do you think about this ?
Sincerly,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Le mardi 27 octobre 2009 ? 08:44 +0100, Thomas Petazzoni a ?crit :
Quoted 35 lines
Le Sat, 24 Oct 2009 20:37:57 +0200,
Peter Korsgaard <[email protected]> a ?crit :Quoted 6 lines
Lionel> Some time ago, an external toolchain improvement patch added Lionel> $(HOST_DIR)/bin and $(HOST_DIR)/usr/bin to the
TARGET_PATHLionel> variable. This patch also removed $(STAGING_DIR)/bin and Lionel> $(STAGING_DIR)/usr/bin toTARGET_PATHwhich breaks the build Lionel> of the WebKit package for example.Quoted 7 lines
Lionel> WebKit depends on the icu package, and when configuring the Lionel> package, there is a need to run icu-config which is Lionel> installed inside the staging directory.
I don't like this, as it mixes binaries/scripts for the host and the target - I think it's better for the icu package to install the host tool into host/usr/bin as well instead.
As part of my work on the package infrastructure, I converted icu over to the new format.
At first, I'd agree with you that icu-config, being executed on the host, should theorically be installed in $(HOST_DIR). However, this raises an issue : icu is compiled twice, once for the host, once for the target.
In the current
package/icu/icu.mk, the host icu is not installed at all, but my package infrastructure work converts it to a more normal package where the host icu is installed in $(HOST_DIR). Therefore, the host icu installs its own icu-config in $(HOST_DIR)/usr/bin.Then, we build the target icu, which also installs its own icu-config, in $(STAGING_DIR)/usr/bin. It is not possible to install this icu-config into $(HOST_DIR)/usr/bin, otherwise it would overwrite the host-icu one.
Quoted 6 lines
Therefore, I think we have no other choice than leaving the *-config stuff into $(STAGING_DIR)/usr/bin and then finding some trick to get them executed even if $(STAGING_DIR)/usr/bin is not in the PATH.
What do you think about this ?
Sounds right to me.
Let's add
WEBKIT_CONF_ENV = icu_config=$(STAGING_DIR)/usr/bin/icu-config
to the webkit makefile.
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Le Tue, 27 Oct 2009 10:45:07 +0100,
Lionel Landwerlin <[email protected]> a ?crit :
Quoted 7 lines
Sounds right to me.
Let's add
WEBKIT_CONF_ENV= icu_config=$(STAGING_DIR)/usr/bin/icu-configto the webkit makefile.
Sure. But we'll have to remember that this is the solution to apply on all similar packages... for some of which a xxx_config configure options might not be present.
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Le mardi 27 octobre 2009 ? 10:53 +0100, Thomas Petazzoni a ?crit :
Quoted 7 lines
Le Tue, 27 Oct 2009 10:45:07 +0100,
Lionel Landwerlin <[email protected]> a ?crit :Quoted 4 lines
Sounds right to me.
Let's add
Quoted 7 lines
Quoted 3 lines
WEBKIT_CONF_ENV= icu_config=$(STAGING_DIR)/usr/bin/icu-configto the webkit makefile.
Sure. But we'll have to remember that this is the solution to apply on all similar packages... for some of which a
xxx_configconfigure options might not be present.
Right.
In my config I built the following packages :
atk-1.22.0
atk-1.22.0-host
autoconf-2.63-host
automake-1.10-host
buildroot-config
busybox-1.15.1
busybox-1.15.2
cairo-1.6.4
cairo-1.6.4-host
dbus-1.2.16
directfb-1.4.2
directfb-examples-1.2.0
eina-0.0.2.060
enchant-1.5.0
ethtool-6
expat-2.0.1
expat-2.0.1-host
fakeroot-1.9.5
fakeroot-1.9.5-host
fontconfig-2.6.0
fontconfig-2.6.0-host
freetype-2.3.9
freetype-2.3.9-host
gettext-0.16.1
icu
icu-host
iostat-2.2
jpeg-6b
libcurl-7.19.2
libecore-0.9.9.062
libe_dbus-0.5.0.062
libedje-0.9.92.062
libeet-1.2.2
libeina-0.0.2.062
libembryo-0.9.9.062
liberation-fonts-1.04
libevas-0.9.9.062
libglib2-2.20.5
libglib2-2.20.5-host
libgtk2-2.12.12
libgtk2-2.12.12-host
libpng-1.2.38
libsoup-2.26.2
libtool-1.5.24-host
libungif-4.1.4
libxml2-2.7.3
libxslt-1.1.24
lzo-2.03
m4-1.4.9-host
makedevs-hostmtd_orig
openssh-5.1p1
openssl-0.9.8k
pango-1.20.5
pango-1.20.5-host
pixman-0.10.0
pixman-0.10.0-host
pkg-config-0.23-host
sawman-1.4.2
sqlite-3.6.18
strace-4.5.18
sysstat-9.0.5
tiff-3.8.2
udev-114
webkit-r44552
zlib-1.2.3
By the way, is there a set of configuration files that cover the list of package that are known to work with buildroot ?
It could be interesting to setup some kind of non regression builds :)
Signature · 1 line
Lionel Landwerlin <[email protected]>
PATCH 1/1 Fix TARGET_PATH variable with external toolchain
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Lionel" == Lionel Landwerlin <[email protected]> writes:
Hi,
Lionel> By the way, is there a set of configuration files that cover Lionel> the list of package that are known to work with buildroot ?
Not a complete list, no. In concept, we would like all packages to work, but as the number of combinations with 600+ packages / 5+ archs is so high, there's bound to be issues.
Lionel> It could be interesting to setup some kind of non regression builds :)
Certainly. I have a buildbot instance running at where I work to test the configs we're using, and I'm planning on setting up a public buildbot soon.
I recently added make all{yes,no,random}packageconfig targets to make it easier to test random package combinations.
Signature · 1 line
Bye, Peter Korsgaard