Pull request for branch for-2012.05/perl
The following changes since commit b5fd0f2709cab72f48da12a5804dd8670fffbf6e:
Change the /etc/mtab symlink to use an absolute path (2012-03-19 17:08:13 +0100)
are available in the git repository at:
git://git.free-electrons.com/users/thomas-petazzoni/buildroot.git for-2012.05/perl
Thomas Petazzoni (2):
microperl: fix MICROPERL_INSTALL_TARGET_CMDS
microperl: install host-microperl in $(HOST_DIR)/opt/perl
package/microperl/microperl.mk | 28 ++++++++++++++++++---------- 1 files changed, 18 insertions(+), 10 deletions(-)
Thanks,
Signature · 1 line
Thomas Petazzoni
PATCH 1/2 microperl: fix MICROPERL_INSTALL_TARGET_CMDS
The MICROPERL_INSTALL_TARGET_CMDS used the following construct in a for loop:
[ -d $(@D)/lib/$$j ] && cp -af $(@D)/lib/$$j \ $(TARGET_DIR)/$(MICROPERL_MODS_DIR) ; \ [ -f $(@D)/lib/$$i ] && $(INSTALL) -m 0644 -D $(@D)/lib/$$i \ $(TARGET_DIR)/$(MICROPERL_MODS_DIR)/$$i; \
The problem is that when at the last iteration, the second test (-f) fails, then the whole loop ends with a non-zero error code, and makes aborts the build. This happens for example if the last Perl modules in the list is Time::Local, because such modules are taken care of by the first condition (that copies a complete directory).
By moving to full if statements, we ensure that the return code is zero even if the condition was false.
--- package/microperl/microperl.mk | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-)
package/microperl/microperl.mk +6 -4
@@ -132,10 +132,12 @@ define MICROPERL_INSTALL_TARGET_CMDS $(MICROPERL_BUILD_EXTENSIONS) for i in $(MICROPERL_MODS); do \ j=`echo $$i|cut -d : -f 1` ; \ - [ -d $(@D)/lib/$$j ] && cp -af $(@D)/lib/$$j \ - $(TARGET_DIR)/$(MICROPERL_MODS_DIR) ; \ - [ -f $(@D)/lib/$$i ] && $(INSTALL) -m 0644 -D $(@D)/lib/$$i \ - $(TARGET_DIR)/$(MICROPERL_MODS_DIR)/$$i; \ + if [ -d $(@D)/lib/$$j ] ; then \ + cp -af $(@D)/lib/$$j $(TARGET_DIR)/$(MICROPERL_MODS_DIR) ; \ + fi ; \ + if [ -f $(@D)/lib/$$i ] ; then \ + $(INSTALL) -m 0644 -D $(@D)/lib/$$i $(TARGET_DIR)/$(MICROPERL_MODS_DIR)/$$i; \ + fi ; \ done # Remove test files find $(TARGET_DIR)/$(MICROPERL_MODS_DIR) -type f -name *.t \
Signature · 1 line
1.7.4.1
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
In reply to Thomas PetazzoniThis commit is a tentative at fixing some Perl-related build problems we are having nowadays, due to the fact that microperl now depends on host-microperl. Since the perl binary from host-microperl is installed in $(HOST_DIR)/usr/bin, it gets picked up when we run host tools like host-intltool, which were normally meant to be used against the distribution-provided Perl.
This problem currently causes between a third to half of the failures in our random build tests.
Another, cleaner, option would be to make host-intltool depend on host-microperl, but this would require every user needing intltool to build a host Perl interpreter, even if the real reason for build the host Perl interpreter is only needed for those who want to build the Perl interpreter for the target. This commit is really a test at trying another solution than adding this dependency, to see whether it turns out to be a reasonable trade-off or not. Later developments and improvements in the Perl area may require to switch to a different solution than the one implemented by this patch.
--- package/microperl/microperl.mk | 18 ++++++++++++------ 1 files changed, 12 insertions(+), 6 deletions(-)
package/microperl/microperl.mk +12 -6
@@ -22,10 +22,16 @@ MICROPERL_MODS += constant.pm CGI CGI.pm Carp.pm Exporter.pm overload.pm \ vars.pm warnings.pm warnings/register.pm endif -# Host microperl is actually full-blown perl +# Host microperl is actually full-blown perl. Instead of installing it +# with prefix=$(HOST_DIR)/usr, we install it with +# prefix=$(HOST_DIR)/opt/perl so that this host-microperl is only used +# for building the target microperl. If we install the host-microperl +# in $(HOST_DIR)/usr, it gets picked up by all Perl-based utilities +# (such as intltool) that Buildroot has installed, creating +# incompatibilites with the Perl provided by the distribution. define HOST_MICROPERL_CONFIGURE_CMDS cd $(@D) ; \ - ./Configure -Dcc="$(HOSTCC)" -Dprefix="$(HOST_DIR)/usr" \ + ./Configure -Dcc="$(HOSTCC)" -Dprefix="$(HOST_DIR)/opt/perl/" \ -Dloclibpth='/lib /lib64 /usr/lib /usr/lib64' -des endef
@@ -95,8 +101,8 @@ define MICROPERL_CONFIGURE_CMDS cp -f $(@D)/uconfig.sh $(@D)/config.sh echo "ccname='$(TARGET_CC)'" >>$(@D)/config.sh echo "PERL_CONFIG_SH=true" >>$(@D)/config.sh - cd $(@D) ; $(HOST_DIR)/usr/bin/perl make_patchnum.pl ; \ - $(HOST_DIR)/usr/bin/perl configpm + cd $(@D) ; $(HOST_DIR)/opt/perl/bin/perl make_patchnum.pl ; \ + $(HOST_DIR)/opt/perl/bin/perl configpm endef define MICROPERL_BUILD_CMDS
@@ -111,9 +117,9 @@ endef # Just ignore make_ext.pl warning/errors define MICROPERL_BUILD_EXTENSIONS for i in $(MICROPERL_MODS); do \ - cd $(@D); ln -sf $(HOST_DIR)/usr/bin/perl miniperl; \ + cd $(@D); ln -sf $(HOST_DIR)/opt/perl/bin/perl miniperl; \ PERL5LIB=$(TARGET_DIR)/$(MICROPERL_ARCH_DIR) \ - $(HOST_DIR)/usr/bin/perl make_ext.pl MAKE="$(MAKE)" --nonxs \ + $(HOST_DIR)/opt/perl/bin/perl make_ext.pl MAKE="$(MAKE)" --nonxs \ `echo $$i|sed -e 's/.pm//'`; \ done endef
Signature · 1 line
1.7.4.1
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Le Tue, 27 Mar 2012 09:06:18 +0200,
Thomas Petazzoni <[email protected]> a ?crit :
Quoted 20 lines
This commit is a tentative at fixing some Perl-related build problems we are having nowadays, due to the fact that microperl now depends on host-microperl. Since the perl binary from host-microperl is installed in $(HOST_DIR)/usr/bin, it gets picked up when we run host tools like host-intltool, which were normally meant to be used against the distribution-provided Perl.
This problem currently causes between a third to half of the failures in our random build tests.
Another, cleaner, option would be to make host-intltool depend on host-microperl, but this would require every user needing intltool to build a host Perl interpreter, even if the real reason for build the host Perl interpreter is only needed for those who want to build the Perl interpreter for the target. This commit is really a test at trying another solution than adding this dependency, to see whether it turns out to be a reasonable trade-off or not. Later developments and improvements in the Perl area may require to switch to a different solution than the one implemented by this patch.
Quoted 1 line
Signed-off-by Thomas Petazzoni <[email protected]>
I have tested
http://autobuild.buildroot.org/results/9980055bf8ebc1ea442f5affc3418eb869c4c90d/defconfig
without the patch, in which case we have the host-intltool problem. With this patch, this problem disappears and this defconfig builds until the end with no problem.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 1/2 microperl: fix MICROPERL_INSTALL_TARGET_CMDS
In reply to Thomas PetazzoniQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
Thomas> The MICROPERL_INSTALL_TARGET_CMDS used the following construct in a Thomas> for loop:
Thomas> [ -d $(@D)/lib/$$j ] && cp -af $(@D)/lib/$$j \
Thomas> $(TARGET_DIR)/$(MICROPERL_MODS_DIR) ; \
Thomas> [ -f $(@D)/lib/$$i ] && $(INSTALL) -m 0644 -D $(@D)/lib/$$i \
Thomas> $(TARGET_DIR)/$(MICROPERL_MODS_DIR)/$$i; \
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
In reply to Thomas PetazzoniQuoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
Thomas> This commit is a tentative at fixing some Perl-related build problems Thomas> we are having nowadays, due to the fact that microperl now depends on Thomas> host-microperl. Since the perl binary from host-microperl is installed Thomas> in $(HOST_DIR)/usr/bin, it gets picked up when we run host tools like Thomas> host-intltool, which were normally meant to be used against the Thomas> distribution-provided Perl.
Thanks, but I have fixed it a bit differently instead:
commit d0e5eb281f0e3b323ecb3446c1b16baf7f3baa69
Author: Peter Korsgaard <[email protected]> Date: Tue Mar 27 17:11:36 2012 +0200
libxml-parser-perl: fix host-intltool breakage when microperl is enabled
Microperl will build host-microperl and install it into HOST_DIR/usr/bin, where other packages will pick it up as _CONFIGURE_OPTS / _MAKE_ENV prepends that to the path.
libxml-parser-perl didn't though, so it would still be built against the system perl, causing host-intltool to fail when it would use host-microperl together with libxml-parser-perl if the system perl isn't compatible with host-microperl.
Fix it by using HOST_CONFIGURE_OPTS and ensuring it is built after (host-)microperl if enabled.
Signature · 1 line
Bye, Peter Korsgaard
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Hello Peter,
Le Wed, 28 Mar 2012 08:54:32 +0200,
Peter Korsgaard <[email protected]> a ?crit :
Quoted 5 lines
Thanks, but I have fixed it a bit differently instead:
commit
d0e5eb281f0e3b323ecb3446c1b16baf7f3baa69Author: Peter Korsgaard <[email protected]> Date: Tue Mar 27 17:11:36 2012 +0200
Thanks, yes. As discussed on IRC, I haven't tested your change, but I agree on the principle.
However, I am worried by the case reported by Will Newton in:
From: Will Newton <[email protected]>
To: buildroot at busybox.net
Subject: [Buildroot] Host libxml-parser-perl build issue
Date: Mon, 26 Mar 2012 17:35:29 +0100
In his case, the microperl for the target was not selected, so the host-microperl was not built, and still he was having issues. Apparently, the problem is that host-libxml-parser-perl didn't pick up the libexpat from the $(HOST_DIR). Normally, we build all binaries with a rpath set to $(HOST_DIR)/usr/lib, but in the specific case of Perl modules that use a native library such as libexpat, I am not sure how we are supposed to tell Perl to build such modules with an rpath set. The current workaround of Will is to pass the LD_LIBRARY_PATH environment variable in TARGET_CONFIGURE_OPTS and TARGET_MAKE_ENV, but I don't think this would work for all packages (I remember libtool being confused by a LD_LIBRARY_PATH being set).
Regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
Hi,
Thomas> However, I am worried by the case reported by Will Newton in:
Thomas> From: Will Newton <[email protected]> Thomas> To: buildroot at busybox.net
Thomas> Subject: [Buildroot] Host libxml-parser-perl build issue Thomas> Date: Mon, 26 Mar 2012 17:35:29 +0100
Thomas> In his case, the microperl for the target was not selected, so the Thomas> host-microperl was not built, and still he was having issues.
Yes, I've seen it. I don't really have a good idea about what we can do about it though :/
Signature · 1 line
Bye, Peter Korsgaard
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
On Wed, Mar 28, 2012 at 10:32 AM, Peter Korsgaard <[email protected]> wrote:
Hi Peter,
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
Hi,
?Thomas> However, I am worried by the case reported by Will Newton in:
?Thomas> From: Will Newton <[email protected]> ?Thomas> To: buildroot at busybox.net
?Thomas> Subject: [Buildroot] Host libxml-parser-perl build issue ?Thomas> Date: Mon, 26 Mar 2012 17:35:29 +0100
Quoted 5 lines
?Thomas> In his case, the microperl for the target was not selected, so the ?Thomas> host-microperl was not built, and still he was having issues.
Yes, I've seen it. I don't really have a good idea about what we can do about it though :/
Do you think that setting LD_LIBRARY_PATH in the way I suggested is unsafe? I had some concerns that it might affect the search path for libraries but I believe this is not an issue with a cross linker (according to the ld man page, I haven't checked the code), and it does not break any of the test builds I have done.
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Le Wed, 28 Mar 2012 11:40:00 +0100,
Will Newton <[email protected]> a ?crit :
Quoted 5 lines
Do you think that setting
LD_LIBRARY_PATHin the way I suggested is unsafe? I had some concerns that it might affect the search path for libraries but I believe this is not an issue with a cross linker (according to the ld man page, I haven't checked the code), and it does not break any of the test builds I have done.
In the past, we had the problem that when we were building target packages that required host utilities installed in $(HOST_DIR)/usr/bin, those utilities were not able to find their libraries in $(HOST_DIR)/usr/lib. We tried using LD_LIBRARY_PATH, but libtool used it as the search path for libraries when compiling target stuff. So, we decided to hardcode the correct rpath in all binaries built and installed in $(HOST_DIR)/usr/bin.
So, yes, I believe that adding LD_LIBRARY_PATH is going to cause problems, unless those problems have disappeared for some other reason.
See:
c1b6242fdcf2cff7ebf09fec4cc1be58963e8427
0d1830b07db4ebfd14e77a258de6fb391e57e960
6b939d40f6a29a43277566adc9d4312d49cb3abf
For a history of what we tried on this topic.
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Quoted 20 lines · Thomas Petazzoni
Le Wed, 28 Mar 2012 11:40:00 +0100,
Will Newton <[email protected]> a ?crit :Quoted 5 lines
Do you think that setting
LD_LIBRARY_PATHin the way I suggested is unsafe? I had some concerns that it might affect the search path for libraries but I believe this is not an issue with a cross linker (according to the ld man page, I haven't checked the code), and it does not break any of the test builds I have done.In the past, we had the problem that when we were building target packages that required host utilities installed in $(HOST_DIR)/usr/bin, those utilities were not able to find their libraries in $(HOST_DIR)/usr/lib. We tried using
LD_LIBRARY_PATH, but libtool used it as the search path for libraries when compiling target stuff. So, we decided to hardcode the correct rpath in all binaries built and installed in $(HOST_DIR)/usr/bin.So, yes, I believe that adding
LD_LIBRARY_PATHis going to cause problems, unless those problems have disappeared for some other reason.
Quoted 7 lines
See:
?c1b6242fdcf2cff7ebf09fec4cc1be58963e8427
?0d1830b07db4ebfd14e77a258de6fb391e57e960
?6b939d40f6a29a43277566adc9d4312d49cb3abfFor a history of what we tried on this topic.
If ld handles this explicitly and correctly then this sounds like a libtool bug to me. Do you have a testcase that can reproduce it? I have spent some time trying to make libtool break but without success, but I am far from an expert in that area...
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
In reply to Thomas PetazzoniLe Wed, 28 Mar 2012 12:55:47 +0200,
Thomas Petazzoni <[email protected]> a ?crit :
Quoted 7 lines
In the past, we had the problem that when we were building target packages that required host utilities installed in $(HOST_DIR)/usr/bin, those utilities were not able to find their libraries in $(HOST_DIR)/usr/lib. We tried using
LD_LIBRARY_PATH, but libtool used it as the search path for libraries when compiling target stuff. So, we decided to hardcode the correct rpath in all binaries built and installed in $(HOST_DIR)/usr/bin.
I had a quick look at this, and here the Perl dynamic module that depends on libexpat.so.1 is built with the correct RPATH:
$ arm-linux-gnueabi-readelf -a host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH 0x000000000000000f (RPATH) Library rpath: [/opt/outputs/perl/host/usr/lib]
So normally, when this Expat.so library is dlopen'ed by Perl, it should use the RPATH of Expat.so to find where libexpat.so.1 is located. And on my system, it does:
$ PERLLIB=/opt/outputs/perl/host/usr/lib/perl/ LD_DEBUG=libs perl -e 'require XML::Parser;'
9529: find library=libperl.so.5.10 [0]; searching
9529: search cache=/etc/ld.so.cache
9529: trying file=/usr/lib/libperl.so.5.10
9529:
9529: find library=libdl.so.2 [0]; searching
9529: search cache=/etc/ld.so.cache
9529: trying file=/lib/x86_64-linux-gnu/libdl.so.2
9529:
9529: find library=libm.so.6 [0]; searching
9529: search cache=/etc/ld.so.cache
9529: trying file=/lib/x86_64-linux-gnu/libm.so.6
9529:
9529: find library=libpthread.so.0 [0]; searching
9529: search cache=/etc/ld.so.cache
9529: trying file=/lib/x86_64-linux-gnu/libpthread.so.0
9529:
9529: find library=libc.so.6 [0]; searching
9529: search cache=/etc/ld.so.cache
9529: trying file=/lib/x86_64-linux-gnu/libc.so.6
9529:
9529: find library=libcrypt.so.1 [0]; searching
9529: search cache=/etc/ld.so.cache
9529: trying file=/lib/x86_64-linux-gnu/libcrypt.so.1
9529:
9529:
9529: calling init: /lib/x86_64-linux-gnu/libpthread.so.0
9529: calling init: /lib/x86_64-linux-gnu/libc.so.6
9529: calling init: /lib/x86_64-linux-gnu/libcrypt.so.1
9529: calling init: /lib/x86_64-linux-gnu/libm.so.6
9529: calling init: /lib/x86_64-linux-gnu/libdl.so.2
9529: calling init: /usr/lib/libperl.so.5.10
9529:
9529:
9529: initialize program: perl
9529: transferring control: perl
9529:
9529: find library=libexpat.so.1 [0]; searching
9529: search path=/opt/outputs/perl/host/usr/lib/tls/x86_64:/opt/outputs/perl/host/usr/lib/tls:/opt/outputs/perl/host/usr/lib/x86_64:/opt/outputs/perl/host/usr/lib (RPATH from file /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so)
9529: trying file=/opt/outputs/perl/host/usr/lib/tls/x86_64/libexpat.so.1
9529: trying file=/opt/outputs/perl/host/usr/lib/tls/libexpat.so.1
9529: trying file=/opt/outputs/perl/host/usr/lib/x86_64/libexpat.so.1
9529: trying file=/opt/outputs/perl/host/usr/lib/libexpat.so.1
9529:
9529: calling init: /opt/outputs/perl/host/usr/lib/libexpat.so.1
9529: calling init: /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so
9529: calling fini: perl [0]
9529: calling fini: /usr/lib/libperl.so.5.10 [0]
9529: calling fini: /lib/x86_64-linux-gnu/libdl.so.2 [0]
9529: calling fini: /lib/x86_64-linux-gnu/libm.so.6 [0]
9529: calling fini: /lib/x86_64-linux-gnu/libpthread.so.0 [0]
9529: calling fini: /lib/x86_64-linux-gnu/libcrypt.so.1 [0]
9529: calling fini: /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so [0]
9529: calling fini: /opt/outputs/perl/host/usr/lib/libexpat.so.1 [0]
9529: calling fini: /lib/x86_64-linux-gnu/libc.so.6 [0]As you can see here, it clearly looks in the RPATH encoded into Expat.so to find where libexpat.so.1 is located. In my
case /opt/outputs/perl/ in the Buildroot output directory, and it clearly uses the libexpat.so.1 from this output directory, and not the one from my system.
Are you sure you don't have any value set in your LD_LIBRARY_PATH environment variable (outside of Buildroot)? Can you try the commands above (check the RPATH of Expat.so and check the resolution of libraries when asking Perl to require the XML::Parser module, when PERLLIB points to the Perl modules built by Buildroot), and post the results?
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Quoted 83 lines · Thomas Petazzoni
Le Wed, 28 Mar 2012 12:55:47 +0200,
Thomas Petazzoni <[email protected]> a ?crit :Quoted 7 lines
In the past, we had the problem that when we were building target packages that required host utilities installed in $(HOST_DIR)/usr/bin, those utilities were not able to find their libraries in $(HOST_DIR)/usr/lib. We tried using
LD_LIBRARY_PATH, but libtool used it as the search path for libraries when compiling target stuff. So, we decided to hardcode the correct rpath in all binaries built and installed in $(HOST_DIR)/usr/bin.I had a quick look at this, and here the Perl dynamic module that depends on libexpat.so.1 is built with the correct RPATH:
$ arm-linux-gnueabi-readelf -a host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH ?0x000000000000000f (RPATH) ? ? ? ? ? ? ?Library rpath: [/opt/outputs/perl/host/usr/lib]So normally, when this Expat.so library is dlopen'ed by Perl, it should use the RPATH of Expat.so to find where libexpat.so.1 is located. And on my system, it does:
$ PERLLIB=/opt/outputs/perl/host/usr/lib/perl/ LD_DEBUG=libs perl -e 'require XML::Parser;' ? ? ?9529: ? ? find library=libperl.so.5.10 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/usr/lib/libperl.so.5.10 ? ? ?9529: ? ? ?9529: ? ? find library=libdl.so.2 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libdl.so.2 ? ? ?9529: ? ? ?9529: ? ? find library=libm.so.6 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libm.so.6 ? ? ?9529: ? ? ?9529: ? ? find library=libpthread.so.0 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libpthread.so.0 ? ? ?9529: ? ? ?9529: ? ? find library=libc.so.6 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libc.so.6 ? ? ?9529: ? ? ?9529: ? ? find library=libcrypt.so.1 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libcrypt.so.1 ? ? ?9529: ? ? ?9529: ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libpthread.so.0 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libc.so.6 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libcrypt.so.1 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libm.so.6 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libdl.so.2 ? ? ?9529: ? ? calling init: /usr/lib/libperl.so.5.10 ? ? ?9529: ? ? ?9529: ? ? ?9529: ? ? initialize program: perl ? ? ?9529: ? ? transferring control: perl ? ? ?9529: ? ? ?9529: ? ? find library=libexpat.so.1 [0]; searching ? ? ?9529: ? ? ?search path=/opt/outputs/perl/host/usr/lib/tls/x86_64:/opt/outputs/perl/host/usr/lib/tls:/opt/outputs/perl/host/usr/lib/x86_64:/opt/outputs/perl/host/usr/lib ? ? ? ? ?(RPATH from file /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so) ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/tls/x86_64/libexpat.so.1 ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/tls/libexpat.so.1 ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/x86_64/libexpat.so.1 ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/libexpat.so.1 ? ? ?9529: ? ? ?9529: ? ? calling init: /opt/outputs/perl/host/usr/lib/libexpat.so.1 ? ? ?9529: ? ? calling init: /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so ? ? ?9529: ? ? calling fini: perl [0] ? ? ?9529: ? ? calling fini: /usr/lib/libperl.so.5.10 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libdl.so.2 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libm.so.6 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libpthread.so.0 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libcrypt.so.1 [0] ? ? ?9529: ? ? calling fini: /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so [0] ? ? ?9529: ? ? calling fini: /opt/outputs/perl/host/usr/lib/libexpat.so.1 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libc.so.6 [0]As you can see here, it clearly looks in the RPATH encoded into Expat.so to find where libexpat.so.1 is located. In my
case/opt/outputs/perl/in the Buildroot output directory, and it clearly uses the libexpat.so.1 from this output directory, and not the one from my system.
Quoted 6 lines
Are you sure you don't have any value set in your
LD_LIBRARY_PATHenvironment variable (outside of Buildroot)? Can you try the commands above (check the RPATH of Expat.so and check the resolution of libraries when asking Perl to require theXML::Parsermodule, when PERLLIB points to the Perl modules built by Buildroot), and post the results?
I haven't got any LD_LIBRARY_PATH set, but it looks like there is no rpath set. I wonder if this is some perl Makefile issue.
[win at lemeta01 buildroot]$ env | grep LD_LIBRARY
[win at lemeta01 buildroot]$ ./output/host/usr/bin/arm-linux-readelf -a ./output/host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH [win at lemeta01 buildroot]$
Although the makefiles do mention rpath, but that one does not seem to be getting used either:
[win at lemeta01 host-libxml-parser-perl-2.36]$ grep -ri rpath * Expat/Makefile:CCDLFLAGS = -Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE Makefile:CCDLFLAGS = -Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Quoted 100 lines · Will Newton
Quoted 83 lines · Thomas Petazzoni
Le Wed, 28 Mar 2012 12:55:47 +0200,
Thomas Petazzoni <[email protected]> a ?crit :Quoted 7 lines
In the past, we had the problem that when we were building target packages that required host utilities installed in $(HOST_DIR)/usr/bin, those utilities were not able to find their libraries in $(HOST_DIR)/usr/lib. We tried using
LD_LIBRARY_PATH, but libtool used it as the search path for libraries when compiling target stuff. So, we decided to hardcode the correct rpath in all binaries built and installed in $(HOST_DIR)/usr/bin.I had a quick look at this, and here the Perl dynamic module that depends on libexpat.so.1 is built with the correct RPATH:
$ arm-linux-gnueabi-readelf -a host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH ?0x000000000000000f (RPATH) ? ? ? ? ? ? ?Library rpath: [/opt/outputs/perl/host/usr/lib]So normally, when this Expat.so library is dlopen'ed by Perl, it should use the RPATH of Expat.so to find where libexpat.so.1 is located. And on my system, it does:
$ PERLLIB=/opt/outputs/perl/host/usr/lib/perl/ LD_DEBUG=libs perl -e 'require XML::Parser;' ? ? ?9529: ? ? find library=libperl.so.5.10 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/usr/lib/libperl.so.5.10 ? ? ?9529: ? ? ?9529: ? ? find library=libdl.so.2 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libdl.so.2 ? ? ?9529: ? ? ?9529: ? ? find library=libm.so.6 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libm.so.6 ? ? ?9529: ? ? ?9529: ? ? find library=libpthread.so.0 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libpthread.so.0 ? ? ?9529: ? ? ?9529: ? ? find library=libc.so.6 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libc.so.6 ? ? ?9529: ? ? ?9529: ? ? find library=libcrypt.so.1 [0]; searching ? ? ?9529: ? ? ?search cache=/etc/ld.so.cache ? ? ?9529: ? ? ? trying file=/lib/x86_64-linux-gnu/libcrypt.so.1 ? ? ?9529: ? ? ?9529: ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libpthread.so.0 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libc.so.6 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libcrypt.so.1 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libm.so.6 ? ? ?9529: ? ? calling init: /lib/x86_64-linux-gnu/libdl.so.2 ? ? ?9529: ? ? calling init: /usr/lib/libperl.so.5.10 ? ? ?9529: ? ? ?9529: ? ? ?9529: ? ? initialize program: perl ? ? ?9529: ? ? transferring control: perl ? ? ?9529: ? ? ?9529: ? ? find library=libexpat.so.1 [0]; searching ? ? ?9529: ? ? ?search path=/opt/outputs/perl/host/usr/lib/tls/x86_64:/opt/outputs/perl/host/usr/lib/tls:/opt/outputs/perl/host/usr/lib/x86_64:/opt/outputs/perl/host/usr/lib ? ? ? ? ?(RPATH from file /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so) ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/tls/x86_64/libexpat.so.1 ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/tls/libexpat.so.1 ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/x86_64/libexpat.so.1 ? ? ?9529: ? ? ? trying file=/opt/outputs/perl/host/usr/lib/libexpat.so.1 ? ? ?9529: ? ? ?9529: ? ? calling init: /opt/outputs/perl/host/usr/lib/libexpat.so.1 ? ? ?9529: ? ? calling init: /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so ? ? ?9529: ? ? calling fini: perl [0] ? ? ?9529: ? ? calling fini: /usr/lib/libperl.so.5.10 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libdl.so.2 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libm.so.6 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libpthread.so.0 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libcrypt.so.1 [0] ? ? ?9529: ? ? calling fini: /opt/outputs/perl/host/usr/lib/perl//auto/XML/Parser/Expat/Expat.so [0] ? ? ?9529: ? ? calling fini: /opt/outputs/perl/host/usr/lib/libexpat.so.1 [0] ? ? ?9529: ? ? calling fini: /lib/x86_64-linux-gnu/libc.so.6 [0]As you can see here, it clearly looks in the RPATH encoded into Expat.so to find where libexpat.so.1 is located. In my
case/opt/outputs/perl/in the Buildroot output directory, and it clearly uses the libexpat.so.1 from this output directory, and not the one from my system.Quoted 6 lines
Are you sure you don't have any value set in your
LD_LIBRARY_PATHenvironment variable (outside of Buildroot)? Can you try the commands above (check the RPATH of Expat.so and check the resolution of libraries when asking Perl to require theXML::Parsermodule, when PERLLIB points to the Perl modules built by Buildroot), and post the results?I haven't got any
LD_LIBRARY_PATHset, but it looks like there is no rpath set. I wonder if this is some perlMakefileissue.[win at lemeta01 buildroot]$ env | grep
LD_LIBRARY
[win at lemeta01 buildroot]$ ./output/host/usr/bin/arm-linux-readelf -a ./output/host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH [win at lemeta01 buildroot]$
Quoted 8 lines
Although the makefiles do mention rpath, but that one does not seem to be getting used either:
[win at lemeta01 host-libxml-parser-perl-2.36]$ grep -ri rpath * Expat/Makefile:CCDLFLAGS = -Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE Makefile:CCDLFLAGS = -Wl,-E -Wl,-rpath,/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE
However it does use LD_RUN_PATH:
[win at lemeta01 host-libxml-parser-perl-2.36]$ grep -r LD_RUN_PATH * Expat/Makefile:LD_RUN_PATH = /meta/home/win/buildroot/output/host/usr/lib
If I change that line to "export LD_RUN_PATH..." it seems to set the rpath and work correctly...
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
In reply to Will NewtonLe Wed, 28 Mar 2012 14:01:56 +0100,
Will Newton <[email protected]> a ?crit :
Quoted 4 lines
[win at lemeta01 buildroot]$ env | grep
LD_LIBRARY
[win at lemeta01 buildroot]$ ./output/host/usr/bin/arm-linux-readelf -a ./output/host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH [win at lemeta01 buildroot]$
Ah, ah, this is our problem.
Quoted 2 lines
Although the makefiles do mention rpath, but that one does not seem to be getting used either:
We have a rpath set in HOST_LDFLAGS, but this variable doesn't seem to be used when building host-libxml-parser-perl. However, by Expat/Makefile contains LD_RUN_PATH="/opt/outputs/perl/host/usr/lib" in the environment when calling ld:
LD_RUN_PATH="/opt/outputs/perl/host/usr/lib" cc -shared -O2 -g -L/usr/local/lib -fstack-protector Expat.o -o ../blib/arch/auto/XML/Parser/Expat/Expat.so \
-L/opt/outputs/perl/host/usr/lib -lexpat
And LD_RUN_PATH is used by ld as the RPATH when no -rpath option is used. I guess this LD_RUN_PATH is set because we run the Makefile.PL script with INSTALLSITELIB=/opt/outputs/perl/host/usr/lib/perl.
Could you post the output of:
rm -rf output/build/host-libxml-parser-perl*
make
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Quoted 21 lines · Thomas Petazzoni
Le Wed, 28 Mar 2012 14:01:56 +0100,
Will Newton <[email protected]> a ?crit :Quoted 4 lines
[win at lemeta01 buildroot]$ env | grep
LD_LIBRARY
[win at lemeta01 buildroot]$ ./output/host/usr/bin/arm-linux-readelf -a ./output/host/usr/lib/perl/auto/XML/Parser/Expat/Expat.so | grep RPATH [win at lemeta01 buildroot]$Ah, ah, this is our problem.
Quoted 2 lines
Although the makefiles do mention rpath, but that one does not seem to be getting used either:
We have a rpath set in
HOST_LDFLAGS, but this variable doesn't seem to be used when building host-libxml-parser-perl. However, by Expat/Makefile contains LD_RUN_PATH="/opt/outputs/perl/host/usr/lib" in the environment when calling ld:LD_RUN_PATH="/opt/outputs/perl/host/usr/lib" cc ?-shared -O2 -g -L/usr/local/lib -fstack-protector Expat.o ?-o ../blib/arch/auto/XML/Parser/Expat/Expat.so ? ? ?\
? ? ? ? ? -L/opt/outputs/perl/host/usr/lib -lexpat
Quoted 8 lines
And
LD_RUN_PATHis used by ld as the RPATH when no -rpath option is used. I guess thisLD_RUN_PATHis set because we run the Makefile.PL script with INSTALLSITELIB=/opt/outputs/perl/host/usr/lib/perl.Could you post the output of:
rm -rf output/build/host-libxml-parser-perl*
make
I think I may have a solution. It looks like Red Hat (and hence CentOS) remove support for LD_RUN_PATH by default:
https://bugzilla.redhat.com/show_bug.cgi?id=136009
However there is a flag to turn it back on. Setting this seems to get me a sane runpath. Does this look ok? I assume it will silently do nothing on non-RH systems.
PATCH 2/2 microperl: install host-microperl in $(HOST_DIR)/opt/perl
Le Wed, 28 Mar 2012 15:00:36 +0100,
Will Newton <[email protected]> a ?crit :
Quoted 8 lines
I think I may have a solution. It looks like Red Hat (and hence CentOS) remove support for
LD_RUN_PATHby default:https://bugzilla.redhat.com/show_bug.cgi?id=136009
However there is a flag to turn it back on. Setting this seems to get me a sane runpath. Does this look ok? I assume it will silently do nothing on non-RH systems.
Ah, ah, it looks like a very good solution indeed.
Thanks for the time you have spent investigating this problem!
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com