perl XS modules
Hi all,
I try to build the libxml-parser-perl package for an ARM target. Unfortunately the compiled perl module fails to load with the following error message:
loadable library and perl binaries are mismatched
After some digging I found that the module needs to be compiled with the same toolchain and compiler defines as the perl binary in order to be ABI compatible. Forcing the defines to be exactly the same as the ones used for the perl binary did not fix the issue so I believe it must be a problem with the toolchain.
I get it right, buildroot builds the host-perl package with the GCC installed on the build system. In my case this is GCC Version 6.1 while the toolchain in buildroot uses 4.9.3. If I get the build procedure right, the ABI for perl is then configured for the 6.1 toolchain while module builds will use the toolchain provided by buildroot which is ABI incompatible.
Is there a way to make buildroot build an initial toolchain to build the host packages with? Did anybody ever successfully build perlXS modules?
Thanks,
/Arne
Hello,
Fran?ois, could you have a look at the below question, that is Perl and cross-compilation related?
Thanks!
Thomas
Quoted 32 lines · Arne Caspari
Hi all,
I try to build the libxml-parser-perl package for an ARM target. Unfortunately the compiled perl module fails to load with the following error message:
loadable library and perl binaries are mismatched
After some digging I found that the module needs to be compiled with the same toolchain and compiler defines as the perl binary in order to be ABI compatible. Forcing the defines to be exactly the same as the ones used for the perl binary did not fix the issue so I believe it must be a problem with the toolchain.
I get it right, buildroot builds the host-perl package with the GCC installed on the build system. In my case this is GCC Version 6.1 while the toolchain in buildroot uses 4.9.3. If I get the build procedure right, the ABI for perl is then configured for the 6.1 toolchain while module builds will use the toolchain provided by buildroot which is ABI incompatible.
Is there a way to make buildroot build an initial toolchain to build the host packages with? Did anybody ever successfully build perlXS modules?
Thanks,
/Arne
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
The BR package libxml-parser-perl is a very specific host package which works only with the the system perl.
(The reason is : this packge is heavy used, and we don't want the cost|time of the host-perl build)
All Perl modules (XS or Pure Perl) use the naming scheme :
perl-module-name.
For example, the wrapper for libxml2 is perl-xml-libxml. All these packages use the BR infrastructure perl-package.
If you want a target package perl-xml-parser (XML expat wrapper), start by running:
$ support/script/scancpan XML-Parser which creates a decent template. After add manually, the specific part required by the expat C library. (And obviously, read https://buildroot.org/downloads/manual/manual.html#_infrastructure_for_perl_cpan_packages )
Fran?ois
2016-07-08 17:40 GMT+02:00 Thomas Petazzoni <
[email protected]>:
Quoted 45 lines
Hello,
Fran?ois, could you have a look at the below question, that is Perl and cross-compilation related?
Thanks!
Thomas
Quoted 25 lines · Arne Caspari
Hi all,
I try to build the libxml-parser-perl package for an ARM target. Unfortunately the compiled perl module fails to load with the following error message:
loadable library and perl binaries are mismatched
After some digging I found that the module needs to be compiled with the same toolchain and compiler defines as the perl binary in order to be ABI compatible. Forcing the defines to be exactly the same as the ones used for the perl binary did not fix the issue so I believe it must be a problem with the toolchain.
I get it right, buildroot builds the host-perl package with the GCC installed on the build system. In my case this is GCC Version 6.1 while the toolchain in buildroot uses 4.9.3. If I get the build procedure right, the ABI for perl is then configured for the 6.1 toolchain while module builds will use the toolchain provided by buildroot which is ABI incompatible.
Is there a way to make buildroot build an initial toolchain to build the host packages with? Did anybody ever successfully build perlXS modules?
Quoted 7 lines
Thanks,
/Arne
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Quoted 5 lines
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20160712/c5cd40bc/attachment.html>
Hello,
Quoted 4 lines · Fran?ois Perrad
The BR package libxml-parser-perl is a very specific host package which works only with the the system perl.
(The reason is : this packge is heavy used, and we don't want the cost|time of the host-perl build)
The question from Arne was not about the host variant of libxml-parser-perl, but about he target variant.
Quoted 4 lines
All Perl modules (XS or Pure Perl) use the naming scheme :
perl-module-name.
For example, the wrapper for libxml2 is perl-xml-libxml. All these packages use the BR infrastructure perl-package.
Right, but Arne seems to indicate that when you build a *target* Perl modules with cross-compiled C code (i.e not a pure Perl module), there is an ABI compatibility issue, because the host perl and the target perl don't use the same ABI.
Arne: can you give a specific Buildroot configuration and commands to test that exhibit the problem?
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
Fran?ois,
Thank you very much for coming back to me. It is highly appreciated.
I did not know about the scancpan-Script - this would have saved me a ton of work, literally since I just ported a package with very very many module dependencies (the Logitech Media Server) to buildroot. Ah well...
I finally found the issue with the perl XS packages though: The perl MakeMaker module fails to generate a correct Makefile but instead messes up the configuration with the host-perl one. I kind of fixed the issue for me in a very hackish way by modifying the generated Makefile with the following script:
define PERL_XML_PARSER_BUILD_CMDS
sed "s~/usr/bin/gcc~$(TARGET_CC) -Os~" -i $(@D)/Makefile sed "s~/usr/bin/gcc~$(TARGET_CC) -Os~" -i $(@D)/Expat/Makefile sed "s~x86_64~$(KERNEL_ARCH)~" -i $(@D)/Expat/Makefile sed "s~host/usr/lib~staging/usr/lib~" -i $(@D)/Expat/Makefile $(MAKE) $(@D)/Makefile -C $(@D)
endef
The module loads correctly afterwards. Unfortunately I do not have any experience with perl or the MakeMaker module to come up with a less insane fix.
/Arne
Am Tue, 12 Jul 2016 10:00:41 +0200
schrieb Fran?ois Perrad <[email protected]>:
Quoted 74 lines
The BR package libxml-parser-perl is a very specific host package which works only with the the system perl.
(The reason is : this packge is heavy used, and we don't want the cost|time of the host-perl build)All Perl modules (XS or Pure Perl) use the naming scheme :
perl-module-name.
For example, the wrapper for libxml2 is perl-xml-libxml. All these packages use the BR infrastructure perl-package.If you want a target package perl-xml-parser (XML expat wrapper), start by running:
$ support/script/scancpan XML-Parser which creates a decent template. After add manually, the specific part required by the expat C library. (And obviously, read https://buildroot.org/downloads/manual/manual.html#_infrastructure_for_perl_cpan_packages )Fran?ois
2016-07-08 17:40 GMT+02:00 Thomas Petazzoni <
thomas.petazzoni at free-electrons.com>:Quoted 50 lines
Hello,
Fran?ois, could you have a look at the below question, that is Perl and cross-compilation related?
Thanks!
Thomas
Quoted 25 lines · Arne Caspari
Hi all,
I try to build the libxml-parser-perl package for an ARM target. Unfortunately the compiled perl module fails to load with the following error message:
loadable library and perl binaries are mismatched
After some digging I found that the module needs to be compiled with the same toolchain and compiler defines as the perl binary in order to be ABI compatible. Forcing the defines to be exactly the same as the ones used for the perl binary did not fix the issue so I believe it must be a problem with the toolchain.
I get it right, buildroot builds the host-perl package with the GCC installed on the build system. In my case this is GCC Version 6.1 while the toolchain in buildroot uses 4.9.3. If I get the build procedure right, the ABI for perl is then configured for the 6.1 toolchain while module builds will use the toolchain provided by buildroot which is ABI incompatible.
Is there a way to make buildroot build an initial toolchain to build the host packages with? Did anybody ever successfully build perlXS modules?
Quoted 7 lines
Thanks,
/Arne
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
Hello,
Quoted 6 lines · Arne Caspari
Thank you very much for coming back to me. It is highly appreciated.
I did not know about the scancpan-Script - this would have saved me a ton of work, literally since I just ported a package with very very many module dependencies (the Logitech Media Server) to buildroot. Ah well...
The scancpan script is mentioned in the Buildroot manual:
"""
So, this file and the Config.in can be generated by running the script supports/scripts/scancpan Foo-Bar in the Buildroot directory (or in the BR2_EXTERNAL directory). This script creates a Config.in file and foo-bar.mk file for the requested package, and also recursively for all dependencies specified by CPAN. You should still manually edit the result. In particular, the following things should be checked.
"""
Quoted 12 lines
I finally found the issue with the perl XS packages though: The perl MakeMaker module fails to generate a correct
Makefilebut instead messes up the configuration with the host-perl one. I kind of fixed the issue for me in a very hackish way by modifying the generatedMakefilewith the following script:define
PERL_XML_PARSER_BUILD_CMDSsed "s~/usr/bin/gcc~$(TARGET_CC) -Os~" -i $(@D)/Makefile sed "s~/usr/bin/gcc~$(TARGET_CC) -Os~" -i $(@D)/Expat/Makefile sed "s~x86_64~$(KERNEL_ARCH)~" -i $(@D)/Expat/Makefile sed "s~host/usr/lib~staging/usr/lib~" -i $(@D)/Expat/Makefile $(MAKE) $(@D)/Makefile -C $(@D)
Quoted 5 lines
endef
The module loads correctly afterwards. Unfortunately I do not have any experience with perl or the MakeMaker module to come up with a less insane fix.
Again, could you give a Buildroot configuration, and a test scenario that allows us to reproduce the issue ?
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
Am Tue, 12 Jul 2016 10:56:24 +0200
schrieb Thomas Petazzoni <[email protected]>:
Quoted 4 lines
Again, could you give a Buildroot configuration, and a test scenario that allows us to reproduce the issue ?
Thomas,
for me the following steps reproduce the issue:
support/scripts/scancpan XML-Parser
[Enable the perl-xml-parser module in the config]
make
Then on the target:
perl -e "require XML::Parser"
The buildroot configuration should not matter as long as the target has a different architecture (eg: ARM ). If it is the same architecture as the host, it could produce binary compatible modules by accident.
You could even verify the presence of this issue by just compiling the perl-xml-parser module (any architecture): If you look at the build output, you will see that the Makefile uses the configuration and destination paths of the host perl.
/Arne
Arne,
Could you try to build a package like perl-gd or perl-net-ssleay, in order to verify your infrastructure ?
These packages are known to build on autobuild.buildroot.net .
Fran?ois
2016-07-12 11:04 GMT+02:00 Arne Caspari <[email protected]>:
Quoted 24 lines
Am Tue, 12 Jul 2016 10:56:24 +0200
schrieb Thomas Petazzoni <[email protected]>:Quoted 4 lines
Again, could you give a Buildroot configuration, and a test scenario that allows us to reproduce the issue ?
Thomas,
for me the following steps reproduce the issue:
support/scripts/scancpan XML-Parser
[Enable the perl-xml-parser module in the config]
makeThen on the target:
perl -e "require
XML::Parser"The buildroot configuration should not matter as long as the target has a different architecture (eg: ARM ). If it is the same architecture as the host, it could produce binary compatible modules by accident.
Quoted 7 lines
You could even verify the presence of this issue by just compiling the perl-xml-parser module (any architecture): If you look at the build output, you will see that the
Makefileuses the configuration and destination paths of the host perl./Arne
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20160712/ed41024a/attachment.html>
Hello,
Quoted 3 lines · Fran?ois Perrad
Could you try to build a package like perl-gd or perl-net-ssleay, in order to verify your infrastructure ?
These packages are known to build on autobuild.buildroot.net .
Arne is not complaining about build problems, but about runtime problems. Runtime problems are not detected by autobuild.buildroot.net.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
2016-07-12 12:16 GMT+02:00 Thomas Petazzoni <
[email protected]>:
Quoted 4 lines
Hello,
On Tue, 12 Jul 2016 11:43:49 +0200, Fran?ois Perrad wrote:
Quoted 9 lines
Quoted 1 line
Could you try to build a package like perl-gd or perl-net-ssleay, in
order
Quoted 2 lines
to verify your infrastructure ?
These packages are known to build on autobuild.buildroot.net .Arne is not complaining about build problems, but about runtime problems. Runtime problems are not detected by autobuild.buildroot.net.
After building, it's easy to test them:
# perl -MGD -e 1
...Fran?ois
Quoted 6 lines
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20160712/dfc21e80/attachment.html>
Hello,
Quoted 6 lines · Fran?ois Perrad
Quoted 3 lines
Arne is not complaining about build problems, but about runtime problems. Runtime problems are not detected by autobuild.buildroot.net.
After building, it's easy to test them:
# perl -MGD -e 1 ...
This involves booting the generated system on a target platform (or Qemu), which is not currently supported by autobuild.b.o.
I did start an effort to add runtime tests for Buildroot, see
https://github.com/tpetazzoni/buildroot-runtime-test, but there's still a lot of work that remains to be done.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com