Fix building in very long paths
cover Fix building in very long paths
Hey all,
The following series fixes building in deep paths. To be clear, the biggest problem is that shebang (#!) is not portable over 80 chars in length and we already eat up 51 chars with sysroot paths on x86_64. The fix, in concept is easy, use /usr/bin/env instead. In practice, perl makes this difficult. In this series we fix python to use /usr/bin/env, add a 'nativeperl' wrapper (RP's name) so that we can do /usr/bin/env nativeperl and not require sub-recipes to have inherit perlnative (eg sato-icon-theme would need perlnative if we didn't do this). Next we have cpan make sure it mangles everything to use nativeperl and then go fix a few hard cases by hand. All of these changes are either direct ports (python) or port + more work (perl/cpan) of changes I did in oe.dev. With this series I'm able to build world for qemux86 in a very long path (which means I could then turn on doing this in our jenkins cluster on the weekend like I did for oe.dev on testing-next).
The following changes since commit 0d03800caffc2341b74190d38f7a372223c89f4c:
Mei Lei (1):
lsbsetup_1.0.bb: Change the hardcoded /usr/lib to support multilib
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib trini/work-in-deep-paths-v1
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=trini/work-in-deep-paths-v1
Tom Rini (5):
python-native: Mangle scripts to use /usr/bin/env python, bump PR
gnome-doc-utils: Use /usr/bin/env python in xml2po, bump PR
perl-native: Add a perl-native wrapper in the normal bindir cpan.bbclass: Perform more mangling for perl path
icon-naming-utils-native: Fix long path to perl in icon-name-mapping
meta/classes/cpan.bbclass | 3 ++ .../icon-naming-utils-native_0.8.7.bb | 7 +++- meta/recipes-devtools/perl/liburi-perl_1.58.bb | 2 +- .../perl/libxml-parser-perl_2.40.bb | 2 +- meta/recipes-devtools/perl/libxml-perl_0.08.bb | 2 +- .../perl/libxml-simple-perl_2.18.bb | 2 +- meta/recipes-devtools/perl/perl-native_5.12.3.bb | 17 +++++++++- .../recipes-devtools/python/python-native_2.6.6.bb | 5 +++ .../sgmlspl/sgmlspl-native_1.03ii.bb | 2 +- .../perl/libconvert-asn1-perl_0.22.bb | 2 +- .../recipes-extended/perl/libtimedate-perl_1.20.bb | 2 +- .../perl/libxml-namespacesupport-perl_1.11.bb | 2 +- meta/recipes-extended/perl/libxml-sax-perl_0.96.bb | 2 +- .../use-usr-bin-env-for-python-in-xml2po.patch | 33 ++++++++++++++++++++ meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb | 5 ++- 15 files changed, 75 insertions(+), 13 deletions(-) create mode 100644 meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch
PATCH 1/5 python-native: Mangle scripts to use /usr/bin/env python, bump PR
This is dd33ced4b8123600efa1f67f704aec13b0842ad1 in oe.dev and needed for building in deep paths.
--- .../recipes-devtools/python/python-native_2.6.6.bb | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-)
meta/recipes-devtools/python/python-native_2.6.6.bb +5 -0
@@ -36,4 +36,9 @@ do_install() { oe_runmake 'DESTDIR=${D}' install install -d ${D}${bindir}/ install -m 0755 Parser/pgen ${D}${bindir}/ + + # Make sure we use /usr/bin/env python + for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do + sed -i -e '1s|^#!.*|#!/usr/bin/env python|' $PYTHSCRIPT + done }
Signature · 1 line
1.7.0.4
PATCH 2/5 gnome-doc-utils: Use /usr/bin/env python in xml2po, bump PR
In reply to Tom RiniIn oe.dev we can just sed over the binary but in newer versions of this program we need to fix the in use copy too. As noted in the patch header, this isn't appropriate for upstream as it could change behavior on Windows.
--- .../use-usr-bin-env-for-python-in-xml2po.patch | 33 ++++++++++++++++++++ meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb | 5 ++- 2 files changed, 36 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch
meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch new file +33 -0
@@ -0,0 +1,33 @@ +Upstream-Status="Inappropriate [Would break behavior on Windows]" + +We can't use #!/full/path/to/python -u as this can be longer than shebang +allows for. In order to be appropraite for upstream more work would be +needed to make sure that the main xml2po code doesn't rely on python +being invoked with -u (force stdin/out/err to be used raw). +--- + xml2po/xml2po/Makefile.am | 1 - + xml2po/xml2po/xml2po.py.in | 2 +- + 2 files changed, 1 insertion(+), 2 deletions(-) + +Index: gnome-doc-utils-0.20.6/xml2po/xml2po/Makefile.am +=================================================================== +--- gnome-doc-utils-0.20.6.orig/xml2po/xml2po/Makefile.am ++++ gnome-doc-utils-0.20.6/xml2po/xml2po/Makefile.am +@@ -7,7 +7,6 @@ CLEANFILES = xml2po + + xml2po: xml2po.py.in + sed -e "s/^VERSION =.*/VERSION = \"@VERSION@\"/" \ +- -e "s+^#!.*python.*+#!$(PYTHON)+" \ + < $(srcdir)/xml2po.py.in > xml2po + chmod +x xml2po + +Index: gnome-doc-utils-0.20.6/xml2po/xml2po/xml2po.py.in +=================================================================== +--- gnome-doc-utils-0.20.6.orig/xml2po/xml2po/xml2po.py.in ++++ gnome-doc-utils-0.20.6/xml2po/xml2po/xml2po.py.in +@@ -1,4 +1,4 @@ +-#!/usr/bin/python -u ++#!/usr/bin/env python + # -*- encoding: utf-8 -*- + # Copyright (c) 2004, 2005, 2006 Danilo Šegan <[email protected]>. + # Copyright (c) 2009 Claude Paroz <[email protected]>.
meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb +3 -2
@@ -1,9 +1,10 @@ require gnome-doc-utils.inc LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=eb723b61539feef013de476e68b5c50a \ file://COPYING.LGPL;md5=a6f89e2100d9b6cdffcea4f398e37343" -PR = "r3" +PR = "r4" -SRC_URI += "file://xsltproc_nonet.patch" +SRC_URI += "file://xsltproc_nonet.patch \ + file://use-usr-bin-env-for-python-in-xml2po.patch" SRC_URI[archive.md5sum] = "8f6e05071599bc073007830ea0a68391" SRC_URI[archive.sha256sum] = "091486e370480bf45349ad09dac799211092a02938b26a0d68206172cb6cebbf"
Signature · 1 line
1.7.0.4
PATCH 3/5 perl-native: Add a perl-native wrapper in the normal bindir
In reply to Tom RiniWe need this to allow for scripts to do #!/usr/bin/env perl-native and not require an 'inherit perlnative' per user of a package.
--- meta/recipes-devtools/perl/perl-native_5.12.3.bb | 17 ++++++++++++++++- 1 files changed, 16 insertions(+), 1 deletions(-)
meta/recipes-devtools/perl/perl-native_5.12.3.bb +16 -1
@@ -4,7 +4,7 @@ SECTION = "libs" LICENSE = "Artistic|GPL" LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \ file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8" -PR = "r3" +PR = "r4" LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \ file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8"
@@ -102,3 +102,18 @@ do_install () { create_wrapper ${D}${bindir}/perl PERL5LIB='$PERL5LIB:${STAGING_LIBDIR}/perl/${PV}:${STAGING_LIBDIR}/perl/' create_wrapper ${D}${bindir}/perl${PV} PERL5LIB='$PERL5LIB:${STAGING_LIBDIR}/perl/${PV}:${STAGING_LIBDIR}/perl/' } + +SYSROOT_PREPROCESS_FUNCS += "perl_sysroot_create_wrapper" + +perl_sysroot_create_wrapper () { + mkdir -p ${SYSROOT_DESTDIR}${bindir} + # Create a wrapper that /usr/bin/env perl will use to get perl-native. + # This MUST live in the normal bindir. + cat > ${SYSROOT_DESTDIR}${bindir}/../nativeperl << EOF +#!/bin/sh +realpath=\`readlink -fn \$0\` +exec \`dirname \$realpath\`/perl-native/perl "\$@" +EOF + chmod 0755 ${SYSROOT_DESTDIR}${bindir}/../nativeperl + cat ${SYSROOT_DESTDIR}${bindir}/../nativeperl +}
Signature · 1 line
1.7.0.4
PATCH 4/5 cpan.bbclass: Perform more mangling for perl path
In reply to Tom RiniOn hosts where we may already be past the #! limit this is required and this is safe on the target as well. This is be7fe31 in oe.dev. Related to this we now bump all PRs for recipes that inherit cpan. Note that in oe.dev we mangle for perl but here we use the new nativeperl script.
--- meta/classes/cpan.bbclass | 3 +++ meta/recipes-devtools/perl/liburi-perl_1.58.bb | 2 +- .../perl/libxml-parser-perl_2.40.bb | 2 +- meta/recipes-devtools/perl/libxml-perl_0.08.bb | 2 +- .../perl/libxml-simple-perl_2.18.bb | 2 +- .../sgmlspl/sgmlspl-native_1.03ii.bb | 2 +- .../perl/libconvert-asn1-perl_0.22.bb | 2 +- .../recipes-extended/perl/libtimedate-perl_1.20.bb | 2 +- .../perl/libxml-namespacesupport-perl_1.11.bb | 2 +- meta/recipes-extended/perl/libxml-sax-perl_0.96.bb | 2 +- 10 files changed, 12 insertions(+), 9 deletions(-)
meta/classes/cpan.bbclass +3 -0
@@ -35,6 +35,9 @@ cpan_do_compile () { cpan_do_install () { oe_runmake DESTDIR="${D}" install_vendor + for PERLSCRIPT in `grep -rIEl '#!${bindir}/perl-native.*/perl' ${D}`; do + sed -i -e 's|^#!${bindir}/perl-native.*/perl|#!/usr/bin/env nativeperl|' $PERLSCRIPT + done } EXPORT_FUNCTIONS do_configure do_compile do_install
meta/recipes-devtools/perl/liburi-perl_1.58.bb +1 -1
@@ -4,7 +4,7 @@ and manipulate the various components that make up these strings." SECTION = "libs" LICENSE = "Artistic | GPLv1+" -PR = "r0" +PR = "r1" LIC_FILES_CHKSUM = "file://README;beginline=26;endline=30;md5=6c33ae5c87fd1c4897714e122dd9c23d"
meta/recipes-devtools/perl/libxml-parser-perl_2.40.bb +1 -1
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://README;beginline=2;endline=6;md5=c8767d7516229f07b26e DEPENDS += "expat expat-native" -PR = "r3" +PR = "r4" SRC_URI = "http://www.cpan.org/modules/by-module/XML/XML-Parser-${PV}.tar.gz" SRC_URI[md5sum] = "c66e9adba003d0667cc40115ccd837a5"
meta/recipes-devtools/perl/libxml-perl_0.08.bb +1 -1
@@ -5,7 +5,7 @@ XML::Grove and others." SUMMARY = "Collection of Perl modules for working with XML." SECTION = "libs" LICENSE = "Artistic | GPLv1+" -PR = "r0" +PR = "r1" LIC_FILES_CHKSUM = "file://README;beginline=33;endline=35;md5=1705549eef7577a3d6ba71123a1f0ce8"
meta/recipes-devtools/perl/libxml-simple-perl_2.18.bb +1 -1
@@ -2,7 +2,7 @@ SECTION = "libs" LICENSE = "Artistic" LIC_FILES_CHKSUM = "file://README;beginline=70;md5=94aa5d46682b411a53a5494cfb22640e" DEPENDS += "libxml-parser-perl" -PR = "r2" +PR = "r3" SRC_URI = "http://www.cpan.org/modules/by-module/XML/XML-Simple-${PV}.tar.gz"
meta/recipes-devtools/sgmlspl/sgmlspl-native_1.03ii.bb +1 -1
@@ -4,7 +4,7 @@ HOMEPAGE = "http://search.cpan.org/src/DMEGG/SGMLSpm-1.03ii/DOC/HTML/SGMLSpm/sgm SECTION = "libs" LICENSE = "GPLv2+" LIC_FILES_CHKSUM = "file://COPYING;md5=18810669f13b87348459e611d31ab760" -PR = "r1" +PR = "r2" SRC_URI = "http://www.cpan.org/authors/id/D/DM/DMEGG/SGMLSpm-${PV}.tar.gz \ file://combined.patch"
meta/recipes-extended/perl/libconvert-asn1-perl_0.22.bb +1 -1
@@ -2,7 +2,7 @@ DESCRIPTION = "Convert::ASN1 - ASN.1 Encode/Decode library" SECTION = "libs" LICENSE = "Artistic|GPLv1+" LIC_FILES_CHKSUM = "file://README;beginline=10;endline=12;md5=a64b291b13ffddeef33b14f047ee8b26" -PR = "r1" +PR = "r2" SRC_URI = "http://search.cpan.org/CPAN/authors/id/G/GB/GBARR/Convert-ASN1-${PV}.tar.gz"
meta/recipes-extended/perl/libtimedate-perl_1.20.bb +1 -1
@@ -3,7 +3,7 @@ SECTION = "libs" # You can redistribute it and/or modify it under the same terms as Perl itself. LICENSE = "Artistic|GPLv1+" LIC_FILES_CHKSUM = "file://README;beginline=21;md5=576b7cb41e5e821501a01ed66f0f9d9e" -PR = "r2" +PR = "r3" SRC_URI = "http://search.cpan.org/CPAN/authors/id/G/GB/GBARR/TimeDate-${PV}.tar.gz"
meta/recipes-extended/perl/libxml-namespacesupport-perl_1.11.bb +1 -1
@@ -5,7 +5,7 @@ DESCRIPTION = "XML::NamespaceSupport offers a simple way to process namespace-ba SECTION = "libs" LICENSE = "Artistic|GPLv1+" -PR = "r0" +PR = "r1" LIC_FILES_CHKSUM = "file://META.yml;beginline=22;endline=22;md5=3b2b564dae8b9af9e8896e85c07dcbe5"
meta/recipes-extended/perl/libxml-sax-perl_0.96.bb +1 -1
@@ -11,7 +11,7 @@ SECTION = "libs" LICENSE = "Artistic|GPLv1+" DEPENDS += "libxml-namespacesupport-perl-native" RDEPENDS_${PN} += "libxml-namespacesupport-perl perl-module-file-temp" -PR = "r0" +PR = "r1" LIC_FILES_CHKSUM = "file://LICENSE;md5=65c4cd8f39c24c7135ed70dacbcb09e3"
Signature · 1 line
1.7.0.4
PATCH 5/5 icon-naming-utils-native: Fix long path to perl in icon-name-mapping
In reply to Tom RiniThis is 5396e0868cf89787746fa6341ddb1f271a770595 in oe.dev but here we instead use the nativeperl wrapper.
--- .../icon-naming-utils-native_0.8.7.bb | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
meta/recipes-devtools/icon-naming-utils/icon-naming-utils-native_0.8.7.bb +6 -1
@@ -1,6 +1,6 @@ LICENSE = "GPLv2" DEPENDS = "libxml-simple-perl-native" -PR = "r2" +PR = "r3" LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f"
@@ -9,3 +9,8 @@ SRC_URI = "http://tango.freedesktop.org/releases/icon-naming-utils-0.8.7.tar.gz" S = "${WORKDIR}/icon-naming-utils-${PV}" inherit autotools native perlnative + +do_configure_append() { + # Make sure we use our nativeperl wrapper. + sed -i -e "1s:#!.*:#!/usr/bin/env nativeperl:" ${S}/icon-name-mapping.pl.in +}
Signature · 1 line
1.7.0.4
Saul Wold
AppliedQuoted 49 lines · Tom Rini
Hey all,
The following series fixes building in deep paths. To be clear, the biggest problem is that shebang (#!) is not portable over 80 chars in length and we already eat up 51 chars with sysroot paths on
x86_64. The fix, in concept is easy, use/usr/bin/envinstead. In practice, perl makes this difficult. In this series we fix python to use/usr/bin/env, add a 'nativeperl' wrapper (RP's name) so that we can do/usr/bin/envnativeperl and not require sub-recipes to have inherit perlnative (eg sato-icon-theme would need perlnative if we didn't do this). Next we have cpan make sure it mangles everything to use nativeperl and then go fix a few hard cases by hand. All of these changes are either direct ports (python) or port + more work (perl/cpan) of changes I did in oe.dev. With this series I'm able to build world for qemux86 in a very long path (which means I could then turn on doing this in our jenkins cluster on the weekend like I did for oe.dev on testing-next).The following changes since commit
0d03800caffc2341b74190d38f7a372223c89f4c:
Mei Lei (1):
lsbsetup_1.0.bb: Change the hardcoded/usr/libto support multilibare available in the git repository at:
git://git.openembedded.org/openembedded-core-contribtrini/work-in-deep-paths-v1
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=trini/work-in-deep-paths-v1Tom Rini (5):
python-native: Mangle scripts to use/usr/bin/envpython, bump PR
gnome-doc-utils: Use/usr/bin/envpython in xml2po, bump PR
perl-native: Add a perl-native wrapper in the normal bindir cpan.bbclass: Perform more mangling for perl path
icon-naming-utils-native: Fix long path to perl in icon-name-mappingmeta/classes/cpan.bbclass | 3 ++ .../icon-naming-utils-native_0.8.7.bb | 7 +++- meta/recipes-devtools/perl/liburi-perl_1.58.bb | 2 +- .../perl/libxml-parser-perl_2.40.bb | 2 +- meta/recipes-devtools/perl/libxml-perl_0.08.bb | 2 +- .../perl/libxml-simple-perl_2.18.bb | 2 +- meta/recipes-devtools/perl/perl-native_5.12.3.bb | 17 +++++++++- .../recipes-devtools/python/python-native_2.6.6.bb | 5 +++ .../sgmlspl/sgmlspl-native_1.03ii.bb | 2 +- .../perl/libconvert-asn1-perl_0.22.bb | 2 +- .../recipes-extended/perl/libtimedate-perl_1.20.bb | 2 +- .../perl/libxml-namespacesupport-perl_1.11.bb | 2 +- meta/recipes-extended/perl/libxml-sax-perl_0.96.bb | 2 +- .../use-usr-bin-env-for-python-in-xml2po.patch | 33 ++++++++++++++++++++ meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb | 5 ++- 15 files changed, 75 insertions(+), 13 deletions(-)
Quoted 8 lines
create mode 100644
meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Merged into oe-core
Thanks
Sau!