Skip to contentSkip to conversation

Fix building in very long paths

PATCH Applied Tom Rini 2 participants 7 messages Jul 20, 2011 – Jul 22, 2011

lore
Share & tools
lore

Tom Rini

author
#0 lore raw

cover Fix building in very long paths

Hey all,

The following series fixes building in deep paths. To be clear, the biggest problem is that shebang (#!) is not portable over 80 chars in length and we already eat up 51 chars with sysroot paths on x86_64. The fix, in concept is easy, use /usr/bin/env instead. In practice, perl makes this difficult. In this series we fix python to use /usr/bin/env, add a 'nativeperl' wrapper (RP's name) so that we can do /usr/bin/env nativeperl and not require sub-recipes to have inherit perlnative (eg sato-icon-theme would need perlnative if we didn't do this). Next we have cpan make sure it mangles everything to use nativeperl and then go fix a few hard cases by hand. All of these changes are either direct ports (python) or port + more work (perl/cpan) of changes I did in oe.dev. With this series I'm able to build world for qemux86 in a very long path (which means I could then turn on doing this in our jenkins cluster on the weekend like I did for oe.dev on testing-next).

The following changes since commit 0d03800caffc2341b74190d38f7a372223c89f4c:
Mei Lei (1):
lsbsetup_1.0.bb: Change the hardcoded /usr/lib to support multilib

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib trini/work-in-deep-paths-v1
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=trini/work-in-deep-paths-v1

Tom Rini (5):
python-native: Mangle scripts to use /usr/bin/env python, bump PR
gnome-doc-utils: Use /usr/bin/env python in xml2po, bump PR
perl-native: Add a perl-native wrapper in the normal bindir cpan.bbclass: Perform more mangling for perl path
icon-naming-utils-native: Fix long path to perl in icon-name-mapping

 meta/classes/cpan.bbclass                          |    3 ++
 .../icon-naming-utils-native_0.8.7.bb              |    7 +++-
 meta/recipes-devtools/perl/liburi-perl_1.58.bb     |    2 +-
 .../perl/libxml-parser-perl_2.40.bb                |    2 +-
 meta/recipes-devtools/perl/libxml-perl_0.08.bb     |    2 +-
 .../perl/libxml-simple-perl_2.18.bb                |    2 +-
 meta/recipes-devtools/perl/perl-native_5.12.3.bb   |   17 +++++++++-
 .../recipes-devtools/python/python-native_2.6.6.bb |    5 +++
 .../sgmlspl/sgmlspl-native_1.03ii.bb               |    2 +-
 .../perl/libconvert-asn1-perl_0.22.bb              |    2 +-
 .../recipes-extended/perl/libtimedate-perl_1.20.bb |    2 +-
 .../perl/libxml-namespacesupport-perl_1.11.bb      |    2 +-
 meta/recipes-extended/perl/libxml-sax-perl_0.96.bb |    2 +-
 .../use-usr-bin-env-for-python-in-xml2po.patch     |   33 ++++++++++++++++++++
 meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb |    5 ++-
 15 files changed, 75 insertions(+), 13 deletions(-)
 create mode 100644 meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch
author
#1 lore raw

PATCH 1/5 python-native: Mangle scripts to use /usr/bin/env python, bump PR

This is dd33ced4b8123600efa1f67f704aec13b0842ad1 in oe.dev and needed for building in deep paths.

---
 .../recipes-devtools/python/python-native_2.6.6.bb |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
meta/recipes-devtools/python/python-native_2.6.6.bb +5 -0
index fa943f4..59ed61a 100644
@@ -36,4 +36,9 @@ do_install() {
 	oe_runmake 'DESTDIR=${D}' install
 	install -d ${D}${bindir}/
 	install -m 0755 Parser/pgen ${D}${bindir}/
+
+	# Make sure we use /usr/bin/env python
+	for PYTHSCRIPT in `grep -rIl ${bindir}/python ${D}${bindir}`; do
+		sed -i -e '1s|^#!.*|#!/usr/bin/env python|' $PYTHSCRIPT
+	done
 }
Signature · 1 line
1.7.0.4
author
#2 lore raw

PATCH 2/5 gnome-doc-utils: Use /usr/bin/env python in xml2po, bump PR

In reply to Tom Rini

In oe.dev we can just sed over the binary but in newer versions of this program we need to fix the in use copy too. As noted in the patch header, this isn't appropriate for upstream as it could change behavior on Windows.

---
 .../use-usr-bin-env-for-python-in-xml2po.patch     |   33 ++++++++++++++++++++
 meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb |    5 ++-
 2 files changed, 36 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch
meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch new file +33 -0
new file mode 100644
index 0000000..a285ad2
@@ -0,0 +1,33 @@
+Upstream-Status="Inappropriate [Would break behavior on Windows]"
+
+We can't use #!/full/path/to/python -u as this can be longer than shebang
+allows for.  In order to be appropraite for upstream more work would be
+needed to make sure that the main xml2po code doesn't rely on python
+being invoked with -u (force stdin/out/err to be used raw).
+---
+ xml2po/xml2po/Makefile.am  |    1 -
+ xml2po/xml2po/xml2po.py.in |    2 +-
+ 2 files changed, 1 insertion(+), 2 deletions(-)
+
+Index: gnome-doc-utils-0.20.6/xml2po/xml2po/Makefile.am
+===================================================================
+--- gnome-doc-utils-0.20.6.orig/xml2po/xml2po/Makefile.am
++++ gnome-doc-utils-0.20.6/xml2po/xml2po/Makefile.am
+@@ -7,7 +7,6 @@ CLEANFILES = xml2po
+ 
+ xml2po: xml2po.py.in
+ 	sed -e "s/^VERSION =.*/VERSION = \"@VERSION@\"/" \
+-	    -e "s+^#!.*python.*+#!$(PYTHON)+" \
+ 	  < $(srcdir)/xml2po.py.in > xml2po
+ 	chmod +x xml2po
+ 
+Index: gnome-doc-utils-0.20.6/xml2po/xml2po/xml2po.py.in
+===================================================================
+--- gnome-doc-utils-0.20.6.orig/xml2po/xml2po/xml2po.py.in
++++ gnome-doc-utils-0.20.6/xml2po/xml2po/xml2po.py.in
+@@ -1,4 +1,4 @@
+-#!/usr/bin/python -u
++#!/usr/bin/env python
+ # -*- encoding: utf-8 -*-
+ # Copyright (c) 2004, 2005, 2006 Danilo Šegan <[email protected]>.
+ # Copyright (c) 2009 Claude Paroz <[email protected]>.
meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb +3 -2
index 4554122..1ec1076 100644
@@ -1,9 +1,10 @@
 require gnome-doc-utils.inc
 LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=eb723b61539feef013de476e68b5c50a \
 		    file://COPYING.LGPL;md5=a6f89e2100d9b6cdffcea4f398e37343"
-PR = "r3"
+PR = "r4"
 
-SRC_URI += "file://xsltproc_nonet.patch"
+SRC_URI += "file://xsltproc_nonet.patch \
+	    file://use-usr-bin-env-for-python-in-xml2po.patch"
 
 SRC_URI[archive.md5sum] = "8f6e05071599bc073007830ea0a68391"
 SRC_URI[archive.sha256sum] = "091486e370480bf45349ad09dac799211092a02938b26a0d68206172cb6cebbf"
Signature · 1 line
1.7.0.4
author
#3 lore raw

PATCH 3/5 perl-native: Add a perl-native wrapper in the normal bindir

In reply to Tom Rini

We need this to allow for scripts to do #!/usr/bin/env perl-native and not require an 'inherit perlnative' per user of a package.

---
 meta/recipes-devtools/perl/perl-native_5.12.3.bb |   17 ++++++++++++++++-
 1 files changed, 16 insertions(+), 1 deletions(-)
meta/recipes-devtools/perl/perl-native_5.12.3.bb +16 -1
index 63f7e13..945ebd4 100644
@@ -4,7 +4,7 @@ SECTION = "libs"
 LICENSE = "Artistic|GPL"
 LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \
 		    file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8"
-PR = "r3"
+PR = "r4"
 
 LIC_FILES_CHKSUM = "file://Copying;md5=2b4c6ffbcfcbdee469f02565f253d81a \
                     file://Artistic;md5=f921793d03cc6d63ec4b15e9be8fd3f8"
@@ -102,3 +102,18 @@ do_install () {
 	create_wrapper ${D}${bindir}/perl PERL5LIB='$PERL5LIB:${STAGING_LIBDIR}/perl/${PV}:${STAGING_LIBDIR}/perl/'
 	create_wrapper ${D}${bindir}/perl${PV} PERL5LIB='$PERL5LIB:${STAGING_LIBDIR}/perl/${PV}:${STAGING_LIBDIR}/perl/'
 }
+
+SYSROOT_PREPROCESS_FUNCS += "perl_sysroot_create_wrapper"
+
+perl_sysroot_create_wrapper () {
+	mkdir -p ${SYSROOT_DESTDIR}${bindir}
+	# Create a wrapper that /usr/bin/env perl will use to get perl-native.
+	# This MUST live in the normal bindir.
+	cat > ${SYSROOT_DESTDIR}${bindir}/../nativeperl << EOF
+#!/bin/sh
+realpath=\`readlink -fn \$0\`
+exec \`dirname \$realpath\`/perl-native/perl "\$@"
+EOF
+	chmod 0755 ${SYSROOT_DESTDIR}${bindir}/../nativeperl
+	cat ${SYSROOT_DESTDIR}${bindir}/../nativeperl
+}
Signature · 1 line
1.7.0.4
author
#4 lore raw

PATCH 4/5 cpan.bbclass: Perform more mangling for perl path

In reply to Tom Rini

On hosts where we may already be past the #! limit this is required and this is safe on the target as well. This is be7fe31 in oe.dev. Related to this we now bump all PRs for recipes that inherit cpan. Note that in oe.dev we mangle for perl but here we use the new nativeperl script.

---
 meta/classes/cpan.bbclass                          |    3 +++
 meta/recipes-devtools/perl/liburi-perl_1.58.bb     |    2 +-
 .../perl/libxml-parser-perl_2.40.bb                |    2 +-
 meta/recipes-devtools/perl/libxml-perl_0.08.bb     |    2 +-
 .../perl/libxml-simple-perl_2.18.bb                |    2 +-
 .../sgmlspl/sgmlspl-native_1.03ii.bb               |    2 +-
 .../perl/libconvert-asn1-perl_0.22.bb              |    2 +-
 .../recipes-extended/perl/libtimedate-perl_1.20.bb |    2 +-
 .../perl/libxml-namespacesupport-perl_1.11.bb      |    2 +-
 meta/recipes-extended/perl/libxml-sax-perl_0.96.bb |    2 +-
 10 files changed, 12 insertions(+), 9 deletions(-)
meta/classes/cpan.bbclass +3 -0
index cbf428d..cb1c2f4 100644
@@ -35,6 +35,9 @@ cpan_do_compile () {
 
 cpan_do_install () {
 	oe_runmake DESTDIR="${D}" install_vendor
+	for PERLSCRIPT in `grep -rIEl '#!${bindir}/perl-native.*/perl' ${D}`; do
+		sed -i -e 's|^#!${bindir}/perl-native.*/perl|#!/usr/bin/env nativeperl|' $PERLSCRIPT
+	done
 }
 
 EXPORT_FUNCTIONS do_configure do_compile do_install
meta/recipes-devtools/perl/liburi-perl_1.58.bb +1 -1
index 76a1231..ebb4911 100644
@@ -4,7 +4,7 @@ and manipulate the various components that make up these strings."
 
 SECTION = "libs"
 LICENSE = "Artistic | GPLv1+"
-PR = "r0"
+PR = "r1"
 
 LIC_FILES_CHKSUM = "file://README;beginline=26;endline=30;md5=6c33ae5c87fd1c4897714e122dd9c23d"
 
meta/recipes-devtools/perl/libxml-parser-perl_2.40.bb +1 -1
index 3f1a0ad..1d1593b 100644
@@ -5,7 +5,7 @@ LIC_FILES_CHKSUM = "file://README;beginline=2;endline=6;md5=c8767d7516229f07b26e
 
 DEPENDS += "expat expat-native"
 
-PR = "r3"
+PR = "r4"
 
 SRC_URI = "http://www.cpan.org/modules/by-module/XML/XML-Parser-${PV}.tar.gz"
 SRC_URI[md5sum] = "c66e9adba003d0667cc40115ccd837a5"
meta/recipes-devtools/perl/libxml-perl_0.08.bb +1 -1
index 9c74cdb..fb637a8 100644
@@ -5,7 +5,7 @@ XML::Grove and others."
 SUMMARY = "Collection of Perl modules for working with XML."
 SECTION = "libs"
 LICENSE = "Artistic | GPLv1+"
-PR = "r0"
+PR = "r1"
 
 LIC_FILES_CHKSUM = "file://README;beginline=33;endline=35;md5=1705549eef7577a3d6ba71123a1f0ce8"
 
meta/recipes-devtools/perl/libxml-simple-perl_2.18.bb +1 -1
index 89b65b2..3a50d7f 100644
@@ -2,7 +2,7 @@ SECTION = "libs"
 LICENSE = "Artistic"
 LIC_FILES_CHKSUM = "file://README;beginline=70;md5=94aa5d46682b411a53a5494cfb22640e"
 DEPENDS += "libxml-parser-perl"
-PR = "r2"
+PR = "r3"
 
 SRC_URI = "http://www.cpan.org/modules/by-module/XML/XML-Simple-${PV}.tar.gz"
 
meta/recipes-devtools/sgmlspl/sgmlspl-native_1.03ii.bb +1 -1
index 4e01f17..28e0e9f 100644
@@ -4,7 +4,7 @@ HOMEPAGE = "http://search.cpan.org/src/DMEGG/SGMLSpm-1.03ii/DOC/HTML/SGMLSpm/sgm
 SECTION = "libs"
 LICENSE = "GPLv2+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=18810669f13b87348459e611d31ab760"
-PR = "r1"
+PR = "r2"
 
 SRC_URI = "http://www.cpan.org/authors/id/D/DM/DMEGG/SGMLSpm-${PV}.tar.gz \
            file://combined.patch"
meta/recipes-extended/perl/libconvert-asn1-perl_0.22.bb +1 -1
index fc77b75..48e107c 100644
@@ -2,7 +2,7 @@ DESCRIPTION = "Convert::ASN1 - ASN.1 Encode/Decode library"
 SECTION = "libs"
 LICENSE = "Artistic|GPLv1+"
 LIC_FILES_CHKSUM = "file://README;beginline=10;endline=12;md5=a64b291b13ffddeef33b14f047ee8b26"
-PR = "r1"
+PR = "r2"
 
 SRC_URI = "http://search.cpan.org/CPAN/authors/id/G/GB/GBARR/Convert-ASN1-${PV}.tar.gz"
 
meta/recipes-extended/perl/libtimedate-perl_1.20.bb +1 -1
index 7298f37..c925980 100644
@@ -3,7 +3,7 @@ SECTION = "libs"
 # You can redistribute it and/or modify it under the same terms as Perl itself.
 LICENSE = "Artistic|GPLv1+"
 LIC_FILES_CHKSUM = "file://README;beginline=21;md5=576b7cb41e5e821501a01ed66f0f9d9e"
-PR = "r2"
+PR = "r3"
 
 
 SRC_URI = "http://search.cpan.org/CPAN/authors/id/G/GB/GBARR/TimeDate-${PV}.tar.gz"
meta/recipes-extended/perl/libxml-namespacesupport-perl_1.11.bb +1 -1
index d96430e..f306ff1 100644
@@ -5,7 +5,7 @@ DESCRIPTION = "XML::NamespaceSupport offers a simple way to process namespace-ba
 
 SECTION = "libs"
 LICENSE = "Artistic|GPLv1+"
-PR = "r0"
+PR = "r1"
 
 LIC_FILES_CHKSUM = "file://META.yml;beginline=22;endline=22;md5=3b2b564dae8b9af9e8896e85c07dcbe5"
 
meta/recipes-extended/perl/libxml-sax-perl_0.96.bb +1 -1
index 13956cb..97d2025 100644
@@ -11,7 +11,7 @@ SECTION = "libs"
 LICENSE = "Artistic|GPLv1+"
 DEPENDS += "libxml-namespacesupport-perl-native"
 RDEPENDS_${PN} += "libxml-namespacesupport-perl perl-module-file-temp"
-PR = "r0"
+PR = "r1"
 
 LIC_FILES_CHKSUM = "file://LICENSE;md5=65c4cd8f39c24c7135ed70dacbcb09e3"
 
Signature · 1 line
1.7.0.4
author
#5 lore raw

PATCH 5/5 icon-naming-utils-native: Fix long path to perl in icon-name-mapping

In reply to Tom Rini

This is 5396e0868cf89787746fa6341ddb1f271a770595 in oe.dev but here we instead use the nativeperl wrapper.

---
 .../icon-naming-utils-native_0.8.7.bb              |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)
meta/recipes-devtools/icon-naming-utils/icon-naming-utils-native_0.8.7.bb +6 -1
index d34fb03..4760c96 100644
@@ -1,6 +1,6 @@
 LICENSE = "GPLv2"
 DEPENDS = "libxml-simple-perl-native"
-PR = "r2"
+PR = "r3"
 
 LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f"
 
@@ -9,3 +9,8 @@ SRC_URI = "http://tango.freedesktop.org/releases/icon-naming-utils-0.8.7.tar.gz"
 S = "${WORKDIR}/icon-naming-utils-${PV}"
 
 inherit autotools native perlnative
+
+do_configure_append() {
+	# Make sure we use our nativeperl wrapper.
+	sed -i -e "1s:#!.*:#!/usr/bin/env nativeperl:" ${S}/icon-name-mapping.pl.in
+}
Signature · 1 line
1.7.0.4
Applied
#6 lore raw
In reply to Tom Rini
Quoted 49 lines · Tom Rini

Hey all,

The following series fixes building in deep paths. To be clear, the biggest problem is that shebang (#!) is not portable over 80 chars in length and we already eat up 51 chars with sysroot paths on x86_64. The fix, in concept is easy, use /usr/bin/env instead. In practice, perl makes this difficult. In this series we fix python to use /usr/bin/env, add a 'nativeperl' wrapper (RP's name) so that we can do /usr/bin/env nativeperl and not require sub-recipes to have inherit perlnative (eg sato-icon-theme would need perlnative if we didn't do this). Next we have cpan make sure it mangles everything to use nativeperl and then go fix a few hard cases by hand. All of these changes are either direct ports (python) or port + more work (perl/cpan) of changes I did in oe.dev. With this series I'm able to build world for qemux86 in a very long path (which means I could then turn on doing this in our jenkins cluster on the weekend like I did for oe.dev on testing-next).

The following changes since commit 0d03800caffc2341b74190d38f7a372223c89f4c:
Mei Lei (1):
lsbsetup_1.0.bb: Change the hardcoded /usr/lib to support multilib

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib trini/work-in-deep-paths-v1
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=trini/work-in-deep-paths-v1

Tom Rini (5):
python-native: Mangle scripts to use /usr/bin/env python, bump PR
gnome-doc-utils: Use /usr/bin/env python in xml2po, bump PR
perl-native: Add a perl-native wrapper in the normal bindir cpan.bbclass: Perform more mangling for perl path
icon-naming-utils-native: Fix long path to perl in icon-name-mapping

  meta/classes/cpan.bbclass                          |    3 ++
  .../icon-naming-utils-native_0.8.7.bb              |    7 +++-
  meta/recipes-devtools/perl/liburi-perl_1.58.bb     |    2 +-
  .../perl/libxml-parser-perl_2.40.bb                |    2 +-
  meta/recipes-devtools/perl/libxml-perl_0.08.bb     |    2 +-
  .../perl/libxml-simple-perl_2.18.bb                |    2 +-
  meta/recipes-devtools/perl/perl-native_5.12.3.bb   |   17 +++++++++-
  .../recipes-devtools/python/python-native_2.6.6.bb |    5 +++
  .../sgmlspl/sgmlspl-native_1.03ii.bb               |    2 +-
  .../perl/libconvert-asn1-perl_0.22.bb              |    2 +-
  .../recipes-extended/perl/libtimedate-perl_1.20.bb |    2 +-
  .../perl/libxml-namespacesupport-perl_1.11.bb      |    2 +-
  meta/recipes-extended/perl/libxml-sax-perl_0.96.bb |    2 +-
  .../use-usr-bin-env-for-python-in-xml2po.patch     |   33 ++++++++++++++++++++
  meta/recipes-gnome/gnome/gnome-doc-utils_0.20.6.bb |    5 ++-
  15 files changed, 75 insertions(+), 13 deletions(-)
Quoted 8 lines

create mode 100644 meta/recipes-gnome/gnome/gnome-doc-utils/use-usr-bin-env-for-python-in-xml2po.patch

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Merged into oe-core

Thanks
Sau!