Skip to contentSkip to conversation

fixes for building in long path

PATCH Superseded Robert Yang 2 participants 15 messages Mar 14, 2017 – Mar 16, 2017

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover fixes for building in long path

The following changes since commit 86b09a7ed67a43a45c805f44778bed0bfdf57361:

u-boot: add option to specify FDT argument in extlinux.conf (2017-03-13 09:43:08 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/long
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/long

Robert Yang (8):
m4: do_configure: fix "Argument list too long"
guile: do_configure: fix "Argument list too long"
kmod: set ac_cv_path_DOLT_BASH to /bin/bash
mc: set ac_cv_path_PERL to /usr/bin/perl
valgrind: set ac_cv_path_PERL to /usr/bin/perl
autogen-native: config/libopts.m4
autogen-native: fix POSIX_SHELL and perl oe/path.py: fix for "Argument list too long"

 meta/lib/oe/path.py                                | 10 +++---
 .../autogen/autogen-native_5.18.12.bb              |  4 +++
 ...pts.m4-regenerate-it-from-config-libopts..patch | 39 ++++++++++++++++++++++
 ...2-autoopts-mk-tpl-config.sh-fix-perl-path.patch | 32 ++++++++++++++++++
 meta/recipes-devtools/guile/guile_2.0.14.bb        |  3 ++
 meta/recipes-devtools/m4/m4-1.4.18.inc             |  3 ++
 meta/recipes-devtools/valgrind/valgrind_3.12.0.bb  |  2 ++
 meta/recipes-extended/mc/mc_4.8.18.bb              |  2 ++
 meta/recipes-kernel/kmod/kmod.inc                  |  2 ++
 9 files changed, 93 insertions(+), 4 deletions(-)
 create mode 100644 meta/recipes-devtools/autogen/autogen/0001-config-libopts.m4-regenerate-it-from-config-libopts..patch
 create mode 100644 meta/recipes-devtools/autogen/autogen/0002-autoopts-mk-tpl-config.sh-fix-perl-path.patch
Signature · 1 line
2.11.0.rc2.dirty
#1 lore raw

PATCH 1/8 m4: do_configure: fix "Argument list too long"

Fixed when len(TMPDIR) = 410:
Can't exec "/bin/sh": Argument list too long at /usr/lib/perl/5.18/IO/File.pm line 65.

This is becuase it has a lot of m4 files, use relative path for them can fix the problem.

---
 meta/recipes-devtools/m4/m4-1.4.18.inc | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-devtools/m4/m4-1.4.18.inc +3 -0
index 5652b74308..d7c8648577 100644
@@ -15,5 +15,8 @@ SRC_URI_append_class-target = "\
            file://0001-Unset-need_charset_alias-when-building-for-musl.patch \
 	   "
 
+# Fix "Argument list too long" error when len(TMPDIR) = 410
+acpaths = "-I ./m4"
+
 SRC_URI[md5sum] = "a077779db287adf4e12a035029002d28"
 SRC_URI[sha256sum] = "ab2633921a5cd38e48797bf5521ad259bdc4b979078034a3b790d7fec5493fab"
Signature · 1 line
2.11.0.rc2.dirty
#2 lore raw

PATCH 2/8 guile: do_configure: fix "Argument list too long"

In reply to Robert Yang

Fixed when len(TMPDIR) = 410:
Can't exec "/bin/sh": Argument list too long at /usr/lib/perl/5.18/IO/File.pm line 65.

This is becuase it has a lot of m4 files, use relative path for them can fix the problem.

---
 meta/recipes-devtools/guile/guile_2.0.14.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-devtools/guile/guile_2.0.14.bb +3 -0
index d2306e6a75..7a36ba0c3b 100644
@@ -30,6 +30,9 @@ SRC_URI[sha256sum] = "e8442566256e1be14e51fc18839cd799b966bc5b16c6a1d7a7c35155a8
 inherit autotools gettext pkgconfig texinfo
 BBCLASSEXTEND = "native"
 
+# Fix "Argument list too long" error when len(TMPDIR) = 410
+acpaths = "-I ./m4"
+
 DEPENDS = "libunistring bdwgc gmp libtool libffi ncurses readline"
 # add guile-native only to the target recipe's DEPENDS
 DEPENDS_append_class-target = " guile-native libatomic-ops"
Signature · 1 line
2.11.0.rc2.dirty
#3 lore raw

Re: [PATCH 2/8] guile: do_configure: fix "Argument list too long"

meta/recipes-devtools/guile/guile_2.0.14.bb SRC_URI[sha256sum] = "e8442566256e1be14e51fc18839cd799b966bc5b16c6a1d7a7c35155a8 · line 34

Quoted 3 lines · Robert Yang
meta/recipes-devtools/guile/guile_2.0.14.bb +2 -0
+# Fix "Argument list too long" error when len(TMPDIR) = 410
+acpaths = "-I ./m4"

If we're fixing this in various recipes, why not fix in autotools.bbclass directly?

Ross

#4 lore raw

Re: [PATCH 2/8] guile: do_configure: fix "Argument list too long"

Quoted 9 lines · Ross

On 14 March 2017 at 08:45, Robert Yang <[email protected] <mailto:[email protected]>> wrote:

+# Fix "Argument list too long" error when len(TMPDIR) = 410 +acpaths = "-I ./m4"

If we're fixing this in various recipes, why not fix in autotools.bbclass directly?

We had tried this about 3 years, but failed since the m4 files may not in ./m4, and the worse is that it may in sub directory, RP didn't think that set it in autotools.bbclass is a good idea, so we fix the recipe one by one.

// Robert

Quoted 2 lines

Ross

#5 lore raw

PATCH 3/8 kmod: set ac_cv_path_DOLT_BASH to /bin/bash

In reply to Robert Yang

The shebang's length is usually 128 as defined in /usr/include/linux/binfmts.h:
#define BINPRM_BUF_SIZE 128

So there would be errors when /path/to/hosttools/bash is longer than 128:

/bin/sh: ./doltcompile: [snip]: bad interpreter: No such file or directory

Set ac_cv_path_DOLT_BASH to /bin/bash to fix the problem.

---
 meta/recipes-kernel/kmod/kmod.inc | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-kernel/kmod/kmod.inc +2 -0
index cfa409dbb3..77bf689db3 100644
@@ -30,6 +30,8 @@ S = "${WORKDIR}/git"
 EXTRA_AUTORECONF += "--install --symlink"
 EXTRA_OECONF +=" --enable-tools --with-zlib"
 
+CACHED_CONFIGUREVARS += "ac_cv_path_DOLT_BASH=/bin/bash"
+
 PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
 PACKAGECONFIG[logging] = " --enable-logging,--disable-logging"
 PACKAGECONFIG[manpages] = "--enable-manpages, --disable-manpages, libxslt-native xmlto-native"
Signature · 1 line
2.11.0.rc2.dirty
#6 lore raw

PATCH 4/8 mc: set ac_cv_path_PERL to /usr/bin/perl

In reply to Robert Yang

The shebang's length is usually 128 as defined in /usr/include/linux/binfmts.h:
#define BINPRM_BUF_SIZE 128

So there would be errors when /path/to/hosttools/perl is longer than 128.

Set ac_cv_path_PERL to /usr/bin/perl to fix the problem.

---
 meta/recipes-extended/mc/mc_4.8.18.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-extended/mc/mc_4.8.18.bb +2 -0
index 78abb275ba..87cc4d0677 100644
@@ -22,6 +22,8 @@ PACKAGECONFIG[sftp] = "--enable-vfs-sftp,--disable-vfs-sftp,libssh2,"
 
 EXTRA_OECONF = "--with-screen=ncurses --without-gpm-mouse --without-x"
 
+CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"
+
 do_install_append () {
 	sed -i -e '1s,#!.*perl,#!${bindir}/env perl,' ${D}${libexecdir}/mc/extfs.d/*
 	sed -i -e '1s,#!.*python,#!${bindir}/env python,' ${D}${libexecdir}/mc/extfs.d/*
Signature · 1 line
2.11.0.rc2.dirty
#7 lore raw

Re: [PATCH 4/8] mc: set ac_cv_path_PERL to /usr/bin/perl

Quoted 2 lines · Robert Yang

+CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"

Would it be sensible to put this in autotools.bbclass?

Ross

#8 lore raw

Re: [PATCH 4/8] mc: set ac_cv_path_PERL to /usr/bin/perl

Quoted 7 lines · Ross
Quoted 2 lines · Robert Yang

+CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"

Would it be sensible to put this in autotools.bbclass?

Hang on. ac_cv_path_PERL is the result of AC_PATH_PROG(perl) which is the path to the perl on the host. Is mc using this path only for writing the hashbang into scripts? If it tries to execute the path on the build host then there's several ways this can fail (perl in /usr/local or with buildtools).

Ross

#9 lore raw

Re: [PATCH 4/8] mc: set ac_cv_path_PERL to /usr/bin/perl

Quoted 10 lines · Ross

On 14 March 2017 at 12:06, Burton, Ross <[email protected] <mailto:[email protected]>> wrote:

On 14 March 2017 at 08:45, Robert Yang <[email protected] <mailto:[email protected]>> wrote:

+CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"

Quoted 7 lines

Would it be sensible to put this in autotools.bbclass?

Hang on. ac_cv_path_PERL is the result of AC_PATH_PROG(perl) which is the path to the perl on the host. Is mc using this path only for writing the hashbang into scripts? If it tries to execute the path on the build host then there's several ways this can fail (perl in /usr/local or with buildtools).

Hi Ross,

The automake and autoconf had already set this:

CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"

So I set it for mc referred to them, should we use "#!/usr/bin/env perl" here ?

// Robert

Quoted 3 lines

Ross

#10 lore raw

Re: [PATCH 4/8] mc: set ac_cv_path_PERL to /usr/bin/perl

Quoted 21 lines · Robert Yang
Quoted 10 lines · Ross

On 14 March 2017 at 12:06, Burton, Ross <[email protected] <mailto:[email protected]>> wrote:

On 14 March 2017 at 08:45, Robert Yang <[email protected] <mailto:[email protected]>> wrote:

+CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"

Quoted 7 lines

Would it be sensible to put this in autotools.bbclass?

Hang on. ac_cv_path_PERL is the result of AC_PATH_PROG(perl) which is the path to the perl on the host. Is mc using this path only for writing the hashbang into scripts? If it tries to execute the path on the build host then there's several ways this can fail (perl in /usr/local or with buildtools).

Quoted 7 lines

Hi Ross,

The automake and autoconf had already set this:

CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"

So I set it for mc referred to them, should we use "#!/usr/bin/env perl" here ?

Hi Ross,

Any comments, please ?

// Robert

Quoted 6 lines

// Robert

Quoted 3 lines

Ross

#11 lore raw

PATCH 5/8 valgrind: set ac_cv_path_PERL to /usr/bin/perl

In reply to Robert Yang

The shebang's length is usually 128 as defined in /usr/include/linux/binfmts.h:
#define BINPRM_BUF_SIZE 128

So there would be errors when /path/to/hosttools/perl is longer than 128.

Set ac_cv_path_PERL to /usr/bin/perl to fix the problem.

---
 meta/recipes-devtools/valgrind/valgrind_3.12.0.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/valgrind/valgrind_3.12.0.bb +2 -0
index a4a3e72ecd..48091420c3 100644
@@ -56,6 +56,8 @@ EXTRA_OECONF_append_arm = " --host=armv7${HOST_VENDOR}-${HOST_OS}"
 
 EXTRA_OEMAKE = "-w"
 
+CACHED_CONFIGUREVARS += "ac_cv_path_PERL=${USRBINPATH}/perl"
+
 # valgrind likes to control its own optimisation flags. It generally defaults
 # to -O2 but uses -O0 for some specific test apps etc. Passing our own flags
 # (via CFLAGS) means we interfere with that. Only pass DEBUG_FLAGS to it
Signature · 1 line
2.11.0.rc2.dirty
#12 lore raw

PATCH 6/8 autogen-native: config/libopts.m4

In reply to Robert Yang

It was out of date compared to config/libopts.def, so regenerate it via "autogen config/libopts.def" command.

---
 .../autogen/autogen-native_5.18.12.bb              |  1 +
 ...pts.m4-regenerate-it-from-config-libopts..patch | 39 ++++++++++++++++++++++
 2 files changed, 40 insertions(+)
 create mode 100644 meta/recipes-devtools/autogen/autogen/0001-config-libopts.m4-regenerate-it-from-config-libopts..patch
meta/recipes-devtools/autogen/autogen-native_5.18.12.bb +1 -0
index cf1744c8c0..0004389b11 100644
@@ -12,6 +12,7 @@ SRC_URI = "${GNU_MIRROR}/autogen/rel${PV}/autogen-${PV}.tar.gz \
            file://increase-timeout-limit.patch \
            file://mk-tpl-config.sh-force-exit-value-to-be-0-in-subproc.patch \
            file://fix-script-err-when-processing-libguile.patch \
+           file://0001-config-libopts.m4-regenerate-it-from-config-libopts..patch \
 "
 
 SRC_URI[md5sum] = "551d15ccbf5b5fc5658da375d5003389"
meta/recipes-devtools/autogen/autogen/0001-config-libopts.m4-regenerate-it-from-config-libopts..patch new file +39 -0
new file mode 100644
index 0000000000..a14018e339
@@ -0,0 +1,39 @@
+From 45040e7d268329ebc40e6cb237c64a6637cfab5c Mon Sep 17 00:00:00 2001
+From: Robert Yang <[email protected]>
+Date: Mon, 13 Mar 2017 20:22:10 -0700
+Subject: [PATCH] config/libopts.m4: regenerate it from config/libopts.def
+
+It was out of date compared to config/libopts.def, so regenerate it via
+"autogen config/libopts.def" command.
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <[email protected]>
+---
+ config/libopts.m4 | 3 ++-
+ 1 file changed, 2 insertions(+), 1 deletion(-)
+
+diff --git a/config/libopts.m4 b/config/libopts.m4
+index c7ba4f3..51e6a39 100644
+--- a/config/libopts.m4
++++ b/config/libopts.m4
+@@ -2,7 +2,7 @@ dnl  -*- buffer-read-only: t -*- vi: set ro:
+ dnl
+ dnl DO NOT EDIT THIS FILE   (libopts.m4)
+ dnl
+-dnl It has been AutoGen-ed
++dnl It has been AutoGen-ed  March 13, 2017 at 08:21:21 PM by AutoGen 5.18
+ dnl From the definitions    libopts.def
+ dnl and the template file   conftest.tpl
+ dnl
+@@ -114,6 +114,7 @@ AC_DEFUN([INVOKE_LIBOPTS_MACROS_FIRST],[
+   AC_PROG_SED
+   [while :
+   do
++      test -x "$POSIX_SHELL" && break
+       POSIX_SHELL=`which bash`
+       test -x "$POSIX_SHELL" && break
+       POSIX_SHELL=`which dash`
+-- 
+2.10.2
+
Signature · 1 line
2.11.0.rc2.dirty
#13 lore raw

PATCH 7/8 autogen-native: fix POSIX_SHELL and perl

In reply to Robert Yang

The shebang's length is usually 128 as defined in /usr/include/linux/binfmts.h:
#define BINPRM_BUF_SIZE 128

So there would be errors when /path/to/hosttools/perl is longer than 128.

This patch fixes the problem when POSIX_SHELL and perl are used as the interpreters.

---
 .../autogen/autogen-native_5.18.12.bb              |  3 ++
 ...2-autoopts-mk-tpl-config.sh-fix-perl-path.patch | 32 ++++++++++++++++++++++
 2 files changed, 35 insertions(+)
 create mode 100644 meta/recipes-devtools/autogen/autogen/0002-autoopts-mk-tpl-config.sh-fix-perl-path.patch
meta/recipes-devtools/autogen/autogen-native_5.18.12.bb +3 -0
index 0004389b11..c3c8287210 100644
@@ -13,6 +13,7 @@ SRC_URI = "${GNU_MIRROR}/autogen/rel${PV}/autogen-${PV}.tar.gz \
            file://mk-tpl-config.sh-force-exit-value-to-be-0-in-subproc.patch \
            file://fix-script-err-when-processing-libguile.patch \
            file://0001-config-libopts.m4-regenerate-it-from-config-libopts..patch \
+           file://0002-autoopts-mk-tpl-config.sh-fix-perl-path.patch \
 "
 
 SRC_URI[md5sum] = "551d15ccbf5b5fc5658da375d5003389"
@@ -30,6 +31,8 @@ inherit autotools texinfo native pkgconfig
 export GUILE_LOAD_PATH = "${STAGING_DATADIR_NATIVE}/guile/2.0"
 export GUILE_LOAD_COMPILED_PATH = "${STAGING_LIBDIR_NATIVE}/guile/2.0/ccache"
 
+export POSIX_SHELL = "/bin/bash"
+
 do_install_append () {
 	create_wrapper ${D}/${bindir}/autogen \
 		GUILE_LOAD_PATH=${STAGING_DATADIR_NATIVE}/guile/2.0 \
meta/recipes-devtools/autogen/autogen/0002-autoopts-mk-tpl-config.sh-fix-perl-path.patch new file +32 -0
new file mode 100644
index 0000000000..d5fe143ce4
@@ -0,0 +1,32 @@
+From 9f69f3f5ef22bf1bcffb0e651efc260889cfaa46 Mon Sep 17 00:00:00 2001
+From: Robert Yang <[email protected]>
+Date: Mon, 13 Mar 2017 20:33:30 -0700
+Subject: [PATCH] autoopts/mk-tpl-config.sh: fix perl path
+
+Use "which perl" as shebang doesn't work when it is longer than
+BINPRM_BUF_SIZE which is 128 usually. So use "/usr/bin/env perl" to
+instead of.
+
+Upstream-Status: Pending
+
+Signed-off-by: Robert Yang <[email protected]>
+---
+ autoopts/mk-tpl-config.sh | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/autoopts/mk-tpl-config.sh b/autoopts/mk-tpl-config.sh
+index 093e808..8dfc6dd 100755
+--- a/autoopts/mk-tpl-config.sh
++++ b/autoopts/mk-tpl-config.sh
+@@ -98,7 +98,7 @@ fix_scripts() {
+         st=`sed 1q $f`
+ 
+         case "$st" in
+-        *perl ) echo '#!' `which perl`
++        *perl ) echo '#!/usr/bin/env perl'
+                  sed 1d $f
+                  ;;
+ 
+-- 
+2.10.2
+
Signature · 1 line
2.11.0.rc2.dirty
#14 lore raw

PATCH 8/8 oe/path.py: fix for "Argument list too long"

In reply to Robert Yang

Issue: LIN9-1648

Fixed when len(TMPDIR) = 410:

$ bitbake core-image-sato-sdk
[snip]
Subprocess output:
/bin/sh: /bin/cp: Argument list too long

ERROR: core-image-sato-sdk-1.0-r0 do_rootfs: Function failed: do_rootfs [snip]

This is because "copyhardlinktree(src, dst)" does "cp -afl src/* dst", while src/* is expanded to "src/file1 src/file2, src/file3..." which causes the "Argument list too long", use ./* as src and change cwd in subprocess.check_output() to fix the problem.

---
 meta/lib/oe/path.py | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
meta/lib/oe/path.py +6 -4
index a2a50c5b39..448a2b944e 100644
@@ -103,12 +103,14 @@ def copyhardlinktree(src, dst):
         source = ''
         if os.path.isdir(src):
             if len(glob.glob('%s/.??*' % src)) > 0:
-                source = '%s/.??* ' % src
-            source = source + '%s/*' % src
+                source = './.??* '
+            source += './*'
+            s_dir = src
         else:
             source = src
-        cmd = 'cp -afl --preserve=xattr %s %s' % (source, dst)
-        subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
+            s_dir = os.getcwd()
+        cmd = 'cp -afl --preserve=xattr %s %s' % (source, os.path.realpath(dst))
+        subprocess.check_output(cmd, shell=True, cwd=s_dir, stderr=subprocess.STDOUT)
     else:
         copytree(src, dst)
 
Signature · 1 line
2.11.0.rc2.dirty