Build results for 2014-03-01

Build statistics for 2014-03-01
===============================

success : 42
failures : 48
timeouts : 0
TOTAL : 90

Classification of failures by reason
====================================

  host-python-setuptools-2.1.2 | 16
               php-gnupg-1.3.3 | 5 
                 lvm2-2.02.103 | 4 
                    parted-3.1 | 4 
             python-pyasn-0.17 | 3 
                      bash-4.3 | 3 
              host-scons-2.3.0 | 2 
                    crda-1.1.3 | 2 
                 mplayer-1.1.1 | 1 
                libnftnl-1.0.0 | 1 
                pciutils-3.2.1 | 1 
                 cairo-1.12.10 | 1 
         host-python3-3.4.0rc1 | 1 
                  zeromq-3.2.4 | 1 
                    fbgrab-1.2 | 1 
          libmicrohttpd-0.9.34 | 1 
               libsigsegv-2.10 | 1 

Detail of failures
===================

      i686 |                       bash-4.3 | NOK | http://autobuild.buildroot.net/results/44a962623be757143a3d6d37fc6cd14964094f79/
      i686 |                       bash-4.3 | NOK | http://autobuild.buildroot.net/results/9c16032f9630e09e3709a69ced6716de725c153d/
    x86_64 |                       bash-4.3 | NOK | http://autobuild.buildroot.net/results/6902eee5e39d9fd79b1e78ccd54382299f1b25ea/
      bfin |                  cairo-1.12.10 | NOK | http://autobuild.buildroot.net/results/c421cde66d8d2d9bdf9ecbe531fb15b86f85d909/
   powerpc |                     crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/87bfdc235ee441153c36489a0a1ec97266899d5a/
   powerpc |                     crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/d442f571f14758a4bdf7d694cecd5a34116be40a/
      bfin |                     fbgrab-1.2 | NOK | http://autobuild.buildroot.net/results/b87b9397f4bf22579e7fca9763c9f099bb63bd8f/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/f22ffbf62d4bc8a1f910269e06636181c324644f/
   powerpc |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5e6f3c45dd480ef2bed8515d6a48579642e28274/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/aa6334d89d47d91fdb696d9248036ec27b4b752a/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/95d0e3656d539258bffa9ca3637d98bc48876a40/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/177f027edd981ed2ec081d93a15993e51f408620/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/87909291c77a45b1fbe99f641ea8f7d4b8b50e13/
   powerpc |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/4952e3a5bd6bac437b8b382dfa18f52eb17505da/
       arm |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/91df531bb3bb57f9b562ed358d6e4a7a970d5e8a/
   powerpc |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5deb94ab960aa2f4428f7e9d2d5315ba79116ce1/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/11725dbff02b07096552a8c0f85d90d969a31fc9/
   powerpc |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/1b4713983e9148408081ee432717cb9f6afa5ac9/
       arm |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b8a4c5dce70d6215945293cd6fd28282c3896a78/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/de0d965f2e1d6e70b234c2708d558273291f3a5f/
   powerpc |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/05c721a141e698d0df49b2867a218d31e7acb311/
      i686 |   host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b01f64c75e71fb4e5282f690ee76a2a49842e889/
      i686 |          host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
       arm |               host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/
   powerpc |               host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/
      i686 |           libmicrohttpd-0.9.34 | NOK | http://autobuild.buildroot.net/results/edb1b5ee1ffaa6a495b4d7c2976f618c8331b5d2/
   powerpc |                 libnftnl-1.0.0 | NOK | http://autobuild.buildroot.net/results/c13ef3757b90838549d6f396155a20890d163ff9/
     nios2 |                libsigsegv-2.10 | NOK | http://autobuild.buildroot.net/results/82dac744ce01c9ebb8c769aea02bf29bc7494aa6/
       arm |                  lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/dd5ceda4e3778a763d26b8a81a08391f13f02307/
   powerpc |                  lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/d6f4d388c62ec2137560aabbff46c4a30c584e18/
      mips |                  lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/bc9b84964667fd37155df97c35252d6b73be724f/
microblaze |                  lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/affded5ab5590e6f7313c59f511b81c1bb645c92/
       arm |                  mplayer-1.1.1 | NOK | http://autobuild.buildroot.net/results/ee409788bbf44434a53c14ede7928367ba8e1c85/
       arm |                     parted-3.1 | NOK | http://autobuild.buildroot.net/results/87ee883d745fafd88fa446244cfe69659ff06f3c/
       arm |                     parted-3.1 | NOK | http://autobuild.buildroot.net/results/1e90929bfaf617381d490dc0b99f333968624ce7/
       arm |                     parted-3.1 | NOK | http://autobuild.buildroot.net/results/c4cb972758a58c4b62e600c14936af4fd559295b/
      i686 |                     parted-3.1 | NOK | http://autobuild.buildroot.net/results/4b5f5dc38a08f622992ea22fd58c1d82b80890c8/
      bfin |                 pciutils-3.2.1 | NOK | http://autobuild.buildroot.net/results/f5c6d4453d546001423244476adf2c6f40411e2c/
      i686 |                php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/e3b2d68b22224fcda95174bd64803a58aae1606a/
   powerpc |                php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/fe26034ee29c7334d09cba8a80ae5bfd2787a552/
      i686 |                php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/df9f7c7c30d9cdba65b4f5c6d46c8aa91de4179a/
   powerpc |                php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/efbce29a2fa1d8561c10613d2145b9b382ddc83c/
   powerpc |                php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/757fb0b12af12de4f128d9c503d45b20776c2313/
       arm |              python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/3c968459f6cc020a3c6bc77f49245d69ed92df20/
       arm |              python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/e009a9a5f8a311b5fcb41212d517a2d52ff2db9d/
       arm |              python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/c11893d19f5614236f1d30a5fca781b94d2f9f80/
      bfin |                   zeromq-3.2.4 | NOK | http://autobuild.buildroot.net/results/59182657d03ca48dc46c96d4ff14667324f4e4f6/

http://autobuild.buildroot.net

Hello all,

Fixed by
http://git.buildroot.net/buildroot/commit/?id=7a95111e48f5fe387f592463effa233834c6fda1

To be fixed.

Issue when host-python3 is built before crda. The easiest solution is probably to make the crda Python script Python 3 compatible and submit the patch upstream.

Missing -l flags?

Fixed by
http://git.buildroot.net/buildroot/commit/?id=bec21dcfa7042d1f7065cf467c4f9c777f0fd072

Parallel installation issue with Python 3. Either we should make Python 3 entirely use MAKE1, or fix the problem. I have analyzed the problem, and it is caused by the parallel installation of libpython into the HOST_DIR and the execution of ./python in the Python source directory.

Scons is not compatible with Python 3. But host-scons is not a target package, it's a host package, so we cannot add a Python 3 related dependency to it. So the two solutions that I see are:

* Change our host-python and host-python3 packages so that they are not exclusive from each other, and then find a way for host-scons to express the fact that it really needs host-python, and not the host-python version matching the selected target python version. Note that achieving this might be difficult.

* Make the three target packages that actually need host-scons "depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next version of SCons is supposed to support Python 3, so we could consider it as a temporary solution.

Suggestions?

Fixed by
http://git.buildroot.net/buildroot/commit/?id=25c438bce1c02775f2c23d8878ab71c3022b0c95

Probably needs some kernel headers dependency. To be checked in detail. Any taker?

Needs a small patch to support NIOS 2. The only information needed is to know whether the stack grows upward or downward on the architecture.

Fixed by
http://git.buildroot.net/buildroot/commit/?id=fb36a82a745e93afc92bb5aff29daea8daecb6d3.

ARM floating point issue.

Fixed by
http://git.buildroot.net/buildroot/commit/?id=8d89fd2b7c09af47f2d1b81cfa4de45a421d8a83.

Blackfin specific issue related to the usage of _ as prefix for assembly symbols.

Fixed by
http://git.buildroot.net/buildroot/commit/?id=1188fcf1667ac316ea9df024b9346415a05b90db.

Fixed by
http://git.buildroot.net/buildroot/commit/?id=e025e49100565617715e0de986ba670532098b14.

Compiler does not provide the right compiler intrinsics. We should check if more recent versions of gcc do not provide the necessary intrinsics.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Thomas, all,

On Sun, Mar 2, 2014 at 9:57 AM, Thomas Petazzoni
<[email protected]> wrote:
[...]

host-python and host-python3 are exclusive at the kconfig level, not the make target one, right or am I missing something? AFAICS the only overlapping thing between the 2 host package is the $(HOST_DIR)/usr/bin/python symlink.

So, we could imagine building host-python and host-python3 in the same build. In such a case, the tricky point becomes the $(HOST_DIR)/usr/bin/python symlink management:
- host-python should not install the symlink if python3 is enabled;
- host-python3 should deactivate the symlink installation when python is enabled.
Then, the SCONS variable should explicity refer to "$(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons"
So, we could be able to build host-python (for host-scons) even if python3 is enabled.

host-python is the only package providing python2 stuff we would allow to build when python3 is enabled.

For other packages, it should not change anything since the $(HOST_DIR)/usr/bin/python symlink still reflects the configuration (ie. its points to python3 if python3 is enabled, to python if python(2) is enabled).
For example, crda needs host-python-m2crypto. Since there is no change in the buildroot configuration (ie. the BR2_PACKAGE_PYTHON and BR2_PACKAGE_PYTHON3 will still have the right value) and $(HOST_DIR)/usr/bin/python still reflect this configuration, host-python-m2crypto should still be built against the right python version.

Of course, if we choose this way, it requires tests.

Comments?

I've just checked out scons repository and it cannot be bootstraped with python3 :'(

See above ;)

Regards,

Samuel

Dear Samuel Martin,

Hum? There are no Kconfig options for host-python and host-python3. It's the target Pythons that are mutually exclusive at the Kconfig level. And since each target Python depends on the same version of host-pythonX, this is what brings host-python3 when target python3 is selected.

You're missing something: the host-scons package is using the host-python-package infrastructure. And this infrastructure uses the host Python version of the selected target Python. This is where the core of the problem is. We would need a way to tell the host-python-package infrastructure to make a package depend on host-python instead of host-python3, even if the selected target Python is Python 3.

See the Scons homepage:

"""
SCons release 2.3.0 now available from the download page at SourceForge. This release adds new features, and fixes and improves a number of issues. This will be the last release to support Python versions earlier than 2.7, as we begin to move toward supporting Python 3.
"""

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Is that because we export an LD_LIBRARY_PATH that points to the freshly
installed libpython? Because I would expect that ./python uses the build directory's libpython, not the installed one...

Oh, but it actually does get called with the correct LD_LIBRARY_PATH:

LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib:
./python ...

The issue is that the executable contains an RPATH pointing to the
install location of libpython.so (in $(HOST_DIR)). When the file doesn't exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the build directory. When the file does exist, it can link against it - except when it hasn't been fully written yet.

The solution would be if the executable would use RUNPATH instead of
RPATH - which according to e.g. [1] is what everybody should do (and I tend to agree). IOW, the option --enable-new-dtags should be passed to the linker ("new" is not really true BTW, the option has existed for since more than 10 years). When we would add a toolchain wrapper for ld, we could add it there.

But for the time being, I'll post a patch for python3 (and python, which
suffers from the same problem) that adds --enable-new-dtags there.

Regards,
Arnout

[1] http://blog.tremily.us/posts/rpath/

Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

During installation, host-python and host-python3 run the freshly built python executable. This is done with a proper LD_LIBRARY_PATH to make sure it picks up the libpython in the build directory. However, the python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib. Therefore, if libpython exists there, it will be used instead.

If the install step is run in parallel, it is possible that libpython is already partially copied to $(HOST_DIR)/usr/lib when python is run. This gives an error like:

python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short

The fix is simple: use RUNPATH instead of RPATH, which allows LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the build directory is always used. RUNPATH is enabled by passing --enable-new-dtags to the linker.

Fixes e.g.
http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4

---
 package/python/python.mk   | 6 ++++++
 package/python3/python3.mk | 6 ++++++
 2 files changed, 12 insertions(+)

index 94765cb..216448e 100644
@@ -34,6 +34,12 @@ HOST_PYTHON_CONF_OPT += 	\
 	--disable-ssl		\
 	--disable-pyo-build
 
+# Make sure that LD_LIBRARY_PATH overrides -rpath.
+# This is needed because libpython may be installed at the same time that
+# python is called.
+HOST_PYTHON_CONF_ENV += \
+	LDFLAGS="$(HOST_LDFLAGS) -Wl,--enable-new-dtags"
+
 # Building host python in parallel sometimes triggers a "Bus error"
 # during the execution of "./python setup.py build" in the
 # installation step. It is probably due to the installation of a

index 880b66e..3d95a6d 100644
@@ -30,6 +30,12 @@ HOST_PYTHON3_CONF_OPT += 	\
 	--disable-idle3		\
 	--disable-pyo-build
 
+# Make sure that LD_LIBRARY_PATH overrides -rpath.
+# This is needed because libpython may be installed at the same time that
+# python is called.
+HOST_PYTHON3_CONF_ENV += \
+	LDFLAGS="$(HOST_LDFLAGS) -Wl,--enable-new-dtags"
+
 PYTHON3_DEPENDENCIES  = host-python3 libffi
 
 HOST_PYTHON3_DEPENDENCIES = host-expat host-zlib

1.9.0

Dear Arnout Vandecappelle (Essensium/Mind),

Applied, thanks.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Dear Arnout Vandecappelle,

Correct. I came to the same conclusion when analyzing the problem the other day (but I reported only my findings on IRC, and didn't send a summary to the list, my bad).

Indeed, DT_RUNPATH would fix the problem, but to me it feels more like a workaround than a real fix, no? Wouldn't the real fix be to ensure that the installation is atomic, i.e either the library is present in $(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs when Python is executed when the library file already exists in $(HOST_DIR)/usr/lib, but its copy hasn't finished yet.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

In this case, that happens to be the problem. But in general, it is
possible that there is already an old version of the library in $(HOST_DIR)/usr/lib, and then that one will be used instead of the newly built one. Nothing catastrophic, but still wrong.

The thing is, RPATH was just a mistake to begin with: you _always_ want
LD_LIBRARY_PATH to take precedence over RPATH, otherwise LD_LIBRARY_PATH doesn't make much sense. That's why I think that adding --enable-new-dtags unconditionally for all builds would be a great idea.

I just noticed now that the idea of an ld wrapper that adds that option
would not work in this case anyway, because this is the host ld... Note that some distros (I think Gentoo and SUSE at least) patch binutils to make the --enable-new-dtags the default, so there you wouldn't even see this issue.

Regards,
Arnout

Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

Dear Arnout Vandecappelle,

Right.

True. So we should add --enable-new-dtags to HOST_LDFLAGS ? Unfortunately, I'm pretty sure there are many packages that don't honor HOST_LDFLAGS.

Ok, yes, indeed we don't have a wrapper for host ld, and it is specifically for host binaries that we need to set a RUNPATH.

Thomas

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com