New: python 2.7 compilation issue

https://bugs.busybox.net/show_bug.cgi?id=7268

Summary: python 2.7 compilation issue
Product: buildroot
Version: unspecified
Platform: PC
OS/Version: Linux
Status: NEW
Severity: normal
Priority: P5
Component: Other
AssignedTo: unassigned at buildroot.uclibc.org
ReportedBy: buildroot-2014 at inbox.com
CC: buildroot at uclibc.org
Estimated Hours: 0.0

Hi,

I am trying to build python for an x86_64 target.

I am using fresh sources of buildroot (exactly the git revision 5ac8f8ce10027ac8285a7e9cec16e74aa38ee2c1).

To achieve that, I go through following steps :
- make defconfig
- in Target Options, change Target Architecture to x86_64 / generic
- in Toolchain, change C Library to glibc
- in Target Packages / Interpreter languages and scripting, enable python
- make

The following tarball encloses configuration file and make output.

It seems host part of python build goes well. Target part, however, fails building "struct" module (I'm not a python expert... so I may be wrong).

Thanks

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #1 from David <[email protected]> 2014-07-07 17:18:26 UTC --- Created attachment 5486
--> https://bugs.busybox.net/attachment.cgi?id=5486
buildroot configuration

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #2 from David <[email protected]> 2014-07-07 17:24:42 UTC --- Created attachment 5492
--> https://bugs.busybox.net/attachment.cgi?id=5492
Make output

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #3 from David <[email protected]> 2014-07-07 17:26:24 UTC --- Hi,

if needed, here is the full make output (5MB, but 797MB uncompressed) :
http://dl.free.fr/eMphhZaBN

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #4 from Thomas Petazzoni <[email protected]> 2014-07-07 17:32:16 UTC --- Thanks for the report, I've just started a build with your configuration to try
to reproduce the problem.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #5 from Thomas Petazzoni <[email protected]> 2014-07-07 17:47:44 UTC --- The build of your .config works fine here (Debian GNU/Linux 6.0).

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #6 from Thomas Petazzoni <[email protected]> 2014-07-07 19:18:57 UTC --- Ok, so the problem is due to the fact that for some reason in your case, Python
has added "-I/usr/include/x86_64-linux-gnu" to the CFLAGS used when building
target modules, which doesn't make sense. I don't have this flag added when
building here, which explains why it works.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #7 from Thomas Petazzoni <[email protected]> 2014-07-07 19:20:25 UTC --- (In reply to comment #3)

Unfortunately, I'm not able to download it. Could you put it somewhere else?

Thanks.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #8 from David <[email protected]> 2014-07-07 20:38:25 UTC --- Thanks for your reply

Here is the download link :
http://we.tl/vSYk0yUI5r

I forgot to say : my host is a Debian 7.5

I'll try to remove this flag tomorrow morning

Regards

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #9 from David <[email protected]> 2014-07-08 11:51:23 UTC --- Hi,

A few more details about this issue...

A git bisect identifies 7e960dc9 as the first "bad" commit : I manage to compile python using revision 577e52ac, but I do not using revision 7e960dc9.

7e960dc9da56d4a484b5480746aaf617ca491274 is the first bad commit commit 7e960dc9da56d4a484b5480746aaf617ca491274

Author: Thomas Petazzoni <[email protected]>
Date:   Thu Feb 13 21:48:53 2014 +0100

python: bump to 2.7.6

Even though jumping from 2.7.3 to 2.7.6 looks like a minor version bump, it is in fact a fairly significant one, because a good number of changes to help cross-compilation have been merged into Python upstream. Therefore, most of our patches are affected by this change.

In detail, this commit:

* Renames all the patches to follow the naming convention of patches in Buildroot: the patch file names should not have any version number.

* The patches numbered above 100, that add configuration options to disable certain modules of the Python standard library, are only renamed and slightly adapted, they didn't change that much.

* The patches numbered below 100 are almost entirely rewritten: many of the cross-compilation problems that used to exist in Python 2.7.3 no longer exist, and the number of remaining problems is smaller, and can be fixed with smaller patches.

:040000 040000 8618414254d7998096c5c87b7774cbd50217cb08
ddcec6a08dbe50e52973f7659789200593fe6861 M package

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

David <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

            Summary|python 2.7 compilation      |python 2.7 compilation
                   |issue                       |issue on a Debian/Ubuntu
                   |                            |host with multiarch support

--- Comment #10 from David <[email protected]> 2014-07-08 17:04:14 UTC --- Hi,

I finally got the bug. Your input about wrong paths really helped me and saved me a lot of time, thanks a lot for this.

In fact, the main setup.py script has a "add_multiarch_paths" function that adapts standard paths to multiarch requirements. For some reason, it adds host /usr/include/x86_64-linux-gnu to include paths for the target python build.

I add a patch that makes multiarch management process available only if python is not being cross-compiled. I do not know if multiarch really makes sense in the case of buildroot. This patch fixes my compilation issue, but it may break something for a different architecture, so I hope someone with a lot more experience about python and buildroot architectures than me will read this patch.

Thanks a lot again
David

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #11 from David <[email protected]> 2014-07-08 17:05:20 UTC --- Created attachment 5504
--> https://bugs.busybox.net/attachment.cgi?id=5504
manage multiarch paths only if not cross-compiling

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

Yann E. MORIN <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |yann.morin.1998 at free.fr

--- Comment #12 from Yann E. MORIN <[email protected]> 2014-07-08 17:19:34 UTC --- David,

Your patch is reversed. ;-)

Care to send the patch to the list, please?

Thanks.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #13 from Thomas Petazzoni <[email protected]> 2014-07-08 20:20:50 UTC --- Patch for python posted at http://patchwork.ozlabs.org/patch/368000/, and patch
for python3 posted at http://patchwork.ozlabs.org/patch/368001/.

Thanks a lot David for your bug report and investigation! Once the patches are merged by Peter, we'll mark this bug as fixed. Of course, it would be good if you could formally test the patches, and give your Tested-by on the patches if their work for you.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #14 from David Bachelart <[email protected]> 2014-07-09 08:56:48 UTC --- Hi Thomas,

I managed to compile python 2.7 and python 3 using your patches.

Please forgive me, I'm not familiar with the "Tested-by" thing... How would you do that ?

Thanks
David

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

--- Comment #15 from Thomas Petazzoni <[email protected]> 2014-07-09 11:26:37 UTC --- (In reply to comment #14)

You would have had to reply to the patches sent to the mailing list, and just add a line like:

In the reply.

However, the patches have been committed now, so it's too late :-)

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7268

Thomas Petazzoni <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #16 from Thomas Petazzoni <[email protected]> 2014-07-09 11:27:10 UTC --- Python 2.x fixed in
http://git.buildroot.net/buildroot/commit/?id=5476b7273f0e5e58b59da9bf6403366d08e6b585,
and Python 3.x fixed in
http://git.buildroot.net/buildroot/commit/?id=5364b22b6c309073d59f593c3b5b1a9b3246e957.

Closing bug.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.