New: Python SSL does not get built for Raspberry Pi

https://bugs.busybox.net/show_bug.cgi?id=7628

Summary: Python SSL does not get built for Raspberry Pi
Product: buildroot
Version: 2014.08
Platform: PC
OS/Version: Linux
Status: NEW
Severity: major
Priority: P5
Component: Other
AssignedTo: unassigned at buildroot.uclibc.org
ReportedBy: tobias.meggendorfer+buildroot at gmail.com
CC: buildroot at uclibc.org
Estimated Hours: 0.0

I'm building for the raspberry pi and want to have the Python SSL package.

Steps i've taken:
- make distclean
- make raspberrypi_defconfig
- make nconfig
- Enable WCHAR support, python3 and python-ssl
- make
- grab coffee

However, there is no "_ssl.cpython-34m.so" ("_ssl.so" in the python2 case) to be found in the lib-dynload folder, thus the ssl module isn't working. I'm fairly new to (cross-)compiling so I might have missed something trivial, although I think I've doublechecked everything. I guess there just is a include or something like that missing, as the python compilation complains about ssl dependencies not being found.

I've also tried this with glibc 2.18 / 2.19 and gcc 4.8 / 4.9, binutils 2.22 / 2.24 and python2 and EABIhf. I actually need glibc + EABIhf for my application, but this problem seems to occur with every configuration.

Host: 3.2.0-70-virtual #105-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
Buildroot: git version 2014.08-0-gfcd720d

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #1 from Thomas Petazzoni <[email protected]> 2014-11-13 23:02:54 UTC --- Can you give your exact Buildroot .config? I've built the following defconfig
here:

BR2_arm=y BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" BR2_PACKAGE_PYTHON3=y BR2_PACKAGE_PYTHON3_SSL=y

And I definitely do have the _ss.cpython-34m.so library built:

# find / -name '_ssl*'
/usr/lib/python3.4/lib-dynload/_ssl.cpython-34m.so

However, when I try to use the ssl module, it complains of an undefined instruction:

# python
Python 3.4.1 (default, Nov 13 2014, 22:55:45) 
[GCC 4.8.3] on linux
Type "help", "copyright", "credits" or "license" for more information.

python (528): undefined instruction: pc=769e536c

Code: 00000000 f26ee1fe e12fff1e ee190f1d (e12fff1e) 

So there is indeed an issue, but it does not appear to be the issue you reported.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #2 from tobias.meggendorfer+buildroot at gmail.com 2014-11-13 23:14:02 UTC --- Created attachment 5774
--> https://bugs.busybox.net/attachment.cgi?id=5774
My config

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #3 from tobias.meggendorfer+buildroot at gmail.com 2014-11-13 23:21:11 UTC --- I attached my config, highlighting the following:
BR2_arm=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_PACKAGE_PYTHON3_PYC_ONLY=y
BR2_PACKAGE_PYTHON3_SSL=y
BR2_PACKAGE_PYTHON3_UNICODEDATA=y

find output/target/ -name "_ssl*" (in the buildroot main directory) definitely is empty with the given configuration (at least on my machine)

The getty port apparently differs but I don't think this causes any issues. I'm currently building with the modified getty port and PYC_ONLY=n, just in case, will edit results in here.

I don't know how well the env is cleaned when compiling, so here is the output of env:
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=<cookie>
SSH_CLIENT=<ip & port>
SSH_TTY=/dev/pts/1
USER=<username>
LS_COLORS=<stuff>
MAIL=/var/mail/<username> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games PWD=/home/<username>
LANG=de_DE.UTF-8
SHLVL=1
HOME=/home/<username>
LOGNAME=<username>
SSH_CONNECTION=<ip & port>
LESSOPEN=| /usr/bin/lesspipe %s
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/env

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #4 from tobias.meggendorfer+buildroot at gmail.com 2014-11-13 23:48:07 UTC --- Apparently I can't edit my comments, so new comment: With the following changes to the attached config:

262c262
< BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
---

652,653c652,653
< BR2_PACKAGE_PYTHON3_PY_ONLY=y
< # BR2_PACKAGE_PYTHON3_PYC_ONLY is not set
---

find output/target/ -name "_ssl*" still is empty (after make clean, make)

output/build/host-python3-3.4.1/build/lib.linux-x86_64-3.4/_ssl.cpython-34m.so (host-python ssl) gets built though (in every configuration I tried so far).

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #5 from Thomas Petazzoni <[email protected]> 2014-11-14 08:25:37 UTC --- Thanks for your feedback. Interesting, it clearly doesn't match what I'm seeing
here. Can you post to some pastebin site or some other location the complete
build log? Something like:

make 2>&1 | tee logfile

And then post 'logfile' somewhere. It would be even nicer if you could after the build do:

find output/target/ | tee -a logfile

So that I can also see the full list of files in your target filesystem. I indeed think something in your host system might be causing the problem, we need to figure out what.

Thanks again!

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #6 from Thomas De Schampheleire <[email protected]> 2014-11-14 09:06:30 UTC --- Possibly related to bug #7478

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #7 from tobias.meggendorfer+buildroot at gmail.com 2014-11-14 10:29:36 UTC --- Again, _ssl* missing in target but present in host. See line 61876, "The necessary bits to build these optional modules were not found:"

I uploaded this compressed file to my gdrive, the uncompressed log has ~23 mb, which probably does not get accepted by any pastebin-ish hoster.
https://drive (dot) google (dot) com/file/d/0BwB7SQYhX6MuZlIwYXdjSDZnZzA/view?usp=sharing

Note: /project is a symlink to /var/data/project on my filesystem (buildroot is too big for my root partition). I just tried compiling after switching to the "real" path, just in case, but same result.

I used the config mentioned in comment 4

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

--- Comment #8 from tobias.meggendorfer+buildroot at gmail.com 2014-11-20 17:46:19 UTC ---
Update: I just setup a new virtual machine (3.13.0-32-generic, Ubuntu 14.04 LTS Server), and apparently there the SSL module gets built and I can run "import ssl" without a problem.

The used config differs from the one I uploaded but it didn't work with this particular one on my original system, too.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=7628

Thomas Petazzoni <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #9 from Thomas Petazzoni <[email protected]> 2015-06-03 01:31:47 UTC --- I believe it might have been a locale related issue, since having a non-english locale would make the Python add_gcc_paths() logic to fail, and therefore it
wouldn't find the headers for OpenSSL.

This has been solved by commit
http://git.buildroot.net/buildroot/commit/package/python?id=238bf198cf4f31a3301b81a924d5e22044945955.

Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.