Python library hashlib is missig hash-code.
A message that is not in the archive (sent off-list or not yet received): [email protected]
My theory is that this is caused by the openssl /usr changes, can anyone confirm or deny that?
Begin doorgestuurd bericht:
Quoted 143 lines
Van: Mattias Millbro <[email protected]>
Onderwerp: [Angstrom-devel] Python library hashlib is missig hash-code.
Datum: 15 april 2012 14:38:56 GMT+02:00
Aan: [email protected]
Antwoord aan: Discussion of the angstrom distribution development <[email protected]>Hi everybody,
as a new member of this mailing list I'm not familiar to the proper procedure of this kids of questions/bug reports. But I'll give it a try and please let me know if any necessary information is missing.
I'm running Angstrom on a Beaglebone using the Angstrom-systemd-image-eglibc-ipk-v2012.01-core-beaglebone.rootfs.tar.bz2 image from http://www.angstrom-distribution.org/demo/beaglebone/. I installed python and pyuthon-modules using opkg.
I recently made a completely fresh install and discovered that the python library hashlib does no longer work properly. It does not seem to find code for sha hashes even tho python.org states
"Included are the FIPS secure hash algorithms SHA1, SHA224, SHA256, SHA384, and SHA512 (defined in FIPS 180-2) as well as RSA’s MD5 algorithm (defined in Internet RFC 1321).".
All related packages seems to be up to date and I have spent several days googling and installing/reinstalling various packages without any success.
This is what happens when importing hashlib:
root@beaglebone:~# python Python 2.7.2 (default, Apr 13 2012, 21:18:55) [GCC 4.5.4 20120305 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information.Quoted 1 line
Quoted 1 line
Quoted 1 line
import hashlib
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha512Quoted 1 line
Quoted 1 line
Quoted 1 line
Here is some additional information that might be useful:
root@beaglebone:~# opkg list-installed | grep python libpython2.7-1.0 - 2.7.2-r2.11 python-2to3 - 2.7.2-r2.11 python-audio - 2.7.2-r2.11 python-bsddb - 2.7.2-r2.11 python-codecs - 2.7.2-r2.11 python-compile - 2.7.2-r2.11 python-compiler - 2.7.2-r2.11 python-compression - 2.7.2-r2.11 python-core - 2.7.2-r2.11 python-crypt - 2.7.2-r2.11 python-ctypes - 2.7.2-r2.11 python-curses - 2.7.2-r2.11 python-datetime - 2.7.2-r2.11 python-db - 2.7.2-r2.11 python-debugger - 2.7.2-r2.11 python-difflib - 2.7.2-r2.11 python-distutils - 2.7.2-r2.11 python-doctest - 2.7.2-r2.11 python-elementtree - 2.7.2-r2.11 python-email - 2.7.2-r2.11 python-fcntl - 2.7.2-r2.11 python-gdbm - 2.7.2-r2.11 python-hotshot - 2.7.2-r2.11 python-html - 2.7.2-r2.11 python-idle - 2.7.2-r2.11 python-image - 2.7.2-r2.11 python-io - 2.7.2-r2.11 python-json - 2.7.2-r2.11 python-lang - 2.7.2-r2.11 python-logging - 2.7.2-r2.11 python-mailbox - 2.7.2-r2.11 python-math - 2.7.2-r2.11 python-mime - 2.7.2-r2.11 python-mmap - 2.7.2-r2.11 python-modules - 2.7.2-r2.11 python-multiprocessing - 2.7.2-r2.11 python-netclient - 2.7.2-r2.11 python-netserver - 2.7.2-r2.11 python-numbers - 2.7.2-r2.11 python-pickle - 2.7.2-r2.11 python-pkgutil - 2.7.2-r2.11 python-pprint - 2.7.2-r2.11 python-profile - 2.7.2-r2.11 python-pydoc - 2.7.2-r2.11 python-re - 2.7.2-r2.11 python-readline - 2.7.2-r2.11 python-resource - 2.7.2-r2.11 python-robotparser - 2.7.2-r2.11 python-shell - 2.7.2-r2.11 python-smtpd - 2.7.2-r2.11 python-sqlite3 - 2.7.2-r2.11 python-sqlite3-tests - 2.7.2-r2.11 python-stringold - 2.7.2-r2.11 python-subprocess - 2.7.2-r2.11 python-syslog - 2.7.2-r2.11 python-terminal - 2.7.2-r2.11 python-tests - 2.7.2-r2.11 python-textutils - 2.7.2-r2.11 python-threading - 2.7.2-r2.11 python-tkinter - 2.7.2-r2.11 python-unittest - 2.7.2-r2.11 python-unixadmin - 2.7.2-r2.11 python-xml - 2.7.2-r2.11 python-xmlrpc - 2.7.2-r2.11 python-zlib - 2.7.2-r2.11root@beaglebone:~# opkg list-installed | grep openssl openssl - 1.0.0h-r15.1I'm grateful for any kind of help solving this issue.
Best Regards
/ Mattias_______________________________________________
Angstrom-distro-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/angstrom-distro-devel
Op 15 apr. 2012, om 16:35 heeft Koen Kooi het volgende geschreven:
Quoted 1 line
My theory is that this is caused by the openssl /usr changes, can anyone confirm or deny that?
I dug into this. The breakage was recorded by buildhistory:
==========================================================================
commit fef22c30d3834abdfe8b5c12cd2dd17e3e031973
Author: Koen Kooi <[email protected]> Date: Fri Feb 24 12:39:55 2012 +0100
Build Angstrom v2012.02-core of angstrom v2012.02-core for machine beagleboard on dominion
packages/armv7a-angstrom-linux-gnueabi/python/python-crypt/latest +3 -3
@@ -1,7 +1,7 @@ PV = 2.7.2 PR = r0.6 -RDEPENDS = eglibc (>= 2.12) libcrypto (>= 1.0.0g) libpython2 (>= 2.7.2) python-core +RDEPENDS = eglibc (>= 2.12) libpython2 (>= 2.7.2) python-core RRECOMMENDS = openssl -PKGSIZE = 25436 +PKGSIZE = 14252 FILES = /usr/lib/python2.7/hashlib.* /usr/lib/python2.7/md5.* /usr/lib/python2.7/sha.* /usr/lib/python2.7/lib-dynload/crypt.so /usr/lib/python2.7/lib-dynload/_hashlib.so /usr/lib/python2.7/lib-dynload/_sha256.so /usr/lib/python2.7/lib-dynload/_sha512.so -FILELIST = /usr/lib/python2.7/hashlib.py /usr/lib/python2.7/hashlib.pyo /usr/lib/python2.7/lib-dynload/_hashlib.so /usr/lib/python2.7/lib-dynload/crypt.so /usr/lib/python2.7/md5.py /usr/lib/python2.7/md5.pyo /usr/lib/python2.7/sha.py /usr/lib/python2.7/sha.pyo +FILELIST = /usr/lib/python2.7/hashlib.py /usr/lib/python2.7/hashlib.pyo /usr/lib/python2.7/lib-dynload/crypt.so /usr/lib/python2.7/md5.py /usr/lib/python2.7/md5.pyo /usr/lib/python2.7/sha.py /usr/lib/python2.7/sha.pyo
==========================================================================
The preceding commit affecting python crypt:
==========================================================================
commit f572f6547bc4b1e2a64e4cae8201b70c0e1b4162
Author: Koen Kooi <[email protected]> Date: Wed Feb 22 14:53:00 2012 +0100
Build Angstrom v2012.02-core of angstrom v2012.02-core for machine beaglebone on dominion
packages/armv7a-angstrom-linux-gnueabi/python/python-crypt/latest +2 -2
@@ -3,5 +3,5 @@ PR = r0.6 RDEPENDS = eglibc (>= 2.12) libcrypto (>= 1.0.0g) libpython2 (>= 2.7.2) python-core RRECOMMENDS = openssl PKGSIZE = 25436 -FILES = /usr/lib/python2.7/hashlib.* /usr/lib/python2.7/md5.* /usr/lib/python2.7/sha.* /usr/lib/python2.7/lib-dynload/crypt.so /usr/lib/python2.7/lib-dynload/_hashlib.so /usr/lib/python2.7/lib-dynload/_sha256.so /usr/lib/python2.7/lib-dynload/_sha512.so -FILELIST = /usr/lib/python2.7/md5.py /usr/lib/python2.7/md5.pyo /usr/lib/python2.7/sha.pyo /usr/lib/python2.7/hashlib.pyo /usr/lib/python2.7/sha.py /usr/lib/python2.7/hashlib.py /usr/lib/python2.7/lib-dynload/_hashlib.so /usr/lib/python2.7/lib-dynload/crypt.so +FILES = /usr/lib/python2.7/hashlib.* /usr/lib/python2.7/md5.* /usr/lib/python2.7/sha.* /usr/lib/python2.7/lib-dynload/crypt.so /usr/lib/python2.7/lib-dynload/_hashlib.so /usr/lib/python2.7/lib-dynload/_sha256.so /usr/lib/python2.7/lib-dynload/_sha512.so +FILELIST = /usr/lib/python2.7/hashlib.py /usr/lib/python2.7/hashlib.pyo /usr/lib/python2.7/lib-dynload/_hashlib.so /usr/lib/python2.7/lib-dynload/crypt.so /usr/lib/python2.7/md5.py /usr/lib/python2.7/md5.pyo /usr/lib/python2.7/sha.py /usr/lib/python2.7/sha.pyo
==========================================================================
And look at that, an openssl patch in between those dates:
==========================================================================
From 01ea85f7f6c53c66c76d6f832518b28bf06ec072 Mon Sep 17 00:00:00 2001
From: Andrei Gherzan <[email protected]>
Date: Tue, 21 Feb 2012 23:03:55 +0000
Subject: openssl: Move libcrypto to base_libdir
This fix is for dhclient. It needs libcrypto at runtime and if libcrypto is in libdir, it's path can be inaccessible on systems where /usr is on nfs for example or dhclient is needed before /usr is mounted.
==========================================================================
I'm going to band-aid it so we can get a working python hashlib module in the 'denzil' branch, hopefully someone will come up with a proper fix soon.
regards,
Koen
Quoted 147 lines
Begin doorgestuurd bericht:
Quoted 143 lines
Van: Mattias Millbro <[email protected]>
Onderwerp: [Angstrom-devel] Python library hashlib is missig hash-code.
Datum: 15 april 2012 14:38:56 GMT+02:00
Aan: [email protected]
Antwoord aan: Discussion of the angstrom distribution development <[email protected]>Hi everybody,
as a new member of this mailing list I'm not familiar to the proper procedure of this kids of questions/bug reports. But I'll give it a try and please let me know if any necessary information is missing.
I'm running Angstrom on a Beaglebone using the Angstrom-systemd-image-eglibc-ipk-v2012.01-core-beaglebone.rootfs.tar.bz2 image from http://www.angstrom-distribution.org/demo/beaglebone/. I installed python and pyuthon-modules using opkg.
I recently made a completely fresh install and discovered that the python library hashlib does no longer work properly. It does not seem to find code for sha hashes even tho python.org states
"Included are the FIPS secure hash algorithms SHA1, SHA224, SHA256, SHA384, and SHA512 (defined in FIPS 180-2) as well as RSA’s MD5 algorithm (defined in Internet RFC 1321).".
All related packages seems to be up to date and I have spent several days googling and installing/reinstalling various packages without any success.
This is what happens when importing hashlib:
root@beaglebone:~# python Python 2.7.2 (default, Apr 13 2012, 21:18:55) [GCC 4.5.4 20120305 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information.Quoted 1 line
Quoted 1 line
Quoted 1 line
import hashlib
ERROR:root:code for hash sha1 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha1
ERROR:root:code for hash sha224 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha224
ERROR:root:code for hash sha256 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha256
ERROR:root:code for hash sha384 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha384
ERROR:root:code for hash sha512 was not found.
Traceback (most recent call last):
File "/usr/lib/python2.7/hashlib.py", line 139, in <module> globals()[__func_name] =__get_hash(__func_name)
File "/usr/lib/python2.7/hashlib.py", line 91, in__get_builtin_constructorraise ValueError('unsupported hash type %s' % name)
ValueError: unsupported hash type sha512Quoted 1 line
Quoted 1 line
Quoted 1 line
Here is some additional information that might be useful:
root@beaglebone:~# opkg list-installed | grep python libpython2.7-1.0 - 2.7.2-r2.11 python-2to3 - 2.7.2-r2.11 python-audio - 2.7.2-r2.11 python-bsddb - 2.7.2-r2.11 python-codecs - 2.7.2-r2.11 python-compile - 2.7.2-r2.11 python-compiler - 2.7.2-r2.11 python-compression - 2.7.2-r2.11 python-core - 2.7.2-r2.11 python-crypt - 2.7.2-r2.11 python-ctypes - 2.7.2-r2.11 python-curses - 2.7.2-r2.11 python-datetime - 2.7.2-r2.11 python-db - 2.7.2-r2.11 python-debugger - 2.7.2-r2.11 python-difflib - 2.7.2-r2.11 python-distutils - 2.7.2-r2.11 python-doctest - 2.7.2-r2.11 python-elementtree - 2.7.2-r2.11 python-email - 2.7.2-r2.11 python-fcntl - 2.7.2-r2.11 python-gdbm - 2.7.2-r2.11 python-hotshot - 2.7.2-r2.11 python-html - 2.7.2-r2.11 python-idle - 2.7.2-r2.11 python-image - 2.7.2-r2.11 python-io - 2.7.2-r2.11 python-json - 2.7.2-r2.11 python-lang - 2.7.2-r2.11 python-logging - 2.7.2-r2.11 python-mailbox - 2.7.2-r2.11 python-math - 2.7.2-r2.11 python-mime - 2.7.2-r2.11 python-mmap - 2.7.2-r2.11 python-modules - 2.7.2-r2.11 python-multiprocessing - 2.7.2-r2.11 python-netclient - 2.7.2-r2.11 python-netserver - 2.7.2-r2.11 python-numbers - 2.7.2-r2.11 python-pickle - 2.7.2-r2.11 python-pkgutil - 2.7.2-r2.11 python-pprint - 2.7.2-r2.11 python-profile - 2.7.2-r2.11 python-pydoc - 2.7.2-r2.11 python-re - 2.7.2-r2.11 python-readline - 2.7.2-r2.11 python-resource - 2.7.2-r2.11 python-robotparser - 2.7.2-r2.11 python-shell - 2.7.2-r2.11 python-smtpd - 2.7.2-r2.11 python-sqlite3 - 2.7.2-r2.11 python-sqlite3-tests - 2.7.2-r2.11 python-stringold - 2.7.2-r2.11 python-subprocess - 2.7.2-r2.11 python-syslog - 2.7.2-r2.11 python-terminal - 2.7.2-r2.11 python-tests - 2.7.2-r2.11 python-textutils - 2.7.2-r2.11 python-threading - 2.7.2-r2.11 python-tkinter - 2.7.2-r2.11 python-unittest - 2.7.2-r2.11 python-unixadmin - 2.7.2-r2.11 python-xml - 2.7.2-r2.11 python-xmlrpc - 2.7.2-r2.11 python-zlib - 2.7.2-r2.11root@beaglebone:~# opkg list-installed | grep openssl openssl - 1.0.0h-r15.1I'm grateful for any kind of help solving this issue.
Best Regards
/ Mattias_______________________________________________
Angstrom-distro-devel mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/angstrom-distro-devel
Op 23 apr. 2012, om 10:15 heeft Koen Kooi het volgende geschreven:
Quoted 5 lines
Op 15 apr. 2012, om 16:35 heeft Koen Kooi het volgende geschreven:
Quoted 1 line
My theory is that this is caused by the openssl /usr changes, can anyone confirm or deny that?
I dug into this. The breakage was recorded by buildhistory:
[..]
Quoted 1 line
I'm going to band-aid it so we can get a working python hashlib module in the 'denzil' branch, hopefully someone will come up with a proper fix soon.
After spending some more time on this and getting a ton of help from RP:
Python can't link to anything in ${base_libdir} due to a combination of python having a stupid buildsystem and the patches we apply to make it less stupid. The only fix I found to get hashlib and ssl to work in python is to revert the patch to openssl that moves libcrypto around.
I really need python unbroken in the release branch, so I see a few scenarios:
1) revert openssl patch, seperate /usr doesn't work properly anyway in this release
2) fix python
3) revert openssl patch, statically link libcrypto into dhclient.
4) bbappend openssl in meta-angstrom to revert the openssl patch
There are probably more scenarios, but I'm going to have lunch before thinking some more about it. My question to all of you:
How do you want to fix this for denzil and how do you want to fix this for master?
And to be clear: I'm mean getting things into the denzil branch of OE-core, not getting it into the release, it's too late for that.
regards,
koen
Quoted 32 lines · Koen Kooi
Op 23 apr. 2012, om 10:15 heeft Koen Kooi het volgende geschreven:
Quoted 3 lines
Op 15 apr. 2012, om 16:35 heeft Koen Kooi het volgende geschreven:
Quoted 1 line
My theory is that this is caused by the openssl /usr changes, can
anyone confirm or deny that?
Quoted 2 lines
I dug into this. The breakage was recorded by buildhistory:
[..]
Quoted 1 line
I'm going to band-aid it so we can get a working python hashlib
module in the 'denzil' branch, hopefully someone will come up with a proper fix soon.
After spending some more time on this and getting a ton of help from RP:
Python can't link to anything in ${base_libdir} due to a combination of python having a stupid buildsystem and the patches we apply to make it less stupid. The only fix I found to get hashlib and ssl to work in python is to revert the patch to openssl that moves libcrypto around.
I really need python unbroken in the release branch, so I see a few scenarios:
1) revert openssl patch, seperate /usr doesn't work properly anyway in this release
2) fix python
3) revert openssl patch, statically link libcrypto into dhclient.
4) bbappend openssl in meta-angstrom to revert the openssl patch
Quoted 8 lines
There are probably more scenarios, but I'm going to have lunch before thinking some more about it. My question to all of you:
How do you want to fix this for denzil and how do you want to fix
this for master?And to be clear: I'm mean getting things into the denzil branch of OE-core, not getting it into the release, it's too late for that.
I've managed to squeeze a fix for this in, I posted the patches last night. It wasn't just the ssl piece that was broken, there were a number of problems in the python recipe causing various modules to fail to build (such as curses and readline). Some of these issues had been present for much longer than the openssl change.
Its worth noting the openssl change was not the problem, it was python passing native paths to a cross-gcc which then understandably got confused.
At this point the release is pretty much done, assuming a favourable QA report.
Cheers,
Richard