Replace Python 3.3.x with Python 3.4.2
PATCH 1/1 Replace Python 3.3.x with Python 3.4.2
From: Artur Wroblewski <[email protected]>
Replace Python 3.3.x with Python 3.4.2 (bug #6857)
This also fixes issue with missing basic Python 3.x modules, see
https://bugzilla.yoctoproject.org/show_bug.cgi?id=6831
The text of the license changed a little, it contains new dates.
This change enables Python 3.4.2 successful build and possiblity of Python 3 being run on a Yocto Project based distribution, but is still work in progress. Beside missing features specific to Python 3.4 (i.e. pip support), some patches need review and updating
- 020-dont-compile-python-files.patch
- 070-dont-clean-ipkg-install.patch
- 080-distutils-dont_adjust_files.patch
- 110-enable-zlib.patch
- 130-readline-setup.patch
- 150-fix-setupterm.patch
- fix-ast.h-dependency.patch
There is new patch - python-debug.patch - to fix build when building Python without debug switch.
Despite further work needed, I hope this patch can be merged, so further work on Python 3 support can be continued in smaller steps.
---
documentation/ref-manual/ref-variables.xml | 2 +-
meta/classes/distutils3-base.bbclass | 2 +-
meta/classes/distutils3-native-base.bbclass | 2 +-
meta/classes/python3native.bbclass | 2 +-
...on-3.3-manifest.inc => python-3.4-manifest.inc} | 142 +++++-----
...on3-native_3.3.3.bb => python3-native_3.4.2.bb} | 34 +--
...2-distutils-prefix-is-inside-staging-area.patch | 29 +-
.../python3/avoid-ncursesw-include-path.patch | 32 +--
.../recipes-devtools/python/python3/makerace.patch | 2 -
....3-multilib.patch => python-3.4-multilib.patch} | 307 ++++++++++-----------
.../python/python3/python-debug.patch | 26 ++
.../python/python3/unixccompiler.patch | 4 +-
.../python/{python3_3.3.3.bb => python3_3.4.2.bb} | 52 ++--
...te-manifest-3.3.py => generate-manifest-3.4.py} | 56 ++--
14 files changed, 326 insertions(+), 366 deletions(-)
rename meta/recipes-devtools/python/{python-3.3-manifest.inc => python-3.4-manifest.inc} (32%)
rename meta/recipes-devtools/python/{python3-native_3.3.3.bb => python3-native_3.4.2.bb} (71%)
rename meta/recipes-devtools/python/python3/{python-3.3-multilib.patch => python-3.4-multilib.patch} (71%)
create mode 100644 meta/recipes-devtools/python/python3/python-debug.patch
rename meta/recipes-devtools/python/{python3_3.3.3.bb => python3_3.4.2.bb} (81%)
rename scripts/contrib/python/{generate-manifest-3.3.py => generate-manifest-3.4.py} (86%)documentation/ref-manual/ref-variables.xml +1 -1
@@ -6953,7 +6953,7 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3" The OpenEmbedded build system uses the ABI to construct directory names used when installing the Python headers and libraries in sysroot - (e.g. <filename>.../python3.3m/...</filename>). + (e.g. <filename>.../python3.4m/...</filename>). </para> <para>
meta/classes/distutils3-base.bbclass +1 -1
@@ -1,7 +1,7 @@ DEPENDS += "${@["${PYTHON_PN}-native ${PYTHON_PN}", ""][(d.getVar('PACKAGES', True) == '')]}" RDEPENDS_${PN} += "${@['', '${PYTHON_PN}-core']['${CLASSOVERRIDE}' == 'class-target']}" -PYTHON_BASEVERSION = "3.3" +PYTHON_BASEVERSION = "3.4" PYTHON_ABI = "m" inherit distutils-common-base python3native
meta/classes/distutils3-native-base.bbclass +1 -1
@@ -1,4 +1,4 @@ -PYTHON_BASEVERSION = "3.3" +PYTHON_BASEVERSION = "3.4" PYTHON_ABI = "m" inherit distutils-native-base
meta/classes/python3native.bbclass +1 -1
@@ -1,4 +1,4 @@ -PYTHON_BASEVERSION = "3.3" +PYTHON_BASEVERSION = "3.4" inherit python-dir
meta/recipes-devtools/python/python-3.4-manifest.inc renamed from meta/recipes-devtools/python/python-3.3-manifest.inc +71 -71
334 lines not on this page · raw message
meta/recipes-devtools/python/python3-native_3.4.2.bb renamed from meta/recipes-devtools/python/python3-native_3.3.3.bb +14 -20
@@ -1,37 +1,29 @@ require recipes-devtools/python/python.inc PR = "${INC_PR}.0" -PYTHON_MAJMIN = "3.3" +PYTHON_MAJMIN = "3.4" DISTRO_SRC_URI ?= "file://sitecustomize.py" DISTRO_SRC_URI_linuxstdbase = "" -SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.bz2 \ +SRC_URI = "http://www.python.org/ftp/python/${PV}/Python-${PV}.tar.xz \ file://12-distutils-prefix-is-inside-staging-area.patch \ file://python-config.patch \ file://000-cross-compile.patch \ -file://020-dont-compile-python-files.patch \ file://030-fixup-include-dirs.patch \ -file://070-dont-clean-ipkg-install.patch \ -file://080-distutils-dont_adjust_files.patch \ -file://110-enable-zlib.patch \ -file://130-readline-setup.patch \ -file://150-fix-setupterm.patch \ -file://python-3.3-multilib.patch \ -file://03-fix-tkinter-detection.patch \ -file://avoid_warning_about_tkinter.patch \ -file://06-ctypes-libffi-fix-configure.patch \ -file://shutil-follow-symlink-fix.patch \ file://0001-h2py-Fix-issue-13032-where-it-fails-with-UnicodeDeco.patch \ +file://host_include_contamination.patch \ +file://avoid-ncursesw-include-path.patch \ +file://python-3.4-multilib.patch \ +file://06-ctypes-libffi-fix-configure.patch \ file://sysroot-include-headers.patch \ file://unixccompiler.patch \ -file://fix-ast.h-dependency.patch \ file://makerace.patch \ -${DISTRO_SRC_URI} \ -file://python3-fix-build-error-with-Readline-6.3.patch \ +file://03-fix-tkinter-detection.patch \ +file://python-debug.patch \ " -SRC_URI[md5sum] = "f3ebe34d4d8695bf889279b54673e10c" -SRC_URI[sha256sum] = "e526e9b612f623888364d30cc9f3dfc34dcef39065c713bdbcddf47df84d8dcb" +SRC_URI[md5sum] = "36fc7327c02c6f12fa24fc9ba78039e3" +SRC_URI[sha256sum] = "1c6d9682d145c056537e477bbfa060ce727f9edd38df1827e0f970dcf04b2def" -LIC_FILES_CHKSUM = "file://LICENSE;md5=4eaea08eaaf6875189b0c49f26fa2005" +LIC_FILES_CHKSUM = "file://LICENSE;md5=64fc2b30b67d0a8423c250e0386ed72f" S = "${WORKDIR}/Python-${PV}"
@@ -40,6 +32,8 @@ DEPENDS = "openssl-native bzip2-replacement-native zlib-native readline-native s inherit native +CONFIGUREOPTS += " --without-pydebug --without-ensurepip" + RPROVIDES += "python3-distutils-native python3-compression-native python3-textutils-native python3-core-native" EXTRA_OECONF_append = " --bindir=${bindir}/${PN}"
@@ -57,7 +51,7 @@ EXTRA_OEMAKE = '\ # No ctypes option for python 3 PYTHONLSBOPTS = "" -do_configure_prepend() { +do_configure_append() { autoreconf --verbose --install --force --exclude=autopoint Modules/_ctypes/libffi || bbnote "_ctypes failed to autoreconf" }
meta/recipes-devtools/python/python3/12-distutils-prefix-is-inside-staging-area.patch +6 -23
@@ -1,14 +1,6 @@ -Upstream-Status: Inappropriate [embedded specific] - -# The proper prefix is inside our staging area. -# Signed-Off: Michael 'Mickey' Lauer <[email protected]> -# Signed-off-by: Phil Blundell <[email protected]> -# Signed-off-by: Khem Raj <[email protected]> - -Index: Python-3.3.2/Lib/distutils/sysconfig.py -=================================================================== ---- Python-3.3.2.orig/Lib/distutils/sysconfig.py 2013-07-30 00:00:52.769749805 -0700 -+++ Python-3.3.2/Lib/distutils/sysconfig.py 2013-07-30 00:16:22.545767248 -0700 +diff -Nur Python-3.4.2.orig/Lib/distutils/sysconfig.py Python-3.4.2/Lib/distutils/sysconfig.py +--- Python-3.4.2.orig/Lib/distutils/sysconfig.py 2014-10-08 09:18:12.000000000 +0100 ++++ Python-3.4.2/Lib/distutils/sysconfig.py 2014-10-23 12:26:27.409663674 +0100 @@ -16,10 +16,11 @@ from .errors import DistutilsPlatformError
@@ -36,7 +28,7 @@ Index: Python-3.3.2/Lib/distutils/sysconfig.py prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX if os.name == "posix": if python_build: -@@ -136,6 +139,12 @@ +@@ -134,6 +137,12 @@ If 'prefix' is supplied, use it instead of sys.base_prefix or sys.base_exec_prefix -- i.e., ignore 'plat_specific'. """
@@ -49,7 +41,7 @@ Index: Python-3.3.2/Lib/distutils/sysconfig.py if prefix is None: if standard_lib: prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX -@@ -144,7 +153,7 @@ +@@ -142,7 +151,7 @@ if os.name == "posix": libpython = os.path.join(prefix,
@@ -58,16 +50,7 @@ Index: Python-3.3.2/Lib/distutils/sysconfig.py if standard_lib: return libpython else: -@@ -249,7 +258,7 @@ - else: - # The name of the config.h file changed in 2.2 - config_h = 'pyconfig.h' -- return os.path.join(inc_dir, config_h) -+ return os.path.join(inc_dir, config_h).replace( os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") ) - - - def get_makefile_filename(): -@@ -258,7 +267,7 @@ +@@ -251,7 +260,7 @@ return os.path.join(_sys_home or project_base, "Makefile") lib_dir = get_python_lib(plat_specific=0, standard_lib=1) config_file = 'config-{}{}'.format(get_python_version(), build_flags)
meta/recipes-devtools/python/python3/avoid-ncursesw-include-path.patch +7 -25
@@ -1,29 +1,11 @@ -We should make sure that sysroot is used by gcc instead of assuming -hardcoded locations for include paths - -Upstream-Status: Pending - -Signed-off-by: Khem Raj <[email protected]> -: -Index: Python-3.3.3/configure.ac -=================================================================== ---- Python-3.3.3.orig/configure.ac 2014-01-29 19:00:15.245312752 -0800 -+++ Python-3.3.3/configure.ac 2014-01-29 23:27:22.938526262 -0800 -@@ -1548,7 +1548,7 @@ - # checks for header files - AC_HEADER_STDC - ac_save_cppflags="$CPPFLAGS" --CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" -+CPPFLAGS="$CPPFLAGS -I=/usr/include/ncursesw" - AC_CHECK_HEADERS(asm/types.h conio.h curses.h direct.h dlfcn.h errno.h \ - fcntl.h grp.h \ - ieeefp.h io.h langinfo.h libintl.h ncurses.h process.h pthread.h \ -@@ -4308,7 +4308,7 @@ - fi +--- Python-3.4.2/configure.ac.orig 2014-10-17 21:58:13.967488345 +0100 ++++ Python-3.4.2/configure.ac 2014-10-17 21:59:01.231689639 +0100 +@@ -4399,7 +4399,7 @@ + # first curses header check ac_save_cppflags="$CPPFLAGS" -CPPFLAGS="$CPPFLAGS -I/usr/include/ncursesw" +CPPFLAGS="$CPPFLAGS -I=/usr/include/ncursesw" - # On HP/UX 11.0, mvwdelch is a block with a return statement - AC_MSG_CHECKING(whether mvwdelch is an expression) - AC_CACHE_VAL(ac_cv_mvwdelch_is_expression, + + AC_CHECK_HEADERS(curses.h ncurses.h) +
meta/recipes-devtools/python/python3/makerace.patch +0 -2
@@ -11,8 +11,6 @@ The easiest way to avoid this is to list the dependencies explictly. RP 28/2/2014 -Upstream-Status: Pending - Index: Python-3.3.3/Makefile.pre.in =================================================================== --- Python-3.3.3.orig/Makefile.pre.in 2014-02-27 14:04:38.268339533 +0000
meta/recipes-devtools/python/python3/python-3.4-multilib.patch renamed from meta/recipes-devtools/python/python3/python-3.3-multilib.patch +144 -163
406 lines not on this page · raw message
meta/recipes-devtools/python/python3/python-debug.patch new file +26 -0
@@ -0,0 +1,26 @@ +diff -Nur Python-3.4.2.orig/Objects/obmalloc.c Python-3.4.2/Objects/obmalloc.c +--- Python-3.4.2.orig/Objects/obmalloc.c 2014-10-08 09:18:15.000000000 +0100 ++++ Python-3.4.2/Objects/obmalloc.c 2014-10-18 02:32:25.576272764 +0100 +@@ -2141,7 +2141,9 @@ + + if (p->ref.count == 0) { + /* currently unused */ ++#ifdef Py_DEBUG + assert(pool_is_in_list(p, arenas[i].freepools)); ++#endif + continue; + } + ++numpools[sz]; +diff -Nur Python-3.4.2.orig/Objects/unicodeobject.c Python-3.4.2/Objects/unicodeobject.c +--- Python-3.4.2.orig/Objects/unicodeobject.c 2014-10-08 09:18:15.000000000 +0100 ++++ Python-3.4.2/Objects/unicodeobject.c 2014-10-18 02:48:59.667861619 +0100 +@@ -418,6 +418,9 @@ + } + return 1; + } ++#else ++int ++_PyUnicode_CheckConsistency(PyObject *op, int check_content) { return 1; } + #endif + + static PyObject*
meta/recipes-devtools/python/python3/unixccompiler.patch +1 -3
@@ -13,13 +13,11 @@ Index: Python-3.3.2/Lib/distutils/unixccompiler.py =================================================================== --- Python-3.3.2.orig/Lib/distutils/unixccompiler.py 2013-05-15 09:32:54.000000000 -0700 +++ Python-3.3.2/Lib/distutils/unixccompiler.py 2013-08-01 00:58:18.629056286 -0700 -@@ -202,7 +202,9 @@ +@@ -202,7 +202,7 @@ # ccompiler.py. def library_dir_option(self, dir): - return "-L" + dir -+ if dir.startswith("."): -+ return "-L" + dir + return "-L=" + dir def _is_gcc(self, compiler_name):
meta/recipes-devtools/python/python3_3.4.2.bb renamed from meta/recipes-devtools/python/python3_3.3.3.bb +16 -36
87 lines not on this page · raw message
scripts/contrib/python/generate-manifest-3.4.py renamed from scripts/contrib/python/generate-manifest-3.3.py +37 -19
122 lines not on this page · raw message
Signature · 1 line
2.1.3
Hi Artur,
Quoted 2 lines · wrobell
Replace Python 3.3.x with Python 3.4.2 (bug #6857)
This patch has long lines so didn't survive SMTP, can you push it to a git repository somewhere?
Quoted 2 lines
Despite further work needed, I hope this patch can be merged, so further
work on Python 3 support can be continued in smaller steps.
Quoted 1 line
Personally I'd prefer to see a working Python 3.4 integrated in one go instead of having a broken 3.4 integrated for an unknown about of time. Hopefully anyone else who is interested in 3.4 can step up and work on the outstanding issues now that this patch is public (which is another good reason to have a public repo with this in).
Ross
Quoted 9 lines · Ross
Hi Artur,
Quoted 2 lines · wrobell
Replace Python 3.3.x with Python 3.4.2 (bug #6857)
This patch has long lines so didn't survive SMTP, can you push it to a git repository somewhere?
I have attached it to bug #6857. Actually, some improvements to the python manifest generation script was next step in my plan. This would tackle the above problem in the future.
Quoted 11 lines
Quoted 4 lines
Despite further work needed, I hope this patch can be merged, so further
work on Python 3 support can be continued in smaller steps.
Personally I'd prefer to see a working Python 3.4 integrated in one go instead of having a broken 3.4 integrated for an unknown about of time. Hopefully anyone else who is interested in 3.4 can step up and work on the outstanding issues now that this patch is public (which is another good reason to have a public repo with this in).
If you specify some exit criteria for this, then I can try to improve the patch. But
please keep in mind that Python 3.3 in Yocto is broken to such extent that it crashes due to missing essential modules.
Regards,
w
Quoted 4 lines · Artur Wroblewski
Quoted 3 lines
Personally I'd prefer to see a working Python 3.4 integrated in one go instead of having a broken 3.4 integrated for an unknown about of time. Hopefully anyone else who is interested in 3.4 can step up and work on
the
Quoted 8 lines
Quoted 2 lines
outstanding issues now that this patch is public (which is another good reason to have a public repo with this in).
If you specify some exit criteria for this, then I can try to improve the patch. But
please keep in mind that Python 3.3 in Yocto is broken to such extent that it
crashes due to missing essential modules.
The exit criteria would be a patch to upgrade to 3.4 that doesn't come
with the caveat that seven of the patches need to be updated. 3.3 being broken is a separate issue to upgrading to 3.4 and we should probably fix 3.3 first, backport that fix to dizzy, and then upgrade to 3.4.
I've filed a bug about the startup problem with just python3 installed (#6967). Note that whilst this is a bug it's not fundamentally broken as installing just python3-core (which python3 is an alias for - more on that in a minute) won't pull in the bulk of the runtime library. Installing python3-core and python3-modules brings in the entire runtime, and the interpreter starts up fine.
You could argue that whilst splitting out the core library is reasonable so that minimal installs are possible, having "python3" not be the full Python experience isn't reasonable. I'd be inclined to agree here, and think that python3 should be a meta-package that depends on all of Python.
Ross
Quoted 16 lines · Ross
Quoted 8 lines · Artur Wroblewski
Quoted 7 lines
Personally I'd prefer to see a working Python 3.4 integrated in one go instead of having a broken 3.4 integrated for an unknown about of time. Hopefully anyone else who is interested in 3.4 can step up and work on the
outstanding issues now that this patch is public (which is another good reason to have a public repo with this in).Quoted 5 lines
If you specify some exit criteria for this, then I can try to improve the patch. But
please keep in mind that Python 3.3 in Yocto is broken to such extent that it
crashes due to missing essential modules.
Quoted 2 lines
The exit criteria would be a patch to upgrade to 3.4 that doesn't come with
the caveat that seven of the patches need to be updated.
Yep, there is more work required.
Quoted 3 lines
3.3 being broken
is a separate issue to upgrading to 3.4 and we should probably fix 3.3 first, backport that fix to dizzy, and then upgrade to 3.4.
No incompatibilities were introduced in Python 3.4
(https://docs.python.org/3/whatsnew/3.4.html).
What is your reason to support version of Python, which practically cannot be started
anyway and is 2 years old?
Quoted 6 lines
I've filed a bug about the startup problem with just python3 installed (#6967). Note that whilst this is a bug it's not fundamentally broken as installing just python3-core (which python3 is an alias for - more on that in a minute) won't pull in the bulk of the runtime library. Installing python3-core and python3-modules brings in the entire runtime, and the interpreter starts up fine.
See also bug #6831.
In the context of your bug report, please note that manifest file still does not contain reprlib (the same in Poky 1.6.1)
and there is no "misc" package defined there, so I am really confused what is the mainstream work here.
Regards,
Artur