Skip to contentSkip to conversation

Python 2 and 3 bumps, support for Python 3 external modules

PATCH Superseded Thomas Petazzoni 5 participants 47 messages Feb 18, 2014 – Oct 9, 2016

lore
Share & tools
lore
#0 lore raw

v2 cover Python 2 and 3 bumps, support for Python 3 external modules

Hello,

Here is a v2 of the patch set that does numerous Python support improvements. Changes since v1:

* Dropped patches that have been merged.

* Rebased on top of the next branch, in order to resolve conflicts with the parallel build support patches that were merged.

Best regards,

Thomas

Przemyslaw Wrzos (1):
python, python3: fix to ensure libpython is stripped

Rohan Fletcher (1):
python-setuptools: bump version to 2.1.2

Thomas Petazzoni (17):
pkg-python: no longer use distutilscross for setuptools
python-distutilscross: remove package that is no longer needed
python3: removal of *.py/*.pyc is now done globally
python3: make it exclusive from python
python3: add python -> python3 symlink for the host variant
python3: use proper PYTHON3_VERSION_MAJOR instead of harcoding 3.3
python3: bump to 3.4.0rc1
python3: provide a PYTHON3_PATH
package: add python3 support in the package infrastructure
package: allow Python packages with Python3
python, python3: enable unicodedata for host-python, needed by
setuptools
python-pyasn: use the real upstream
python-bottle: allow to build with Python 3
python-serial: allow to build with Python 3
python-pyasn: allow to build with Python 3
python-pycrypto: allow to build with Python 3 python-pysnmp{,-apps,-mibs}: allow to build with Python 3

kpa_info at yahoo.fr (1):
python3: add config directory symbolic link

 package/Config.in                                  |   4 +-
 package/pkg-python.mk                              |  29 ++--
 package/python-bottle/Config.in                    |   2 +-
 .../python-distutilscross/python-distutilscross.mk |  12 --
 package/python-pyasn/Config.in                     |   2 +-
 package/python-pyasn/python-pyasn.mk               |  12 +-
 package/python-pycrypto/Config.in                  |   2 +-
 package/python-pysnmp-apps/Config.in               |   2 +-
 package/python-pysnmp-mibs/Config.in               |   2 +-
 package/python-pysnmp/Config.in                    |   2 +-
 package/python-serial/Config.in                    |   2 +-
 package/python-setuptools/python-setuptools.mk     |  11 +-
 package/python/python.mk                           |  11 +-
 package/python3/Config.in                          |   1 +
 ...ython3-000-generate-sysconfigdata-buildir.patch | 158 ---------------------
 .../python3-001-remove-host-header-path.patch      |  36 +++++
 .../python3/python3-001-support-for-build.patch    |  68 ---------
 .../python3/python3-002-no-host-headers-libs.patch |  72 ----------
 ...thon3-002-properly-detect-if-python-build.patch |  23 +++
 .../python3/python3-003-staging-header-libs.patch  |  41 ------
 ...ython3-003-sysconfigdata-install-location.patch |  76 ++++++++++
 ...ython3-004-no-import-when-cross-compiling.patch |  27 ----
 ...he.patch => python3-004-old-stdlib-cache.patch} |  39 +++--
 .../python3-005-do-not-generate-pyo-files.patch    |  40 ------
 .../python3/python3-005-pyc-pyo-conditional.patch  |  60 ++++++++
 ...=> python3-006-cross-compile-getaddrinfo.patch} |   8 +-
 .../python3/python3-006-reread-environment.patch   |  63 --------
 .../python3-007-change-pyconfig-h-location.patch   |  76 ----------
 ....patch => python3-007-disable-extensions.patch} |  44 +++---
 ...008-distutils-sysconfig-use-sysconfigdata.patch |  58 ++++++++
 package/python3/python3-008-no-rpath.patch         |  51 -------
 .../python3-009-distutils-use-python-sysroot.patch |  57 ++++++++
 .../python3/python3-009-verbose-module-build.patch |  26 ----
 ...3-010-distutils-cross-compilation-support.patch |  71 ---------
 .../python3/python3-010-no-termcap-host-path.patch |  22 +++
 .../python3-100-optional-test-modules.patch        |  60 ++++----
 package/python3/python3-101-optional-pydoc.patch   |  46 +++---
 package/python3/python3-102-optional-2to3.patch    |  68 +++++----
 package/python3/python3-103-optional-sqlite.patch  |  53 ++++---
 package/python3/python3-104-optional-tk.patch      |  51 ++++---
 package/python3/python3-105-optional-curses.patch  |  30 ++--
 package/python3/python3-106-optional-expat.patch   |  36 ++---
 .../python3/python3-107-optional-codecs-cjk.patch  |   8 +-
 package/python3/python3-108-optional-nis.patch     |   8 +-
 .../python3/python3-109-optional-unicodedata.patch |   8 +-
 package/python3/python3-110-optional-idle.patch    |  39 +++--
 package/python3/python3.mk                         | 102 ++++++-------
 47 files changed, 671 insertions(+), 1048 deletions(-)
 delete mode 100644 package/python-distutilscross/python-distutilscross.mk
 delete mode 100644 package/python3/python3-000-generate-sysconfigdata-buildir.patch
 create mode 100644 package/python3/python3-001-remove-host-header-path.patch
 delete mode 100644 package/python3/python3-001-support-for-build.patch
 delete mode 100644 package/python3/python3-002-no-host-headers-libs.patch
 create mode 100644 package/python3/python3-002-properly-detect-if-python-build.patch
 delete mode 100644 package/python3/python3-003-staging-header-libs.patch
 create mode 100644 package/python3/python3-003-sysconfigdata-install-location.patch
 delete mode 100644 package/python3/python3-004-no-import-when-cross-compiling.patch
 rename package/python3/{python3-112-old-stdlib-cache.patch => python3-004-old-stdlib-cache.patch} (61%)
 delete mode 100644 package/python3/python3-005-do-not-generate-pyo-files.patch
 create mode 100644 package/python3/python3-005-pyc-pyo-conditional.patch
 rename package/python3/{python3-011-cross-compile-getaddrinfo.patch => python3-006-cross-compile-getaddrinfo.patch} (81%)
 delete mode 100644 package/python3/python3-006-reread-environment.patch
 delete mode 100644 package/python3/python3-007-change-pyconfig-h-location.patch
 rename package/python3/{python3-012-disable-extensions.patch => python3-007-disable-extensions.patch} (78%)
 create mode 100644 package/python3/python3-008-distutils-sysconfig-use-sysconfigdata.patch
 delete mode 100644 package/python3/python3-008-no-rpath.patch
 create mode 100644 package/python3/python3-009-distutils-use-python-sysroot.patch
 delete mode 100644 package/python3/python3-009-verbose-module-build.patch
 delete mode 100644 package/python3/python3-010-distutils-cross-compilation-support.patch
 create mode 100644 package/python3/python3-010-no-termcap-host-path.patch
Signature · 1 line
1.8.3.2
#1 lore raw

PATCH v2 1/20 pkg-python: no longer use distutilscross for setuptools

Thanks to the bump of Python 2.x, distutilscross is no longer needed to achieve cross-compilation for setuptools packages. The host Python 2.x interpreter can be tricked into using the target compiler thanks to pointing it to a different sysconfigdata module, which is achieved using PYTHON_PATH.

---
 package/pkg-python.mk | 21 ++++++++-------------
 1 file changed, 8 insertions(+), 13 deletions(-)
package/pkg-python.mk +8 -13
index a862895..11ad75d 100644
@@ -114,7 +114,7 @@ endif
 else ifeq ($$($(2)_SETUP_TYPE),setuptools)
 ifeq ($(4),target)
 $(2)_BASE_ENV         = $$(PKG_PYTHON_SETUPTOOLS_ENV)
-$(2)_BASE_BUILD_TGT   = build -x
+$(2)_BASE_BUILD_TGT   = build
 $(2)_BASE_BUILD_OPT   =
 $(2)_BASE_INSTALL_OPT = $$(PKG_PYTHON_SETUPTOOLS_INSTALL_OPT)
 else
@@ -133,13 +133,12 @@ endif
 # front of the dependencies.
 #
 # However it must be repeated from inner-generic-package, as we need
-# to exclude the python, host-python, host-python-setuptools and
-# host-distutilscross packages, which are added below in the list of
-# dependencies depending on the package characteristics, and shouldn't
-# be derived automatically from the dependencies of the corresponding
-# target package. For example, target packages need
-# host-python-distutilscross, but not host packages.
-$(2)_DEPENDENCIES ?= $(filter-out host-python host-python-setuptools host-python-distutilscross host-toolchain $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
+# to exclude the python, host-python and host-python-setuptools
+# packages, which are added below in the list of dependencies
+# depending on the package characteristics, and shouldn't be derived
+# automatically from the dependencies of the corresponding target
+# package.
+$(2)_DEPENDENCIES ?= $(filter-out host-python host-python-setuptools host-toolchain $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
 
 # Target packages need both the python interpreter on the target (for
 # runtime) and the python interpreter on the host (for
@@ -152,17 +151,13 @@ $(2)_DEPENDENCIES += host-python
 endif
 
 # Setuptools based packages will need host-python-setuptools (both
-# host and target) and host-python-distutilscross (only target
-# packages). We need to have a special exclusion for the
+# host and target). We need to have a special exclusion for the
 # host-setuptools package itself: it is setuptools-based, but
 # shouldn't depend on host-setuptools (because it would otherwise
 # depend on itself!).
 ifeq ($$($(2)_SETUP_TYPE),setuptools)
 ifneq ($(2),HOST_PYTHON_SETUPTOOLS)
 $(2)_DEPENDENCIES += host-python-setuptools
-ifeq ($(4),target)
-$(2)_DEPENDENCIES += host-python-distutilscross
-endif
 endif
 endif
 
Signature · 1 line
1.8.3.2
#2 lore raw

PATCH v2 1/20 pkg-python: no longer use distutilscross for setuptools

Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thanks to the bump of Python 2.x, distutilscross is no longer needed to achieve cross-compilation for setuptools packages. The host Python 2.x interpreter can be tricked into using the target compiler thanks to pointing it to a different sysconfigdata module, which is achieved using PYTHON_PATH.

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#3 lore raw

PATCH v2 2/20 python-distutilscross: remove package that is no longer needed

In reply to Thomas Petazzoni

Thanks to the previous commit that makes distutilscross unecessary for setuptools packages, the host-distutilscross package can now be removed. There is no need for any Config.in.legacy handling, since there is no target variant, or visible Config.in option for host-distutilscross.

---
 package/python-distutilscross/python-distutilscross.mk | 12 ------------
 1 file changed, 12 deletions(-)
 delete mode 100644 package/python-distutilscross/python-distutilscross.mk
package/python-distutilscross/python-distutilscross.mk deleted +0 -12
deleted file mode 100644
index cfe2811..0000000
@@ -1,12 +0,0 @@
-################################################################################
-#
-# python-distutilscross
-#
-################################################################################
-
-PYTHON_DISTUTILSCROSS_VERSION = 0.1
-PYTHON_DISTUTILSCROSS_SOURCE  = distutilscross-$(PYTHON_DISTUTILSCROSS_VERSION).tar.gz
-PYTHON_DISTUTILSCROSS_SITE    = http://pypi.python.org/packages/source/d/distutilscross
-PYTHON_DISTUTILSCROSS_SETUP_TYPE = setuptools
-
-$(eval $(host-python-package))
Signature · 1 line
1.8.3.2
#4 lore raw

PATCH v2 2/20 python-distutilscross: remove package that is no longer needed

Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Thanks to the previous commit that makes distutilscross unecessary for setuptools packages, the host-distutilscross package can now be removed. There is no need for any Config.in.legacy handling, since there is no target variant, or visible Config.in option for host-distutilscross.

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#5 lore raw

PATCH v2 3/20 python3: removal of *.py/*.pyc is now done globally

In reply to Thomas Petazzoni
---
 package/python3/python3.mk | 21 ---------------------
 1 file changed, 21 deletions(-)
package/python3/python3.mk +0 -21
index b5e9689..d49fe48 100644
@@ -159,26 +159,5 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
 endif
 
-ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y)
-define PYTHON3_REMOVE_MODULES_FILES
-	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) \
-		 -name __pycache__` ; do \
-		rm -rf $$i ; \
-	done
-endef
-endif
-
-ifeq ($(BR2_PACKAGE_PYTHON3_PYC_ONLY),y)
-define PYTHON3_REMOVE_MODULES_FILES
-	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) \
-		 -name *.py` ; do \
-		rm -f $$i ; \
-	done
-endef
-endif
-
-PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_REMOVE_MODULES_FILES
-
-
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
Signature · 1 line
1.8.3.2
#7 lore raw

PATCH v2 4/20 python3: make it exclusive from python

In reply to Thomas Petazzoni

In Buildroot, we do not support installing both Python 2.x and Python 3.x on the target.

---
 package/python3/Config.in | 1 +
 1 file changed, 1 insertion(+)
package/python3/Config.in +1 -0
index 6580cfd..12928f6 100644
@@ -1,5 +1,6 @@
 config BR2_PACKAGE_PYTHON3
 	bool "python3"
+	depends on !BR2_PACKAGE_PYTHON
 	depends on BR2_USE_WCHAR
 	# uses fork()
 	depends on BR2_USE_MMU
Signature · 1 line
1.8.3.2
#8 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Quoted 6 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

In Buildroot, we do not support installing both Python 2.x and Python 3.x on the target.

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#9 lore raw

PATCH v2 4/20 python3: make it exclusive from python

In reply to Thomas Petazzoni

Am Tue, 18 Feb 2014 21:39:57 +0100 schrieb Thomas Petazzoni:

Quoted 2 lines

In Buildroot, we do not support installing both Python 2.x and Python 3.x on the target.

Hi,

sorry for warming up this old commit but I would like to ask whether it is possible at all to allow python2 and python3 together on the target?

The topic was discussed on the python mailinglist (https://
mail.python.org/pipermail/python-list/2011-March/600724.html) and it did not mention any problems.

I need python2 for Kodi, afaics[1] there are no current plans to support python3.

For a new function[2] I would like to add to my buildroot-based system I need python3 on the target as well.

Regards, Bernd

[1] http://forum.kodi.tv/showthread.php?tid=216598
[2] https://github.com/itkach/slob/

#10 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Hello,

Quoted 2 lines · Bernd Kuhls

sorry for warming up this old commit but I would like to ask whether it is possible at all to allow python2 and python3 together on the target?

No, this is not possible, because we have no way in kconfig to say which Python modules should be installed for python2, and which Python modules should be installed for python3.

So while having both interpreters and the Python standard library side-by-side would work, it doesn't work for all the external Python modules we have.

Quoted 2 lines

I need python2 for Kodi, afaics[1] there are no current plans to support python3.

I think that's the main problem. Staying on python2 only is really not a wise choice.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#11 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Thomas, Bernd, All,

On 2016-10-05 21:30 +0200, Thomas Petazzoni spake thusly:

Quoted 11 lines · Bernd Kuhls
Quoted 2 lines

sorry for warming up this old commit but I would like to ask whether it is possible at all to allow python2 and python3 together on the target?

No, this is not possible, because we have no way in kconfig to say which Python modules should be installed for python2, and which Python modules should be installed for python3.

So while having both interpreters and the Python standard library side-by-side would work, it doesn't work for all the external Python modules we have.

For what is worth, I've also been thinking on how we could solve this in Buildroot. We've discussed this on IRC (with Romain or Samuel, I can't remember). Here's what we came up with...

First, as Thomas said, we'd need to have the kcofnig-side sorted out. When looking only at external python modules for themselves, it is not very complex: we already have some modules depend on either python xor python3 for those modules that have an explicit dependency.

Where it gets more complex is when a package needs a modules that works only with python and another module that works only with python3. I don't know if this situation is realistic, but we have to at least take it in consideration.

Then, there is the problem of building modules for either or both of the pythons, when those modules need compiling (e.g. are not written entirely in python, but use C or whatever that needs compiling).

When a module is only python xor python3, this is realtively easy; we just need to do a single build of that module. It does not change much from what we currently have for that module.

But then, when the module is compatible with both python and python3, then we need to build it twice, which is currently not supported by our python-package infra, and is definitely not trivial to do.

Finally, comes the installation phase. Again, modules that are python xor python3, this is mostly status-quo with what we currently have.

Modules that are compatible with both python and python3 would need to be installed for both. Again, our python-package infra does not support that, and it is far from trivial to do.

I've started looking into this, but I have absolutely no code to show. I think it *might* be possible to come up with something, but that may probably add quite a bit of complexity and uglyness in the python-package infra. I'm not sure it would be worth the effort... :-/

Quoted 5 lines
Quoted 2 lines

I need python2 for Kodi, afaics[1] there are no current plans to support python3.

I think that's the main problem. Staying on python2 only is really not a wise choice.

Yes, this is a real pain... :-(

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#12 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Hello,

Quoted 9 lines · Yann E. MORIN

First, as Thomas said, we'd need to have the kcofnig-side sorted out. When looking only at external python modules for themselves, it is not very complex: we already have some modules depend on either python xor python3 for those modules that have an explicit dependency.

Where it gets more complex is when a package needs a modules that works only with python and another module that works only with python3. I don't know if this situation is realistic, but we have to at least take it in consideration.

I don't understand how you solve the Kconfig side of the problem. How will you select which of the modules should be compiled for python2 only, for python3 only or for both ?

Let's say you have three Python packages A, B and C. They are all compatible with both Python 2 and Python 3. And for the purpose of my system, I need A only with Python 2, I need B only for Python 3 and I need C for both Python 2 and Python 3.

Quoted 4 lines

I've started looking into this, but I have absolutely no code to show. I think it *might* be possible to come up with something, but that may probably add quite a bit of complexity and uglyness in the python-package infra. I'm not sure it would be worth the effort... :-/

Please explain how to solve the Kconfig problem first. Until there is no solution for the Kconfig problem, there is really no point in even thinking about the build problems.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#13 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Thomas, All,

On 2016-10-08 14:51 +0200, Thomas Petazzoni spake thusly:

Quoted 11 lines · Yann E. MORIN
Quoted 9 lines

First, as Thomas said, we'd need to have the kcofnig-side sorted out. When looking only at external python modules for themselves, it is not very complex: we already have some modules depend on either python xor python3 for those modules that have an explicit dependency.

Where it gets more complex is when a package needs a modules that works only with python and another module that works only with python3. I don't know if this situation is realistic, but we have to at least take it in consideration.

Quoted 8 lines

I don't understand how you solve the Kconfig side of the problem. How will you select which of the modules should be compiled for python2 only, for python3 only or for both ?

Let's say you have three Python packages A, B and C. They are all compatible with both Python 2 and Python 3. And for the purpose of my system, I need A only with Python 2, I need B only for Python 3 and I need C for both Python 2 and Python 3.

Plain and simple: if a python module supports both python and python3, and both are enabled, that python module is built for both python and python3.

If for your system you only need A for python and B for python3, then it is *your* responsibility to remove A for python3 and B for python in a post-build script. That's what we've been advertising in similar cases when people only wanted a subset of a package, and what you've been suggesting when we were arguing about the libudev stuff (i.e. build the full eudev and get rid of the udevd program in a post-build script). ;-)

Quoted 8 lines
Quoted 4 lines

I've started looking into this, but I have absolutely no code to show. I think it *might* be possible to come up with something, but that may probably add quite a bit of complexity and uglyness in the python-package infra. I'm not sure it would be worth the effort... :-/

Please explain how to solve the Kconfig problem first. Until there is no solution for the Kconfig problem, there is really no point in even thinking about the build problems.

I don't think there is a "kconfig problem" to start with. ;-)

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#14 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Hello,

Quoted 3 lines · Yann E. MORIN

Plain and simple: if a python module supports both python and python3, and both are enabled, that python module is built for both python and python3.

Well, that's horrible from a filesystem size point of view, and I'm not really sure this would solve Bernd's issue. I'm pretty sure Bernd would like to have a given set of modules built for Python 2, and another set of modules built for Python 3.

Quoted 6 lines

If for your system you only need A for python and B for python3, then it is *your* responsibility to remove A for python3 and B for python in a post-build script. That's what we've been advertising in similar cases when people only wanted a subset of a package, and what you've been suggesting when we were arguing about the libudev stuff (i.e. build the full eudev and get rid of the udevd program in a post-build script). ;-)

Well, yes, you could see it this way.

But on the eudev thing, you still get it wrong: having libudev installed separately from udevd is not an option offered by upstream. You had to hack all around the place to make this possible.

Building Python modules with just python2 or just python3 is perfectly fine from upstream's point of view.

So please don't confuse things :)

Quoted 5 lines
Quoted 3 lines

Please explain how to solve the Kconfig problem first. Until there is no solution for the Kconfig problem, there is really no point in even thinking about the build problems.

I don't think there is a "kconfig problem" to start with. ;-)

With your solution, indeed. However, I'm not sure it is worth the hassle supporting this mechanism. But it can certainly be done.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#15 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Hi Thomas, hi Yann,

Am Sat, 08 Oct 2016 16:02:28 +0200 schrieb Thomas Petazzoni:

Quoted 8 lines
Quoted 3 lines

Plain and simple: if a python module supports both python and python3, and both are enabled, that python module is built for both python and python3.

Well, that's horrible from a filesystem size point of view, and I'm not really sure this would solve Bernd's issue. I'm pretty sure Bernd would like to have a given set of modules built for Python 2, and another set of modules built for Python 3.

actually I solved the dependency problem by copying python2 and python3 to my target system ;) This is possible for me because I do not use a filesystem created by buildroot but use files from output/target/ stored as optional packages for the fli4l project, which then creates the filesystem image for the target machine by including these packages.

To achieve this I am building with one .config which has python2 + kodi/ mwscrape and later another .config build is done with python3 + mwscrape2slob.

To clarify:
mwscrape[1] downloads articles from mediawiki-based sites, stores them on a couchdb server and has runtime dependencies to the following python
modules: couchdb, futures, mwclient and pylru. Because mwclient is python2-only so is mwscrape.

mwscrape2slob[2] downloads the stored mediawiki articles from the couchdb server and creates files in .slob format for use in the Android app aard2 [3], which is a offline-reader. It depends on the following python
modules: couchdb, cssselect, cssutils, lxml and slob. The modules slob and mwscrape2slob are python3-only.

Luckily no module with a python version restriction depends on a module with a different python version restriction :)

So yes, there are some modules which I only need in python2, others only in python3, for this purpose I am using two different .config files and build python2 and python3 seperately. Filesystem space is not a problem for me, my target system is not an embedded device, therefore my solution may not be suitable for the general use in the buildroot project.

I expect to send the mwscrape2slob patch series in the coming days after testing a self-build .slob file. Currently I am busy with downloading Wikipedia data ;) The CouchDB server built with buildroot runs rock-solid.

Regards, Bernd

[1] https://github.com/itkach/mwscrape
[2] https://github.com/itkach/mwscrape2slob
[3] https://github.com/itkach/aard2-android

#16 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Quoted 23 lines · Bernd Kuhls

Hi Thomas, hi Yann,

Am Sat, 08 Oct 2016 16:02:28 +0200 schrieb Thomas Petazzoni:

Quoted 8 lines
Quoted 3 lines

Plain and simple: if a python module supports both python and python3, and both are enabled, that python module is built for both python and python3.

Well, that's horrible from a filesystem size point of view, and I'm not really sure this would solve Bernd's issue. I'm pretty sure Bernd would like to have a given set of modules built for Python 2, and another set of modules built for Python 3.

actually I solved the dependency problem by copying python2 and python3 to my target system ;) This is possible for me because I do not use a filesystem created by buildroot but use files from output/target/ stored as optional packages for the fli4l project, which then creates the filesystem image for the target machine by including these packages.

To achieve this I am building with one .config which has python2 + kodi/ mwscrape and later another .config build is done with python3 + mwscrape2slob.

Quoted 5 lines

To clarify:
mwscrape[1] downloads articles from mediawiki-based sites, stores them on a couchdb server and has runtime dependencies to the following python
modules: couchdb, futures, mwclient and pylru. Because mwclient is python2-only so is mwscrape.

isn't mwclient Python 3 compatible? See [1]

[1] https://github.com/mwclient/mwclient/blob/master/setup.py#L48

Yegor

Quoted 29 lines

mwscrape2slob[2] downloads the stored mediawiki articles from the couchdb server and creates files in .slob format for use in the Android app aard2 [3], which is a offline-reader. It depends on the following python
modules: couchdb, cssselect, cssutils, lxml and slob. The modules slob and mwscrape2slob are python3-only.

Luckily no module with a python version restriction depends on a module with a different python version restriction :)

So yes, there are some modules which I only need in python2, others only in python3, for this purpose I am using two different .config files and build python2 and python3 seperately. Filesystem space is not a problem for me, my target system is not an embedded device, therefore my solution may not be suitable for the general use in the buildroot project.

I expect to send the mwscrape2slob patch series in the coming days after testing a self-build .slob file. Currently I am busy with downloading Wikipedia data ;) The CouchDB server built with buildroot runs rock-solid.

Regards, Bernd

[1] https://github.com/itkach/mwscrape
[2] https://github.com/itkach/mwscrape2slob
[3] https://github.com/itkach/aard2-android

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#17 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Am Sat, 08 Oct 2016 20:23:09 +0200 schrieb Yegor Yefremov:

Quoted 3 lines

isn't mwclient Python 3 compatible? See [1]

[1] https://github.com/mwclient/mwclient/blob/master/setup.py#L48

Hi Yegor,

using this defconfig

BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_MWCLIENT=y

this error occurs with plain make:
-------------------------------------------------

Quoted 1 line
Quoted 1 line
Quoted 1 line

Finalizing target directory

PYTHONPATH="/home/buildroot/br3_couchdb/output/target/usr/lib/python3.5/ sysconfigdata/:/home/buildroot/br3_couchdb/output/target/usr/lib/ python3.5/site-packages/" /home/buildroot/br3_couchdb/output/host/usr/bin/ python3.5 support/scripts/pycompile.py /home/buildroot/br3_couchdb/output/ target/usr/lib/python3.5
Traceback (most recent call last):
File "/home/buildroot/br3_couchdb/output/host/usr/lib/python3.5/
py_compile.py", line 125, in compile
_optimize=optimize)
File "<frozen importlib._bootstrap_external>", line 700, in
source_to_code
File "<frozen importlib._bootstrap>", line 222, in
_call_with_frames_removed
File "/home/buildroot/br3_couchdb/output/target/usr/lib/python3.5/site-
packages/mwclient/ex.py", line 15
exec _file in globals(), predata
^
SyntaxError: invalid syntax
-------------------------------------------------
This error does not occur with python2.

Regards, Bernd

#18 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Quoted 29 lines · Bernd Kuhls

Am Sat, 08 Oct 2016 20:23:09 +0200 schrieb Yegor Yefremov:

Quoted 3 lines

isn't mwclient Python 3 compatible? See [1]

[1] https://github.com/mwclient/mwclient/blob/master/setup.py#L48

Hi Yegor,

using this defconfig

BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_MWCLIENT=y

this error occurs with plain make:
-------------------------------------------------

Quoted 1 line
Quoted 1 line
Quoted 1 line

Finalizing target directory

PYTHONPATH="/home/buildroot/br3_couchdb/output/target/usr/lib/python3.5/ sysconfigdata/:/home/buildroot/br3_couchdb/output/target/usr/lib/ python3.5/site-packages/" /home/buildroot/br3_couchdb/output/host/usr/bin/ python3.5 support/scripts/pycompile.py /home/buildroot/br3_couchdb/output/ target/usr/lib/python3.5
Traceback (most recent call last):
File "/home/buildroot/br3_couchdb/output/host/usr/lib/python3.5/
py_compile.py", line 125, in compile
_optimize=optimize)
File "<frozen importlib._bootstrap_external>", line 700, in
source_to_code
File "<frozen importlib._bootstrap>", line 222, in

Quoted 8 lines

_call_with_frames_removed
File "/home/buildroot/br3_couchdb/output/target/usr/lib/python3.5/site-
packages/mwclient/ex.py", line 15
exec _file in globals(), predata
^
SyntaxError: invalid syntax
-------------------------------------------------
This error does not occur with python2.

OK. I'll try to take a look at this issue.

Yegor

#19 lore raw

PATCH v2 4/20 python3: make it exclusive from python

Quoted 39 lines · Yegor Yefremov
Quoted 29 lines · Bernd Kuhls

Am Sat, 08 Oct 2016 20:23:09 +0200 schrieb Yegor Yefremov:

Quoted 3 lines

isn't mwclient Python 3 compatible? See [1]

[1] https://github.com/mwclient/mwclient/blob/master/setup.py#L48

Hi Yegor,

using this defconfig

BR2_TOOLCHAIN_BUILDROOT_WCHAR=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_MWCLIENT=y

this error occurs with plain make:
-------------------------------------------------

Quoted 1 line
Quoted 1 line
Quoted 1 line

Finalizing target directory

PYTHONPATH="/home/buildroot/br3_couchdb/output/target/usr/lib/python3.5/ sysconfigdata/:/home/buildroot/br3_couchdb/output/target/usr/lib/ python3.5/site-packages/" /home/buildroot/br3_couchdb/output/host/usr/bin/ python3.5 support/scripts/pycompile.py /home/buildroot/br3_couchdb/output/ target/usr/lib/python3.5
Traceback (most recent call last):
File "/home/buildroot/br3_couchdb/output/host/usr/lib/python3.5/
py_compile.py", line 125, in compile
_optimize=optimize)
File "<frozen importlib._bootstrap_external>", line 700, in
source_to_code
File "<frozen importlib._bootstrap>", line 222, in

Quoted 8 lines

_call_with_frames_removed
File "/home/buildroot/br3_couchdb/output/target/usr/lib/python3.5/site-
packages/mwclient/ex.py", line 15
exec _file in globals(), predata
^
SyntaxError: invalid syntax
-------------------------------------------------
This error does not occur with python2.

Quoted 1 line

OK. I'll try to take a look at this issue.

Applying this upstream patch [1] solves the issue.

[1] https://github.com/mwclient/mwclient/commit/5de1bb82465d39962e26175c62f644a3e423d030

Yegor

#20 lore raw

PATCH v2 4/20 python3: make it exclusive from python

In reply to Thomas Petazzoni

Thomas, All,

On 2016-10-08 16:02 +0200, Thomas Petazzoni spake thusly:

Quoted 9 lines · Yann E. MORIN
Quoted 3 lines

Plain and simple: if a python module supports both python and python3, and both are enabled, that python module is built for both python and python3.

Well, that's horrible from a filesystem size point of view, and I'm not really sure this would solve Bernd's issue. I'm pretty sure Bernd would like to have a given set of modules built for Python 2, and another set of modules built for Python 3.

Well, short of duplicating (tripling? more?) the kconfig symbols, there is no easy solution.

But again, I would argue that, for Buildroot, we would be just happy to at least have that situation: having both python2 and python3 on the target, even with duplicated and useless python modules in some cases. That would be an improvement against the current situation.

If anything, that would make scenarii currently impossible, possible.

*Then*, we could look at solving how to remove the useless duplicated modules, if at all possible in a sane way...

Quoted 7 lines
Quoted 6 lines

If for your system you only need A for python and B for python3, then it is *your* responsibility to remove A for python3 and B for python in a post-build script. That's what we've been advertising in similar cases when people only wanted a subset of a package, and what you've been suggesting when we were arguing about the libudev stuff (i.e. build the full eudev and get rid of the udevd program in a post-build script). ;-)

Quoted 5 lines

Well, yes, you could see it this way.

But on the eudev thing, you still get it wrong: having libudev installed separately from udevd is not an option offered by upstream. You had to hack all around the place to make this possible.

Hmm... Don't get me started on this topic (woops, I did start it!). There are things not supporrted by upstream for which we patch some stuff; and in some situations, I believe this is insane. The best (worst!) example being the patch to build openssl in parallel; that one is insane and dangerous: openssl is a tricky and critical piece of code, and I'm not sure patching it with a patch that was not blessed (and was even refused!) by upstream is sane... :-/

But OK, I'll first push upstream eudev to enable building only libudev. I already started that, but time being always in short supplies, this hasn't progressed too much for now... :-/

Topic closed. ;-)

Quoted 2 lines

With your solution, indeed. However, I'm not sure it is worth the hassle supporting this mechanism. But it can certainly be done.

On contrary, I do believe this is worth the hassle.

But until I have some code, I'll close the topic... ;-)

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#21 lore raw

PATCH v2 5/20 python3: add python -> python3 symlink for the host variant

In reply to Thomas Petazzoni

The target python3 depends on host-python3, but most of the scripts call "python", so we need to ensure that $(HOST_DIR)/usr/bin/python exists. This patch achieves this by creating a python -> python3 symbolic link in $(HOST_DIR), just like we are already doing for the target Python 3.

---
 package/python3/python3.mk | 6 ++++++
 1 file changed, 6 insertions(+)
package/python3/python3.mk +6 -0
index d49fe48..ba72e75 100644
@@ -159,5 +159,11 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
 endif
 
+define HOST_PYTHON3_INSTALL_SYMLINK
+	ln -fs python3 $(HOST_DIR)/usr/bin/python
+endef
+
+HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
Signature · 1 line
1.8.3.2
#22 lore raw

PATCH v2 5/20 python3: add python -> python3 symlink for the host variant

Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

The target python3 depends on host-python3, but most of the scripts call "python", so we need to ensure that $(HOST_DIR)/usr/bin/python exists. This patch achieves this by creating a python -> python3 symbolic link in $(HOST_DIR), just like we are already doing for the target Python 3.

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#23 lore raw

PATCH v2 6/20 python3: add config directory symbolic link

In reply to Thomas Petazzoni

From: "[email protected]" <[email protected]>

When using host python to install an external module, the setup tool look for some files in python3.3/config/. The python3 package config directory is called config-3.3m. This is why we need to alias config to config-3.3m.

[Thomas: use PYTHON3_VERSION_MAJOR instead of hardcoding 3.3, update commit title.]

---
 package/python3/python3.mk | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
package/python3/python3.mk +10 -1
index ba72e75..d98ca38 100644
@@ -163,7 +163,16 @@ define HOST_PYTHON3_INSTALL_SYMLINK
 	ln -fs python3 $(HOST_DIR)/usr/bin/python
 endef
 
-HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+# Alias the python<version>/config-<version>m to python<version>/config
+# This is needed when installing an external python module, because
+# the setup is looking for files in python<version>/config/
+define HOST_PYTHON3_INSTALL_CONFIG_SYMLINK
+	ln -fs config-$(PYTHON3_VERSION_MAJOR)m $(HOST_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOE)config
+endef
+
+HOST_PYTHON3_POST_INSTALL_HOOKS += \
+	HOST_PYTHON3_INSTALL_SYMLINK \
+	HOST_PYTHON3_INSTALL_CONFIG_SYMLINK
 
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
Signature · 1 line
1.8.3.2
#24 lore raw

PATCH v2 6/20 python3: add config directory symbolic link

Quoted 22 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

From: "kpa_info at yahoo.fr" <[email protected]>
When using host python to install an external module, the setup tool look for some files in python3.3/config/. The python3 package config directory is called config-3.3m. This is why we need to alias config to config-3.3m.

[Thomas: use PYTHON3_VERSION_MAJOR instead of hardcoding 3.3, update commit title.]

---
 package/python3/python3.mk | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
package/python3/python3.mk +0 -0
index ba72e75..d98ca38 100644

package/python3/python3.mk define HOST_PYTHON3_INSTALL_SYMLINK · line 170

Quoted 9 lines
package/python3/python3.mk +5 -1
@@ -163,7 +163,16 @@ define HOST_PYTHON3_INSTALL_SYMLINK
 	ln -fs python3 $(HOST_DIR)/usr/bin/python
 endef
 
-HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+# Alias the python<version>/config-<version>m to python<version>/config
+# This is needed when installing an external python module, because
+# the setup is looking for files in python<version>/config/
+define HOST_PYTHON3_INSTALL_CONFIG_SYMLINK
+	ln -fs config-$(PYTHON3_VERSION_MAJOR)m $(HOST_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOE)config

Ehh, shouldn't this be ..$(HOST_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config?

As you didn't notice, perhaps the symlink isn't really needed after all?

Signature · 1 line
Bye, Peter Korsgaard
#25 lore raw

PATCH v2 6/20 python3: add config directory symbolic link

Dear Peter Korsgaard,

Quoted 10 lines · Peter Korsgaard

package/python3/python3.mk define HOST_PYTHON3_INSTALL_SYMLINK · line 170

Quoted 6 lines
package/python3/python3.mk +5 -1
-HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+# Alias the python<version>/config-<version>m to python<version>/config
+# This is needed when installing an external python module, because
+# the setup is looking for files in python<version>/config/
+define HOST_PYTHON3_INSTALL_CONFIG_SYMLINK
+	ln -fs config-$(PYTHON3_VERSION_MAJOR)m $(HOST_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOE)config

Ehh, shouldn't this be ..$(HOST_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config?

As you didn't notice, perhaps the symlink isn't really needed after all?

I tested without this patch, and indeed, I could build Python 3 and all the external modules that can be enabled with Python 3 without problem. So I believe it may no longer be necessary with the Python 3 bump to 3.4.

I have tested, and patches 7 onwards apply without problem if you discard PATCH 6. So I guess there is no need for me to resend the series, you can just continue to apply it starting from PATCH 7, if you don't mind.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#26 lore raw

PATCH v2 6/20 python3: add config directory symbolic link

Quoted 11 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Ehh, shouldn't this be ..$(HOST_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config?

As you didn't notice, perhaps the symlink isn't really needed after all?

I tested without this patch, and indeed, I could build Python 3 and all the external modules that can be enabled with Python 3 without problem. So I believe it may no longer be necessary with the Python 3 bump to 3.4.

Quoted 4 lines

I have tested, and patches 7 onwards apply without problem if you discard PATCH 6. So I guess there is no need for me to resend the series, you can just continue to apply it starting from PATCH 7, if you don't mind.

Ok, thanks. Will do so.

Signature · 1 line
Bye, Peter Korsgaard
#27 lore raw

PATCH v2 7/20 python3: use proper PYTHON3_VERSION_MAJOR instead of harcoding 3.3

In reply to Thomas Petazzoni

Some parts of python3.mk were hardcoding the 3.3 version as the major version, which does not work for Python 3.4 and other future versions. Instead, use the existing PYTHON3_VERSION_MAJOR.

---
 package/python3/python3.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/python3/python3.mk +2 -2
index d98ca38..477bdfb 100644
@@ -127,7 +127,7 @@ PYTHON3_MAKE_ENV = \
 # trouble for cross compilation
 define PYTHON3_FIXUP_LIBDIR
 	$(SED) 's|^LIBDIR=.*|LIBDIR= $(STAGING_DIR)/usr/lib|' \
-	   $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-3.3m/Makefile
+	   $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)m/Makefile
 endef
 
 PYTHON3_POST_INSTALL_STAGING_HOOKS += PYTHON3_FIXUP_LIBDIR
@@ -141,7 +141,7 @@ define PYTHON3_REMOVE_USELESS_FILES
 	rm -f $(TARGET_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR)m-config
 	rm -f $(TARGET_DIR)/usr/bin/python3-config
 	rm -f $(TARGET_DIR)/usr/bin/smtpd.py.3
-	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-3.3m/ \
+	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/config-$(PYTHON3_VERSION_MAJOR)m/ \
 		-type f -not -name pyconfig.h -a -not -name Makefile` ; do \
 		rm -f $$i ; \
 	done
Signature · 1 line
1.8.3.2
#28 lore raw

PATCH v2 7/20 python3: use proper PYTHON3_VERSION_MAJOR instead of harcoding 3.3

Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Some parts of python3.mk were hardcoding the 3.3 version as the major version, which does not work for Python 3.4 and other future versions. Instead, use the existing PYTHON3_VERSION_MAJOR.

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#29 lore raw

PATCH v2 8/20 python3: bump to 3.4.0rc1

In reply to Thomas Petazzoni

This commit bumps the Python3 package to use Python 3.4.0rc1.

About the patches:

* The patches below 100 are significantly changed, because like for Python 2.x, a good number of improvements have been made in the upstream Python for cross-compilation. Therefore, almost all of these patches have been modified.

* All the patches above 100 are simply updated for Python 3.4.0, with a small refactoring for the handling of test modules.

The details of the python3.mk changes are:

* --without-ensurepip to tell Python to not use PIP and build time.

* Many environment variables are no longer passed, they were specific to our cross-compilation patches

* The fixup of the LIBDIR in the Python Makefile is no longer needed since Python has switched to _sysconfigdata.py for distutils configuration instead of parsing the Makefile.

* A new post patch hooks touches the two files generated by pgen to make sure they are newer than the pgen sources, which ensures pgen is not built/executed.

---
 ...ython3-000-generate-sysconfigdata-buildir.patch | 158 ---------------------
 .../python3-001-remove-host-header-path.patch      |  36 +++++
 .../python3/python3-001-support-for-build.patch    |  68 ---------
 .../python3/python3-002-no-host-headers-libs.patch |  72 ----------
 ...thon3-002-properly-detect-if-python-build.patch |  23 +++
 .../python3/python3-003-staging-header-libs.patch  |  41 ------
 ...ython3-003-sysconfigdata-install-location.patch |  76 ++++++++++
 ...ython3-004-no-import-when-cross-compiling.patch |  27 ----
 ...he.patch => python3-004-old-stdlib-cache.patch} |  39 +++--
 .../python3-005-do-not-generate-pyo-files.patch    |  40 ------
 .../python3/python3-005-pyc-pyo-conditional.patch  |  60 ++++++++
 ...=> python3-006-cross-compile-getaddrinfo.patch} |   8 +-
 .../python3/python3-006-reread-environment.patch   |  63 --------
 .../python3-007-change-pyconfig-h-location.patch   |  76 ----------
 ....patch => python3-007-disable-extensions.patch} |  44 +++---
 ...008-distutils-sysconfig-use-sysconfigdata.patch |  58 ++++++++
 package/python3/python3-008-no-rpath.patch         |  51 -------
 .../python3-009-distutils-use-python-sysroot.patch |  57 ++++++++
 .../python3/python3-009-verbose-module-build.patch |  26 ----
 ...3-010-distutils-cross-compilation-support.patch |  71 ---------
 .../python3/python3-010-no-termcap-host-path.patch |  22 +++
 .../python3-100-optional-test-modules.patch        |  60 ++++----
 package/python3/python3-101-optional-pydoc.patch   |  46 +++---
 package/python3/python3-102-optional-2to3.patch    |  68 +++++----
 package/python3/python3-103-optional-sqlite.patch  |  53 ++++---
 package/python3/python3-104-optional-tk.patch      |  51 ++++---
 package/python3/python3-105-optional-curses.patch  |  30 ++--
 package/python3/python3-106-optional-expat.patch   |  36 ++---
 .../python3/python3-107-optional-codecs-cjk.patch  |   8 +-
 package/python3/python3-108-optional-nis.patch     |   8 +-
 .../python3/python3-109-optional-unicodedata.patch |   8 +-
 package/python3/python3-110-optional-idle.patch    |  39 +++--
 package/python3/python3.mk                         |  62 +++-----
 33 files changed, 610 insertions(+), 975 deletions(-)
 delete mode 100644 package/python3/python3-000-generate-sysconfigdata-buildir.patch
 create mode 100644 package/python3/python3-001-remove-host-header-path.patch
 delete mode 100644 package/python3/python3-001-support-for-build.patch
 delete mode 100644 package/python3/python3-002-no-host-headers-libs.patch
 create mode 100644 package/python3/python3-002-properly-detect-if-python-build.patch
 delete mode 100644 package/python3/python3-003-staging-header-libs.patch
 create mode 100644 package/python3/python3-003-sysconfigdata-install-location.patch
 delete mode 100644 package/python3/python3-004-no-import-when-cross-compiling.patch
 rename package/python3/{python3-112-old-stdlib-cache.patch => python3-004-old-stdlib-cache.patch} (61%)
 delete mode 100644 package/python3/python3-005-do-not-generate-pyo-files.patch
 create mode 100644 package/python3/python3-005-pyc-pyo-conditional.patch
 rename package/python3/{python3-011-cross-compile-getaddrinfo.patch => python3-006-cross-compile-getaddrinfo.patch} (81%)
 delete mode 100644 package/python3/python3-006-reread-environment.patch
 delete mode 100644 package/python3/python3-007-change-pyconfig-h-location.patch
 rename package/python3/{python3-012-disable-extensions.patch => python3-007-disable-extensions.patch} (78%)
 create mode 100644 package/python3/python3-008-distutils-sysconfig-use-sysconfigdata.patch
 delete mode 100644 package/python3/python3-008-no-rpath.patch
 create mode 100644 package/python3/python3-009-distutils-use-python-sysroot.patch
 delete mode 100644 package/python3/python3-009-verbose-module-build.patch
 delete mode 100644 package/python3/python3-010-distutils-cross-compilation-support.patch
 create mode 100644 package/python3/python3-010-no-termcap-host-path.patch
package/python3/python3-000-generate-sysconfigdata-buildir.patch deleted +0 -158
deleted file mode 100644
index a29aa0f..0000000
@@ -1,158 +0,0 @@
-changeset:   79745:f85c3f4d9b98
-parent:      79743:36b2ca7dc893
-parent:      79744:24d52d3060e8
-user:        Trent Nelson <[email protected]>
-date:        Tue Oct 16 08:17:11 2012 -0400
-summary:     Merge issue #15298: ensure _sysconfigdata is generated in build directory,
-
-Taken from upstream.
-
----
- Lib/sysconfig.py |   11 ++++++++++-
- Makefile.pre.in  |   24 +++++++++++++-----------
- setup.py         |   14 --------------
- 3 files changed, 23 insertions(+), 26 deletions(-)
-
-Index: Python-3.3.0/Lib/sysconfig.py
-===================================================================
---- Python-3.3.0.orig/Lib/sysconfig.py
-+++ Python-3.3.0/Lib/sysconfig.py
-@@ -390,13 +390,22 @@
-     if _PYTHON_BUILD:
-         vars['LDSHARED'] = vars['BLDSHARED']
- 
--    destfile = os.path.join(os.path.dirname(__file__), '_sysconfigdata.py')
-+    pybuilddir = 'build/lib.%s-%s' % (get_platform(), sys.version[:3])
-+    if hasattr(sys, "gettotalrefcount"):
-+        pybuilddir += '-pydebug'
-+    os.makedirs(pybuilddir, exist_ok=True)
-+    destfile = os.path.join(pybuilddir, '_sysconfigdata.py')
-+
-     with open(destfile, 'w', encoding='utf8') as f:
-         f.write('# system configuration generated and used by'
-                 ' the sysconfig module\n')
-         f.write('build_time_vars = ')
-         pprint.pprint(vars, stream=f)
- 
-+    # Create file used for sys.path fixup -- see Modules/getpath.c
-+    with open('pybuilddir.txt', 'w', encoding='ascii') as f:
-+        f.write(pybuilddir)
-+
- def _init_posix(vars):
-     """Initialize the module as appropriate for POSIX systems."""
-     # _sysconfigdata is generated at build time, see _generate_posix_vars()
-Index: Python-3.3.0/Makefile.pre.in
-===================================================================
---- Python-3.3.0.orig/Makefile.pre.in
-+++ Python-3.3.0/Makefile.pre.in
-@@ -410,8 +410,6 @@
- 		Objects/unicodectype.o \
- 		Objects/weakrefobject.o
- 
--SYSCONFIGDATA=$(srcdir)/Lib/_sysconfigdata.py
--
- ##########################################################################
- # objects that get linked into the Python library
- LIBRARY_OBJS_OMIT_FROZEN=	\
-@@ -432,7 +430,7 @@
- 
- # Default target
- all:		build_all
--build_all:	$(BUILDPYTHON) $(SYSCONFIGDATA) oldsharedmods sharedmods gdbhooks Modules/_testembed
-+build_all:	$(BUILDPYTHON) oldsharedmods sharedmods gdbhooks Modules/_testembed
- 
- # Compile a binary with gcc profile guided optimization.
- profile-opt:
-@@ -466,15 +464,17 @@
- $(BUILDPYTHON):	Modules/python.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
- 	$(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Modules/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
- 
--platform: $(BUILDPYTHON) $(SYSCONFIGDATA)
-+platform: $(BUILDPYTHON)
- 	$(RUNSHARED) $(PYTHON_FOR_BUILD) -c 'import sys ; from sysconfig import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
- 
--# Generate the sysconfig build-time data
--$(SYSCONFIGDATA): $(BUILDPYTHON)
-+# Create build directory and generate the sysconfig build-time data there.
-+# pybuilddir.txt contains the name of the build dir and is used for
-+# sys.path fixup -- see Modules/getpath.c.
-+pybuilddir.txt: $(BUILDPYTHON)
- 	$(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars
- 
- # Build the shared modules
--sharedmods: $(BUILDPYTHON) $(SYSCONFIGDATA)
-+sharedmods: $(BUILDPYTHON) pybuilddir.txt
- 	case $$MAKEFLAGS in *s*) quiet=-q; esac; \
- 	$(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' \
- 		$(PYTHON_FOR_BUILD) $(srcdir)/setup.py $$quiet build
-@@ -1036,7 +1036,7 @@
- 		else	true; \
- 		fi; \
- 	done
--	@for i in $(srcdir)/Lib/*.py ; \
-+	@for i in $(srcdir)/Lib/*.py `cat pybuilddir.txt`/_sysconfigdata.py; \
- 	do \
- 		if test -x $$i; then \
- 			$(INSTALL_SCRIPT) $$i $(DESTDIR)$(LIBDEST); \
-@@ -1196,6 +1196,8 @@
- 		--install-scripts=$(BINDIR) \
- 		--install-platlib=$(DESTSHARED) \
- 		--root=$(DESTDIR)/
-+	-rm $(DESTDIR)$(DESTSHARED)/_sysconfigdata.py
-+	-rm -r $(DESTDIR)$(DESTSHARED)/__pycache__
- 
- # Here are a couple of targets for MacOSX again, to install a full
- # framework-based Python. frameworkinstall installs everything, the
-@@ -1341,9 +1343,10 @@
- 	find . -name '*.s[ol]' -exec rm -f {} ';'
- 	find . -name '*.so.[0-9]*.[0-9]*' -exec rm -f {} ';'
- 	find build -name 'fficonfig.h' -exec rm -f {} ';' || true
--	find build -name 'fficonfig.py' -exec rm -f {} ';' || true
-+	find build -name '*.py' -exec rm -f {} ';' || true
-+	find build -name '*.py[co]' -exec rm -f {} ';' || true
-+	-rm -f pybuilddir.txt
- 	-rm -f Lib/lib2to3/*Grammar*.pickle
--	-rm -f $(SYSCONFIGDATA)
- 	-rm -f Modules/_testembed Modules/_freeze_importlib
- 
- profile-removal:
-@@ -1367,7 +1370,6 @@
- 		Modules/Setup Modules/Setup.local Modules/Setup.config \
- 		Modules/ld_so_aix Modules/python.exp Misc/python.pc
- 	-rm -f python*-gdb.py
--	-rm -f pybuilddir.txt
- 	find $(srcdir) '(' -name '*.fdc' -o -name '*~' \
- 			   -o -name '[@,#]*' -o -name '*.old' \
- 			   -o -name '*.orig' -o -name '*.rej' \
-Index: Python-3.3.0/setup.py
-===================================================================
---- Python-3.3.0.orig/setup.py
-+++ Python-3.3.0/setup.py
-@@ -33,10 +33,6 @@
- # This global variable is used to hold the list of modules to be disabled.
- disabled_module_list = []
- 
--# File which contains the directory for shared mods (for sys.path fixup
--# when running from the build dir, see Modules/getpath.c)
--_BUILDDIR_COOKIE = "pybuilddir.txt"
--
- def add_dir_to_list(dirlist, dir):
-     """Add the directory 'dir' to the list 'dirlist' (after any relative
-     directories) if:
-@@ -252,16 +248,6 @@
-             args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags
-         self.compiler.set_executables(**args)
- 
--        # Not only do we write the builddir cookie, but we manually install
--        # the shared modules directory if it isn't already in sys.path.
--        # Otherwise trying to import the extensions after building them
--        # will fail.
--        with open(_BUILDDIR_COOKIE, "wb") as f:
--            f.write(self.build_lib.encode('utf-8', 'surrogateescape'))
--        abs_build_lib = os.path.join(os.getcwd(), self.build_lib)
--        if abs_build_lib not in sys.path:
--            sys.path.append(abs_build_lib)
--
-         build_ext.build_extensions(self)
- 
-         longest = max([len(e.name) for e in self.extensions])
package/python3/python3-001-remove-host-header-path.patch new file +36 -0
new file mode 100644
index 0000000..e159e86
@@ -0,0 +1,36 @@
+setup.py: do not add invalid header locations
+
+This piece of code incorrectly adds /usr/include to
+self.compiler.include_dirs, and results in the following invalid
+compilation line:
+
+/home/thomas/projets/buildroot/output/host/usr/bin/arm-none-linux-gnueabi-gcc
+  -fPIC -Wno-unused-result -Werror=declaration-after-statement -DNDEBUG -g
+  -O3 -Wall -Wstrict-prototypes -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
+  -D_FILE_OFFSET_BITS=64 -pipe -Os
+  -I./Include -I/usr/include -I. -IInclude
+  -I/home/thomas/projets/buildroot/output/host/usr/arm-buildroot-linux-gnueabi/sysroot/usr/include
+  -I/home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Include
+  -I/home/thomas/projets/buildroot/output/build/python3-3.4.0b1
+  -c /home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Modules/_struct.c
+  -o build/temp.linux-arm-3.4/home/thomas/projets/buildroot/output/build/python3-3.4.0b1/Modules/_struct.o
+cc1: warning: include location "/usr/include" is unsafe for cross-compilation [-Wpoison-system-directories]
+
+The -I/usr/include is wrong when cross compiling, so we disable adding
+INCLUDEDIR and LIBDIR from the host when cross compiling.
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+Index: b/setup.py
+===================================================================
+--- a/setup.py
++++ b/setup.py
+@@ -487,7 +487,8 @@
+                         add_dir_to_list(dir_list, directory)
+ 
+         if os.path.normpath(sys.base_prefix) != '/usr' \
+-                and not sysconfig.get_config_var('PYTHONFRAMEWORK'):
++                and not sysconfig.get_config_var('PYTHONFRAMEWORK') \
++                and not cross_compiling:
+             # OSX note: Don't add LIBDIR and INCLUDEDIR to building a framework
+             # (PYTHONFRAMEWORK is set) to avoid # linking problems when
+             # building a framework with different architectures than
package/python3/python3-001-support-for-build.patch deleted +0 -68

68 lines not on this page · raw message

package/python3/python3-002-no-host-headers-libs.patch deleted +0 -72

72 lines not on this page · raw message

package/python3/python3-002-properly-detect-if-python-build.patch new file +23 -0
new file mode 100644
index 0000000..ce874c6
@@ -0,0 +1,23 @@
+distutils: fix build_ext check to find whether we're building Python or not
+
+The build_ext logic uses
+sys.executable.startswith(os.path.join(sys.exec_prefix, "bin")) to
+determine whether we're building a third-party Python extension, or a
+built-in Python extension. However, this check is wrong in
+cross-compilation mode, and instead, the sysconfig.python_build
+variable should be used.
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+Index: b/Lib/distutils/command/build_ext.py
+===================================================================
+--- a/Lib/distutils/command/build_ext.py
++++ b/Lib/distutils/command/build_ext.py
+@@ -246,7 +246,7 @@
+         # Python's library directory must be appended to library_dirs
+         # See Issues: #1600860, #4366
+         if (sysconfig.get_config_var('Py_ENABLE_SHARED')):
+-            if sys.executable.startswith(os.path.join(sys.exec_prefix, "bin")):
++            if not sysconfig.python_build:
+                 # building third party extensions
+                 self.library_dirs.append(sysconfig.get_config_var('LIBDIR'))
+             else:
package/python3/python3-003-staging-header-libs.patch deleted +0 -41

41 lines not on this page · raw message

package/python3/python3-003-sysconfigdata-install-location.patch new file +76 -0

76 lines not on this page · raw message

package/python3/python3-004-no-import-when-cross-compiling.patch deleted +0 -27
deleted file mode 100644
index cd5a437..0000000
@@ -1,27 +0,0 @@
-Disable import check when cross-compiling
-
-Once Python has compiled an extension (i.e some C code, potentially
-linked to a library), it tries to import it. This cannot work in
-cross-compilation mode, so we just disable this check.
-
-Signed-off-by: Thomas Petazzoni <[email protected]>
-Signed-off-by: Maxime Ripard <[email protected]>
----
- setup.py |    4 ++++
- 1 file changed, 4 insertions(+)
-
-Index: Python-3.3.0/setup.py
-===================================================================
---- Python-3.3.0.orig/setup.py
-+++ Python-3.3.0/setup.py
-@@ -318,6 +318,10 @@
-             self.announce('WARNING: skipping import check for Cygwin-based "%s"'
-                 % ext.name)
-             return
-+        if os.environ.get('CROSS_COMPILING') == 'yes':
-+            self.announce('WARNING: skipping import check for cross compiled "%s"'
-+                          % ext.name)
-+            return
-         ext_filename = os.path.join(
-             self.build_lib,
-             self.get_ext_filename(self.get_ext_fullname(ext.name)))
package/python3/python3-004-old-stdlib-cache.patch renamed from package/python3/python3-112-old-stdlib-cache.patch +27 -12
similarity index 61%
rename from package/python3/python3-112-old-stdlib-cache.patch
rename to package/python3/python3-004-old-stdlib-cache.patch
index 5a2cf8b..ac34f8d 100644
@@ -8,11 +8,13 @@ See http://www.python.org/dev/peps/pep-3147
 
 Signed-off-by: Daniel Nelson <[email protected]>
 
---- python3-3.3.0/configure.ac	2013-01-27 16:35:52.429067797 -0800
-+++ python3-3.3.0/configure.ac	2013-01-27 16:43:43.528601443 -0800
-@@ -1827,6 +1827,23 @@
-     LIBTOOL_CRUFT=$LIBTOOL_CRUFT' -compatibility_version $(VERSION) -current_version $(VERSION)';;
- esac
+Index: b/configure.ac
+===================================================================
+--- a/configure.ac
++++ b/configure.ac
+@@ -349,6 +349,23 @@
+ AC_SUBST(FRAMEWORKUNIXTOOLSPREFIX)
+ AC_SUBST(FRAMEWORKINSTALLAPPSPREFIX)
  
 +STDLIB_CACHE_FLAGS=
 +AC_MSG_CHECKING(for --enable-old-stdlib-cache)
@@ -31,12 +33,14 @@ Signed-off-by: Daniel Nelson <[email protected]>
 +])
 +AC_SUBST(STDLIB_CACHE_FLAGS)
 +
- AC_MSG_CHECKING(for --enable-framework)
- if test "$enable_framework"
- then
---- python3-3.3.0/Makefile.pre.in	2013-01-27 16:35:52.422067642 -0800
-+++ python3-3.3.0/Makefile.pre.in	2013-01-27 16:37:03.710650439 -0800
-@@ -143,6 +143,9 @@
+ ##AC_ARG_WITH(dyld,
+ ##            AS_HELP_STRING([--with-dyld],
+ ##                           [Use (OpenStep|Rhapsody) dynamic linker]))
+Index: b/Makefile.pre.in
+===================================================================
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -157,6 +157,9 @@
  # Options to enable prebinding (for fast startup prior to Mac OS X 10.3)
  OTHER_LIBTOOL_OPT=@OTHER_LIBTOOL_OPT@
  
@@ -46,7 +50,7 @@ Signed-off-by: Daniel Nelson <[email protected]>
  # Environment to run shared python without installed libraries
  RUNSHARED=       @RUNSHARED@
  
-@@ -1079,12 +1082,12 @@
+@@ -1231,21 +1234,21 @@
  	fi
  	-PYTHONPATH=$(DESTDIR)$(LIBDEST)  $(RUNSHARED) \
  		$(PYTHON_FOR_BUILD) -Wi $(DESTDIR)$(LIBDEST)/compileall.py \
@@ -54,10 +58,21 @@ Signed-off-by: Daniel Nelson <[email protected]>
 +		-d $(LIBDEST) -f $(STDLIB_CACHE_FLAGS) \
  		-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
  		$(DESTDIR)$(LIBDEST)
+ 	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
+ 		$(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
+-		-d $(LIBDEST) -f \
++		-d $(LIBDEST) -f $(STDLIB_CACHE_FLAGS) \
+ 		-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
+ 		$(DESTDIR)$(LIBDEST)
  	-PYTHONPATH=$(DESTDIR)$(LIBDEST)  $(RUNSHARED) \
  		$(PYTHON_FOR_BUILD) -Wi $(DESTDIR)$(LIBDEST)/compileall.py \
 -		-d $(LIBDEST)/site-packages -f \
 +		-d $(LIBDEST)/site-packages -f $(STDLIB_CACHE_FLAGS) \
  		-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
  	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
+ 		$(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
+-		-d $(LIBDEST)/site-packages -f \
++		-d $(LIBDEST)/site-packages -f $(STDLIB_CACHE_FLAGS) \
+ 		-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
+ 	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
  		$(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/Grammar.txt
package/python3/python3-005-do-not-generate-pyo-files.patch deleted +0 -40
deleted file mode 100644
index 983d546..0000000
@@ -1,40 +0,0 @@
-Do not generate .pyo files
-
-By default, the Python installation byte-compiles all modules in two
-forms: the normal bytecode (.pyc) and an optimized bytecode (.pyo).
-
-According to
-http://www.network-theory.co.uk/docs/pytut/CompiledPythonfiles.html,
-the optimization do not do anything useful, and generating both the
-"non-optimized" and "optimized" bytecode variants takes time.
-
-Signed-off-by: Thomas Petazzoni <[email protected]>
----
- Makefile.pre.in |    9 ---------
- 1 file changed, 9 deletions(-)
-
-Index: Python-3.3.0/Makefile.pre.in
-===================================================================
---- Python-3.3.0.orig/Makefile.pre.in
-+++ Python-3.3.0/Makefile.pre.in
-@@ -1082,20 +1082,11 @@
- 		-d $(LIBDEST) -f \
- 		-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
- 		$(DESTDIR)$(LIBDEST)
--	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
--		$(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
--		-d $(LIBDEST) -f \
--		-x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
--		$(DESTDIR)$(LIBDEST)
- 	-PYTHONPATH=$(DESTDIR)$(LIBDEST)  $(RUNSHARED) \
- 		$(PYTHON_FOR_BUILD) -Wi $(DESTDIR)$(LIBDEST)/compileall.py \
- 		-d $(LIBDEST)/site-packages -f \
- 		-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
- 	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
--		$(PYTHON_FOR_BUILD) -Wi -O $(DESTDIR)$(LIBDEST)/compileall.py \
--		-d $(LIBDEST)/site-packages -f \
--		-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
--	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
- 		$(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/Grammar.txt
- 	-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
- 		$(PYTHON_FOR_BUILD) -m lib2to3.pgen2.driver $(DESTDIR)$(LIBDEST)/lib2to3/PatternGrammar.txt
package/python3/python3-005-pyc-pyo-conditional.patch new file +60 -0

60 lines not on this page · raw message

package/python3/python3-006-cross-compile-getaddrinfo.patch renamed from package/python3/python3-011-cross-compile-getaddrinfo.patch +4 -4
similarity index 81%
rename from package/python3/python3-011-cross-compile-getaddrinfo.patch
rename to package/python3/python3-006-cross-compile-getaddrinfo.patch
index 33286ce..512a534 100644
@@ -6,11 +6,11 @@ Signed-off-by: Vanya Sergeev <[email protected]>
  configure.ac |    2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
-Index: Python-3.3.0/configure.ac
+Index: b/configure.ac
 ===================================================================
---- Python-3.3.0.orig/configure.ac
-+++ Python-3.3.0/configure.ac
-@@ -3204,7 +3204,7 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -3423,7 +3423,7 @@
  
  AC_MSG_RESULT($ac_cv_buggy_getaddrinfo)
  
package/python3/python3-006-reread-environment.patch deleted +0 -63

63 lines not on this page · raw message

package/python3/python3-007-change-pyconfig-h-location.patch deleted +0 -76

76 lines not on this page · raw message

package/python3/python3-007-disable-extensions.patch renamed from package/python3/python3-012-disable-extensions.patch +23 -21

73 lines not on this page · raw message

package/python3/python3-008-distutils-sysconfig-use-sysconfigdata.patch new file +58 -0

58 lines not on this page · raw message

package/python3/python3-008-no-rpath.patch deleted +0 -51

51 lines not on this page · raw message

package/python3/python3-009-distutils-use-python-sysroot.patch new file +57 -0

57 lines not on this page · raw message

package/python3/python3-009-verbose-module-build.patch deleted +0 -26
deleted file mode 100644
index 60b6d79..0000000
@@ -1,26 +0,0 @@
-Enables verbose output when building modules
-
-Patch borrowed from OpenBricks.
-
-Signed-off-by: Thomas Petazzoni <[email protected]>
-
----
- Makefile.pre.in |    3 +--
- 1 file changed, 1 insertion(+), 2 deletions(-)
-
-Index: Python-3.3.0/Makefile.pre.in
-===================================================================
---- Python-3.3.0.orig/Makefile.pre.in
-+++ Python-3.3.0/Makefile.pre.in
-@@ -476,10 +476,9 @@
- 
- # Build the shared modules
- sharedmods: $(BUILDPYTHON) pybuilddir.txt
--	case $$MAKEFLAGS in *s*) quiet=-q; esac; \
- 	$(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' \
- 		CONFIG_ARGS="$(CONFIG_ARGS)" \
--		$(PYTHON_FOR_BUILD) $(srcdir)/setup.py $$quiet build
-+		$(PYTHON_FOR_BUILD) $(srcdir)/setup.py build
- 
- # Build static library
- # avoid long command lines, same as LIBRARY_OBJS
package/python3/python3-010-distutils-cross-compilation-support.patch deleted +0 -71

71 lines not on this page · raw message

package/python3/python3-010-no-termcap-host-path.patch new file +22 -0
new file mode 100644
index 0000000..57fe47a
@@ -0,0 +1,22 @@
+Don't look in /usr/lib/termcap for libraries
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+
+Index: b/setup.py
+===================================================================
+--- a/setup.py
++++ b/setup.py
+@@ -725,12 +725,9 @@
+                 pass # Issue 7384: Already linked against curses or tinfo.
+             elif curses_library:
+                 readline_libs.append(curses_library)
+-            elif self.compiler.find_library_file(lib_dirs +
+-                                                     ['/usr/lib/termcap'],
+-                                                     'termcap'):
++            elif self.compiler.find_library_file(lib_dirs, 'termcap'):
+                 readline_libs.append('termcap')
+             exts.append( Extension('readline', ['readline.c'],
+-                                   library_dirs=['/usr/lib/termcap'],
+                                    extra_link_args=readline_extra_link_args,
+                                    libraries=readline_libs) )
+         else:
package/python3/python3-100-optional-test-modules.patch +32 -28

110 lines not on this page · raw message

package/python3/python3-101-optional-pydoc.patch +23 -23

84 lines not on this page · raw message

package/python3/python3-102-optional-2to3.patch +33 -35

112 lines not on this page · raw message

package/python3/python3-103-optional-sqlite.patch +26 -27

75 lines not on this page · raw message

package/python3/python3-104-optional-tk.patch +24 -27

74 lines not on this page · raw message

package/python3/python3-105-optional-curses.patch +15 -15
index adb3183..75e2558 100644
@@ -8,35 +8,35 @@ Signed-off-by: Samuel Martin <[email protected]>
  configure.ac    |    9 +++++++++
  2 files changed, 14 insertions(+), 1 deletion(-)
 
-Index: cpython/Makefile.pre.in
+Index: b/Makefile.pre.in
 ===================================================================
---- cpython.orig/Makefile.pre.in
-+++ cpython/Makefile.pre.in
-@@ -993,7 +993,7 @@
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -1140,7 +1140,7 @@
  		multiprocessing multiprocessing/dummy \
  		unittest \
  		venv venv/scripts venv/scripts/posix \
 -		curses $(MACHDEPS)
 +		$(MACHDEPS)
  
- ifeq (@TEST_MODULES@,yes)
- LIBSUBDIRS += 	test \
-@@ -1061,6 +1061,10 @@
- endif
+ TESTSUBDIRS = 	test test/test_asyncio ctypes/test \
+ 		test/test_email test/test_email/data \
+@@ -1201,6 +1201,10 @@
+ 	tkinter/test/test_ttk
  endif
  
 +ifeq (@CURSES@,yes)
 +LIBSUBDIRS += curses
 +endif
 +
- libinstall:	build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c
- 	@for i in $(SCRIPTDIR) $(LIBDEST); \
- 	do \
-Index: cpython/configure.ac
+ ifeq (@TEST_MODULES@,yes)
+ LIBSUBDIRS += $(TESTSUBDIRS)
+ endif
+Index: b/configure.ac
 ===================================================================
---- cpython.orig/configure.ac
-+++ cpython/configure.ac
-@@ -2466,6 +2466,15 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -2684,6 +2684,15 @@
     DISABLED_EXTENSIONS="${DISABLED_EXTENSIONS} _tkinter"
  fi
  
package/python3/python3-106-optional-expat.patch +18 -18
index d171b79..80da939 100644
@@ -15,20 +15,20 @@ Signed-off-by: Samuel Martin <[email protected]>
  setup.py        |    2 +-
  3 files changed, 19 insertions(+), 7 deletions(-)
 
-Index: cpython/Makefile.pre.in
+Index: b/Makefile.pre.in
 ===================================================================
---- cpython.orig/Makefile.pre.in
-+++ cpython/Makefile.pre.in
-@@ -987,7 +987,7 @@
- 		html json http dbm xmlrpc \
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -1134,7 +1134,7 @@
  		logging csv wsgiref urllib \
- 		ctypes ctypes/macholib idlelib idlelib/Icons \
+ 		ctypes ctypes/macholib \
+ 		idlelib idlelib/Icons \
 -		distutils distutils/command $(XMLLIBSUBDIRS) \
 +		distutils distutils/command \
  		importlib \
  		turtledemo \
  		multiprocessing multiprocessing/dummy \
-@@ -1065,6 +1065,10 @@
+@@ -1205,6 +1205,10 @@
  LIBSUBDIRS += curses
  endif
  
@@ -36,14 +36,14 @@ Index: cpython/Makefile.pre.in
 +LIBSUBDIRS += $(XMLLIBSUBDIRS)
 +endif
 +
- libinstall:	build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c
- 	@for i in $(SCRIPTDIR) $(LIBDEST); \
- 	do \
-Index: cpython/configure.ac
+ ifeq (@TEST_MODULES@,yes)
+ LIBSUBDIRS += $(TESTSUBDIRS)
+ endif
+Index: b/configure.ac
 ===================================================================
---- cpython.orig/configure.ac
-+++ cpython/configure.ac
-@@ -2178,13 +2178,21 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -2363,13 +2363,21 @@
  AC_SUBST(DISABLED_EXTENSIONS)
  
  # Check for use of the system expat library
@@ -70,11 +70,11 @@ Index: cpython/configure.ac
  
  # Check for use of the system libffi library
  AC_MSG_CHECKING(for --with-system-ffi)
-Index: cpython/setup.py
+Index: b/setup.py
 ===================================================================
---- cpython.orig/setup.py
-+++ cpython/setup.py
-@@ -1404,7 +1404,7 @@
+--- a/setup.py
++++ b/setup.py
+@@ -1396,7 +1396,7 @@
          #
          # More information on Expat can be found at www.libexpat.org.
          #
package/python3/python3-107-optional-codecs-cjk.patch +4 -4
index de7f910..f9d377d 100644
@@ -6,11 +6,11 @@ Signed-off-by: Thomas Petazzoni <[email protected]>
  configure.ac |    6 ++++++
  1 file changed, 6 insertions(+)
 
-Index: cpython/configure.ac
+Index: b/configure.ac
 ===================================================================
---- cpython.orig/configure.ac
-+++ cpython/configure.ac
-@@ -2465,6 +2465,12 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -2683,6 +2683,12 @@
     DISABLED_EXTENSIONS="${DISABLED_EXTENSIONS} _sqlite3"
  fi
  
package/python3/python3-108-optional-nis.patch +4 -4
index 3c91497..7ec5b19 100644
@@ -9,11 +9,11 @@ Signed-off-by: Thomas Petazzoni <[email protected]>
  configure.ac |    6 ++++++
  1 file changed, 6 insertions(+)
 
-Index: cpython/configure.ac
+Index: b/configure.ac
 ===================================================================
---- cpython.orig/configure.ac
-+++ cpython/configure.ac
-@@ -2471,6 +2471,12 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -2689,6 +2689,12 @@
  		DISABLED_EXTENSIONS="${DISABLED_EXTENSIONS} _codecs_kr _codecs_jp _codecs_cn _codecs_tw _codecs_hk _codecs_iso2022"
  	fi])
  
package/python3/python3-109-optional-unicodedata.patch +4 -4
index c75ee27..4b671ed 100644
@@ -6,11 +6,11 @@ Signed-off-by: Thomas Petazzoni <[email protected]>
  configure.ac |    6 ++++++
  1 file changed, 6 insertions(+)
 
-Index: cpython/configure.ac
+Index: b/configure.ac
 ===================================================================
---- cpython.orig/configure.ac
-+++ cpython/configure.ac
-@@ -2477,6 +2477,12 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -2695,6 +2695,12 @@
      	     DISABLED_EXTENSIONS="${DISABLED_EXTENSIONS} nis"
    	  fi])
  
package/python3/python3-110-optional-idle.patch +19 -20
index 1a7fa65..f657cbd 100644
@@ -11,11 +11,11 @@ Signed-off-by: Maxime Ripard <[email protected]>
  setup.py        |    4 +++-
  3 files changed, 16 insertions(+), 2 deletions(-)
 
-Index: cpython/Makefile.pre.in
+Index: b/Makefile.pre.in
 ===================================================================
---- cpython.orig/Makefile.pre.in
-+++ cpython/Makefile.pre.in
-@@ -950,7 +950,9 @@
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -1085,7 +1085,9 @@
  	-rm -f $(DESTDIR)$(LIBPC)/python3.pc
  	(cd $(DESTDIR)$(LIBPC); $(LN) -s python-$(VERSION).pc python3.pc)
  	-rm -f $(DESTDIR)$(BINDIR)/idle3
@@ -25,16 +25,15 @@ Index: cpython/Makefile.pre.in
  	-rm -f $(DESTDIR)$(BINDIR)/pydoc3
  ifeq (@PYDOC@,yes)
  	(cd $(DESTDIR)$(BINDIR); $(LN) -s pydoc$(VERSION) pydoc3)
-@@ -986,7 +988,7 @@
- 		email email/mime \
+@@ -1133,7 +1135,6 @@
  		html json http dbm xmlrpc \
  		logging csv wsgiref urllib \
--		ctypes ctypes/macholib idlelib idlelib/Icons \
-+		ctypes ctypes/macholib \
+ 		ctypes ctypes/macholib \
+-		idlelib idlelib/Icons \
  		distutils distutils/command \
  		importlib \
  		turtledemo \
-@@ -1069,6 +1071,10 @@
+@@ -1209,6 +1210,10 @@
  LIBSUBDIRS += $(XMLLIBSUBDIRS)
  endif
  
@@ -42,14 +41,14 @@ Index: cpython/Makefile.pre.in
 +LIBSUBDIRS += idlelib idlelib/Icons
 +endif
 +
- libinstall:	build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c
- 	@for i in $(SCRIPTDIR) $(LIBDEST); \
- 	do \
-Index: cpython/configure.ac
+ ifeq (@TEST_MODULES@,yes)
+ LIBSUBDIRS += $(TESTSUBDIRS)
+ endif
+Index: b/configure.ac
 ===================================================================
---- cpython.orig/configure.ac
-+++ cpython/configure.ac
-@@ -2519,6 +2519,12 @@
+--- a/configure.ac
++++ b/configure.ac
+@@ -2737,6 +2737,12 @@
  	AS_HELP_STRING([--disable-lib2to3], [disable lib2to3]),
  	[ LIB2TO3="${enableval}" ], [ LIB2TO3=yes ])
  
@@ -62,11 +61,11 @@ Index: cpython/configure.ac
  # Check for enable-ipv6
  AH_TEMPLATE(ENABLE_IPV6, [Define if --enable-ipv6 is specified])
  AC_MSG_CHECKING([if --enable-ipv6 is specified])
-Index: cpython/setup.py
+Index: b/setup.py
 ===================================================================
---- cpython.orig/setup.py
-+++ cpython/setup.py
-@@ -2124,11 +2124,13 @@
+--- a/setup.py
++++ b/setup.py
+@@ -2183,11 +2183,13 @@
      import warnings
      warnings.filterwarnings("ignore",category=DeprecationWarning)
  
package/python3/python3.mk +22 -40

106 lines not on this page · raw message

Signature · 1 line
1.8.3.2
#30 lore raw

PATCH v2 8/20 python3: bump to 3.4.0rc1

Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

This commit bumps the Python3 package to use Python 3.4.0rc1. About the patches:

* The patches below 100 are significantly changed, because like for Python 2.x, a good number of improvements have been made in the upstream Python for cross-compilation. Therefore, almost all of these patches have been modified.

Quoted 6 lines

* All the patches above 100 are simply updated for Python 3.4.0, with a small refactoring for the handling of test modules.

The details of the python3.mk changes are:

* --without-ensurepip to tell Python to not use PIP and build time.

s/and/at/

Committed to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#31 lore raw

PATCH v2 9/20 python3: provide a PYTHON3_PATH

In reply to Thomas Petazzoni

The Python package infrastructure will need the Python 3 package to provide a PYTHON3_PATH environment variable in order to build third-party Python modules.

---
 package/python3/python3.mk | 3 +++
 1 file changed, 3 insertions(+)
package/python3/python3.mk +3 -0
index 232862a..8bf9518 100644
@@ -156,5 +156,8 @@ HOST_PYTHON3_POST_INSTALL_HOOKS += \
 	HOST_PYTHON3_INSTALL_SYMLINK \
 	HOST_PYTHON3_INSTALL_CONFIG_SYMLINK
 
+# Provided to other packages
+PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
+
 $(eval $(autotools-package))
 $(eval $(host-autotools-package))
Signature · 1 line
1.8.3.2
#32 lore raw

PATCH v2 9/20 python3: provide a PYTHON3_PATH

Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

The Python package infrastructure will need the Python 3 package to provide a PYTHON3_PATH environment variable in order to build third-party Python modules.

Committed to next, thanks.

This btw didn't cleanly apply with git am.

Signature · 1 line
Bye, Peter Korsgaard
#33 lore raw

PATCH v2 9/20 python3: provide a PYTHON3_PATH

Dear Peter Korsgaard,

Quoted 11 lines · Peter Korsgaard
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

The Python package infrastructure will need the Python 3 package to provide a PYTHON3_PATH environment variable in order to build third-party Python modules.

Committed to next, thanks.

This btw didn't cleanly apply with git am.

Most likely due to PATCH 06/20 not being applied, since the changes are close. I did test the removal of PATCH 06/20 by doing an interactive rebase, so the result might be slightly different than with git am.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#34 lore raw

PATCH v2 10/20 package: add python3 support in the package infrastructure

In reply to Thomas Petazzoni

This commit improves the Python package infrastructure to allow Python packages to be built with Python 3. The changes are fairly simple:

* Use either PYTHON_PATH or PYTHON3_PATH as the PYTHONPATH depending on which Python is used.

* Depend on host-python or host-python3 and python or python3 depending on which Python is used.

---
 package/pkg-python.mk | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
package/pkg-python.mk +5 -5
index 11ad75d..512ef66 100644
@@ -27,7 +27,7 @@ PKG_PYTHON_DISTUTILS_ENV = \
 	CFLAGS="$(TARGET_CFLAGS)" \
 	LDFLAGS="$(TARGET_LDFLAGS)" \
 	LDSHARED="$(TARGET_CROSS)gcc -shared" \
-	PYTHONPATH="$(PYTHON_PATH)" \
+	PYTHONPATH="$(if $(BR2_PACKAGE_PYTHON3),$(PYTHON3_PATH),$(PYTHON_PATH))" \
 	_python_sysroot=$(STAGING_DIR) \
 	_python_prefix=/usr \
 	_python_exec_prefix=/usr
@@ -48,7 +48,7 @@ HOST_PKG_PYTHON_DISTUTILS_INSTALL_OPT = \
 # Target setuptools-based packages
 PKG_PYTHON_SETUPTOOLS_ENV = \
 	PATH="$(TARGET_PATH)" \
-	PYTHONPATH="$(PYTHON_PATH)" \
+	PYTHONPATH="$(if $(BR2_PACKAGE_PYTHON3),$(PYTHON3_PATH),$(PYTHON_PATH))" \
 	_python_sysroot=$(STAGING_DIR) \
 	_python_prefix=/usr \
 	_python_exec_prefix=/usr
@@ -138,16 +138,16 @@ endif
 # depending on the package characteristics, and shouldn't be derived
 # automatically from the dependencies of the corresponding target
 # package.
-$(2)_DEPENDENCIES ?= $(filter-out host-python host-python-setuptools host-toolchain $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
+$(2)_DEPENDENCIES ?= $(filter-out host-python host-python3 host-python-setuptools host-toolchain $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
 
 # Target packages need both the python interpreter on the target (for
 # runtime) and the python interpreter on the host (for
 # compilation). However, host packages only need the python
 # interpreter on the host.
 ifeq ($(4),target)
-$(2)_DEPENDENCIES += host-python python
+$(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3 python3,host-python python)
 else
-$(2)_DEPENDENCIES += host-python
+$(2)_DEPENDENCIES += $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
 endif
 
 # Setuptools based packages will need host-python-setuptools (both
Signature · 1 line
1.8.3.2
#35 lore raw

PATCH v2 11/20 package: allow Python packages with Python3

In reply to Thomas Petazzoni

Until now, Python external modules were only visible when Python 2.x was selected. With this commit, we now source all the Config.in files of Python external modules, as soon as one of the two Python interpreters is enabled.

Since all Python external modules have a "depends on BR2_PACKAGE_PYTHON" in their Config.in, this commit in practice does not allow to enable any Python external module. However, thanks to this, we can progressively and safely enable more and more Python external modules to build with Python 3, by simply changing their dependency to "depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3".

---
 package/Config.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
package/Config.in +2 -2
index cb2d31b..0f702a3 100644
@@ -411,7 +411,8 @@ comment "External PHP extensions need a toolchain w/ dynamic library"
 endif
 endif
 source "package/python/Config.in"
-if BR2_PACKAGE_PYTHON
+source "package/python3/Config.in"
+if BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 menu "external python modules"
 source "package/python-bottle/Config.in"
 source "package/python-configobj/Config.in"
@@ -447,7 +448,6 @@ source "package/python-tornado/Config.in"
 source "package/python-versiontools/Config.in"
 endmenu
 endif
-source "package/python3/Config.in"
 source "package/ruby/Config.in"
 source "package/tcl/Config.in"
 if BR2_PACKAGE_TCL
Signature · 1 line
1.8.3.2
#36 lore raw

PATCH v2 12/20 python, python3: enable unicodedata for host-python, needed by setuptools

In reply to Thomas Petazzoni

As we are going to bump setuptools to a much newer version, the host python needs to be built with support for unicodedata.

---
 package/python/python.mk   | 2 +-
 package/python3/python3.mk | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
package/python/python.mk +1 -1
index 0469820..3b0a4dd 100644
@@ -25,7 +25,7 @@ HOST_PYTHON_CONF_OPT += 	\
 	--disable-curses	\
 	--disable-codecs-cjk	\
 	--disable-nis		\
-	--disable-unicodedata	\
+	--enable-unicodedata	\
 	--disable-dbm		\
 	--disable-gdbm		\
 	--disable-bsddb		\
package/python3/python3.mk +1 -1
index 8bf9518..62642c8 100644
@@ -25,7 +25,7 @@ HOST_PYTHON3_CONF_OPT += 	\
 	--disable-curses	\
 	--disable-codecs-cjk	\
 	--disable-nis		\
-	--disable-unicodedata	\
+	--enable-unicodedata	\
 	--disable-test-modules	\
 	--disable-idle3		\
 	--disable-pyo-build
Signature · 1 line
1.8.3.2
#37 lore raw

PATCH v2 13/20 python-setuptools: bump version to 2.1.2

In reply to Thomas Petazzoni

From: Rohan Fletcher <[email protected]>

[Thomas: bump to 2.1.2 instead of 0.8, remove comment that no longer made sense about setuptools being forked.]

---
 package/python-setuptools/python-setuptools.mk | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
package/python-setuptools/python-setuptools.mk +3 -8
index 13df448..48bf7c5 100644
@@ -4,14 +4,9 @@
 #
 ################################################################################
 
-# "distribute" is a fork of the unmaintained setuptools package. There
-# are plans to re-merge it into setuptools; if this happens, we can
-# switch back to it.
-# See http://pypi.python.org/packages/source/s/setuptools
-
-PYTHON_SETUPTOOLS_VERSION = 0.6.36
-PYTHON_SETUPTOOLS_SOURCE  = distribute-$(PYTHON_SETUPTOOLS_VERSION).tar.gz
-PYTHON_SETUPTOOLS_SITE    = http://pypi.python.org/packages/source/d/distribute
+PYTHON_SETUPTOOLS_VERSION = 2.1.2
+PYTHON_SETUPTOOLS_SOURCE  = setuptools-$(PYTHON_SETUPTOOLS_VERSION).tar.gz
+PYTHON_SETUPTOOLS_SITE    = http://pypi.python.org/packages/source/s/setuptools
 PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 
 $(eval $(python-package))
Signature · 1 line
1.8.3.2
#38 lore raw

PATCH v2 14/20 python, python3: fix to ensure libpython is stripped

In reply to Thomas Petazzoni

From: Przemyslaw Wrzos <[email protected]>

The python and python3 builds mark libpython as read-only which prevents it from being stripped out correctly for the target.

---
 package/python/python.mk   | 9 +++++++++
 package/python3/python3.mk | 9 +++++++++
 2 files changed, 18 insertions(+)
package/python/python.mk +9 -0
index 3b0a4dd..94765cb 100644
@@ -160,6 +160,15 @@ endef
 
 PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_REMOVE_USELESS_FILES
 
+#
+# Make sure libpython gets stripped out on target
+#
+define PYTHON_ENSURE_LIBPYTHON_STRIPPED
+	chmod u+w $(TARGET_DIR)/usr/lib/libpython$(PYTHON_VERSION_MAJOR)*.so
+endef
+
+PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_ENSURE_LIBPYTHON_STRIPPED
+
 PYTHON_AUTORECONF = YES
 
 # Provided to other packages
package/python3/python3.mk +9 -0
index 62642c8..2d9d1f2 100644
@@ -131,6 +131,15 @@ endef
 
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_REMOVE_USELESS_FILES
 
+#
+# Make sure libpython gets stripped out on target
+#
+define PYTHON3_ENSURE_LIBPYTHON_STRIPPED
+	chmod u+w $(TARGET_DIR)/usr/lib/libpython$(PYTHON3_VERSION_MAJOR)*.so
+endef
+
+PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_ENSURE_LIBPYTHON_STRIPPED
+
 PYTHON3_AUTORECONF = YES
 
 define PYTHON3_INSTALL_SYMLINK
Signature · 1 line
1.8.3.2
#39 lore raw

PATCH v2 15/20 python-pyasn: use the real upstream

In reply to Thomas Petazzoni

The https://code.google.com/p/pyasn/ project is not really the real upstream for PyASN, and at least not the upstream for the PyASN implementation recommended by the PySNMP developers.

Instead, the real upstream is
https://pypi.python.org/packages/source/p/pyasn1/, which has had much more regular releases than the other PyASN implementation.

Therefore, we switch to using this implementation, as recommended by the PySNMP developers on http://pysnmp.sourceforge.net/download.html.

---
 package/python-pyasn/python-pyasn.mk | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)
package/python-pyasn/python-pyasn.mk +3 -9
index b007222..43fabe9 100644
@@ -4,16 +4,10 @@
 #
 ################################################################################
 
-PYTHON_PYASN_VERSION = 1.2
-PYTHON_PYASN_SOURCE = PyASN-$(PYTHON_PYASN_VERSION).zip
-PYTHON_PYASN_SITE = https://pyasn.googlecode.com/files
+PYTHON_PYASN_VERSION = 0.17
+PYTHON_PYASN_SOURCE = pyasn1-$(PYTHON_PYASN_VERSION).tar.gz
+PYTHON_PYASN_SITE = https://pypi.python.org/packages/source/p/pyasn1/
 PYTHON_PYASN_LICENSE = LGPLv3+ (pyasn.cpp), GPLv2+ (libgds)
 PYTHON_PYASN_SETUP_TYPE = distutils
 
-define PYTHON_PYASN_EXTRACT_CMDS
-	unzip -d $(@D) $(DL_DIR)/$(PYTHON_PYASN_SOURCE)
-	mv $(@D)/PyASN-$(PYTHON_PYASN_VERSION)/* $(@D)
-	$(RM) -r $(@D)/PyASN-$(PYTHON_PYASN_VERSION)
-endef
-
 $(eval $(python-package))
Signature · 1 line
1.8.3.2
#40 lore raw

PATCH v2 16/20 python-bottle: allow to build with Python 3

In reply to Thomas Petazzoni
---
 package/python-bottle/Config.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/python-bottle/Config.in +1 -1
index 07f490c..7b13c1d 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_BOTTLE
 	bool "python-bottle"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	help
 	  Bottle is a fast, simple and lightweight WSGI micro web-framework
 	  for Python. It is distributed as a single file module and has no
Signature · 1 line
1.8.3.2
#41 lore raw

PATCH v2 17/20 python-serial: allow to build with Python 3

In reply to Thomas Petazzoni
---
 package/python-serial/Config.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/python-serial/Config.in +1 -1
index df26af1..72f5567 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_SERIAL
 	bool "python-serial"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	help
 	  python-serial is a Python library to access serial ports.
 
Signature · 1 line
1.8.3.2
#42 lore raw

PATCH v2 18/20 python-pyasn: allow to build with Python 3

In reply to Thomas Petazzoni
---
 package/python-pyasn/Config.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/python-pyasn/Config.in +1 -1
index 13095f9..eca53db 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_PYASN
 	bool "python-pyasn"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	depends on BR2_INSTALL_LIBSTDCPP
 	help
 	  PyASN is a Python extension module that enables you to
Signature · 1 line
1.8.3.2
#43 lore raw

PATCH v2 19/20 python-pycrypto: allow to build with Python 3

In reply to Thomas Petazzoni
---
 package/python-pycrypto/Config.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/python-pycrypto/Config.in +1 -1
index 60a3446..fe545b5 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_PYCRYPTO
 	bool "python-pycrypto"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	select BR2_PACKAGE_GMP
 	help
 	  PyCrypto is a collection of cryptographic algorithms and
Signature · 1 line
1.8.3.2
#44 lore raw

PATCH v2 20/20 python-pysnmp{, -apps, -mibs}: allow to build with Python 3

In reply to Thomas Petazzoni
---
 package/python-pysnmp-apps/Config.in | 2 +-
 package/python-pysnmp-mibs/Config.in | 2 +-
 package/python-pysnmp/Config.in      | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
package/python-pysnmp-apps/Config.in +1 -1
index 304e812..8edd04c 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_PYSNMP_APPS
 	bool "python-pysnmp-apps"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	select BR2_PACKAGE_PYTHON_PYSNMP
 	depends on BR2_INSTALL_LIBSTDCPP # pysnmp -> pyasn
 	help
package/python-pysnmp-mibs/Config.in +1 -1
index 90ec933..cc03676 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_PYSNMP_MIBS
 	bool "python-pysnmp-mibs"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	select BR2_PACKAGE_PYTHON_PYSNMP
 	depends on BR2_INSTALL_LIBSTDCPP # pysnmp -> pyasn
 	help
package/python-pysnmp/Config.in +1 -1
index d5f90ed..6cf18ee 100644
@@ -1,6 +1,6 @@
 config BR2_PACKAGE_PYTHON_PYSNMP
 	bool "python-pysnmp"
-	depends on BR2_PACKAGE_PYTHON
+	depends on BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3
 	depends on BR2_INSTALL_LIBSTDCPP # pyasn
 	select BR2_PACKAGE_PYTHON_PYASN
 	select BR2_PACKAGE_PYTHON_PYCRYPTO
Signature · 1 line
1.8.3.2
#45 lore raw

v2 cover Python 2 and 3 bumps, support for Python 3 external modules

In reply to Thomas Petazzoni
Quoted 41 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hello,
Here is a v2 of the patch set that does numerous Python support improvements. Changes since v1:

* Dropped patches that have been merged.

* Rebased on top of the next branch, in order to resolve conflicts with the parallel build support patches that were merged.

Best regards,

Thomas

Przemyslaw Wrzos (1):
python, python3: fix to ensure libpython is stripped

Rohan Fletcher (1):
python-setuptools: bump version to 2.1.2

Thomas Petazzoni (17):
pkg-python: no longer use distutilscross for setuptools
python-distutilscross: remove package that is no longer needed
python3: removal of *.py/*.pyc is now done globally
python3: make it exclusive from python
python3: add python -> python3 symlink for the host variant
python3: use proper PYTHON3_VERSION_MAJOR instead of harcoding 3.3
python3: bump to 3.4.0rc1
python3: provide a PYTHON3_PATH
package: add python3 support in the package infrastructure
package: allow Python packages with Python3
python, python3: enable unicodedata for host-python, needed by
setuptools
python-pyasn: use the real upstream
python-bottle: allow to build with Python 3
python-serial: allow to build with Python 3
python-pyasn: allow to build with Python 3
python-pycrypto: allow to build with Python 3 python-pysnmp{,-apps,-mibs}: allow to build with Python 3

Quoted 2 lines

kpa_info at yahoo.fr (1):
python3: add config directory symbolic link

Committed the rest of the series (except for patch 6) to next, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#46 lore raw

v2 cover Python 2 and 3 bumps, support for Python 3 external modules

Dear Peter Korsgaard,

Quoted 26 lines · Peter Korsgaard
Quoted 25 lines

Przemyslaw Wrzos (1):
python, python3: fix to ensure libpython is stripped

Rohan Fletcher (1):
python-setuptools: bump version to 2.1.2

Thomas Petazzoni (17):
pkg-python: no longer use distutilscross for setuptools
python-distutilscross: remove package that is no longer needed
python3: removal of *.py/*.pyc is now done globally
python3: make it exclusive from python
python3: add python -> python3 symlink for the host variant
python3: use proper PYTHON3_VERSION_MAJOR instead of harcoding 3.3
python3: bump to 3.4.0rc1
python3: provide a PYTHON3_PATH
package: add python3 support in the package infrastructure
package: allow Python packages with Python3
python, python3: enable unicodedata for host-python, needed by
setuptools
python-pyasn: use the real upstream
python-bottle: allow to build with Python 3
python-serial: allow to build with Python 3
python-pyasn: allow to build with Python 3
python-pycrypto: allow to build with Python 3 python-pysnmp{,-apps,-mibs}: allow to build with Python 3

Quoted 4 lines
Quoted 2 lines

kpa_info at yahoo.fr (1):
python3: add config directory symbolic link

Committed the rest of the series (except for patch 6) to next, thanks.

Great, thanks. If there are Python users on the list, please give a test to the next branch, and see if it works for you. It was a fairly major bump, so there might be some issues I have overlooked.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com