Skip to contentSkip to conversation

Python 3, simple, external package support

PATCH New Patrick Gerber 2 participants 7 messages Feb 15, 2013 – Feb 13, 2014

lore
Share & tools
lore
#0 lore raw

v4 cover Python 3, simple, external package support

This patch series refactor python support in buildroot to enable the use of external package with both; python2 and python3. The packages python-bottle and python-serial have been adapted as examples.

To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.

Changes between v3 and v4:

* Remove the python-common.mk file and move the PYTHON variable declaration into python.mk and python3.mk (Requested by Maxime Ripard)

Changes between v2 and v3:

* Same new menu organisation as v2 but with less modifications in package/Config.in (Requested by Thomas Petazzoni)

* New directory called python-common instead of python generic. (Requested by Thomas Petazzoni)

* No more symlink from python to python3 in staging (Requested by Samuel Martin)

Patrick Gerber (5):
Python menu reorganization
Add python3 config directory simlink
Add PYTHON variable to ease the support of both version
python-bottle: Convert package to be used with python2 or python3
python-serial: Convert package to be used with python2 or python3

 package/Config.in                      |    3 ++-
 package/python-bottle/python-bottle.mk |    6 +++---
 package/python-common/Config.in        |   35 ++++++++++++++++++++++++++++++++
 package/python-serial/Config.in        |    1 -
 package/python-serial/python-serial.mk |    6 +++---
 package/python/Config.in               |   14 -------------
 package/python/python.mk               |    6 ++++++
 package/python3/Config.in              |   14 -------------
 package/python3/python3.mk             |   15 ++++++++++++++
 9 files changed, 64 insertions(+), 36 deletions(-)
 create mode 100644 package/python-common/Config.in
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v4 1/5 Python menu reorganization

In order to have a better support for python external package make the python menu clearer. Python2 and Python3 are now mutually exclusive

---
 package/Config.in               |    3 ++-
 package/python-common/Config.in |   35 +++++++++++++++++++++++++++++++++++
 package/python/Config.in        |   14 --------------
 package/python3/Config.in       |   14 --------------
 4 files changed, 37 insertions(+), 29 deletions(-)
 create mode 100644 package/python-common/Config.in
package/Config.in +2 -1
index 2fad94d..5f2c1bd 100644
@@ -315,9 +315,10 @@ endmenu
 endif
 source "package/microperl/Config.in"
 source "package/php/Config.in"
+source "package/python-common/Config.in"
 source "package/python/Config.in"
 source "package/python3/Config.in"
-if BR2_PACKAGE_PYTHON
+if (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3)
 menu "external python modules"
 source "package/python-bottle/Config.in"
 source "package/python-dpkt/Config.in"
package/python-common/Config.in new file +35 -0
new file mode 100644
index 0000000..6523c21
@@ -0,0 +1,35 @@
+config BR2_PACKAGE_PYTHON_COMMON
+	bool "Python"
+	depends on BR2_USE_WCHAR
+	# uses fork()
+	depends on BR2_USE_MMU
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
+
+comment "python3 requires a toolchain with WCHAR support"
+	depends on !BR2_USE_WCHAR
+
+choice
+	depends on BR2_PACKAGE_PYTHON_COMMON
+	prompt "Python interpreter"
+	help
+	  Select which version of Python you would like to use.
+
+config BR2_PACKAGE_PYTHON
+	bool "python2"
+	select BR2_PACKAGE_LIBFFI
+	help
+	  Use Python 2.x 
+
+
+config BR2_PACKAGE_PYTHON3
+	bool "python3"
+	# uses fork()
+	depends on BR2_USE_MMU
+	select BR2_PACKAGE_LIBFFI
+	help
+	  Use Python 3.x 
+
+endchoice
package/python/Config.in +0 -14
index 4089cce..b4fd6c2 100644
@@ -1,17 +1,3 @@
-config BR2_PACKAGE_PYTHON
-	bool "python"
-	depends on BR2_USE_WCHAR
-	# uses fork()
-	depends on BR2_USE_MMU
-	select BR2_PACKAGE_LIBFFI
-	help
-	  The python language interpreter.
-
-	  http://www.python.org/
-
-comment "python requires a toolchain with WCHAR support"
-	depends on !BR2_USE_WCHAR
-
 if BR2_PACKAGE_PYTHON
 
 choice
package/python3/Config.in +0 -14
index 5959d31..279f31b 100644
@@ -1,17 +1,3 @@
-config BR2_PACKAGE_PYTHON3
-	bool "python3"
-	depends on BR2_USE_WCHAR
-	# uses fork()
-	depends on BR2_USE_MMU
-	select BR2_PACKAGE_LIBFFI
-	help
-	  The python language interpreter.
-
-	  http://www.python.org/
-
-comment "python3 requires a toolchain with WCHAR support"
-	depends on !BR2_USE_WCHAR
-
 if BR2_PACKAGE_PYTHON3
 
 choice
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v4 2/5 Add python3 config directory simlink

In reply to Patrick Gerber

When using host python to install an external module, the setup tool look for some files in python3.3/config/. The python3 package config directory is called config-3.3m. This is why we need to alias config to config-3.3m.

---
 package/python3/python3.mk |    9 +++++++++
 1 file changed, 9 insertions(+)
package/python3/python3.mk +9 -0
index 7551a7b..74af9b4 100644
@@ -160,6 +160,15 @@ ifneq ($(BR2_PACKAGE_PYTHON),y)
 PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK
 endif
 
+# Alias the python3.3/config-3.3m to python3.3/config
+# This is needed when installing an external python module, because
+# the setup is looking for files in python3.3/config/
+define HOST_PYTHON3_INSTALL_SYMLINK
+	ln -fs config-3.3m $(HOST_DIR)/usr/lib/python3.3/config
+endef
+
+HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+
 ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y)
 define PYTHON3_REMOVE_MODULES_FILES
 	for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) \
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH v4 3/5 Add PYTHON variable to ease the support of both version

In reply to Patrick Gerber

Define a new variable called PYTHON that contains either "python" or "python3" depending of the selected Python version. This is needed to easily set dependencies and to call the right version of python for packages that support both version of python.

---
 package/python/python.mk   |    6 ++++++
 package/python3/python3.mk |    6 ++++++
 2 files changed, 12 insertions(+)
package/python/python.mk +6 -0
index 71591b7..6a7de43 100644
@@ -53,6 +53,12 @@ HOST_PYTHON_DEPENDENCIES = host-expat host-zlib
 
 PYTHON_INSTALL_STAGING = YES
 
+# The PYTHON variable could be used by external python package to
+# works with both python's version (python and python3)
+ifeq ($(BR2_PACKAGE_PYTHON),y)
+	PYTHON = python
+endif
+
 ifeq ($(BR2_PACKAGE_PYTHON_READLINE),y)
 PYTHON_DEPENDENCIES += readline
 endif
package/python3/python3.mk +6 -0
index 74af9b4..44136fc 100644
@@ -55,6 +55,12 @@ HOST_PYTHON3_DEPENDENCIES = host-expat host-zlib
 
 PYTHON3_INSTALL_STAGING = YES
 
+# The PYTHON variable could be used by external python package to
+# works with both python's version (python and python3)
+ifeq ($(BR2_PACKAGE_PYTHON3),y)
+	PYTHON = python3
+endif
+
 ifeq ($(BR2_PACKAGE_PYTHON3_READLINE),y)
 PYTHON3_DEPENDENCIES += readline
 endif
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH v4 4/5 python-bottle: Convert package to be used with python2 or python3

In reply to Patrick Gerber
---
 package/python-bottle/python-bottle.mk |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
package/python-bottle/python-bottle.mk +3 -3
index 370b31b..b8a1847 100644
@@ -7,16 +7,16 @@
 PYTHON_BOTTLE_VERSION = 0.11.4
 PYTHON_BOTTLE_SOURCE = bottle-$(PYTHON_BOTTLE_VERSION).tar.gz
 PYTHON_BOTTLE_SITE = http://pypi.python.org/packages/source/b/bottle
-PYTHON_BOTTLE_DEPENDENCIES = python
+PYTHON_BOTTLE_DEPENDENCIES = $(PYTHON)
 PYTHON_BOTTLE_LICENSE = MIT
 # README.rst refers to the file "LICENSE" but it's not included
 
 define PYTHON_BOTTLE_BUILD_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py build)
 endef
 
 define PYTHON_BOTTLE_INSTALL_TARGET_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py install --prefix=$(TARGET_DIR)/usr)
 endef
 
 $(eval $(generic-package))
Signature · 1 line
1.7.9.5
#5 lore raw

PATCH v4 5/5 python-serial: Convert package to be used with python2 or python3

In reply to Patrick Gerber
---
 package/python-serial/Config.in        |    1 -
 package/python-serial/python-serial.mk |    6 +++---
 2 files changed, 3 insertions(+), 4 deletions(-)
package/python-serial/Config.in +0 -1
index ea090bd..55a9cc1 100644
@@ -1,6 +1,5 @@
 config BR2_PACKAGE_PYTHON_SERIAL
 	bool "python-serial"
-	depends on BR2_PACKAGE_PYTHON
 	help
 	  python-serial is a Python library to access serial ports.
 
package/python-serial/python-serial.mk +3 -3
index fb6b31c..6528a6b 100644
@@ -8,14 +8,14 @@ PYTHON_SERIAL_VERSION = 2.6
 PYTHON_SERIAL_SOURCE  = pyserial-$(PYTHON_SERIAL_VERSION).tar.gz
 PYTHON_SERIAL_SITE    = http://pypi.python.org/packages/source/p/pyserial/
 
-PYTHON_SERIAL_DEPENDENCIES = python
+PYTHON_SERIAL_DEPENDENCIES = $(PYTHON)
 
 define PYTHON_SERIAL_BUILD_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py build)
 endef
 
 define PYTHON_SERIAL_INSTALL_TARGET_CMDS
-	(cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr)
+	(cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py install --prefix=$(TARGET_DIR)/usr)
 endef
 
 $(eval $(generic-package))
Signature · 1 line
1.7.9.5
#6 lore raw

v4 cover Python 3, simple, external package support

In reply to Patrick Gerber

Dear Patrick Gerber,

Quoted 8 lines · Patrick Gerber

This patch series refactor python support in buildroot to enable the use of external package with both; python2 and python3. The packages python-bottle and python-serial have been adapted as examples.

To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.

It's been a very long time (one year), but I have sent a patch series that bumps Python 2.x and 3.x to their latest version, and adds the possibility of building Python external modules with Python 3.

If you have some time, I would definitely be interested by receiving comments, and testing reports. For details, see:

http://lists.busybox.net/pipermail/buildroot/2014-February/089309.html

The code is available at:

http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=python3-pkg

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com