Python 3, simple, external package support
v3 cover Python 3, simple, external package support
This patch series refactor python support in buildroot to enable the use of external package with both; python2 and python3. The packages python-bottle and python-serial have been adapted as examples.
To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.
Changes between v2 and v3:
* Same new menu organisation as v2 but with less modifications in package/Config.in (Requested by Thomas Petazzoni)
* New directory called python-common instead of python generic. (Requested by Thomas Petazzoni)
* No more symlink from python to python3 in staging (Requested by Samuel Martin)
Patrick Gerber (5):
Python menu reorganization
Add python3 config directory simlink
Add common declaration to ease the support of both version
python-bottle: Convert package to be used with python2 or python3
python-serial: Convert package to be used with python2 or python3
package/Config.in | 3 ++- package/python-bottle/python-bottle.mk | 6 +++--- package/python-common/Config.in | 35 ++++++++++++++++++++++++++++++++ package/python-common/python-common.mk | 16 +++++++++++++++ package/python-serial/Config.in | 1 - package/python-serial/python-serial.mk | 6 +++--- package/python/Config.in | 14 ------------- package/python3/Config.in | 14 ------------- package/python3/python3.mk | 6 ++++++ 9 files changed, 65 insertions(+), 36 deletions(-) create mode 100644 package/python-common/Config.in create mode 100644 package/python-common/python-common.mk
Signature · 1 line
1.7.9.5
PATCH v3 1/5 Python menu reorganization
In order to have a better support for python external package make the python menu clearer. Python2 and Python3 are now mutually exclusive
--- package/Config.in | 3 ++- package/python-common/Config.in | 35 +++++++++++++++++++++++++++++++++++ package/python/Config.in | 14 -------------- package/python3/Config.in | 14 -------------- 4 files changed, 37 insertions(+), 29 deletions(-) create mode 100644 package/python-common/Config.in
package/Config.in +2 -1
@@ -315,9 +315,10 @@ endmenu endif source "package/microperl/Config.in" source "package/php/Config.in" +source "package/python-common/Config.in" source "package/python/Config.in" source "package/python3/Config.in" -if BR2_PACKAGE_PYTHON +if (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3) menu "external python modules" source "package/python-bottle/Config.in" source "package/python-dpkt/Config.in"
package/python-common/Config.in new file +35 -0
@@ -0,0 +1,35 @@ +config BR2_PACKAGE_PYTHON_COMMON + bool "Python" + depends on BR2_USE_WCHAR + # uses fork() + depends on BR2_USE_MMU + help + The python language interpreter. + + http://www.python.org/ + +comment "python3 requires a toolchain with WCHAR support" + depends on !BR2_USE_WCHAR + +choice + depends on BR2_PACKAGE_PYTHON_COMMON + prompt "Python interpreter" + help + Select which version of Python you would like to use. + +config BR2_PACKAGE_PYTHON + bool "python2" + select BR2_PACKAGE_LIBFFI + help + Use Python 2.x + + +config BR2_PACKAGE_PYTHON3 + bool "python3" + # uses fork() + depends on BR2_USE_MMU + select BR2_PACKAGE_LIBFFI + help + Use Python 3.x + +endchoice
package/python/Config.in +0 -14
@@ -1,17 +1,3 @@ -config BR2_PACKAGE_PYTHON - bool "python" - depends on BR2_USE_WCHAR - # uses fork() - depends on BR2_USE_MMU - select BR2_PACKAGE_LIBFFI - help - The python language interpreter. - - http://www.python.org/ - -comment "python requires a toolchain with WCHAR support" - depends on !BR2_USE_WCHAR - if BR2_PACKAGE_PYTHON choice
package/python3/Config.in +0 -14
@@ -1,17 +1,3 @@ -config BR2_PACKAGE_PYTHON3 - bool "python3" - depends on BR2_USE_WCHAR - # uses fork() - depends on BR2_USE_MMU - select BR2_PACKAGE_LIBFFI - help - The python language interpreter. - - http://www.python.org/ - -comment "python3 requires a toolchain with WCHAR support" - depends on !BR2_USE_WCHAR - if BR2_PACKAGE_PYTHON3 choice
Signature · 1 line
1.7.9.5
PATCH v3 2/5 Add python3 config directory simlink
In reply to Patrick GerberAdd a simlink to alias the python3.3/config-3.3m directory to python3.3/config in the staging directory
--- package/python3/python3.mk | 6 ++++++ 1 file changed, 6 insertions(+)
package/python3/python3.mk +6 -0
@@ -160,6 +160,12 @@ ifneq ($(BR2_PACKAGE_PYTHON),y) PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK endif +define HOST_PYTHON3_INSTALL_SYMLINK + ln -fs config-3.3m $(HOST_DIR)/usr/lib/python3.3/config +endef + +HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK + ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y) define PYTHON3_REMOVE_MODULES_FILES for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) \
Signature · 1 line
1.7.9.5
PATCH v3 2/5 Add python3 config directory simlink
Dear Patrick Gerber,
Quoted 2 lines · Patrick Gerber
Add a simlink to alias the python3.3/config-3.3m directory to python3.3/config in the staging directory
This description doesn't match what the code is doing. The code creates a symbolic link in $(HOST_DIR), not $(STAGING_DIR).
Also, this commit log doesn't explain *why* this is needed in the first place. The commit log only explains what the patch is doing (which we can easily find out by reading the code), but lacks an explanation of the underlying reasons.
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH v3 2/5 Add python3 config directory simlink
Quoted 9 lines · Thomas Petazzoni
Dear Patrick Gerber,
Quoted 2 lines · Patrick Gerber
Add a simlink to alias the python3.3/config-3.3m directory to python3.3/config in the staging directory
This description doesn't match what the code is doing. The code creates a symbolic link in $(HOST_DIR), not $(STAGING_DIR).
Quoted 9 lines
Also, this commit log doesn't explain *why* this is needed in the first place. The commit log only explains what the patch is doing (which we can easily find out by reading the code), but lacks an explanation of the underlying reasons.
Thanks,
Thomas
When using host python to install an external module, the setup tools look for a python3.3/config/pyconfig.h file. With the python3 package from buildroot this directory is called config-3.3m. This is why we need to alias config to config-3.3m.
Patrick
PATCH v3 2/5 Add python3 config directory simlink
Dear Patrick,
Quoted 4 lines · Patrick
When using host python to install an external module, the setup tools look for a
python3.3/config/pyconfig.hfile. With the python3 package from buildroot this directory is called config-3.3m. This is why we need to alias config to config-3.3m.
Thanks. This explanation should go in the commit log, and maybe even as a comment in the code explaining why the symlink is needed.
Thanks,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH v3 2/5 Add python3 config directory simlink
Quoted 11 lines · Thomas Petazzoni
Dear Patrick,
Quoted 4 lines · Patrick
When using host python to install an external module, the setup tools look for a
python3.3/config/pyconfig.hfile. With the python3 package from buildroot this directory is called config-3.3m. This is why we need to alias config to config-3.3m.Thanks. This explanation should go in the commit log, and maybe even as a comment in the code explaining why the symlink is needed.>
I'll do it.
Quoted 4 lines
Thanks,
Thomas
PATCH v3 3/5 Add common declaration to ease the support of both version
In reply to Patrick GerberDefine a new variable called PYTHON that contains either "python" or "python3" depending of the selected Python version. This is needed to easily set dependencies and to call the right version of python for packages that support both version of python.
--- package/python-common/python-common.mk | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 package/python-common/python-common.mk
package/python-common/python-common.mk new file +16 -0
@@ -0,0 +1,16 @@ +############################################################# +# +# python-common +# +############################################################# +# Define some symbol usefull to make python package that works +# with python2 and python3. + +ifeq ($(BR2_PACKAGE_PYTHON),y) + PYTHON = python +endif + +ifeq ($(BR2_PACKAGE_PYTHON3),y) + PYTHON = python3 +endif +
Signature · 1 line
1.7.9.5
PATCH v3 4/5 python-bottle: Convert package to be used with python2 or python3
In reply to Patrick Gerber--- package/python-bottle/python-bottle.mk | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
package/python-bottle/python-bottle.mk +3 -3
@@ -7,16 +7,16 @@ PYTHON_BOTTLE_VERSION = 0.11.4 PYTHON_BOTTLE_SOURCE = bottle-$(PYTHON_BOTTLE_VERSION).tar.gz PYTHON_BOTTLE_SITE = http://pypi.python.org/packages/source/b/bottle -PYTHON_BOTTLE_DEPENDENCIES = python +PYTHON_BOTTLE_DEPENDENCIES = $(PYTHON) PYTHON_BOTTLE_LICENSE = MIT # README.rst refers to the file "LICENSE" but it's not included define PYTHON_BOTTLE_BUILD_CMDS - (cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build) + (cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py build) endef define PYTHON_BOTTLE_INSTALL_TARGET_CMDS - (cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr) + (cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py install --prefix=$(TARGET_DIR)/usr) endef $(eval $(generic-package))
Signature · 1 line
1.7.9.5
PATCH v3 5/5 python-serial: Convert package to be used with python2 or python3
In reply to Patrick Gerber--- package/python-serial/Config.in | 1 - package/python-serial/python-serial.mk | 6 +++--- 2 files changed, 3 insertions(+), 4 deletions(-)
package/python-serial/Config.in +0 -1
@@ -1,6 +1,5 @@ config BR2_PACKAGE_PYTHON_SERIAL bool "python-serial" - depends on BR2_PACKAGE_PYTHON help python-serial is a Python library to access serial ports.
package/python-serial/python-serial.mk +3 -3
@@ -8,14 +8,14 @@ PYTHON_SERIAL_VERSION = 2.6 PYTHON_SERIAL_SOURCE = pyserial-$(PYTHON_SERIAL_VERSION).tar.gz PYTHON_SERIAL_SITE = http://pypi.python.org/packages/source/p/pyserial/ -PYTHON_SERIAL_DEPENDENCIES = python +PYTHON_SERIAL_DEPENDENCIES = $(PYTHON) define PYTHON_SERIAL_BUILD_CMDS - (cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build) + (cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py build) endef define PYTHON_SERIAL_INSTALL_TARGET_CMDS - (cd $(@D); $(HOST_DIR)/usr/bin/python setup.py install --prefix=$(TARGET_DIR)/usr) + (cd $(@D); $(HOST_DIR)/usr/bin/$(PYTHON) setup.py install --prefix=$(TARGET_DIR)/usr) endef $(eval $(generic-package))
Signature · 1 line
1.7.9.5
v3 cover Python 3, simple, external package support
In reply to Patrick GerberHi Patrick,
Le 13/02/2013 10:20, Patrick Gerber a ?crit :
Quoted 8 lines
This patch series refactor python support in buildroot to enable the use of external package with both; python2 and python3. The packages python-bottle and python-serial have been adapted as examples.
To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.
I don't get why this python-common directory is needed. If the two python are mutually exclusive, why can't we just declare the PYTHON variable in their respective package makefiles? The Config.in file is useless as well, just make a depends on !PYTHON3 in python, and !PYTHON in python3, like Thomas suggested.
Moreover, I'm pretty concerned about the breakage of many configurations. You seem to assume here that all external modules can build on both python and python3, and I'm really not sure about that.
Maxime
Signature · 4 lines
Maxime Ripard, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
v3 cover Python 3, simple, external package support
Hi Maxime,
Quoted 7 lines · Maxime Ripard
Hi Patrick,
Le 13/02/2013 10:20, Patrick Gerber a ?crit :
Quoted 4 lines
This patch series refactor python support in buildroot to enable the use of external package with both; python2 and python3. The packages python-bottle and python-serial have been adapted as examples.
Quoted 8 lines
Quoted 4 lines
To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.
I don't get why this python-common directory is needed. If the two python are mutually exclusive, why can't we just declare the PYTHON variable in their respective package makefiles?
You are right it could be done as suggested.
The Config.in file is
Quoted 2 lines
useless as well, just make a depends on !PYTHON3 in python, and !PYTHON in python3, like Thomas suggested.
As I replied to Thomas this result in strange behaviour of the menu. When selecting/deselecting one of the python entry the menu do not handle properly the "circular reference" and it's almost unusable.
Quoted 3 lines
Moreover, I'm pretty concerned about the breakage of many configurations. You seem to assume here that all external modules can build on both python and python3, and I'm really not sure about that.
No. All the package that don't support python3 have in their Config.in a line like this: depends on BR2_PACKAGE_PYTHON
The same way potential package that support only python3 could have:
depends on BR2_PACKAGE_PYTHON3
Quoted 2 lines
Maxime
Patrick
v3 cover Python 3, simple, external package support
Quoted 11 lines · Patrick
Hi Maxime,
Quoted 5 lines · Maxime Ripard
Hi Patrick,
I don't get why this python-common directory is needed. If the two python are mutually exclusive, why can't we just declare the PYTHON variable in their respective package makefiles?
You are right it could be done as suggested.
Quoted 8 lines
The Config.in file is
Quoted 2 lines
useless as well, just make a depends on !PYTHON3 in python, and !PYTHON in python3, like Thomas suggested.
As I replied to Thomas this result in strange behaviour of the menu. When selecting/deselecting one of the python entry the menu do not handle properly the "circular reference" and it's almost unusable.
Maxim, Thomas,
Does it seems acceptable to you if I keep the menu like proposed in this patch series (including the file python-common/Config.in) but move the PYTHON variable definition in both python.mk and python3.mk and remove the python-common.mk file ?
Comments are welcomes
Patrick