Python 3, simple, external package support
PATCH v2 1/1 Python 3, simple, external package support
This patch series refactor python support in buildroot to enable the use of from line 'This patch series refactor python support in buildroot to enable the use of external package with both python2 and python3. The packages python-bottle and python-serial have adapted as examples.
To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.
v2: Add short patch title
PATCH 1/5 Python menu reorganization
In order to have a better support for python external package make the python menu clearer. Python2 and Python3 are now mutually exclusive
--- package/Config.in | 19 +------------ package/python-generic/Config.in | 56 ++++++++++++++++++++++++++++++++++++++ package/python/Config.in | 14 ---------- package/python3/Config.in | 14 ---------- 4 files changed, 57 insertions(+), 46 deletions(-) create mode 100644 package/python-generic/Config.in
package/Config.in +1 -18
@@ -315,24 +315,7 @@ endmenu endif source "package/microperl/Config.in" source "package/php/Config.in" -source "package/python/Config.in" -source "package/python3/Config.in" -if BR2_PACKAGE_PYTHON -menu "external python modules" -source "package/python-bottle/Config.in" -source "package/python-dpkt/Config.in" -source "package/python-id3/Config.in" -source "package/python-mad/Config.in" -source "package/python-meld3/Config.in" -source "package/python-netifaces/Config.in" -source "package/python-nfc/Config.in" -source "package/python-protobuf/Config.in" -source "package/python-pygame/Config.in" -source "package/python-pyparsing/Config.in" -source "package/python-serial/Config.in" -source "package/python-setuptools/Config.in" -endmenu -endif +source "package/python-generic/Config.in" source "package/ruby/Config.in" source "package/tcl/Config.in" if BR2_PACKAGE_TCL
package/python-generic/Config.in new file +56 -0
56 lines not on this page · raw message
package/python/Config.in +0 -14
@@ -1,17 +1,3 @@ -config BR2_PACKAGE_PYTHON - bool "python" - depends on BR2_USE_WCHAR - # uses fork() - depends on BR2_USE_MMU - select BR2_PACKAGE_LIBFFI - help - The python language interpreter. - - http://www.python.org/ - -comment "python requires a toolchain with WCHAR support" - depends on !BR2_USE_WCHAR - if BR2_PACKAGE_PYTHON choice
package/python3/Config.in +0 -14
@@ -1,17 +1,3 @@ -config BR2_PACKAGE_PYTHON3 - bool "python3" - depends on BR2_USE_WCHAR - # uses fork() - depends on BR2_USE_MMU - select BR2_PACKAGE_LIBFFI - help - The python language interpreter. - - http://www.python.org/ - -comment "python3 requires a toolchain with WCHAR support" - depends on !BR2_USE_WCHAR - if BR2_PACKAGE_PYTHON3 choice
Signature · 1 line
1.7.10.4
PATCH 2/5 Add python3 to python simlinks in staging
In reply to Patrick GerberWhen using python3 a symlink "python" is added in staging and point to python3. This is needed to call python setup for external package. A second simlink is added to alias the config-3.3m directory to config
--- package/python3/python3.mk | 8 ++++++++ 1 file changed, 8 insertions(+)
package/python3/python3.mk +8 -0
@@ -160,6 +160,14 @@ ifneq ($(BR2_PACKAGE_PYTHON),y) PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK endif +define HOST_PYTHON3_INSTALL_SYMLINK + ln -fs python3 $(HOST_DIR)/usr/bin/python + ln -fs config-3.3m $(HOST_DIR)/usr/lib/python3.3/config +endef + +HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK + + ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y) define PYTHON3_REMOVE_MODULES_FILES for i in `find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) \
Signature · 1 line
1.7.10.4
PATCH 2/5 Add python3 to python simlinks in staging
Hi Patrick,
2013/2/12 Patrick Gerber <[email protected]>:
Quoted 1 line
When using python3 a symlink "python" is added in staging and point to
s/staging/host directory/
Quoted 2 lines
python3. This is needed to call python setup for external package. A second simlink is added to alias the config-3.3m directory to config
s/simlink/symlink/
Quoted 11 lines
Signed-off-by Patrick Gerber <[email protected]>--- package/python3/python3.mk | 8 ++++++++ 1 file changed, 8 insertions(+)package/python3/python3.mk +0
-0
package/python3/python3.mk ifneq ($(BR2_PACKAGE_PYTHON),y) · line 168
Quoted 9 lines
package/python3/python3.mk +6
-0@@ -160,6 +160,14 @@ ifneq ($(BR2_PACKAGE_PYTHON),y) PYTHON3_POST_INSTALL_TARGET_HOOKS += PYTHON3_INSTALL_SYMLINK endif +define HOST_PYTHON3_INSTALL_SYMLINK + ln -fs python3 $(HOST_DIR)/usr/bin/python + ln -fs config-3.3m $(HOST_DIR)/usr/lib/python3.3/config +endef + +HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
BTW, I'm a bit concerned about the consequencies this may have...
From my experience, having python poiting to python3 in the PATH has some bad effect,
and some patch have been posted some time ago to fix this issue:
http://patchwork.ozlabs.org/patch/208803/
Also, I'm dubious about having python poiting to python3. Though there is nothing really stated, there is usually a consensus:
"python" refers to the
python-2 interpreter if it exists on the system, otherwise it refers to the python-3 one (if existing).
What worries me is that, python (i.e. python 2.6 or 2.7) is currently a mandatory for Buildroot
(see: http://buildroot.org/downloads/manual/manual.html#requirement-mandatory),
and here,
you are overriding it with python3 in the PATH inside Buildroot.
I undersatnd that, when BR2_PACKAGE_PYTHON3_PY_ONLY is enabled, you want to build
all modules using python3, but meanwhile some other build scripts may use 'python' referring
to python2 and may badly failed because they will trigger python3...
I may have missed something... so, enlighten me please.
Regards,
Signature · 1 line
Samuel
PATCH 3/5 Add common declaration to ease the support of both version
In reply to Patrick GerberDefine a new variable called PYTHON_GENERIC_DEPENDENCIES either to "python" or "python3" depending of the selected Python version. This is needed to easily set dependencies for packages that could be used either with Python 2 or Python 3.
--- package/python-generic/python-generic.mk | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 package/python-generic/python-generic.mk
package/python-generic/python-generic.mk new file +16 -0
@@ -0,0 +1,16 @@ +############################################################# +# +# python-generic +# +############################################################# +# PYTHON_GENERIC_DEPENDENCIES is defined to the selected python +# version. + +ifeq ($(BR2_PACKAGE_PYTHON),y) + PYTHON_GENERIC_DEPENDENCIES = python +endif + +ifeq ($(BR2_PACKAGE_PYTHON3),y) + PYTHON_GENERIC_DEPENDENCIES = python3 +endif +
Signature · 1 line
1.7.10.4
PATCH 4/5 python-bottle: Convert package to be used with python2 or python3
In reply to Patrick Gerber--- package/python-bottle/Config.in | 1 + package/python-bottle/python-bottle.mk | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
package/python-bottle/Config.in +1 -0
@@ -1,5 +1,6 @@ config BR2_PACKAGE_PYTHON_BOTTLE bool "python-bottle" + depends on (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3) help Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no
package/python-bottle/python-bottle.mk +1 -1
@@ -7,7 +7,7 @@ PYTHON_BOTTLE_VERSION = 0.11.4 PYTHON_BOTTLE_SOURCE = bottle-$(PYTHON_BOTTLE_VERSION).tar.gz PYTHON_BOTTLE_SITE = http://pypi.python.org/packages/source/b/bottle -PYTHON_BOTTLE_DEPENDENCIES = python +PYTHON_BOTTLE_DEPENDENCIES = $(PYTHON_GENERIC_DEPENDENCIES) PYTHON_BOTTLE_LICENSE = MIT # README.rst refers to the file "LICENSE" but it's not included
Signature · 1 line
1.7.10.4
PATCH 4/5 python-bottle: Convert package to be used with python2 or python3
Dear Patrick Gerber,
Quoted 12 lines · Patrick Gerber
Signed-off-by Patrick Gerber <[email protected]>--- package/python-bottle/Config.in | 1 + package/python-bottle/python-bottle.mk | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)package/python-bottle/Config.in +0
-0
package/python-bottle/Config.in line 3
Quoted 3 lines
package/python-bottle/Config.in +1
-0@@ -1,5 +1,6 @@ config BR2_PACKAGE_PYTHON_BOTTLE bool "python-bottle" + depends on (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3)
This change is useless, since the python-bottle package is only available when either BR2_PACKAGE_PYTHON or BR2_PACKAGE_PYTHON3 are enabled:
+if (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3)
+menu "external python modules"
+source "package/python-bottle/Config.in"
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH 5/5 python-serial: Convert package to be used with python2 or python3
In reply to Patrick Gerber--- package/python-serial/Config.in | 2 +- package/python-serial/python-serial.mk | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
package/python-serial/Config.in +1 -1
@@ -1,6 +1,6 @@ config BR2_PACKAGE_PYTHON_SERIAL bool "python-serial" - depends on BR2_PACKAGE_PYTHON + depends on (BR2_PACKAGE_PYTHON || BR2_PACKAGE_PYTHON3) help python-serial is a Python library to access serial ports.
package/python-serial/python-serial.mk +1 -1
@@ -8,7 +8,7 @@ PYTHON_SERIAL_VERSION = 2.6 PYTHON_SERIAL_SOURCE = pyserial-$(PYTHON_SERIAL_VERSION).tar.gz PYTHON_SERIAL_SITE = http://pypi.python.org/packages/source/p/pyserial/ -PYTHON_SERIAL_DEPENDENCIES = python +PYTHON_SERIAL_DEPENDENCIES = $(PYTHON_GENERIC_DEPENDENCIES) define PYTHON_SERIAL_BUILD_CMDS (cd $(@D); $(HOST_DIR)/usr/bin/python setup.py build)
Signature · 1 line
1.7.10.4
PATCH v2 1/1 Python 3, simple, external package support
In reply to Patrick GerberDear Patrick Gerber,
Quoted 8 lines · Patrick Gerber
This patch series refactor python support in buildroot to enable the use of from line 'This patch series refactor python support in buildroot to enable the use of external package with both python2 and python3. The packages python-bottle and python-serial have adapted as examples.
To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.
Thanks, but it still doesn't answer the question:
"""
As Samuel pointed it out, this unfortunately may break a number of things in Buildroot. They are packages that depend on host-python, and would only be capable of using an Python 2.x host-python.
Or at least, I would like to have the confirmation that all packages that depend on host-python continue to build normally when host-python is Python 3.
That said, some packages are not really clear about this. For example, Scons uses the Python provided with the distribution, and not the host-python built by Buildroot.
As you can see, this is not a trivial topic.
"""
Best regards,
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH v2 1/1 Python 3, simple, external package support
Quoted 27 lines · Thomas Petazzoni
Dear Patrick Gerber,
Quoted 8 lines · Patrick Gerber
This patch series refactor python support in buildroot to enable the use of from line 'This patch series refactor python support in buildroot to enable the use of external package with both python2 and python3. The packages python-bottle and python-serial have adapted as examples.
To support others external package more work is needed as they are based on python setuptools. It's no more supported by python3 and should be replaced by "Distribute". This should be done in another patch series.
Thanks, but it still doesn't answer the question:
"""
As Samuel pointed it out, this unfortunately may break a number of things in Buildroot. They are packages that depend on host-python, and would only be capable of using an Python 2.x host-python.Or at least, I would like to have the confirmation that all packages that depend on host-python continue to build normally when host-python is Python 3.
That said, some packages are not really clear about this. For example, Scons uses the Python provided with the distribution, and not the host-python built by Buildroot.
Quoted 7 lines
As you can see, this is not a trivial topic.
"""Best regards,
Thomas
Related with host-python, the only difference is that now the patch add a symlink in HOST_DIR. I agree that this could be a problem.
I could edit the external python package to use "python" or "python3" depending of the selected version. This will not impact host-python at all. Does it seems good to you ?
Patrick
PATCH v2 1/1 Python 3, simple, external package support
Dear Patrick,
Quoted 6 lines · Patrick
Related with host-python, the only difference is that now the patch add a symlink in
HOST_DIR. I agree that this could be a problem.I could edit the external python package to use "python" or "python3" depending of the selected version. This will not impact host-python at all. Does it seems good to you ?
No, it doesn't solve the problem of packages that currently depend on host-python, and that expect a Python 2 to be installed.
You seem to be focused only on Python external modules, but you completely forget that there are packages in Buildroot that depend on host-python as their build dependencies, even though those packages are not directly Python related.
For example, are you sure that mesa3d and libxcb will built properly once host-python is Python 3 ?
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH v2 1/1 Python 3, simple, external package support
Quoted 19 lines · Thomas Petazzoni
Dear Patrick,
Quoted 6 lines · Patrick
Related with host-python, the only difference is that now the patch add a symlink in
HOST_DIR. I agree that this could be a problem.I could edit the external python package to use "python" or "python3" depending of the selected version. This will not impact host-python at all. Does it seems good to you ?
No, it doesn't solve the problem of packages that currently depend on host-python, and that expect a Python 2 to be installed.
You seem to be focused only on Python external modules, but you completely forget that there are packages in Buildroot that depend on host-python as their build dependencies, even though those packages are not directly Python related.
Quoted 5 lines
For example, are you sure that mesa3d and libxcb will built properly once host-python is Python 3 ?
Thomas
Thomas, I am aware that there is others packages that depends on host-python, mostly x11. But I have not changed the way host-python nor host-python3 are build and used. I don't understand how the changes I have made could break this compared as previously ? If I'm not mistaken, host-python is selected directly by the package (mesa3d for example) when needed, and this is independent of the selected python package, right ?
Patrick
PATCH v2 1/1 Python 3, simple, external package support
Dear Patrick,
Quoted 6 lines · Patrick
Thomas, I am aware that there is others packages that depends on host-python, mostly x11. But I have not changed the way host-python nor host-python3 are build and used. I don't understand how the changes I have made could break this compared as previously ? If I'm not mistaken, host-python is selected directly by the package (mesa3d for example) when needed, and this is independent of the selected python package, right ?
True.
So we might end up with two versions of Python installed in $(HOST_DIR): python2 and python3. Hopefully, it should work.
Regarding the patch set, I think I would prefer to rename the current python package to python2, and have the 'python' package be the virtual package that provides either python2 or python3. That's indeed more work, but I find the name "python-generic" not so great.
Thomas
Signature · 4 lines
Thomas Petazzoni, Free Electrons Kernel, drivers, real-time and embedded Linux development, consulting, training and support. http://free-electrons.com
PATCH v2 1/1 Python 3, simple, external package support
Dear Thomas,
Quoted 11 lines · Thomas Petazzoni
Dear Patrick,
Quoted 6 lines · Patrick
Thomas, I am aware that there is others packages that depends on host-python, mostly x11. But I have not changed the way host-python nor host-python3 are build and used. I don't understand how the changes I have made could break this compared as previously ? If I'm not mistaken, host-python is selected directly by the package (mesa3d for example) when needed, and this is independent of the selected python package, right ?
Quoted 5 lines
True.
So we might end up with two versions of Python installed in $(HOST_DIR): python2 and python3. Hopefully, it should work.
This is already the case if I enable python3, no ?
Quoted 4 lines
Regarding the patch set, I think I would prefer to rename the current python package to python2, and have the 'python' package be the virtual package that provides either python2 or python3. That's indeed more work, but I find the name "python-generic" not so great.
I have tried like you suggest, but then I have to rename python.mk to python2.mk and then any occurrence of python or host-python to python2 and host-python2 in many package dependencies, no ?
But I could rename python-generic to python-common or something like that ?