host-python2 hard dependency

This change adds a patch to python disabling the installation of the python and python-config symlinks.

Signed-off-by Samuel Martin <[email protected]>

Cc Gustavo Zacarias <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 .../python/python-011-remove-python-symlink.patch  | 29 ++++++++++++++++++++
 package/python/python.mk                           | 32 ++++++++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 package/python/python-011-remove-python-symlink.patch

package/python/python-011-remove-python-symlink.patch new file +19 -0

new file mode 100644
index 0000000..a9b1daa
@@ -0,0 +1,29 @@
+Remove the python symlink install rules.
+
+The python symlink installation will be handled by Buildroot itself, because
+Buildroot needs to control to what python interpreter (python2 or python3) the
+python symlink points to.
+
+Signed-off-by: Samuel Martin <[email protected]>
+
+diff -purN a/Makefile.pre.in b/Makefile.pre.in
+--- a/Makefile.pre.in	2014-03-02 17:56:56.529132499 +0100
++++ b/Makefile.pre.in	2014-03-02 19:24:02.346491849 +0100
+@@ -857,17 +857,10 @@ $(DESTSHARED):
+ #  $(PYTHON) -> python2 -> python$(VERSION))
+ # Also create equivalent chains for other installed files
+ bininstall:	altbininstall
+-	-if test -f $(DESTDIR)$(BINDIR)/$(PYTHON) -o -h $(DESTDIR)$(BINDIR)/$(PYTHON); \
+-	then rm -f $(DESTDIR)$(BINDIR)/$(PYTHON); \
+-	else true; \
+-	fi

package/python/python.mk +10 -0

+define PYTHON_INSTALL_TARGET_PYTHON_SYMLINK
+	ln -sf python2 $(TARGET_DIR)/usr/bin/python
+endef
+
+PYTHON_POST_INSTALL_TARGET_HOOKS += PYTHON_INSTALL_TARGET_PYTHON_SYMLINK
+
+#
+# Always install the python-config symlink in the staging tree
+#
+define PYTHON_INSTALL_STAGING_PYTHON_CONFIG_SYMLINK

package/python/python.mk +11 -0

+# for the target.
+#
+ifeq ($(BR2_PACKAGE_PYTHON),y)
+define HOST_PYTHON_INSTALL_PYTHON_SYMLINK
+	ln -sf python2 $(HOST_DIR)/usr/bin/python
+	ln -sf python2-config $(HOST_DIR)/usr/bin/python-config
+endef
+
+HOST_PYTHON_POST_INSTALL_HOOKS += HOST_PYTHON_INSTALL_PYTHON_SYMLINK
+endif
+
 # Provided to other packages
 PYTHON_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/
 

Signature · 6 lines

1.9.0

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

This patch reworks the way python3 and python3-config symlink are installed.

Buildroot wants to control these symlinks' installation:

* the python3 symlink should be unconditionally installed in the target tree, and the python3-config symlink in the staging tree, since it is the only python package built and installed in the target tree if the user selected it;

* the python3 and python3-config symlinks should only be installed in the host tree when python3 is the selection of the user for the target.

Signed-off-by Samuel Martin <[email protected]>

Cc Gustavo Zacarias <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 package/python3/python3.mk | 9 +++++++++
 1 file changed, 9 insertions(+)

package/python3/python3.mk +0 -0

index 3d95a6d..19bef1e 100644

package/python3/python3.mk +2 -0

+	ln -fs python3-config $(HOST_DIR)/usr/bin/python-config
 endef
 
 HOST_PYTHON3_POST_INSTALL_HOOKS += HOST_PYTHON3_INSTALL_SYMLINK
+endif
 
 # Provided to other packages
 PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/sysconfigdata/:$(TARGET_DIR)/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages/

Signature · 1 line

1.9.0

Some packages need a host-python interpreter with a version different from the one installed in the target to run some build scripts (eg. scons requires python2 to run, so build any kind of packages even if

the python interpreter selected for the target is python3).

In such cases, we need to add the right host-python dependency to the package using the host-python-package infrastructure, and we also want to invoke the right host python interpreter during the build steps.

This patch add a *_FORCE_HOST_PYTHON variable that can be set either

to 'python2' or 'python3'. This variable can be set by any package using the host-python-package infrastructure can set to force the python interpreter for the build.

This variable also takes care of
setting the right host-python dependency.

This *_FORCE_HOST_PYTHON variable only affects packages using the host-python-package infrastructure.

If some configure/build/install commands are overloaded in the *.mk file, the right python interpreter should be explicitly called.

If the package define some tool variable (eg.: SCONS), the variable

should explicitly called the right python interpreter.

Signed-off-by Samuel Martin <[email protected]>

Cc Gustavo Zacarias <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 docs/manual/adding-packages-python.txt |  8 ++++++
 package/pkg-python.mk                  | 50 +++++++++++++++++++++++++++++++---
 2 files changed, 54 insertions(+), 4 deletions(-)

docs/manual/adding-packages-python.txt +0 -0

index b8d5331..da97654 100644

docs/manual/adding-packages-python.txt +4 -0

+  the build and will invoke it for the build. If some build steps
+  are overloaded, the right python interpreter must be explicitly
+  called in the commands.
+
 With the Python infrastructure, all the steps required to build and
 install the packages are already defined, and they generally work well
 for most Python-based packages. However, when required, it is still

package/pkg-python.mk +11 -1

index 512ef66..178b494 100644
@@ -143,11 +143,29 @@ $(2)_DEPENDENCIES ?= $(filter-out host-python host-python3 host-python-setuptool
 # Target packages need both the python interpreter on the target (for
 # runtime) and the python interpreter on the host (for
 # compilation). However, host packages only need the python
-# interpreter on the host.
+# interpreter on the host, whose version may be enforced by setting
+# the *_FORCE_HOST_PYTHON variable.
+#
+# So:
+# - for target packages, we always depend on the default python interpreter
+#   (the one selected by the config);
+# - for host packages:
+#   - if *_FORCE_HOST_PYTHON is not set, then we depend on use the default
+#     interperter;
+#   - otherwise, we depend on the one requested by *_FORCE_HOST_PYTHON.
+#

package/pkg-python.mk +13 -0

+endif
+endif
 endif
 
 # Setuptools based packages will need host-python-setuptools (both

@@ -161,6 +179,30 @@ $(2)_DEPENDENCIES += host-python-setuptools
 endif
 endif
 
+# Python interpreter to use for building the package.
+#
+# We may want to specify the python interpreter toi be used for building a
+# package, especially for host-packages (target packages must be built using
+# the same version of the interpreter as the one installed on the target).
+#
+# So:
+# - for target packages, we always use the default python interpreter (which
+#   is the same version as the one built and installed on the target);
+# - for host packages:
+#   - if *_FORCE_HOST_PYTHON is not set, then we use use the default

I thionk we should explicitly check for (empty), 'python2' or 'python3', and abort on any other value:

    ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
    $(2)_DEPENDENCIES += host-python
    else ifeq ($($(2)_FORCE_HOST_PYTHON),python3)
    $(2)_DEPENDENCIES += host-python3
    else
    $$(error Incorrect value '$($(2)_FORCE_HOST_PYTHON)' for $(2)_FORCE_HOST_PYTHON)
    endif
    endif

Quoted 4 lines

I thionk we should explicitly check for (empty), 'python2' or 'python3', and abort on any other value:

Quoted 8 lines

    ifeq ($($(2)_FORCE_HOST_PYTHON),python2)
    $(2)_DEPENDENCIES += host-python
    else ifeq ($($(2)_FORCE_HOST_PYTHON),python3)
    $(2)_DEPENDENCIES += host-python3
    else
    $$(error Incorrect value '$($(2)_FORCE_HOST_PYTHON)' for $(2)_FORCE_HOST_PYTHON)
    endif
    endif

Hum, most packages will *not* define <pkg>_FORCE_HOST_PYTHON I believe. Or do you plan to modify all packages that depend on host-python so that they all define a value for <pkg>_FORCE_HOST_PYTHON ?

I believe the idea was that if <pkg>_FORCE_HOST_PYTHON is not defined, then we default to whatever Python version has been selected.

Scons build-system needs python2 as interpreter (it does not support python3 yet).

So, we need to force the host dependency to get the python2 interperter built and available in the host tree to be able to build host-scons itself and to build scons-based packages, whatever is the python interpreter for the target.

This patch also makes sure scons will in be called using the right python interpreter when invoked via $(SCONS).

Fixes:
http://autobuild.buildroot.net/results/11e6c8c8d79d56ed43daf52e6d6dc80847709926/

Signed-off-by Samuel Martin <[email protected]>

Cc Gustavo Zacarias <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 package/scons/scons.mk | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

package/scons/scons.mk +3 -1

index e3fad1b..88396d4 100644
@@ -10,10 +10,12 @@ SCONS_LICENSE = MIT
 SCONS_LICENSE_FILES = LICENSE.txt
 SCONS_SETUP_TYPE = distutils
 
+HOST_SCONS_FORCE_HOST_PYTHON = python2
+
 HOST_SCONS_INSTALL_OPT = \
 	--install-lib=$(HOST_DIR)/usr/lib/scons-$(SCONS_VERSION)
 
 $(eval $(host-python-package))
 
 # variables used by other packages
-SCONS = $(HOST_DIR)/usr/bin/scons
+SCONS = $(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons

Signature · 6 lines

1.9.0

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Acording to its homepage [1], m2crypto does not support python3 yet.

So, this patch forces building the host m2crypto module for pyhton2.

[1] https://pypi.python.org/pypi/M2Crypto

Signed-off-by Samuel Martin <[email protected]>

Cc Gustavo Zacarias <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 package/python-m2crypto/python-m2crypto.mk | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

package/python-m2crypto/python-m2crypto.mk +7 -3

index dd0f77e..62dc720 100644
@@ -10,12 +10,16 @@ PYTHON_M2CRYPTO_SOURCE = M2Crypto-$(PYTHON_M2CRYPTO_VERSION).tar.gz
 PYTHON_M2CRYPTO_SETUP_TYPE = setuptools
 HOST_PYTHON_M2CRYPTO_DEPENDENCIES = host-openssl host-swig
 
-# We need to override the build commands to be able to use build_ext,
-# which accepts the --openssl option.
+# We need to use python2 because m2crypto is not python3 compliant.
+HOST_PYTHON_M2CRYPTO_FORCE_HOST_PYTHON = python2
+
+# * We need to override the build commands to be able to use build_ext,
+#   which accepts the --openssl option.
+# * Use python2 interpreter to avoid trying building some python3 objects.
 define HOST_PYTHON_M2CRYPTO_BUILD_CMDS
 	(cd $(@D); \
 		$(HOST_PKG_PYTHON_SETUPTOOLS_ENV) \
-		$(HOST_DIR)/usr/bin/python setup.py build_ext \
+		$(HOST_DIR)/usr/bin/python2 setup.py build_ext \
 			--openssl=$(HOST_DIR)/usr)
 endef
 

Signature · 6 lines

1.9.0

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

During its build, crda calls a python script importing m2crypto module which requires python2 interpreter.

So, we have a build dependency against host-python.

Also, we make sure this python script is interpreted using the right python interpreter; we add a patch allowing to enforce this and explicitly set it.

Fixes:
http://autobuild.buildroot.net/results/a1e/a1e53d66d3962fd5431e7601937b071a0a3c2084/

Signed-off-by Samuel Martin <[email protected]>

Cc Gustavo Zacarias <[email protected]>

Cc Thomas Petazzoni <[email protected]>

---
 ...file-allow-to-override-python-interpreter.patch | 34 ++++++++++++++++++++++
 package/crda/crda.mk                               |  8 ++++-
 2 files changed, 41 insertions(+), 1 deletion(-)
 create mode 100644 package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch

package/crda/crda-0001-Makefile-allow-to-override-python-interpreter.patch new file +0 -0

new file mode 100644
index 0000000..2481390

package/crda/crda.mk +1 -1

index ce87d44..0a10816 100644
@@ -8,7 +8,7 @@ CRDA_VERSION = 1.1.3
 CRDA_SOURCE = crda-$(CRDA_VERSION).tar.bz2
 CRDA_SITE = http://wireless.kernel.org/download/crda
 CRDA_DEPENDENCIES = host-pkgconf host-python-m2crypto \
-	libnl libgcrypt
+	libnl libgcrypt host-python
 CRDA_LICENSE = ISC
 CRDA_LICENSE_FILES = LICENSE
 

Samuel Martin (6):
python: rework python symlinks installation
python3: rework python symlinks installation
pkg-python: support host-python dependency different from the python
in the target
scons: force host-python dependency to be python2
python-m2crypto: requires host-python2
crda: override python interperter