Skip to contentSkip to conversation

python: clean up host version logic.

PATCH New Adam Duskett 4 participants 17 messages Jan 2, 2018 – Jan 9, 2018

lore
Share & tools
lore

Adam Duskett

author
#0 lore raw

cover python: clean up host version logic.

Currently, packages that rely on host-python, and are compatible with both python2 and 3, only build python3 if python3 is selected for the target.

This causes a problem in the following scenario:

- A user does not have a target python selected.
- Package A depends on host-python3
- Package B is compatible with host-python and host-python3

In this scenario, host-python is not needed, as package A and B are both compatible with python3. However; both host-python and host-python3 will be built because no target-python has been selected.

To fix this, the following patch series introduces a few minimal changes that will allow the user to manually select a host version of python. (Note: I have changed only the packages that I personally use and have tested.)

Adam Duskett (8):
python: add host-entry
python3: add host-entry
python-setuptools: check host-python version
pkg-waf.mk: check host-python version
ninja: check host-python version
libselinux: check host-python version
setools: check host-python version
libselinux: check host-python version

 package/Config.in.host                         |  2 ++
 package/libselinux/libselinux.mk               |  2 +-
 package/libsemanage/libsemanage.mk             |  2 +-
 package/ninja/ninja.mk                         |  2 +-
 package/pkg-waf.mk                             | 20 +++++++++++++-------
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 package/python/Config.in.host                  |  6 ++++++
 package/python3/Config.in.host                 |  6 ++++++
 package/setools/setools.mk                     |  6 ++++++
 9 files changed, 45 insertions(+), 11 deletions(-)
 create mode 100644 package/python/Config.in.host
 create mode 100644 package/python3/Config.in.host
Signature · 1 line
2.14.3
author
#1 lore raw

PATCH 1/8 python: add selectable host entry

Add a user selectable entry to build python2 as a host package.

Signed-off-by Adam Duskett <[email protected]>
---
 package/Config.in.host        | 1 +
 package/python/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python/Config.in.host
package/Config.in.host +1 -0
index dd6415bba5..7d908c409c 100644
@@ -41,6 +41,7 @@ menu "Host utilities"
 	source "package/pkgconf/Config.in.host"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
+	source "package/python/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
package/python/Config.in.host new file +6 -0
new file mode 100644
index 0000000000..0a8d6d179f
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON
+	bool "host python"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
Signature · 1 line
2.14.3
author
#2 lore raw

PATCH 2/8 python3: add selectable host entry

In reply to Adam Duskett

Add a user selectable entry to build python3 as a host package.

Signed-off-by Adam Duskett <[email protected]>
---
 package/Config.in.host         | 1 +
 package/python3/Config.in.host | 6 ++++++
 2 files changed, 7 insertions(+)
 create mode 100644 package/python3/Config.in.host
package/Config.in.host +1 -0
index 7d908c409c..19d45e9d9e 100644
@@ -42,6 +42,7 @@ menu "Host utilities"
 	source "package/pru-software-support/Config.in.host"
 	source "package/pwgen/Config.in.host"
 	source "package/python/Config.in.host"
+	source "package/python3/Config.in.host"
 	source "package/python-lxml/Config.in.host"
 	source "package/python-six/Config.in.host"
 	source "package/qemu/Config.in.host"
package/python3/Config.in.host new file +6 -0
new file mode 100644
index 0000000000..876661e71b
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_PYTHON3
+	bool "host python3"
+	help
+	  The python language interpreter.
+
+	  http://www.python.org/
Signature · 1 line
2.14.3
author
#3 lore raw

PATCH 3/8 python-setuptools: check host-python version

In reply to Adam Duskett

python-setuptools is compatible with both python2 and python3, as such, there is no need to force python2 as a dependency.

Signed-off-by Adam Duskett <[email protected]>
---
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
package/python-setuptools/python-setuptools.mk +9 -1
index 28d4f29b5b..a21494a288 100644
@@ -10,10 +10,18 @@ PYTHON_SETUPTOOLS_LICENSE = MIT
 PYTHON_SETUPTOOLS_LICENSE_FILES = LICENSE
 PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif
+
 # recent setuptools versions require bootstrap.py to be invoked
 # before the standard setup process.
 define PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
-	cd  $(@D) && $(HOST_DIR)/bin/python ./bootstrap.py
+	cd  $(@D) && $(HOST_DIR)/bin/$(HOST_PYTHON_SETUPTOOLS_PYLIBVER) ./bootstrap.py
 endef
 
 PYTHON_SETUPTOOLS_PRE_CONFIGURE_HOOKS = PYTHON_SETUPTOOLS_RUN_BOOTSTRAP
Signature · 1 line
2.14.3
#4 lore raw

PATCH 3/8 python-setuptools: check host-python version

Hello,

Quoted 13 lines · Adam Duskett

python-setuptools is compatible with both python2 and python3, as such, there is no need to force python2 as a dependency.

Signed-off-by Adam Duskett <[email protected]>
---
 package/python-setuptools/python-setuptools.mk | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
package/python-setuptools/python-setuptools.mk +0 -0
index 28d4f29b5b..a21494a288 100644

package/python-setuptools/python-setuptools.mk PYTHON_SETUPTOOLS_LICENSE = MIT · line 19

Quoted 10 lines
package/python-setuptools/python-setuptools.mk +7 -0
@@ -10,10 +10,18 @@ PYTHON_SETUPTOOLS_LICENSE = MIT
 PYTHON_SETUPTOOLS_LICENSE_FILES = LICENSE
 PYTHON_SETUPTOOLS_SETUP_TYPE = setuptools
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif

So, now, if the user selects:

BR2_PACKAGE_PYTHON=y
BR2_PACKAGE_PYTHON_ARROW=y
BR2_PACKAGE_HOST_PYTHON3=y

then the build will break. Indeed, since python2 is selected as the Python for the target, python-arrow should be built and installed for python2, so it will expect host-python-setuptools to be built and installed for host-python.

But because the user has selected BR2_PACKAGE_HOST_PYTHON3, host-python-setuptools will be installed for host-python3.

So, I believe the build breaks with your change.

Basically the current reasoning of the current Python packaging is:

- The user selects the target version of Python, and from that we derive:

* That we need to build the same Python version for the host

* That all host Python modules should be built installed for that Python version

- Optionally, packages may depend on the other host-python{,3} if they really need that specific Python version. However, we do not support installing third party Python modules for this secondary host Python version.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#5 lore raw

PATCH 3/8 python-setuptools: check host-python version

In reply to Adam Duskett

package/python-setuptools/python-setuptools.mk PYTHON_SETUPTOOLS_LICENSE = MIT · line 19

Quoted 11 lines · Adam Duskett

python-setuptools is compatible with both python2 and python3, as such, there is no need to force python2 as a dependency.

package/python-setuptools/python-setuptools.mk +7 -0
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif

From why I see in pkg-python, the default behavior for host packages is to depend on the default interpreter, which is python3 if enabled otherwise python2. Isn't the above block basically doing the same thing? Other than using HOST_PYTHON3 rather than PYTHON3.

But shouldn't this be changed in pkg-python?

Then setting <PKG>_NEEDS_HOST_PYTHON based on why python version is in use could be eliminated, unless the package needs a specific version, which appear to be how it's supposed to work.

#6 lore raw

PATCH 3/8 python-setuptools: check host-python version

Hello,

Quoted 18 lines · Trent Piepho

package/python-setuptools/python-setuptools.mk PYTHON_SETUPTOOLS_LICENSE = MIT · line 19

Quoted 11 lines · Adam Duskett

python-setuptools is compatible with both python2 and python3, as such, there is no need to force python2 as a dependency.

package/python-setuptools/python-setuptools.mk +7 -0
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif

From why I see in pkg-python, the default behavior for host packages is to depend on the default interpreter, which is python3 if enabled otherwise python2. Isn't the above block basically doing the same thing? Other than using HOST_PYTHON3 rather than PYTHON3.

Quoted 5 lines

But shouldn't this be changed in pkg-python?

Then setting <PKG>_NEEDS_HOST_PYTHON based on why python version is in use could be eliminated, unless the package needs a specific version, which appear to be how it's supposed to work.

See my reply to this patch, which explains why it is (IMO) broken.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#7 lore raw

PATCH 3/8 python-setuptools: check host-python version

Quoted 16 lines · Thomas Petazzoni
Quoted 15 lines · Trent Piepho

package/python-setuptools/python-setuptools.mk PYTHON_SETUPTOOLS_LICENSE = MIT · line 19

Quoted 8 lines · Adam Duskett
package/python-setuptools/python-setuptools.mk +7 -0
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python3
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
+else
+HOST_PYTHON_SETUPTOOLS_NEEDS_HOST_PYTHON += python
+HOST_PYTHON_SETUPTOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
+endif

From why I see in pkg-python, the default behavior for host packages is to depend on the default interpreter, which is python3 if enabled otherwise python2. Isn't the above block basically doing the same thing? Other than using HOST_PYTHON3 rather than PYTHON3.

Quoted 8 lines
Quoted 5 lines

But shouldn't this be changed in pkg-python?

Then setting <PKG>_NEEDS_HOST_PYTHON based on why python version is in use could be eliminated, unless the package needs a specific version, which appear to be how it's supposed to work.

See my reply to this patch, which explains why it is (IMO) broken.

If I understand correctly, the fundamental problem is that Adam's patch allows for simultaneously enabling both host python2 and host python3 and building host python packages using both python versions or a version different than the target python version.

But I think Adam pointed out a real flaw that can be fixed. It's caused problems for me as well.

The issue is that the host python version is linked to the target python version. Which is fine, if there is a target python. But the logic fails if there is no target python.

Right now, one gets this:
Target Python 2 -> Host Python 2
Target Python 3 -> Host Python 3
No Target Python -> Host Python 2

I'd like to build host python 3 utilities and don't have python on the target. Can't see a way to configure buildroot to do that. I can add a build dep on host-python3 and get the interpreter built, but I can't make a host-python-package using python3.

I think this could be fixed by making host python config options, which are forced by the target python configs, so that host python can only be selected manually if no target python is enabled.

Packages that want to depend on host python should use the host python config to select which python to depends on, rather than the target python config as they do now. If target python is enabled, then host python == target python so there is no change. But without target python it will be possible to build select either host-python3 or host- python as the python version to be used by host-python-setuptools.

#8 lore raw

PATCH 3/8 python-setuptools: check host-python version

Hello,

Quoted 4 lines · Trent Piepho

If I understand correctly, the fundamental problem is that Adam's patch allows for simultaneously enabling both host python2 and host python3 and building host python packages using both python versions or a version different than the target python version.

Correct.

Quoted 11 lines

But I think Adam pointed out a real flaw that can be fixed. It's caused problems for me as well.

The issue is that the host python version is linked to the target python version. Which is fine, if there is a target python. But the logic fails if there is no target python.

Right now, one gets this:
Target Python 2 -> Host Python 2
Target Python 3 -> Host Python 3
No Target Python -> Host Python 2

Absolutely.

Quoted 9 lines

I'd like to build host python 3 utilities and don't have python on the target. Can't see a way to configure buildroot to do that. I can add a build dep on host-python3 and get the interpreter built, but I can't make a host-python-package using python3.

I think this could be fixed by making host python config options, which are forced by the target python configs, so that host python can only be selected manually if no target python is enabled.

Quoted 6 lines

Packages that want to depend on host python should use the host python config to select which python to depends on, rather than the target python config as they do now. If target python is enabled, then host python == target python so there is no change. But without target python it will be possible to build select either host-python3 or host- python as the python version to be used by host-python-setuptools.

Yes, this would be possible. Feel free to make a proposal in this direction.

In general, we have already discussed several times adding Config.in options for all host packages. They would be hidden options for most of the packages, but they would still be visible for some packages (like we have today in Config.in.host).

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
author
#9 lore raw

PATCH 4/8 pkg-waf.mk: check host-python version

In reply to Adam Duskett

waf is compatible with both python2 and python3. Now that there are host entries for python, the make logic can be changed to check if host-python3 has been selected, and if so, depend on it instead of explicitly depending only on python2.

Signed-off-by Adam Duskett <[email protected]>
---
 package/pkg-waf.mk | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
package/pkg-waf.mk +13 -7
index e5b606f063..8df287074b 100644
@@ -36,9 +36,6 @@
 
 define inner-waf-package
 
-# We need host-python to run waf
-$(2)_DEPENDENCIES += host-python
-
 $(2)_NEEDS_EXTERNAL_WAF ?= NO
 
 # If the package does not have its own waf, use our own.

@@ -49,6 +46,15 @@ else
 $(2)_WAF = ./waf
 endif
 
+# We need a host-python to run waf
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+$(2)_DEPENDENCIES += host-python3
+$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python3
+else
+$(2)_DEPENDENCIES += host-python
+$(2)_WAF_PYTHON_PATH ?= $$(HOST_DIR)/bin/python2
+endif
+
 $(2)_BUILD_OPTS				?=
 $(2)_INSTALL_STAGING_OPTS		?=
 $(2)_INSTALL_TARGET_OPTS		?=

@@ -63,7 +69,7 @@ define $(2)_CONFIGURE_CMDS
 	cd $$(@D) && \
 	$$(TARGET_CONFIGURE_OPTS) \
 	$$($(2)_CONF_ENV) \
-	$$(HOST_DIR)/bin/python2 $$($(2)_WAF) configure \
+	$$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) configure \
 		--prefix=/usr \
 		--libdir=/usr/lib \
 		$$($(2)_CONF_OPTS) \

@@ -78,7 +84,7 @@ endif
 ifndef $(2)_BUILD_CMDS
 define $(2)_BUILD_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		build -j $$(PARALLEL_JOBS) $$($(2)_BUILD_OPTS) \
 		$$($(2)_WAF_OPTS)
 endef

@@ -91,7 +97,7 @@ endif
 ifndef $(2)_INSTALL_STAGING_CMDS
 define $(2)_INSTALL_STAGING_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		install --destdir=$$(STAGING_DIR) \
 		$$($(2)_INSTALL_STAGING_OPTS) \
 		$$($(2)_WAF_OPTS)

@@ -105,7 +111,7 @@ endif
 ifndef $(2)_INSTALL_TARGET_CMDS
 define $(2)_INSTALL_TARGET_CMDS
 	cd $$(@D) && \
-	$$(TARGET_MAKE_ENV) $$(HOST_DIR)/bin/python2 $$($(2)_WAF) \
+	$$(TARGET_MAKE_ENV) $$($(2)_WAF_PYTHON_PATH) $$($(2)_WAF) \
 		install --destdir=$$(TARGET_DIR) \
 		$$($(2)_INSTALL_TARGET_OPTS) \
 		$$($(2)_WAF_OPTS)
Signature · 1 line
2.14.3
author
#10 lore raw

PATCH 5/8 ninja: check host-python version

In reply to Adam Duskett

Check if host-python3 is selected instead of target-python3.

Signed-off-by Adam Duskett <[email protected]>
---
 package/ninja/ninja.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/ninja/ninja.mk +1 -1
index 259a87d216..b58d1a0f43 100644
@@ -9,7 +9,7 @@ NINJA_SITE = $(call github,ninja-build,ninja,$(NINJA_VERSION))
 NINJA_LICENSE = Apache-2.0
 NINJA_LICENSE_FILES = COPYING
 
-HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)
+HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_HOST_PYTHON3),host-python3,host-python)
 
 define HOST_NINJA_BUILD_CMDS
 	(cd $(@D); ./configure.py --bootstrap)
Signature · 1 line
2.14.3
author
#11 lore raw

PATCH 6/8 libselinux: check host-python version

In reply to Adam Duskett

Check if host-python3 is selected instead of target-python3.

Signed-off-by Adam Duskett <[email protected]>
---
 package/libselinux/libselinux.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/libselinux/libselinux.mk +1 -1
index 8ac8000de5..f8e8ea1285 100644
@@ -75,7 +75,7 @@ endef
 HOST_LIBSELINUX_DEPENDENCIES = \
 	host-libsepol host-pcre host-swig
 
-ifeq ($(BR2_PACKAGE_PYTHON3),y)
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
 HOST_LIBSELINUX_DEPENDENCIES += host-python3
 HOST_LIBSELINUX_PYINC = -I$(HOST_DIR)/include/python$(PYTHON3_VERSION_MAJOR)m/
 HOST_LIBSELINUX_PYLIBVER = python$(PYTHON3_VERSION_MAJOR)
Signature · 1 line
2.14.3
author
#12 lore raw

PATCH 7/8 setools: check host-python version

In reply to Adam Duskett

Check if host-python3 is selected instead of target-python3.

Signed-off-by Adam Duskett <[email protected]>
---
 package/setools/setools.mk | 6 ++++++
 1 file changed, 6 insertions(+)
package/setools/setools.mk +6 -0
index 6748c95c23..f4644b091c 100644
@@ -20,6 +20,12 @@ SETOOLS_PYLIBVER = python$(PYTHON_VERSION_MAJOR)
 SETOOLS_DEPENDENCIES += python-enum34
 endif
 
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
+HOST_SETOOLS_NEEDS_HOST_PYTHON += python3
+else
+HOST_SETOOLS_NEEDS_HOST_PYTHON += python
+endif
+
 define SETOOLS_FIX_SETUP
 	# By default, setup.py will look for libsepol.a in the host machines
 	# /usr/lib directory. This needs to be changed to the staging directory.
Signature · 1 line
2.14.3
author
#13 lore raw

PATCH 8/8 libselinux: check host-python version

In reply to Adam Duskett

Check if host-python3 is selected instead of target-python3.

Signed-off-by Adam Duskett <[email protected]>
---
 package/libsemanage/libsemanage.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/libsemanage/libsemanage.mk +1 -1
index 829409f527..b9a98c7b6c 100644
@@ -37,7 +37,7 @@ HOST_LIBSEMANAGE_MAKE_OPTS += \
 	PREFIX=$(HOST_DIR) \
 	SWIG_LIB="$(HOST_DIR)/share/swig/$(SWIG_VERSION)/"
 
-ifeq ($(BR2_PACKAGE_PYTHON3),y)
+ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y)
 HOST_LIBSEMANAGE_DEPENDENCIES += host-python3
 HOST_LIBSEMANAGE_MAKE_OPTS += \
 	PYINC="-I$(HOST_DIR)/include/python$(PYTHON3_VERSION_MAJOR)m/" \
Signature · 1 line
2.14.3
#14 lore raw

cover python: clean up host version logic.

In reply to Adam Duskett

Hello,

Quoted 9 lines · Adam Duskett

Currently, packages that rely on host-python, and are compatible with both python2 and 3, only build python3 if python3 is selected for the target.

This causes a problem in the following scenario:

- A user does not have a target python selected.
- Package A depends on host-python3
- Package B is compatible with host-python and host-python3

Quoted 7 lines

In this scenario, host-python is not needed, as package A and B are both compatible with python3. However; both host-python and host-python3 will be built because no target-python has been selected.

To fix this, the following patch series introduces a few minimal changes that will allow the user to manually select a host version of python. (Note: I have changed only the packages that I personally use and have tested.)

I will have to think a lot to fully grasp the potential consequences of your changes.

However, there is one thing that I believe is missing in your patch
series: BR2_PACKAGE_PYTHON3 should select BR2_PACKAGE_HOST_PYTHON3, and BR2_PACKAGE_PYTHON should select BR2_PACKAGE_HOST_PYTHON.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
#15 lore raw

cover python: clean up host version logic.

Thomas, Adam, All,

On 2018-01-02 21:40 +0100, Thomas Petazzoni spake thusly:

Quoted 18 lines · Adam Duskett
Quoted 9 lines

Currently, packages that rely on host-python, and are compatible with both python2 and 3, only build python3 if python3 is selected for the target.

This causes a problem in the following scenario:

- A user does not have a target python selected.
- Package A depends on host-python3
- Package B is compatible with host-python and host-python3

Quoted 7 lines

In this scenario, host-python is not needed, as package A and B are both compatible with python3. However; both host-python and host-python3 will be built because no target-python has been selected.

To fix this, the following patch series introduces a few minimal changes that will allow the user to manually select a host version of python. (Note: I have changed only the packages that I personally use and have tested.)

Quoted 6 lines

I will have to think a lot to fully grasp the potential consequences of your changes.

However, there is one thing that I believe is missing in your patch
series: BR2_PACKAGE_PYTHON3 should select BR2_PACKAGE_HOST_PYTHON3, and BR2_PACKAGE_PYTHON should select BR2_PACKAGE_HOST_PYTHON.

Furthermore, I believe the prompt should be hidden, because it does not make sense for the user to actually select the host version for python.

Besides, the current series allows for building both host-python and host-python3 in the same configuration, and we do not support this for now (or we would have made it possible for the target variant too).

Regards,
Yann E. MORIN.

Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#16 lore raw

cover python: clean up host version logic.

Hello,

Quoted 6 lines · Yann E. MORIN
Quoted 3 lines

However, there is one thing that I believe is missing in your patch
series: BR2_PACKAGE_PYTHON3 should select BR2_PACKAGE_HOST_PYTHON3, and BR2_PACKAGE_PYTHON should select BR2_PACKAGE_HOST_PYTHON.

Furthermore, I believe the prompt should be hidden, because it does not make sense for the user to actually select the host version for python.

If the prompt is not user-selectable, then I don't see the point of the patch series. Indeed, you can already do:

<pkg>_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python)

to use host-python3 if already built as part of the python3 build process, or fallback to host-python otherwise. We already use this construct in a number of places.

Quoted 3 lines

Besides, the current series allows for building both host-python and host-python3 in the same configuration, and we do not support this for now (or we would have made it possible for the target variant too).

Of course we support building both host-python and host-python3. What we don't support is:

- python and python3 in the same configuration

- building third-party host python modules for a host python version different from the selected target python version

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com