Allow host python3 without a target python
cover Allow host python3 without a target python
It's currently not possible to select the default host python to be python3 without also turning on target python3.
There was a previous patch that attemped to resolve this
https://patchwork.ozlabs.org/patch/854602/ but there were issues.
This should solve those issues, using the approach I described in comment #4.
This series does not yet hunt down every non-python-package that uses host-python to make sure it gets the proper host python version. I just did the packages I needed to test with our project and will wait to see if the approach is accepted before working on the remaining packages.
Trent Piepho (2):
host-python: Select host python version when no target python
packages: Use PKG_PYTHON_HOST_PYTHON for host python
package/Config.in.host | 1 + package/gtest/gtest.mk | 5 ++--- package/ninja/ninja.mk | 3 +-- package/pkg-python.mk | 6 +++++- package/python/python.mk | 2 +- package/python3/Config.in.host | 8 ++++++++ package/python3/python3.mk | 2 +- 7 files changed, 19 insertions(+), 8 deletions(-) create mode 100644 package/python3/Config.in.host
Signature · 1 line
2.14.3
PATCH 1/2 host-python: Select host python version when no target python
Buildroot will select the host python version, 2 or 3, based on the target python version. Without a target python, one gets host python 2.
While both versions of host python can be built, only one can be the default and have setuptools and distutils support. This must match the target python version, but if there is no target python version it could be either 2 or 3. Currently, the default host python will be 2 if there is no target python selected and there is no way to change that.
This patch allows one to select host python 3 in this instance. Iff there is no target python selected, then a host package option for python3 will be available.
If there is a target python selected, then this new option is forced by that and no longer user selectable.
Add variable PKG_PYTHON_HOST_PYTHON to be used by non-python-packages that need host-python. If they can work with either host-python version, they should use this variable to depend on the correct one.
--- package/Config.in.host | 1 + package/pkg-python.mk | 6 +++++- package/python/python.mk | 2 +- package/python3/Config.in.host | 8 ++++++++ package/python3/python3.mk | 2 +- 5 files changed, 16 insertions(+), 3 deletions(-) create mode 100644 package/python3/Config.in.host
package/Config.in.host +1 -0
@@ -43,6 +43,7 @@ menu "Host utilities" source "package/pkgconf/Config.in.host" source "package/pru-software-support/Config.in.host" source "package/pwgen/Config.in.host" + source "package/python3/Config.in.host" source "package/python-lxml/Config.in.host" source "package/python-six/Config.in.host" source "package/python-xlrd/Config.in.host"
package/pkg-python.mk +5 -1
@@ -24,6 +24,10 @@ define PKG_PYTHON_SYSCONFIGDATA_NAME $(basename $(notdir $(wildcard $(STAGING_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/_sysconfigdata_m_linux_*.py))) endef +# If a package, which is not a python-package, needs a host python, but it can +# be either version, use this as the dependency +PKG_PYTHON_HOST_PYTHON = $(if $(BR2_PACKAGE_HOST_PYTHON3),host-python3,host-python) + # Target distutils-based packages PKG_PYTHON_DISTUTILS_ENV = \ PATH=$(BR_PATH) \
@@ -166,7 +170,7 @@ ifeq ($(4),target) $(2)_DEPENDENCIES += $$(if $$(BR2_PACKAGE_PYTHON3),host-python3 python3,host-python python) else ifeq ($$($(2)_NEEDS_HOST_PYTHON),) -$(2)_DEPENDENCIES += $$(if $$(BR2_PACKAGE_PYTHON3),host-python3,host-python) +$(2)_DEPENDENCIES += $$(if $$(BR2_PACKAGE_HOST_PYTHON3),host-python3,host-python) else ifeq ($$($(2)_NEEDS_HOST_PYTHON),python2) $(2)_DEPENDENCIES += host-python
package/python/python.mk +1 -1
@@ -225,7 +225,7 @@ PYTHON_AUTORECONF = YES # Some packages may have build scripts requiring python2. # Only install the python symlink in the host tree if python3 is not enabled # for the target, otherwise the default python program may be missing. -ifneq ($(BR2_PACKAGE_PYTHON3),y) +ifneq ($(BR2_PACKAGE_HOST_PYTHON3),y) define HOST_PYTHON_INSTALL_PYTHON_SYMLINK ln -sf python2 $(HOST_DIR)/bin/python ln -sf python2-config $(HOST_DIR)/bin/python-config
package/python3/Config.in.host new file +8 -0
@@ -0,0 +1,8 @@ +config BR2_PACKAGE_HOST_PYTHON3 + bool "host python3" if !BR2_PACKAGE_PYTHON3 && !BR2_PACKAGE_PYTHON + default BR2_PACKAGE_PYTHON3 + help + Use Python version 3 for the host Python version. + + If Python is enabled for the target, then this must match the target's + Python version and this option is not user selectable.
package/python3/python3.mk +1 -1
@@ -244,7 +244,7 @@ endif # python version chosen for the target. # Only install the python symlink in the host tree if python3 is enabled # for the target. -ifeq ($(BR2_PACKAGE_PYTHON3),y) +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y) define HOST_PYTHON3_INSTALL_SYMLINK ln -fs python3 $(HOST_DIR)/bin/python ln -fs python3-config $(HOST_DIR)/bin/python-config
Signature · 1 line
2.14.3
PATCH 2/2 packages: Use PKG_PYTHON_HOST_PYTHON for host python
In reply to Trent PiephoThese are not python packages, but they still depend on host-python. Have them use $(PKG_PYTHON_HOST_PYTHON) as the dependency so they use the correct python version.
--- package/gtest/gtest.mk | 5 ++--- package/ninja/ninja.mk | 3 +-- 2 files changed, 3 insertions(+), 5 deletions(-)
package/gtest/gtest.mk +2 -3
@@ -17,13 +17,12 @@ endif HOST_GTEST_LICENSE = Apache-2.0 HOST_GTEST_LICENSE_FILES = googlemock/scripts/generator/LICENSE -ifeq ($(BR2_PACKAGE_PYTHON3),y) +ifeq ($(BR2_PACKAGE_HOST_PYTHON3),y) HOST_GTEST_PYTHON_VERSION = $(PYTHON3_VERSION_MAJOR) -HOST_GTEST_DEPENDENCIES += host-python3 else HOST_GTEST_PYTHON_VERSION = $(PYTHON_VERSION_MAJOR) -HOST_GTEST_DEPENDENCIES += host-python endif +HOST_GTEST_DEPENDENCIES += $(PKG_PYTHON_HOST_PYTHON) HOST_GTEST_GMOCK_PYTHONPATH = \ $(HOST_DIR)/lib/python$(HOST_GTEST_PYTHON_VERSION)/site-packages
package/ninja/ninja.mk +1 -2
@@ -8,8 +8,7 @@ NINJA_VERSION = v1.8.2 NINJA_SITE = $(call github,ninja-build,ninja,$(NINJA_VERSION)) NINJA_LICENSE = Apache-2.0 NINJA_LICENSE_FILES = COPYING - -HOST_NINJA_DEPENDENCIES = $(if $(BR2_PACKAGE_PYTHON3),host-python3,host-python) +HOST_NINJA_DEPENDENCIES = $(PKG_PYTHON_HOST_PYTHON) define HOST_NINJA_BUILD_CMDS (cd $(@D); ./configure.py --bootstrap)
Signature · 1 line
2.14.3
cover Allow host python3 without a target python
In reply to Trent PiephoHi Trent,
Thank you for these patches.
Quoted 6 lines · Trent Piepho
It's currently not possible to select the default host python to be python3 without also turning on target python3.
There was a previous patch that attemped to resolve this
https://patchwork.ozlabs.org/patch/854602/ but there were issues.
Quoted 7 lines
This should solve those issues, using the approach I described in comment #4.
This series does not yet hunt down every non-python-package that uses host-python to make sure it gets the proper host python version. I just did the packages I needed to test with our project and will wait to see if the approach is accepted before working on the remaining packages.
We discussed this at the BR developer meeting. The problem is, this change is
not very useful if you don't hunt down all the packages that use host-python. Now, that list is still somewhat reasonable (100-ish). However, remember that any package that uses host-python through the infra (e.g. meson-package) would also need to select a python version. We feel that this is going to add too much to the Config.in files everywhere, and that the potential gains from it are not worth the effort.
Specifically for python, there were two cases where the
host-python-if-no-target-python selection was relevant: setuptools and meson. The setuptools case is already handled through python3-setuptools. For the meson case, there is a really simple solution: make host-ninja depend on host-python3 instead of host-python. host-ninja is currently *only* directly depended upon by host-meson, so in practice we *always* need host-python3...
If there are other cases where we would really like to have a
host-python-2-or-3 config option, it can probably also be worked around on a case-by-case basis.
Therefore, these patches are marked as Rejected in patchwork. Thanks for
working on it though, it was still useful!
Regards,
Arnout
Quoted 15 lines
Trent Piepho (2):
host-python: Select host python version when no target python
packages: UsePKG_PYTHON_HOST_PYTHONfor host pythonpackage/Config.in.host | 1 + package/gtest/gtest.mk | 5 ++--- package/ninja/ninja.mk | 3 +-- package/pkg-python.mk | 6 +++++- package/python/python.mk | 2 +- package/python3/Config.in.host | 8 ++++++++ package/python3/python3.mk | 2 +- 7 files changed, 19 insertions(+), 8 deletions(-) create mode 100644 package/python3/Config.in.host