Skip to contentSkip to conversation

support/dependencies: add a check for python3

PATCH New Romain Naour 4 participants 6 messages Mar 12, 2019 – Mar 15, 2019

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 support/dependencies: add a check for python3

Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library [1].

We add a new check to verify the version of python3 interpreter installed on the host. If no suitable python3 interpreter is found, define BR2_PYTHON3_HOST_DEPENDENCY to add host-python3 in package dependencies when needed.

[1] https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

---
v2: fix the third argument handling in the check-host-python3.sh (Yann)
rework the python version handling to simplify the check (Adam)

---
 support/dependencies/check-host-python3.mk | 13 +++++++++++++
 support/dependencies/check-host-python3.sh | 31 ++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
 create mode 100644 support/dependencies/check-host-python3.mk
 create mode 100755 support/dependencies/check-host-python3.sh
support/dependencies/check-host-python3.mk new file +13 -0
new file mode 100644
index 0000000000..ceb48c969e
@@ -0,0 +1,13 @@
+# Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library.
+# https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html
+
+# Set this to either 3.4 or higher, depending on the highest minimum
+# version required by any of the packages bundled in Buildroot. If a
+# package is bumped or a new one added, and it requires a higher
+# version, our package infra will catch it and whine.
+#
+BR2_PYTHON3_VERSION_MIN = 3.4
+
+ifeq (,$(call suitable-host-package,python3,$(BR2_PYTHON3_VERSION_MIN) python3 python))
+BR2_PYTHON3_HOST_DEPENDENCY = host-python3
+endif
support/dependencies/check-host-python3.sh new file +31 -0
new file mode 100755
index 0000000000..17cafd2883
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+# prevent shift error
+[ $# -lt 2 ] && exit 1
+
+version_min="$(echo ${1} | awk '{ split($1, v, "."); print v[1] v[2] }')"
+
+shift
+
+# The host python interpreter is already checked by dependencies.sh but
+# it only check if the version is at least 2.7.
+# We want to check the version number of the python3 interpreter even
+# if Buildroot is able to use any version but some packages may require
+# a more recent version.
+
+for candidate in "${@}" ; do
+	python3=`which $candidate 2>/dev/null`
+	if [ ! -x "$python3" ]; then
+		continue
+	fi
+	version=`$python3 -V 2>&1 | awk '{ split($2, v, "."); print v[1] v[2] }'`
+
+	if [ $version -lt $version_min ]; then
+		# no suitable python3 found
+		continue
+	fi
+
+	# suitable python3 found
+	echo $python3
+	break
+done
Signature · 1 line
2.14.5
#1 lore raw

PATCH v2 2/2 package/glibc: bump to version 2.29

Changes to build and runtime requirements:

* Python 3.4 or later is required to build the GNU C Library.

* On most architectures, GCC 5 or later is required to build the GNU C Library. (On powerpc64le, GCC 6.2 or later is still required, as before.)

While at it, remove the double "glibc-" prefix in the version.

https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

---
v2: remove "glibc-" prefix in the version (Yann).

---
 package/gcc/Config.in.host                                          | 6 ++----
 .../glibc.hash                                                      | 2 +-
 package/glibc/glibc.mk                                              | 6 +++---
 3 files changed, 6 insertions(+), 8 deletions(-)
 rename package/glibc/{glibc-2.28-69-g1e5c5303a522764d7e9d2302a60e4a32cdb902f1 => 2.29-6-g067fc32968b601493f4b247a3ac00caeea3f3d61}/glibc.hash (69%)
package/gcc/Config.in.host +2 -4
index 8ec406a117..222251f43c 100644
@@ -30,10 +30,8 @@ config BR2_GCC_VERSION_4_9_X
 	depends on !(BR2_TOOLCHAIN_USES_MUSL && (BR2_microblazeel || BR2_microblazebe))
 	depends on !(BR2_TOOLCHAIN_USES_MUSL && (BR2_powerpc64 || BR2_powerpc64le))
 	depends on !(BR2_TOOLCHAIN_USES_MUSL && (BR2_mips64 || BR2_mips64el))
-	# glibc >= 2.26 needs gcc >= 6.2
-	depends on !(BR2_TOOLCHAIN_USES_GLIBC && BR2_powerpc64le)
-	# glibc >= 2.27 needs gcc >= 5
-	depends on !(BR2_TOOLCHAIN_USES_GLIBC && (BR2_aarch64 || BR2_aarch64_be))
+	# glibc >= 2.29 needs gcc >= 5 on all architectures.
+	depends on !BR2_TOOLCHAIN_USES_GLIBC
 	select BR2_TOOLCHAIN_GCC_AT_LEAST_4_9
 
 config BR2_GCC_VERSION_5_X
package/glibc/2.29-6-g067fc32968b601493f4b247a3ac00caeea3f3d61/glibc.hash renamed from package/glibc/glibc-2.28-69-g1e5c5303a522764d7e9d2302a60e4a32cdb902f1/glibc.hash +1 -1
similarity index 69%
rename from package/glibc/glibc-2.28-69-g1e5c5303a522764d7e9d2302a60e4a32cdb902f1/glibc.hash
rename to package/glibc/2.29-6-g067fc32968b601493f4b247a3ac00caeea3f3d61/glibc.hash
index e83b1caf4c..56e4bf1c18 100644
@@ -1,5 +1,5 @@
 # Locally calculated (fetched from Github)
-sha256  ebf04c7b00153d6df8beceec0666d4b13e1ac613b40d5774d1b8c6f61c1686e6  glibc-glibc-2.28-69-g1e5c5303a522764d7e9d2302a60e4a32cdb902f1.tar.gz
+sha256  fdc2f7966eac7071ac4d66bc38d9236476d670f042645f9566746a1fd42a6a9d  glibc-2.29-6-g067fc32968b601493f4b247a3ac00caeea3f3d61.tar.gz
 
 # Hashes for license files
 sha256  8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643  COPYING
package/glibc/glibc.mk +3 -3
index ec5b3cedc9..cb867d4d5b 100644
@@ -12,8 +12,8 @@ GLIBC_VERSION = 4e2943456e690d89f48e6e710757dd09404b0c9a
 GLIBC_SITE = $(call github,riscv,riscv-glibc,$(GLIBC_VERSION))
 else
 # Generate version string using:
-#   git describe --match 'glibc-*' --abbrev=40 origin/release/MAJOR.MINOR/master
-GLIBC_VERSION = glibc-2.28-69-g1e5c5303a522764d7e9d2302a60e4a32cdb902f1
+#   git describe --match 'glibc-*' --abbrev=40 origin/release/MAJOR.MINOR/master | cut -d '-' -f 2-
+GLIBC_VERSION = 2.29-6-g067fc32968b601493f4b247a3ac00caeea3f3d61
 # Upstream doesn't officially provide an https download link.
 # There is one (https://sourceware.org/git/glibc.git) but it's not reliable,
 # sometimes the connection times out. So use an unofficial github mirror.
@@ -32,7 +32,7 @@ GLIBC_ADD_TOOLCHAIN_DEPENDENCY = NO
 # Before glibc is configured, we must have the first stage
 # cross-compiler and the kernel headers
 GLIBC_DEPENDENCIES = host-gcc-initial linux-headers host-bison host-gawk \
-	$(BR2_MAKE_HOST_DEPENDENCY)
+	$(BR2_MAKE_HOST_DEPENDENCY) $(BR2_PYTHON3_HOST_DEPENDENCY)
 
 GLIBC_SUBDIR = build
 
Signature · 1 line
2.14.5
#2 lore raw

PATCH v2 2/2 package/glibc: bump to version 2.29

Quoted 13 lines · On Tue, 12 Mar 2019 23:35:25 +0100 Romain Naour <[email protected]> wrote:

Changes to build and runtime requirements:

* Python 3.4 or later is required to build the GNU C Library.

* On most architectures, GCC 5 or later is required to build the GNU C Library. (On powerpc64le, GCC 6.2 or later is still required, as before.)

While at it, remove the double "glibc-" prefix in the version.

https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

Quoted 8 lines

---
v2: remove "glibc-" prefix in the version (Yann).

---
 package/gcc/Config.in.host                                          | 6 ++----
 .../glibc.hash                                                      | 2 +-
 package/glibc/glibc.mk                                              | 6 +++---
 3 files changed, 6 insertions(+), 8 deletions(-)
 rename package/glibc/{glibc-2.28-69-g1e5c5303a522764d7e9d2302a60e4a32cdb902f1 => 2.29-6-g067fc32968b601493f4b247a3ac00caeea3f3d61}/glibc.hash (69%)

Applied to master, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw

PATCH v2 1/2 support/dependencies: add a check for python3

In reply to Romain Naour

All;

Quoted 83 lines · Romain Naour

Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library [1].

We add a new check to verify the version of python3 interpreter installed on the host. If no suitable python3 interpreter is found, define BR2_PYTHON3_HOST_DEPENDENCY to add host-python3 in package dependencies when needed.

[1] https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

---
v2: fix the third argument handling in the check-host-python3.sh (Yann)
rework the python version handling to simplify the check (Adam)

---
 support/dependencies/check-host-python3.mk | 13 +++++++++++++
 support/dependencies/check-host-python3.sh | 31 ++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
 create mode 100644 support/dependencies/check-host-python3.mk
 create mode 100755 support/dependencies/check-host-python3.sh
support/dependencies/check-host-python3.mk new file +13 -0
new file mode 100644
index 0000000000..ceb48c969e
@@ -0,0 +1,13 @@
+# Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library.
+# https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html
+
+# Set this to either 3.4 or higher, depending on the highest minimum
+# version required by any of the packages bundled in Buildroot. If a
+# package is bumped or a new one added, and it requires a higher
+# version, our package infra will catch it and whine.
+#
+BR2_PYTHON3_VERSION_MIN = 3.4
+
+ifeq (,$(call suitable-host-package,python3,$(BR2_PYTHON3_VERSION_MIN) python3 python))
+BR2_PYTHON3_HOST_DEPENDENCY = host-python3
+endif
support/dependencies/check-host-python3.sh new file +31 -1
new file mode 100755
index 0000000000..17cafd2883
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+# prevent shift error
+[ $# -lt 2 ] && exit 1
+
+version_min="$(echo ${1} | awk '{ split($1, v, "."); print v[1] v[2] }')"
+
+shift
+
+# The host python interpreter is already checked by dependencies.sh but
+# it only check if the version is at least 2.7.
+# We want to check the version number of the python3 interpreter even
+# if Buildroot is able to use any version but some packages may require
+# a more recent version.
+
+for candidate in "${@}" ; do
+       python3=`which $candidate 2>/dev/null`
+       if [ ! -x "$python3" ]; then
+               continue
+       fi
+       version=`$python3 -V 2>&1 | awk '{ split($2, v, "."); print v[1] v[2] }'`
+
+       if [ $version -lt $version_min ]; then
+               # no suitable python3 found
+               continue
+       fi
+
+       # suitable python3 found
+       echo $python3
+       break
+done
--

2.14.5

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#4 lore raw

PATCH v2 1/2 support/dependencies: add a check for python3

In reply to Romain Naour

Romain, All,

On 2019-03-12 23:35 +0100, Romain Naour spake thusly:

Quoted 9 lines

Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library [1].

We add a new check to verify the version of python3 interpreter installed on the host. If no suitable python3 interpreter is found, define BR2_PYTHON3_HOST_DEPENDENCY to add host-python3 in package dependencies when needed.

[1] https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

However, I have a nit-pick, see below...

Quoted 16 lines

---
v2: fix the third argument handling in the check-host-python3.sh (Yann)
rework the python version handling to simplify the check (Adam)

---
 support/dependencies/check-host-python3.mk | 13 +++++++++++++
 support/dependencies/check-host-python3.sh | 31 ++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
 create mode 100644 support/dependencies/check-host-python3.mk
 create mode 100755 support/dependencies/check-host-python3.sh
support/dependencies/check-host-python3.mk new file +0 -0
new file mode 100644
index 0000000000..ceb48c969e

support/dependencies/check-host-python3.mk line 7

Quoted 7 lines
support/dependencies/check-host-python3.mk new file +7 -0
@@ -0,0 +1,13 @@
+# Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library.
+# https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html
+
+# Set this to either 3.4 or higher, depending on the highest minimum
+# version required by any of the packages bundled in Buildroot. If a
+# package is bumped or a new one added, and it requires a higher
+# version, our package infra will catch it and whine.

So, I know that you just mimicked the existing blurb from another check, but I find the wording pretty confusing: it seems to imply that if a new package (or package version) is added, and it has a requirement on a newer python, then the infra would cnotice and whine.

This is wrong. The infra can't detect such new requirement.

This is inherited from the 'make' test, which has the same wording, that it in turn inherited from the 'cmake' test, which has a similar, but not equivalent, wording:

If a package is bumped or a new one added, and it requires a higher version, our cmake infra will catch it and build its own.

This again is wrong: we can't detect that a package has a requirement on another cmake version.

However, in the cmake case, what happens is that cmake will actually whine and break at build time, because cmake itself does that check.

Similarly, it's glibc's ./configure script that would whine and fail when it notices that make's version is too old.

And similarly as well, it's again glibc's ./configure that will whine and break if python3's version is too old.

It's never the infra that automatically detects that a package has a requirement on a newer version of a tool. If that were so, then we would not have to write the version string in the test-script to begin with...

So, I'd suggest we just drop that last sentence in the whole three test-scripts.

Regards,
Yann E. MORIN.

Quoted 49 lines
support/dependencies/check-host-python3.mk new file +5 -0
+BR2_PYTHON3_VERSION_MIN = 3.4
+
+ifeq (,$(call suitable-host-package,python3,$(BR2_PYTHON3_VERSION_MIN) python3 python))
+BR2_PYTHON3_HOST_DEPENDENCY = host-python3
+endif
support/dependencies/check-host-python3.sh new file +31 -0
new file mode 100755
index 0000000000..17cafd2883
@@ -0,0 +1,31 @@
+#!/bin/sh
+
+# prevent shift error
+[ $# -lt 2 ] && exit 1
+
+version_min="$(echo ${1} | awk '{ split($1, v, "."); print v[1] v[2] }')"
+
+shift
+
+# The host python interpreter is already checked by dependencies.sh but
+# it only check if the version is at least 2.7.
+# We want to check the version number of the python3 interpreter even
+# if Buildroot is able to use any version but some packages may require
+# a more recent version.
+
+for candidate in "${@}" ; do
+	python3=`which $candidate 2>/dev/null`
+	if [ ! -x "$python3" ]; then
+		continue
+	fi
+	version=`$python3 -V 2>&1 | awk '{ split($2, v, "."); print v[1] v[2] }'`
+
+	if [ $version -lt $version_min ]; then
+		# no suitable python3 found
+		continue
+	fi
+
+	# suitable python3 found
+	echo $python3
+	break
+done
Signature · 6 lines
2.14.5

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#5 lore raw

PATCH v2 1/2 support/dependencies: add a check for python3

In reply to Romain Naour
Quoted 9 lines · On Tue, 12 Mar 2019 23:35:24 +0100 Romain Naour <[email protected]> wrote:

Since version 2.29, glibc requires python 3.4 or later to build the GNU C Library [1].

We add a new check to verify the version of python3 interpreter installed on the host. If no suitable python3 interpreter is found, define BR2_PYTHON3_HOST_DEPENDENCY to add host-python3 in package dependencies when needed.

[1] https://www.sourceware.org/ml/libc-alpha/2019-01/msg00723.html

Applied to master, with one change (see below). Thanks!

support/dependencies/check-host-python3.mk line 7

Quoted 4 lines
support/dependencies/check-host-python3.mk new file +4 -0
+# Set this to either 3.4 or higher, depending on the highest minimum
+# version required by any of the packages bundled in Buildroot. If a
+# package is bumped or a new one added, and it requires a higher
+# version, our package infra will catch it and whine.

Yann complained about this comment, and particularly the last sentence. In fact to me, even the first sentence doesn't make much sense. So I dropped the entire paragraph when applying.

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com