allow build infrastructure to pick up installed meson tool
PATCH 1/2 allow build infrastructure to pick up installed meson tool
From: Norbert Lange <[email protected]>
Automatically check for an available meson tool,
and use it aslond the version is fitting.
Some concerns are about being able to figure out the correct version for all available packages in buildroot. The min version could be set to version 1.0 to postpone the problem, but still have the infrastructure in place to allow users to override the version.
Currently host-ninja will still be always built.
--- package/meson/meson.mk | 1 - package/pkg-meson.mk | 4 +-- support/dependencies/check-host-meson.mk | 16 +++++++++ support/dependencies/check-host-meson.sh | 45 ++++++++++++++++++++++++ 4 files changed, 63 insertions(+), 3 deletions(-) create mode 100644 support/dependencies/check-host-meson.mk create mode 100755 support/dependencies/check-host-meson.sh
package/meson/meson.mk +0 -1
@@ -10,7 +10,6 @@ MESON_LICENSE = Apache-2.0 MESON_LICENSE_FILES = COPYING MESON_SETUP_TYPE = setuptools -HOST_MESON_DEPENDENCIES = host-ninja HOST_MESON_NEEDS_HOST_PYTHON = python3 HOST_MESON_TARGET_ENDIAN = $(call LOWERCASE,$(BR2_ENDIAN))
package/pkg-meson.mk +2 -2
@@ -25,7 +25,7 @@ # $(HOST_DIR)/bin/python3 will not look for Meson modules in # $HOME/.local/lib/python3.x/site-packages # -MESON = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/meson +MESON = PYTHONNOUSERSITE=y $(BR2_MESON) NINJA = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/ninja NINJA_OPTS = $(if $(VERBOSE),-v) -j$(PARALLEL_JOBS)
@@ -105,7 +105,7 @@ endef endif endif -$(2)_DEPENDENCIES += host-meson +$(2)_DEPENDENCIES += $(BR2_MESON_HOST_DEPENDENCY) # # Build step. Only define it if not already defined by the package .mk
support/dependencies/check-host-meson.mk new file +16 -0
@@ -0,0 +1,16 @@ +# Set this to either 0.49 or higher, depending on the highest minimum +# version required by any of the packages bundled in Buildroot. If a +# package is bumped or a new one added, and it requires a higher +# version, our meson infra will catch it and build its own. +# +BR2_MESON_VERSION_MIN = 0.49 + +BR2_MESON_CANDIDATES ?= meson +BR2_MESON ?= $(call suitable-host-package,meson,\ + $(BR2_MESON_VERSION_MIN) $(BR2_MESON_CANDIDATES)) +ifeq ($(BR2_MESON),) +BR2_MESON = $(HOST_DIR)/bin/meson +BR2_MESON_HOST_DEPENDENCY = host-meson host-ninja +else +BR2_MESON_HOST_DEPENDENCY = host-ninja +endif
support/dependencies/check-host-meson.sh new file +45 -0
45 lines not on this page · raw message
--
2.20.1
PATCH 2/2 allow build infrastructure to pick up installed ninja tool
From: Norbert Lange <[email protected]>
If a fitting ninja tool is detected it will be used, otherwise the tool will be built from source.
Replace the fixed dependencies to host-ninja,
notably from the meson infrastructure.
--- package/pkg-meson.mk | 2 +- support/dependencies/check-host-meson.mk | 4 +-- support/dependencies/check-host-ninja.mk | 14 ++++++++ support/dependencies/check-host-ninja.sh | 45 ++++++++++++++++++++++++ 4 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 support/dependencies/check-host-ninja.mk create mode 100755 support/dependencies/check-host-ninja.sh
package/pkg-meson.mk +1 -1
@@ -26,7 +26,7 @@ # $HOME/.local/lib/python3.x/site-packages # MESON = PYTHONNOUSERSITE=y $(BR2_MESON) -NINJA = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/ninja +NINJA = PYTHONNOUSERSITE=y $(BR2_NINJA) NINJA_OPTS = $(if $(VERBOSE),-v) -j$(PARALLEL_JOBS) ################################################################################
support/dependencies/check-host-meson.mk +2 -2
@@ -10,7 +10,7 @@ BR2_MESON ?= $(call suitable-host-package,meson,\ $(BR2_MESON_VERSION_MIN) $(BR2_MESON_CANDIDATES)) ifeq ($(BR2_MESON),) BR2_MESON = $(HOST_DIR)/bin/meson -BR2_MESON_HOST_DEPENDENCY = host-meson host-ninja +BR2_MESON_HOST_DEPENDENCY = host-meson $(BR2_NINJA_HOST_DEPENDENCY) else -BR2_MESON_HOST_DEPENDENCY = host-ninja +BR2_MESON_HOST_DEPENDENCY = $(BR2_NINJA_HOST_DEPENDENCY) endif
support/dependencies/check-host-ninja.mk new file +14 -0
@@ -0,0 +1,14 @@ +# Set this to either 1.8.2 or higher, depending on the highest minimum +# version required by any of the packages bundled in Buildroot. If a +# package is bumped or a new one added, and it requires a higher +# version, our ninja infra will catch it and build its own. +# +BR2_NINJA_VERSION_MIN = 1.8.0 + +BR2_NINJA_CANDIDATES ?= ninja +BR2_NINJA ?= $(call suitable-host-package,ninja,\ + $(BR2_NINJA_VERSION_MIN) $(BR2_NINJA_CANDIDATES)) +ifeq ($(BR2_NINJA),) +BR2_NINJA = $(HOST_DIR)/bin/ninja +BR2_NINJA_HOST_DEPENDENCY = host-ninja +endif
support/dependencies/check-host-ninja.sh new file +45 -0
45 lines not on this page · raw message
--
2.20.1
PATCH 2/2 allow build infrastructure to pick up installed ninja tool
Hello Norbert,
Quoted 9 lines · On Fri, 3 May 2019 15:18:51 +0200 Norbert Lange <[email protected]> wrote:
From: Norbert Lange <[email protected]>
If a fitting ninja tool is detected it will be used, otherwise the tool will be built from source.
Replace the fixed dependencies to host-ninja,
notably from the meson infrastructure.Signed-off-by Norbert Lange <[email protected]>
Since host-meson cannot be changed to use the system-provided meson, as discussed with Peter Seiderer on PATCH 1/2, I don't think it's really worth doing the effort for host-ninja, which requires the same thing as
host-meson: a host Python interpreter.
I think the approaches to reduce the build time are:
- Make sure we only need to build one of host-python or host-python3, and not both. This is what J?rg Krause is proposing in
http://patchwork.ozlabs.org/patch/1047119/.
- Or perhaps, try to go one step further, and see if we can use the system-provided Python instead of a Python built by Buildroot. But that would only work if the system provides Python 3, since Meson requires Python 3.
support/dependencies/check-host-ninja.mk line 4
Quoted 10 lines
support/dependencies/check-host-ninja.mk new file +4
-0@@ -0,0 +1,14 @@ +# Set this to either 1.8.2 or higher, depending on the highest minimum +# version required by any of the packages bundled in Buildroot. If a +# package is bumped or a new one added, and it requires a higher +# version, our ninja infra will catch it and build its own.
I know this comment is in other .mk files in the same directory, but it doesn't make any sense. There is no such thing as a "ninja infra" in Buildroot, and there is nothing that will "catch it" and build its own. If we forget to update BR2_NINJA_VERSION_MIN to the oldest Ninja version that is acceptable by all packages using Ninja in Buildroot, we will never notice until a user hits the problem.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 2/2 allow build infrastructure to pick up installed ninja tool
Am Di., 7. Mai 2019 um 23:00 Uhr schrieb Thomas Petazzoni <[email protected]>:
Quoted 14 lines
Hello Norbert,
Quoted 8 lines · On Fri, 3 May 2019 15:18:51 +0200 Norbert Lange <[email protected]> wrote:
From: Norbert Lange <[email protected]>
If a fitting ninja tool is detected it will be used, otherwise the tool will be built from source.
Replace the fixed dependencies to host-ninja,
notably from the meson infrastructure.
Quoted 6 lines
Quoted 1 line
Signed-off-by Norbert Lange <[email protected]>Since host-meson cannot be changed to use the system-provided meson, as discussed with Peter Seiderer on PATCH 1/2, I don't think it's really worth doing the effort for host-ninja, which requires the same thing as
host-meson: a host Python interpreter.
The effort is already done in the patch and not really a big one. Is there anything bad about
having some additional requirements matched?
But yes, most of the time is spent building python (twice).
Quoted 10 lines
I think the approaches to reduce the build time are:
- Make sure we only need to build one of host-python or host-python3, and not both. This is what J?rg Krause is proposing in
http://patchwork.ozlabs.org/patch/1047119/.- Or perhaps, try to go one step further, and see if we can use the system-provided Python instead of a Python built by Buildroot. But that would only work if the system provides Python 3, since Meson requires Python 3.
maybe add some python-any dependency (resolves to either 2 or 3), which is enough for bootstrapping host-ninja?
Quoted 10 lines
Quoted 10 lines
support/dependencies/check-host-ninja.mk new file +3
-0@@ -0,0 +1,14 @@ +# Set this to either 1.8.2 or higher, depending on the highest minimum +# version required by any of the packages bundled in Buildroot. If a +# package is bumped or a new one added, and it requires a higher
Quoted 8 lines
support/dependencies/check-host-ninja.mk line 4
Quoted 1 line
support/dependencies/check-host-ninja.mk new file +1
-0+# version, our ninja infra will catch it and build its own.I know this comment is in other .mk files in the same directory, but it doesn't make any sense. There is no such thing as a "ninja infra" in Buildroot, and there is nothing that will "catch it" and build its own. If we forget to update
BR2_NINJA_VERSION_MINto the oldest Ninja version that is acceptable by all packages using Ninja in Buildroot, we will never notice until a user hits the problem.
That's something what your CI builder could catch.
packages break all the time with new compilers and libc's and I think ninja should be pretty much painless in comparison.
The meta-buildsystems like mesons should be able to check for a matching version (which might depend on the build, like C++20 modules will need some features in ninja)
Norbert.
PATCH 1/2 allow build infrastructure to pick up installed meson tool
In reply to Norbert LangeHello Norbert,
appreciating every patch which speeds up the buildroot build process ;-), but some nits, one minor comments and one major concern, see below...
Quoted 4 lines · Norbert Lange
From: Norbert Lange <[email protected]>
Automatically check for an available meson tool,
and use it aslond the version is fitting.
s/aslond/as long/
Quoted 4 lines
Some concerns are about being able to figure out the correct version for all available packages in buildroot. The min version could be set to version 1.0 to postpone the
You mean 1.0 to disable host meson usage?
Quoted 55 lines
problem, but still have the infrastructure in place to allow users to override the version.
Currently host-ninja will still be always built.
Signed-off-by Norbert Lange <[email protected]>--- package/meson/meson.mk | 1 - package/pkg-meson.mk | 4 +-- support/dependencies/check-host-meson.mk | 16 +++++++++ support/dependencies/check-host-meson.sh | 45 ++++++++++++++++++++++++ 4 files changed, 63 insertions(+), 3 deletions(-) create mode 100644 support/dependencies/check-host-meson.mk create mode 100755 support/dependencies/check-host-meson.shpackage/meson/meson.mk +0
-1@@ -10,7 +10,6 @@ MESON_LICENSE = Apache-2.0 MESON_LICENSE_FILES = COPYING MESON_SETUP_TYPE = setuptools -HOST_MESON_DEPENDENCIES = host-ninja HOST_MESON_NEEDS_HOST_PYTHON = python3 HOST_MESON_TARGET_ENDIAN = $(call LOWERCASE,$(BR2_ENDIAN))package/pkg-meson.mk +2
-2@@ -25,7 +25,7 @@ # $(HOST_DIR)/bin/python3 will not look for Meson modules in # $HOME/.local/lib/python3.x/site-packages # -MESON = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/meson +MESON = PYTHONNOUSERSITE=y $(BR2_MESON) NINJA = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/ninja NINJA_OPTS = $(if $(VERBOSE),-v) -j$(PARALLEL_JOBS)@@ -105,7 +105,7 @@ endef endif endif -$(2)_DEPENDENCIES += host-meson +$(2)_DEPENDENCIES += $(BR2_MESON_HOST_DEPENDENCY) # # Build step. Only define it if not already defined by the package .mksupport/dependencies/check-host-meson.mk new file +0
-0
support/dependencies/check-host-meson.mk line 6
Quoted 6 lines
support/dependencies/check-host-meson.mk new file +6
-0@@ -0,0 +1,16 @@ +# Set this to either 0.49 or higher, depending on the highest minimum +# version required by any of the packages bundled in Buildroot. If a +# package is bumped or a new one added, and it requires a higher +# version, our meson infra will catch it and build its own. +# +BR2_MESON_VERSION_MIN = 0.49
Needs support for third version number, see latest meson bump to 0.50 and revert, but 0.50.1 worked ([1], [2], [3])...
The major concern is no distribution will ship meson with the still needed buildroot patch, see [4]...
Regards,
Peter
[1] https://git.buildroot.net/buildroot/commit/?id=114e9dcd28e1001b74689215ec669b8940dc3ea9
[2] https://git.buildroot.net/buildroot/commit/?id=c37b81af64dd8d6729325d8edbe633e6a186bb3d
[3] https://git.buildroot.net/buildroot/commit/?id=70fb5e610761dd6468a058c580acc3d4f81fe547
[4] https://git.buildroot.net/buildroot/tree/package/meson/0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch
Quoted 68 lines
support/dependencies/check-host-meson.mk new file +10
-0+ +BR2_MESON_CANDIDATES ?= meson +BR2_MESON ?= $(call suitable-host-package,meson,\ + $(BR2_MESON_VERSION_MIN) $(BR2_MESON_CANDIDATES)) +ifeq ($(BR2_MESON),) +BR2_MESON = $(HOST_DIR)/bin/meson +BR2_MESON_HOST_DEPENDENCY = host-meson host-ninja +else +BR2_MESON_HOST_DEPENDENCY = host-ninja +endifsupport/dependencies/check-host-meson.sh new file +45
-1@@ -0,0 +1,45 @@ +#!/bin/sh + +# prevent shift error +[ $# -lt 2 ] && exit 1 + +split_version() { + local VARPREFIX + local NUMBERS + local major + local minor + + VARPREFIX=$1 + NUMBERS=$2 + + major=${NUMBERS%%\.*} + NUMBERS=${NUMBERS#$major}; NUMBERS=${NUMBERS#\.} + minor=${NUMBERS%%\.*} + NUMBERS=${NUMBERS#$minor}; NUMBERS=${NUMBERS#\.} + + # ensure that missing values are 0 + eval "${VARPREFIX}_major=\$major; ${VARPREFIX}_minor=\$((minor + 0)); ${VARPREFIX}_bugfix=\$((NUMBERS + 0));" +} + +split_version req "$1" + +shift + +for candidate; do + + # Try to locate the candidate. Discard it if not located. + meson=$(which "${candidate}" 2>/dev/null) + [ -n "${meson}" ] || continue + + split_version cur "$("${meson}" --version)" + + [ -n "${cur_major}" -a "${cur_major}" -ge "${req_major}" ] || continue + [ "${cur_major}" -gt "${req_major}" ] || [ "${cur_minor}" -ge "${req_minor}" ] || continue + [ "${cur_minor}" -gt "${req_minor}" ] || [ "${cur_bugfix}" -ge "${req_bugfix}" ] || continue + + echo "${meson}" + exit +done + +# echo nothing: no suitable meson found +exit 1 --2.20.1
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
PATCH 1/2 allow build infrastructure to pick up installed meson tool
Am Fr., 3. Mai 2019 um 18:57 Uhr schrieb Peter Seiderer <[email protected]>:
Quoted 6 lines
Hello Norbert,
appreciating every patch which speeds up the buildroot build process ;-), but some nits, one minor comments and one major concern, see below...
Quoted 8 lines
Quoted 4 lines · Norbert Lange
From: Norbert Lange <[email protected]>
Automatically check for an available meson tool,
and use it aslond the version is fitting.s/aslond/as long/
Ack
Quoted 6 lines
Quoted 4 lines
Some concerns are about being able to figure out the correct version for all available packages in buildroot. The min version could be set to version 1.0 to postpone the
You mean 1.0 to disable host meson usage?
Yes, so meson is always built like it is now, if this is too critical to enable by default.
Quoted 56 lines
Quoted 56 lines
problem, but still have the infrastructure in place to allow users to override the version.
Currently host-ninja will still be always built.
Signed-off-by Norbert Lange <[email protected]>--- package/meson/meson.mk | 1 - package/pkg-meson.mk | 4 +-- support/dependencies/check-host-meson.mk | 16 +++++++++ support/dependencies/check-host-meson.sh | 45 ++++++++++++++++++++++++ 4 files changed, 63 insertions(+), 3 deletions(-) create mode 100644 support/dependencies/check-host-meson.mk create mode 100755 support/dependencies/check-host-meson.shpackage/meson/meson.mk +0
-1@@ -10,7 +10,6 @@ MESON_LICENSE = Apache-2.0 MESON_LICENSE_FILES = COPYING MESON_SETUP_TYPE = setuptools -HOST_MESON_DEPENDENCIES = host-ninja HOST_MESON_NEEDS_HOST_PYTHON = python3 HOST_MESON_TARGET_ENDIAN = $(call LOWERCASE,$(BR2_ENDIAN))package/pkg-meson.mk +2
-2@@ -25,7 +25,7 @@ # $(HOST_DIR)/bin/python3 will not look for Meson modules in # $HOME/.local/lib/python3.x/site-packages # -MESON = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/meson +MESON = PYTHONNOUSERSITE=y $(BR2_MESON) NINJA = PYTHONNOUSERSITE=y $(HOST_DIR)/bin/ninja NINJA_OPTS = $(if $(VERBOSE),-v) -j$(PARALLEL_JOBS)@@ -105,7 +105,7 @@ endef endif endif -$(2)_DEPENDENCIES += host-meson +$(2)_DEPENDENCIES += $(BR2_MESON_HOST_DEPENDENCY) # # Build step. Only define it if not already defined by the package .mksupport/dependencies/check-host-meson.mk new file +1
-0@@ -0,0 +1,16 @@ +# Set this to either 0.49 or higher, depending on the highest minimum
Quoted 8 lines
support/dependencies/check-host-meson.mk line 6
Quoted 5 lines
support/dependencies/check-host-meson.mk new file +5
-0+# version required by any of the packages bundled in Buildroot. If a +# package is bumped or a new one added, and it requires a higher +# version, our meson infra will catch it and build its own. +# +BR2_MESON_VERSION_MIN = 0.49Needs support for third version number, see latest meson bump to 0.50 and revert, but 0.50.1 worked ([1], [2], [3])...
There is support for up to 3 version components in support/dependencies/check-host-meson.sh
Quoted 3 lines
The major concern is no distribution will ship meson with the still needed buildroot patch, see [4]...
Ok, news to me.
Whats the practical effect though, does rpath even work if you relocated buildroot?
Is it necessary if you set LD_LIBRARY_PATH?
(From my personal experience I was never happy with RPATH).
Quoted 3 lines
Regards,
Peter
Regards, Norbert
PATCH 1/2 allow build infrastructure to pick up installed meson tool
In reply to Peter SeidererHello,
Quoted 2 lines · On Fri, 3 May 2019 18:57:21 +0200 Peter Seiderer <[email protected]> wrote:
The major concern is no distribution will ship meson with the still needed buildroot patch, see [4]...
Indeed, that makes it impractical/impossible to use a system-provided meson. If we want to get there at some point, the 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch patch needs to be upstreamed in a form or another.
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/2 allow build infrastructure to pick up installed meson tool
Am Di., 7. Mai 2019 um 22:56 Uhr schrieb Thomas Petazzoni <[email protected]>:
Quoted 6 lines
Hello,
On Fri, 3 May 2019 18:57:21 +0200
Peter Seiderer <[email protected]> wrote:
Quoted 7 lines
Quoted 2 lines
The major concern is no distribution will ship meson with the still needed buildroot patch, see [4]...
Indeed, that makes it impractical/impossible to use a system-provided meson. If we want to get there at some point, the 0001-Only-fix-RPATH-if-install_rpath-is-not-empty.patch patch needs to be upstreamed in a form or another.
I see why you would not want to depend on a system provided meson, but it does work for target libraries.
So my proposal would be to incorporate the changes now, with min-version set to a value that always evaluates to false.
So a user like me can enable the system meson easily, even if its not officially supported or tested.
Norbert