Skip to contentSkip to conversation

support/dependencies: introduce BR2_HOST_CMAKE_AT_LEAST

RFC In review Romain Naour 2 participants 2 messages Jun 3, 2023 – Jun 4, 2023

lore
Share & tools
lore
#0 lore raw

RFC 1/1 support/dependencies: introduce BR2_HOST_CMAKE_AT_LEAST

libjxl requires cmake 3.19 since version v0.7 due to the new behavior of cmake [1].

-- Configuring done
CMake Error at cmake/FindLCMS2.cmake:40 (add_library):
INTERFACE_LIBRARY targets may only have whitelisted properties. The property "INCLUDE_DIRECTORIES" is not allowed.
Call Stack (most recent call first):
third_party/CMakeLists.txt:114 (find_package)

The portability issue has already been reported upstream [2].

host-cmake is know to take a long time to build and we want to avoid bumping BR2_CMAKE_VERSION_MIN only for one package. As discussed on the mailing list [3] introduce BR2_HOST_CMAKE_AT_LEAST_X_Y selected by package that requires a higher cmake version than the one provided by the host.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4322819095

[1] https://gitlab.kitware.com/cmake/cmake/-/commit/afb998704e67d3d3ce5b24c112cb06e770fca78d
[2] https://github.com/libjxl/libjxl/issues/1425
[3] http://lists.busybox.net/pipermail/buildroot/2023-June/668287.html

Cc Julien Olivain <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Yann E. MORIN <[email protected]>
Cc Thomas Petazzoni <[email protected]>
---
 package/cmake/Config.in.host             | 31 ++++++++++++++++++++++++
 package/cmake/cmake.mk                   |  1 +
 package/libjxl/Config.in                 |  1 +
 support/dependencies/check-host-cmake.mk | 12 +++++----
 4 files changed, 40 insertions(+), 5 deletions(-)
package/cmake/Config.in.host +31 -0
index b2e210ee2c..fe5640088e 100644
@@ -9,3 +9,34 @@ config BR2_PACKAGE_HOST_CMAKE
 	  the compiler environment of your choice.
 
 	  http://www.cmake.org/
+
+# The latest version in this list must match the one provided by host-cmake.
+config BR2_HOST_CMAKE_AT_LEAST_3_18 # Debian bullseye provide cmake 3.18.x
+	bool
+	default y
+
+config BR2_HOST_CMAKE_AT_LEAST_3_19
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_18
+
+config BR2_HOST_CMAKE_AT_LEAST_3_20
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_19
+
+config BR2_HOST_CMAKE_AT_LEAST_3_21
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_20
+
+config BR2_HOST_CMAKE_AT_LEAST_3_22
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_21
+
+# This order guarantees that the highest version is set, as kconfig
+# stops affecting a value on the first matching default.
+config BR2_HOST_CMAKE_AT_LEAST
+	string
+	default "3.22"	if BR2_HOST_CMAKE_AT_LEAST_3_22
+	default "3.21"	if BR2_HOST_CMAKE_AT_LEAST_3_21
+	default "3.20"	if BR2_HOST_CMAKE_AT_LEAST_3_20
+	default "3.19"	if BR2_HOST_CMAKE_AT_LEAST_3_19
+	default "3.18"	if BR2_HOST_CMAKE_AT_LEAST_3_18
package/cmake/cmake.mk +1 -0
index dc92c8bb4e..b99877696a 100644
@@ -4,6 +4,7 @@
 #
 ################################################################################
 
+# When updating the version, please also update BR2_HOST_CMAKE_AT_LEAST_X_Y
 CMAKE_VERSION_MAJOR = 3.22
 CMAKE_VERSION = $(CMAKE_VERSION_MAJOR).3
 CMAKE_SITE = https://cmake.org/files/v$(CMAKE_VERSION_MAJOR)
package/libjxl/Config.in +1 -0
index da04b89f9b..923560298f 100644
@@ -7,6 +7,7 @@ config BR2_PACKAGE_LIBJXL
 	# libjxl fail to link statically due to libatomic issue
 	depends on !BR2_STATIC_LIBS
 	depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_81426
+	select BR2_HOST_CMAKE_AT_LEAST_3_19
 	select BR2_PACKAGE_BROTLI
 	select BR2_PACKAGE_HIGHWAY
 	select BR2_PACKAGE_LCMS2
support/dependencies/check-host-cmake.mk +7 -5
index 5760e4ded1..e367690319 100644
@@ -1,9 +1,11 @@
-# Set this to either 3.18 or higher, depending on the highest minimum
-# version required by any of the packages bundled in Buildroot. If a
-# package is bumped or a new one added, and it requires a higher
-# version, our cmake infra will catch it and build its own.
+# The cmake minimum version is set to either 3.18 or higher,
+# depending on the highest minimum version required by any
+# of the packages bundled in Buildroot. If a package is
+# bumped or a new one added, and it requires a higher
+# cmake version than the one provided by the host, our
+# cmake infra will catch it and build its own.
 #
-BR2_CMAKE_VERSION_MIN = 3.18
+BR2_CMAKE_VERSION_MIN = $(BR2_HOST_CMAKE_AT_LEAST)
 
 BR2_CMAKE_CANDIDATES ?= cmake cmake3
 BR2_CMAKE ?= $(call suitable-host-package,cmake,\
Signature · 6 lines
2.34.3

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

Romain, All,

On 2023-06-04 00:16 +0200, Romain Naour spake thusly:

Quoted 2 lines

libjxl requires cmake 3.19 since version v0.7 due to the new behavior of cmake [1].

This needs to be split in two:
1. add the host-cmake versions
2. fix libjxl to select the required version

This needs a more detailed commit log, please. ;-)

For example:

Some packages (e.g. libjxl) requires a quite recent cmake version, that is not yet available in most distributions, especially those LTS versions.

Currently, when we bump the minum cmake version we require, it gets bumped for all packages, regardless of their own minimum required version, which means that a given configuration will trigger the build of our host-cmake even if the packages that require it are not enabled and those that are would be content with the system-provided cmake.

Since host-cmake can take quite some time to build, this can get a bit annoying to pay the price of a host-cmake build that would otherwise not be needed.

We intriduce config options that packages can select to indicate what minimal cmake version they require, and use that version as the required minimal version required by the current configuration [0].

We would like to ensure that the currently selected minimum cmake version is indeed lower (or equal) to the cmake version we package, but that is not possible: dependencies.mk is parsed before we parse packages, so we do not yet know the cmake version we have, and we can't invert the parsing order as we need to know the requires dependencies before we parse packages (so that we can build their dependency rules in Makefile). So we can only add comments in both places, that refer to the other location.

[0] note that this is yet not optimal, as in such a case, host-cmake would be in the dependency chain of all cmake-based packages, even for those packages that do not require it. The optimum would be for each package to gain such a dependency on an as-needed basis, but this is by far more complex to achieve, and would only speed up cases where a single package is built from scratch (e.g. with:
make clean; make foo), which is not worth optimising (yet?)

Quoted 8 lines

-- Configuring done
CMake Error at cmake/FindLCMS2.cmake:40 (add_library):
INTERFACE_LIBRARY targets may only have whitelisted properties. The property "INCLUDE_DIRECTORIES" is not allowed.
Call Stack (most recent call first):
third_party/CMakeLists.txt:114 (find_package)

The portability issue has already been reported upstream [2].

This part above would have to be moved into the commit that change libjxl to request host-cmake 3.22.

Quoted 32 lines

host-cmake is know to take a long time to build and we want to avoid bumping BR2_CMAKE_VERSION_MIN only for one package. As discussed on the mailing list [3] introduce BR2_HOST_CMAKE_AT_LEAST_X_Y selected by package that requires a higher cmake version than the one provided by the host.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4322819095

[1] https://gitlab.kitware.com/cmake/cmake/-/commit/afb998704e67d3d3ce5b24c112cb06e770fca78d
[2] https://github.com/libjxl/libjxl/issues/1425
[3] http://lists.busybox.net/pipermail/buildroot/2023-June/668287.html

Cc Julien Olivain <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Peter Korsgaard <[email protected]>
Cc Yann E. MORIN <[email protected]>
Cc Thomas Petazzoni <[email protected]>
---
 package/cmake/Config.in.host             | 31 ++++++++++++++++++++++++
 package/cmake/cmake.mk                   |  1 +
 package/libjxl/Config.in                 |  1 +
 support/dependencies/check-host-cmake.mk | 12 +++++----
 4 files changed, 40 insertions(+), 5 deletions(-)
package/cmake/Config.in.host +0 -0
index b2e210ee2c..fe5640088e 100644

package/cmake/Config.in.host config BR2_PACKAGE_HOST_CMAKE · line 16

Quoted 8 lines
package/cmake/Config.in.host +5 -0
@@ -9,3 +9,34 @@ config BR2_PACKAGE_HOST_CMAKE
 	  the compiler environment of your choice.
 
 	  http://www.cmake.org/
+
+# The latest version in this list must match the one provided by host-cmake.
+config BR2_HOST_CMAKE_AT_LEAST_3_18 # Debian bullseye provide cmake 3.18.x
+	bool
+	default y

There is no need for this symbol, if...

Quoted 15 lines
package/cmake/Config.in.host +15 -0
+config BR2_HOST_CMAKE_AT_LEAST_3_19
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_18
+
+config BR2_HOST_CMAKE_AT_LEAST_3_20
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_19
+
+config BR2_HOST_CMAKE_AT_LEAST_3_21
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_20
+
+config BR2_HOST_CMAKE_AT_LEAST_3_22
+	bool
+	select BR2_HOST_CMAKE_AT_LEAST_3_21

package/cmake/Config.in.host config BR2_PACKAGE_HOST_CMAKE · line 42

Quoted 10 lines
package/cmake/Config.in.host +10 -0
+
+# This order guarantees that the highest version is set, as kconfig
+# stops affecting a value on the first matching default.
+config BR2_HOST_CMAKE_AT_LEAST
+	string
+	default "3.22"	if BR2_HOST_CMAKE_AT_LEAST_3_22
+	default "3.21"	if BR2_HOST_CMAKE_AT_LEAST_3_21
+	default "3.20"	if BR2_HOST_CMAKE_AT_LEAST_3_20
+	default "3.19"	if BR2_HOST_CMAKE_AT_LEAST_3_19
+	default "3.18"	if BR2_HOST_CMAKE_AT_LEAST_3_18

... you remove the conditional here, e.g. just:

    default "3.18"

And I prefer that there is always a non-conditional default.

In which case the first comment would change to:

# The minimum system cmake version we expect if 3.18 as provided by # Debian bullseye, that we use in our reference build docker image.

Quoted 17 lines
package/cmake/cmake.mk +1 -0
index dc92c8bb4e..b99877696a 100644
@@ -4,6 +4,7 @@
 #
 ################################################################################
 
+# When updating the version, please also update BR2_HOST_CMAKE_AT_LEAST_X_Y
 CMAKE_VERSION_MAJOR = 3.22
 CMAKE_VERSION = $(CMAKE_VERSION_MAJOR).3
 CMAKE_SITE = https://cmake.org/files/v$(CMAKE_VERSION_MAJOR)
package/libjxl/Config.in +0 -0
index da04b89f9b..923560298f 100644

package/libjxl/Config.in config BR2_PACKAGE_LIBJXL · line 10

Quoted 4 lines
package/libjxl/Config.in +1 -0
@@ -7,6 +7,7 @@ config BR2_PACKAGE_LIBJXL
 	# libjxl fail to link statically due to libatomic issue
 	depends on !BR2_STATIC_LIBS
 	depends on !BR2_TOOLCHAIN_HAS_GCC_BUG_81426
+	select BR2_HOST_CMAKE_AT_LEAST_3_19

As suggested above, this needs to be in a separate patch.

Otherwise, this is a very good idea, much simpler than what I was thinking about (i.e. having each package depend on their required version).

Regards,
Yann E. MORIN.

Quoted 27 lines
package/libjxl/Config.in +0 -0
 	select BR2_PACKAGE_BROTLI
 	select BR2_PACKAGE_HIGHWAY
 	select BR2_PACKAGE_LCMS2
support/dependencies/check-host-cmake.mk +7 -5
index 5760e4ded1..e367690319 100644
@@ -1,9 +1,11 @@
-# Set this to either 3.18 or higher, depending on the highest minimum
-# version required by any of the packages bundled in Buildroot. If a
-# package is bumped or a new one added, and it requires a higher
-# version, our cmake infra will catch it and build its own.
+# The cmake minimum version is set to either 3.18 or higher,
+# depending on the highest minimum version required by any
+# of the packages bundled in Buildroot. If a package is
+# bumped or a new one added, and it requires a higher
+# cmake version than the one provided by the host, our
+# cmake infra will catch it and build its own.
 #
-BR2_CMAKE_VERSION_MIN = 3.18
+BR2_CMAKE_VERSION_MIN = $(BR2_HOST_CMAKE_AT_LEAST)
 
 BR2_CMAKE_CANDIDATES ?= cmake cmake3
 BR2_CMAKE ?= $(call suitable-host-package,cmake,\
Signature · 1 line
2.34.3
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot