cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

PATCH Superseded Pascal Bach 3 participants 8 messages Oct 10, 2018

Share & tools

Pascal Bach

author

2018-10-10 08:11 UTC #0 lore raw

PATCH v3 1/4 cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH

CMAKE_LIBRARY_PATH is intended to be set by projects. CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain file.

Signed-off-by Pascal Bach <[email protected]>

---
 meta/classes/cmake.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

meta/classes/cmake.bbclass +1 -1

index fd40a9863e..86b1d0e9aa 100644
@@ -106,7 +106,7 @@ set( CMAKE_INSTALL_RPATH ${OECMAKE_RPATH} )
 list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
 
 # add for non /usr/lib libdir, e.g. /usr/lib64
-set( CMAKE_LIBRARY_PATH ${libdir} ${base_libdir})
+set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir})
 
 EOF
 }

Signature · 1 line

2.11.0

Pascal Bach

author

2018-10-10 08:12 UTC #1 lore raw

PATCH v3 2/4 cmake.bbclass: search both sysroot-native and host for native packages

Certain headers and libraries like math.h an -m are only available on the host as they are provided by the host toolchain.

This leads to issues that a find_library in CMake doesn't find the m library of a find_path doesn't find math.h. This issue occurred in the wireshark recipe for example.

This change enables CMake to also look on the host for libraries and includes when building a native package.

Signed-off-by Pascal Bach <[email protected]>

---
 meta/classes/cmake.bbclass | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

meta/classes/cmake.bbclass +6 -6

index 86b1d0e9aa..ce3c0278ff 100644
@@ -43,8 +43,8 @@ OECMAKE_RPATH ?= ""
 OECMAKE_PERLNATIVE_DIR ??= ""
 OECMAKE_EXTRA_ROOT_PATH ?= ""
 
-OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM = "ONLY"
-OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM_class-native = "BOTH"
+OECMAKE_FIND_ROOT_PATH_MODE = "ONLY"
+OECMAKE_FIND_ROOT_PATH_MODE_class-native = "BOTH"
 
 EXTRA_OECMAKE_append = " ${PACKAGECONFIG_CONFARGS}"

@@ -90,10 +90,10 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" )
 # only search in the paths provided so cmake doesnt pick
 # up libraries and tools from the native build machine
 set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN})
-set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY )
-set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM} )
-set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY )
-set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY )
+set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ${OECMAKE_FIND_ROOT_PATH_MODE} )
+set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE} )
+set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ${OECMAKE_FIND_ROOT_PATH_MODE} )
+set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ${OECMAKE_FIND_ROOT_PATH_MODE} )
 
 # Use qt.conf settings
 set( ENV{QT_CONF_PATH} ${WORKDIR}/qt.conf )

Signature · 1 line

2.11.0

Pascal Bach

author

2018-10-10 08:12 UTC #2 lore raw

PATCH v3 3/4 cmake.bbclass: move CMAKE_NO_SYSTEM_FROM_IMPORTED to toolchain.cmake

In reply to Pascal Bach

The setting influences the build like other settings already in toolchain.cmake. It is more appropriate to set it there instead of providing it as a random command line parameter to CMake.

It also makes it easier to use the toolchain.cmake file independent of bitbake. Like the devshell for example.

Signed-off-by Pascal Bach <[email protected]>

---
 meta/classes/cmake.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

meta/classes/cmake.bbclass +3 -1

index ce3c0278ff..0ef63795eb 100644
@@ -108,6 +108,9 @@ list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
 # add for non /usr/lib libdir, e.g. /usr/lib64
 set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir})
 
+# avoid treating imports as system includes
+set( CMAKE_NO_SYSTEM_FROM_IMPORTED ON)
+
 EOF
 }

@@ -152,7 +155,6 @@ cmake_do_configure() {
 	  -DCMAKE_INSTALL_SO_NO_EXE=0 \
 	  -DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
 	  -DCMAKE_VERBOSE_MAKEFILE=1 \
-	  -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \
 	  ${EXTRA_OECMAKE} \
 	  -Wno-dev
 }

Signature · 1 line

2.11.0

Pascal Bach

author

2018-10-10 08:12 UTC #3 lore raw

PATCH v3 4/4 cmake.bbclass: allow cmake to find hosttools

In reply to Pascal Bach

Currently the generated toolchain file is unable to find hosttools as they do not appear in the search paths.

Just adding HOSTTOOLS_DIR is not enough as binaries are located directly under ${HOSTTOOLS_DIR}. Like ${HOSTTOOLS_DIR}/git for example.
CMake however only searches in [s]bin sub directories of the paths specified in CMAKE_FIND_ROOT_PATH. Explicitly adding / to CMAKE_SYSTEM_PROGRAM_PATH makes CMake look in the right location.

Signed-off-by Pascal Bach <[email protected]>

---
 meta/classes/cmake.bbclass | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

meta/classes/cmake.bbclass +5 -1

index 0ef63795eb..c8a079c417 100644
@@ -89,7 +89,7 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" )
 
 # only search in the paths provided so cmake doesnt pick
 # up libraries and tools from the native build machine
-set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN})
+set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR})
 set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ${OECMAKE_FIND_ROOT_PATH_MODE} )
 set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE} )
 set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ${OECMAKE_FIND_ROOT_PATH_MODE} )

@@ -108,6 +108,10 @@ list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
 # add for non /usr/lib libdir, e.g. /usr/lib64
 set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir})
 
+# by default CMake only looks in [s]bin subdirectories of CMAKE_FIND_ROOT_PATH
+# adding / makes CMake look for binaries in hosttools too.
+set( CMAKE_SYSTEM_PROGRAM_PATH /)
+
 # avoid treating imports as system includes
 set( CMAKE_NO_SYSTEM_FROM_IMPORTED ON)

Signature · 1 line

2.11.0

Richard Purdie

2018-10-10 12:36 UTC #4 lore raw

In reply to Pascal Bach

Quoted 8 lines · Pascal Bach

CMAKE_LIBRARY_PATH is intended to be set by projects. CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain file.
Signed-off-by Pascal Bach <[email protected]>
---
 meta/classes/cmake.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

I have a feeling something in this series may have caused:

https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/42753/raw

but haven't bisected to confirm it is the cmake changes yet (it is from something in master-next).

Cheers,

Richard

Pascal Bach

author

2018-10-10 13:32 UTC #5 lore raw

Quoted 9 lines · Richard Purdie

Quoted 8 lines · Pascal Bach
CMAKE_LIBRARY_PATH is intended to be set by projects. CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain file.
Signed-off-by Pascal Bach <[email protected]>
---
 meta/classes/cmake.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Quoted 6 lines

I have a feeling something in this series may have caused:
https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/42753/raw
but haven't bisected to confirm it is the cmake changes yet (it is from something in master-next).

You are right the recipe is using cmake. I will investigate the issue.

Pascal Bach

author

2018-10-10 14:10 UTC #6 lore raw

In reply to Richard Purdie

Quoted 9 lines · Richard Purdie

Quoted 8 lines · Pascal Bach
CMAKE_LIBRARY_PATH is intended to be set by projects. CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain file.
Signed-off-by Pascal Bach <[email protected]>
---
 meta/classes/cmake.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Quoted 7 lines

I have a feeling something in this series may have caused:
https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/42753/raw
but haven't bisected to confirm it is the cmake changes yet (it is from something in master-next).

The problem is "cmake.bbclass: allow cmake to find hosttools" it has the side effect of making all hosttools available to all recipes. In the case of libproxy this leads to it finding python, which is included in hosttools, and thus building it's bindings.

This kind of defeats the purpose of having recipe specific sysroots.
Is there a way to make only selected hosttools available to a recipe. Like for example git? Maybe use git-native?

I think the patch "cmake.bbclass: allow cmake to find hosttools" in this form does more harm then good. The rest of the series should be fine. Should I resubmit the series without this specific patch?

One more thing for libproxy. It currently disables python explicitly via -WITH_PYTHON=no. But this option doesn't exist anymore as it got replaces by -WITH_PYTHON2=no -WITH_PYTHON3=no. This is why the auto detection kicks in. I will submit a patch to clean this recipe up too.

Pascal

Burton, Ross

2018-10-10 14:16 UTC #7 lore raw

Quoted 9 lines · Pascal Bach

The problem is "cmake.bbclass: allow cmake to find hosttools" it has the side effect of making all hosttools available to all recipes. In the case of libproxy this leads to it finding python, which is included in hosttools, and thus building it's bindings.
This kind of defeats the purpose of having recipe specific sysroots. Is there a way to make only selected hosttools available to a recipe. Like for example git? Maybe use git-native?

Quoted 8 lines

I think the patch "cmake.bbclass: allow cmake to find hosttools" in this form does more harm then good. The rest of the series should be fine. Should I resubmit the series without this specific patch?
One more thing for libproxy. It currently disables python explicitly via -WITH_PYTHON=no. But this option doesn't exist anymore as it got replaces by -WITH_PYTHON2=no -WITH_PYTHON3=no. This is why the auto detection kicks in. I will submit a patch to clean this recipe up too.

Per-recipe hosttools is an interesting idea but I'd say it's quite likely too invasive. This change has actually fixed cmake, and exposed the problem that the recipe is broken and should be using WITH_PYTHON2/WITH_PYTHON3: fixing that is the correct fix.

Ross