cmake.bbclass improvements
v4 cover cmake.bbclass improvements
This patchset is unmodified from v3. It is just rebased on top of master which already includes a fixed version of libproxy and piglit.
I built all cmake based recipes in oe-core (qemuarm64) and did not find any more issues.
Pascal Bach (4):
cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of
CMAKE_LIBRARY_PATH
cmake.bbclass: search both sysroot-native and host for native packages cmake.bbclass: move CMAKE_NO_SYSTEM_FROM_IMPORTED to toolchain.cmake cmake.bbclass: allow cmake to find hosttools
meta/classes/cmake.bbclass | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)
Signature · 1 line
2.11.0
PATCH v4 1/4 cmake.bbclass: use CMAKE_SYSTEM_LIBRARY_PATH instead of CMAKE_LIBRARY_PATH
CMAKE_LIBRARY_PATH is intended to be set by projects. CMAKE_SYSTEM_LIBRARY_PATH is better suited to be used in a toolchain file.
--- meta/classes/cmake.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/cmake.bbclass +1 -1
@@ -106,7 +106,7 @@ set( CMAKE_INSTALL_RPATH ${OECMAKE_RPATH} ) list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/") # add for non /usr/lib libdir, e.g. /usr/lib64 -set( CMAKE_LIBRARY_PATH ${libdir} ${base_libdir}) +set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir}) EOF }
Signature · 1 line
2.11.0
PATCH v4 2/4 cmake.bbclass: search both sysroot-native and host for native packages
In reply to Pascal BachCertain headers and libraries like math.h an -m are only available on the host as they are provided by the host toolchain.
This leads to issues that a find_library in CMake doesn't find the m library of a find_path doesn't find math.h. This issue occurred in the wireshark recipe for example.
This change enables CMake to also look on the host for libraries and includes when building a native package.
--- meta/classes/cmake.bbclass | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
meta/classes/cmake.bbclass +6 -6
@@ -43,8 +43,8 @@ OECMAKE_RPATH ?= "" OECMAKE_PERLNATIVE_DIR ??= "" OECMAKE_EXTRA_ROOT_PATH ?= "" -OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM = "ONLY" -OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM_class-native = "BOTH" +OECMAKE_FIND_ROOT_PATH_MODE = "ONLY" +OECMAKE_FIND_ROOT_PATH_MODE_class-native = "BOTH" EXTRA_OECMAKE_append = " ${PACKAGECONFIG_CONFARGS}"
@@ -90,10 +90,10 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" ) # only search in the paths provided so cmake doesnt pick # up libraries and tools from the native build machine set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN}) -set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY ) -set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM} ) -set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY ) -set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY ) +set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ${OECMAKE_FIND_ROOT_PATH_MODE} ) +set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE} ) +set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ${OECMAKE_FIND_ROOT_PATH_MODE} ) +set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ${OECMAKE_FIND_ROOT_PATH_MODE} ) # Use qt.conf settings set( ENV{QT_CONF_PATH} ${WORKDIR}/qt.conf )
Signature · 1 line
2.11.0
Re: [PATCH v4 2/4] cmake.bbclass: search both sysroot-native and host for native packages
I think I figured out why we're seeing odd failures on the autobuilder, such as
https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/125.
The important bit of the log is:
File "/home/pokybuild/yocto-worker/poky-tiny/build/build/tmp/work/qemux86-poky-linux-musl/core-image-minimal/1.0-r0/recipe-sysroot-native/usr/lib/python3.5/site-packages/libcomps/__init__.py", line 1, in <module>
from ._libpycomps import *
ImportError: libpython3.6m.so.1.0: cannot open shared object file: No
such file or directory
The smoking gun is that oe-core doesn't have Python 3.6, but 3.5.
My suspicion is that for systems where the host has python3-devel installed, libcomp-native's cmake is using the host python instead of the native python in the sysroot. I'm not sure yet if BOTH means it searches the host before the sysroots, or if it is finding the python binary in HOSTTOOLS and then asking that what the paths are. Either way, it ends up linking to the host libpython3.5.so and going into sstate. Another machine in the pool reuses this sstate but it doesn't have Python 3.6 installed, so the library link is broken.
Ross
Re: [PATCH v4 2/4] cmake.bbclass: search both sysroot-native and host for native packages
Quoted 14 lines · Ross
I think I figured out why we're seeing odd failures on the autobuilder, such as
https://autobuilder.yoctoproject.org/typhoon/#/builders/15/builds/125.The important bit of the log is:
File "
/home/pokybuild/yocto-worker/poky-tiny/build/build/tmp/work/qemux86-poky-linux-musl/core-image-minimal/1.0-r0/recipe-sysroot-native/usr/lib/python3.5/site-packages/libcomps/__init__.py", line 1, in <module>
from ._libpycomps import *
ImportError: libpython3.6m.so.1.0: cannot open shared object file: No
such file or directoryThe smoking gun is that oe-core doesn't have Python 3.6, but 3.5.
Quoted 8 lines
My suspicion is that for systems where the host has python3-devel installed, libcomp-native's cmake is using the host python instead of the native python in the sysroot. I'm not sure yet if BOTH means it searches the host before the sysroots, or if it is finding the python binary in HOSTTOOLS and then asking that what the paths are. Either way, it ends up linking to the host libpython3.5.so and going into sstate. Another machine in the pool reuses this sstate but it doesn't have Python 3.6 installed, so the library link is broken.
That is definitively not the intention here.
For target builds it is pretty clear that there should be no usage of host dependencies.
For native it is not so clear to me. There seems to be some dependencies like the compiler and associated libs.
Maybe there is a way to bring this in a different way that is more selective instead of a general fallback to host as it would be with this patch?
What I'm traying to solve here was to get rid of these two lines in the wireshark recipe [1]:
-DM_INCLUDE_DIR=${includedir} \
-DM_LIBRARY=${libdir} \
As they effectively add /usr/include and /usr/lib to the search path of cmake.
Any toughts?
Pascal
Re: [PATCH v4 2/4] cmake.bbclass: search both sysroot-native and host for native packages
Quoted 8 lines · Pascal Bach
Quoted 8 lines
My suspicion is that for systems where the host has python3-devel installed, libcomp-native's cmake is using the host python instead of the native python in the sysroot. I'm not sure yet if BOTH means it searches the host before the sysroots, or if it is finding the python binary in HOSTTOOLS and then asking that what the paths are. Either way, it ends up linking to the host libpython3.5.so and going into sstate. Another machine in the pool reuses this sstate but it doesn't have Python 3.6 installed, so the library link is broken.
Quoted 4 lines
That is definitively not the intention here.
For target builds it is pretty clear that there should be no usage of host dependencies.
For native it is not so clear to me. There seems to be some dependencies like the compiler and associated libs.
I think ensuring the ordering for native searches is sysroot then host would be sufficient.
Quoted 6 lines
What I'm traying to solve here was to get rid of these two lines in the wireshark recipe [1]:
(quoted diff) +0
-2-DM_INCLUDE_DIR=${includedir} \ -DM_LIBRARY=${libdir} \As they effectively add
/usr/includeand/usr/libto the search path of cmake.
Well that's terrifying. What is that trying to solve? It's building for the target, right? So why is it passing host paths?
Ross
PATCH v4 3/4 cmake.bbclass: move CMAKE_NO_SYSTEM_FROM_IMPORTED to toolchain.cmake
In reply to Pascal BachThe setting influences the build like other settings already in toolchain.cmake. It is more appropriate to set it there instead of providing it as a random command line parameter to CMake.
It also makes it easier to use the toolchain.cmake file independent of bitbake. Like the devshell for example.
--- meta/classes/cmake.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes/cmake.bbclass +3 -1
@@ -108,6 +108,9 @@ list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/") # add for non /usr/lib libdir, e.g. /usr/lib64 set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir}) +# avoid treating imports as system includes +set( CMAKE_NO_SYSTEM_FROM_IMPORTED ON) + EOF }
@@ -152,7 +155,6 @@ cmake_do_configure() { -DCMAKE_INSTALL_SO_NO_EXE=0 \ -DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \ -DCMAKE_VERBOSE_MAKEFILE=1 \ - -DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \ ${EXTRA_OECMAKE} \ -Wno-dev }
Signature · 1 line
2.11.0
PATCH v4 4/4 cmake.bbclass: allow cmake to find hosttools
In reply to Pascal BachCurrently the generated toolchain file is unable to find hosttools as they do not appear in the search paths.
Just adding HOSTTOOLS_DIR is not enough as binaries are located directly under ${HOSTTOOLS_DIR}. Like ${HOSTTOOLS_DIR}/git for example.
CMake however only searches in [s]bin sub directories of the paths specified in CMAKE_FIND_ROOT_PATH. Explicitly adding / to CMAKE_SYSTEM_PROGRAM_PATH makes CMake look in the right location.
--- meta/classes/cmake.bbclass | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
meta/classes/cmake.bbclass +5 -1
@@ -89,7 +89,7 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" ) # only search in the paths provided so cmake doesnt pick # up libraries and tools from the native build machine -set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN}) +set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR}) set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ${OECMAKE_FIND_ROOT_PATH_MODE} ) set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE} ) set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ${OECMAKE_FIND_ROOT_PATH_MODE} )
@@ -108,6 +108,10 @@ list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/") # add for non /usr/lib libdir, e.g. /usr/lib64 set( CMAKE_SYSTEM_LIBRARY_PATH ${libdir} ${base_libdir}) +# by default CMake only looks in [s]bin subdirectories of CMAKE_FIND_ROOT_PATH +# adding / makes CMake look for binaries in hosttools too. +set( CMAKE_SYSTEM_PROGRAM_PATH /) + # avoid treating imports as system includes set( CMAKE_NO_SYSTEM_FROM_IMPORTED ON)
Signature · 1 line
2.11.0
Quoted 6 lines · Pascal Bach
This patchset is unmodified from v3. It is just rebased on top of master which already includes a fixed version of libproxy and piglit.
I built all cmake based recipes in oe-core (qemuarm64) and did not find any more issues.
Can you run world builds on meta-openembedded repositories with this series applied and also compile some other large projects which use cmake like meta-browser
Quoted 18 lines
Pascal Bach (4):
cmake.bbclass: useCMAKE_SYSTEM_LIBRARY_PATHinstead of
CMAKE_LIBRARY_PATH
cmake.bbclass: search both sysroot-native and host for native packages cmake.bbclass: moveCMAKE_NO_SYSTEM_FROM_IMPORTEDto toolchain.cmake cmake.bbclass: allow cmake to find hosttoolsmeta/classes/cmake.bbclass | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)--
2.11.0--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 9 lines · Khem Raj
Quoted 5 lines · Pascal Bach
This patchset is unmodified from v3. It is just rebased on top of master which already includes a fixed version of libproxy and piglit.
I built all cmake based recipes in oe-core (qemuarm64) and did not find any more issues.
Can you run world builds on meta-openembedded repositories with this series applied and also compile some other large projects which use cmake like meta-browser
I can try but usually I get into trouble with the proxy because there are always some recipes that fetch from some location via some protocol that doesn't work.
Quoted 19 lines
Quoted 19 lines
Pascal Bach (4):
cmake.bbclass: useCMAKE_SYSTEM_LIBRARY_PATHinstead of
CMAKE_LIBRARY_PATH
cmake.bbclass: search both sysroot-native and host for native packages cmake.bbclass: moveCMAKE_NO_SYSTEM_FROM_IMPORTEDto toolchain.cmake cmake.bbclass: allow cmake to find hosttoolsmeta/classes/cmake.bbclass | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)--
2.11.0--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 7 lines · Pascal Bach
Quoted 4 lines · Khem Raj
Quoted 3 lines · Pascal Bach
This patchset is unmodified from v3. It is just rebased on top of master which already includes a fixed version of libproxy and piglit.
Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
I built all cmake based recipes in oe-core (qemuarm64) and did not find any more issues.
Can you run world builds on meta-openembedded repositories with this series applied and also compile some other large projects which use cmake like meta-browser
I can try but usually I get into trouble with the proxy because there are always some recipes that fetch from some location via some protocol that doesn't work.
Once the current cycle of builds is done on OE builders I will try to schedule this in OE builds and see where it ends up. hopefully by this weekend.
Quoted 21 lines
Quoted 20 lines
Quoted 20 lines
Pascal Bach (4):
cmake.bbclass: useCMAKE_SYSTEM_LIBRARY_PATHinstead of
CMAKE_LIBRARY_PATH
cmake.bbclass: search both sysroot-native and host for native packages cmake.bbclass: moveCMAKE_NO_SYSTEM_FROM_IMPORTEDto toolchain.cmake cmake.bbclass: allow cmake to find hosttoolsmeta/classes/cmake.bbclass | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)--
2.11.0--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 9 lines · Khem Raj
Quoted 8 lines · Pascal Bach
Quoted 6 lines · Khem Raj
Quoted 5 lines · Pascal Bach
This patchset is unmodified from v3. It is just rebased on top of master which already includes a fixed version of libproxy and piglit.
I built all cmake based recipes in oe-core (qemuarm64) and did not find any more issues.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
Can you run world builds on meta-openembedded repositories with this series applied and also compile some other large projects which use cmake like meta-browser
I can try but usually I get into trouble with the proxy because there are always some recipes that fetch from some location via some protocol that doesn't work.
Once the current cycle of builds is done on OE builders I will try to schedule this in OE builds and see where it ends up. hopefully by this weekend.
I was able to do a world build with all layers of meta-oe and there are a few failures:
Two of wich I think are related to cmake:
- libyui-ncurses => messing around with hardcoded path in their cmake setup
- civetweb => still trying to figure out what exactly goes wrong
The following I was unable to build due to fetch problems:
- oscam
I will try to send patches for these two recipes.
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Pascal Bach (4):
cmake.bbclass: useCMAKE_SYSTEM_LIBRARY_PATHinstead of
CMAKE_LIBRARY_PATH
cmake.bbclass: search both sysroot-native and host for native packages cmake.bbclass: moveCMAKE_NO_SYSTEM_FROM_IMPORTEDto toolchain.cmake cmake.bbclass: allow cmake to find hosttoolsmeta/classes/cmake.bbclass | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)--
2.11.0--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
I would like to understand why this patch series was never merged. Specifically, the patch that moves CMAKE_NO_SYSTEM_FROM_IMPORTED to toolchain.cmake seems to fix a common issue when building with OE: By default, CMake adds the SYSTEM property to all imported libraries, which lets consuming targets use stricter compiler warnings and treat warnings as errors without triggering errors from dependent targets' headers. Setting CMAKE_NO_SYSTEM_FROM_IMPORTED disables this default behaviour, oddly forcing consuming targets to fix issues that were created by the build system itself as CMAKE_NO_SYSTEM_FROM_IMPORTED applies uniformly to all imported targets the consumer links to. I've successfully built core-image-sato with CMAKE_NO_SYSTEM_FROM_IMPORTED=1 completely removed on current master. Looking at the commit history, this setting seems to have been added to address an issue that may no longer exist. Is there a rationale for keeping this setting?