Skip to contentSkip to conversation

CMake packages improvements

PATCH New Luca Ceresoli 6 participants 31 messages Mar 6, 2013 – Mar 29, 2013 Kbuild

lore
Share & tools
lore
#0 lore raw

cover CMake packages improvements

Hi,

Here are a few improvements to the current handling of CMake packages in Buildroot.

The first three patch cleanup the CMake host package and bump to the latest version.
Patch 4 is the most interesting, as it allows CMake packages to benefit from ccache to dramatically reduce build times. This was impossile so far because CMake would fail detecting the compiler.
Patch 5 removes .cmake files from the target. This is similar to what BR alreay does with pkg-config's .pc files.

Luca

Luca Ceresoli (5):
cmake: convert spaces to tabs in make rules
cmake: remove target package macro (not really implemented)
cmake: bump to 2.8.10.2
Enable ccache for cmake packages
CMake packages: remove .cmake files from target directory

 Makefile               |    6 ++++--
 package/cmake/cmake.mk |   17 ++++++++---------
 package/pkg-cmake.mk   |   31 +++++++++++++++++++++++++++++--
 3 files changed, 41 insertions(+), 13 deletions(-)
Signature · 1 line
1.7.10.4
#1 lore raw

PATCH 1/5 cmake: convert spaces to tabs in make rules

---
 package/cmake/cmake.mk |   14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
package/cmake/cmake.mk +7 -7
index 6836172..d0db502e 100644
@@ -10,19 +10,19 @@ CMAKE_LICENSE = BSD-3c
 CMAKE_LICENSE_FILES = Copyright.txt
 
 define HOST_CMAKE_CONFIGURE_CMDS
- (cd $(@D); \
-	LDFLAGS="$(HOST_LDFLAGS)" \
-	CFLAGS="$(HOST_CFLAGS)" \
-	./bootstrap --prefix=$(HOST_DIR)/usr --parallel=$(PARALLEL_JOBS) \
- )
+	(cd $(@D); \
+		LDFLAGS="$(HOST_LDFLAGS)" \
+		CFLAGS="$(HOST_CFLAGS)" \
+		./bootstrap --prefix=$(HOST_DIR)/usr --parallel=$(PARALLEL_JOBS) \
+	)
 endef
 
 define HOST_CMAKE_BUILD_CMDS
- $(HOST_MAKE_ENV) $(MAKE) -C $(@D)
+	$(HOST_MAKE_ENV) $(MAKE) -C $(@D)
 endef
 
 define HOST_CMAKE_INSTALL_CMDS
- $(HOST_MAKE_ENV) $(MAKE) -C $(@D) install
+	$(HOST_MAKE_ENV) $(MAKE) -C $(@D) install
 endef
 
 $(eval $(generic-package))
Signature · 1 line
1.7.10.4
#4 lore raw

PATCH 2/5 cmake: remove target package macro (not really implemented)

In reply to Luca Ceresoli

CMake is declared as a generic target package, but the corresponding rules do not even exist.

---
 package/cmake/cmake.mk |    1 -
 1 file changed, 1 deletion(-)
package/cmake/cmake.mk +0 -1
index d0db502e..cfe8ab4 100644
@@ -25,5 +25,4 @@ define HOST_CMAKE_INSTALL_CMDS
 	$(HOST_MAKE_ENV) $(MAKE) -C $(@D) install
 endef
 
-$(eval $(generic-package))
 $(eval $(host-generic-package))
Signature · 1 line
1.7.10.4
#5 lore raw

PATCH 2/5 cmake: remove target package macro (not really implemented)

Dear Luca Ceresoli,

Quoted 2 lines · Luca Ceresoli

CMake is declared as a generic target package, but the corresponding rules do not even exist.

This was needed some time ago: the package infrastructures had some issues when there was a host-<foo> package but no <foo> package. I think it was fixed by 79bfcd556027901b847cae063cbcfbedf75e394a.

Quoted 1 line
Signature · 4 lines
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#6 lore raw

PATCH v2 2/5 cmake: remove target package macro

The dummy target package is not needed anymore since commit 79bfcd556027901.

---

Changed in v2: reworded according to Thomas' comment.

 package/cmake/cmake.mk |    1 -
 1 file changed, 1 deletion(-)
package/cmake/cmake.mk +0 -1
index d0db502e..cfe8ab4 100644
@@ -25,5 +25,4 @@ define HOST_CMAKE_INSTALL_CMDS
 	$(HOST_MAKE_ENV) $(MAKE) -C $(@D) install
 endef
 
-$(eval $(generic-package))
 $(eval $(host-generic-package))
Signature · 1 line
1.7.10.4
#7 lore raw

PATCH v2 2/5 cmake: remove target package macro

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Luca" == Luca Ceresoli <[email protected]> writes:

Luca> The dummy target package is not needed anymore since commit Luca> 79bfcd556027901.
Luca> Signed-off-by: Luca Ceresoli <[email protected]> Luca> Acked-by: Thomas Petazzoni <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#8 lore raw

PATCH 3/5 cmake: bump to 2.8.10.2

In reply to Luca Ceresoli
---
 package/cmake/cmake.mk |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
package/cmake/cmake.mk +1 -1
index cfe8ab4..8e3e751 100644
@@ -4,7 +4,7 @@
 #
 #############################################################
 
-CMAKE_VERSION = 2.8.9
+CMAKE_VERSION = 2.8.10.2
 CMAKE_SITE = http://www.cmake.org/files/v2.8/
 CMAKE_LICENSE = BSD-3c
 CMAKE_LICENSE_FILES = Copyright.txt
Signature · 1 line
1.7.10.4
#9 lore raw

PATCH 3/5 cmake: bump to 2.8.10.2

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Luca" == Luca Ceresoli <[email protected]> writes:

Luca> Signed-off-by: Luca Ceresoli <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#10 lore raw

PATCH 4/5 Enable ccache for cmake packages

In reply to Luca Ceresoli

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

---
 Makefile             |    2 +-
 package/pkg-cmake.mk |   31 +++++++++++++++++++++++++++++--
 2 files changed, 30 insertions(+), 3 deletions(-)
Makefile +1 -1
index 1300725..f6d8494 100644
@@ -387,7 +387,7 @@ $(TARGETS_ALL): __real_tgt_%: $(BASE_TARGETS) %
 dirs: $(TOOLCHAIN_DIR) $(BUILD_DIR) $(STAGING_DIR) $(TARGET_DIR) \
 	$(HOST_DIR) $(BINARIES_DIR) $(STAMP_DIR)
 
-$(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
+$(BASE_TARGETS): dirs $(CMAKE_PKG_DEPENDENCIES)
 
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
package/pkg-cmake.mk +29 -2
index 626a0b0..0a44cfe 100644
@@ -183,6 +183,26 @@ cmake-package = $(call inner-cmake-package,$(call pkgname),$(call UPPERCASE,$(ca
 host-cmake-package = $(call inner-cmake-package,host-$(call pkgname),$(call UPPERCASE,host-$(call pkgname)),$(call UPPERCASE,$(call pkgname)),$(call pkgparentdir),host)
 
 ################################################################################
+# Generate wrappers for the compiler.
+# If CMAKE_C_COMPILER were set to $(TARGET_CC), which may contanin a ccache
+# prefix, CMake would fail to detect the compiler. Generate two simple wrappers
+# that make it believe it is calling the compiler directly.
+################################################################################
+
+CMAKE_CC_WRAPPER  = $(HOST_DIR)/usr/bin/cmake-cc-wrapper
+CMAKE_CXX_WRAPPER = $(HOST_DIR)/usr/bin/cmake-cxx-wrapper
+
+$(CMAKE_CC_WRAPPER):
+	echo "#!/bin/sh" >$@
+	echo "$(TARGET_CC)" '$$@' >>$@
+	chmod a+x $@
+
+$(CMAKE_CXX_WRAPPER):
+	echo "#!/bin/sh" >$@
+	echo "$(TARGET_CXX)" '$$@' >>$@
+	chmod a+x $@
+
+################################################################################
 # Generation of the CMake toolchain file
 ################################################################################
 
@@ -190,8 +210,8 @@ $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:
 	@mkdir -p $(@D)
 	@echo -en "\
 	set(CMAKE_SYSTEM_NAME Linux)\n\
-	set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
-	set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+	set(CMAKE_C_COMPILER $(CMAKE_CC_WRAPPER))\n\
+	set(CMAKE_CXX_COMPILER $(CMAKE_CXX_WRAPPER))\n\
 	set(CMAKE_C_FLAGS \"\$${CMAKE_C_FLAGS} $(TARGET_CFLAGS)\" CACHE STRING \"Buildroot CFLAGS\" FORCE)\n\
 	set(CMAKE_CXX_FLAGS \"\$${CMAKE_CXX_FLAGS} $(TARGET_CXXFLAGS)\" CACHE STRING \"Buildroot CXXFLAGS\" FORCE)\n\
 	set(CMAKE_INSTALL_SO_NO_EXE 0)\n\
@@ -203,3 +223,10 @@ $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:
 	set(ENV{PKG_CONFIG_SYSROOT_DIR} \"$(STAGING_DIR)\")\n\
 	" > $@
 
+################################################################################
+# dependencies for the top-level Makefile
+################################################################################
+
+CMAKE_PKG_DEPENDENCIES = \
+	$(CMAKE_CC_WRAPPER) $(CMAKE_CXX_WRAPPER) \
+	$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
Signature · 1 line
1.7.10.4
#11 lore raw

PATCH 4/5 Enable ccache for cmake packages

Dear Luca Ceresoli,

Quoted 2 lines · Luca Ceresoli

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

This raises the question of whether we want to generalize this in some
way: should we be doing the ccache wrapper thing in a generic way, and use it everywhere? Since we already have a wrapper for external toolchain tools, does it really make sense to have a shell wrapper around a C wrapper? Should we have a single C wrapper used in all cases (internal, external, crosstool-ng), that handles everything?

I don't (yet) have a strong opinion on this, I just wanted to have a more global reflection about wrappers, and avoid creating many wrappers to solve different problems.

Best regards,

Thomas

Signature · 4 lines
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#12 lore raw

PATCH 4/5 Enable ccache for cmake packages

Quoted 6 lines · Thomas Petazzoni

Dear Luca Ceresoli,

Quoted 2 lines · Luca Ceresoli

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

Quoted 6 lines

This raises the question of whether we want to generalize this in some
way: should we be doing the ccache wrapper thing in a generic way, and use it everywhere? Since we already have a wrapper for external toolchain tools, does it really make sense to have a shell wrapper around a C wrapper? Should we have a single C wrapper used in all cases (internal, external, crosstool-ng), that handles everything?

Yes, I would be very much in favour of that. It also solves the issue
that some configure systems don't deal very well with a CC that contains spaces. And for external toolchains, it's nice that the actual options that are passed to the external toolchain are also visible to ccache.

I see two issues however:

- How to deal with HOSTCC?

- How to deal with HOSTCC_NOCCACHE? (TARGET_CC_NOCCACHE can probably be removed)

Anyway, for cmake the solution proposed by Samuel works even better.

Regards,
Arnout

Quoted 8 lines

I don't (yet) have a strong opinion on this, I just wanted to have a more global reflection about wrappers, and avoid creating many wrappers to solve different problems.

Best regards,

Thomas

Signature · 6 lines
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
#13 lore raw

PATCH 4/5 Enable ccache for cmake packages

In reply to Luca Ceresoli

Hi Lucas,

2013/3/6 Luca Ceresoli <[email protected]>:

Quoted 2 lines

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

It is possible to do this without any wrapper, setting CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the ccache binary path, and CMAKE_C_COMPILER_ARG1 and CMAKE_CXX_COMPILER_ARG1 to the actual C-compiler, respectively to the C++-compiler.

BTW, note that we currently don't handle space in the paths at all; hope that nobody triggers issues related to this ;)

Regards,

Signature · 1 line
Samuel
#14 lore raw

PATCH 4/5 Enable ccache for cmake packages

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Samuel" == Samuel Martin <[email protected]> writes:

Samuel> Hi Lucas,
Samuel> 2013/3/6 Luca Ceresoli <[email protected]>:

Quoted 2 lines
Quoted 2 lines

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

Samuel> It is possible to do this without any wrapper, setting Samuel> CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the ccache binary path, and Samuel> CMAKE_C_COMPILER_ARG1 and CMAKE_CXX_COMPILER_ARG1 to the actual Samuel> C-compiler, respectively to the C++-compiler.

Samuel> BTW, note that we currently don't handle space in the paths at all; Samuel> hope that nobody triggers issues related to this ;)

I bet a lot of (make based) build systems don't handle paths with spaces either.

Signature · 1 line
Bye, Peter Korsgaard
#15 lore raw

PATCH 4/5 Enable ccache for cmake packages

In reply to Samuel Martin
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Samuel" == Samuel Martin <[email protected]> writes:

Samuel> Hi Lucas,
Samuel> 2013/3/6 Luca Ceresoli <[email protected]>:

Quoted 2 lines
Quoted 2 lines

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

Samuel> It is possible to do this without any wrapper, setting Samuel> CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the ccache binary path, and Samuel> CMAKE_C_COMPILER_ARG1 and CMAKE_CXX_COMPILER_ARG1 to the actual Samuel> C-compiler, respectively to the C++-compiler.

Samuel> BTW, note that we currently don't handle space in the paths at all; Samuel> hope that nobody triggers issues related to this ;)

Luca, could you respin the patch to do this instead?

Signature · 1 line
Bye, Peter Korsgaard
#16 lore raw

PATCH 4/5 Enable ccache for cmake packages

Quoted 11 lines · Peter Korsgaard
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Samuel" == Samuel Martin <[email protected]> writes:

Samuel> Hi Lucas,
Samuel> 2013/3/6 Luca Ceresoli <[email protected]>:

Quoted 2 lines
Quoted 2 lines

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

Samuel> It is possible to do this without any wrapper, setting Samuel> CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the ccache binary path, and Samuel> CMAKE_C_COMPILER_ARG1 and CMAKE_CXX_COMPILER_ARG1 to the actual Samuel> C-compiler, respectively to the C++-compiler.

Quoted 5 lines

Samuel> BTW, note that we currently don't handle space in the paths at all; Samuel> hope that nobody triggers issues related to this ;)

Luca, could you respin the patch to do this instead?

Sure, will do.

Luca

#17 lore raw

PATCH 4/5 Enable ccache for cmake packages

In reply to Samuel Martin
Quoted 10 lines · Samuel Martin

Hi Lucas,

2013/3/6 Luca Ceresoli <[email protected]>:

Quoted 2 lines

CMake fails in detecting the compiler when ccache is used. Add a wrapper script to make it happy.

It is possible to do this without any wrapper, setting CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the ccache binary path, and CMAKE_C_COMPILER_ARG1 and CMAKE_CXX_COMPILER_ARG1 to the actual C-compiler, respectively to the C++-compiler.

Do you meansomething like this?

$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:

         @mkdir -p $(@D)
         @echo -en "\
         set(CMAKE_SYSTEM_NAME Linux)\n\
-       set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
-       set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+       set(CMAKE_C_COMPILER $(CCACHE))\n\
+       set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+       set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE))\n\
+       set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE))\n\
         set(CMAKE_C_FLAGS \"\$${CMAKE_C_FLAGS} $(TARGET_CFLAGS)\" CACHE 
STRING \"Buildroot CFLAGS\" FORCE)\n\
         set(CMAKE_CXX_FLAGS \"\$${CMAKE_CXX_FLAGS} $(TARGET_CXXFLAGS)\" 
CACHE STRING \"Buildroot CXXFLAGS\" F
         set(CMAKE_INSTALL_SO_NO_EXE 0)\n\
         set(CMAKE_PROGRAM_PATH \"$(HOST_DIR)/usr/bin\")\n\

This is undocumented in the official CMake manual, yet OpenWRT uses it, so it looks like a serious thing. And it actually works, except it breaks rpi-userland:

$ LANG= make V=1 rpi-userland{-dirclean,}
...
Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring

(cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ && rm -f CMakeCache.txt &&
/home/murray/devel/buildroot/output/host/usr/bin/cmake /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ -DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake" -DCMAKE_INSTALL_PREFIX="/usr" -DVMCS_INSTALL_PREFIX=/usr )
Re-run cmake no build system arguments
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 4.7.3
...

Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Building

PATH="/home/murray/devel/buildroot/output/host/bin:/home/murray/devel/buildroot/output/host/usr/bin:/home/murray/devel/buildroot/output/host/usr/sbin/:/home/murray/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games" PERLLIB="/home/murray/devel/buildroot/output/host/usr/lib/perl"
/usr/bin/make -j8 -C
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/
...
[ 8%] [ 8%] /home/murray/devel/buildroot/output/host/usr/bin/cmake -E
cmake_progress_report
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/CMakeFiles
/home/murray/devel/buildroot/output/host/usr/bin/ccache: invalid option
-- 'D'
Usage:
ccache [options]
ccache compiler [compiler options]
compiler [compiler options] (via symbolic link)

Options:
-c, --cleanup delete old files and recalculate size counters
(normally not needed as this is done
automatically)
-C, --clear clear the cache completely
-F, --max-files=N set maximum number of files in cache to N
(use 0 for
no limit)
-M, --max-size=SIZE set maximum size of cache to SIZE (use 0 for no
limit; available suffixes: G, M and K; default
suffix: G)
-s, --show-stats show statistics summary
-z, --zero-stats zero statistics counters

-h, --help print this help text
-V, --version print version and copyright information

See also <http://ccache.samba.org>.
make[3]: *** [interface/khronos/CMakeFiles/khrn_client.dir/common/khrn_int_hash_asm.s.o] Error 1
...

I don't know if rpi-userland is doing something wrong or the _ARG1 implementation is unreliable. I'll investigate on that, but if you have any clues they would be helpful. I am no CMake guru.

Luca

#18 lore raw

PATCH 4/5 Enable ccache for cmake packages

Quoted 81 lines · Luca Ceresoli
Quoted 4 lines · Samuel Martin

Hi Lucas,

2013/3/6 Luca Ceresoli <[email protected]>:

Quoted 1 line

CMake fails in detecting the compiler when ccache is used. Add a

wrapper

Quoted 6 lines
Quoted 1 line

script to make it happy.

It is possible to do this without any wrapper, setting CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the ccache binary path, and CMAKE_C_COMPILER_ARG1 and CMAKE_CXX_COMPILER_ARG1 to the actual C-compiler, respectively to the C++-compiler.

Do you meansomething like this?

$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:

        @mkdir -p $(@D)
        @echo -en "\
        set(CMAKE_SYSTEM_NAME Linux)\n\
(quoted diff) +4 -2
-       set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
-       set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+       set(CMAKE_C_COMPILER $(CCACHE))\n\
+       set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+       set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE))\n\
+       set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE))\n\
        set(CMAKE_C_FLAGS \"\$${CMAKE_C_FLAGS} $(TARGET_CFLAGS)\" 
CACHE STRING \"Buildroot CFLAGS\" FORCE)\n\
        set(CMAKE_CXX_FLAGS \"\$${CMAKE_CXX_FLAGS} 
$(TARGET_CXXFLAGS)\" CACHE STRING \"Buildroot CXXFLAGS\" F
        set(CMAKE_INSTALL_SO_NO_EXE 0)\n\
        set(CMAKE_PROGRAM_PATH \"$(HOST_DIR)/usr/bin\")\n\

This is undocumented in the official CMake manual, yet OpenWRT uses it, so it looks like a serious thing. And it actually works, except it breaks rpi-userland:

$ LANG= make V=1 rpi-userland{-dirclean,}
...
Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring

(cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ && rm -f CMakeCache.txt && /home/murray/devel/buildroot/output/host/usr/bin/cmake /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/

(quoted diff) +0 -2
-DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake" 
-DCMAKE_INSTALL_PREFIX="/usr" -DVMCS_INSTALL_PREFIX=/usr )

Re-run cmake no build system arguments
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 4.7.3
...

Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Building

PATH="/home/murray/devel/buildroot/output/host/bin:/home/murray/devel/buildroot/output/host/usr/bin:/home/murray/devel/buildroot/output/host/usr/sbin/:/home/murray/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games" PERLLIB="/home/murray/devel/buildroot/output/host/usr/lib/perl"
/usr/bin/make -j8 -C
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/
...
[ 8%] [ 8%] /home/murray/devel/buildroot/output/host/usr/bin/cmake
-E cmake_progress_report
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/CMakeFiles
/home/murray/devel/buildroot/output/host/usr/bin/ccache: invalid
option -- 'D'
Usage:
ccache [options]
ccache compiler [compiler options]
compiler [compiler options] (via symbolic link)

Options:
-c, --cleanup delete old files and recalculate size counters
(normally not needed as this is done
automatically)
-C, --clear clear the cache completely
-F, --max-files=N set maximum number of files in cache to N
(use 0 for
no limit)
-M, --max-size=SIZE set maximum size of cache to SIZE (use 0 for no
limit; available suffixes: G, M and K; default
suffix: G)
-s, --show-stats show statistics summary
-z, --zero-stats zero statistics counters

Quoted 8 lines

-h, --help print this help text
-V, --version print version and copyright information

See also <http://ccache.samba.org>.
make[3]: *** [interface/khronos/CMakeFiles/khrn_client.dir/common/khrn_int_hash_asm.s.o] Error 1
...

Oh, now I see, this happens when trying to assemble a .s file. With -DCMAKE_VERBOSE_MAKEFILE=ON you can see it does the wrong thing:

Quoted 39 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring

(cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ && rm -f CMakeCache.txt && /home/murray/devel/buildroot/output/host/usr/bin/cmake /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/

(quoted diff) +0 -2
-DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake" 
-DCMAKE_INSTALL_PREFIX="/usr" -DVMCS_INSTALL_PREFIX=/usr )

Re-run cmake no build system arguments
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 4.7.3
-- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- The ASM compiler identification is GNU
-- Found assembler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache CMake Warning at interface/usbdk/CMakeLists.txt:2 (message):
usbdk: using stubbed out hostreq, HDMI buttons and gestures code

-- Configuring done
-- Generating done
-- Build files have been written to:
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 touch /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/.stamp_configured

Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Building

...

Quoted 7 lines

[ 1%] Building C object
interface/vcos/pthreads/CMakeFiles/vcos.dir/vcos_pthreads.c.o
cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/interface/vcos/pthreads && /home/murray/devel/buildroot/output/host/usr/bin/ccache /home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gnueabihf-gcc
<blah blah>

...

Quoted 7 lines

[ 17%] Building ASM object interface/khronos/CMakeFiles/khrn_client.dir/common/khrn_int_hash_asm.s.o
cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/interface/khronos && /home/murray/devel/buildroot/output/host/usr/bin/ccache
-DEGL_SERVER_DISPMANX -DHAVE_CMAKE_CONFIG <blah blah>

Note in the above line the assembler executable is missing. Apparently CMake automagically set CMAKE_ASM_COMPILER equal to CMAKE_C_COMPILER or so, but it looks like CMAKE_ASM_COMPILER_ARG1 is empty.

So I set CMAKE_ASM_COMPILER{,_ARG1}just like I did for C and CXX:

-       set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
-       set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+       set(CMAKE_ASM_COMPILER $(CCACHE))\n\
+       set(CMAKE_C_COMPILER $(CCACHE))\n\
+       set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+       set(CMAKE_ASM_COMPILER_ARG1 $(TARGET_CC_NOCCACHE))\n\
+       set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE))\n\
+       set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE))\n\

Now it compiles correctly but with two bad side effects.

First, when building without ccache, CMake detects /usr/bin/cc as the (supposedly cross-) compiler, leading to an obvious link failure. Ok, this can be worked around by generating a different toolchainfile.cmake for the ccache and non-ccache cases, although it is annoying.

Second, during both the staging install and the target install phases, it complains saying:

Quoted 3 lines

Re-run cmake: build system dependency is missing
-- The ASM compiler identification is unknown
-- Found assembler:

/home/murray/devel/buildroot/output/host/usr/bin/ccache

Quoted 8 lines

-- Warning: Did not find file Compiler/-ASM
CMake Warning at interface/usbdk/CMakeLists.txt:2 (message):
usbdk: using stubbed out hostreq, HDMI buttons and gestures code

-- Configuring done
-- Generating done
-- Build files have been written to:

/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108

and recompiles the whole package again. These amount to three compilations! And they are ccached separately, so the first compilation actually takes an unreasonably long time. Well, during subsequent compilations with a full ccache the time loss is not more than 15 seconds on a dual core i5, but this clearly shows something is not working properly.

Samuel, your idea was certainly good, but I think CMake is thinking too much instead of just doing what it should... the result is a mess I have no time to dig into. I guess I'll stay with my stupid (but working) shell wrappers for the moment. :-(

Luca

#19 lore raw

PATCH 4/5 Enable ccache for cmake packages

Hi Lucas,

2013/3/21 Luca Ceresoli <[email protected]>:

Quoted 1 line

Luca Ceresoli wrote:

[...]

Quoted 64 lines

Oh, now I see, this happens when trying to assemble a .s file. With -DCMAKE_VERBOSE_MAKEFILE=ON you can see it does the wrong thing:

Quoted 39 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring

(cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ && rm -f CMakeCache.txt && /home/murray/devel/buildroot/output/host/usr/bin/cmake /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/

(quoted diff) +0 -2
-DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake"
-DCMAKE_INSTALL_PREFIX="/usr" -DVMCS_INSTALL_PREFIX=/usr )

Re-run cmake no build system arguments
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 4.7.3
-- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- The ASM compiler identification is GNU
-- Found assembler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache CMake Warning at interface/usbdk/CMakeLists.txt:2 (message):
usbdk: using stubbed out hostreq, HDMI buttons and gestures code

-- Configuring done
-- Generating done
-- Build files have been written to:
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 touch /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/.stamp_configured

Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Building

...

Quoted 8 lines

[ 1%] Building C object
interface/vcos/pthreads/CMakeFiles/vcos.dir/vcos_pthreads.c.o
cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/interface/vcos/pthreads && /home/murray/devel/buildroot/output/host/usr/bin/ccache /home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gnueabihf-gcc
<blah blah>

...

Quoted 8 lines

[ 17%] Building ASM object interface/khronos/CMakeFiles/khrn_client.dir/common/khrn_int_hash_asm.s.o
cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/interface/khronos && /home/murray/devel/buildroot/output/host/usr/bin/ccache
-DEGL_SERVER_DISPMANX -DHAVE_CMAKE_CONFIG <blah blah>

Quoted 5 lines

Note in the above line the assembler executable is missing. Apparently CMake automagically set CMAKE_ASM_COMPILER equal to CMAKE_C_COMPILER or so, but it looks like CMAKE_ASM_COMPILER_ARG1 is empty.

So I set CMAKE_ASM_COMPILER{,_ARG1}just like I did for C and CXX:

AFAIK, ccache only supports C, C++, Objective-C and Objective-C++. So, I think, it is useless to set it for anything else.

BTW, I also use to set most of the binutils/gcc binaries in my CMake toolchain files,
this includes setting the following variables:
CMAKE_{LINKER,AR,RANDLIB,NM,OBJCOPY,OBJDUMP,STRIP}.

So, IMHO, it is not stupid to set CMAKE_ASM_COMPILER as well, though I never need it so far.

[...]

Quoted 8 lines

Now it compiles correctly but with two bad side effects.

First, when building without ccache, CMake detects /usr/bin/cc as the (supposedly cross-) compiler, leading to an obvious link failure. Ok, this can be worked around by generating a different toolchainfile.cmake for the ccache and non-ccache cases, although it is annoying.

Well, here I use to conditionnaly enable ccache if found using something like this:

--
find_program(CCACHE "ccache")
if (CCACHE)
message( STATUS "Using ccache")
endif(CCACHE)

if (CCACHE AND NOT FORCE_NO_CCACHE)
set(CMAKE_C_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE) set(CMAKE_CXX_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE) set(CMAKE_C_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc") set(CMAKE_CXX_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
else(CCACHE AND NOT FORCE_NO_CCACHE)
set(CMAKE_C_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc") set(CMAKE_CXX_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
endif(CCACHE AND NOT FORCE_NO_CCACHE)
--

But I have to check that find program will actually look in ${HOST_DIR}/usr/bin, not in the system...

About the /usr/bin/cc issue, I think it will vanish by correctly handling the ccache presence;
if not, ccache seems to use a number of environment variables, so... our salute might eventually be there ;)

Quoted 17 lines

Second, during both the staging install and the target install phases, it complains saying:

Quoted 13 lines

Re-run cmake: build system dependency is missing
-- The ASM compiler identification is unknown
-- Found assembler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Warning: Did not find file Compiler/-ASM
CMake Warning at interface/usbdk/CMakeLists.txt:2 (message):
usbdk: using stubbed out hostreq, HDMI buttons and gestures code

-- Configuring done
-- Generating done
-- Build files have been written to:
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108

hum... here I think it is because ccache with the assembler is misleading CMake...

Quoted 8 lines

and recompiles the whole package again. These amount to three compilations! And they are ccached separately, so the first compilation actually takes an unreasonably long time. Well, during subsequent compilations with a full ccache the time loss is not more than 15 seconds on a dual core i5, but this clearly shows something is not working properly.

Quoted 4 lines

Samuel, your idea was certainly good, but I think CMake is thinking too much instead of just doing what it should... the result is a mess I have no time to dig into. I guess I'll stay with my stupid (but working) shell wrappers for the moment. :-(

Ok, I'll try to find some time to play with the toolchainfile.cmake generation reusing what you've already done.

Regards,

Signature · 1 line
Samuel
#20 lore raw

PATCH 4/5 Enable ccache for cmake packages

Quoted 64 lines · Samuel Martin

Hi Lucas,

2013/3/21 Luca Ceresoli <[email protected]>:

Quoted 1 line

Luca Ceresoli wrote:

[...]

Quoted 59 lines

Oh, now I see, this happens when trying to assemble a .s file. With -DCMAKE_VERBOSE_MAKEFILE=ON you can see it does the wrong thing:

Quoted 39 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring

(cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ && rm -f CMakeCache.txt && /home/murray/devel/buildroot/output/host/usr/bin/cmake /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/

(quoted diff) +0 -2
-DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake"
-DCMAKE_INSTALL_PREFIX="/usr" -DVMCS_INSTALL_PREFIX=/usr )

Re-run cmake no build system arguments
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 4.7.3
-- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Looking for execinfo.h
-- Looking for execinfo.h - found
-- The ASM compiler identification is GNU
-- Found assembler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache CMake Warning at interface/usbdk/CMakeLists.txt:2 (message):
usbdk: using stubbed out hostreq, HDMI buttons and gestures code

-- Configuring done
-- Generating done
-- Build files have been written to:
/home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 touch /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/.stamp_configured

Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Building

...

Quoted 7 lines

[ 1%] Building C object
interface/vcos/pthreads/CMakeFiles/vcos.dir/vcos_pthreads.c.o
cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/interface/vcos/pthreads && /home/murray/devel/buildroot/output/host/usr/bin/ccache /home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gnueabihf-gcc
<blah blah>

...

Quoted 7 lines

[ 17%] Building ASM object interface/khronos/CMakeFiles/khrn_client.dir/common/khrn_int_hash_asm.s.o
cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/interface/khronos && /home/murray/devel/buildroot/output/host/usr/bin/ccache
-DEGL_SERVER_DISPMANX -DHAVE_CMAKE_CONFIG <blah blah>

Quoted 7 lines
Quoted 5 lines

Note in the above line the assembler executable is missing. Apparently CMake automagically set CMAKE_ASM_COMPILER equal to CMAKE_C_COMPILER or so, but it looks like CMAKE_ASM_COMPILER_ARG1 is empty.

So I set CMAKE_ASM_COMPILER{,_ARG1}just like I did for C and CXX:

AFAIK, ccache only supports C, C++, Objective-C and Objective-C++. So, I think, it is useless to set it for anything else.

Ouch, right!
The fix is as easy as:

Quoted 1 line
    set(CMAKE_ASM_COMPILER $(TARGET_CC_NOCCACHE))\n\

Now when ccache is enabled it all works OK.

Quoted 25 lines

BTW, I also use to set most of the binutils/gcc binaries in my CMake toolchain files,
this includes setting the following variables:
CMAKE_{LINKER,AR,RANDLIB,NM,OBJCOPY,OBJDUMP,STRIP}.

So, IMHO, it is not stupid to set CMAKE_ASM_COMPILER as well, though I never need it so far.

[...]

Quoted 7 lines

Now it compiles correctly but with two bad side effects.

First, when building without ccache, CMake detects /usr/bin/cc as the (supposedly cross-) compiler, leading to an obvious link failure. Ok, this can be worked around by generating a different toolchainfile.cmake for the ccache and non-ccache cases, although it is annoying.

Well, here I use to conditionnaly enable ccache if found using something like this:

--
find_program(CCACHE "ccache")
if (CCACHE)
message( STATUS "Using ccache")
endif(CCACHE)

Quoted 10 lines

if (CCACHE AND NOT FORCE_NO_CCACHE)
set(CMAKE_C_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE) set(CMAKE_CXX_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE) set(CMAKE_C_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc") set(CMAKE_CXX_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
else(CCACHE AND NOT FORCE_NO_CCACHE)
set(CMAKE_C_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc") set(CMAKE_CXX_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
endif(CCACHE AND NOT FORCE_NO_CCACHE)
--

I'm not following you here. This would require patching the CMakeLists.txt file for every CMake package, right?

If we want to solve this cleanly, it should be in the BR infrastructure. I think we shouldgenerate the toolchainfile.cmakefile with a different content for the two cases (ccache/no-ccache). But we also should generate it unconditionally at every build, in case the user changes the BR2_CCACHE setting.

Luca

#21 lore raw

PATCH 4/5 Enable ccache for cmake packages

Lucas,

2013/3/22 Luca Ceresoli <[email protected]>:

Quoted 5 lines
Quoted 4 lines · Samuel Martin

Hi Lucas,

2013/3/21 Luca Ceresoli <[email protected]>:

[...]

Quoted 50 lines
Quoted 2 lines

AFAIK, ccache only supports C, C++, Objective-C and Objective-C++. So, I think, it is useless to set it for anything else.

Ouch, right!
The fix is as easy as:

Quoted 1 line
    set(CMAKE_ASM_COMPILER $(TARGET_CC_NOCCACHE))\n\

Now when ccache is enabled it all works OK.

Quoted 27 lines

BTW, I also use to set most of the binutils/gcc binaries in my CMake toolchain files,
this includes setting the following variables:
CMAKE_{LINKER,AR,RANDLIB,NM,OBJCOPY,OBJDUMP,STRIP}.

So, IMHO, it is not stupid to set CMAKE_ASM_COMPILER as well, though I never need it so far.

[...]

Quoted 8 lines

Now it compiles correctly but with two bad side effects.

First, when building without ccache, CMake detects /usr/bin/cc as the (supposedly cross-) compiler, leading to an obvious link failure. Ok, this can be worked around by generating a different toolchainfile.cmake for the ccache and non-ccache cases, although it is annoying.

Well, here I use to conditionnaly enable ccache if found using something like this:

--
find_program(CCACHE "ccache")
if (CCACHE)
message( STATUS "Using ccache")
endif(CCACHE)

Quoted 10 lines

if (CCACHE AND NOT FORCE_NO_CCACHE)
set(CMAKE_C_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE) set(CMAKE_CXX_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE) set(CMAKE_C_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc") set(CMAKE_CXX_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
else(CCACHE AND NOT FORCE_NO_CCACHE)
set(CMAKE_C_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc") set(CMAKE_CXX_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
endif(CCACHE AND NOT FORCE_NO_CCACHE)
--

Quoted 2 lines

I'm not following you here. This would require patching the CMakeLists.txt file for every CMake package, right?

Nope, just the toolchainfile.cmake.
And when ccache is disabled, we can just force it by setting FORCE_NO_CCACHE in the CMake infra.

Quoted 2 lines

If we want to solve this cleanly, it should be in the BR infrastructure.

Agree

Quoted 4 lines

I think we shouldgenerate the toolchainfile.cmakefile with a different content for the two cases (ccache/no-ccache). But we also should generate it unconditionally at every build, in case the user changes the BR2_CCACHE setting.

I'm not sure I follow you here...

If we are able to generate one toolchain.cmake file that handle both case (even enabling/disabling ccache from one run to another), we don't need anything else, nor regenerate the toolchainfile.cmake file at each build start.

Anyway, there are few things to check and test, I'll look into this. :)

Regards,

Signature · 1 line
Samuel
#22 lore raw

PATCH 4/5 Enable ccache for cmake packages

Quoted 53 lines · Samuel Martin

Lucas,

2013/3/22 Luca Ceresoli <[email protected]>:

Quoted 4 lines
Quoted 3 lines · Samuel Martin

Hi Lucas,

2013/3/21 Luca Ceresoli <[email protected]>:

[...]

Quoted 45 lines
Quoted 2 lines

AFAIK, ccache only supports C, C++, Objective-C and Objective-C++. So, I think, it is useless to set it for anything else.

Ouch, right!
The fix is as easy as:

Quoted 1 line
     set(CMAKE_ASM_COMPILER $(TARGET_CC_NOCCACHE))\n\

Now when ccache is enabled it all works OK.

Quoted 35 lines

BTW, I also use to set most of the binutils/gcc binaries in my CMake toolchain files,
this includes setting the following variables:
CMAKE_{LINKER,AR,RANDLIB,NM,OBJCOPY,OBJDUMP,STRIP}.

So, IMHO, it is not stupid to set CMAKE_ASM_COMPILER as well, though I never need it so far.

[...]

Quoted 7 lines

Now it compiles correctly but with two bad side effects.

First, when building without ccache, CMake detects /usr/bin/cc as the (supposedly cross-) compiler, leading to an obvious link failure. Ok, this can be worked around by generating a different toolchainfile.cmake for the ccache and non-ccache cases, although it is annoying.

Well, here I use to conditionnaly enable ccache if found using something like this:

--
find_program(CCACHE "ccache")
if (CCACHE)
message( STATUS "Using ccache")
endif(CCACHE)

if (CCACHE AND NOT FORCE_NO_CCACHE)

    set(CMAKE_C_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE)
    set(CMAKE_CXX_COMPILER "${CCACHE}" CACHE FILEPATH "" FORCE)
    set(CMAKE_C_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc")
    set(CMAKE_CXX_COMPILER_ARG1 "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")
else(CCACHE AND NOT FORCE_NO_CCACHE)
    set(CMAKE_C_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-gcc")
    set(CMAKE_CXX_COMPILER "${HOST_DIR}/usr/bin/${TARGET_TUPLE}-g++")

endif(CCACHE AND NOT FORCE_NO_CCACHE)
--

Quoted 5 lines
Quoted 2 lines

I'm not following you here. This would require patching the CMakeLists.txt file for every CMake package, right?

Nope, just the toolchainfile.cmake.
And when ccache is disabled, we can just force it by setting FORCE_NO_CCACHE in the CMake infra.

Ahh, now I got you idea! Thanks!

I worked on it, and although it was a bit tricky (probably because of my limited experience with CMake) it seems to be working.
Even if it is not yet fully tested, I'm submitting a v2 patch now to let you have a preliminary look at this work.

Luca

#23 lore raw

PATCH v2 1/1 Enable ccache for cmake packages

Use the *_ARG1 CMake variables to let CMake use ccache without failing in detecting the compiler.

Implementation based on an idea suggested by Samuel Martin.

---

NOTE! This patch is not yet well tested and may be largely non-optimal. Commit with care!

It was a bit tricky to implement, because the CMake-generated Makefiles do call cmake without supplying the same commandline arguments that we passed to the top-level cmake invocation.
The solution that I implemented for this issue is to use a tristate variable, named USE_CCACHE, that Buildroot passes to cmake:
- if set to ON, it triggers the configuration for ccache to be saved in the CMake cache;
- if set to OFF, it triggers the no-ccache configuration to be saved in the CMake cache;
- if it is not set, this means cmake is not being called directly by Buildroot; in this case it does not set any of the compiler CMake variables, thus using those previously cached.

Changed in v2:
- totally reimplemented based on Samuel's suggestion;
- added dependency on host-ccache for cmake packages if ccache is enabled.

 package/pkg-cmake.mk |   26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)
package/pkg-cmake.mk +24 -2
index b2fcd67..3d37502 100644
@@ -20,6 +20,13 @@
 #
 ################################################################################
 
+# Options to CMake that are equal for all packages
+ifeq ($(BR2_CCACHE),y)
+CMAKE_EXTRA_OPT += -DUSE_CCACHE=ON
+else
+CMAKE_EXTRA_OPT += -DUSE_CCACHE=OFF
+endif
+
 ################################################################################
 # inner-cmake-package -- defines how the configuration, compilation and
 # installation of a CMake package should be done, implements a few hooks to
@@ -65,6 +72,7 @@ define $(2)_CONFIGURE_CMDS
 	$$($$(PKG)_CONF_ENV) $(HOST_DIR)/usr/bin/cmake $$($$(PKG)_SRCDIR) \
 		-DCMAKE_TOOLCHAIN_FILE="$$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake" \
 		-DCMAKE_INSTALL_PREFIX="/usr" \
+		$(CMAKE_EXTRA_OPT) \
 		-DCMAKE_VERBOSE_MAKEFILE=ON \
 		$$($$(PKG)_CONF_OPT) \
 	)
@@ -82,6 +90,7 @@ define $(2)_CONFIGURE_CMDS
 		-DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY="BOTH" \
 		-DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE="BOTH" \
 		-DCMAKE_INSTALL_PREFIX="$$(HOST_DIR)/usr" \
+		$(CMAKE_EXTRA_OPT) \
 		$$($$(PKG)_CONF_OPT) \
 	)
 endef
@@ -94,6 +103,10 @@ $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix
 
 $(2)_DEPENDENCIES += host-cmake
 
+ifeq ($(BR2_CCACHE),y)
+$(2)_DEPENDENCIES += host-ccache
+endif
+
 #
 # Build step. Only define it if not already defined by the package .mk
 # file.
@@ -191,8 +204,17 @@ $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:
 	@mkdir -p $(@D)
 	@echo -en "\
 	set(CMAKE_SYSTEM_NAME Linux)\n\
-	set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
-	set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+	if(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+		set(CMAKE_ASM_COMPILER $(TARGET_CC_NOCCACHE))\n\
+		set(CMAKE_C_COMPILER $(CCACHE))\n\
+		set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+		set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+		set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+	endif(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+	if(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\
+		set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
+		set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+	endif(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\
 	set(CMAKE_C_FLAGS \"\$${CMAKE_C_FLAGS} $(TARGET_CFLAGS)\" CACHE STRING \"Buildroot CFLAGS\" FORCE)\n\
 	set(CMAKE_CXX_FLAGS \"\$${CMAKE_CXX_FLAGS} $(TARGET_CXXFLAGS)\" CACHE STRING \"Buildroot CXXFLAGS\" FORCE)\n\
 	set(CMAKE_INSTALL_SO_NO_EXE 0)\n\
Signature · 1 line
1.7.10.4
#24 lore raw

PATCH v2 1/1 Enable ccache for cmake packages

Hi Lucas,

2013/3/22 Luca Ceresoli <[email protected]>:

Quoted 51 lines

Use the *_ARG1 CMake variables to let CMake use ccache without failing in detecting the compiler.

Implementation based on an idea suggested by Samuel Martin.

---

NOTE! This patch is not yet well tested and may be largely non-optimal. Commit with care!

It was a bit tricky to implement, because the CMake-generated Makefiles do call cmake without supplying the same commandline arguments that we passed to the top-level cmake invocation.
The solution that I implemented for this issue is to use a tristate variable, named USE_CCACHE, that Buildroot passes to cmake:
- if set to ON, it triggers the configuration for ccache to be saved in the CMake cache;
- if set to OFF, it triggers the no-ccache configuration to be saved in the CMake cache;
- if it is not set, this means cmake is not being called directly by Buildroot; in this case it does not set any of the compiler CMake variables, thus using those previously cached.

Changed in v2:
- totally reimplemented based on Samuel's suggestion;
- added dependency on host-ccache for cmake packages if ccache is enabled.

 package/pkg-cmake.mk |   26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)
package/pkg-cmake.mk +7 -0
index b2fcd67..3d37502 100644
@@ -20,6 +20,13 @@
 #
 ################################################################################
 
+# Options to CMake that are equal for all packages
+ifeq ($(BR2_CCACHE),y)
+CMAKE_EXTRA_OPT += -DUSE_CCACHE=ON
+else
+CMAKE_EXTRA_OPT += -DUSE_CCACHE=OFF
+endif
+
 ################################################################################
 # inner-cmake-package -- defines how the configuration, compilation and
 # installation of a CMake package should be done, implements a few hooks to

package/pkg-cmake.mk define $(2)_CONFIGURE_CMDS · line 75

Quoted 4 lines
package/pkg-cmake.mk +1 -0
@@ -65,6 +72,7 @@ define $(2)_CONFIGURE_CMDS
        $$($$(PKG)_CONF_ENV) $(HOST_DIR)/usr/bin/cmake $$($$(PKG)_SRCDIR) \
                -DCMAKE_TOOLCHAIN_FILE="$$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake" \
                -DCMAKE_INSTALL_PREFIX="/usr" \
+               $(CMAKE_EXTRA_OPT) \

Or the inlined if works as well ;)

package/pkg-cmake.mk define $(2)_CONFIGURE_CMDS · line 76

Quoted 1 line
package/pkg-cmake.mk +0 -0
                -DCMAKE_VERBOSE_MAKEFILE=ON \

Ahem... I think this belongs to some debug remains ;)

BTW, with CMake, to switch verbose log, just run: 'VERBOSE=1 make' In Buildroot, 'make V=1' also works... because of some magics in the Makefile in the topdir.

Quoted 11 lines
package/pkg-cmake.mk +1 -0
                $$($$(PKG)_CONF_OPT) \
        )
@@ -82,6 +90,7 @@ define $(2)_CONFIGURE_CMDS
                -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY="BOTH" \
                -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE="BOTH" \
                -DCMAKE_INSTALL_PREFIX="$$(HOST_DIR)/usr" \
+               $(CMAKE_EXTRA_OPT) \
                $$($$(PKG)_CONF_OPT) \
        )
 endef

package/pkg-cmake.mk $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix · line 108

Quoted 6 lines
package/pkg-cmake.mk +3 -0
@@ -94,6 +103,10 @@ $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix
 
 $(2)_DEPENDENCIES += host-cmake
 
+ifeq ($(BR2_CCACHE),y)
+$(2)_DEPENDENCIES += host-ccache
+endif

I don't think this is need since host-ccache is one of the first target add to the list in the topdir Makefile.

package/pkg-cmake.mk $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake: · line 208

Quoted 12 lines
package/pkg-cmake.mk +3 -2
+
 #
 # Build step. Only define it if not already defined by the package .mk
 # file.
@@ -191,8 +204,17 @@ $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake:
        @mkdir -p $(@D)
        @echo -en "\
        set(CMAKE_SYSTEM_NAME Linux)\n\
-       set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
-       set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+       if(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+               set(CMAKE_ASM_COMPILER $(TARGET_CC_NOCCACHE))\n\

I think it does not hurt to put the CMAKE_ASM_COMPILER definition outside the if block...

package/pkg-cmake.mk $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake: · line 214

Quoted 6 lines
package/pkg-cmake.mk +6 -0
+               set(CMAKE_C_COMPILER $(CCACHE))\n\
+               set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+               set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+               set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+       endif(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+       if(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\

You can replace the 2 above lines by:
else ()

Quoted 9 lines
package/pkg-cmake.mk +3 -1
+               set(CMAKE_C_COMPILER $(TARGET_CC_NOCCACHE))\n\
+               set(CMAKE_CXX_COMPILER $(TARGET_CXX_NOCCACHE))\n\
+       endif(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\
        set(CMAKE_C_FLAGS \"\$${CMAKE_C_FLAGS} $(TARGET_CFLAGS)\" CACHE STRING \"Buildroot CFLAGS\" FORCE)\n\
        set(CMAKE_CXX_FLAGS \"\$${CMAKE_CXX_FLAGS} $(TARGET_CXXFLAGS)\" CACHE STRING \"Buildroot CXXFLAGS\" FORCE)\n\
        set(CMAKE_INSTALL_SO_NO_EXE 0)\n\
--

1.7.10.4

I have not tested this patch but something really close, nor done extensive tests, but I expect it will works fine.

Regards,

Signature · 1 line
Samuel
#25 lore raw

PATCH v2 1/1 Enable ccache for cmake packages

Dear Samuel Martin,

Quoted 10 lines · Samuel Martin

package/pkg-cmake.mk define $(2)_CONFIGURE_CMDS · line 75

Quoted 8 lines

################################################################################

 # inner-cmake-package -- defines how the configuration, compilation and
 # installation of a CMake package should be done, implements a few hooks to
package/pkg-cmake.mk +1 -0
@@ -65,6 +72,7 @@ define $(2)_CONFIGURE_CMDS
        $$($$(PKG)_CONF_ENV) $(HOST_DIR)/usr/bin/cmake $$($$(PKG)_SRCDIR) \
                -DCMAKE_TOOLCHAIN_FILE="$$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake" \
                -DCMAKE_INSTALL_PREFIX="/usr" \
+               $(CMAKE_EXTRA_OPT) \

Or the inlined if works as well ;)

Quoted 2 lines

package/pkg-cmake.mk define $(2)_CONFIGURE_CMDS · line 76

Quoted 1 line
package/pkg-cmake.mk +0 -0
                -DCMAKE_VERBOSE_MAKEFILE=ON \

Ahem... I think this belongs to some debug remains ;)

But it hasn't been added by this patch from Luca.

Quoted 5 lines

package/pkg-cmake.mk $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix · line 108

Quoted 3 lines
package/pkg-cmake.mk +3 -0
+ifeq ($(BR2_CCACHE),y)
+$(2)_DEPENDENCIES += host-ccache
+endif

I don't think this is need since host-ccache is one of the first target add to the list in the topdir Makefile.

Correct.

Quoted 8 lines

package/pkg-cmake.mk $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake: · line 214

Quoted 6 lines
package/pkg-cmake.mk +6 -0
+               set(CMAKE_C_COMPILER $(CCACHE))\n\
+               set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+               set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+               set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+       endif(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+       if(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\

You can replace the 2 above lines by:
else ()

I don't think so. See the explanation in Luca's commit log. He needs to differentiate three cases:

* USE_CCACHE is defined and set to ON
* USE_CCACHE is defined and set to OFF
* USE_CCACHE is not defined

Luca, I think a portion of the message you've added below the '---' should be added either in the commit log, or maybe as a comment in pkg-cmake.mk.

Thanks,

Thomas

Signature · 4 lines
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
#26 lore raw

PATCH v2 1/1 Enable ccache for cmake packages

Hi Thomas, Lucas,

2013/3/23 Thomas Petazzoni <[email protected]>:

Quoted 13 lines

Dear Samuel Martin,

Quoted 10 lines · Samuel Martin

package/pkg-cmake.mk define $(2)_CONFIGURE_CMDS · line 75

Quoted 8 lines

################################################################################

 # inner-cmake-package -- defines how the configuration, compilation and
 # installation of a CMake package should be done, implements a few hooks to
package/pkg-cmake.mk +1 -0
@@ -65,6 +72,7 @@ define $(2)_CONFIGURE_CMDS
        $$($$(PKG)_CONF_ENV) $(HOST_DIR)/usr/bin/cmake $$($$(PKG)_SRCDIR) \
                -DCMAKE_TOOLCHAIN_FILE="$$(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake" \
                -DCMAKE_INSTALL_PREFIX="/usr" \
+               $(CMAKE_EXTRA_OPT) \

Or the inlined if works as well ;)

Quoted 4 lines
Quoted 2 lines

package/pkg-cmake.mk define $(2)_CONFIGURE_CMDS · line 76

Quoted 1 line
package/pkg-cmake.mk +0 -0
                -DCMAKE_VERBOSE_MAKEFILE=ON \

Ahem... I think this belongs to some debug remains ;)

But it hasn't been added by this patch from Luca.

Not this patch, but for sure, it's not on master ;)

Quoted 19 lines
Quoted 7 lines

package/pkg-cmake.mk $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix · line 108

Quoted 5 lines
package/pkg-cmake.mk +3 -0
+ifeq ($(BR2_CCACHE),y)
+$(2)_DEPENDENCIES += host-ccache
+endif

I don't think this is need since host-ccache is one of the first target add to the list in the topdir Makefile.

Correct.

Quoted 8 lines

package/pkg-cmake.mk $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake: · line 214

Quoted 6 lines
package/pkg-cmake.mk +6 -0
+               set(CMAKE_C_COMPILER $(CCACHE))\n\
+               set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+               set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+               set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+       endif(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+       if(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\

You can replace the 2 above lines by:
else ()

Quoted 6 lines

I don't think so. See the explanation in Luca's commit log. He needs to differentiate three cases:

* USE_CCACHE is defined and set to ON
* USE_CCACHE is defined and set to OFF
* USE_CCACHE is not defined

Well, I didn't really pay enough/that much attention to the tristate thing because I didn't really get the point...
Now, I got it, but I'm really dubious about behavior in the last case.

I mean: the goal of this toolchainfile.cmake is, among other thing, to be able to reuse the Buildroot toolchain
to build a projects outside Buildroot because CMake is good at this.

So, in that case, we want to be sure we correctly export the toolchain properties.

If we don't set any compiler in the case USE_CCACHE is undefined, then we totally miss this goal,
and IMHO, it does not make sense because the toolchainfile.cmake file is generated by Buildroot,
for the Buildroot toolchain and nothing else.

Maybe the logic is a bit more complex... something like that:

set(FIND_PROGRAM_PATH "$(HOST_DIR)/usr/bin")
find_program(CCACHE "ccache")
if (DEFINED USE_CCACHE)
if (USE_CCACHE AND CCACHE)

    # set the compiler variables with ccache support
  else ()
    # set the compiler variables without ccache support
  endif ()
else ()
  message("To enable ccache usage, add -DUSE_CACHE=ON on the command line")
  # fallback: set only the compiler variables (with no ccache support)
endif ()

Notes:
* The find_program call and CCACHE variable are here to check/ensure the ccache binary existence.
* We only look for ccache in the Buildroot host bindir, ie. there is no way to use the binary from the
system in the above example.

Regards,

Signature · 1 line
Samuel
#27 lore raw

PATCH v2 1/1 Enable ccache for cmake packages

Quoted 5 lines · Samuel Martin

[...]

Quoted 3 lines
Quoted 2 lines
Quoted 1 line
                 -DCMAKE_VERBOSE_MAKEFILE=ON \

Ahem... I think this belongs to some debug remains ;)

But it hasn't been added by this patch from Luca.

Not this patch, but for sure, it's not on master ;)

Sorry, my mistake! :)

[...]

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

package/pkg-cmake.mk $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake: · line 214

Quoted 6 lines
package/pkg-cmake.mk +6 -0
+               set(CMAKE_C_COMPILER $(CCACHE))\n\
+               set(CMAKE_CXX_COMPILER $(CCACHE))\n\
+               set(CMAKE_C_COMPILER_ARG1 $(TARGET_CC_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+               set(CMAKE_CXX_COMPILER_ARG1 $(TARGET_CXX_NOCCACHE) CACHE STRING \"\" FORCE)\n\
+       endif(DEFINED USE_CCACHE AND USE_CCACHE)\n\
+       if(DEFINED USE_CCACHE AND NOT USE_CCACHE)\n\

You can replace the 2 above lines by:
else ()

Quoted 8 lines
Quoted 6 lines

I don't think so. See the explanation in Luca's commit log. He needs to differentiate three cases:

* USE_CCACHE is defined and set to ON
* USE_CCACHE is defined and set to OFF
* USE_CCACHE is not defined

Well, I didn't really pay enough/that much attention to the tristate thing because I didn't really get the point...

If you add at the beginning of toolchainfile.cmake the line:
message("USE_CCACHE=${USE_CCACHE}")
and then run make rpi-userland{-dirclean,} you'll see:

Quoted 28 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

rpi-userland 5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108 Configuring

(cd /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/ && rm -f CMakeCache.txt && /home/murray/devel/buildroot/output/host/usr/bin/cmake /home/murray/devel/buildroot/output/build/rpi-userland-5e9a740a88a889dfc8a18bb1b00c17e5dd9d0108/

(quoted diff) +0 -2
-DCMAKE_TOOLCHAIN_FILE="/home/murray/devel/buildroot/output/host/usr/share/buildroot/toolchainfile.cmake" 
-DCMAKE_INSTALL_PREFIX="/usr" -DUSE_CCACHE=ON -DVMCS_INSTALL_PREFIX=/usr )

Re-run cmake no build system arguments
USE_CCACHE=ON
USE_CCACHE=ON
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 4.7.3
-- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache USE_CCACHE=
-- Check for working C compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting C compiler ABI info
USE_CCACHE=
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache USE_CCACHE=
-- Check for working CXX compiler:
/home/murray/devel/buildroot/output/host/usr/bin/ccache -- works -- Detecting CXX compiler ABI info
USE_CCACHE=

Quoted 8 lines

-- Detecting CXX compiler ABI info - done
USE_CCACHE=
-- Looking for execinfo.h
USE_CCACHE=
-- Looking for execinfo.h - found
-- The ASM compiler identification is GNU
-- Found assembler:
/home/murray/devel/buildroot/output/host/usr/bin/arm-linux-gnueabihf-gcc

Apparently CMake interprets multiple times the toolchainfile.cmake, and it does not always pass the original parameters, so USE_CCACHE is sometimes undefined. Hence my idea to handle that variable as a tristate.

Quoted 6 lines

Now, I got it, but I'm really dubious about behavior in the last case.

I mean: the goal of this toolchainfile.cmake is, among other thing, to be able to reuse the Buildroot toolchain
to build a projects outside Buildroot because CMake is good at this.

Quoted 8 lines

So, in that case, we want to be sure we correctly export the toolchain properties.

If we don't set any compiler in the case USE_CCACHE is undefined, then we totally miss this goal,
and IMHO, it does not make sense because the toolchainfile.cmake file is generated by Buildroot,
for the Buildroot toolchain and nothing else.

I actually overlookedthe need to make the whole thing usable from outside Buildroot. It should be made working indeed.

Quoted 6 lines

Maybe the logic is a bit more complex... something like that:

set(FIND_PROGRAM_PATH "$(HOST_DIR)/usr/bin")
find_program(CCACHE "ccache")
if (DEFINED USE_CCACHE)
if (USE_CCACHE AND CCACHE)

Quoted 8 lines
     # set the compiler variables with ccache support
   else ()
     # set the compiler variables without ccache support
   endif ()
else ()
   message("To enable ccache usage, add -DUSE_CACHE=ON on the command line")
   # fallback: set only the compiler variables (with no ccache support)
endif ()

I don't like very much automagic things, such as using a tool transparently when it'sfound. We have a BR2_CCACHE config knob, that should rule IMO.

Still, yours is an interesting proposal. The only alternative is probably to put the ccachelogic in the wrappers as Thomas proposed in reply to my v1 patch.

But I see a problem here: Buildroot exports BUILDROOT_CACHE_DIR, and $(HOST_DIR)/usr/bin/ccacheuses that environment variable to locate its cache directory. If the same ccache executable is executed standalone it does won't find that environment variable and default to ~/.ccache.

Anyway, thinking more about how to implement this feature, I think maybe Thomas' proposal can avoid the mentioned issues, as the call to ccache (as well as the ccache directory path) can be coded inside a (C?) wrapper if the wrapper has been built with BR2_CCACHE=y. But we would need to regenerate the wrapper whenever the user changes the BR2_CCACHE...

Also, the BR-built ccache would be used transparently even when calling ${HOST_DIR}/usr/bin/*-gcc. This shows clearly an advantage, although the user might not understand well that he's using ccache under the hood (maybe not a problem).

But I'd like to understand better your idea, Thomas. The ccache usage is independent from the toolchain type (Buildroot/ct-NG/ external). Would you implement a unique wrapper to be used for all three toolchain types, and replace the current ext-toolchain-wrapper.c?

Luca

#28 lore raw

PATCH v2 1/1 Enable ccache for cmake packages

Hi Lucas, all,

2013/3/28 Luca Ceresoli <[email protected]>:
[...]

Quoted 4 lines

If you add at the beginning of toolchainfile.cmake the line:
message("USE_CCACHE=${USE_CCACHE}")
and then run make rpi-userland{-dirclean,} you'll see:

[...]

Quoted 6 lines

Apparently CMake interprets multiple times the toolchainfile.cmake, and it does not always pass the original parameters, so USE_CCACHE is sometimes undefined. Hence my idea to handle that variable as a tristate.

I don't know that much about CMake innards, but i'm pretty sure, in the end, it is what is set in the CMakeCache.txt file that is used.

Quoted 38 lines
Quoted 7 lines

Now, I got it, but I'm really dubious about behavior in the last case.

I mean: the goal of this toolchainfile.cmake is, among other thing, to be able to reuse the Buildroot toolchain
to build a projects outside Buildroot because CMake is good at this.

Quoted 8 lines

So, in that case, we want to be sure we correctly export the toolchain properties.

If we don't set any compiler in the case USE_CCACHE is undefined, then we totally miss this goal,
and IMHO, it does not make sense because the toolchainfile.cmake file is generated by Buildroot,
for the Buildroot toolchain and nothing else.

I actually overlookedthe need to make the whole thing usable from outside Buildroot. It should be made working indeed.

Quoted 6 lines

Maybe the logic is a bit more complex... something like that:

set(FIND_PROGRAM_PATH "$(HOST_DIR)/usr/bin")
find_program(CCACHE "ccache")
if (DEFINED USE_CCACHE)
if (USE_CCACHE AND CCACHE)

Quoted 9 lines
     # set the compiler variables with ccache support
   else ()
     # set the compiler variables without ccache support
   endif ()
else ()
   message("To enable ccache usage, add -DUSE_CACHE=ON on the command
line")
   # fallback: set only the compiler variables (with no ccache support)
endif ()
Quoted 3 lines

I don't like very much automagic things, such as using a tool transparently when it'sfound. We have a BR2_CCACHE config knob, that should rule IMO.

An alternative, with no magics when used by Buildroot:

In the CONFIGURE_CMDS functions in pkg-cmake, add -DCCACHE="$(CCACHE)", then, in the toolchainfile.cmake:

---
set(FIND_PROGRAM_PATH "$(HOST_DIR)/usr/bin")

if(DEFINED CCACHE) # check if run by buildroot.
# Note that to make it very cristal and enforce this check, we can
use the following condition:
# if(DEFINED CCACHE AND NOT "$ENV{BR2_VERSION}" STREQUAL "") #
# Run inside buildroot
if (DEFINED USE_CCACHE NOT "${CCACHE}" STREQUAL "")

    # set the compiler variables with ccache support
  else ()
    # set the compiler variables without ccache support
  endif ()
else ()
  # Run outside buildroot
  if (DEFINED USE_CCACHE)
    # ccache support is set
    if (USE_CCACHE)
      # ccache support is requested
      find_program(CCACHE "ccache")
      if(CCACHE)
        # ccache support is available
        # set the compiler variables with ccache support
      else ()
        # ccache support is not available
        message("No ccache binary found")
        # set the compiler variables without ccache support
      endif ()
    else ()
      # ccache support is explicitly disabled
      # set the compiler variables without ccache support
    endif ()
  else ()
    # ccache support is undefined
    message("To enable ccache usage, add -DUSE_CACHE=ON on the command line")
    # fallback: set only the compiler variables (with no ccache support)
  endif ()
endif ()
---
Quoted 9 lines

Still, yours is an interesting proposal. The only alternative is probably to put the ccachelogic in the wrappers as Thomas proposed in reply to my v1 patch.

But I see a problem here: Buildroot exports BUILDROOT_CACHE_DIR, and $(HOST_DIR)/usr/bin/ccacheuses that environment variable to locate its cache directory. If the same ccache executable is executed standalone it does won't find that environment variable and default to ~/.ccache.

Is that a problem to fallback on ~/.ccache when used outside Buildroot?

BTW, if you really want to reuse the Buildroot ccache directory, you can add:

---
if(DEFINED CCACHE_DIR)
set(ENV{BUILDROOT_CACHE_DIR} "${CCACHE_DIR}")
else()
message("Using the default ccache directory.") message("To use an alternative ccache directory, add
-DCCACHE_DIR=/path/to/somewhere")
endif()
---

Quoted 8 lines

Anyway, thinking more about how to implement this feature, I think maybe Thomas' proposal can avoid the mentioned issues, as the call to ccache (as well as the ccache directory path) can be coded inside a (C?) wrapper if the wrapper has been built with BR2_CCACHE=y. But we would need to regenerate the wrapper whenever the user changes the BR2_CCACHE...

Quoted 6 lines

Also, the BR-built ccache would be used transparently even when calling ${HOST_DIR}/usr/bin/*-gcc. This shows clearly an advantage, although the user might not understand well that he's using ccache under the hood (maybe not a problem).

But I'd like to understand better your idea, Thomas.

Me too, though I'm not so fond of wrappers...

Quoted 8 lines

The ccache
usage is independent from the toolchain type (Buildroot/ct-NG/ external). Would you implement a unique wrapper to be used for all three toolchain types, and replace the current ext-toolchain-wrapper.c?

Luca

Regards,

Signature · 1 line
Samuel
#29 lore raw

PATCH 5/5 CMake packages: remove .cmake files from target directory

In reply to Luca Ceresoli

Just like pkgconfig files.

---
 Makefile |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Makefile +3 -1
index f6d8494..8346f27 100644
@@ -452,7 +452,9 @@ ifeq ($(BR2_HAVE_DEVFILES),y)
 	( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )
 else
 	rm -rf $(TARGET_DIR)/usr/include $(TARGET_DIR)/usr/share/aclocal \
-		$(TARGET_DIR)/usr/lib/pkgconfig $(TARGET_DIR)/usr/share/pkgconfig
+		$(TARGET_DIR)/usr/lib/pkgconfig $(TARGET_DIR)/usr/share/pkgconfig \
+		$(TARGET_DIR)/usr/lib/cmake $(TARGET_DIR)/usr/share/cmake
+	find $(TARGET_DIR)/usr/{lib,share}/ -name '*.cmake' -print0 | xargs -0 rm -f
 	find $(TARGET_DIR)/lib \( -name '*.a' -o -name '*.la' \) -print0 | xargs -0 rm -f
 	find $(TARGET_DIR)/usr/lib \( -name '*.a' -o -name '*.la' \) -print0 | xargs -0 rm -f
 endif
Signature · 1 line
1.7.10.4
#30 lore raw

PATCH 5/5 CMake packages: remove .cmake files from target directory

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Luca" == Luca Ceresoli <[email protected]> writes:

Luca> Just like pkgconfig files.
Luca> Signed-off-by: Luca Ceresoli <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard