package/ccache: add universal wrapper script
PATCH 1/1 package/ccache: add universal wrapper script
This patch adds a wrapper script to call ccache, which will differentiate if the HOST or either internal or external TARGET compiler is called.
Ccache needs to know what compiler we use, but can be simply fooled to fetch invalid objects. To circumvent this, it uses different algorythms to distiguish the compilers. This needs to be tuned for Buildroot's usage, as the internal toolchains' characteristics change with every recompilation, while ccache can not directly access the external toolchains (symlinks).
This patch does the following:
- Reenable using mtime (hash of binary size and modification date) as the default compiler_check algorythm. This will be used for the host compiler.
- Renames the ccache binary, so the wrapper script can be installed on it's place.
- Creates a wrapper script with hardcoded values for performance, which will disable the mtime compiler check and add an extra file to hash if it is called for a target compiler (either internal or external). It is basically all the same if we are using internal or external toolchains, ccache only sees a path.
- Creates a checksum file for the above method from the extract of the current Buildroot config, which should not change when configuring packages or recompiling the whole project. However it will change if the compiler options change, thus safely pointing to cached objects.
--- Config.in | 7 ------- package/ccache/ccache.mk | 25 ++++++++++++++++++++----- 2 files changed, 20 insertions(+), 12 deletions(-)
Config.in +0 -7
@@ -257,13 +257,6 @@ config BR2_CCACHE up future builds. By default, the cache is stored in $HOME/.buildroot-ccache. - Note that Buildroot does not try to invalidate the cache - contents when the compiler changes in an incompatible - way. Therefore, if you make a change to the compiler version - and/or configuration, you are responsible for purging the - ccache cache by removing the $HOME/.buildroot-ccache - directory. - if BR2_CCACHE config BR2_CCACHE_DIR
package/ccache/ccache.mk +20 -5
@@ -26,16 +26,14 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib # is already used by autotargets for the ccache package. # BR_CACHE_DIR is exported by Makefile based on config option # BR2_CCACHE_DIR. -# - ccache shouldn't use the compiler binary mtime to detect a change in -# the compiler, because in the context of Buildroot, that completely -# defeats the purpose of ccache. Of course, that leaves the user -# responsible for purging its cache when the compiler changes. # - Change hard-coded last-ditch default to match path in .config, to avoid # the need to specify BR_CACHE_DIR when invoking ccache directly. +# - Rename ccache binary to ccache.bin, so later the wrapper can be safely +# named ccache, as referred everywhere define HOST_CCACHE_PATCH_CONFIGURATION sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c - sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c + sed -i 's,#define MYNAME "ccache",#define MYNAME "ccache.bin",' $(@D)/ccache.h endef HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION
@@ -58,6 +56,23 @@ endef HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_DO_INITIAL_SETUP endif +# Install a wrapper script, which, when using the TARGET_ toolchain, will make +# ccache use the hash of the toolchain options instead of the compiler's mtime +define HOST_CCACHE_INSTALL_WRAPPER + mv $(CCACHE) $(CCACHE).bin + echo "#!/bin/sh" > $(CCACHE) + echo "if [ \"$$"'1'"\" = \"$(TARGET_CC_NOCCACHE)\" ] || \\" >> $(CCACHE) + echo " [ \"$$"'1'"\" = \"$(TARGET_CXX_NOCCACHE)\" ]; then" >> $(CCACHE) + echo " export CCACHE_COMPILERCHECK=none CCACHE_EXTRAFILES=$(HOST_DIR)/ccache_toolchain.config.hash" >> $(CCACHE) + echo "fi" >> $(CCACHE) + echo "$(CCACHE).bin \"$$"'@'"\"" >> $(CCACHE) + grep "BR2_ARCH\|BR2_TOOLCHAIN\|BR2_GCC\|BR2_BINUTILS\|BR2_UCLIBC\|BR2_GLIBC\|BR2_EGLIBC" $(BR2_CONFIG) | \ + grep -v "#" | md5sum > $(HOST_DIR)/ccache_toolchain.config.hash + chmod 755 $(CCACHE) +endef + +HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_INSTALL_WRAPPER + $(eval $(host-autotools-package)) ifeq ($(BR2_CCACHE),y)
Signature · 1 line
1.7.10.4
PATCH 1/1 package/ccache: add universal wrapper script
Karoly,
Quoted 77 lines · Karoly Kasza
This patch adds a wrapper script to call ccache, which will differentiate if the HOST or either internal or external TARGET compiler is called.
Ccache needs to know what compiler we use, but can be simply fooled to fetch invalid objects. To circumvent this, it uses different algorythms to distiguish the compilers. This needs to be tuned for Buildroot's usage, as the internal toolchains' characteristics change with every recompilation, while ccache can not directly access the external toolchains (symlinks).
This patch does the following:
- Reenable using mtime (hash of binary size and modification date) as the defaultcompiler_checkalgorythm. This will be used for the host compiler.
- Renames the ccache binary, so the wrapper script can be installed on it's place.
- Creates a wrapper script with hardcoded values for performance, which will disable the mtime compiler check and add an extra file to hash if it is called for a target compiler (either internal or external). It is basically all the same if we are using internal or external toolchains, ccache only sees a path.
- Creates a checksum file for the above method from the extract of the current Buildroot config, which should not change when configuring packages or recompiling the whole project. However it will change if the compiler options change, thus safely pointing to cached objects.Signed-off-by Karoly Kasza <[email protected]>--- Config.in | 7 ------- package/ccache/ccache.mk | 25 ++++++++++++++++++++----- 2 files changed, 20 insertions(+), 12 deletions(-)Config.in +0
-7@@ -257,13 +257,6 @@ config BR2_CCACHE up future builds. By default, the cache is stored in $HOME/.buildroot-ccache. - Note that Buildroot does not try to invalidate the cache - contents when the compiler changes in an incompatible - way. Therefore, if you make a change to the compiler version - and/or configuration, you are responsible for purging the - ccache cache by removing the $HOME/.buildroot-ccache - directory. - if BR2_CCACHE config BR2_CCACHE_DIRpackage/ccache/ccache.mk +5
-5@@ -26,16 +26,14 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib # is already used by autotargets for the ccache package. # BR_CACHE_DIR is exported by Makefile based on config option # BR2_CCACHE_DIR. -# - ccache shouldn't use the compiler binary mtime to detect a change in -# the compiler, because in the context of Buildroot, that completely -# defeats the purpose of ccache. Of course, that leaves the user -# responsible for purging its cache when the compiler changes. # - Change hard-coded last-ditch default to match path in .config, to avoid # the need to specify BR_CACHE_DIR when invoking ccache directly. +# - Rename ccache binary to ccache.bin, so later the wrapper can be safely +# named ccache, as referred everywhere define HOST_CCACHE_PATCH_CONFIGURATION sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c - sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c + sed -i 's,#define MYNAME "ccache",#define MYNAME "ccache.bin",' $(@D)/ccache.h endef HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION@@ -58,6 +56,23 @@ endef HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_DO_INITIAL_SETUP endif +# Install a wrapper script, which, when using the TARGET_ toolchain, will make +# ccache use the hash of the toolchain options instead of the compiler's mtime
package/ccache/ccache.mk endef · line 70
Quoted 10 lines
package/ccache/ccache.mk +10
-0+define HOST_CCACHE_INSTALL_WRAPPER + mv $(CCACHE) $(CCACHE).bin + echo "#!/bin/sh" > $(CCACHE) + echo "if [ \"$$"'1'"\" = \"$(TARGET_CC_NOCCACHE)\" ] || \\" >> $(CCACHE) + echo " [ \"$$"'1'"\" = \"$(TARGET_CXX_NOCCACHE)\" ]; then" >> $(CCACHE) + echo " export CCACHE_COMPILERCHECK=none CCACHE_EXTRAFILES=$(HOST_DIR)/ccache_toolchain.config.hash" >> $(CCACHE) + echo "fi" >> $(CCACHE) + echo "$(CCACHE).bin \"$$"'@'"\"" >> $(CCACHE) + grep "BR2_ARCH\|BR2_TOOLCHAIN\|BR2_GCC\|BR2_BINUTILS\|BR2_UCLIBC\|BR2_GLIBC\|BR2_EGLIBC" $(BR2_CONFIG) | \ + grep -v "#" | md5sum > $(HOST_DIR)/ccache_toolchain.config.hash
Does the list used for grep need these as well, as they influence the ext-toolchain-wrapper.c?
* BR2_TARGET_OPTIMIZATION
* BR2_SOFT_FLOAT
Danomi -
Quoted 15 lines
package/ccache/ccache.mk +5
-1+ chmod 755 $(CCACHE) +endef + +HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_INSTALL_WRAPPER + $(eval $(host-autotools-package)) ifeq ($(BR2_CCACHE),y) --1.7.10.4
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
PATCH 1/1 package/ccache: add universal wrapper script
Hi Danomi,
Quoted 12 lines
+ grep
"BR2_ARCH\|BR2_TOOLCHAIN\|BR2_GCC\|BR2_BINUTILS\|BR2_UCLIBC\|BR2_GLIBC\|BR2_EGLIBC" $(BR2_CONFIG) | \Quoted 1 line
+ grep -v "#" | md5sum > $(HOST_DIR)/ccache_toolchain.config.hash
Does the list used for grep need these as well, as they influence the
ext-toolchain-wrapper.c?*
BR2_TARGET_OPTIMIZATION
*BR2_SOFT_FLOAT
This patch don't really calculates with ext-toolchain-wrapper.c, but it's a good question if these config options should be factored in, I am not sure. If these are passed at compilation time to the target compiler (by the wrapper binary), then theoretically ccache gets them and calculates them in, as ccache is not influenced by the wrapper (mtime turned off) - so we don't need them in the hash file.
If the target compiler binary changes based on their values, then they should be calculated in - I think this doesn't happen, as the external binaries won't be changed.
Opinions?
(It's not a be a big deal to add them also into the grep line, but if we restrict the hash too much, then it won't match the cache hits even when it should. If we don't restrict it enough, it may generate false cache hits and through that corrupted binaries.)
BR
Karoly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20150430/47bff646/attachment.html>
PATCH 1/1 package/ccache: add universal wrapper script
Quoted 14 lines · K?roly Kasza
Hi Danomi,
Quoted 1 line
+ grep
"BR2_ARCH\|BR2_TOOLCHAIN\|BR2_GCC\|BR2_BINUTILS\|BR2_UCLIBC\|BR2_GLIBC\|BR2_EGLIBC" $(BR2_CONFIG) | \
> + grep -v "#" | md5sum > $(HOST_DIR)/ccache_toolchain.config.hashDoes the list used for grep need these as well, as they influence the
ext-toolchain-wrapper.c?*
BR2_TARGET_OPTIMIZATION
*BR2_SOFT_FLOAT
Quoted 6 lines
This patch don't really calculates with
ext-toolchain-wrapper.c, but it's a good question if these config options should be factored in, I am not sure. If these are passed at compilation time to the target compiler (by the wrapper binary), then theoretically ccache gets them and calculates them in, as ccache is not influenced by the wrapper (mtime turned off) - so we don't need them in the hash file.
The wrapper adds these values, and ccache calls the wrapper. So if these values
change, ccache won't notice it (at least if the wrapper is used).
The list of affected options is actually a lot longer I'm afraid. I think this
is the full list of options affecting the compiler. Some of these are Config.in options, some of these are options calculated in the makefiles, but all of them are available to make.
Options that affect the build of the compiler (internal toolchain):BR2_GCC_TARGET_ARCHBR2_GCC_TARGET_ABIBR2_GCC_TARGET_CPUBR2_GCC_TARGET_CPU_REVISIONBR2_GCC_TARGET_FPUBR2_GCC_TARGET_FLOAT_ABIBR2_GCC_TARGET_MODEBR2_EXTRA_GCC_CONFIG_OPTIONSBR2_SOFT_FLOAT
BR2_PTHREADS_*BR2_TOOLCHAIN_BUILDROOT_LIBCBR2_TOOLCHAIN_BUILDROOT_WCHARBR2_GCC_ENABLE_TLSBR2_GCC_ENABLE_LTOBR2_GCC_ENABLE_GRAPHITEBR2_GCC_ENABLE_OPENMP (probably not needed, only affects headers/libraries) BR2_GCC_ENABLE_LIBMUDFLAP (probably not needed, only affects headers/libraries) GCC_VERSION
The contents of package/gcc/$(GCC_VERSION)/*
The contents of $(BR2_XTENSA_OVERLAY_DIR)/xtensa_$(call qstrip,\ $(BR2_XTENSA_CORE_NAME)).tar
The contents of package/gcc/gcc.mk (although that may be too conservative) The contents of package/gcc/gcc-final.mk (although that may be too conservative)
Note that the libc options are not relevant, since they will affect only
libraries and header files. ccache doesn't deal with libraries, and ccache will still fingerprint header files.
A simpler way of getting the correct fingerprint would be to look atHOST_GCC_FINAL_CONF_OPTS, after removing all occurences of $(HOST_DIR) and $(TARGET_DIR) from it. It's slightly too conservative since some unimportant options (e.g. BR2_INSTALL_LIBSTDCPP) still appear in there, but on the other hand it's more future safe since any option that affects the compiler will appear in it.
Options that affect external toolchains:
The BR2_GCC_TARGET_* options mentioned above BR2_TARGET_OPTIMIZATIONBR2_SOFT_FLOAT
The contents of $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) The contents of $(BR2_TOOLCHAIN_EXTERNAL_PATH) (if preinstalled)
As it happens, all of the flags that affect the toolchain wrapper are already
in TOOLCHAIN_EXTERNAL_CFLAGS.
So I think the easiest solution is to calculate the ccache hash in two places:
- In HOST_GCC_FINAL_POST_PATCH_HOOKS, calculate a hash of the contents of $(@D) and $(subst $(HOST_DIR),@,$(HOST_GCC_FINAL_CONF_OPTS))
- In TOOLCHAIN_EXTERNAL_INSTALL_WRAPPER, calculate a hash of $(DL_DIR)/$(TOOLCHAIN_EXTERNAL_SOURCE) or of $(BR2_TOOLCHAIN_EXTERNAL_PATH) and $(TOOLCHAIN_EXTERNAL_CFLAGS)
Obviously, this needs a bit of experimentation to see if it really works.
Regards,
Arnout
Quoted 19 lines
If the target compiler binary changes based on their values, then they should be calculated in - I think this doesn't happen, as the external binaries won't be changed.
Opinions?(It's not a be a big deal to add them also into the grep line, but if we restrict the hash too much, then it won't match the cache hits even when it should. If we don't restrict it enough, it may generate false cache hits and through that corrupted binaries.)
BR
Karoly_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
PATCH 1/1 package/ccache: add universal wrapper script
In reply to Karoly KaszaQuoted 75 lines · Karoly Kasza
This patch adds a wrapper script to call ccache, which will differentiate if the HOST or either internal or external TARGET compiler is called.
Ccache needs to know what compiler we use, but can be simply fooled to fetch invalid objects. To circumvent this, it uses different algorythms to distiguish the compilers. This needs to be tuned for Buildroot's usage, as the internal toolchains' characteristics change with every recompilation, while ccache can not directly access the external toolchains (symlinks).
This patch does the following:
- Reenable using mtime (hash of binary size and modification date) as the defaultcompiler_checkalgorythm. This will be used for the host compiler.
- Renames the ccache binary, so the wrapper script can be installed on it's place.
- Creates a wrapper script with hardcoded values for performance, which will disable the mtime compiler check and add an extra file to hash if it is called for a target compiler (either internal or external). It is basically all the same if we are using internal or external toolchains, ccache only sees a path.
- Creates a checksum file for the above method from the extract of the current Buildroot config, which should not change when configuring packages or recompiling the whole project. However it will change if the compiler options change, thus safely pointing to cached objects.Signed-off-by Karoly Kasza <[email protected]>--- Config.in | 7 ------- package/ccache/ccache.mk | 25 ++++++++++++++++++++----- 2 files changed, 20 insertions(+), 12 deletions(-)Config.in +0
-7@@ -257,13 +257,6 @@ config BR2_CCACHE up future builds. By default, the cache is stored in $HOME/.buildroot-ccache. - Note that Buildroot does not try to invalidate the cache - contents when the compiler changes in an incompatible - way. Therefore, if you make a change to the compiler version - and/or configuration, you are responsible for purging the - ccache cache by removing the $HOME/.buildroot-ccache - directory. - if BR2_CCACHE config BR2_CCACHE_DIRpackage/ccache/ccache.mk +3
-5@@ -26,16 +26,14 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib # is already used by autotargets for the ccache package. # BR_CACHE_DIR is exported by Makefile based on config option # BR2_CCACHE_DIR. -# - ccache shouldn't use the compiler binary mtime to detect a change in -# the compiler, because in the context of Buildroot, that completely -# defeats the purpose of ccache. Of course, that leaves the user -# responsible for purging its cache when the compiler changes. # - Change hard-coded last-ditch default to match path in .config, to avoid # the need to specify BR_CACHE_DIR when invoking ccache directly. +# - Rename ccache binary to ccache.bin, so later the wrapper can be safely +# named ccache, as referred everywhere define HOST_CCACHE_PATCH_CONFIGURATION sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c - sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c + sed -i 's,#define MYNAME "ccache",#define MYNAME "ccache.bin",' $(@D)/ccache.h endef HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION@@ -58,6 +56,23 @@ endef HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_DO_INITIAL_SETUP endif
package/ccache/ccache.mk endef · line 68
Quoted 10 lines
package/ccache/ccache.mk +10
-0+# Install a wrapper script, which, when using the TARGET_ toolchain, will make +# ccache use the hash of the toolchain options instead of the compiler's mtime +define HOST_CCACHE_INSTALL_WRAPPER + mv $(CCACHE) $(CCACHE).bin + echo "#!/bin/sh" > $(CCACHE) + echo "if [ \"$$"'1'"\" = \"$(TARGET_CC_NOCCACHE)\" ] || \\" >> $(CCACHE) + echo " [ \"$$"'1'"\" = \"$(TARGET_CXX_NOCCACHE)\" ]; then" >> $(CCACHE) + echo " export CCACHE_COMPILERCHECK=none CCACHE_EXTRAFILES=$(HOST_DIR)/ccache_toolchain.config.hash" >> $(CCACHE) + echo "fi" >> $(CCACHE) + echo "$(CCACHE).bin \"$$"'@'"\"" >> $(CCACHE)
I don't really like this additional wrapper script. Can't we just define two
variables, HOST_CCACHE and TARGET_CCACHE:
HOST_CCACHE = $(HOST_DIR)/usr/bin/ccache
TARGET_CCACHE = \ CCACHE_COMPILERCHECK=none \ CCACHE_EXTRAFILES=$(HOST_DIR)/ccache_toolchain.config.hash \ $(HOST_DIR)/usr/bin/ccache
TARGET_CC := $(TARGET_CCACHE) $(TARGET_CC)
(and subsitute the other references to $(CCACHE) as appropriate)
The handling of cmake and qmake will require a bit of thinking however.
If you do keep the wrapper script, then I'd prefer if you create it as
package/ccache/ccache_wrapper.in and fill in the relevant variables with sed.
package/ccache/ccache.mk endef · line 70
Quoted 2 lines
package/ccache/ccache.mk +2
-0+ grep "BR2_ARCH\|BR2_TOOLCHAIN\|BR2_GCC\|BR2_BINUTILS\|BR2_UCLIBC\|BR2_GLIBC\|BR2_EGLIBC" $(BR2_CONFIG) | \ + grep -v "#" | md5sum > $(HOST_DIR)/ccache_toolchain.config.hash
Since CCACHE_EXTRAFILES is anyway taken together with the input file into one
big hash, I wouldn't bother with hashing it here, but just put the actual options to be hashed in that file. That gives a good reference when you want to debug ccache, and the speed penalty is negligible (the overhead of opening the file is larger than reading and hashing it).
Regards,
Arnout
Quoted 9 lines
package/ccache/ccache.mk +5
-0+ chmod 755 $(CCACHE) +endef + +HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_INSTALL_WRAPPER + $(eval $(host-autotools-package)) ifeq ($(BR2_CCACHE),y)
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
PATCH 1/1 package/ccache: add universal wrapper script
Hi Karoly,
Quoted 107 lines · Arnout Vandecappelle
Quoted 75 lines · Karoly Kasza
This patch adds a wrapper script to call ccache, which will differentiate if the HOST or either internal or external TARGET compiler is called.
Ccache needs to know what compiler we use, but can be simply fooled to fetch invalid objects. To circumvent this, it uses different algorythms to distiguish the compilers. This needs to be tuned for Buildroot's usage, as the internal toolchains' characteristics change with every recompilation, while ccache can not directly access the external toolchains (symlinks).
This patch does the following:
- Reenable using mtime (hash of binary size and modification date) as the defaultcompiler_checkalgorythm. This will be used for the host compiler.
- Renames the ccache binary, so the wrapper script can be installed on it's place.
- Creates a wrapper script with hardcoded values for performance, which will disable the mtime compiler check and add an extra file to hash if it is called for a target compiler (either internal or external). It is basically all the same if we are using internal or external toolchains, ccache only sees a path.
- Creates a checksum file for the above method from the extract of the current Buildroot config, which should not change when configuring packages or recompiling the whole project. However it will change if the compiler options change, thus safely pointing to cached objects.Signed-off-by Karoly Kasza <[email protected]>--- Config.in | 7 ------- package/ccache/ccache.mk | 25 ++++++++++++++++++++----- 2 files changed, 20 insertions(+), 12 deletions(-)Config.in +0
-7@@ -257,13 +257,6 @@ config BR2_CCACHE up future builds. By default, the cache is stored in $HOME/.buildroot-ccache. - Note that Buildroot does not try to invalidate the cache - contents when the compiler changes in an incompatible - way. Therefore, if you make a change to the compiler version - and/or configuration, you are responsible for purging the - ccache cache by removing the $HOME/.buildroot-ccache - directory. - if BR2_CCACHE config BR2_CCACHE_DIRpackage/ccache/ccache.mk +3
-5@@ -26,16 +26,14 @@ HOST_CCACHE_CONF_OPTS += --with-bundled-zlib # is already used by autotargets for the ccache package. # BR_CACHE_DIR is exported by Makefile based on config option # BR2_CCACHE_DIR. -# - ccache shouldn't use the compiler binary mtime to detect a change in -# the compiler, because in the context of Buildroot, that completely -# defeats the purpose of ccache. Of course, that leaves the user -# responsible for purging its cache when the compiler changes. # - Change hard-coded last-ditch default to match path in .config, to avoid # the need to specify BR_CACHE_DIR when invoking ccache directly. +# - Rename ccache binary to ccache.bin, so later the wrapper can be safely +# named ccache, as referred everywhere define HOST_CCACHE_PATCH_CONFIGURATION sed -i 's,getenv("CCACHE_DIR"),getenv("BR_CACHE_DIR"),' $(@D)/ccache.c - sed -i 's,conf->compiler_check = x_strdup("mtime"),conf->compiler_check = x_strdup("none"),' $(@D)/conf.c sed -i 's,"%s/.ccache","$(BR_CACHE_DIR)",' $(@D)/conf.c + sed -i 's,#define MYNAME "ccache",#define MYNAME "ccache.bin",' $(@D)/ccache.h endef HOST_CCACHE_POST_PATCH_HOOKS += HOST_CCACHE_PATCH_CONFIGURATION@@ -58,6 +56,23 @@ endef HOST_CCACHE_POST_INSTALL_HOOKS += HOST_CCACHE_DO_INITIAL_SETUP endifpackage/ccache/ccache.mk endef · line 68
Quoted 10 lines
package/ccache/ccache.mk +10
-0+# Install a wrapper script, which, when using the TARGET_ toolchain, will make +# ccache use the hash of the toolchain options instead of the compiler's mtime +define HOST_CCACHE_INSTALL_WRAPPER + mv $(CCACHE) $(CCACHE).bin + echo "#!/bin/sh" > $(CCACHE) + echo "if [ \"$$"'1'"\" = \"$(TARGET_CC_NOCCACHE)\" ] || \\" >> $(CCACHE) + echo " [ \"$$"'1'"\" = \"$(TARGET_CXX_NOCCACHE)\" ]; then" >> $(CCACHE) + echo " export CCACHE_COMPILERCHECK=none CCACHE_EXTRAFILES=$(HOST_DIR)/ccache_toolchain.config.hash" >> $(CCACHE) + echo "fi" >> $(CCACHE) + echo "$(CCACHE).bin \"$$"'@'"\"" >> $(CCACHE)I don't really like this additional wrapper script. Can't we just define two
variables,HOST_CCACHEandTARGET_CCACHE:
HOST_CCACHE= $(HOST_DIR)/usr/bin/ccacheTARGET_CCACHE = \ CCACHE_COMPILERCHECK=none \ CCACHE_EXTRAFILES=$(HOST_DIR)/ccache_toolchain.config.hash \ $(HOST_DIR)/usr/bin/ccache
TARGET_CC:= $(TARGET_CCACHE) $(TARGET_CC)(and subsitute the other references to $(CCACHE) as appropriate)
The handling of cmake and qmake will require a bit of thinking however.
If you do keep the wrapper script, then I'd prefer if you create it as
package/ccache/ccache_wrapper.in and fill in the relevant variables with sed.
Quoted 10 lines
package/ccache/ccache.mk endef · line 70
Quoted 2 lines
package/ccache/ccache.mk +2
-0+ grep "BR2_ARCH\|BR2_TOOLCHAIN\|BR2_GCC\|BR2_BINUTILS\|BR2_UCLIBC\|BR2_GLIBC\|BR2_EGLIBC" $(BR2_CONFIG) | \ + grep -v "#" | md5sum > $(HOST_DIR)/ccache_toolchain.config.hashSince
CCACHE_EXTRAFILESis anyway taken together with the input file into one
big hash, I wouldn't bother with hashing it here, but just put the actual options to be hashed in that file. That gives a good reference when you want to debug ccache, and the speed penalty is negligible (the overhead of opening the file is larger than reading and hashing it).
Did you have a chance to look any further into this topic ?
Thanks,
Thomas
PATCH 1/1 package/ccache: add universal wrapper script
Hi Thomas, all
I had (have) an IRL interrupt, but I still plan to take this topic further, based on Arnout's comments.
Also, ccache released a new minor version, I'll send a bump in a few days, as soon as I have time to test it properly.
The new CCACHE_COMPILERCHECK=string:<value> option could provide useful after the bump.
Kind regards,
Karoly
Quoted 5 lines
Did you have a chance to look any further into this topic ?
Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20150529/37ebcd6d/attachment.html>