ccache: Fixes for 4.1
cover ccache: Fixes for 4.1
The following changes since commit 23cb39a5fa2a55681e7bc2605f435135cec9173b:
diffstat: point the license checksum at the license (2021-01-05 13:48:07 +0000)
are available in the Git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/ccache
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ccache
Robert Yang (3):
ccache: Extend to nativesdk
buildtools-tarball: Add nativesdk-ccache
ccache.bbclass: Set CCACHE_TEMPDIR
meta/classes/ccache.bbclass | 4 ++++ meta/recipes-core/meta/buildtools-tarball.bb | 1 + meta/recipes-devtools/ccache/ccache_4.1.bb | 2 +- 3 files changed, 6 insertions(+), 1 deletion(-)
Signature · 1 line
2.17.1
PATCH 1/3 ccache: Extend to nativesdk
Now we have to use host's ccache as described by:f5b29367af ccache.bbclass: use ccache from host distribution
So extend it to nativesdk and will add it to buildtools-tarball.
--- meta/recipes-devtools/ccache/ccache_4.1.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/ccache/ccache_4.1.bb +1 -1
@@ -23,4 +23,4 @@ inherit cmake PATCHTOOL = "patch" -BBCLASSEXTEND = "native" +BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.17.1
PATCH 2/3 buildtools-tarball: Add nativesdk-ccache
In reply to Robert YangAdd it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.
--- meta/recipes-core/meta/buildtools-tarball.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ nativesdk-rpcsvc-proto \ nativesdk-patch \ nativesdk-mtools \ + nativesdk-ccache \ " MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"
Signature · 1 line
2.17.1
PATCH 3/3 ccache.bbclass: Set CCACHE_TEMPDIR
In reply to Robert YangFixed when build with buildtools-tarball:
$ bitbake linux-libc-headers HOSTCC arch/x86/tools/relocs_common.o ccache: error: Failed to create directory /run/user/0/ccache-tmp: Permission denied
--- meta/classes/ccache.bbclass | 4 ++++ 1 file changed, 4 insertions(+)
meta/classes/ccache.bbclass +4 -0
@@ -35,6 +35,10 @@ export CCACHE_CONFIGPATH ?= "${COREBASE}/meta/conf/ccache.conf" export CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${MULTIMACH_TARGET_SYS}/${PN}" +# Fixed errors: +# ccache: error: Failed to create directory /run/user/0/ccache-tmp: Permission denied +export CCACHE_TEMPDIR ?= "${CCACHE_DIR}/tmp" + # We need to stop ccache considering the current directory or the # debug-prefix-map target directory to be significant when calculating # its hash. Without this the cache would be invalidated every time
Signature · 1 line
2.17.1
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
In reply to Robert YangQuoted 17 lines · Robert Yang
Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.
Signed-off-by Robert Yang <[email protected]>---
meta/recipes-core/meta/buildtools-tarball.bb | 1 +1 file changed, 1 insertion(+)meta/recipes-core/meta/buildtools-tarball.bb +1
-0@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ nativesdk-rpcsvc-proto \ nativesdk-patch \ nativesdk-mtools \ + nativesdk-ccache \
Quoted 1 line
"
My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it but its not essential, required or helps much in standard builds (which would reuse from sstate if built previously).
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
Hi RP,
Quoted 9 lines · Richard Purdie
Quoted 8 lines · Robert Yang
Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.
Signed-off-by Robert Yang <[email protected]>--- meta/recipes-core/meta/buildtools-tarball.bb | 1 + 1 file changed, 1 insertion(+)
Quoted 13 lines
meta/recipes-core/meta/buildtools-tarball.bb TOOLCHAIN_HOST_TASK ?= "\ · line 33
Quoted 10 lines
meta/recipes-core/meta/buildtools-tarball.bb +1
-0@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ nativesdk-rpcsvc-proto \ nativesdk-patch \ nativesdk-mtools \ + nativesdk-ccache \ "My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it
The problem is that user installed ccache may not work with our recipes, for example, we found the following recipes failed to build with ccache on different hosts:
cdrtools-native
fribidi
rocksdb
So have a fixed version of ccache can make sure the recipes work well, it also helps us to reproduce the errors easily when user reported a bug.
// Robert
Quoted 7 lines
but its not essential, required or helps much in standard builds (which would reuse from sstate if built previously).
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
Quoted 31 lines · Robert Yang
Hi RP,
Quoted 9 lines · Richard Purdie
Quoted 8 lines · Robert Yang
Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.
Signed-off-by Robert Yang <[email protected]>---
meta/recipes-core/meta/buildtools-tarball.bb | 1 +1 file changed, 1 insertion(+)Quoted 13 lines
Quoted 10 lines
meta/recipes-core/meta/buildtools-tarball.bb +1
-0@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ nativesdk-rpcsvc-proto \ nativesdk-patch \ nativesdk-mtools \ + nativesdk-ccache \"
My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it
The problem is that user installed ccache may not work with our recipes, for example, we found the following recipes failed to build with ccache on different hosts:
Quoted 7 lines
cdrtools-native
fribidi
rocksdbSo have a fixed version of ccache can make sure the recipes work well, it also helps us to reproduce the errors easily when user reported a bug.
I really do not want ccache in buildtools. If this is such a problem, we need to work out how to resolve the circular dependency issues so ccache-native can work again.
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
Hi RP,
Quoted 37 lines · Richard Purdie
Quoted 36 lines · Robert Yang
Hi RP,
Quoted 9 lines · Richard Purdie
Quoted 8 lines · Robert Yang
Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.
Signed-off-by Robert Yang <[email protected]>--- meta/recipes-core/meta/buildtools-tarball.bb | 1 + 1 file changed, 1 insertion(+)Quoted 13 lines
meta/recipes-core/meta/buildtools-tarball.bb TOOLCHAIN_HOST_TASK ?= "\ · line 33
Quoted 10 lines
meta/recipes-core/meta/buildtools-tarball.bb +1
-0@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ nativesdk-rpcsvc-proto \ nativesdk-patch \ nativesdk-mtools \ + nativesdk-ccache \ "My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it
The problem is that user installed ccache may not work with our recipes, for example, we found the following recipes failed to build with ccache on different hosts:
cdrtools-native
fribidi
rocksdb
Quoted 6 lines
Quoted 2 lines
So have a fixed version of ccache can make sure the recipes work well, it also helps us to reproduce the errors easily when user reported a bug.
I really do not want ccache in buildtools. If this is such a problem, we need to work out how to resolve the circular dependency issues so ccache-native can work again.
There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build
* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.
It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native.
Host's ccache isn't reliable from the history.
// Robert
Quoted 7 lines
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
Quoted 60 lines · Robert Yang
Hi RP,
Quoted 6 lines · Richard Purdie
Quoted 5 lines · Robert Yang
Hi RP,
Quoted 2 lines · Richard Purdie
Quoted 1 line · Robert Yang
Add it to buildtools-tarball so that there will be a unify version of
ccache,
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
which can help avoid various compile errors.
Signed-off-by Robert Yang <[email protected]>--- meta/recipes-core/meta/buildtools-tarball.bb | 1 + 1 file changed, 1 insertion(+)(quoted diff) +0
-0b/meta/recipes-core/meta/buildtools-tarball.bb
Quoted 14 lines
Quoted 14 lines
Quoted 12 lines
meta/recipes-core/meta/buildtools-tarball.bb TOOLCHAIN_HOST_TASK ?= "\ · line 33
Quoted 9 lines
index
9da81d55235..2ffdd7c7253100644meta/recipes-core/meta/buildtools-tarball.bb +1
-0@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ nativesdk-rpcsvc-proto \ nativesdk-patch \ nativesdk-mtools \ + nativesdk-ccache \ "My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it
Quoted 1 line
Quoted 1 line
The problem is that user installed ccache may not work with our
recipes, for
Quoted 1 line
Quoted 1 line
example, we found the following recipes failed to build with ccache on
different
Quoted 8 lines
Quoted 8 lines
hosts:
cdrtools-native
fribidi
rocksdbSo have a fixed version of ccache can make sure the recipes work well,
it also
Quoted 5 lines
Quoted 1 line
helps us to reproduce the errors easily when user reported a bug.
I really do not want ccache in buildtools. If this is such a problem, we need to work out how to resolve the circular dependency issues so ccache-native can work again.
There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires
more time to build, so it might be a bottleneck for the build* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.
Quoted 8 lines
It seems that there isn't a good way to fix the problem, or we have to disable
ccache for native recipes, only make other recipes (cross, target and nativesdk)
use ccache-native.Host's ccache isn't reliable from the history.
We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.
Alternatively, I think it's possible to roll your own buildtools that has ccache?
Quoted 13 lines
// Robert
Quoted 7 lines
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
In reply to Robert YangQuoted 4 lines · Robert Yang
There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build
Quoted 8 lines
* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.
It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native.
Host's ccache isn't reliable from the history.
Thanks for the analysis. I think I'd be inclined to disable ccache for native recipes then. If anyone does want to use it for native recipes, they can build a buildtools tarball of their own that contains it but I don't want to default to that.
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
Quoted 12 lines · Richard Purdie
Quoted 11 lines · Robert Yang
There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.
It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native.
Quoted 6 lines
Quoted 1 line
Host's ccache isn't reliable from the history.
Thanks for the analysis. I think I'd be inclined to disable ccache for native recipes then. If anyone does want to use it for native recipes, they can build a buildtools tarball of their own that contains it but I don't want to default to that.
Yes, make sense, I will send a patch to disable ccache for native recipes and enable ccache-native for other recipes.
// Robert
Quoted 5 lines
Cheers,
Richard
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
In reply to Joshua WattQuoted 74 lines · Joshua Watt
On Thu, Jan 7, 2021, 5:00 AM Robert Yang <[email protected] <mailto:[email protected]>> wrote:
Hi RP, On 1/7/21 6:40 PM, Richard Purdie wrote: > On Thu, 2021-01-07 at 10:44 +0800, Robert Yang wrote: >> Hi RP, >> >> On 1/6/21 9:56 PM, Richard Purdie wrote: >>> On Wed, 2021-01-06 at 04:09 -0800, Robert Yang wrote: >>>> Add it to buildtools-tarball so that there will be a unify version of ccache, >>>> which can help avoid various compile errors. >>>> >>>> Signed-off-by: Robert Yang <[email protected] <mailto:[email protected]>> >>>> --- >>>> meta/recipes-core/meta/buildtools-tarball.bb <http://buildtools-tarball.bb> | 1 + >>>> 1 file changed, 1 insertion(+) >>>> >>>> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb <http://buildtools-tarball.bb> b/meta/recipes-core/meta/buildtools-tarball.bb <http://buildtools-tarball.bb> >>>> index 9da81d55235..2ffdd7c7253 100644 >>>> --- a/meta/recipes-core/meta/buildtools-tarball.bb <http://buildtools-tarball.bb> >>>> +++ b/meta/recipes-core/meta/buildtools-tarball.bb <http://buildtools-tarball.bb> >>>> @@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\ >>>> nativesdk-rpcsvc-proto \ >>>> nativesdk-patch \ >>>> nativesdk-mtools \ >>>> + nativesdk-ccache \ >>>> " >>> >>> My view is we should not start to ship ccache with buildtools by >>> default. Its something that the user can install if they want/need it >> >> >> The problem is that user installed ccache may not work with our recipes, for >> example, we found the following recipes failed to build with ccache on different >> hosts: >> >> cdrtools-native >> fribidi >> rocksdb >> >> >> So have a fixed version of ccache can make sure the recipes work well, it also >> helps us to reproduce the errors easily when user reported a bug. > > I really do not want ccache in buildtools. If this is such a problem, > we need to work out how to resolve the circular dependency issues so > ccache-native can work again. There are two problems to use cache-native now: * ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build * We need disable ccache-native for about 20 native recipes to resolve the circular dependency. It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native. Host's ccache isn't reliable from the history.
Quoted 3 lines
We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.
Alternatively, I think it's possible to roll your own buildtools that has ccache?
Yes, thanks, it's a good suggestion.
// Robert
Quoted 14 lines
// Robert
>
> Cheers,
>
> Richard
>
>
>Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
In reply to Joshua WattQuoted 5 lines · Joshua Watt
We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.
Alternatively, I think it's possible to roll your own buildtools that has ccache?
Yea, it's a long standing best practice that an LXC or docker container or buildtools tarball must be used to get reproducible builds in CI systems and on developers machines with random Linux distros and versions. Host contamination issues have reduced considerably in recent years, thanks for all fixes, but I still can't recommend mixing builds from different Linux distros when developing yocto based products.
ccache version is far from the only host tool which has incompatibilities between versions and distros.
Cheers,
-Mikko
Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache
Quoted 7 lines · On Fri, 2021-01-08 at 07:47 +0000, [email protected] wrote:
Quoted 5 lines · Joshua Watt
We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.
Alternatively, I think it's possible to roll your own buildtools that has ccache?
Quoted 7 lines
Yea, it's a long standing best practice that an LXC or docker container or buildtools tarball must be used to get reproducible builds in CI systems and on developers machines with random Linux distros and versions. Host contamination issues have reduced considerably in recent years, thanks for all fixes, but I still can't recommend mixing builds from different Linux distros when developing yocto based products.
Whilst this has been an issue I am quite proud of the work we've done with the autobuilder to detect and resolve reproducibility issues which has now been extended to world builds, i.e. apart from a small subset of known issues (~65 packages), OE-Core is fully reproducible over all out tested distros.
Whilst containers are a way of avoiding such issues, I think some of the recent issues would even have caused differences between containers (though kernel config or the geo of nameservers).
I appreciate that it hasn't been extended beyond OE-Core as yet but think it does show what is possible and that there aren't that many issues, certainly the core and tools are in a strong position.
Cheers,
Richard