Skip to contentSkip to conversation

ccache: Fixes for 4.1

PATCH In review Robert Yang 4 participants 14 messages Jan 6, 2021 – Jan 8, 2021

lore
Share & tools
lore
#0 lore raw

cover ccache: Fixes for 4.1

The following changes since commit 23cb39a5fa2a55681e7bc2605f435135cec9173b:

diffstat: point the license checksum at the license (2021-01-05 13:48:07 +0000)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/ccache
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/ccache

Robert Yang (3):
ccache: Extend to nativesdk
buildtools-tarball: Add nativesdk-ccache
ccache.bbclass: Set CCACHE_TEMPDIR

 meta/classes/ccache.bbclass                  | 4 ++++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 meta/recipes-devtools/ccache/ccache_4.1.bb   | 2 +-
 3 files changed, 6 insertions(+), 1 deletion(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/3 ccache: Extend to nativesdk

Now we have to use host's ccache as described by:
f5b29367af ccache.bbclass: use ccache from host distribution

So extend it to nativesdk and will add it to buildtools-tarball.

---
 meta/recipes-devtools/ccache/ccache_4.1.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/ccache/ccache_4.1.bb +1 -1
index 96254a38753..1205f4996cb 100644
@@ -23,4 +23,4 @@ inherit cmake
 
 PATCHTOOL = "patch"
 
-BBCLASSEXTEND = "native"
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/3 buildtools-tarball: Add nativesdk-ccache

In reply to Robert Yang

Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.

---
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
index 9da81d55235..2ffdd7c7253 100644
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
     nativesdk-rpcsvc-proto \
     nativesdk-patch \
     nativesdk-mtools \
+    nativesdk-ccache \
     "
 
 MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"
Signature · 1 line
2.17.1
#13 lore raw

PATCH 3/3 ccache.bbclass: Set CCACHE_TEMPDIR

In reply to Robert Yang

Fixed when build with buildtools-tarball:

$ bitbake linux-libc-headers
  HOSTCC  arch/x86/tools/relocs_common.o
  ccache: error: Failed to create directory /run/user/0/ccache-tmp: Permission denied
---
 meta/classes/ccache.bbclass | 4 ++++
 1 file changed, 4 insertions(+)
meta/classes/ccache.bbclass +4 -0
index 11a3f1cb52e..f00fafc292a 100644
@@ -35,6 +35,10 @@ export CCACHE_CONFIGPATH ?= "${COREBASE}/meta/conf/ccache.conf"
 
 export CCACHE_DIR ?= "${CCACHE_TOP_DIR}/${MULTIMACH_TARGET_SYS}/${PN}"
 
+# Fixed errors:
+# ccache: error: Failed to create directory /run/user/0/ccache-tmp: Permission denied
+export CCACHE_TEMPDIR ?= "${CCACHE_DIR}/tmp"
+
 # We need to stop ccache considering the current directory or the
 # debug-prefix-map target directory to be significant when calculating
 # its hash. Without this the cache would be invalidated every time
Signature · 1 line
2.17.1
#3 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

In reply to Robert Yang
Quoted 17 lines · Robert Yang

Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.

---
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +

 1 file changed, 1 insertion(+)
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
index 9da81d55235..2ffdd7c7253 100644
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
      nativesdk-rpcsvc-proto \
      nativesdk-patch \
      nativesdk-mtools \
+    nativesdk-ccache \
Quoted 1 line

     "

My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it but its not essential, required or helps much in standard builds (which would reuse from sstate if built previously).

Cheers,

Richard

#4 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

Hi RP,

Quoted 9 lines · Richard Purdie
Quoted 8 lines · Robert Yang

Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.

---
  meta/recipes-core/meta/buildtools-tarball.bb | 1 +
  1 file changed, 1 insertion(+)
Quoted 13 lines

meta/recipes-core/meta/buildtools-tarball.bb TOOLCHAIN_HOST_TASK ?= "\ · line 33

Quoted 10 lines
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
index 9da81d55235..2ffdd7c7253 100644
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
      nativesdk-rpcsvc-proto \
      nativesdk-patch \
      nativesdk-mtools \
+    nativesdk-ccache \
      "

My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it

The problem is that user installed ccache may not work with our recipes, for example, we found the following recipes failed to build with ccache on different hosts:

cdrtools-native
fribidi
rocksdb

So have a fixed version of ccache can make sure the recipes work well, it also helps us to reproduce the errors easily when user reported a bug.

// Robert

Quoted 7 lines

but its not essential, required or helps much in standard builds (which would reuse from sstate if built previously).

Cheers,

Richard

#5 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

Quoted 31 lines · Robert Yang

Hi RP,

Quoted 9 lines · Richard Purdie
Quoted 8 lines · Robert Yang

Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.

---
  meta/recipes-core/meta/buildtools-tarball.bb | 1 +

  1 file changed, 1 insertion(+)
Quoted 13 lines
Quoted 10 lines
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
index 9da81d55235..2ffdd7c7253 100644
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
       nativesdk-rpcsvc-proto \
       nativesdk-patch \
       nativesdk-mtools \
+    nativesdk-ccache \

      "

My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it

The problem is that user installed ccache may not work with our recipes, for example, we found the following recipes failed to build with ccache on different hosts:

Quoted 7 lines

cdrtools-native
fribidi
rocksdb

So have a fixed version of ccache can make sure the recipes work well, it also helps us to reproduce the errors easily when user reported a bug.

I really do not want ccache in buildtools. If this is such a problem, we need to work out how to resolve the circular dependency issues so ccache-native can work again.

Cheers,

Richard

#6 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

Hi RP,

Quoted 37 lines · Richard Purdie
Quoted 36 lines · Robert Yang

Hi RP,

Quoted 9 lines · Richard Purdie
Quoted 8 lines · Robert Yang

Add it to buildtools-tarball so that there will be a unify version of ccache, which can help avoid various compile errors.

---
   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
   1 file changed, 1 insertion(+)
Quoted 13 lines

meta/recipes-core/meta/buildtools-tarball.bb TOOLCHAIN_HOST_TASK ?= "\ · line 33

Quoted 10 lines
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
index 9da81d55235..2ffdd7c7253 100644
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
       nativesdk-rpcsvc-proto \
       nativesdk-patch \
       nativesdk-mtools \
+    nativesdk-ccache \
       "

My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it

The problem is that user installed ccache may not work with our recipes, for example, we found the following recipes failed to build with ccache on different hosts:

cdrtools-native
fribidi
rocksdb

Quoted 6 lines
Quoted 2 lines

So have a fixed version of ccache can make sure the recipes work well, it also helps us to reproduce the errors easily when user reported a bug.

I really do not want ccache in buildtools. If this is such a problem, we need to work out how to resolve the circular dependency issues so ccache-native can work again.

There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build

* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.

It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native.

Host's ccache isn't reliable from the history.

// Robert

Quoted 7 lines

Cheers,

Richard

#7 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

Quoted 60 lines · Robert Yang

Hi RP,

Quoted 6 lines · Richard Purdie
Quoted 5 lines · Robert Yang

Hi RP,

Quoted 2 lines · Richard Purdie
Quoted 1 line · Robert Yang

Add it to buildtools-tarball so that there will be a unify version of

ccache,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 8 lines

which can help avoid various compile errors.

---
   meta/recipes-core/meta/buildtools-tarball.bb | 1 +
   1 file changed, 1 insertion(+)
(quoted diff) +0 -0

b/meta/recipes-core/meta/buildtools-tarball.bb

Quoted 14 lines
Quoted 14 lines
Quoted 12 lines

meta/recipes-core/meta/buildtools-tarball.bb TOOLCHAIN_HOST_TASK ?= "\ · line 33

Quoted 9 lines

index 9da81d55235..2ffdd7c7253 100644

meta/recipes-core/meta/buildtools-tarball.bb +1 -0
@@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
       nativesdk-rpcsvc-proto \
       nativesdk-patch \
       nativesdk-mtools \
+    nativesdk-ccache \
       "

My view is we should not start to ship ccache with buildtools by default. Its something that the user can install if they want/need it

Quoted 1 line
Quoted 1 line

The problem is that user installed ccache may not work with our

recipes, for

Quoted 1 line
Quoted 1 line

example, we found the following recipes failed to build with ccache on

different

Quoted 8 lines
Quoted 8 lines

hosts:

cdrtools-native
fribidi
rocksdb

So have a fixed version of ccache can make sure the recipes work well,

it also

Quoted 5 lines
Quoted 1 line

helps us to reproduce the errors easily when user reported a bug.

I really do not want ccache in buildtools. If this is such a problem, we need to work out how to resolve the circular dependency issues so ccache-native can work again.

There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires
more time to build, so it might be a bottleneck for the build

* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.

Quoted 8 lines

It seems that there isn't a good way to fix the problem, or we have to disable
ccache for native recipes, only make other recipes (cross, target and nativesdk)
use ccache-native.

Host's ccache isn't reliable from the history.

We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.

Alternatively, I think it's possible to roll your own buildtools that has ccache?

Quoted 13 lines

// Robert

Quoted 7 lines

Cheers,

Richard

#11 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

In reply to Robert Yang
Quoted 4 lines · Robert Yang

There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build

Quoted 8 lines

* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.

It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native.

Host's ccache isn't reliable from the history.

Thanks for the analysis. I think I'd be inclined to disable ccache for native recipes then. If anyone does want to use it for native recipes, they can build a buildtools tarball of their own that contains it but I don't want to default to that.

Cheers,

Richard

#12 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

Quoted 12 lines · Richard Purdie
Quoted 11 lines · Robert Yang

There are two problems to use cache-native now:
* ccache-native depends on cmake-native and zstd-native which makes it requires more time to build, so it might be a bottleneck for the build

* We need disable ccache-native for about 20 native recipes to resolve the circular dependency.

It seems that there isn't a good way to fix the problem, or we have to disable ccache for native recipes, only make other recipes (cross, target and nativesdk) use ccache-native.

Quoted 6 lines
Quoted 1 line

Host's ccache isn't reliable from the history.

Thanks for the analysis. I think I'd be inclined to disable ccache for native recipes then. If anyone does want to use it for native recipes, they can build a buildtools tarball of their own that contains it but I don't want to default to that.

Yes, make sense, I will send a patch to disable ccache for native recipes and enable ccache-native for other recipes.

// Robert

Quoted 5 lines

Cheers,

Richard

#8 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

In reply to Joshua Watt
Quoted 74 lines · Joshua Watt

On Thu, Jan 7, 2021, 5:00 AM Robert Yang <[email protected] <mailto:[email protected]>> wrote:

    Hi RP,

    On 1/7/21 6:40 PM, Richard Purdie wrote:
     > On Thu, 2021-01-07 at 10:44 +0800, Robert Yang wrote:
     >> Hi RP,
     >>
     >> On 1/6/21 9:56 PM, Richard Purdie wrote:
     >>> On Wed, 2021-01-06 at 04:09 -0800, Robert Yang wrote:
     >>>> Add it to buildtools-tarball so that there will be a unify version of
    ccache,
     >>>> which can help avoid various compile errors.
     >>>>
     >>>> Signed-off-by: Robert Yang <[email protected]
    <mailto:[email protected]>>
     >>>> ---
     >>>>    meta/recipes-core/meta/buildtools-tarball.bb
    <http://buildtools-tarball.bb> | 1 +
     >>>>    1 file changed, 1 insertion(+)
     >>>>
     >>>> diff --git a/meta/recipes-core/meta/buildtools-tarball.bb
    <http://buildtools-tarball.bb>
    b/meta/recipes-core/meta/buildtools-tarball.bb <http://buildtools-tarball.bb>
     >>>> index 9da81d55235..2ffdd7c7253 100644
     >>>> --- a/meta/recipes-core/meta/buildtools-tarball.bb
    <http://buildtools-tarball.bb>
     >>>> +++ b/meta/recipes-core/meta/buildtools-tarball.bb
    <http://buildtools-tarball.bb>
     >>>> @@ -29,6 +29,7 @@ TOOLCHAIN_HOST_TASK ?= "\
     >>>>        nativesdk-rpcsvc-proto \
     >>>>        nativesdk-patch \
     >>>>        nativesdk-mtools \
     >>>> +    nativesdk-ccache \
     >>>>        "
     >>>
     >>> My view is we should not start to ship ccache with buildtools by
     >>> default. Its something that the user can install if they want/need it
     >>
     >>
     >> The problem is that user installed ccache may not work with our recipes, for
     >> example, we found the following recipes failed to build with ccache on
    different
     >> hosts:
     >>
     >> cdrtools-native
     >> fribidi
     >> rocksdb
     >>
     >>
     >> So have a fixed version of ccache can make sure the recipes work well,
    it also
     >> helps us to reproduce the errors easily when user reported a bug.
     >
     > I really do not want ccache in buildtools. If this is such a problem,
     > we need to work out how to resolve the circular dependency issues so
     > ccache-native can work again.

    There are two problems to use cache-native now:
    * ccache-native depends on cmake-native and zstd-native which makes it requires
        more time to build, so it might be a bottleneck for the build

    * We need disable ccache-native for about 20 native recipes to resolve the
    circular dependency.

    It seems that there isn't a good way to fix the problem, or we have to disable
    ccache for native recipes, only make other recipes (cross, target and nativesdk)
    use ccache-native.

    Host's ccache isn't reliable from the history.
Quoted 3 lines

We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.

Alternatively, I think it's possible to roll your own buildtools that has ccache?

Yes, thanks, it's a good suggestion.

// Robert

Quoted 14 lines

    // Robert

     >
     > Cheers,
     >
     > Richard
     >
     >
     >
#9 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

In reply to Joshua Watt
Quoted 5 lines · Joshua Watt

We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.

Alternatively, I think it's possible to roll your own buildtools that has ccache?

Yea, it's a long standing best practice that an LXC or docker container or buildtools tarball must be used to get reproducible builds in CI systems and on developers machines with random Linux distros and versions. Host contamination issues have reduced considerably in recent years, thanks for all fixes, but I still can't recommend mixing builds from different Linux distros when developing yocto based products.

ccache version is far from the only host tool which has incompatibilities between versions and distros.

Cheers,

-Mikko

#10 lore raw

Re: [OE-core] [PATCH 2/3] buildtools-tarball: Add nativesdk-ccache

Quoted 7 lines · On Fri, 2021-01-08 at 07:47 +0000, [email protected] wrote:
Quoted 5 lines · Joshua Watt

We had the same problem with icecream and solved it by building in a docker container with the version of ice cream that we wanted FWIW.

Alternatively, I think it's possible to roll your own buildtools that has ccache?

Quoted 7 lines

Yea, it's a long standing best practice that an LXC or docker container or buildtools tarball must be used to get reproducible builds in CI systems and on developers machines with random Linux distros and versions. Host contamination issues have reduced considerably in recent years, thanks for all fixes, but I still can't recommend mixing builds from different Linux distros when developing yocto based products.

Whilst this has been an issue I am quite proud of the work we've done with the autobuilder to detect and resolve reproducibility issues which has now been extended to world builds, i.e. apart from a small subset of known issues (~65 packages), OE-Core is fully reproducible over all out tested distros.

Whilst containers are a way of avoiding such issues, I think some of the recent issues would even have caused differences between containers (though kernel config or the geo of nameservers).

I appreciate that it hasn't been extended beyond OE-Core as yet but think it does show what is possible and that there aren't that many issues, certainly the core and tools are in a strong position.

Cheers,

Richard