Skip to contentSkip to conversation

Integrate ccache-native

PATCH Superseded [email protected] 3 participants 5 messages Jun 17, 2011 – Jun 22, 2011

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Integrate ccache-native

From: Wenzong Fan <[email protected]>

1) Add ccache as a native tool, for getting it built automatically make it as the dependency of bzip2; Also, the ccache will be included by SDK images;

2) Set CCACHE on a per recipe basis and add task 'mkccachedir' to create the CCACHE_DIR while start a package building;

3) Remove the duplicate 'ccache.inc' from 'meta/class/'.

The following changes since commit 2163461ec94528ecf046a04edc5db3d2dd3a6b8b:
Tom Zanussi (1):
systemtap: remove non-core COMPATIBLE_MACHINES

are available in the git repository at:

git://git.pokylinux.org/poky-contrib wenzong/ccache
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=wenzong/ccache

Wenzong Fan (1):
ccache: Integrate ccache-native

 meta/classes/base.bbclass                    |    1 +
 meta/classes/ccache.bbclass                  |   21 +++++++++++++++++++++
 meta/classes/ccache.inc                      |   11 -----------
 meta/conf/bitbake.conf                       |    1 +
 meta/recipes-core/tasks/task-core-sdk.bb     |    1 +
 meta/recipes-devtools/ccache/ccache.inc      |   16 ++++++++++++++++
 meta/recipes-devtools/ccache/ccache_3.1.5.bb |    8 ++++++++
 meta/recipes-extended/bzip2/bzip2_1.0.6.bb   |    2 ++
 8 files changed, 50 insertions(+), 11 deletions(-)
 create mode 100644 meta/classes/ccache.bbclass
 delete mode 100644 meta/classes/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache_3.1.5.bb
#1 lore raw

PATCH 1/1 ccache: Integrate ccache-native

From: Wenzong Fan <[email protected]>

1) Add ccache as a native tool, for getting it built automatically make it as the dependency of bzip2; Also, the ccache will be included by SDK images;

2) Set CCACHE on a per recipe basis and add task 'mkccachedir' to create the CCACHE_DIR while start a package building;

3) Remove the duplicate 'ccache.inc' from 'meta/class/'.

---
 meta/classes/base.bbclass                    |    1 +
 meta/classes/ccache.bbclass                  |   21 +++++++++++++++++++++
 meta/classes/ccache.inc                      |   11 -----------
 meta/conf/bitbake.conf                       |    1 +
 meta/recipes-core/tasks/task-core-sdk.bb     |    1 +
 meta/recipes-devtools/ccache/ccache.inc      |   16 ++++++++++++++++
 meta/recipes-devtools/ccache/ccache_3.1.5.bb |    8 ++++++++
 meta/recipes-extended/bzip2/bzip2_1.0.6.bb   |    2 ++
 8 files changed, 50 insertions(+), 11 deletions(-)
 create mode 100644 meta/classes/ccache.bbclass
 delete mode 100644 meta/classes/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache_3.1.5.bb
meta/classes/base.bbclass +1 -0
index 119b052..a329b4e 100644
@@ -2,6 +2,7 @@ BB_DEFAULT_TASK ?= "build"
 
 inherit patch
 inherit staging
+inherit ccache
 
 inherit mirrors
 inherit utils
meta/classes/ccache.bbclass new file +21 -0
new file mode 100644
index 0000000..be37c18
@@ -0,0 +1,21 @@
+# Make ${CCACHE_DIR} if it is not existing
+
+python ccache_do_mkccachedir(){
+	import os
+
+	ccache = bb.data.getVar('CCACHE', d, True)
+	ccache_dir = bb.data.getVar('CCACHE_DIR', d, True)
+	if len(ccache) == 0 or len(ccache_dir) == 0:
+		return
+
+	try:
+		if not os.path.isdir(ccache_dir):
+			bb.note("Creating " + ccache_dir)
+			os.makedirs(ccache_dir)
+	except bb.fetch2.BBFetchException, e:
+		raise bb.build.FuncFailed(e)
+}
+
+addtask mkccachedir before do_configure
+
+EXPORT_FUNCTIONS do_mkccachedir
meta/classes/ccache.inc deleted +0 -11
deleted file mode 100644
index d830a1b..0000000
@@ -1,11 +0,0 @@
-# Make ccache use a TMPDIR specific ccache directory if using the crosscompiler,
-# since it isn't likely to be useful with any other toolchain than the one we just
-# built, and would otherwise push more useful things out of the default cache.
-
-CCACHE_DIR_TARGET = "${TMPDIR}/ccache"
-
-python () {
-    if not bb.data.inherits_class('native', d) and not bb.data.inherits_class('cross', d):
-        bb.data.setVar('CCACHE_DIR', '${CCACHE_DIR_TARGET}', d)
-        bb.data.setVarFlag('CCACHE_DIR', 'export', '1', d)
-}
meta/conf/bitbake.conf +1 -0
index 6d8a674..64b3651 100644
@@ -391,6 +391,7 @@ export PATH
 CCACHE = "${@bb.which(bb.data.getVar('PATH', d, 1), 'ccache') and 'ccache '}"
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
 
+export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/${PN}"
 export CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
 export CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
 export F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
meta/recipes-core/tasks/task-core-sdk.bb +1 -0
index 4aee0c2..52fdd75 100644
@@ -25,6 +25,7 @@ RDEPENDS_task-core-sdk = "\
     coreutils \
     cpp \
     cpp-symlinks \
+    ccache \
     diffutils \
     gcc \
     gcc-symlinks \
meta/recipes-devtools/ccache/ccache.inc new file +16 -0
new file mode 100644
index 0000000..e9f7344
@@ -0,0 +1,16 @@
+SUMMARY = "a fast C/C++ compiler cache"
+DESCRIPTION = "ccache is a compiler cache. It speeds up recompilation \
+by caching the result of previous compilations and detecting when the \
+same compilation is being done again. Supported languages are C, C\+\+, \
+Objective-C and Objective-C++."
+HOMEPAGE = "http://ccache.samba.org"
+SECTION = "devel"
+LICENSE = "GPLv3+"
+
+SRC_URI = "http://samba.org/ftp/ccache/ccache-${PV}.tar.gz"
+
+inherit autotools
+
+BBCLASSEXTEND = "native"
+
+TARGET_CC_ARCH += "${LDFLAGS}"
meta/recipes-devtools/ccache/ccache_3.1.5.bb new file +8 -0
new file mode 100644
index 0000000..9a967b2
@@ -0,0 +1,8 @@
+require ccache.inc
+
+PR = "r0"
+LICENSE = "GPLv3+"
+LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=80e92ec45d4fca91f127864fb9e5d932"
+
+SRC_URI[md5sum] = "b1a9684828eae68382d6afc98ce80d24"
+SRC_URI[sha256sum] = "54afc35c672ce451e04a478cfc0eb74c1ba184e27ef24881206602aa0eb94d63"
meta/recipes-extended/bzip2/bzip2_1.0.6.bb +2 -0
index f5b9ac1..4d445a5 100644
@@ -6,6 +6,8 @@ HOMEPAGE = "http://www.bzip.org/"
 SECTION = "console/utils"
 LICENSE = "bzip2"
 LIC_FILES_CHKSUM = "file://LICENSE;beginline=8;endline=37;md5=40d9d1eb05736d1bfc86cfdd9106e6b2"
+
+DEPENDS = "ccache-native"
 PR = "r3"
 
 SRC_URI = "http://www.bzip.org/${PV}/${BPN}-${PV}.tar.gz \
Signature · 1 line
1.7.0.4
#3 lore raw
In reply to [email protected]

Just looking for a status review on this item.

Does this implement the task:

Performance Improvements - CCACHE per recipe

c) Set CCACHE on a per recipe basis. need to figure out whether ccache data can be shared and under what circumstances.

?

I just need to update them with the status, if this is the full item, I'll make sure it gets reviewed when Richard starts integrating patches again.

Thanks!

--Mark

Quoted 42 lines · On 6/17/11 4:19 AM, [email protected] wrote:

From: Wenzong Fan <[email protected]>

1) Add ccache as a native tool, for getting it built automatically make it as the dependency of bzip2; Also, the ccache will be included by SDK images;

2) Set CCACHE on a per recipe basis and add task 'mkccachedir' to create the CCACHE_DIR while start a package building;

3) Remove the duplicate 'ccache.inc' from 'meta/class/'.

The following changes since commit 2163461ec94528ecf046a04edc5db3d2dd3a6b8b:
Tom Zanussi (1):
systemtap: remove non-core COMPATIBLE_MACHINES

are available in the git repository at:

git://git.pokylinux.org/poky-contrib wenzong/ccache
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=wenzong/ccache

Wenzong Fan (1):
ccache: Integrate ccache-native

 meta/classes/base.bbclass                    |    1 +
 meta/classes/ccache.bbclass                  |   21 +++++++++++++++++++++
 meta/classes/ccache.inc                      |   11 -----------
 meta/conf/bitbake.conf                       |    1 +
 meta/recipes-core/tasks/task-core-sdk.bb     |    1 +
 meta/recipes-devtools/ccache/ccache.inc      |   16 ++++++++++++++++
 meta/recipes-devtools/ccache/ccache_3.1.5.bb |    8 ++++++++
 meta/recipes-extended/bzip2/bzip2_1.0.6.bb   |    2 ++
 8 files changed, 50 insertions(+), 11 deletions(-)
 create mode 100644 meta/classes/ccache.bbclass
 delete mode 100644 meta/classes/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache_3.1.5.bb

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#4 lore raw
Quoted 12 lines · Mark Hatle

Just looking for a status review on this item.

Does this implement the task:

Performance Improvements - CCACHE per recipe

c) Set CCACHE on a per recipe basis. need to figure out whether ccache data can be shared and under what circumstances.

?

Correct, this task also be included in this review.

Thanks
Wenzong

Quoted 60 lines

I just need to update them with the status, if this is the full item, I'll make sure it gets reviewed when Richard starts integrating patches again.

Thanks!

--Mark

Quoted 38 lines · On 6/17/11 4:19 AM, [email protected] wrote:

From: Wenzong Fan<[email protected]>

1) Add ccache as a native tool, for getting it built automatically make it as the dependency of bzip2; Also, the ccache will be included by SDK images;

2) Set CCACHE on a per recipe basis and add task 'mkccachedir' to create the CCACHE_DIR while start a package building;

3) Remove the duplicate 'ccache.inc' from 'meta/class/'.

The following changes since commit 2163461ec94528ecf046a04edc5db3d2dd3a6b8b:
Tom Zanussi (1):
systemtap: remove non-core COMPATIBLE_MACHINES

are available in the git repository at:

git://git.pokylinux.org/poky-contrib wenzong/ccache
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=wenzong/ccache

Wenzong Fan (1):
ccache: Integrate ccache-native

  meta/classes/base.bbclass                    |    1 +
  meta/classes/ccache.bbclass                  |   21 +++++++++++++++++++++
  meta/classes/ccache.inc                      |   11 -----------
  meta/conf/bitbake.conf                       |    1 +
  meta/recipes-core/tasks/task-core-sdk.bb     |    1 +
  meta/recipes-devtools/ccache/ccache.inc      |   16 ++++++++++++++++
  meta/recipes-devtools/ccache/ccache_3.1.5.bb |    8 ++++++++
  meta/recipes-extended/bzip2/bzip2_1.0.6.bb   |    2 ++
  8 files changed, 50 insertions(+), 11 deletions(-)

create mode 100644 meta/classes/ccache.bbclass
delete mode 100644 meta/classes/ccache.inc
create mode 100644 meta/recipes-devtools/ccache/ccache.inc create mode 100644 meta/recipes-devtools/ccache/ccache_3.1.5.bb

Quoted 5 lines

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#2 lore raw

Re: [PATCH 1/1] ccache: Integrate ccache-native

In reply to [email protected]
Quoted 56 lines · On Fri, 2011-06-17 at 17:19 +0800, [email protected] wrote:

From: Wenzong Fan <[email protected]>

1) Add ccache as a native tool, for getting it built automatically make it as the dependency of bzip2; Also, the ccache will be included by SDK images;

2) Set CCACHE on a per recipe basis and add task 'mkccachedir' to create the CCACHE_DIR while start a package building;

3) Remove the duplicate 'ccache.inc' from 'meta/class/'.

---
 meta/classes/base.bbclass                    |    1 +
 meta/classes/ccache.bbclass                  |   21 +++++++++++++++++++++
 meta/classes/ccache.inc                      |   11 -----------
 meta/conf/bitbake.conf                       |    1 +
 meta/recipes-core/tasks/task-core-sdk.bb     |    1 +
 meta/recipes-devtools/ccache/ccache.inc      |   16 ++++++++++++++++
 meta/recipes-devtools/ccache/ccache_3.1.5.bb |    8 ++++++++
 meta/recipes-extended/bzip2/bzip2_1.0.6.bb   |    2 ++
 8 files changed, 50 insertions(+), 11 deletions(-)
 create mode 100644 meta/classes/ccache.bbclass
 delete mode 100644 meta/classes/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache.inc
 create mode 100644 meta/recipes-devtools/ccache/ccache_3.1.5.bb
meta/classes/base.bbclass +1 -0
index 119b052..a329b4e 100644
@@ -2,6 +2,7 @@ BB_DEFAULT_TASK ?= "build"
 
 inherit patch
 inherit staging
+inherit ccache
 
 inherit mirrors
 inherit utils
meta/classes/ccache.bbclass new file +11 -0
new file mode 100644
index 0000000..be37c18
@@ -0,0 +1,21 @@
+# Make ${CCACHE_DIR} if it is not existing
+
+python ccache_do_mkccachedir(){
+	import os
+
+	ccache = bb.data.getVar('CCACHE', d, True)
+	ccache_dir = bb.data.getVar('CCACHE_DIR', d, True)
+	if len(ccache) == 0 or len(ccache_dir) == 0:
+		return
+
+	try:

meta/classes/ccache.bbclass line 21

Quoted 10 lines
meta/classes/ccache.bbclass new file +10 -0
+		if not os.path.isdir(ccache_dir):
+			bb.note("Creating " + ccache_dir)
+			os.makedirs(ccache_dir)
+	except bb.fetch2.BBFetchException, e:
+		raise bb.build.FuncFailed(e)
+}
+
+addtask mkccachedir before do_configure
+
+EXPORT_FUNCTIONS do_mkccachedir

You can do this in a much simpler way by just adding:

do_configure[dirs] += "${CCACHE_DIR}"

Quoted 22 lines
meta/classes/ccache.inc deleted +0 -11
deleted file mode 100644
index d830a1b..0000000
@@ -1,11 +0,0 @@
-# Make ccache use a TMPDIR specific ccache directory if using the crosscompiler,
-# since it isn't likely to be useful with any other toolchain than the one we just
-# built, and would otherwise push more useful things out of the default cache.
-
-CCACHE_DIR_TARGET = "${TMPDIR}/ccache"
-
-python () {
-    if not bb.data.inherits_class('native', d) and not bb.data.inherits_class('cross', d):
-        bb.data.setVar('CCACHE_DIR', '${CCACHE_DIR_TARGET}', d)
-        bb.data.setVarFlag('CCACHE_DIR', 'export', '1', d)
-}
meta/conf/bitbake.conf +0 -0
index 6d8a674..64b3651 100644

meta/conf/bitbake.conf export PATH · line 394

Quoted 4 lines
meta/conf/bitbake.conf +1 -0
@@ -391,6 +391,7 @@ export PATH
 CCACHE = "${@bb.which(bb.data.getVar('PATH', d, 1), 'ccache') and 'ccache '}"
 TOOLCHAIN_OPTIONS = " --sysroot=${STAGING_DIR_TARGET}"
 
+export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/${PN}"

Should this be HOST_SYS instead of TARGET_SYS?

meta/recipes-extended/bzip2/bzip2_1.0.6.bb

Quoted 53 lines
meta/conf/bitbake.conf +0 -0
 export CC = "${CCACHE}${HOST_PREFIX}gcc ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
 export CXX = "${CCACHE}${HOST_PREFIX}g++ ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
 export F77 = "${CCACHE}${HOST_PREFIX}g77 ${HOST_CC_ARCH}${TOOLCHAIN_OPTIONS}"
meta/recipes-core/tasks/task-core-sdk.bb +1 -0
index 4aee0c2..52fdd75 100644
@@ -25,6 +25,7 @@ RDEPENDS_task-core-sdk = "\
     coreutils \
     cpp \
     cpp-symlinks \
+    ccache \
     diffutils \
     gcc \
     gcc-symlinks \
meta/recipes-devtools/ccache/ccache.inc new file +16 -0
new file mode 100644
index 0000000..e9f7344
@@ -0,0 +1,16 @@
+SUMMARY = "a fast C/C++ compiler cache"
+DESCRIPTION = "ccache is a compiler cache. It speeds up recompilation \
+by caching the result of previous compilations and detecting when the \
+same compilation is being done again. Supported languages are C, C\+\+, \
+Objective-C and Objective-C++."
+HOMEPAGE = "http://ccache.samba.org"
+SECTION = "devel"
+LICENSE = "GPLv3+"
+
+SRC_URI = "http://samba.org/ftp/ccache/ccache-${PV}.tar.gz"
+
+inherit autotools
+
+BBCLASSEXTEND = "native"
+
+TARGET_CC_ARCH += "${LDFLAGS}"
meta/recipes-devtools/ccache/ccache_3.1.5.bb new file +8 -0
new file mode 100644
index 0000000..9a967b2
@@ -0,0 +1,8 @@
+require ccache.inc
+
+PR = "r0"
+LICENSE = "GPLv3+"
+LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=80e92ec45d4fca91f127864fb9e5d932"
+
+SRC_URI[md5sum] = "b1a9684828eae68382d6afc98ce80d24"
+SRC_URI[sha256sum] = "54afc35c672ce451e04a478cfc0eb74c1ba184e27ef24881206602aa0eb94d63"
meta/recipes-extended/bzip2/bzip2_1.0.6.bb +0 -0
index f5b9ac1..4d445a5 100644

Please split this patch into a couple of changes, one of which just adds the ccache-native recipe.

meta/recipes-extended/bzip2/bzip2_1.0.6.bb HOMEPAGE = "http://www.bzip.org/" · line 13

Quoted 11 lines
meta/recipes-extended/bzip2/bzip2_1.0.6.bb +2 -0
@@ -6,6 +6,8 @@ HOMEPAGE = "http://www.bzip.org/"
 SECTION = "console/utils"
 LICENSE = "bzip2"
 LIC_FILES_CHKSUM = "file://LICENSE;beginline=8;endline=37;md5=40d9d1eb05736d1bfc86cfdd9106e6b2"
+
+DEPENDS = "ccache-native"
 PR = "r3"
 
 SRC_URI = "http://www.bzip.org/${PV}/${BPN}-${PV}.tar.gz \

I'm not sure how we're going to trigger the generation of ccache-native and I can understand the attraction of doing this but it isn't acceptable unfortunately. Lets just drop this component of the patch for now.

Cheers,

Richard