ccache dependencies

PATCH New Thomas De Schampheleire 3 participants 10 messages Aug 2, 2013 – Aug 7, 2013 Kbuild

Share & tools

Thomas De Schampheleire

author

2013-08-02 07:33 UTC #0 lore raw

PATCH 1/1 ccache dependencies

As discussed, this series includes some changes related to ccache dependencies.
Important: see note in the second patch.
I included Samuel's patches here for convenience.

Best regards,
Thomas

Thomas De Schampheleire

author

2013-08-02 07:33 UTC #1 lore raw

PATCH 1/1 dependencies: build without ccache

This patch forces the plain host compiler to be used during the building of dependencies, without ccache as it is not yet built.

Signed-off-by Thomas De Schampheleire <[email protected]>

---
 support/dependencies/dependencies.mk |  2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

support/dependencies/dependencies.mk +2 -0

@@ -25,6 +25,8 @@ core-dependencies:
 		DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \
 		$(TOPDIR)/support/dependencies/dependencies.sh
 
+dependencies: HOSTCC=$(HOSTCC_NOCCACHE)
+dependencies: HOSTCXX=$(HOSTCXX_NOCCACHE)
 dependencies: core-dependencies $(DEPENDENCIES_HOST_PREREQ)
 
 dependencies-source:

Thomas De Schampheleire

author

2013-08-02 07:33 UTC #2 lore raw

PATCH 1/1 host-ccache: turn into a proper dependency

In reply to Thomas De Schampheleire

This patch moves the host-ccache build target from BASE_TARGETS in Makefile to an actual host prerequisite in support/dependencies. This causes host-ccache to be built as part of the dependencies, before any real package is built.
Since the dependencies are built without ccache anyway, there is no need to set HOST_CCACHE_CONF_ENV anymore.

Signed-off-by Thomas De Schampheleire <[email protected]>

---

 Makefile                             |  4 ----
 package/ccache/ccache.mk             |  6 ------
 support/dependencies/dependencies.mk |  4 ++++
 3 files changed, 4 insertions(+), 10 deletions(-)

Note: this in itself works, when the dependencies are built (regular 'make' for example, or explicit 'make dependencies'). However, if we want to supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants the following to work:

$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig

then I think something additional is necessary. In the above flow, the dependencies target is never used, so host-ccache doesn't get a chance to build.

Makefile +0 -4

@@ -222,10 +222,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig
 #
 ################################################################################
 
-ifeq ($(BR2_CCACHE),y)
-BASE_TARGETS += host-ccache
-endif
-
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
 BASE_TARGETS += toolchain-buildroot
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)

package/ccache/ccache.mk +0 -6

@@ -10,12 +10,6 @@ CCACHE_SOURCE  = ccache-$(CCACHE_VERSION
 CCACHE_LICENSE = GPLv3+, others
 CCACHE_LICENSE_FILES = LICENSE.txt GPL-3.0.txt
 
-# When ccache is being built for the host, ccache is not yet
-# available, so we have to use the special C compiler without the
-# cache.
-HOST_CCACHE_CONF_ENV = \
-	CC="$(HOSTCC_NOCCACHE)"
-
 # Force ccache to use its internal zlib. The problem is that without
 # this, ccache would link against the zlib of the build system, but we
 # might build and install a different version of zlib in $(O)/host

support/dependencies/dependencies.mk +4 -0

@@ -20,6 +20,10 @@ ifeq ($(BR2_STRIP_sstrip),y)
 DEPENDENCIES_HOST_PREREQ+=host-sstrip
 endif
 
+ifeq ($(BR2_CCACHE),y)
+DEPENDENCIES_HOST_PREREQ += host-ccache
+endif
+
 core-dependencies:
 	@HOSTCC="$(firstword $(HOSTCC))" MAKE="$(MAKE)" \
 		DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \

Samuel Martin

2013-08-02 08:27 UTC #3 lore raw

PATCH 1/1 host-ccache: turn into a proper dependency

Hi Thomas (both), all,

2013/8/2 Thomas De Schampheleire <[email protected]>:

Quoted 9 lines

This patch moves the host-ccache build target from BASE_TARGETS in Makefile to an actual host prerequisite in support/dependencies. This causes host-ccache to be built as part of the dependencies, before any real package is built.
Since the dependencies are built without ccache anyway, there is no need to set HOST_CCACHE_CONF_ENV anymore.
Suggested-by Thomas Petazzoni <[email protected]>
Signed-off-by Thomas De Schampheleire <[email protected]>

Acked-by Samuel Martin <[email protected]>

Quoted 14 lines

---
 Makefile                             |  4 ----
 package/ccache/ccache.mk             |  6 ------
 support/dependencies/dependencies.mk |  4 ++++
 3 files changed, 4 insertions(+), 10 deletions(-)
Note: this in itself works, when the dependencies are built (regular 'make' for example, or explicit 'make dependencies'). However, if we want to supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants the following to work:

Quoted 6 lines

$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
then I think something additional is necessary. In the above flow, the dependencies target is never used, so host-ccache doesn't get a chance to build.

Humm... I don't think so. This patch should be enough to fix the bug #5678. I mean, to run *-menuconfig, the package need to be extracted beforehand. I expect the 'dependencies' target is part of dependency list of the *-extract target (because of host-tar).
If it's done so, then it's ok tm.

Quoted 35 lines

Makefile +0 -4

@@ -222,10 +222,6 @@ GNU_HOST_NAME:=$(shell support/gnuconfig
 #
 ################################################################################
 
-ifeq ($(BR2_CCACHE),y)
-BASE_TARGETS += host-ccache
-endif
-
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)
 BASE_TARGETS += toolchain-buildroot
 else ifeq ($(BR2_TOOLCHAIN_EXTERNAL),y)

package/ccache/ccache.mk +0 -6

@@ -10,12 +10,6 @@ CCACHE_SOURCE  = ccache-$(CCACHE_VERSION
 CCACHE_LICENSE = GPLv3+, others
 CCACHE_LICENSE_FILES = LICENSE.txt GPL-3.0.txt
 
-# When ccache is being built for the host, ccache is not yet
-# available, so we have to use the special C compiler without the
-# cache.
-HOST_CCACHE_CONF_ENV = \
-       CC="$(HOSTCC_NOCCACHE)"
-
 # Force ccache to use its internal zlib. The problem is that without
 # this, ccache would link against the zlib of the build system, but we
 # might build and install a different version of zlib in $(O)/host

support/dependencies/dependencies.mk ifeq ($(BR2_STRIP_sstrip),y) · line 29

Quoted 10 lines

support/dependencies/dependencies.mk +4 -0

@@ -20,6 +20,10 @@ ifeq ($(BR2_STRIP_sstrip),y)
 DEPENDENCIES_HOST_PREREQ+=host-sstrip
 endif
 
+ifeq ($(BR2_CCACHE),y)
+DEPENDENCIES_HOST_PREREQ += host-ccache
+endif
+
 core-dependencies:
        @HOSTCC="$(firstword $(HOSTCC))" MAKE="$(MAKE)" \
                DL_TOOLS="$(sort $(DL_TOOLS_DEPENDENCIES))" \

Regards,

Signature · 1 line

Samuel

Thomas De Schampheleire

author

2013-08-02 08:45 UTC #4 lore raw

PATCH 1/1 host-ccache: turn into a proper dependency

Hi Samuel,

Quoted 31 lines · Samuel Martin

Hi Thomas (both), all,
2013/8/2 Thomas De Schampheleire <[email protected]>:
Quoted 9 lines
This patch moves the host-ccache build target from BASE_TARGETS in Makefile to an actual host prerequisite in support/dependencies. This causes host-ccache to be built as part of the dependencies, before any real package is built.
Since the dependencies are built without ccache anyway, there is no need to set HOST_CCACHE_CONF_ENV anymore.
Suggested-by Thomas Petazzoni <[email protected]>
Signed-off-by Thomas De Schampheleire <[email protected]>
Acked-by Samuel Martin <[email protected]>
Quoted 17 lines
---
 Makefile                             |  4 ----
 package/ccache/ccache.mk             |  6 ------
 support/dependencies/dependencies.mk |  4 ++++
 3 files changed, 4 insertions(+), 10 deletions(-)
Note: this in itself works, when the dependencies are built (regular 'make' for example, or explicit 'make dependencies'). However, if we want to supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants the following to work:
$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig

Quoted 8 lines

Quoted 3 lines
then I think something additional is necessary. In the above flow, the dependencies target is never used, so host-ccache doesn't get a chance to build.
Humm... I don't think so. This patch should be enough to fix the bug #5678. I mean, to run *-menuconfig, the package need to be extracted beforehand. I expect the 'dependencies' target is part of dependency list of the *-extract target (because of host-tar).
If it's done so, then it's ok tm.

I may be missing something, but I don't see a dependency of *-extract on 'dependencies'.
Also, I tried the scenario and it doesn't work. The compiler used to do the linux-menuconfig is with ccache, and it isn't built automatically, unless you specify 'dependencies' manually.

Samuel Martin

2013-08-02 08:54 UTC #5 lore raw

PATCH 1/1 host-ccache: turn into a proper dependency

2013/8/2 Thomas De Schampheleire <[email protected]>:

Quoted 37 lines

Hi Samuel,
Quoted 34 lines · Samuel Martin
Hi Thomas (both), all,
2013/8/2 Thomas De Schampheleire <[email protected]>:
Quoted 9 lines
This patch moves the host-ccache build target from BASE_TARGETS in Makefile to an actual host prerequisite in support/dependencies. This causes host-ccache to be built as part of the dependencies, before any real package is built.
Since the dependencies are built without ccache anyway, there is no need to set HOST_CCACHE_CONF_ENV anymore.
Suggested-by Thomas Petazzoni <[email protected]>
Signed-off-by Thomas De Schampheleire <[email protected]>
Acked-by Samuel Martin <[email protected]>
Quoted 20 lines
---
 Makefile                             |  4 ----
 package/ccache/ccache.mk             |  6 ------
 support/dependencies/dependencies.mk |  4 ++++
 3 files changed, 4 insertions(+), 10 deletions(-)
Note: this in itself works, when the dependencies are built (regular 'make' for example, or explicit 'make dependencies'). However, if we want to supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants the following to work:
$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
then I think something additional is necessary. In the above flow, the dependencies target is never used, so host-ccache doesn't get a chance to build.

Quoted 8 lines

Quoted 5 lines
Humm... I don't think so. This patch should be enough to fix the bug #5678. I mean, to run *-menuconfig, the package need to be extracted beforehand. I expect the 'dependencies' target is part of dependency list of the *-extract target (because of host-tar).
If it's done so, then it's ok tm.
I may be missing something, but I don't see a dependency of *-extract on 'dependencies'.

I didn't say it was, but how I expect it should be. ;)

Quoted 3 lines

Also, I tried the scenario and it doesn't work. The compiler used to do the linux-menuconfig is with ccache, and it isn't built automatically, unless you specify 'dependencies' manually.

Arf... Maybe more work is needed to fix that bug.

Signature · 1 line

Samuel

Thomas De Schampheleire

author

2013-08-07 09:28 UTC #6 lore raw

PATCH 1/1 host-ccache: turn into a proper dependency

Hi Thomas,

Quoted 44 lines · Samuel Martin

2013/8/2 Thomas De Schampheleire <[email protected]>:
Quoted 43 lines
Hi Samuel,
Quoted 39 lines · Samuel Martin
Hi Thomas (both), all,
2013/8/2 Thomas De Schampheleire <[email protected]>:
Quoted 9 lines
This patch moves the host-ccache build target from BASE_TARGETS in Makefile to an actual host prerequisite in support/dependencies. This causes host-ccache to be built as part of the dependencies, before any real package is built.
Since the dependencies are built without ccache anyway, there is no need to set HOST_CCACHE_CONF_ENV anymore.
Suggested-by Thomas Petazzoni <[email protected]>
Signed-off-by Thomas De Schampheleire <[email protected]>
Acked-by Samuel Martin <[email protected]>
Quoted 14 lines
---
 Makefile                             |  4 ----
 package/ccache/ccache.mk             |  6 ------
 support/dependencies/dependencies.mk |  4 ++++
 3 files changed, 4 insertions(+), 10 deletions(-)
Note: this in itself works, when the dependencies are built (regular 'make' for example, or explicit 'make dependencies'). However, if we want to supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants the following to work:
Quoted 6 lines
$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig
then I think something additional is necessary. In the above flow, the dependencies target is never used, so host-ccache doesn't get a chance to build.
Humm... I don't think so. This patch should be enough to fix the bug #5678. I mean, to run *-menuconfig, the package need to be extracted beforehand. I expect the 'dependencies' target is part of dependency list of the *-extract target (because of host-tar).
If it's done so, then it's ok tm.

Quoted 8 lines

Quoted 2 lines
I may be missing something, but I don't see a dependency of *-extract on 'dependencies'.
I didn't say it was, but how I expect it should be. ;)
Quoted 3 lines
Also, I tried the scenario and it doesn't work. The compiler used to do the linux-menuconfig is with ccache, and it isn't built automatically, unless you specify 'dependencies' manually.
Arf... Maybe more work is needed to fix that bug.

What should we do with this series?
And how to proceed with the menuconfig thing?

Thanks,
Thomas

Thomas Petazzoni

2013-08-02 08:52 UTC #7 lore raw

PATCH 1/1 host-ccache: turn into a proper dependency

In reply to Samuel Martin

Dear Samuel Martin,

Quoted 8 lines · Samuel Martin

Quoted 8 lines
Note: this in itself works, when the dependencies are built (regular 'make' for example, or explicit 'make dependencies'). However, if we want to supersede patch http://patchwork.ozlabs.org/patch/257372/, that wants the following to work:
$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig

Quoted 7 lines

Quoted 3 lines
then I think something additional is necessary. In the above flow, the dependencies target is never used, so host-ccache doesn't get a chance to build.
Humm... I don't think so. This patch should be enough to fix the bug #5678. I mean, to run *-menuconfig, the package need to be extracted beforehand. I expect the 'dependencies' target is part of dependency list of the *-extract target (because of host-tar).

But it's not the case as far as I can see.

in busybox.mk:

busybox-menuconfig busybox-xconfig busybox-gconfig: busybox-patch

in pkg-generic.mk

$(1)-patch: $(1)-extract $$($(2)_TARGET_PATCH)

$(1)-extract:           $(1)-source \
                        $$($(2)_TARGET_EXTRACT)

$(1)-source: $$($(2)_TARGET_SOURCE)

and that's it. Nowhere it pulls 'dependencies' as far as I can see.

Thomas

Signature · 4 lines

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Thomas De Schampheleire

author

2013-08-02 07:33 UTC #8 lore raw

PATCH 1/1 crosstool-ng: remove unneed explicit ccache dependency

In reply to Thomas De Schampheleire

This ccache dependency was here to enforce build order. It is not needed anymore.

Signed-off-by Samuel Martin <[email protected]>

Signed-off-by Thomas De Schampheleire <[email protected]>

---
package/crosstool-ng/crosstool-ng.mk | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)

package/crosstool-ng/crosstool-ng.mk +1 -1

@@ -11,7 +11,7 @@ CROSSTOOL_NG_INSTALL_TARGET    = NO
 CROSSTOOL_NG_MAKE              = $(MAKE1)
 
 HOST_CROSSTOOL_NG_DEPENDENCIES = \
-	$(if $(BR2_CCACHE),host-ccache) host-gawk \
+	host-gawk \
 	host-automake host-gperf host-bison host-flex \
 	host-texinfo

Thomas De Schampheleire

author

2013-08-02 07:33 UTC #9 lore raw

PATCH 1/1 sstrip: remove unneed explicit ccache dependency

In reply to Thomas De Schampheleire

This ccache dependency was here to enforce the build order. It is not needed anymore.

Signed-off-by Samuel Martin <[email protected]>

Signed-off-by Thomas De Schampheleire <[email protected]>

---
package/sstrip/sstrip.mk | 5 -----

 1 file changed, 5 deletions(-)

package/sstrip/sstrip.mk +0 -5

@@ -8,11 +8,6 @@ SSTRIP_SITE = svn://dev.openwrt.org/open
 SSTRIP_VERSION = 20154
 HOST_SSTRIP_BINARY = $(GNU_TARGET_NAME)-sstrip
 
-# This is a kludge to get host-ccache built before us or it fails
-ifeq ($(BR2_CCACHE),y)
-HOST_SSTRIP_DEPENDENCIES = host-ccache
-endif
-
 define SSTRIP_BUILD_CMDS
 	cd $(@D) ; \
 	$(TARGET_CC) $(TARGET_CFLAGS) -include endian.h -include byteswap.h \