infra: make possible to run 'make *-menuconfig' from a clean output dir

PATCH New Samuel Martin 2 participants 5 messages Jul 7, 2013 – Jul 9, 2013 Kbuild

lore

Share & tools

lore

Samuel Martin

author

2013-07-07 18:43 UTC #0 lore raw

PATCH 1/3 infra: make possible to run 'make *-menuconfig' from a clean output dir

Fixes #5678

This patch allows to run:

$ make clean menuconfig
$ make {barebox,busybox,ctng,linux,uclibc}-menuconfig

with ccache enabled or not.

This patch introduce a new make variable BASE_PKGS for packages like host-ccache that need to be handled beforehand.

These packages are added to the dependency list of any packages, except for those from the BASE_PKGS list.

This also implies that if there are some dependency relationships between two packages from the BASE_PKGS list, then these dependencies should be manually handled.

Signed-off-by Samuel Martin <[email protected]>

---
 Makefile               | 6 ++++--
 package/pkg-generic.mk | 3 +++
 2 files changed, 7 insertions(+), 2 deletions(-)

Makefile +4 -2

index 955e684..015fbdf 100644
@@ -220,7 +220,7 @@ GNU_HOST_NAME:=$(shell support/gnuconfig/config.guess)
 ################################################################################
 
 ifeq ($(BR2_CCACHE),y)
-BASE_TARGETS += host-ccache
+BASE_PKGS += host-ccache
 endif
 
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)

@@ -361,7 +361,7 @@ include fs/common.mk
 TARGETS+=target-post-image
 
 TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
+TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS) $(BASE_PKGS))
 TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
 TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))

@@ -397,6 +397,8 @@ $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
 
+BASE_TARGETS += $(BASE_PKGS)
+
 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
 toolchain: prepare dirs dependencies $(BASE_TARGETS)

package/pkg-generic.mk +3 -0

index 668f011..282ce99 100644
@@ -303,6 +303,9 @@ $(2)_REDISTRIBUTE		?= YES
 
 
 $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
+ifeq ($(filter $(1),$(BASE_PKGS)),)
+$(2)_DEPENDENCIES += $(BASE_PKGS)
+endif
 
 $(2)_INSTALL_STAGING		?= NO
 $(2)_INSTALL_IMAGES		?= NO

Signature · 1 line

1.8.3.2

Samuel Martin

author

2013-07-07 18:43 UTC #1 lore raw

PATCH 2/3 crosstool-ng: remove unneed explicit ccache dependency

This ccache dependency was here to enforce build order. It is not needed anymore.

Signed-off-by Samuel Martin <[email protected]>

---
 package/crosstool-ng/crosstool-ng.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

package/crosstool-ng/crosstool-ng.mk +1 -1

index d3ee5f8..8d98db4 100644
@@ -11,7 +11,7 @@ CROSSTOOL_NG_INSTALL_TARGET    = NO
 CROSSTOOL_NG_MAKE              = $(MAKE1)
 
 HOST_CROSSTOOL_NG_DEPENDENCIES = \
-	$(if $(BR2_CCACHE),host-ccache) host-gawk \
+	host-gawk \
 	host-automake host-gperf host-bison host-flex \
 	host-texinfo

Signature · 1 line

1.8.3.2

Samuel Martin

author

2013-07-07 18:43 UTC #2 lore raw

PATCH 3/3 sstrip: remove unneed explicit ccache dependency

In reply to Samuel Martin

This ccache dependency was here to enforce the build order. It is not needed anymore.

Signed-off-by Samuel Martin <[email protected]>

---
 package/sstrip/sstrip.mk | 5 -----
 1 file changed, 5 deletions(-)

package/sstrip/sstrip.mk +0 -5

index be8c563..c79a501 100644
@@ -8,11 +8,6 @@ SSTRIP_SITE = svn://dev.openwrt.org/openwrt/trunk/tools/sstrip
 SSTRIP_VERSION = 20154
 HOST_SSTRIP_BINARY = $(GNU_TARGET_NAME)-sstrip
 
-# This is a kludge to get host-ccache built before us or it fails
-ifeq ($(BR2_CCACHE),y)
-HOST_SSTRIP_DEPENDENCIES = host-ccache
-endif
-
 define SSTRIP_BUILD_CMDS
 	cd $(@D) ; \
 	$(TARGET_CC) $(TARGET_CFLAGS) -include endian.h -include byteswap.h \

Signature · 1 line

1.8.3.2

Thomas Petazzoni

2013-07-07 19:37 UTC #3 lore raw

PATCH 1/3 infra: make possible to run 'make *-menuconfig' from a clean output dir

In reply to Samuel Martin

Dear Samuel Martin,

Quoted 16 lines · Samuel Martin

ifeq ($(BR2_CCACHE),y)

Makefile +2 -2

-BASE_TARGETS += host-ccache
+BASE_PKGS += host-ccache
 endif
 
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)

@@ -361,7 +361,7 @@ include fs/common.mk
 TARGETS+=target-post-image
 
 TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
+TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS) $(BASE_PKGS))
 TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
 TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))

Makefile $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake · line 401

Quoted 5 lines

Makefile +2 -0

@@ -397,6 +397,8 @@ $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
 	$(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig
 
+BASE_TARGETS += $(BASE_PKGS)
+

Is this really the correct fix? Currently BASE_TARGETS contains host-ccache if enabled, and the toolchain-<backend> target. So, normally, all packages should depend on both, and there should be no need to introduce a separate $(BASE_PKGS).

Quoted 8 lines

Makefile +0 -0

 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
 toolchain: prepare dirs dependencies $(BASE_TARGETS)

package/pkg-generic.mk +0 -0

index 668f011..282ce99 100644

package/pkg-generic.mk $(2)_REDISTRIBUTE ?= YES · line 308

Quoted 6 lines

package/pkg-generic.mk +3 -0

@@ -303,6 +303,9 @@ $(2)_REDISTRIBUTE		?= YES
 
 
 $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
+ifeq ($(filter $(1),$(BASE_PKGS)),)
+$(2)_DEPENDENCIES += $(BASE_PKGS)
+endif

I.e, wouldn't

$(2)_DEPENDENCIES += $(BASE_TARGETS)

work?

Hum, the issue would of course be a recursive dependency on the Crosstool-NG package, because toolchain-crosstool-ng depends on crosstool-ng, and crosstool-ng being a package, it would in turn depend on toolchain-crosstool-ng. Ditto for some other packages like gcc-{initial,intermediate,final}, binutils and so on. So maybe what I'm proposing in fact doesn't work, unless such packages are identified specifically, like <pkg>_TOOLCHAIN_COMPONENT = YES, which would prevent $(BASE_TARGETS) from being added to the dependencies of such packages.

Hum, now that I think of it, technically speaking, only the target packages do need to have the dependency on the toolchain, host packages do not. But both need the dependency on host-ccache.

Ok, the problem is complex, but I would really prefer to see a global reflexion on how to name all those BASE_TARGETS, BASE_PKGS, TARGETS variables rather than adding yet another hack into this mess.

Note that I am also currently fiddling in this area, since I have issues getting 'make source' and 'make external-deps' to work properly since the conversion of the internal backend to packages.

Best regards,

Thomas

Signature · 4 lines

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

Samuel Martin

author

2013-07-09 21:02 UTC #4 lore raw

PATCH 1/3 infra: make possible to run 'make *-menuconfig' from a clean output dir

Hi Thomas, all,

2013/7/7 Thomas Petazzoni <[email protected]>:

Quoted 23 lines

Dear Samuel Martin,

Quoted 19 lines · Samuel Martin

ifeq ($(BR2_CCACHE),y)

Makefile +2 -2

-BASE_TARGETS += host-ccache
+BASE_PKGS += host-ccache
 endif
 
 ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y)

@@ -361,7 +361,7 @@ include fs/common.mk
 TARGETS+=target-post-image
 
 TARGETS_CLEAN:=$(patsubst %,%-clean,$(TARGETS))
-TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS))
+TARGETS_SOURCE:=$(patsubst %,%-source,$(TARGETS) $(BASE_TARGETS) $(BASE_PKGS))
 TARGETS_DIRCLEAN:=$(patsubst %,%-dirclean,$(TARGETS))
 TARGETS_ALL:=$(patsubst %,__real_tgt_%,$(TARGETS))

@@ -397,6 +397,8 @@ $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake
 $(BUILD_DIR)/buildroot-config/auto.conf: $(BUILDROOT_CONFIG)
      $(MAKE) $(EXTRAMAKEARGS) HOSTCC="$(HOSTCC_NOCCACHE)" HOSTCXX="$(HOSTCXX_NOCCACHE)" silentoldconfig

Quoted 7 lines

Makefile $(BASE_TARGETS): dirs $(HOST_DIR)/usr/share/buildroot/toolchainfile.cmake · line 401
Quoted 2 lines
Makefile +2 -0
+BASE_TARGETS += $(BASE_PKGS)
+
Is this really the correct fix? Currently BASE_TARGETS contains host-ccache if enabled, and the toolchain-<backend> target. So, normally, all packages should depend on both, and there should be no need to introduce a separate $(BASE_PKGS).

ahem. I knew this patch would be controversial ;) There are also few point I'm not entirely happy wit them.

Quoted 18 lines

Quoted 9 lines

Makefile +0 -0

 prepare: $(BUILD_DIR)/buildroot-config/auto.conf
 
 toolchain: prepare dirs dependencies $(BASE_TARGETS)

package/pkg-generic.mk +0 -0

index 668f011..282ce99 100644

package/pkg-generic.mk $(2)_REDISTRIBUTE ?= YES · line 308

Quoted 6 lines

package/pkg-generic.mk +3 -0

@@ -303,6 +303,9 @@ $(2)_REDISTRIBUTE         ?= YES
 
 
 $(2)_DEPENDENCIES ?= $(filter-out $(1),$(patsubst host-host-%,host-%,$(addprefix host-,$($(3)_DEPENDENCIES))))
+ifeq ($(filter $(1),$(BASE_PKGS)),)
+$(2)_DEPENDENCIES += $(BASE_PKGS)
+endif

I.e, wouldn't

Quoted 3 lines

$(2)_DEPENDENCIES += $(BASE_TARGETS)
work?

This was my first (naive) though, but it ended up like this:
---
make: *** No rule to make target host-toolchain-buildroot-source', needed by source'. Stop.
---
On the very first attemp, I even got the same error for 'dirs-source'... which a non-sense!
That's why I split dependencies that are packages from the rest of the ones in BASE_TARGETS.

Quoted 10 lines

Hum, the issue would of course be a recursive dependency on the Crosstool-NG package, because toolchain-crosstool-ng depends on crosstool-ng, and crosstool-ng being a package, it would in turn depend on toolchain-crosstool-ng. Ditto for some other packages like gcc-{initial,intermediate,final}, binutils and so on. So maybe what I'm proposing in fact doesn't work, unless such packages are identified specifically, like <pkg>_TOOLCHAIN_COMPONENT = YES, which would prevent $(BASE_TARGETS) from being added to the dependencies of such packages.

Quoted 7 lines

Hum, now that I think of it, technically speaking, only the target packages do need to have the dependency on the toolchain, host packages do not. But both need the dependency on host-ccache.
Ok, the problem is complex, but I would really prefer to see a global reflexion on how to name all those BASE_TARGETS, BASE_PKGS, TARGETS variables rather than adding yet another hack into this mess.

Agree, that's why I prefer post this RFC.
I may not have all thing in mind... and I'm not a make guru...

Quoted 5 lines

Note that I am also currently fiddling in this area, since I have issues getting 'make source' and 'make external-deps' to work properly since the conversion of the internal backend to packages.

Ok, let me know about your progress

Regards,

Signature · 1 line

Samuel