Skip to contentSkip to conversation

Makefile: Support merged and nested defconfigs.

PATCH Superseded Sam Bobroff 2 participants 10 messages Jun 21, 2016 Kbuild

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Makefile: Support merged and nested defconfigs.

This is a combination and cleanup of the previous efforts by Sam Bobroff and Elizabeth Liner to introduce merged defconfigs and nested defconfig directories respectively.

Merged defconfigs allow us to create defconfig snippets for aspects that are common across multiple machines and create a combined defconfig from the snippets. The support scripts for merged kernel defconfigs is reused.

Nested defconfig directories allow us to organize the defconfig directory into sub-directories. A likely use for this is to place defconfigs into architecture-specific subdirectories.

Changes v1 -> v2:

This is a complete rework of the config merging system, discarding the .merge files and instead using Makefile fragments included from the configuration directories that invoke a Make function. This allows much better error handling as well as a cleaner looking system (somewhat similar to the package .mk files).

I've also added some documentation to explain what is supported and an example to show how it's used. I haven't touched the documentation before and I'm not familiar with the formatting so I hope I've done it the right way and that it's useful (feedback would be great). "make manual-html" seems to work at least.

Elizabeth Liner (1):
Makefile: Support nested config directories

Patrick Williams (3):
Makefile: Generate %_defconfig recipes from macro.
Makefile: Add nested config dirs to list-defconfigs
Makefile: Add merged defconfigs to list-defconfigs.

Sam Bobroff (1):
Add support for merged defconfigs

 Makefile                             | 58 +++++++++++++++++++++++++++++-----
 docs/manual/adding-board-support.txt |  3 ++
 docs/manual/appendix.txt             |  1 +
 docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
 4 files changed, 114 insertions(+), 8 deletions(-)
 create mode 100644 docs/manual/merged-defconfigs.txt
Signature · 1 line
2.1.0
#1 lore raw

PATCH v2 1/5 Add support for merged defconfigs

---
 Makefile                             | 38 +++++++++++++++++++++++
 docs/manual/adding-board-support.txt |  3 ++
 docs/manual/appendix.txt             |  1 +
 docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+)
 create mode 100644 docs/manual/merged-defconfigs.txt
Makefile +38 -0
index 78b44c5..9f3fc92 100644
@@ -858,6 +858,44 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 .PHONY: defconfig savedefconfig
 
+# Find the path of a config file that may be either in TOPDIR or BR2_EXTERNAL.
+# Missing or duplicate matches must be checked for elsewhere.
+define FIND_CONFIG_FILE =
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(wildcard $(d)/$(1)))
+endef
+
+define CONFIG_FILE_LIST =
+$(foreach f,$1,$(call FIND_CONFIG_FILE,$f))
+endef
+
+# Check that each config file is found once and only once.
+define CHECK_CONFIG_FILE_LIST =
+$(foreach f,$(1),
+ifeq "$(words $(call FIND_CONFIG_FILE,$(f)))" "0"
+$$(error ERROR: Missing input file: $(f))
+else
+ifneq "$(words $(call FIND_CONFIG_FILE,$(f)))" "1"
+$$(error ERROR: Duplicate input file: $(f))
+endif
+endif
+)
+endef
+
+# To be called by configuration fragments (*.mk) to set up defconfigs built
+# by merge_config.sh, via the include that follows:
+define merge_config =
+$(call CHECK_CONFIG_FILE_LIST,$2 $3)
+$(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputmakefile
+	@mkdir $$(CONFIG_DIR)/.merge_config
+	@$$(TOPDIR)/support/kconfig/merge_config.sh -m -O $$(CONFIG_DIR)/.merge_config \
+		$(call CONFIG_FILE_LIST,$2 $3)
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(CONFIG_DIR)/.merge_config/.config \
+		$$< --defconfig=$$(CONFIG_DIR)/.merge_config/.config $$(CONFIG_CONFIG_IN)
+	@rm -rf $$(CONFIG_DIR)/.merge_config
+endef
+
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+
 ################################################################################
 #
 # Cleanup and misc junk
docs/manual/adding-board-support.txt +3 -0
index f6d74ae..086a67e 100644
@@ -24,6 +24,9 @@ savedefconfig+. This will generate a minimal +defconfig+ file@the
 root of the Buildroot source tree. Move this file into the +configs/+
 directory, and rename it +<boardname>_defconfig+.
 
+It is also possible to create configurations by specifying differences from
+an existing configuration. See xref:merged-defconfigs[].
+
 It is recommended to use as much as possible upstream versions of the
 Linux kernel and bootloaders, and to use as much as possible default
 kernel and bootloader configurations. If they are incorrect for your
docs/manual/appendix.txt +1 -0
index 87a20bd..9abc37e 100644
@@ -3,6 +3,7 @@
 
 include::makedev-syntax.txt[]
 include::makeusers-syntax.txt[]
+include::merged-defconfigs.txt[]
 
 
 // Automatically generated lists:
docs/manual/merged-defconfigs.txt new file +60 -0

60 lines not on this page · raw message

Signature · 1 line
2.1.0
#4 lore raw

PATCH v2 2/5 Makefile: Generate %_defconfig recipes from macro.

In reply to Sam Bobroff

From: Patrick Williams <[email protected]>

To reduce duplication in the %_defconfig recipes with $(TOPDIR) and $(BR2_EXTERNAL) versions, generate these from a macro. The macro is now called on a list of directories containing the appropriate ones.

---
 Makefile | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
Makefile +8 -7
index 9f3fc92..e78b9e2 100644
@@ -842,13 +842,14 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
 # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
-		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
+define CREATE_DEFCONFIG_RECIPES
+%_defconfig: $$(BUILD_DIR)/buildroot-config/conf $1/%_defconfig outputmakefile
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(TOPDIR)/configs/$$@ \
+		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
+endef
 
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
-		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
+BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
+$(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< \
@@ -894,7 +895,7 @@ $(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputm
 	@rm -rf $$(CONFIG_DIR)/.merge_config
 endef
 
-$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+$(foreach d,$(BR2_DEFCONFIG_PATHS),$(eval -include $(d)/*.mk))
 
 ################################################################################
 #
Signature · 1 line
2.1.0
#7 lore raw

PATCH v2 3/5 Makefile: Support nested config directories

In reply to Sam Bobroff

From: Elizabeth Liner <[email protected]>

Extend the BR2_DEFCONFIG_PATH to support nested config directories. This allows us to place config files in both .../config and .../config/<arch>.

Both $(TOPDIR) and $(BR2_EXTERNAL) are supported.

---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Makefile +2 -1
index e78b9e2..fd07f8c 100644
@@ -848,7 +848,8 @@ define CREATE_DEFCONFIG_RECIPES
 		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
 endef
 
-BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
+BR2_DEFCONFIG_PATHS=$(sort $(dir $(wildcard $(TOPDIR)/configs/*/))) \
+		$(sort $(dir $(wildcard $(BR2_EXTERNAL)/configs/*/)))
 $(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
Signature · 1 line
2.1.0
#8 lore raw

PATCH v2 4/5 Makefile: Add nested config dirs to list-defconfigs

In reply to Sam Bobroff

From: Patrick Williams <[email protected]>

---
 Makefile | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
Makefile +6 -4
index fd07f8c..cd94f52 100644
@@ -206,6 +206,10 @@ LEGAL_LICENSES_TXT_HOST = $(LEGAL_INFO_DIR)/host-licenses.txt
 LEGAL_WARNINGS = $(LEGAL_INFO_DIR)/.warnings
 LEGAL_REPORT = $(LEGAL_INFO_DIR)/README
 
+BR2_DEFCONFIG_PATHS_LOCAL=$(sort $(dir $(wildcard $(TOPDIR)/configs/*/)))
+BR2_DEFCONFIG_PATHS_USER=$(sort $(dir $(wildcard $(BR2_EXTERNAL)/configs/*/)))
+BR2_DEFCONFIG_PATHS=$(BR2_DEFCONFIG_PATHS_LOCAL) $(BR2_DEFCONFIG_PATHS_USER)
+
 BR2_CONFIG = $(CONFIG_DIR)/.config
 
 # Pull in the user's configuration file
@@ -848,8 +852,6 @@ define CREATE_DEFCONFIG_RECIPES
 		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
 endef
 
-BR2_DEFCONFIG_PATHS=$(sort $(dir $(wildcard $(TOPDIR)/configs/*/))) \
-		$(sort $(dir $(wildcard $(BR2_EXTERNAL)/configs/*/)))
 $(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
@@ -1009,12 +1011,12 @@ help:
 
 list-defconfigs:
 	@echo 'Built-in configs:'
-	@$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_LOCAL),$(wildcard $(path)/*_defconfig)))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 ifneq ($(wildcard $(BR2_EXTERNAL)/configs/*_defconfig),)
 	@echo
 	@echo 'User-provided configs:'
-	@$(foreach b, $(sort $(notdir $(wildcard $(BR2_EXTERNAL)/configs/*_defconfig))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_USER),$(wildcard $(path)/*_defconfig)))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 endif
 	@echo
Signature · 1 line
2.1.0
#9 lore raw

PATCH v2 5/5 Makefile: Add merged defconfigs to list-defconfigs.

In reply to Sam Bobroff

From: Patrick Williams <[email protected]>

---
 Makefile | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Makefile +2 -2
index cd94f52..461066f 100644
@@ -1011,12 +1011,12 @@ help:
 
 list-defconfigs:
 	@echo 'Built-in configs:'
-	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_LOCAL),$(wildcard $(path)/*_defconfig)))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_LOCAL),$(basename $(wildcard $(path)/*_defconfig $(path)/*_defconfig.mk))))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 ifneq ($(wildcard $(BR2_EXTERNAL)/configs/*_defconfig),)
 	@echo
 	@echo 'User-provided configs:'
-	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_USER),$(wildcard $(path)/*_defconfig)))), \
+	@$(foreach b, $(sort $(notdir $(foreach path,$(BR2_DEFCONFIG_PATHS_USER),$(basename $(wildcard $(path)/*_defconfig $(path)/*_defconfig.mk))))), \
 	  printf "  %-35s - Build for %s\\n" $(b) $(b:_defconfig=);)
 endif
 	@echo
Signature · 1 line
2.1.0
#2 lore raw

PATCH v2 1/5 Add support for merged defconfigs

In reply to Sam Bobroff

Hi Sam, All,

Thanks for this new series!

I tested it by merging qemu_arm_vexpress_defconfig as base configuration file with a arm_toolchain.config and efl.config partial configuration files. The merged defconfig is named efl_on_arm_defconfig.

I think it's a "must have" for 2016.08.

Some comments bellow.

Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :

Quoted 8 lines
---
 Makefile                             | 38 +++++++++++++++++++++++
 docs/manual/adding-board-support.txt |  3 ++
 docs/manual/appendix.txt             |  1 +
 docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+)
 create mode 100644 docs/manual/merged-defconfigs.txt

I would be good to split in two commits the Makefile changes and the documentation.

Quoted 34 lines
Makefile +25 -0
index 78b44c5..9f3fc92 100644
@@ -858,6 +858,44 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 .PHONY: defconfig savedefconfig
 
+# Find the path of a config file that may be either in TOPDIR or BR2_EXTERNAL.
+# Missing or duplicate matches must be checked for elsewhere.
+define FIND_CONFIG_FILE =
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(wildcard $(d)/$(1)))
+endef
+
+define CONFIG_FILE_LIST =
+$(foreach f,$1,$(call FIND_CONFIG_FILE,$f))
+endef
+
+# Check that each config file is found once and only once.
+define CHECK_CONFIG_FILE_LIST =
+$(foreach f,$(1),
+ifeq "$(words $(call FIND_CONFIG_FILE,$(f)))" "0"
+$$(error ERROR: Missing input file: $(f))
+else
+ifneq "$(words $(call FIND_CONFIG_FILE,$(f)))" "1"
+$$(error ERROR: Duplicate input file: $(f))
+endif
+endif
+)
+endef
+
+# To be called by configuration fragments (*.mk) to set up defconfigs built
+# by merge_config.sh, via the include that follows:

Makefile savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile · line 895

Quoted 10 lines
Makefile +10 -0
+define merge_config =
+$(call CHECK_CONFIG_FILE_LIST,$2 $3)
+$(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputmakefile
+	@mkdir $$(CONFIG_DIR)/.merge_config
+	@$$(TOPDIR)/support/kconfig/merge_config.sh -m -O $$(CONFIG_DIR)/.merge_config \
+		$(call CONFIG_FILE_LIST,$2 $3)
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(CONFIG_DIR)/.merge_config/.config \
+		$$< --defconfig=$$(CONFIG_DIR)/.merge_config/.config $$(CONFIG_CONFIG_IN)
+	@rm -rf $$(CONFIG_DIR)/.merge_config
+endef

There is a problem here with make savedefconfig since BR2_DEFCONFIG = $$(CONFIG_DIR)/.merge_config/.config and .merge_config directory is removed with the last command.

I get:
*** Error while saving defconfig to: test/efl-1.17.1-merged/.merge_config/.config

BR2_DEFCONFIG should be set to configs/efl_on_arm_defconfig

I need more time to continue the review and testing.

Best regards,
Romain Naour

Quoted 99 lines
Makefile +3 -0
+
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+
 ################################################################################
 #
 # Cleanup and misc junk
docs/manual/adding-board-support.txt +3 -0
index f6d74ae..086a67e 100644
@@ -24,6 +24,9 @@ savedefconfig+. This will generate a minimal +defconfig+ file at the
 root of the Buildroot source tree. Move this file into the +configs/+
 directory, and rename it +<boardname>_defconfig+.
 
+It is also possible to create configurations by specifying differences from
+an existing configuration. See xref:merged-defconfigs[].
+
 It is recommended to use as much as possible upstream versions of the
 Linux kernel and bootloaders, and to use as much as possible default
 kernel and bootloader configurations. If they are incorrect for your
docs/manual/appendix.txt +1 -0
index 87a20bd..9abc37e 100644
@@ -3,6 +3,7 @@
 
 include::makedev-syntax.txt[]
 include::makeusers-syntax.txt[]
+include::merged-defconfigs.txt[]
 
 
 // Automatically generated lists:
docs/manual/merged-defconfigs.txt new file +60 -0
new file mode 100644
index 0000000..e1f50fd
@@ -0,0 +1,60 @@
+// -*- mode:doc -*- ;
+// vim: set syntax=asciidoc:
+
+[[merged-defconfigs]]
+== Merged configurations
+
+Buildroot supports the use of the Linux kernel's +merge_config.sh+ script to
+create configurations by merging a set of partial configurations (or overlays)
+on top of a base configuration. This may be useful when there are several
+configurations that are specializations of a single generic configuration.
+
+Files in the +configs/+ directory with the extension
++.mk+ will be read by the top level Makefile and the Make function
++merge_config+ can be used within them to define new merged configurations.
+
+The merge_config function takes three parameters: the first is the name of the
+new configuration (which should end with +_defconfig+), the second is the name
+of the base configuration file and the third is a space separated list of the
+partial configuration files to merge from the +configs/+ directory. All files
+should be specified relative to the +configs/+ directory and the name of the
+Make fragment file should match the name of the configuration it creates.
+
+The new configuration can be selected as if it were a normal configuration, i.e. with
++make foo_defconfig+.
+
+Partial configurations have the same format as full configuration
+files except that they contain only a subset of values. They can be created by
+starting with a normal configuration file and deleting lines as appropriate.
+
+External configuration files from the +$(BR2_EXTERNAL)/configs+ directory are fully
+supported, and merge configurations may freely refer to files from either
+location.
+
+Example:
+
+If configs/foo_defconfig contains:
+-----
+BR2_TARGET_GENERIC_HOSTNAME="buildroot"
+-----
+
+And configs/bar_defconfig.mk contains:
+-----
+$(eval $(call merge_config,bar_defconfig,foo_defconfig,b.config))
+-----
+
+And configs/b.config contains:
+-----
+BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
+-----
+
+After running the command:
+-----
+make bar_defconfig
+-----
+
+The resulting .config file would contain:
+-----
+BR2_TARGET_GENERIC_HOSTNAME="buildroot"
+BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
+-----
#5 lore raw

PATCH v2 2/5 Makefile: Generate %_defconfig recipes from macro.

In reply to Sam Bobroff

Hi Sam, All

Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :

Quoted 7 lines

From: Patrick Williams <[email protected]>

To reduce duplication in the %_defconfig recipes with $(TOPDIR) and $(BR2_EXTERNAL) versions, generate these from a macro. The macro is now called on a list of directories containing the appropriate ones.

Your Sob is missing here and for the following patches, see
http://nightly.buildroot.org/manual.html#submitting-patches

Best regards,
Romain

Quoted 39 lines
---
 Makefile | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
Makefile +8 -7
index 9f3fc92..e78b9e2 100644
@@ -842,13 +842,14 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
 # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
-		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
+define CREATE_DEFCONFIG_RECIPES
+%_defconfig: $$(BUILD_DIR)/buildroot-config/conf $1/%_defconfig outputmakefile
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(TOPDIR)/configs/$$@ \
+		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
+endef
 
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
-		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
+BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
+$(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< \
@@ -894,7 +895,7 @@ $(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputm
 	@rm -rf $$(CONFIG_DIR)/.merge_config
 endef
 
-$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+$(foreach d,$(BR2_DEFCONFIG_PATHS),$(eval -include $(d)/*.mk))
 
 ################################################################################
 #
#3 lore raw

PATCH v2 1/5 Add support for merged defconfigs

In reply to Romain Naour
Quoted 22 lines · Romain Naour

Hi Sam, All,

Thanks for this new series!

I tested it by merging qemu_arm_vexpress_defconfig as base configuration file with a arm_toolchain.config and efl.config partial configuration files. The merged defconfig is named efl_on_arm_defconfig.

I think it's a "must have" for 2016.08.

Some comments bellow.

Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :

Quoted 8 lines
---
 Makefile                             | 38 +++++++++++++++++++++++
 docs/manual/adding-board-support.txt |  3 ++
 docs/manual/appendix.txt             |  1 +
 docs/manual/merged-defconfigs.txt    | 60 ++++++++++++++++++++++++++++++++++++
 4 files changed, 102 insertions(+)
 create mode 100644 docs/manual/merged-defconfigs.txt
Quoted 1 line

I would be good to split in two commits the Makefile changes and the documentation.

Will do.

Quoted 48 lines
Quoted 33 lines
Makefile +25 -0
index 78b44c5..9f3fc92 100644
@@ -858,6 +858,44 @@ savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 
 .PHONY: defconfig savedefconfig
 
+# Find the path of a config file that may be either in TOPDIR or BR2_EXTERNAL.
+# Missing or duplicate matches must be checked for elsewhere.
+define FIND_CONFIG_FILE =
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(wildcard $(d)/$(1)))
+endef
+
+define CONFIG_FILE_LIST =
+$(foreach f,$1,$(call FIND_CONFIG_FILE,$f))
+endef
+
+# Check that each config file is found once and only once.
+define CHECK_CONFIG_FILE_LIST =
+$(foreach f,$(1),
+ifeq "$(words $(call FIND_CONFIG_FILE,$(f)))" "0"
+$$(error ERROR: Missing input file: $(f))
+else
+ifneq "$(words $(call FIND_CONFIG_FILE,$(f)))" "1"
+$$(error ERROR: Duplicate input file: $(f))
+endif
+endif
+)
+endef
+
+# To be called by configuration fragments (*.mk) to set up defconfigs built
+# by merge_config.sh, via the include that follows:

Makefile savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile · line 895

Quoted 10 lines
Makefile +10 -0
+define merge_config =
+$(call CHECK_CONFIG_FILE_LIST,$2 $3)
+$(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputmakefile
+	@mkdir $$(CONFIG_DIR)/.merge_config
+	@$$(TOPDIR)/support/kconfig/merge_config.sh -m -O $$(CONFIG_DIR)/.merge_config \
+		$(call CONFIG_FILE_LIST,$2 $3)
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(CONFIG_DIR)/.merge_config/.config \
+		$$< --defconfig=$$(CONFIG_DIR)/.merge_config/.config $$(CONFIG_CONFIG_IN)
+	@rm -rf $$(CONFIG_DIR)/.merge_config
+endef

There is a problem here with make savedefconfig since BR2_DEFCONFIG = $$(CONFIG_DIR)/.merge_config/.config and .merge_config directory is removed with the last command.

Quoted 4 lines

I get:
*** Error while saving defconfig to: test/efl-1.17.1-merged/.merge_config/.config

BR2_DEFCONFIG should be set to configs/efl_on_arm_defconfig

Ah, I'll take a look at that.

Quoted 1 line

I need more time to continue the review and testing.

Thanks for the review :-) I'll wait for more comments before I send v3.

Quoted 102 lines

Best regards,
Romain Naour

Quoted 99 lines
Makefile +3 -0
+
+$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+
 ################################################################################
 #
 # Cleanup and misc junk
docs/manual/adding-board-support.txt +3 -0
index f6d74ae..086a67e 100644
@@ -24,6 +24,9 @@ savedefconfig+. This will generate a minimal +defconfig+ file at the
 root of the Buildroot source tree. Move this file into the +configs/+
 directory, and rename it +<boardname>_defconfig+.
 
+It is also possible to create configurations by specifying differences from
+an existing configuration. See xref:merged-defconfigs[].
+
 It is recommended to use as much as possible upstream versions of the
 Linux kernel and bootloaders, and to use as much as possible default
 kernel and bootloader configurations. If they are incorrect for your
docs/manual/appendix.txt +1 -0
index 87a20bd..9abc37e 100644
@@ -3,6 +3,7 @@
 
 include::makedev-syntax.txt[]
 include::makeusers-syntax.txt[]
+include::merged-defconfigs.txt[]
 
 
 // Automatically generated lists:
docs/manual/merged-defconfigs.txt new file +60 -0
new file mode 100644
index 0000000..e1f50fd
@@ -0,0 +1,60 @@
+// -*- mode:doc -*- ;
+// vim: set syntax=asciidoc:
+
+[[merged-defconfigs]]
+== Merged configurations
+
+Buildroot supports the use of the Linux kernel's +merge_config.sh+ script to
+create configurations by merging a set of partial configurations (or overlays)
+on top of a base configuration. This may be useful when there are several
+configurations that are specializations of a single generic configuration.
+
+Files in the +configs/+ directory with the extension
++.mk+ will be read by the top level Makefile and the Make function
++merge_config+ can be used within them to define new merged configurations.
+
+The merge_config function takes three parameters: the first is the name of the
+new configuration (which should end with +_defconfig+), the second is the name
+of the base configuration file and the third is a space separated list of the
+partial configuration files to merge from the +configs/+ directory. All files
+should be specified relative to the +configs/+ directory and the name of the
+Make fragment file should match the name of the configuration it creates.
+
+The new configuration can be selected as if it were a normal configuration, i.e. with
++make foo_defconfig+.
+
+Partial configurations have the same format as full configuration
+files except that they contain only a subset of values. They can be created by
+starting with a normal configuration file and deleting lines as appropriate.
+
+External configuration files from the +$(BR2_EXTERNAL)/configs+ directory are fully
+supported, and merge configurations may freely refer to files from either
+location.
+
+Example:
+
+If configs/foo_defconfig contains:
+-----
+BR2_TARGET_GENERIC_HOSTNAME="buildroot"
+-----
+
+And configs/bar_defconfig.mk contains:
+-----
+$(eval $(call merge_config,bar_defconfig,foo_defconfig,b.config))
+-----
+
+And configs/b.config contains:
+-----
+BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
+-----
+
+After running the command:
+-----
+make bar_defconfig
+-----
+
+The resulting .config file would contain:
+-----
+BR2_TARGET_GENERIC_HOSTNAME="buildroot"
+BR2_TARGET_GENERIC_ISSUE="Welcome to BAR"
+-----
#6 lore raw

PATCH v2 2/5 Makefile: Generate %_defconfig recipes from macro.

In reply to Romain Naour
Quoted 5 lines · Romain Naour

Hi Sam, All

Le 21/06/2016 ? 06:56, Sam Bobroff a ?crit :

Quoted 2 lines

From: Patrick Williams <[email protected]>

Quoted 8 lines
Quoted 5 lines

To reduce duplication in the %_defconfig recipes with $(TOPDIR) and $(BR2_EXTERNAL) versions, generate these from a macro. The macro is now called on a list of directories containing the appropriate ones.

Your Sob is missing here and for the following patches, see
http://nightly.buildroot.org/manual.html#submitting-patches

Ah, thanks. I'll fix that for the next version.

Cheers,
Sam.

Quoted 42 lines

Best regards,
Romain

Quoted 39 lines
---
 Makefile | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
Makefile +8 -7
index 9f3fc92..e78b9e2 100644
@@ -842,13 +842,14 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< --defconfig$(if $(DEFCONFIG),=$(DEFCONFIG)) $(CONFIG_CONFIG_IN)
 
 # Override the BR2_DEFCONFIG from COMMON_CONFIG_ENV with the new defconfig
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/configs/$@ \
-		$< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN)
+define CREATE_DEFCONFIG_RECIPES
+%_defconfig: $$(BUILD_DIR)/buildroot-config/conf $1/%_defconfig outputmakefile
+	@$$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$$(TOPDIR)/configs/$$@ \
+		$$< --defconfig=$1/$$@ $$(CONFIG_CONFIG_IN)
+endef
 
-%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile
-	@$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \
-		$< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN)
+BR2_DEFCONFIG_PATHS=$(TOPDIR)/configs $(BR2_EXTERNAL)/configs
+$(foreach path,$(BR2_DEFCONFIG_PATHS),$(eval $(call CREATE_DEFCONFIG_RECIPES,$(path))))
 
 savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile
 	@$(COMMON_CONFIG_ENV) $< \
@@ -894,7 +895,7 @@ $(1): $$(BUILD_DIR)/buildroot-config/conf $(call CONFIG_FILE_LIST,$2 $3) outputm
 	@rm -rf $$(CONFIG_DIR)/.merge_config
 endef
 
-$(foreach d,$(TOPDIR)/configs $(BR2_EXTERNAL)/configs,$(eval -include $(d)/*.mk))
+$(foreach d,$(BR2_DEFCONFIG_PATHS),$(eval -include $(d)/*.mk))
 
 ################################################################################
 #