Makefile: Support merged defconfigs
PATCH 1/1 Makefile: Support merged defconfigs
Within the Linux kernel, and several other packages, it's currently possible to maintain defconfigs as diffs (fragments) against other configs but this is not possible with buildroot itself.
This patch adds the capability, although using a slightly different implementation. Files may be added in the normal config directories ($TOPDIR/configs or $BR2_EXTERNAL/configs) with the format "xxx_defconfig.merge" that contain, one per line, the files to be passed to merge_config.sh to create the matching xxx_defconfig file (the first line should contain the base config followed by the fragments). The generated defconfig file is then handled as it would normally be.
---
For a discussion about this from the OP build project, see
https://github.com/open-power/op-build/issues/457
A couple of things about the implementation:
* merge_config.sh will only output to a file called ".config" so I've set it up to output directly to $TOPDIR/.config, since that's about to be overwritten anyway, but I don't feel this is particularly nice.
* It seems strange to pass --defconfig=.config when running conf but I couldn't quickly tell if there was something better to do. Using --oldconfig would pick up .config by default but I don't know if it would then process the file correctly so I went with what seemed safe.
* I realise that there are now four very similar rules that handle defconfigs. I could look at factoring them but it didn't seem worth the effort and complexity at this point.
Makefile | 10 ++++++++++ 1 file changed, 10 insertions(+)
Makefile +10 -0
@@ -845,6 +845,16 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \ $< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN) +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig.merge outputmakefile + cd $(TOPDIR)/configs && xargs -a [email protected] -x -P 1 $(TOPDIR)/support/kconfig/merge_config.sh -m -O $(TOPDIR) + $(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/.config \ + $< --defconfig=$(TOPDIR)/.config $(CONFIG_CONFIG_IN) + +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig.merge outputmakefile + cd $(BR2_EXTERNAL)/configs && xargs -a [email protected] -x -P 1 $(TOPDIR)/support/kconfig/merge_config.sh -m -O $(TOPDIR) + $(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/.config \ + $< --defconfig=$(TOPDIR)/.config $(CONFIG_CONFIG_IN) + savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @$(COMMON_CONFIG_ENV) $< \ --savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
Signature · 1 line
2.1.0
PATCH 1/1 Makefile: Support merged defconfigs
Quoted 13 lines · Sam Bobroff
Within the Linux kernel, and several other packages, it's currently possible to maintain defconfigs as diffs (fragments) against other configs but this is not possible with buildroot itself.
This patch adds the capability, although using a slightly different implementation. Files may be added in the normal config directories ($TOPDIR/configs or $BR2_EXTERNAL/configs) with the format "xxx_defconfig.merge" that contain, one per line, the files to be passed to
merge_config.shto create the matchingxxx_defconfigfile (the first line should contain the base config followed by the fragments). The generated defconfig file is then handled as it would normally be.
Quoted 1 line
Signed-off-by Sam Bobroff <[email protected]>
This neatly solves some problems for us over in op-build land. Tested with several fragments and output is what I expect:
Quoted 48 lines
---
For a discussion about this from the OP build project, see
https://github.com/open-power/op-build/issues/457A couple of things about the implementation:
*
merge_config.shwill only output to a file called ".config" so I've set it up to output directly to $TOPDIR/.config, since that's about to be overwritten anyway, but I don't feel this is particularly nice.
* It seems strange to pass --defconfig=.config when running conf but I couldn't quickly tell if there was something better to do. Using --oldconfig would pick up .config by default but I don't know if it would then process the file correctly so I went with what seemed safe.
* I realise that there are now four very similar rules that handle defconfigs. I could look at factoring them but it didn't seem worth the effort and complexity at this point.Makefile | 10 ++++++++++ 1 file changed, 10 insertions(+)Makefile +10
-0@@ -845,6 +845,16 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \ $< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN) +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig.merge outputmakefile + cd $(TOPDIR)/configs && xargs -a [email protected] -x -P 1 $(TOPDIR)/support/kconfig/merge_config.sh -m -O $(TOPDIR) + $(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/.config \ + $< --defconfig=$(TOPDIR)/.config $(CONFIG_CONFIG_IN) + +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig.merge outputmakefile + cd $(BR2_EXTERNAL)/configs && xargs -a [email protected] -x -P 1 $(TOPDIR)/support/kconfig/merge_config.sh -m -O $(TOPDIR) + $(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/.config \ + $< --defconfig=$(TOPDIR)/.config $(CONFIG_CONFIG_IN) + savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @$(COMMON_CONFIG_ENV) $< \ --savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \Signature · 6 lines
2.1.0 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
PATCH 1/1 Makefile: Support merged defconfigs
In reply to Sam BobroffQuoted 13 lines · On Mon, 9 May 2016 15:22:32 +1000 Sam Bobroff <[email protected]> wrote:
Within the Linux kernel, and several other packages, it's currently possible to maintain defconfigs as diffs (fragments) against other configs but this is not possible with buildroot itself.
This patch adds the capability, although using a slightly different implementation. Files may be added in the normal config directories ($TOPDIR/configs or $BR2_EXTERNAL/configs) with the format "xxx_defconfig.merge" that contain, one per line, the files to be passed to
merge_config.shto create the matchingxxx_defconfigfile (the first line should contain the base config followed by the fragments). The generated defconfig file is then handled as it would normally be.
Quoted 5 lines
Signed-off-by Sam Bobroff <[email protected]>---
For a discussion about this from the OP build project, see
https://github.com/open-power/op-build/issues/457
Hi Sam,
Quoted 9 lines
A couple of things about the implementation:
*
merge_config.shwill only output to a file called ".config" so I've set it up to output directly to $TOPDIR/.config, since that's about to be overwritten anyway, but I don't feel this is particularly nice.
* It seems strange to pass --defconfig=.config when running conf but I couldn't quickly tell if there was something better to do. Using --oldconfig would pick up .config by default but I don't know if it would then process the file correctly so I went with what seemed safe.
Looks like what you've done will work. I've come to the same conclusion as you that --oldconfig might actually be a better fit, and by extension --olddefconfig may be even better moving forward as the fragments age and new symbols get added.
Then it occurred to me I've had this question before and I had to go look what I did for linux in arch/powerpc/Makefile and turns out, olddefconfig.
Quoted 4 lines
* I realise that there are now four very similar rules that handle defconfigs. I could look at factoring them but it didn't seem worth the effort and complexity at this point.
I have no doubt there may be a refactored solution in there somewhere but I don't believe it will be as easy to follow as what currently exists. Currently it is easy to understand why there are four, what they are trying to achieve, I'd bet that with any refactoring that clarity will be dulled.
Quoted 24 lines
Makefile | 10 ++++++++++ 1 file changed, 10 insertions(+)Makefile +10
-0@@ -845,6 +845,16 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @$(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(BR2_EXTERNAL)/configs/$@ \ $< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN) +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(TOPDIR)/configs/%_defconfig.merge outputmakefile + cd $(TOPDIR)/configs && xargs -a [email protected] -x -P 1 $(TOPDIR)/support/kconfig/merge_config.sh -m -O $(TOPDIR) + $(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/.config \ + $< --defconfig=$(TOPDIR)/.config $(CONFIG_CONFIG_IN) + +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig.merge outputmakefile + cd $(BR2_EXTERNAL)/configs && xargs -a [email protected] -x -P 1 $(TOPDIR)/support/kconfig/merge_config.sh -m -O $(TOPDIR) + $(COMMON_CONFIG_ENV) BR2_DEFCONFIG=$(TOPDIR)/.config \ + $< --defconfig=$(TOPDIR)/.config $(CONFIG_CONFIG_IN) + savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @$(COMMON_CONFIG_ENV) $< \ --savedefconfig=$(if $(DEFCONFIG),$(DEFCONFIG),$(CONFIG_DIR)/defconfig) \
PATCH 1/1 Makefile: Support merged defconfigs
In reply to Sam BobroffHello,
Quoted 13 lines · Sam Bobroff
Within the Linux kernel, and several other packages, it's currently possible to maintain defconfigs as diffs (fragments) against other configs but this is not possible with buildroot itself.
This patch adds the capability, although using a slightly different implementation. Files may be added in the normal config directories ($TOPDIR/configs or $BR2_EXTERNAL/configs) with the format "xxx_defconfig.merge" that contain, one per line, the files to be passed to
merge_config.shto create the matchingxxx_defconfigfile (the first line should contain the base config followed by the fragments). The generated defconfig file is then handled as it would normally be.
Quoted 1 line
Signed-off-by Sam Bobroff <[email protected]>
Warning: the below is definitely not a final opinion or decision. It is merely the start of a discussion.
I do like the usage of fragments. Your Github link at
https://github.com/open-power/op-build/issues/457 even points to Buildroot training materials that I wrote, which explains how to use fragments.
However, I am not sure whether we want in Buildroot a mechanism to build a defconfig from fragments. To me, Buildroot should remain a simple tool that takes as input a complete configuration, and generates a Linux system from it.
Whether this "complete configuration" comes from a full .config, a minimal defconfig, a defconfig generated from fragments, a defconfig generated by a separate tool, by a webpage, or anything else, is not Buildroot's matter. Buildroot's job starts when the configuration is ready.
Essentially, the part that's missing in your commit is the update to the documentation. And when you'll start updating the manual to explain how this feature is working, you'll probably see that it's a bit tricky to explain.
So, I'm definitely not saying "no" to this, but my belief is that it's on the edges of Buildroot features. On the one hand, it brings a well-defined and correct way of storing fragments and generated a defconfig from fragments, that everyone can use and rely on. On the other hand, it adds one more slightly "tricky" feature to Buildroot, increasing its learning curve.
For a customer project, Gustavo has written a "genconfig" shell script, that takes as input various parameters that define the system to be built (which HW platform, which software stack, whether it should be a debug or release build) and produces the defconfig for Buildroot.
What do others think?
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH 1/1 Makefile: Support merged defconfigs
Quoted 49 lines · Thomas Petazzoni
Hello,
Quoted 13 lines · Sam Bobroff
Within the Linux kernel, and several other packages, it's currently possible to maintain defconfigs as diffs (fragments) against other configs but this is not possible with buildroot itself.
This patch adds the capability, although using a slightly different implementation. Files may be added in the normal config directories ($TOPDIR/configs or $BR2_EXTERNAL/configs) with the format "xxx_defconfig.merge" that contain, one per line, the files to be passed to
merge_config.shto create the matchingxxx_defconfigfile (the first line should contain the base config followed by the fragments). The generated defconfig file is then handled as it would normally be.Quoted 1 line
Signed-off-by Sam Bobroff <[email protected]>Warning: the below is definitely not a final opinion or decision. It is merely the start of a discussion.
I do like the usage of fragments. Your Github link at
https://github.com/open-power/op-build/issues/457 even points to Buildroot training materials that I wrote, which explains how to use fragments.However, I am not sure whether we want in Buildroot a mechanism to build a defconfig from fragments. To me, Buildroot should remain a simple tool that takes as input a complete configuration, and generates a Linux system from it.
Whether this "complete configuration" comes from a full .config, a minimal defconfig, a defconfig generated from fragments, a defconfig generated by a separate tool, by a webpage, or anything else, is not Buildroot's matter. Buildroot's job starts when the configuration is ready.
Essentially, the part that's missing in your commit is the update to the documentation. And when you'll start updating the manual to explain how this feature is working, you'll probably see that it's a bit tricky to explain.
So, I'm definitely not saying "no" to this, but my belief is that it's on the edges of Buildroot features. On the one hand, it brings a well-defined and correct way of storing fragments and generated a defconfig from fragments, that everyone can use and rely on. On the other hand, it adds one more slightly "tricky" feature to Buildroot, increasing its learning curve.
Quoted 4 lines
For a customer project, Gustavo has written a "genconfig" shell script, that takes as input various parameters that define the system to be built (which HW platform, which software stack, whether it should be a debug or release build) and produces the defconfig for Buildroot.
I tend to agree with Thomas's reasoning. However, I think it would be good if
there was a clearer canonical way of using buildroot. Gustavoz' genconfig script could be part of that. I, for example, have a local.mk fragment that autodetects packages you have checked out in $(BR2_EXTERNAL)/src and adds OVERRIDE_SRCDIR for them. I remember there was this template br2-external github repo at some point that you could use as a basis to start your own br2-external project (but I can't find it back).
I do think it is good if such supporting functionality is carried in the core
buildroot repo. But it's better if it's a bit out of the way, not in the main Makefile, and probably called through some helper script.
I have a similar feeling about the genimage template script BTW.
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
PATCH 1/1 Makefile: Support merged defconfigs
Hello,
Quoted 7 lines · Arnout Vandecappelle
I tend to agree with Thomas's reasoning. However, I think it would be good if
there was a clearer canonical way of using buildroot. Gustavoz' genconfig script could be part of that. I, for example, have a local.mk fragment that autodetects packages you have checked out in $(BR2_EXTERNAL)/src and addsOVERRIDE_SRCDIRfor them. I remember there was this template br2-external github repo at some point that you could use as a basis to start your own br2-external project (but I can't find it back).
You're probably referring to
https://github.com/Openwide-Ingenierie/buildroot-submodule.
Quoted 5 lines
I do think it is good if such supporting functionality is carried in the core
buildroot repo. But it's better if it's a bit out of the way, not in the mainMakefile, and probably called through some helper script.I have a similar feeling about the genimage template script BTW.
Should we create a section in the manual with some kind of "best practices" or "going further" ideas?
Regarding Gustavo's 'genconfig' script, I am not sure it makes a lot of sense to have something like that in the tree. All what it does is very project-specific, and down the road, it simply calls the merge_config.sh script, which is already in our tree.
So, for the specific situation of creating a defconfig by combining fragments, isn't the problem more a documentation problem than anything else?
Best regards,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com