Keeping customizations outside the Buildroot tree with BR2_EXTERNAL
v6 cover Keeping customizations outside the Buildroot tree with BR2_EXTERNAL
Hello,
Here is the sixth version of the BR2_EXTERNAL patch series. Its main purpose is to take into account the feedback received recently on the list.
People interested in testing this can find a branch at
http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=things-outside-br-v6.
Changes since v5
================
* Fix some typos in the documentation, noticed by Yann E. Morin.
* Split the manual generation fixes patch into two patches: one changing the regexp, now first in the series, and another one passing BR2_EXTERNAL, squashed with the core infrastructure.
Changes since v4
================
* Include Arnout implementation of PATCH 1 (core infrastructure), which is indeed much simpler.
* Add a dummy external.mk in support/dummy-external, which allows to remove the BR2_EXTERNAL_USED variable entirely.
* Fix various typos in the documentation, reported by Yann and Arnout.
* Integrate the new version of PATCH 5 (manual generation fixes), which includes a lot more details.
* Fix the "make help" text according to Yann's suggestions.
Changes since v3
================
* Fix the issue reported by Yann, where not having a output/.br-external file would make the build fail completely. I've fixed it in a slightly different way than the patch posted by Yann, though.
* Revert back to the idea used in v2: include only a top-level BR2_EXTERNAL/Config.in file and a top-level BR2_EXTERNAL/external.mk. This way, the user of BR2_EXTERNAL is free to use it for target packages, host packages, bootloaders, and more. People have raised concern that enforcing the inclusion of BR2_EXTERNAL/package/Config.in was problematic, because they also wanted to add host tools in BR2_EXTERNAL/package/Config.in, and so on. So since people have differerent use cases, let's give some more flexibility.
Changes since v2
================
* Remove the mechanism that was storing the BR2_EXTERNAL value in the external wrapper.
* Add logic that stores the BR2_EXTERNAL value in a file called $(O)/.br-external. Whenever the user passes a BR2_EXTERNAL=<value>, it gets stored in this file, and Buildroot re-uses for all subsequent make invocations. The user can later override it again by passing another BR2_EXTERNAL=<value>. He can also remove it entirely by passing just BR2_EXTERNAL= (empty value).
* Enforce usage of a package/ directory in the BR2_EXTERNAL directory. Buildroot only includes $(BR2_EXTERNAL)/Config.in and shows it under the "Target packages" menu, and Buildroot will only include $(BR2_EXTERNAL)/package/*/*.mk.
* Clearly separate the Buildroot built-in defconfigs from the user-defined external defconfigs in the 'make help' output.
* Include a fix for building the documentation, provided by Samuel Martin.
* Update the documentation accordingly.
Changes since v1
================
* Store BR2_EXTERNAL in the Makefile generated in the output directory, so that BR2_EXTERNAL doesn't have to be passed over and over again. Suggested by Ryan Barnett.
* Patch set splitted in more fine-grained changes, as suggested by Arnout.
* Instead of having the top-level Config.in file generated at run-time, use instead a trick proposed by Arnout: make BR2_EXTERNAL point to support/dummy-external/, which contains an empty Config.in. This way, regardless of whether BR2_EXTERNAL is specified by the user or not, kconfig is happy as 'source "$BR2_EXTERNAL/Config.in"' will always point to an existing file. If BR2_EXTERNAL is not used, it will point to the dummy file, if BR2_EXTERNAL is used, it will point to Config.in file in the provided BR2_EXTERNAL directory.
With this, BR2_EXTERNAL always has a value, so testing whether BR2_EXTERNAL is empty no longer tells us if the user has provided a value or not. Therefore, we have another variable, BR2_EXTERNAL_USED, which tells if the mechanism is used by the user or not. This variable avoids trying to use defconfigs from the dummy directory, or to encode the BR2_EXTERNAL value in the Makefile wrapper.
* As a consequence of the previous change, it is no longer needed to refactor the *config dependencies using a variable, so I've dropped what was previously the first patch of the series.
* BR2_EXTERNAL is turned into an absolute path, in order to avoid any problem. Suggested by Arnout.
* Fixed the usage of <some board> vs. <boardname> in the documentation, as noted by Arnout.
Thomas
Samuel Martin (1):
manual: fix manual generation in preparation for BR2_EXTERNAL support
Thomas Petazzoni (4):
core: introduce the BR2_EXTERNAL variable
core: allow external Config.in/makefile code to be integrated
core: allow external defconfigs to be used docs/manual: add explanations about BR2_EXTERNAL
Config.in | 6 ++ Makefile | 39 +++++++++- docs/manual/customize-outside-br.txt | 137 +++++++++++++++++++++++++++++++++++ docs/manual/customize.txt | 2 + docs/manual/manual.mk | 3 + support/dummy-external/Config.in | 0 support/dummy-external/external.mk | 0 support/scripts/kconfiglib.py | 2 +- 8 files changed, 187 insertions(+), 2 deletions(-) create mode 100644 docs/manual/customize-outside-br.txt create mode 100644 support/dummy-external/Config.in create mode 100644 support/dummy-external/external.mk
Signature · 1 line
1.8.1.2
PATCH v6 1/5 manual: fix manual generation in preparation for BR2_EXTERNAL support
From: Samuel Martin <[email protected]>
This patch fixes an issue that occurs during the manual build process which will occur when BR2_EXTERNAL is introduced.
During the package list generation, the python script using kconfiglib module reads and parses the Config.in files. So, symbols, including environment variables, got expanded and/or resolved. In kconfiglib.py, this patch fixes the regex that did not allow to use numbers in the environment variable names, so '$BR2_EXTERNAL' got wrongly expanded like it was '${BR}2_EXTERNAL':
<snip>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Updating the manual lists...
Traceback (most recent call last):
File "/opt/buildroot/master/support/scripts/gen-manual-lists.py", line 375, in <module>
buildroot = Buildroot()
File "/opt/buildroot/master/support/scripts/gen-manual-lists.py", line 216, in __init__
self.root_config))
File "/opt/buildroot/master/support/scripts/kconfiglib.py", line 214, in __init__
self.top_block = self._parse_file(filename, None, None, None)
File "/opt/src/buildroot/master/support/scripts/kconfiglib.py", line 919, in _parse_file
return self._parse_block(line_feeder, None, parent, deps, visible_if_deps, res)
File "/opt/buildroot/master/support/scripts/kconfiglib.py", line 1114, in _parse_block
self.base_dir))
IOError: /opt/buildroot/master/Config.in:490: sourced file "$BR2_EXTERNAL/Config.in" (expands to "2_EXTERNAL/Config.in") not found. Perhaps base_dir
(argument to Config.__init__(), currently
"/opt/buildroot/master") is set to the wrong value.docs/manual/manual.mk:2: recipe for target 'manual-update-lists' failed
make: *** [manual-update-lists] Error 1
</snip>
--- support/scripts/kconfiglib.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
support/scripts/kconfiglib.py +1 -1
@@ -2074,7 +2074,7 @@ set_re = re.compile(r"CONFIG_(\w+)=(.*)") unset_re = re.compile(r"# CONFIG_(\w+) is not set") # Regular expression for finding $-references to symbols in strings -sym_ref_re = re.compile(r"\$[A-Za-z_]+") +sym_ref_re = re.compile(r"\$[A-Za-z_][0-9A-Za-z_]*") # Integers representing symbol types UNKNOWN, BOOL, TRISTATE, STRING, HEX, INT = range(0, 6)
Signature · 1 line
1.8.1.2
PATCH v6 1/5 manual: fix manual generation in preparation for BR2_EXTERNAL support
Thomas, Samuel, All,
On 2013-12-05 00:14 +0100, Thomas Petazzoni spake thusly:
Quoted 33 lines
From: Samuel Martin <[email protected]>
This patch fixes an issue that occurs during the manual build process which will occur when
BR2_EXTERNALis introduced.During the package list generation, the python script using kconfiglib module reads and parses the Config.in files. So, symbols, including environment variables, got expanded and/or resolved. In
kconfiglib.py, this patch fixes the regex that did not allow to use numbers in the environment variable names, so '$BR2_EXTERNAL' got wrongly expanded like it was '${BR}2_EXTERNAL':<snip>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Updating the manual lists...
Traceback (most recent call last):
File "/opt/buildroot/master/support/scripts/gen-manual-lists.py", line 375, in <module>
buildroot =Buildroot()
File "/opt/buildroot/master/support/scripts/gen-manual-lists.py", line 216, in__init__
self.root_config))
File "/opt/buildroot/master/support/scripts/kconfiglib.py", line 214, in__init__
self.top_block =self._parse_file(filename, None, None, None)
File "/opt/src/buildroot/master/support/scripts/kconfiglib.py", line 919, in_parse_file
returnself._parse_block(line_feeder, None, parent, deps, visible_if_deps, res)
File "/opt/buildroot/master/support/scripts/kconfiglib.py", line 1114, in_parse_blockself.base_dir))IOError:
/opt/buildroot/master/Config.in:490: sourced file "$BR2_EXTERNAL/Config.in" (expands to "2_EXTERNAL/Config.in") not found. Perhapsbase_dir
(argument toConfig.__init__(), currently
"/opt/buildroot/master") is set to the wrong value.docs/manual/manual.mk:2: recipe for target 'manual-update-lists' failed
make: *** [manual-update-lists] Error 1
</snip>
Quoted 3 lines
Reported-by Ryan Barnett <[email protected]>Signed-off-by Samuel Martin <[email protected]>Signed-off-by Thomas Petazzoni <[email protected]>
Regards,
Yann E. MORIN.
Quoted 24 lines
--- support/scripts/kconfiglib.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)support/scripts/kconfiglib.py +1
-1@@ -2074,7 +2074,7 @@ set_re = re.compile(r"CONFIG_(\w+)=(.*)") unset_re = re.compile(r"# CONFIG_(\w+) is not set") # Regular expression for finding $-references to symbols in strings -sym_ref_re = re.compile(r"\$[A-Za-z_]+") +sym_ref_re = re.compile(r"\$[A-Za-z_][0-9A-Za-z_]*") # Integers representing symbol types UNKNOWN, BOOL, TRISTATE, STRING, HEX, INT = range(0, 6)Signature · 6 lines
1.8.1.2 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
[Buildroot] What is an Acked-by?
Quoted 2 lines · Yann E. MORIN
Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>
If you've reviewed it and tested it, you would commit it if you had
commit access, right? So this could actually be an Acked-by, right? Or is my understanding of these tags incorrect?
Regards,
Arnout
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
[Buildroot] What is an Acked-by?
Arnout, All,
On 2013-12-05 19:12 +0100, Arnout Vandecappelle spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 2 lines
Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>If you've reviewed it and tested it, you would commit it if you had commit
access, right? So this could actually be an Acked-by, right? Or is my understanding of these tags incorrect?
I'm following the definitions of Documentation/SubmittingPatches in my Linux kernel tree.
For example, I refer to:
Acked-by: does not necessarily indicate acknowledgement of the entire patch.
Reviewed-by:, instead, indicates that the patch has been reviewed and found acceptable according to the Reviewer's Statement [--SNIP statement--]
So, by providing both Reviewed-by and Tested by, I am explicitly stating that I did a review of the patch, and I tested it. Which, from my understanding, Acked-by does.
Also, I do not believe to be in a position to provide my Acked-by on the core infrastructure, which is rather Thomas' domain. So, Thomas would be right to provide his Acked-by on such patches (but obviously he can't on those, since he's the author).
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
[Buildroot] What is an Acked-by?
Arnout, All,
On 2013-12-05 21:12 +0100, Yann E. MORIN spake thusly:
Quoted 16 lines
On 2013-12-05 19:12 +0100, Arnout Vandecappelle spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 2 lines
Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>If you've reviewed it and tested it, you would commit it if you had commit
access, right? So this could actually be an Acked-by, right? Or is my understanding of these tags incorrect?I'm following the definitions of
Documentation/SubmittingPatchesin my Linux kernel tree.For example, I refer to:
Acked-by: does not necessarily indicate acknowledgement of the entire patch.
Quoted 7 lines
Reviewed-by:, instead, indicates that the patch has been reviewed and found acceptable according to the Reviewer's Statement [--SNIP statement--]
So, by providing both Reviewed-by and Tested by, I am explicitly stating that I did a review of the patch, and I tested it. Which, from my understanding, Acked-by does.
... Acked-by does *not*.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
[Buildroot] What is an Acked-by?
Here we go again... ;-)
Quoted 23 lines · Yann E. MORIN
Arnout, All,
On 2013-12-05 21:12 +0100, Yann E. MORIN spake thusly:
Quoted 20 lines
On 2013-12-05 19:12 +0100, Arnout Vandecappelle spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 2 lines
Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>If you've reviewed it and tested it, you would commit it if you had commit
access, right? So this could actually be an Acked-by, right? Or is my understanding of these tags incorrect?I'm following the definitions of
Documentation/SubmittingPatchesin my Linux kernel tree.For example, I refer to:
Acked-by: does not necessarily indicate acknowledgement of the entire patch.Reviewed-by:, instead, indicates that the patch has been reviewed and found acceptable according to the Reviewer's Statement [--SNIP statement--]
Quoted 5 lines
Quoted 3 lines
So, by providing both Reviewed-by and Tested by, I am explicitly stating that I did a review of the patch, and I tested it. Which, from my understanding, Acked-by does.
... Acked-by does *not*.
We discussed this on the Buildroot developer day at FOSDEM 2012, see the report [1] and addition in the buildroot manual [2]. The manual says:
Acked-by: Indicates that the patch can be committed.
Tested-by: Indicates that the patch has been tested. It is useful but not necessary to add a comment about what has been tested.
I must admit that in the mean time it has become common practice to also use Reviewed-by, so we'll need to clarify that.
By no means authoritative, but here is what I mean when I add the following tags on a patch:
- Tested-by: as in the manual: I performed some kind of test (typically described below the tag) on the patch.
- Reviewed-by: I code-reviewed the patch and did my best in spotting problems, but I am not sufficiently familiar with the area touched to provide an Acked-by. This means that, although I reviewed the patch, there may be remaining problems that would be spotted by someone with more experience in that area. The detection of such problems should not mean that my Reviewed-by: was too hasty.
- Acked-by: I code-reviewed the patch (note: not necessarily tested) and am familiar enough with the area touched that I can indicate it is a good patch. If someone else detects a serious problem with this patch afterwards, then this Acked-by may have been too hasty.
So for me, Acked-by is stronger than Reviewed-by, but orthogonal to Tested-by.
Note that my definition of Acked-by does not really rely on 'module owners', contrary to how Yann interprets it. For example in the case of the core infrastructure I don't believe that only ThomasP can provide an Acked-by. Several developers other than ThomasP have made good changes there, and are thus sufficiently knowledgeable to indicate their Ack. In my opinion, it is up to the maintainer to assess the weight of an Ack. He is free to wait until an ack by ThomasP, or not.
(for the record: with the above I do not want to minimize ThomasP's contribution in this area, not at all. His work has been and is of great importance for the buildroot project, and I deeply respect it (and him)
Best regards,
Thomas
[1] http://lists.busybox.net/pipermail/buildroot/2012-February/050371.html
[2] http://buildroot.uclibc.org/downloads/manual/manual.html#_reviewing_testing_patches
Arnout Vandecappelle
Acked-by[Buildroot] What is an Acked-by?
Quoted 55 lines · Thomas De Schampheleire
Here we go again... ;-)
Quoted 23 lines · Yann E. MORIN
Arnout, All,
On 2013-12-05 21:12 +0100, Yann E. MORIN spake thusly:
Quoted 20 lines
On 2013-12-05 19:12 +0100, Arnout Vandecappelle spake thusly:
Quoted 7 lines · Yann E. MORIN
Quoted 2 lines
Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>If you've reviewed it and tested it, you would commit it if you had commit
access, right? So this could actually be an Acked-by, right? Or is my understanding of these tags incorrect?I'm following the definitions of
Documentation/SubmittingPatchesin my Linux kernel tree.For example, I refer to:
Acked-by: does not necessarily indicate acknowledgement of the entire patch.Reviewed-by:, instead, indicates that the patch has been reviewed and found acceptable according to the Reviewer's Statement [--SNIP statement--]
Quoted 5 lines
Quoted 3 lines
So, by providing both Reviewed-by and Tested by, I am explicitly stating that I did a review of the patch, and I tested it. Which, from my understanding, Acked-by does.
... Acked-by does *not*.
We discussed this on the Buildroot developer day at FOSDEM 2012, see the report [1] and addition in the buildroot manual [2]. The manual says:
Acked-by: Indicates that the patch can be committed.
Tested-by: Indicates that the patch has been tested. It is useful but not necessary to add a comment about what has been tested.I must admit that in the mean time it has become common practice to also use Reviewed-by, so we'll need to clarify that.
By no means authoritative, but here is what I mean when I add the following tags on a patch:
- Tested-by: as in the manual: I performed some kind of test (typically described below the tag) on the patch.- Reviewed-by: I code-reviewed the patch and did my best in spotting problems, but I am not sufficiently familiar with the area touched to provide an Acked-by. This means that, although I reviewed the patch, there may be remaining problems that would be spotted by someone with more experience in that area. The detection of such problems should not mean that my Reviewed-by: was too hasty.
Quoted 4 lines
- Acked-by: I code-reviewed the patch (note: not necessarily tested) and am familiar enough with the area touched that I can indicate it is a good patch. If someone else detects a serious problem with this patch afterwards, then this Acked-by may have been too hasty.
If this text makes it into the documentation, it gets my
:-)
Now I do think I understand why Yann didn't give an Ack, but just a
Reviewed: he doesn't feel familiar enough with the infrastructure to be really sure the patch is OK.
You could also say: with Acked-by you are prepared to share the blame
if something is wrong with the patch, with Reviewed-by you're not.
Regards,
Arnout
Quoted 23 lines
So for me, Acked-by is stronger than Reviewed-by, but orthogonal to Tested-by.
Note that my definition of Acked-by does not really rely on 'module owners', contrary to how Yann interprets it. For example in the case of the core infrastructure I don't believe that only ThomasP can provide an Acked-by. Several developers other than ThomasP have made good changes there, and are thus sufficiently knowledgeable to indicate their Ack. In my opinion, it is up to the maintainer to assess the weight of an Ack. He is free to wait until an ack by ThomasP, or not.
(for the record: with the above I do not want to minimize ThomasP's contribution in this area, not at all. His work has been and is of great importance for the buildroot project, and I deeply respect it (and him)
Best regards,
Thomas[1] http://lists.busybox.net/pipermail/buildroot/2012-February/050371.html
[2] http://buildroot.uclibc.org/downloads/manual/manual.html#_reviewing_testing_patches
Signature · 6 lines
Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
PATCH v6 1/5 manual: fix manual generation in preparation for BR2_EXTERNAL support
In reply to Thomas PetazzoniThomas P.
Thomas Petazzoni <[email protected]> wrote on 12/04/2013 05:14:47 PM:
Quoted 12 lines
From: Samuel Martin <[email protected]>
This patch fixes an issue that occurs during the manual build process which will occur when
BR2_EXTERNALis introduced.During the package list generation, the python script using kconfiglib module reads and parses the Config.in files. So, symbols, including environment variables, got expanded and/or resolved. In
kconfiglib.py, this patch fixes the regex that did not allow to use numbers in the environment variable names, so '$BR2_EXTERNAL' got wrongly expanded like it was '${BR}2_EXTERNAL':
Quoted 4 lines
<snip>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Updating the manual lists...
Traceback (most recent call last):
File "/opt/buildroot/master/support/scripts/gen-manual-lists.py", line
375, in <module>
Quoted 2 lines
buildroot =
Buildroot()
File "/opt/buildroot/master/support/scripts/gen-manual-lists.py", line
216, in __init__
Quoted 2 lines
self.root_config))
File "/opt/buildroot/master/support/scripts/kconfiglib.py", line 214,
in __init__
Quoted 2 lines
self.top_block =
self._parse_file(filename, None, None, None)
File "/opt/src/buildroot/master/support/scripts/kconfiglib.py", line
919, in _parse_file
Quoted 1 line
return self._parse_block(line_feeder, None, parent, deps,
visible_if_deps, res)
Quoted 1 line
File "
/opt/buildroot/master/support/scripts/kconfiglib.py", line 1114,
in _parse_block
Quoted 2 lines
self.base_dir))IOError:
/opt/buildroot/master/Config.in:490: sourced file
"$BR2_EXTERNAL/Config.in" (expands to
Quoted 7 lines
"2_EXTERNAL/Config.in") not found. Perhaps
base_dir(argument toConfig.__init__(), currently "/opt/buildroot/master") is set to the wrong value.docs/manual/manual.mk:2: recipe for target 'manual-update-lists' failed
make: *** [manual-update-lists] Error 1
</snip>
Quoted 6 lines
Reported-by Ryan Barnett <[email protected]>Signed-off-by Samuel Martin <[email protected]>Signed-off-by Thomas Petazzoni <[email protected]>--- support/scripts/kconfiglib.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
PATCH v6 2/5 core: introduce the BR2_EXTERNAL variable
In reply to Thomas PetazzoniThis commit introduces the BR2_EXTERNAL environment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree.
This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in.
Following patches extend the usage of BR2_EXTERNAL to other areas (packages and defconfigs).
In details, this commit:
* Introduces the BR2_EXTERNAL Kconfig option. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of the BR2_EXTERNAL environment variable. The only purpose of this BR2_EXTERNAL Kconfig option is to allow $BR2_EXTERNAL to be properly expanded when used inside Kconfig source statements.
* Calculates the BR2_EXTERNAL value to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass the BR2_EXTERNAL value at each make invocation. If no BR2_EXTERNAL value is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include.
* Passes the BR2_EXTERNAL into the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values.
--- Config.in | 4 ++++ Makefile | 26 +++++++++++++++++++++++++- docs/manual/manual.mk | 3 +++ 3 files changed, 32 insertions(+), 1 deletion(-)
Config.in +4 -0
@@ -14,6 +14,10 @@ config BR2_HOSTARCH string option env="HOSTARCH" +config BR2_EXTERNAL + string + option env="BR2_EXTERNAL" + # Hidden boolean selected by pre-built packages for x86, when they # need to run on x86-64 machines (example: pre-built external # toolchains, binary tools like SAM-BA, etc.).
Makefile +25 -1
@@ -99,6 +99,29 @@ export CDPATH:= BASE_DIR := $(shell mkdir -p $(O) && cd $(O) >/dev/null && pwd) $(if $(BASE_DIR),, $(error output directory "$(O)" does not exist)) + +# Handling of BR2_EXTERNAL. +# +# The value of BR2_EXTERNAL is stored in .br-external in the output directory. +# On subsequent invocations of make, it is read in. It can still be overridden +# on the command line, therefore the file is re-created every time make is run. +# +# When BR2_EXTERNAL is not set, the .br-external file is removed and we point +# to support/dummy-external. This makes sure we can unconditionally include the +# Config.in and external.mk from the BR2_EXTERNAL directory. In this case, +# override is necessary so the user can clear BR2_EXTERNAL from the command +# line, but the dummy path is still used internally. + +BR2_EXTERNAL_FILE = $(BASE_DIR)/.br-external +-include $(BR2_EXTERNAL_FILE) +ifeq ($(BR2_EXTERNAL),) + override BR2_EXTERNAL = support/dummy-external + $(shell rm -f $(BR2_EXTERNAL_FILE)) +else + $(shell echo BR2_EXTERNAL ?= $(BR2_EXTERNAL) > $(BR2_EXTERNAL_FILE)) +endif + + BUILD_DIR:=$(BASE_DIR)/build STAMP_DIR:=$(BASE_DIR)/stamps BINARIES_DIR:=$(BASE_DIR)/images
@@ -632,7 +655,8 @@ COMMON_CONFIG_ENV = \ KCONFIG_AUTOCONFIG=$(BUILD_DIR)/buildroot-config/auto.conf \ KCONFIG_AUTOHEADER=$(BUILD_DIR)/buildroot-config/autoconf.h \ KCONFIG_TRISTATE=$(BUILD_DIR)/buildroot-config/tristate.config \ - BUILDROOT_CONFIG=$(BUILDROOT_CONFIG) + BUILDROOT_CONFIG=$(BUILDROOT_CONFIG) \ + BR2_EXTERNAL=$(BR2_EXTERNAL) xconfig: $(BUILD_DIR)/buildroot-config/qconf outputmakefile @mkdir -p $(BUILD_DIR)/buildroot-config
docs/manual/manual.mk +3 -0
@@ -1,6 +1,9 @@ +# Packages included in BR2_EXTERNAL are not part of buildroot, so they +# should not be included in the manual. manual-update-lists: manual-check-dependencies-lists $(Q)$(call MESSAGE,"Updating the manual lists...") $(Q)BR2_DEFCONFIG="" TOPDIR=$(TOPDIR) O=$(O)/docs/manual/.build \ + BR2_EXTERNAL=$(TOPDIR)/support/dummy-external \ $(TOPDIR)/support/scripts/gen-manual-lists.py # we can't use suitable-host-package here because that's not available in
Signature · 1 line
1.8.1.2
PATCH v6 2/5 core: introduce the BR2_EXTERNAL variable
Thomas P,
Thomas Petazzoni <[email protected]> wrote on 12/04/2013 05:14:48 PM:
Quoted 39 lines
This commit introduces the
BR2_EXTERNALenvironment variable, which will allow to keep Buildroot customization (board-specific configuration files or root filesystem overlays, package Config.in and makefiles, as well as defconfigs) outside of the Buildroot tree.This commit only introduces the variable itself, and ensures that it is available within Config.in options. This allows us to use $BR2_EXTERNAL in a 'source' statement in Config.in.
Following patches extend the usage of
BR2_EXTERNALto other areas (packages and defconfigs).In details, this commit:
* Introduces the
BR2_EXTERNALKconfigoption. This option has no prompt, and is therefore not visible to the user and also not stored in the .config file. It is automatically set to the value of theBR2_EXTERNALenvironment variable. The only purpose of thisBR2_EXTERNALKconfigoption is to allow $BR2_EXTERNAL to be properly expanded when used insideKconfigsource statements.* Calculates the
BR2_EXTERNALvalue to use. If passed on the command line, then this value is taken in priority, and saved to a .br-external hidden file in the output directory. If not passed on the command line, then we read the .br-external file from the output directory. This allows the user to not pass theBR2_EXTERNALvalue at each make invocation. If noBR2_EXTERNALvalue is passed, we define it to support/dummy-external, so that the kconfig code finds an existing $(BR2_EXTERNAL)/package/Config.in file to include.* Passes the
BR2_EXTERNALinto the *config environment, so that its value is found when parsing/evaluating Config.in files and .config values.Signed-off-by Thomas Petazzoni <[email protected]>Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>
Quoted 5 lines
--- Config.in | 4 ++++ Makefile | 26 +++++++++++++++++++++++++- docs/manual/manual.mk | 3 +++ 3 files changed, 32 insertions(+), 1 deletion(-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131205/58dcc03a/attachment.html>
PATCH v6 3/5 core: allow external Config.in/makefile code to be integrated
In reply to Thomas PetazzoniThis commit allows the BR2_EXTERNAL directory to contain Config.in and Makefile code, which gets integrated into the Buildroot build logic:
- Buildroot automatically includes the $BR2_EXTERNAL/Config.in in the top-level configuration menu.
- Buildroot automatically includes the BR2_EXTERNAL/external.mk in the build logic, so it can for example be used to include other .mk files that define package recipes.
This is typically intended to be used to create target packages in the BR2_EXTERNAL directory, but can also be used for bootloaders, host packages, or other custom make logic.
We also add a dummy Config.in file in support/dummy-external/ to ensure that the source "$BR2_EXTERNAL/Config.in" line will point to an existing file even when BR2_EXTERNAL is not used by the user.
--- Config.in | 2 ++ Makefile | 2 ++ support/dummy-external/Config.in | 0 support/dummy-external/external.mk | 0 4 files changed, 4 insertions(+) create mode 100644 support/dummy-external/Config.in create mode 100644 support/dummy-external/external.mk
Config.in +2 -0
@@ -492,3 +492,5 @@ source "boot/Config.in" source "package/Config.in.host" source "Config.in.legacy" + +source "$BR2_EXTERNAL/Config.in"
Makefile +2 -0
@@ -364,6 +364,8 @@ include boot/common.mk include linux/linux.mk include system/system.mk +include $(BR2_EXTERNAL)/external.mk + TARGETS+=target-finalize ifeq ($(BR2_ENABLE_LOCALE_PURGE),y)
support/dummy-external/Config.in new file +0 -0
support/dummy-external/external.mk new file +0 -0
Signature · 1 line
1.8.1.2
PATCH v6 3/5 core: allow external Config.in/makefile code to be integrated
Thomas P,
Thomas Petazzoni <[email protected]> wrote on 12/04/2013 05:14:49 PM:
Quoted 21 lines
This commit allows the
BR2_EXTERNALdirectory to contain Config.in andMakefilecode, which gets integrated into the Buildroot build logic:- Buildroot automatically includes the $BR2_EXTERNAL/Config.in in the top-level configuration menu.
- Buildroot automatically includes the
BR2_EXTERNAL/external.mkin the build logic, so it can for example be used to include other .mk files that define package recipes.This is typically intended to be used to create target packages in the
BR2_EXTERNALdirectory, but can also be used for bootloaders, host packages, or other custom make logic.We also add a dummy Config.in file in support/dummy-external/ to ensure that the source "$BR2_EXTERNAL/Config.in" line will point to an existing file even when
BR2_EXTERNALis not used by the user.Signed-off-by Thomas Petazzoni <[email protected]>Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>
Quoted 8 lines
--- Config.in | 2 ++ Makefile | 2 ++ support/dummy-external/Config.in | 0 support/dummy-external/external.mk | 0 4 files changed, 4 insertions(+) create mode 100644 support/dummy-external/Config.in create mode 100644 support/dummy-external/external.mk
PATCH v6 4/5 core: allow external defconfigs to be used
In reply to Thomas PetazzoniThis commit allows the user to store defconfigs in $BR2_EXTERNAL/configs/. To achieve this:
* It adds a new %_defconfig that looks in $BR2_EXTERNAL/configs/ for the corresponding defconfig file.
* Updates the help target to also list external defconfigs.
--- Makefile | 11 +++++++++++ 1 file changed, 11 insertions(+)
Makefile +11 -0
@@ -742,6 +742,10 @@ defconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @mkdir -p $(BUILD_DIR)/buildroot-config @$(COMMON_CONFIG_ENV) $< --defconfig=$(TOPDIR)/configs/$@ $(CONFIG_CONFIG_IN) +%_defconfig: $(BUILD_DIR)/buildroot-config/conf $(BR2_EXTERNAL)/configs/%_defconfig outputmakefile + @mkdir -p $(BUILD_DIR)/buildroot-config + @$(COMMON_CONFIG_ENV) $< --defconfig=$(BR2_EXTERNAL)/configs/$@ $(CONFIG_CONFIG_IN) + savedefconfig: $(BUILD_DIR)/buildroot-config/conf outputmakefile @mkdir -p $(BUILD_DIR)/buildroot-config @$(COMMON_CONFIG_ENV) $< \
@@ -850,8 +854,15 @@ endif @echo ' make V=0|1 - 0 => quiet build (default), 1 => verbose build' @echo ' make O=dir - Locate all output files in "dir", including .config' @echo + @echo 'Built-in configs:' @$(foreach b, $(sort $(notdir $(wildcard $(TOPDIR)/configs/*_defconfig))), \ printf " %-35s - Build for %s\\n" $(b) $(b:_defconfig=);) +ifneq ($(wildcard $(BR2_EXTERNAL)/configs/*_defconfig),) + @echo + @echo 'User-provided configs:' + @$(foreach b, $(sort $(notdir $(wildcard $(BR2_EXTERNAL)/configs/*_defconfig))), \ + printf " %-35s - Build for %s\\n" $(b) $(b:_defconfig=);) +endif @echo @echo 'See docs/README, or generate the Buildroot manual for further details' @echo
Signature · 1 line
1.8.1.2
PATCH v6 4/5 core: allow external defconfigs to be used
Thomas P,
Thomas Petazzoni <[email protected]> wrote on 12/04/2013 05:14:50 PM:
Quoted 6 lines
This commit allows the user to store defconfigs in $BR2_EXTERNAL/configs/. To achieve this:
* It adds a new %_defconfig that looks in $BR2_EXTERNAL/configs/ for the corresponding defconfig file.
Quoted 8 lines
* Updates the help target to also list external defconfigs.
Signed-off-by Thomas Petazzoni <[email protected]>Reviewed-by "Yann E. MORIN" <[email protected]>Tested-by "Yann E. MORIN" <[email protected]>--- Makefile | 11 +++++++++++ 1 file changed, 11 insertions(+)
PATCH v6 5/5 docs/manual: add explanations about BR2_EXTERNAL
In reply to Thomas PetazzoniThis commit updates the manual to add details on how to use the BR2_EXTERNAL feature.
--- docs/manual/customize-outside-br.txt | 137 +++++++++++++++++++++++++++++++++++ docs/manual/customize.txt | 2 + 2 files changed, 139 insertions(+) create mode 100644 docs/manual/customize-outside-br.txt
docs/manual/customize-outside-br.txt new file +137 -0
@@ -0,0 +1,137 @@ +// -*- mode:doc -*- ; + +[[outside-br-custom]] +Keeping customization outside Buildroot +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Buildroot community recommends and encourages upstreaming to the +official Buildroot version the packages and board supports that are +written by developers. However, it is sometimes not possible or +desirable because some of these packages or board supports are highly +specific or proprietary. + +In this case, Buildroot users are offered two choices: + + * They can add their packages, board support and configuration files + directly within the Buildroot tree, and maintain them by using + branches in a version control system. + + * They can use the +BR2_EXTERNAL+ mechanism, which allows to keep + package recipes, board support and configuration files outside of + the Buildroot tree, while still having them nicely integrated in + the build logic. The following paragraphs give details on how to + use +BR2_EXTERNAL+. + ++BR2_EXTERNAL+ is an environment variable that one can use to point to +a directory that contains Buildroot customizations. It can be passed +to any Buildroot +make+ invocation. It is automatically saved in the +hidden +.br-external+ file in the output directory, so that it is not +needed to pass +BR2_EXTERNAL+ at every +make+ invocation. It can +however be changed at any time by passing a new value, and can be +removed by passing an empty value. + +The +BR2_EXTERNAL+ path can be either an absolute or a relative path, +but if it's passed as a relative path, it is important to note that it +is interpreted relatively to the main Buildroot source directory, not +the Buildroot output directory. + +Some examples: + +----- + buildroot/ $ make BR2_EXTERNAL=../foobar menuconfig +----- + +Starting from now on, external definitions from the +../foobar+ +directory will be used: + +----- + buildroot/ $ make + buildroot/ $ make legal-info +----- + +We can switch to another external definitions directory at any time: + +----- + buildroot/ $ make BR2_EXTERNAL=../barfoo xconfig +----- + +Or disable the usage of external definitions: + +----- + buildroot/ $ make BR2_EXTERNAL= xconfig +----- + +This +BR2_EXTERNAL+ then allows three different things: + + * One can store all the board-specific configuration files there, + such as the kernel configuration, the root filesystem overlay, or + any other configuration file for which Buildroot allows to set its + location. The +BR2_EXTERNAL+ value is available within the + Buildroot configuration using +$(BR2_EXTERNAL)+. As an example, one + could set the +BR2_ROOTFS_OVERLAY+ Buildroot option to + +$(BR2_EXTERNAL)/board/<boardname>/overlay/+ (to specify a root + filesystem overlay), or the +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE+ + Buildroot option to + +$(BR2_EXTERNAL)/board/<boardname>/kernel.config+ (to specify the + location of the kernel configuration file). To achieve this, it is + recommended but not mandatory, to store those details in + directories called +board/<boardname>/+ under +BR2_EXTERNAL+. This + matches the directory structure used within Buildroot. + + * One can store package recipes (i.e +Config.in+ and + +<packagename>.mk+), or even custom configuration options and make + logic. Buildroot automatically includes +BR2_EXTERNAL/Config.in+ to + make it appear in the top-level configuration menu, and includes + +BR2_EXTERNAL/external.mk+ with the rest of the makefile logic. ++ +The main usage of this is to store package recipes. The recommended + way to do this is to write a +BR2_EXTERNAL/Config.in+ that looks + like: ++ +------ +menu "<somecompany> packages" + +source "$BR2_EXTERNAL/package/package1/Config.in" +source "$BR2_EXTERNAL/package/package2/Config.in" + +endmenu +------ ++ +Then, have a +BR2_EXTERNAL/external.mk+ file that looks like: ++ +------ +include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk)) +------ ++ +And then in +BR2_EXTERNAL/package/package1+ and + +BR2_EXTERNAL/package/package2+ create normal Buildroot package + recipes, as explained in xref:adding-packages[]. + + * One can store Buildroot defconfigs in the +configs+ subdirectory of + +BR2_EXTERNAL+. Buildroot will automatically show them in the + output of +make help+ and allow them to be loaded with the normal + +make <name>_defconfig+ command. + +In the end, a typical +BR2_EXTERNAL+ directory organization would +generally be: + +----- +$(BR2_EXTERNAL)/ ++-- Config.in ++-- external.mk ++-- board/ +| +-- <boardname>/ +| +-- linux.config +| +-- overlay/ +| +-- etc/ +| +-- <some file> ++-- configs/ +| +-- <boardname>_defconfig ++-- package/ + +-- package1/ + | +-- Config.in + | +-- package1.mk + +-- package2/ + +-- Config.in + +-- package2.mk +------
docs/manual/customize.txt +2 -0
@@ -17,3 +17,5 @@ include::customize-toolchain.txt[] include::customize-store.txt[] include::customize-packages.txt[] + +include::customize-outside-br.txt[]
Signature · 1 line
1.8.1.2
PATCH v6 5/5 docs/manual: add explanations about BR2_EXTERNAL
Thomas,
Thomas Petazzoni <[email protected]> wrote on 12/04/2013 05:14:51 PM:
Quoted 7 lines
This commit updates the manual to add details on how to use the
BR2_EXTERNALfeature.Signed-off-by Thomas Petazzoni <[email protected]>Reviewed-by "Yann E. MORIN" <[email protected]>--- docs/manual/customize-outside-br.txt | 137
+++++++++++++++++++++++++++++++++++
Review
Quoted 5 lines
docs/manual/customize.txt | 2 + 2 files changed, 139 insertions(+) create mode 100644 docs/manual/customize-outside-br.txt(quoted diff) +0
-0
b/docs/manual/customize-outside-br.txt
Quoted 20 lines
new file mode 100644
index 0000000..4069834docs/manual/customize-outside-br.txt new file +15
-0@@ -0,0 +1,137 @@ +// -*- mode:doc -*- ; + +[[outside-br-custom]] +Keeping customization outside Buildroot +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Buildroot community recommends and encourages upstreaming to the +official Buildroot version the packages and board supports that are +written by developers. However, it is sometimes not possible or +desirable because some of these packages or board supports are highly +specific or proprietary. + +In this case, Buildroot users are offered two choices: + + * They can add their packages, board support and configuration files
docs/manual/customize-outside-br.txt line 25
Quoted 10 lines
docs/manual/customize-outside-br.txt new file +10
-0+ directly within the Buildroot tree, and maintain them by using + branches in a version control system. + + * They can use the +BR2_EXTERNAL+ mechanism, which allows to keep + package recipes, board support and configuration files outside of + the Buildroot tree, while still having them nicely integrated in + the build logic. The following paragraphs give details on how to + use +BR2_EXTERNAL+. + ++BR2_EXTERNAL+ is an environment variable that one can use to point to
I would remove the 'one' from the above line. So it would read:
+BR2_EXTERNAL+ is an environment variable that can be used to point to
docs/manual/customize-outside-br.txt line 29
Quoted 4 lines
docs/manual/customize-outside-br.txt new file +4
-0+a directory that contains Buildroot customizations. It can be passed +to any Buildroot +make+ invocation. It is automatically saved in the +hidden +.br-external+ file in the output directory, so that it is not +needed to pass +BR2_EXTERNAL+ at every +make+ invocation. It can
I would consider rewording the below sentence:
It is automatically saved in the hidden +.br-external+ file in the output directory, so that it is not needed to pass +BR2_EXTERNAL+ at every +make+ invocation.
to be:
It is automatically saved in the hidden +.br-external+ file in the output directory. By doing this, there is no need to pass +BR2_EXTERNAL+ at every +make+ invocation.
Quoted 25 lines
docs/manual/customize-outside-br.txt new file +25
-0+however be changed at any time by passing a new value, and can be +removed by passing an empty value. + +The +BR2_EXTERNAL+ path can be either an absolute or a relative path, +but if it's passed as a relative path, it is important to note that it +is interpreted relatively to the main Buildroot source directory, not +the Buildroot output directory. + +Some examples: + +----- + buildroot/ $ make BR2_EXTERNAL=../foobar menuconfig +----- + +Starting from now on, external definitions from the +../foobar+ +directory will be used: + +----- + buildroot/ $ make + buildroot/ $ make legal-info +----- + +We can switch to another external definitions directory at any time: + +-----
docs/manual/customize-outside-br.txt line 64
Quoted 10 lines
docs/manual/customize-outside-br.txt new file +10
-0+ buildroot/ $ make BR2_EXTERNAL=../barfoo xconfig +----- + +Or disable the usage of external definitions: + +----- + buildroot/ $ make BR2_EXTERNAL= xconfig +----- + +This +BR2_EXTERNAL+ then allows three different things:
Remove 'This' from the previous line.
Quoted 39 lines
docs/manual/customize-outside-br.txt new file +39
-0+ + * One can store all the board-specific configuration files there, + such as the kernel configuration, the root filesystem overlay, or + any other configuration file for which Buildroot allows to set its + location. The +BR2_EXTERNAL+ value is available within the + Buildroot configuration using +$(BR2_EXTERNAL)+. As an example, one + could set the +BR2_ROOTFS_OVERLAY+ Buildroot option to + +$(BR2_EXTERNAL)/board/<boardname>/overlay/+ (to specify a root + filesystem overlay), or the +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE+ + Buildroot option to + +$(BR2_EXTERNAL)/board/<boardname>/kernel.config+ (to specify the + location of the kernel configuration file). To achieve this, it is + recommended but not mandatory, to store those details in + directories called +board/<boardname>/+ under +BR2_EXTERNAL+. This + matches the directory structure used within Buildroot. + + * One can store package recipes (i.e +Config.in+ and + +<packagename>.mk+), or even custom configuration options and make + logic. Buildroot automatically includes +BR2_EXTERNAL/Config.in+ to + make it appear in the top-level configuration menu, and includes + +BR2_EXTERNAL/external.mk+ with the rest of the makefile logic. ++ +The main usage of this is to store package recipes. The recommended + way to do this is to write a +BR2_EXTERNAL/Config.in+ that looks + like: ++ +------ +menu "<somecompany> packages" + +source "$BR2_EXTERNAL/package/package1/Config.in" +source "$BR2_EXTERNAL/package/package2/Config.in" + +endmenu +------ ++ +Then, have a +BR2_EXTERNAL/external.mk+ file that looks like: ++ +------ +include $(sort $(wildcard $(BR2_EXTERNAL)/package/*/*.mk))
docs/manual/customize-outside-br.txt line 113
Quoted 10 lines
docs/manual/customize-outside-br.txt new file +10
-0+------ ++ +And then in +BR2_EXTERNAL/package/package1+ and + +BR2_EXTERNAL/package/package2+ create normal Buildroot package + recipes, as explained in xref:adding-packages[]. + + * One can store Buildroot defconfigs in the +configs+ subdirectory of + +BR2_EXTERNAL+. Buildroot will automatically show them in the + output of +make help+ and allow them to be loaded with the normal + +make <name>_defconfig+ command.
Maybe add a comment that they are shown under 'User-provided configs:'
Thanks,
-Ryan
[...]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131205/8acf2d7b/attachment.html>