core: enhance printvars for variables with newlines
PATCH 1/1 core: enhance printvars for variables with newlines
If the variable content has newlines in it then the currently dumped content can't be fed again to GNU make. Add the option DEFINE_VARS which causes the variables to be dumped using
define VAR
... line 1 ...
... line 2 ...
...
endef
Updated the manual accordingly.
--- Makefile | 10 +++++++--- docs/manual/make-tips.txt | 20 ++++++++++++++++++++ 2 files changed, 27 insertions(+), 3 deletions(-)
Makefile +7 -3
@@ -974,9 +974,13 @@ printvars: $(sort $(if $(VARS),$(filter $(VARS),$(.VARIABLES)),$(.VARIABLES))), \ $(if $(filter-out environment% default automatic, \ $(origin $V)), \ - $(if $(QUOTED_VARS),\ - $(info $V='$(subst ','\'',$(if $(RAW_VARS),$(value $V),$($V)))'), \ - $(info $V=$(if $(RAW_VARS),$(value $V),$($V)))))) + $(if $(DEFINE_VARS), \ + $(info define $V) \ + $(info $(if $(RAW_VARS),$(value $V),$($V))) \ + $(info endef), \ + $(if $(QUOTED_VARS),\ + $(info $V='$(subst ','\'',$(if $(RAW_VARS),$(value $V),$($V)))'), \ + $(info $V=$(if $(RAW_VARS),$(value $V),$($V))))))) # ' Syntax colouring... .PHONY: clean
docs/manual/make-tips.txt +20 -0
@@ -92,6 +92,8 @@ It is possible to tweak the output using some variables: - +VARS+ will limit the listing to variables which names match the specified make-pattern +- +DEFINE_VARS+, if set to +YES+, will use define...endef to preserve + newlines in the value - +QUOTED_VARS+, if set to +YES+, will single-quote the value - +RAW_VARS+, if set to +YES+, will print the unexpanded value
@@ -106,6 +108,24 @@ For example: BUSYBOX_RDEPENDENCIES=ncurses util-linux ---- +---- + $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES DEFINE_VARS=YES + define BUSYBOX_DEPENDENCIES + skeleton toolchain + endef + define BUSYBOX_FINAL_ALL_DEPENDENCIES + skeleton toolchain + endef + define BUSYBOX_FINAL_DEPENDENCIES + skeleton toolchain + endef + define BUSYBOX_FINAL_PATCH_DEPENDENCIES + endef + define BUSYBOX_RDEPENDENCIES + ncurses util-linux'linux-pam skeleton toolchain host-skeleton host-ccache + endef +---- + ---- $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES QUOTED_VARS=YES BUSYBOX_DEPENDENCIES='skeleton toolchain'
Signature · 1 line
2.14.3
PATCH 1/1 core: enhance printvars for variables with newlines
Hello,
Quoted 11 lines · Stefan Becker
If the variable content has newlines in it then the currently dumped content can't be fed again to GNU make. Add the option
DEFINE_VARSwhich causes the variables to be dumped usingdefine VAR ... line 1 ... ... line 2 ... ... endefUpdated the manual accordingly.
Just curious, what is the use case for feeding the printvars output back into GNU Make ?
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 core: enhance printvars for variables with newlines
Hi,
Quoted 3 lines · Thomas Petazzoni
Just curious, what is the use case for feeding the printvars output back into GNU Make ?
Short answer: because we have factored out buildroot image creation step to a separate build.
Previously it was enough to provide the SDK, output/target/ and a selected list of source *.mk file s from the "upstream" buildroot build to be able to do that. With todays @master update the root fs creation requires internal variables, like PACKAGES_USERS, to be set correctly to create a valid, bootable image. Which took me a while to root cause...
Hence we need to use "printvars" to extract the content of those internal variables from the "upstream" buildroot build into a .mk file that is passed down to the "downstream" image build, which include's it in its top-level Makefile (a stripped-down version of buildroot top-level Makefile). The variables we are interested in unfortunately contain newlines.
Regards, Stefan
PATCH 1/1 core: enhance printvars for variables with newlines
Stefan, All,
On 2018-04-03 18:15 +0300, Stefan Becker spake thusly:
Quoted 11 lines
Hi,
Quoted 3 lines · Thomas Petazzoni
Just curious, what is the use case for feeding the printvars output back into GNU Make ?
Short answer: because we have factored out buildroot image creation step to a separate build.
Quoted 6 lines
Previously it was enough to provide the SDK, output/target/ and a selected list of source *.mk file s from the "upstream" buildroot build to be able to do that. With todays @master update the root fs creation requires internal variables, like
PACKAGES_USERS, to be set correctly to create a valid, bootable image. Which took me a while to root cause...
The real solution IMHO is that you base your iamge creatikon out of the rootfs.tar image generated by Buildroot.
If you are using a br2-external tree, you can also define your own filesystem iamge type that depends on rootfs.tar. Then your image generator (or a wrapper around that) would extract (under fakeroot) rootfs.tar, and generate your own image format out of that.
(Note: this is not a reason not to apply your patch, of course, just an alternate solution for your use-case).
Regards,
Yann E. MORIN.
Quoted 8 lines
Hence we need to use "printvars" to extract the content of those internal variables from the "upstream" buildroot build into a .mk file that is passed down to the "downstream" image build, which include's it in its top-level
Makefile(a stripped-down version of buildroot top-levelMakefile). The variables we are interested in unfortunately contain newlines.Regards, Stefan
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 core: enhance printvars for variables with newlines
Hi,
Quoted 3 lines · Yann E. MORIN
The real solution IMHO is that you base your iamge creatikon out of the rootfs.tar image generated by Buildroot.
I'm not sure that is feasible, because then you would loose access to things that need to be done inside fakeroot (f.ex.).
Plus @master the new rootfs-common.tar has been added, but the build deletes it again and thus it can't be exported as build artifact.
If you are using a br2-external tree, you can also define your own
Quoted 4 lines
filesystem iamge type that depends on rootfs.tar. Then your image generator (or a wrapper around that) would extract (under fakeroot) rootfs.tar, and generate your own image format out of that.
...which would mean we are back in the slow monolithic meta build system with our components. Sorry, but that won't do :-)
BTW: we do have a br2-external tree but it is basically "pure". It is only needed so that we do not pollute the buildroot tree with things that are needed to run the buildroot build system and create the build artifacts afterwards.
Regards, Stefan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180403/5cfb7e63/attachment.html>
PATCH 1/1 core: enhance printvars for variables with newlines
Stefan, All,
On 2018-04-03 19:37 +0300, Stefan Becker spake thusly:
Quoted 7 lines
Hi,
On Tue, Apr 3, 2018 at 7:21 PM, Yann E. MORIN < [1][email protected]> wrote:The real solution IMHO is that you base your iamge creatikon out of the rootfs.tar image generated by Buildroot.
I'm not sure that is feasible, because then you would loose access to things that need to be done inside fakeroot (f.ex.).
Of course, as I wrote below, your script that does the work woudl need to run under fakeroot, obviously. This can be easily achieved with a snippet like that:
#!/bin/sh
if [ $(id -u) -ne 0 ]; then
fakeroot "${0]" "${@}"
exit ${?}
fi
# here goes your real filesystem code...Quoted 2 lines
Plus @master the new rootfs-common.tar has been added, but the build deletes it again and thus it can't be exported as build artifact.
Beause this is only an internal temporary file that you should not be concerned with. Besides, it is not even totaly complete either...
I was talking about the *real* final rootfs.tar that you get with BR2_TARGET_ROOTFS_TAR.
Quoted 7 lines
If you are using a br2-external tree, you can also define your own filesystem iamge type that depends on rootfs.tar. Then your image generator (or a wrapper around that) would extract (under fakeroot) rootfs.tar, and generate your own image format out of that.
...which would mean we are back in the slow monolithic meta build system with our components. Sorry, but that won't do :-)
Of course, if you are trying to fast-track Buildroot with local hacks, you are back on your own...
Quoted 2 lines
BTW: we do have a br2-external tree but it is basically "pure". It is only needed so that we do not pollute the buildroot tree with things that are needed to run the buildroot build system and create the build artifacts afterwards.
Sorry, I don't understand what you mean by "pure". There is no "purity" to talk of about a br2-external tree: by definition, you put in there whatever you want. You probably meant that you are not touching the buildroot tree. Yes, I understand. But nothing prevents you from putting your own filesystem (or rather, image) generation tool as a new filesystem type. See for example what I explained at ELCE last year:
https://elinux.org/images/8/8e/2017-10-24_-_ELCE-Buildroot.pdf (slide 16) https://youtu.be/SN2hYO2rYtk?t=728
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 core: enhance printvars for variables with newlines
In reply to Stefan BeckerPing?
Is there anything left which prevents this from getting merged to master?
Regards, Stefan
Quoted 80 lines · Stefan Becker
If the variable content has newlines in it then the currently dumped content can't be fed again to GNU make. Add the option
DEFINE_VARSwhich causes the variables to be dumped usingdefine VAR ... line 1 ... ... line 2 ... ... endefUpdated the manual accordingly.
Signed-off-by Stefan Becker <[email protected]>--- Makefile | 10 +++++++--- docs/manual/make-tips.txt | 20 ++++++++++++++++++++ 2 files changed, 27 insertions(+), 3 deletions(-)Makefile +7
-3@@ -974,9 +974,13 @@ printvars: $(sort $(if $(VARS),$(filter $(VARS),$(.VARIABLES)),$(.VARIABLES))), \ $(if $(filter-out environment% default automatic, \ $(origin $V)), \ - $(if $(QUOTED_VARS),\ - $(info $V='$(subst ','\'',$(if $(RAW_VARS),$(value $V),$($V)))'), \ - $(info $V=$(if $(RAW_VARS),$(value $V),$($V)))))) + $(if $(DEFINE_VARS), \ + $(info define $V) \ + $(info $(if $(RAW_VARS),$(value $V),$($V))) \ + $(info endef), \ + $(if $(QUOTED_VARS),\ + $(info $V='$(subst ','\'',$(if $(RAW_VARS),$(value $V),$($V)))'), \ + $(info $V=$(if $(RAW_VARS),$(value $V),$($V))))))) # ' Syntax colouring... .PHONY: cleandocs/manual/make-tips.txt +20
-1@@ -92,6 +92,8 @@ It is possible to tweak the output using some variables: - +VARS+ will limit the listing to variables which names match the specified make-pattern +- +DEFINE_VARS+, if set to +YES+, will use define...endef to preserve + newlines in the value - +QUOTED_VARS+, if set to +YES+, will single-quote the value - +RAW_VARS+, if set to +YES+, will print the unexpanded value@@ -106,6 +108,24 @@ For example: BUSYBOX_RDEPENDENCIES=ncurses util-linux ---- +---- + $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES DEFINE_VARS=YES + define BUSYBOX_DEPENDENCIES + skeleton toolchain + endef + define BUSYBOX_FINAL_ALL_DEPENDENCIES + skeleton toolchain + endef + define BUSYBOX_FINAL_DEPENDENCIES + skeleton toolchain + endef + define BUSYBOX_FINAL_PATCH_DEPENDENCIES + endef + define BUSYBOX_RDEPENDENCIES + ncurses util-linux'linux-pam skeleton toolchain host-skeleton host-ccache + endef +---- + ---- $ make -s printvars VARS=BUSYBOX_%DEPENDENCIES QUOTED_VARS=YES BUSYBOX_DEPENDENCIES='skeleton toolchain' --2.14.3