Pull request for branch yem/instrument-build
From: "Yann E. MORIN" <[email protected]>
Hello All!
This series is an RFC to add support for instrumenting the build process, reworked after discussion both on the ML and during the developpers' day.
It works as thus;
- we register a set of hooks
- hooks are called before and after each steps
There are currently two hooks defined:
- log steps with timings (as an example of hos to do a hook)
- call a user-supplied script
This is just a RFC for now, and if it is accepted, new hooks can be defined later on.
I'm planning on adding at least one other hook, that runs all scripts present in support/step-hooks/ (or whatever its name).
Open for comments! ;-)
Changes v1 -> v2:
- removed the intra-steps instrumentation (Thomas, Peter)
- accept a list of user-supplied hooks, not only one hook (Thomas)
Regards,
Yann E. MORIN.
The following changes since commit 200bf74e8e4686c3f132a6600ae6803eda8b588a:
Update config.guess and config.sub scripts to latest versions. (2013-10-27 15:29:38 +0100)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/instrument-build
for you to fetch changes up to a4b9a0c71f89f16b00a75ba0b37f19c71c0300c3:
pkg-infra: add user-supplied step-hooks (2013-10-27 16:51:34 +0000)
----------------------------------------------------------------
Yann E. MORIN (4):
post-{build,images} hooks: export BUILD_DIR too
pkg-infra: introduce pre/post-step hooks
pkg-infra: add hook to log timing of steps
pkg-infra: add user-supplied step-hooks
Makefile | 4 +-- docs/manual/customize-rootfs.txt | 5 ++-- docs/manual/debugging-buildroot.txt | 33 +++++++++++++++++++++ docs/manual/developer-guide.txt | 2 ++ package/Makefile.in | 5 ++++ package/pkg-generic.mk | 59 +++++++++++++++++++++++++++++++++++++ 6 files changed, 104 insertions(+), 4 deletions(-) create mode 100644 docs/manual/debugging-buildroot.txt
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/4 post-{build, images} hooks: export BUILD_DIR too
From: "Yann E. MORIN" <[email protected]>
Also export BUILD_DIR for post-{build,images} hooks, so they do have a place to store generated files.
Note: this will be more einteresting for the instrumentation of steps, to come in a later patch.
--- Makefile | 4 ++-- docs/manual/customize-rootfs.txt | 5 +++-- package/Makefile.in | 5 +++++ 3 files changed, 10 insertions(+), 4 deletions(-)
Makefile +2 -2
@@ -511,7 +511,7 @@ endif @$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_BUILD_SCRIPT)), \ $(call MESSAGE,"Executing post-build script $(s)"); \ - $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) + $(USER_HOOKS_EXTRA_ENV) $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) ifeq ($(BR2_ENABLE_LOCALE_PURGE),y) LOCALE_WHITELIST=$(BUILD_DIR)/locales.nopurge
@@ -557,7 +557,7 @@ endif target-post-image: @$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \ $(call MESSAGE,"Executing post-image script $(s)"); \ - $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) + $(USER_HOOKS_EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) toolchain-eclipse-register: ./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)
docs/manual/customize-rootfs.txt +3 -2
@@ -41,6 +41,7 @@ there are a few ways to customize the resulting target filesystem. - +BUILDROOT_CONFIG+: the path to the Buildroot .config file - +HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+: see xref:generic-package-reference[] + - +BUILD_DIR+: the directory where packages are extracted and built - +BINARIES_DIR+: the place where all binary files (aka images) are stored - +BASE_DIR+: the base output directory
@@ -78,8 +79,8 @@ in one of these _post-image scripts_ will require special handling Just like for the _post-build scripts_ mentioned above, you also have access to the following environment variables from your _post-image -scripts_: +BUILDROOT_CONFIG+, +HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+, -+BINARIES_DIR+ and +BASE_DIR+. +scripts_: +BUILDROOT_CONFIG+, +BUILD_DIR+, +HOST_DIR+, +STAGING_DIR+, ++TARGET_DIR+, +BINARIES_DIR+ and +BASE_DIR+. Additionally, each of the +BR2_ROOTFS_POST_BUILD_SCRIPT+ and +BR2_ROOTFS_POST_IMAGE_SCRIPT+ scripts will be passed the arguments
package/Makefile.in +5 -0
@@ -275,6 +275,11 @@ HOST_MAKE_ENV=PATH=$(HOST_PATH) \ PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig" \ PERLLIB="$(HOST_DIR)/usr/lib/perl" +# This extra environment we can not export ourselves, so we have +# to expicitly pass it to user-supplied external hooks (eg. +# post-build, post-images) +USER_HOOKS_EXTRA_ENV=\ + BUILD_DIR=$(BUILD_DIR) ################################################################################ # settings we need to pass to configure
Signature · 1 line
1.8.1.2
PATCH 2/4 pkg-infra: introduce pre/post-step hooks
From: "Yann E. MORIN" <[email protected]>
This hooks will let us instrument the build process in many ways:
- log current step to see what broke
- time each step to see what is worth optimising
- sanity-check installed files (rpath, overwritten files...)
- call user-provided script
- ...
The steps are coarse-grain, and all have a 'start' and a 'end' hooks. Here is the list of available steps (8 total):
- extract
- patch
- configure
- build
- install-host
- install-staging
- install-image
- install-target
The download, clean and uninstall steps are not instrumented on purpose.
--- package/pkg-generic.mk | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
package/pkg-generic.mk +39 -0
@@ -21,6 +21,29 @@ ################################################################################ ################################################################################ +# Helper functions to catch start/end of each steps +################################################################################ + +# Those two functions are called by each steps below. +# They are responsible for calling all hooks defined in $(STEP_HOOKS) +# and pass each of them two arguments: +# $1: either 'start' or 'end' +# $2: the name of the step +# $3: the name of the package + +# Start step +# $1: step name +define step_start + $(foreach hook,$(STEP_HOOKS),$(call $(hook),start,$(1),$($(PKG)_NAME))$(sep)) +endef + +# End step +# $1: step name +define step_end + $(foreach hook,$(STEP_HOOKS),$(call $(hook),end,$(1),$($(PKG)_NAME))$(sep)) +endef + +################################################################################ # Implicit targets -- produce a stamp file for each step of a package build ################################################################################
@@ -55,6 +78,7 @@ endif # Unpack the archive $(BUILD_DIR)/%/.stamp_extracted: + @$(call step_start,extract) @$(call MESSAGE,"Extracting") $(Q)mkdir -p $(@D) $($(PKG)_EXTRACT_CMDS)
@@ -62,6 +86,7 @@ $(BUILD_DIR)/%/.stamp_extracted: $(Q)chmod -R +rw $(@D) $(foreach hook,$($(PKG)_POST_EXTRACT_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + @$(call step_end,extract) # Rsync the source directory if the <pkg>_OVERRIDE_SRCDIR feature is # used.
@@ -90,6 +115,7 @@ endif $(BUILD_DIR)/%/.stamp_patched: NAMEVER = $(RAWNAME)-$($(PKG)_VERSION) $(BUILD_DIR)/%/.stamp_patched: PATCH_BASE_DIRS = $($(PKG)_DIR_PREFIX)/$(RAWNAME) $(call qstrip,$(BR2_GLOBAL_PATCH_DIR))/$(RAWNAME) $(BUILD_DIR)/%/.stamp_patched: + @$(call step_start,patch) @$(call MESSAGE,"Patching $($(PKG)_DIR_PREFIX)/$(RAWNAME)") $(foreach hook,$($(PKG)_PRE_PATCH_HOOKS),$(call $(hook))$(sep)) $(foreach p,$($(PKG)_PATCH),support/scripts/apply-patches.sh $(@D) $(DL_DIR) $(notdir $(p))$(sep))
@@ -106,31 +132,39 @@ $(BUILD_DIR)/%/.stamp_patched: ) $(foreach hook,$($(PKG)_POST_PATCH_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + @$(call step_end,patch) # Configure $(BUILD_DIR)/%/.stamp_configured: + @$(call step_start,configure) $(foreach hook,$($(PKG)_PRE_CONFIGURE_HOOKS),$(call $(hook))$(sep)) @$(call MESSAGE,"Configuring") $($(PKG)_CONFIGURE_CMDS) $(foreach hook,$($(PKG)_POST_CONFIGURE_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + @$(call step_end,configure) # Build $(BUILD_DIR)/%/.stamp_built:: + @$(call step_start,build) @$(call MESSAGE,"Building") $($(PKG)_BUILD_CMDS) $(foreach hook,$($(PKG)_POST_BUILD_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + @$(call step_end,build) # Install to host dir $(BUILD_DIR)/%/.stamp_host_installed: + @$(call step_start,install-host) @$(call MESSAGE,"Installing to host directory") $($(PKG)_INSTALL_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + @$(call step_end,install-host) # Install to staging dir $(BUILD_DIR)/%/.stamp_staging_installed: + @$(call step_start,install-staging) @$(call MESSAGE,"Installing to staging directory") $($(PKG)_INSTALL_STAGING_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep))
@@ -142,16 +176,20 @@ $(BUILD_DIR)/%/.stamp_staging_installed: $(addprefix $(STAGING_DIR)/usr/bin/,$($(PKG)_CONFIG_SCRIPTS)) ;\ fi $(Q)touch $@ + @$(call step_end,install-staging) # Install to images dir $(BUILD_DIR)/%/.stamp_images_installed: + @$(call step_start,install-image) @$(call MESSAGE,"Installing to images directory") $($(PKG)_INSTALL_IMAGES_CMDS) $(foreach hook,$($(PKG)_POST_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ + @$(call step_end,install-image) # Install to target dir $(BUILD_DIR)/%/.stamp_target_installed: + @$(call step_start,install-target) @$(call MESSAGE,"Installing to target") $(if $(BR2_INIT_SYSTEMD),\ $($(PKG)_INSTALL_INIT_SYSTEMD))
@@ -163,6 +201,7 @@ $(BUILD_DIR)/%/.stamp_target_installed: $(RM) -f $(addprefix $(TARGET_DIR)/usr/bin/,$($(PKG)_CONFIG_SCRIPTS)) ; \ fi $(Q)touch $@ + @$(call step_end,install-target) # Clean package $(BUILD_DIR)/%/.stamp_cleaned:
Signature · 1 line
1.8.1.2
PATCH 2/4 pkg-infra: introduce pre/post-step hooks
Quoted 32 lines · Yann E. MORIN
From: "Yann E. MORIN" <[email protected]>
This hooks will let us instrument the build process in many ways:
- log current step to see what broke
- time each step to see what is worth optimising
- sanity-check installed files (rpath, overwritten files...)
- call user-provided script
- ...The steps are coarse-grain, and all have a 'start' and a 'end' hooks. Here is the list of available steps (8 total):
- extract
- patch
- configure
- build
- install-host
- install-staging
- install-image
- install-targetThe download, clean and uninstall steps are not instrumented on purpose.
Signed-off-by "Yann E. MORIN" <[email protected]>---
package/pkg-generic.mk| 39 +++++++++++++++++++++++++++++++++++++++1 file changed, 39 insertions(+)package/pkg-generic.mk +0
-0
package/pkg-generic.mk line 24
Quoted 4 lines
package/pkg-generic.mk +1
-0@@ -21,6 +21,29 @@ ################################################################################ ################################################################################ +# Helper functions to catch start/end of each steps
each step
package/pkg-generic.mk line 27
Quoted 3 lines
package/pkg-generic.mk +3
-0+################################################################################ + +# Those two functions are called by each steps below.
each step
package/pkg-generic.mk line 28
Quoted 1 line
package/pkg-generic.mk +1
-0+# They are responsible for calling all hooks defined in $(STEP_HOOKS)
I personally find the name STEP_HOOKS pretty vague, if you don't know the context you have no clue what it could be. The other hooks POST_EXTRACT etc are clear from their name. Maybe, GLOBAL_INSTRUMENTATION_HOOKS?
Best regards,
Thomas
PATCH 2/4 pkg-infra: introduce pre/post-step hooks
Thomas D, Yann,
Thomas De Schampheleire <[email protected]> wrote on 10/27/2013 04:40:09 PM:
Quoted 1 line
Quoted 1 line
################################################################################
Quoted 2 lines
Quoted 2 lines
################################################################################
Quoted 6 lines
Quoted 1 line
+# Helper functions to catch start/end of each steps
each step
+################################################################################
Quoted 8 lines
package/pkg-generic.mk line 27
Quoted 2 lines
package/pkg-generic.mk +2
-0+ +# Those two functions are called by each steps below.each step
Quoted 1 line
+# They are responsible for calling all hooks defined in $(STEP_HOOKS)
I personally find the name
STEP_HOOKSpretty vague, if you don't know
the
Quoted 1 line
context you have no clue what it could be. The other hooks
POST_EXTRACT
etc
Quoted 3 lines
are clear from their name.
Maybe,
GLOBAL_INSTRUMENTATION_HOOKS?
But that is so long to type :)
Thanks,
-Ryan
PATCH 2/4 pkg-infra: introduce pre/post-step hooks
In reply to Thomas De SchampheleireOn 2013-10-27 22:40 +0100, Thomas De Schampheleire spake thusly:
Quoted 3 lines
Quoted 2 lines · Yann E. MORIN
From: "Yann E. MORIN" <[email protected]>
This hooks will let us instrument the build process in many ways:
[--SNIP--]
Quoted 12 lines
package/pkg-generic.mk line 24
Quoted 9 lines
package/pkg-generic.mk +1
-0@@ -21,6 +21,29 @@ ################################################################################ ################################################################################ +# Helper functions to catch start/end of each stepseach step
Quoted 5 lines
package/pkg-generic.mk line 27
Quoted 3 lines
package/pkg-generic.mk +3
-0+################################################################################ + +# Those two functions are called by each steps below.each step
Both: OK.
Quoted 6 lines
package/pkg-generic.mk line 28
Quoted 1 line
package/pkg-generic.mk +1
-0+# They are responsible for calling all hooks defined in $(STEP_HOOKS)I personally find the name
STEP_HOOKSpretty vague, if you don't know the context you have no clue what it could be. The other hooksPOST_EXTRACTetc are clear from their name.
Maybe,GLOBAL_INSTRUMENTATION_HOOKS?
I'm on the side of Ryan here: it is way too long. :-p I'll try to see if I can come with a better name, though.
Thanks for the review!
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 2/4 pkg-infra: introduce pre/post-step hooks
Hi Yann, Ryan,
Quoted 10 lines · Yann E. MORIN
On 2013-10-27 22:40 +0100, Thomas De Schampheleire spake thusly:
Quoted 4 lines
I personally find the name
STEP_HOOKSpretty vague, if you don't know the context you have no clue what it could be. The other hooksPOST_EXTRACTetc are clear from their name.
Maybe,GLOBAL_INSTRUMENTATION_HOOKS?I'm on the side of Ryan here: it is way too long. :-p I'll try to see if I can come with a better name, though.
I'm fine with looking at other names, of course, but I don't see how GLOBAL_INSTRUMENTATION_HOOKS
is longer thanFOO_POST_INSTALL_STAGING_HOOKS FOO_POST_INSTALL_TARGET_HOOKS
orCONNTRACK_TOOLS_PRE_CONFIGURE_HOOKS
Besides, it's not that you're writing this so regularly that the length is really so important, IMHO.
Best regards,
Thomas
PATCH 2/4 pkg-infra: introduce pre/post-step hooks
Thomas DS, All,
On 2013-10-29 21:03 +0100, Thomas De Schampheleire spake thusly:
Quoted 22 lines
Hi Yann, Ryan,
Quoted 10 lines · Yann E. MORIN
On 2013-10-27 22:40 +0100, Thomas De Schampheleire spake thusly:
Quoted 4 lines
I personally find the name
STEP_HOOKSpretty vague, if you don't know the context you have no clue what it could be. The other hooksPOST_EXTRACTetc are clear from their name.
Maybe,GLOBAL_INSTRUMENTATION_HOOKS?I'm on the side of Ryan here: it is way too long. :-p I'll try to see if I can come with a better name, though.
I'm fine with looking at other names, of course, but I don't see how
GLOBAL_INSTRUMENTATION_HOOKS
is longer thanFOO_POST_INSTALL_STAGING_HOOKSFOO_POST_INSTALL_TARGET_HOOKS
orCONNTRACK_TOOLS_PRE_CONFIGURE_HOOKS
Quoted 2 lines
Besides, it's not that you're writing this so regularly that the length is really so important, IMHO.
Yeah, yeah, right. :-)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 3/4 pkg-infra: add hook to log timing of steps
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
The timing information is stored in the file $(O)/build-time.log
--- package/pkg-generic.mk | 11 +++++++++++ 1 file changed, 11 insertions(+)
package/pkg-generic.mk +11 -0
@@ -43,6 +43,17 @@ define step_end $(foreach hook,$(STEP_HOOKS),$(call $(hook),end,$(1),$($(PKG)_NAME))$(sep)) endef +####################################### +# Actual steps hooks + +# Time steps +define step_time + printf "%s:%-5.5s:%-20.20s: %s\n" \ + "$$(date +%s)" "$(1)" "$(2)" "$(3)" \ + >>"$(BUILD_DIR)/build-time.log" +endef +STEP_HOOKS += step_time + ################################################################################ # Implicit targets -- produce a stamp file for each step of a package build ################################################################################
Signature · 1 line
1.8.1.2
PATCH 4/4 pkg-infra: add user-supplied step-hooks
In reply to Yann E. MORINFrom: "Yann E. MORIN" <[email protected]>
Allow user to supply their own step-hooks by passing a variable on the make command-line:
make BR2_STEP_USER_HOOK=/path/to/my/script
This can be usefull to run site-specific actions at each step of the build process, such as logging installed, removed or modified files, do sanity checks on installed files...
It is possible to call more than one script, by passing a space-separated lists of scripts to call.
--- docs/manual/debugging-buildroot.txt | 33 +++++++++++++++++++++++++++++++++ docs/manual/developer-guide.txt | 2 ++ package/pkg-generic.mk | 9 +++++++++ 3 files changed, 44 insertions(+) create mode 100644 docs/manual/debugging-buildroot.txt
docs/manual/debugging-buildroot.txt new file +33 -0
@@ -0,0 +1,33 @@ +// -*- mode:doc; -*- +// vim: set syntax=asciidoc: + +[[debugging-buildroot]] + +Debugging Buildroot +------------------- + +It is possible to instrument the steps +Buildroot+ does when building +packages. Define the variable +BR2_STEPS_USER_HOOKS+ to contain the path +of one or more scripts (or other executables), in a space-separated list, +you want called before and after each step. The script is called with +three parameters: + + - +start+ or +end+ to denote the start (resp. the end) of a step; + - the name of the step about to be started, or which just ended. + - the name of the package + +For example : + +---- +make BR2_STEPS_USER_HOOKS="/path/to/my/script1 /path/to/my/script2" +---- + +That script has access to the following variables: + + - +BUILDROOT_CONFIG+: the path to the Buildroot .config file + - +HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+: see + xref:generic-package-reference[] + - +BUILD_DIR+: the directory where packages are extracted and built + - +BINARIES_DIR+: the place where all binary files (aka images) are + stored + - +BASE_DIR+: the base output directory
docs/manual/developer-guide.txt +2 -0
@@ -13,3 +13,5 @@ include::patch-policy.txt[] include::download-infra.txt[] include::board-support.txt[] + +include::debugging-buildroot.txt[]
package/pkg-generic.mk +9 -0
@@ -54,6 +54,15 @@ define step_time endef STEP_HOOKS += step_time +# User-supplied script +define step_user + @$(foreach user_hook, $(BR2_STEPS_USER_HOOKS), \ + $(USER_HOOKS_EXTRA_ENV) $(user_hook) "$(1)" "$(2)" "$(3)"$(sep)) +endef +ifneq ($(BR2_STEPS_USER_HOOKS),) +STEP_HOOKS += step_user +endif + ################################################################################ # Implicit targets -- produce a stamp file for each step of a package build ################################################################################
Signature · 1 line
1.8.1.2
PATCH 1/4 post-{build, images} hooks: export BUILD_DIR too
In reply to Yann E. MORINQuoted 65 lines · Yann E. MORIN
From: "Yann E. MORIN" <[email protected]>
Also export
BUILD_DIRfor post-{build,images} hooks, so they do have a place to store generated files.Note: this will be more einteresting for the instrumentation of steps, to come in a later patch.
Signed-off-by "Yann E. MORIN" <[email protected]>---
Makefile | 4 ++-- docs/manual/customize-rootfs.txt | 5 +++-- package/Makefile.in | 5 +++++ 3 files changed, 10 insertions(+), 4 deletions(-)Makefile +1
-1@@ -511,7 +511,7 @@ endif @$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_BUILD_SCRIPT)), \ $(call MESSAGE,"Executing post-build script $(s)"); \ - $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) + $(USER_HOOKS_EXTRA_ENV) $(s) $(TARGET_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) ifeq ($(BR2_ENABLE_LOCALE_PURGE),y)LOCALE_WHITELIST=$(BUILD_DIR)/locales.nopurge
Makefile +0
-0@@ -557,7 +557,7 @@ endiftarget-post-image:
@$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_IMAGE_SCRIPT)), \ $(call MESSAGE,"Executing post-image script $(s)"); \Makefile +1
-1- $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep)) + $(USER_HOOKS_EXTRA_ENV) $(s) $(BINARIES_DIR) $(call qstrip,$(BR2_ROOTFS_POST_SCRIPT_ARGS))$(sep))toolchain-eclipse-register:
./support/scripts/eclipse-register-toolchain `readlink -f $(O)` $(notdir $(TARGET_CROSS)) $(BR2_ARCH)docs/manual/customize-rootfs.txt +1
-0@@ -41,6 +41,7 @@ there are a few ways to customize the resulting target filesystem. - +BUILDROOT_CONFIG+: the path to the Buildroot .config file - +HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+: see xref:generic-package-reference[] + - +BUILD_DIR+: the directory where packages are extracted and built - +BINARIES_DIR+: the place where all binary files (aka images) are stored - +BASE_DIR+: the base output directory@@ -78,8 +79,8 @@ in one of these _post-image scripts_ will require special handlingJust like for the _post-build
scripts_mentioned above, you also have access to the following environment variables from your _post-imagedocs/manual/customize-rootfs.txt +2
-2-scripts_: +BUILDROOT_CONFIG+, +HOST_DIR+, +STAGING_DIR+, +TARGET_DIR+, -+BINARIES_DIR+ and +BASE_DIR+. +scripts_: +BUILDROOT_CONFIG+, +BUILD_DIR+, +HOST_DIR+, +STAGING_DIR+, ++TARGET_DIR+, +BINARIES_DIR+ and +BASE_DIR+.Additionally, each of the +BR2_ROOTFS_POST_BUILD_SCRIPT+ and
docs/manual/customize-rootfs.txt +1
-0+BR2_ROOTFS_POST_IMAGE_SCRIPT+ scripts will be passed the argumentspackage/Makefile.in +0
-0
package/Makefile.in HOST_MAKE_ENV=PATH=$(HOST_PATH) \ · line 282
Quoted 8 lines
package/Makefile.in +5
-0@@ -275,6 +275,11 @@ HOST_MAKE_ENV=PATH=$(HOST_PATH) \ PKG_CONFIG_LIBDIR="$(HOST_DIR)/usr/lib/pkgconfig" \ PERLLIB="$(HOST_DIR)/usr/lib/perl" +# This extra environment we can not export ourselves, so we have +# to expicitly pass it to user-supplied external hooks (eg. +# post-build, post-images) +USER_HOOKS_EXTRA_ENV=\ + BUILD_DIR=$(BUILD_DIR)
I'd add a bit more info here: we cannot export BUILD_DIR globally because some packages, like uboot, also use this same variable for another purpose.
Best regards,
Thomas
PATCH 1/4 post-{build, images} hooks: export BUILD_DIR too
Dear Thomas De Schampheleire,
Quoted 7 lines · Thomas De Schampheleire
package/Makefile.in HOST_MAKE_ENV=PATH=$(HOST_PATH) \ · line 282
Quoted 5 lines
package/Makefile.in +5
-0+# This extra environment we can not export ourselves, so we have +# to expicitly pass it to user-supplied external hooks (eg. +# post-build, post-images) +USER_HOOKS_EXTRA_ENV=\ + BUILD_DIR=$(BUILD_DIR)I'd add a bit more info here: we cannot export
BUILD_DIRglobally because some packages, like uboot, also use this same variable for another purpose.
(Unfortunately, we've asked you to not wrapped patches you reply to, but now, it has the consequence that you no longer wrap what you write. Well, I guess with K9 it's hard to get both).
Regarding BUILD_DIR and other variables being exported, is it really safe to export other variables with a relatively generic name such as TARGET_DIR, BASE_DIR, HOST_DIR, BINARIES_DIR, etc. ? Should we instead expose them to post-build/post-image scripts as BR2_<something> ? But well, of course, it means breaking the existing API, which isn't nice.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH 1/4 post-{build, images} hooks: export BUILD_DIR too
Thomas P, All,
On 2013-10-28 06:17 +0100, Thomas Petazzoni spake thusly:
Quoted 9 lines
Dear Thomas De Schampheleire,
Quoted 5 lines · Thomas De Schampheleire
package/Makefile.in HOST_MAKE_ENV=PATH=$(HOST_PATH) \ · line 282
Quoted 5 lines
package/Makefile.in +5
-0+# This extra environment we can not export ourselves, so we have +# to expicitly pass it to user-supplied external hooks (eg. +# post-build, post-images) +USER_HOOKS_EXTRA_ENV=\ + BUILD_DIR=$(BUILD_DIR)
[--SNIP--]
Quoted 5 lines
Regarding
BUILD_DIRand other variables being exported, is it really safe to export other variables with a relatively generic name such asTARGET_DIR,BASE_DIR,HOST_DIR,BINARIES_DIR, etc. ? Should we instead expose them to post-build/post-image scripts as BR2_<something> ? But well, of course, it means breaking the existing API, which isn't nice.
If we export all of those variables with this mechanism rathe than with the make's 'export', then we again dwell on the sagfe side, and we need not rename them.
We should have named them BR2_something from the beginning, even for our internal use, to avoid any name-clashing with the calling environment. For example:
$ export BUILD_DIR=/path/to/my/project/build.dir
$ mkdir -p "${BUILD_DIR}/buildroot.output"
$ make -C "${BUILD_DIR}//buildroot.output"Suppose the user has post-{build,image} or other rsource files (eg. kernel .config) in its ${BUILD_DIR}, then his/her configuration is broken.
But that's the way story goes, we fscked up at the beginning, we have to live with them, now... :-(
Anyway, I'll keep this change as-is, and make a new cset that export all these variables to using this new mechanism, so they only get exported for post-{build,image} scripts (and the new steps-user-hook).
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/4 post-{build, images} hooks: export BUILD_DIR too
In reply to Thomas PetazzoniHi Thomas,
On Mon, Oct 28, 2013 at 6:17 AM, Thomas Petazzoni
<[email protected]> wrote:
[..]
Quoted 5 lines
(Unfortunately, we've asked you to not wrapped patches you reply to, but now, it has the consequence that you no longer wrap what you write. Well, I guess with K9 it's hard to get both).
Hmm :-o
I'm not sure if I can easily fix this with K9. My earlier bug report and even partial patch on the k9 mailing list didn't get any response. I'll check the source but I'll try not to write long sentences in the mean time ;-)
Best regards,
Thomas
PATCH 1/4 post-{build, images} hooks: export BUILD_DIR too
In reply to Thomas De SchampheleireThomas DS, All,
On 2013-10-27 22:12 +0100, Thomas De Schampheleire spake thusly:
Quoted 5 lines
Quoted 4 lines · Yann E. MORIN
From: "Yann E. MORIN" <[email protected]>
Also export
BUILD_DIRfor post-{build,images} hooks, so they do have a place to store generated files.
[--SNIP--]
Quoted 9 lines
package/Makefile.in HOST_MAKE_ENV=PATH=$(HOST_PATH) \ · line 282
Quoted 5 lines
package/Makefile.in +5
-0+# This extra environment we can not export ourselves, so we have +# to expicitly pass it to user-supplied external hooks (eg. +# post-build, post-images) +USER_HOOKS_EXTRA_ENV=\ + BUILD_DIR=$(BUILD_DIR)I'd add a bit more info here: we cannot export
BUILD_DIRglobally because some packages, like uboot, also use this same variable for another purpose.
Will do, thanks.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'