Add POST_RSYNC hook
PATCH v2 1/1 Add POST_RSYNC hook
This patch series adopts the POST_RSYNC hook patch by Tzu-Jung Lee and changes the default rsync exclusion list, as discussed on the Buildroot developer days.
The other rsync-related patch submitted by Tzu-Jung Lee (custom rsync command) should not be needed anymore as its use case is covered by the second patch in this series.
--- Makefile | 14 ++++++++------ docs/manual/adding-packages-autotools.txt | 3 +-- docs/manual/adding-packages-cmake.txt | 3 +-- docs/manual/adding-packages-generic.txt | 39 +-------------------------------------- docs/manual/adding-packages-hooks.txt | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ docs/manual/adding-packages.txt | 2 ++ package/pkg-generic.mk | 4 +++- 7 files changed, 77 insertions(+), 49 deletions(-)
PATCH v2 1/1 infra: Add POST_RSYNC_HOOKS support
One of the use cases is for the 'local packages' to restore the SCM info. Some packages use this information to generate version info during build time. In this case, the local package can have this hook to restore it by symbolic link for example.
---
v2: no changes
docs/manual/adding-packages-generic.txt | 1 + package/pkg-generic.mk | 2 ++ 2 files changed, 3 insertions(+)
docs/manual/adding-packages-generic.txt +1 -0
@@ -467,6 +467,7 @@ The following hook points are available: * +LIBFOO_POST_DOWNLOAD_HOOKS+ * +LIBFOO_POST_EXTRACT_HOOKS+ +* +LIBFOO_POST_RSYNC_HOOKS+ * +LIBFOO_PRE_PATCH_HOOKS+ * +LIBFOO_POST_PATCH_HOOKS+ * +LIBFOO_PRE_CONFIGURE_HOOKS+
package/pkg-generic.mk +2 -0
@@ -69,6 +69,7 @@ endif @$(call MESSAGE,"Syncing from source dir $(SRCDIR)") @test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1) rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D) + $(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@ # Handle the SOURCE_CHECK and SHOW_EXTERNAL_DEPS cases for rsynced
@@ -339,6 +340,7 @@ endif # post-steps hooks $(2)_POST_DOWNLOAD_HOOKS ?= $(2)_POST_EXTRACT_HOOKS ?= +$(2)_POST_RSYNC_HOOKS ?= $(2)_PRE_PATCH_HOOKS ?= $(2)_POST_PATCH_HOOKS ?= $(2)_PRE_CONFIGURE_HOOKS ?=
PATCH v2 1/1 infra: centralize rsync exclude list for VCS files
In reply to Thomas De SchampheleireBuildroot has three places where rsync is used:
1. to copy the target skeleton
2. to copy the rootfs overlay(s)
3. to copy overridden package sources
In all of these cases, we want to exclude version control files by default. Place 1 and 2 used an identical set of explicit --exclude options, while place 3 used the option --cvs-exclude. This last option, however, not only excludes version control files, but also binary files (.o, .so) and any file or directory named 'core' (a problem for the linux kernel that has several directories with this name). Moreover, the exact list of excluded files when using --cvs-exclude depends on the version of rsync.
This patch creates one global variable RSYNC_VCS_EXCLUSIONS that can be used by the various rsync commands. It excludes the version control files of svn, git, hg, cvs and bzr.
---
v2: no changes
Makefile | 14 ++++++++------ package/pkg-generic.mk | 2 +- 2 files changed, 9 insertions(+), 7 deletions(-)
Makefile +8 -6
@@ -426,11 +426,14 @@ ifeq ($(BR2_ROOTFS_SKELETON_CUSTOM),y) TARGET_SKELETON=$(BR2_ROOTFS_SKELETON_CUSTOM_PATH) endif +RSYNC_VCS_EXCLUSIONS = \ + --exclude .svn --exclude .git --exclude .hg --exclude .bzr \ + --exclude CVS + $(BUILD_DIR)/.root: mkdir -p $(TARGET_DIR) - rsync -a \ - --exclude .empty --exclude .svn --exclude .git \ - --exclude .hg --exclude=CVS --exclude '*~' \ + rsync -a $(RSYNC_VCS_EXCLUSIONS) \ + --exclude .empty --exclude '*~' \ $(TARGET_SKELETON)/ $(TARGET_DIR)/ cp support/misc/target-dir-warning.txt $(TARGET_DIR_WARNING_FILE) @ln -snf lib $(TARGET_DIR)/$(LIB_SYMLINK)
@@ -504,9 +507,8 @@ endif @$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \ $(call MESSAGE,"Copying overlay $(d)"); \ - rsync -a \ - --exclude .empty --exclude .svn --exclude .git \ - --exclude .hg --exclude=CVS --exclude '*~' \ + rsync -a $(RSYNC_VCS_EXCLUSIONS) \ + --exclude .empty --exclude '*~' \ $(d)/ $(TARGET_DIR)$(sep)) @$(foreach s, $(call qstrip,$(BR2_ROOTFS_POST_BUILD_SCRIPT)), \
package/pkg-generic.mk +1 -1
@@ -68,7 +68,7 @@ endif $(BUILD_DIR)/%/.stamp_rsynced: @$(call MESSAGE,"Syncing from source dir $(SRCDIR)") @test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1) - rsync -au --cvs-exclude --include core $(SRCDIR)/ $(@D) + rsync -au $(RSYNC_VCS_EXCLUSIONS) $(SRCDIR)/ $(@D) $(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep)) $(Q)touch $@
PATCH v2 1/1 manual: split info on hooks to a separate section/file
In reply to Thomas De SchampheleireSplit out the information on hooks to a separate section (and source file). Not only because the hooks are useful for all infrastructures (and thus don't really fit specifically in the generic infrastructure section), but also for clarity when the info on hooks will be expanded in later patches.
---
v2: no changes
docs/manual/adding-packages-autotools.txt | 3 +- docs/manual/adding-packages-cmake.txt | 3 +- docs/manual/adding-packages-generic.txt | 40 +------------------------- docs/manual/adding-packages-hooks.txt | 41 +++++++++++++++++++++++++++ docs/manual/adding-packages.txt | 2 + 5 files changed, 46 insertions(+), 43 deletions(-)
docs/manual/adding-packages-autotools.txt +1 -2
@@ -162,8 +162,7 @@ well for most autotools-based packages. still possible to customize what is done in any particular step: * By adding a post-operation hook (after extract, patch, configure, - build or install). See the reference documentation of the generic - infrastructure for details. + build or install). See xref:hooks[] for details. * By overriding one of the steps. For example, even if the autotools infrastructure is used, if the package +.mk+ file defines its
docs/manual/adding-packages-cmake.txt +1 -2
@@ -135,8 +135,7 @@ for most CMake-based packages. However, possible to customize what is done in any particular step: * By adding a post-operation hook (after extract, patch, configure, - build or install). See the reference documentation of the generic - infrastructure for details. + build or install). See xref:hooks[] for details. * By overriding one of the steps. For example, even if the CMake infrastructure is used, if the package +.mk+ file defines its own
docs/manual/adding-packages-generic.txt +1 -39
@@ -451,42 +451,4 @@ In the action definitions, you can use t * Of course the +$(HOST_DIR)+, +$(STAGING_DIR)+ and +$(TARGET_DIR)+ variables to install the packages properly. -The last feature of the generic infrastructure is the ability to add -hooks. These define further actions to perform after existing steps. -Most hooks aren't really useful for generic packages, since the +.mk+ -file already has full control over the actions performed in each step -of the package construction. The hooks are more useful for packages -using the autotools infrastructure described below. However, since -they are provided by the generic infrastructure, they are documented -here. The exception is +LIBFOO_POST_PATCH_HOOKS+. Patching the -package and producing legal info are not user definable, so -+LIBFOO_POST_PATCH_HOOKS+ and +LIBFOO_POST_LEGAL_INFO_HOOKS+ are -useful for generic packages. - -The following hook points are available: - -* +LIBFOO_POST_DOWNLOAD_HOOKS+ -* +LIBFOO_POST_EXTRACT_HOOKS+ -* +LIBFOO_POST_RSYNC_HOOKS+ -* +LIBFOO_PRE_PATCH_HOOKS+ -* +LIBFOO_POST_PATCH_HOOKS+ -* +LIBFOO_PRE_CONFIGURE_HOOKS+ -* +LIBFOO_POST_CONFIGURE_HOOKS+ -* +LIBFOO_POST_BUILD_HOOKS+ -* +LIBFOO_POST_INSTALL_HOOKS+ (for host packages only) -* +LIBFOO_POST_INSTALL_STAGING_HOOKS+ (for target packages only) -* +LIBFOO_POST_INSTALL_TARGET_HOOKS+ (for target packages only) -* +LIBFOO_POST_LEGAL_INFO_HOOKS+ - -These variables are 'lists' of variable names containing actions to be -performed at this hook point. This allows several hooks to be -registered at a given hook point. Here is an example: - ----------------------- -define LIBFOO_POST_PATCH_FIXUP - action1 - action2 -endef - -LIBFOO_POST_PATCH_HOOKS += LIBFOO_POST_PATCH_FIXUP ----------------------- +Finally, you can also use hooks. See xref:hooks[] for more information.
docs/manual/adding-packages-hooks.txt new file +41 -0
41 lines not on this page · raw message
docs/manual/adding-packages.txt +2 -0
@@ -18,6 +18,8 @@ include::adding-packages-autotools.txt[] include::adding-packages-cmake.txt[] +include::adding-packages-hooks.txt[] + include::adding-packages-gettext.txt[] include::adding-packages-tips.txt[]
PATCH v2 1/1 manual: add some info on the POST_RSYNC hook
In reply to Thomas De Schampheleire---
v2: rsync is also used for site method 'local' (comment ThomasP on IRC)
docs/manual/adding-packages-hooks.txt | 20 ++++++++++++++++++++ 1 files changed, 20 insertions(+), 0 deletions(-)
docs/manual/adding-packages-hooks.txt +20 -0
@@ -39,3 +39,23 @@ endef LIBFOO_POST_PATCH_HOOKS += LIBFOO_POST_PATCH_FIXUP ---------------------- + +Using the +POST_RSYNC+ hook +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The +POST_RSYNC+ hook is run only for packages that use a local source, +either through the +local+ site method or the +OVERRIDE_SRCDIR+ +mechanism. In this case, package sources are copied using +rsync+ from +the local location into the buildroot build directory. The +rsync+ +command does not copy all files from the source directory, though. +Files belonging to a version control system, like the directories ++.git+, +.hg+, etc. are not copied. For most packages this is +sufficient, but a given package can perform additional actions using +the +POST_RSYNC+ hook. + +In principle, the hook can contain any command you want. One specific +use case, though, is the intentional copying of the version control +directory using +rsync+. The +rsync+ command you use in the hook can, among +others, use the following variables: + +* +$(SRCDIR)+: the path to the overridden source directory +* +$(@D)+: the path to the build directory
PATCH v2 1/1 manual: add some info on the POST_RSYNC hook
2013/11/7 Thomas De Schampheleire <[email protected]>
Quoted 2 lines
Signed-off-by Thomas De Schampheleire <[email protected]>
Quoted 41 lines
---
v2: rsync is also used for site method 'local' (comment ThomasP on IRC)docs/manual/adding-packages-hooks.txt | 20 ++++++++++++++++++++ 1 files changed, 20 insertions(+), 0 deletions(-)docs/manual/adding-packages-hooks.txt +16
-0@@ -39,3 +39,23 @@ endef LIBFOO_POST_PATCH_HOOKS += LIBFOO_POST_PATCH_FIXUP ---------------------- + +Using the +POST_RSYNC+ hook +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The +POST_RSYNC+ hook is run only for packages that use a local source, +either through the +local+ site method or the +OVERRIDE_SRCDIR+ +mechanism. In this case, package sources are copied using +rsync+ from +the local location into the buildroot build directory. The +rsync+ +command does not copy all files from the source directory, though. +Files belonging to a version control system, like the directories ++.git+, +.hg+, etc. are not copied. For most packages this is +sufficient, but a given package can perform additional actions using +the +POST_RSYNC+ hook. + +In principle, the hook can contain any command you want. One specific +use case, though, is the intentional copying of the version control +directory using +rsync+. The +rsync+ command you use in the hook can,among
docs/manual/adding-packages-hooks.txt +4
-0+others, use the following variables: + +* +$(SRCDIR)+: the path to the overridden source directory +* +$(@D)+: the path to the build directory_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 4 lines
Samuel -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20131107/ed00a074/attachment.html>
PATCH v2 1/1 Add POST_RSYNC hook
In reply to Thomas De SchampheleireQuoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas De Schampheleire <[email protected]> writes:
This patch series adopts the
POST_RSYNChook patch by Tzu-Jung Lee and changes the default rsync exclusion list, as discussed on the Buildroot developer days.
The other rsync-related patch submitted by Tzu-Jung Lee (custom rsync command) should not be needed anymore as its use case is covered by the second patch in this series.Signed-off-by Thomas De Schampheleire <[email protected]>
Committed series, thanks.
Signature · 1 line
Bye, Peter Korsgaard