Add find utility functions / exclude files from strip

This patch series adds a few utility functions to Makefile.package.in that help in composing find commands, and uses these functions to exclude user-configured files and directories from being stripped in target-finalize.

The previous version of this series also modified the Python configuration to allow for a 'reduced Python'. These patches were left out for now, as Thomas Petazzoni raised some objections regarding the two-faced configuration. I hope we can continue the discussion about this topic in that thread.

Thomas, you also had comments on the complexity of the find*clauses functions. If you have alternative suggestions, please let me know. Thanks.

---
v3: update based on review comments, remove Python changes for now
v2: simplify find utilities

 Config.in                   |  19 +++++++++++++++++++
 Makefile                    |   9 +++++++--
 package/Makefile.package.in |  11 +++++++++++
 3 files changed, 37 insertions(+), 2 deletions(-)

This patch adds a few utility functions to Makefile.package.in.

Functions finddirclauses and findfileclauses help in building a find command that skips a set of directories and performs operations on a set of files. This pattern can for example be used to keep certain files or directories from being stripped, or to remove certain files from a package installation.

The notfirstword function is the inverse of the 'firstword' function in make:
it returns all but the first word.

---
v3: use -path instead of -wholename; fix a bug in finddirclauses. Thanks, Cam.
v2: integrate _cont variants in main find*clauses functions. Thanks, Arnout.

 package/Makefile.package.in |  11 +++++++++++
 1 files changed, 11 insertions(+), 0 deletions(-)

@@ -87,6 +87,17 @@ MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)
 TERM_BOLD := $(shell tput smso)
 TERM_RESET := $(shell tput rmso)
 
+# Utility functions for 'find'
+# findfileclauses: creates "-name 'X' -o -name 'Y'"
+# [1:namelist]
+findfileclauses=-name '$(firstword $(1))' $(patsubst %,-o -name '%',$(call notfirstword,$(1)))
+# finddirclauses: creates "-path 'basedir/dirX' -o -path 'basedir/dirY'"
+# [1:basedir, 2:namelist]
+finddirclauses=-path '$(1)/$(firstword $(2))' $(patsubst %,-o -path '$(1)/%',$(call notfirstword,$(2)))
+
+# Miscellaneous utility functions
+notfirstword=$(wordlist 2,$(words $(1)),$(1))
+
 # Download method commands
 WGET:=$(call qstrip,$(BR2_WGET)) $(QUIET)
 SVN:=$(call qstrip,$(BR2_SVN))

Sometimes it may be desirable to keep debug symbols for some binaries and libraries on the target. This commit introduces the config option BR2_STRIP_EXCLUDE_FILES, which is interpreted as a list of such binaries and libraries, and the option BR2_STRIP_EXCLUDE_DIRS, which indicates directories excluded from stripping entirely.
These exclusions are passed to the find command in the target-finalize step.

---
v3: merge with redundant patch 'stripping: use findfileclauses utility function'
v2: add the possibility to exclude entire directories

 Config.in |  19 +++++++++++++++++++
 Makefile  |   9 +++++++--
 2 files changed, 26 insertions(+), 2 deletions(-)

@@ -262,6 +262,25 @@ config BR2_STRIP_none
 	  none    do not strip (only for debugging!)
 endchoice
 
+config BR2_STRIP_EXCLUDE_FILES
+	string "executables that should not be stripped"
+	depends on !BR2_STRIP_none
+	default ""
+	help
+	  You may specify a space-separated list of binaries and libraries
+	  here that should not be stripped on the target.
+
+config BR2_STRIP_EXCLUDE_DIRS
+	string "directories that should be skipped when stripping"
+	depends on !BR2_STRIP_none
+	default ""
+	help
+	  You may specify a space-separated list of directories that should
+	  be skipped when stripping. Binaries and libraries in these
+	  directories will not be touched.
+	  The directories should be specified relative to the target directory,
+	  without leading slash.
+
 choice
 	prompt "gcc optimization level"
 	default BR2_OPTIMIZE_S

@@ -429,6 +429,12 @@ endif
 erase-fakeroots:
 	rm -f $(BUILD_DIR)/.fakeroot*
 
+STRIP_FIND_CMD=find $(TARGET_DIR)
+ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
+STRIP_FIND_CMD+=\( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
+endif
+STRIP_FIND_CMD+=-type f -perm +111 -not \( $(call findfileclauses,libthread_db*.so* $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) -print
+
 target-finalize:
 ifeq ($(BR2_HAVE_DEVFILES),y)
 	( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )

@@ -453,8 +459,7 @@ endif
 ifeq ($(BR2_PACKAGE_PYTHON_PYC_ONLY),y)
 	find $(TARGET_DIR)/usr/lib/ -name '*.py' -print0 | xargs -0 rm -f
 endif
-	find $(TARGET_DIR) -type f -perm +111 '!' -name 'libthread_db*.so*' | \
-		xargs $(STRIPCMD) 2>/dev/null || true
+	$(STRIP_FIND_CMD) | xargs $(STRIPCMD) 2>/dev/null || true
 	find $(TARGET_DIR)/lib/modules -type f -name '*.ko' | \
 		xargs -r $(KSTRIPCMD) || true
 

[snip]

Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

Hi Arnout,

Thanks for these suggestions. I eventually used the second one. I tried the first one, because the join function could have been useful in other situations. However, there were two issues.
1. 'join' already is a make function (of course another name would suffice).
2. Because make uses whitespace to separate words, join cannot be used
here: the words are in fact groups of words: '-name X'. As a result, 'firstword $(2)' would be '-name' and 'notfirstword $(2)' would be 'X'. This conflicts with our purpose.

The second suggestion is also very nice and easily understandable, so I took that as alternative.

Best regards,
Thomas