Add find utility functions / exclude files from strip

This patch adds a few utility functions to Makefile.package.in.

Functions finddirclauses and findfileclauses help in building a find command that skips a set of directories and performs operations on a set of files. This pattern can for example be used to keep certain files or directories from being stripped, or to remove certain files from a package installation.

The notfirstword function is the inverse of the 'firstword' function in make:
it returns all but the first word.

Signed-off-by Thomas De Schampheleire <[email protected]>

Acked-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
v4: simplify find*clauses even further. Thanks, Arnout!
v3: use -path instead of -wholename; fix a bug in finddirclauses. Thanks, Cam.
v2: integrate _cont variants in main find*clauses functions. Thanks, Arnout.

?package/Makefile.package.in | ?10 ++++++++++ ?1 files changed, 10 insertions(+), 0 deletions(-)

package/Makefile.package.in +10 -0

@@ -87,6 +87,16 @@ MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)
 ?TERM_BOLD := $(shell tput smso)
 ?TERM_RESET := $(shell tput rmso)
 
+# Utility functions for 'find'
+# findfileclauses(filelist) => -name 'X' -o -name 'Y'
+findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
+# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
+finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
+
+# Miscellaneous utility functions
+# notfirstword(wordlist): returns all but the first word in wordlist
+notfirstword = $(wordlist 2,$(words $(1)),$(1))
+

?# Download method commands
?WGET:=$(call qstrip,$(BR2_WGET)) $(QUIET)
?SVN:=$(call qstrip,$(BR2_SVN))

Quoted 28 lines · Thomas De Schampheleire

This patch adds a few utility functions to Makefile.package.in.

Functions finddirclauses and findfileclauses help in building a find command that skips a set of directories and performs operations on a set of files. This pattern can for example be used to keep certain files or directories from being stripped, or to remove certain files from a package installation.

The notfirstword function is the inverse of the 'firstword' function in make:
it returns all but the first word.

Signed-off-by Thomas De Schampheleire <[email protected]>

Acked-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
v4: simplify find*clauses even further. Thanks, Arnout!
v3: use -path instead of -wholename; fix a bug in finddirclauses. Thanks, Cam.
v2: integrate _cont variants in main find*clauses functions. Thanks, Arnout.

?package/Makefile.package.in | ?10 ++++++++++ ?1 files changed, 10 insertions(+), 0 deletions(-)

package/Makefile.package.in +0 -0

@@ -87,6 +87,16 @@ MESSAGE = echo "$(TERM_BOLD)>>> $($(PKG)
 ?TERM_BOLD := $(shell tput smso)
 ?TERM_RESET := $(shell tput rmso)
 

Quoted 10 lines

package/Makefile.package.in +10 -0

+# Utility functions for 'find'
+# findfileclauses(filelist) => -name 'X' -o -name 'Y'
+findfileclauses = $(call notfirstword,$(patsubst %,-o -name '%',$(1)))
+# finddirclauses(base, dirlist) => -path 'base/dirX' -o -path 'base/dirY'
+finddirclauses = $(call notfirstword,$(patsubst %,-o -path '$(1)/%',$(2)))
+
+# Miscellaneous utility functions
+# notfirstword(wordlist): returns all but the first word in wordlist
+notfirstword = $(wordlist 2,$(words $(1)),$(1))
+

Quoted 3 lines

?# Download method commands
?WGET:=$(call qstrip,$(BR2_WGET)) $(QUIET)
?SVN:=$(call qstrip,$(BR2_SVN))

bump

Quoted 1 line

package/Makefile.package.in

Quoted 1 line

package/Makefile.package.in +0 -0

Quoted 2 lines

Quoted 2 lines

package/Makefile.package.in

Quoted 2 lines

package/Makefile.package.in +0 -0

Unfortunately, Makefile.package.in no longer exists...

Sometimes it may be desirable to keep debug symbols for some binaries and libraries on the target. This commit introduces the config option BR2_STRIP_EXCLUDE_FILES, which is interpreted as a list of such binaries and libraries, and the option BR2_STRIP_EXCLUDE_DIRS, which indicates directories excluded from stripping entirely.
These exclusions are passed to the find command in the target-finalize step.

Signed-off-by Sven Neumann <[email protected]>

Signed-off-by Thomas De Schampheleire <[email protected]>

Acked-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
v4: whitespace changes
v3: merge with redundant patch 'stripping: use findfileclauses utility function'
v2: add the possibility to exclude entire directories

?Config.in | ?19 +++++++++++++++++++ ?Makefile ?| ?10 ++++++++--
?2 files changed, 27 insertions(+), 2 deletions(-)

Config.in +0 -0

@@ -269,6 +269,25 @@ config BR2_STRIP_none
 ? ? ? ? ?none ? ?do not strip (only for debugging!)

?endchoice

Config.in +19 -0

+config BR2_STRIP_EXCLUDE_FILES
+ ? ? ? string "executables that should not be stripped"
+ ? ? ? depends on !BR2_STRIP_none
+ ? ? ? default ""
+ ? ? ? help
+ ? ? ? ? You may specify a space-separated list of binaries and libraries
+ ? ? ? ? here that should not be stripped on the target.
+
+config BR2_STRIP_EXCLUDE_DIRS
+ ? ? ? string "directories that should be skipped when stripping"
+ ? ? ? depends on !BR2_STRIP_none
+ ? ? ? default ""
+ ? ? ? help
+ ? ? ? ? You may specify a space-separated list of directories that should
+ ? ? ? ? be skipped when stripping. Binaries and libraries in these
+ ? ? ? ? directories will not be touched.
+ ? ? ? ? The directories should be specified relative to the target directory,
+ ? ? ? ? without leading slash.
+

?choice
? ? ? ?prompt "gcc optimization level"
? ? ? ?default BR2_OPTIMIZE_S

Makefile +0 -0

@@ -438,6 +438,13 @@ endif

?erase-fakeroots:
? ? ? ?rm -f $(BUILD_DIR)/.fakeroot*

Makefile +7 -0

+STRIP_FIND_CMD = find $(TARGET_DIR)
+ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
+STRIP_FIND_CMD += \( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
+endif
+STRIP_FIND_CMD += -type f -perm +111
+STRIP_FIND_CMD += -not \( $(call findfileclauses,libthread_db*.so* $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) -print
+

?target-finalize:
?ifeq ($(BR2_HAVE_DEVFILES),y)
? ? ? ?( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )

Makefile +0 -0

@@ -462,8 +469,7 @@ endif
 ?ifeq ($(BR2_PACKAGE_PYTHON_PYC_ONLY),y)

? ? ? ?find $(TARGET_DIR)/usr/lib/ -name '*.py' -print0 | xargs -0 rm -f ?endif
- ? ? ? find $(TARGET_DIR) -type f -perm +111 '!' -name 'libthread_db*.so*' | \
- ? ? ? ? ? ? ? xargs $(STRIPCMD) 2>/dev/null || true
+ ? ? ? $(STRIP_FIND_CMD) | xargs $(STRIPCMD) 2>/dev/null || true ? ? ? ?find $(TARGET_DIR)/lib/modules -type f -name '*.ko' | \
? ? ? ? ? ? ? ?xargs -r $(KSTRIPCMD) || true

Quoted 70 lines · Thomas De Schampheleire

Sometimes it may be desirable to keep debug symbols for some binaries and libraries on the target. This commit introduces the config option BR2_STRIP_EXCLUDE_FILES, which is interpreted as a list of such binaries and libraries, and the option BR2_STRIP_EXCLUDE_DIRS, which indicates directories excluded from stripping entirely.
These exclusions are passed to the find command in the target-finalize step.

Signed-off-by Sven Neumann <[email protected]>

Signed-off-by Thomas De Schampheleire <[email protected]>

Acked-by Arnout Vandecappelle (Essensium/Mind) <[email protected]>

---
v4: whitespace changes
v3: merge with redundant patch 'stripping: use findfileclauses utility function'
v2: add the possibility to exclude entire directories

?Config.in | ?19 +++++++++++++++++++ ?Makefile ?| ?10 ++++++++--
?2 files changed, 27 insertions(+), 2 deletions(-)

Config.in +0 -0

@@ -269,6 +269,25 @@ config BR2_STRIP_none
 ? ? ? ? ?none ? ?do not strip (only for debugging!)

?endchoice

Config.in +19 -0

+config BR2_STRIP_EXCLUDE_FILES
+ ? ? ? string "executables that should not be stripped"
+ ? ? ? depends on !BR2_STRIP_none
+ ? ? ? default ""
+ ? ? ? help
+ ? ? ? ? You may specify a space-separated list of binaries and libraries
+ ? ? ? ? here that should not be stripped on the target.
+
+config BR2_STRIP_EXCLUDE_DIRS
+ ? ? ? string "directories that should be skipped when stripping"
+ ? ? ? depends on !BR2_STRIP_none
+ ? ? ? default ""
+ ? ? ? help
+ ? ? ? ? You may specify a space-separated list of directories that should
+ ? ? ? ? be skipped when stripping. Binaries and libraries in these
+ ? ? ? ? directories will not be touched.
+ ? ? ? ? The directories should be specified relative to the target directory,
+ ? ? ? ? without leading slash.
+

?choice
? ? ? ?prompt "gcc optimization level"
? ? ? ?default BR2_OPTIMIZE_S

Makefile +0 -0

@@ -438,6 +438,13 @@ endif

?erase-fakeroots:
? ? ? ?rm -f $(BUILD_DIR)/.fakeroot*

Makefile +7 -0

+STRIP_FIND_CMD = find $(TARGET_DIR)
+ifneq (,$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS)))
+STRIP_FIND_CMD += \( $(call finddirclauses,$(TARGET_DIR),$(call qstrip,$(BR2_STRIP_EXCLUDE_DIRS))) \) -prune -o
+endif
+STRIP_FIND_CMD += -type f -perm +111
+STRIP_FIND_CMD += -not \( $(call findfileclauses,libthread_db*.so* $(call qstrip,$(BR2_STRIP_EXCLUDE_FILES))) \) -print
+

?target-finalize:
?ifeq ($(BR2_HAVE_DEVFILES),y)
? ? ? ?( support/scripts/copy.sh $(STAGING_DIR) $(TARGET_DIR) )

Makefile +0 -0

@@ -462,8 +469,7 @@ endif
 ?ifeq ($(BR2_PACKAGE_PYTHON_PYC_ONLY),y)

? ? ? ?find $(TARGET_DIR)/usr/lib/ -name '*.py' -print0 | xargs -0 rm -f ?endif

Quoted 6 lines

- ? ? ? find $(TARGET_DIR) -type f -perm +111 '!' -name 'libthread_db*.so*' | \
- ? ? ? ? ? ? ? xargs $(STRIPCMD) 2>/dev/null || true
+ ? ? ? $(STRIP_FIND_CMD) | xargs $(STRIPCMD) 2>/dev/null || true ? ? ? ?find $(TARGET_DIR)/lib/modules -type f -name '*.ko' | \
? ? ? ? ? ? ? ?xargs -r $(KSTRIPCMD) || true

bump

This patch series adds a few utility functions to Makefile.package.in that help in composing find commands, and uses these functions to exclude user-configured files and directories from being stripped in target-finalize.

Thanks to input from Arnout, the find functions are now much simpler and should be clearly understandable by everyone.

Signed-off-by Thomas De Schampheleire <[email protected]>

---
v4: simplify find functions even further; whitespace changes
v3: update based on review comments, remove Python changes for now
v2: simplify find functions

?Config.in ? ? ? ? ? ? ? ? ? | ?19 +++++++++++++++++++
?Makefile ? ? ? ? ? ? ? ? ? ?| ?10 ++++++++--
?package/Makefile.package.in | ?10 ++++++++++

?3 files changed, 37 insertions(+), 2 deletions(-)