headercheck: add dependency check and improve speed

Introduced a perl based script to do the actual check of the headers. Modified Makfile.headerinst so all files in a dir is checked with one call to checkhdr.pl script.

The file check is used as marker when last run was executed. And the file .check.cmd contains a list of dependencies used by make to determine if a new check should be executed.

The perl script was named check* to follow same naming as other check scripts.

The speedup is > 10 seconds for a build with headercheck enabled.

---

Linus complained that the header check part was too slow on his mac-mini.
So as the well trained monkey^Whacker I am I went ahead and optimized it.

The unidef and install part could be optimized too but that will be another time.

David - I assume you will take it in your tree?

Sam

 b/lib/Kconfig.debug            |    1 
 b/scripts/Makefile.headersinst |   51 +++++++++++++++++++++------------------
 b/scripts/checkhdr.pl          |   53 +++++++++++++++++++++++++++++++++++++++++
 scripts/hdrcheck.sh            |   10 -------
 4 files changed, 82 insertions(+), 33 deletions(-)

index fbc5c62..596f973 100644
@@ -70,6 +70,7 @@ config HEADERS_CHECK
 	  relevant for userspace, say 'Y', and check the headers
 	  exported to $(INSTALL_HDR_PATH) (usually 'usr/include' in
 	  your build tree), to make sure they're suitable.
+	  Note: Enabling this check requires perl to be installed
 
 config DEBUG_KERNEL
 	bool "Kernel debugging"

deleted file mode 100755
index 3159858..0000000
@@ -1,10 +0,0 @@
-#!/bin/sh
-
-for FILE in `grep '^[ \t]*#[ \t]*include[ \t]*<' $2 | cut -f2 -d\< | cut -f1 -d\> | egrep ^linux\|^asm` ; do
-    if [ ! -r $1/$FILE ]; then
-	echo $2 requires $FILE, which does not exist in exported headers
-	exit 1
-    fi
-done
-# FIXME: List dependencies into $3
-touch $3

Modified Makfile.headerinst so all files in a dir is checked with one call to checkhdr.pl script.

The file check is used as marker when last run was executed. And the file .check.cmd contains a list of dependencies used by make to determine if a new check should be executed.

The perl script was named check* to follow same naming as other check scripts.

The speedup is > 10 seconds for a build with headercheck enabled.

---

This version works on powerpc.
The assignment of checkfile had to move below the point where we check for biarch.

Sam

 lib/Kconfig.debug            |    1 +
 scripts/Makefile.headersinst |   55 +++++++++++++++++++++++------------------
 scripts/checkhdr.pl          |   53 ++++++++++++++++++++++++++++++++++++++++
 scripts/hdrcheck.sh          |   10 -------
 4 files changed, 85 insertions(+), 34 deletions(-)
 create mode 100644 scripts/checkhdr.pl
 delete mode 100755 scripts/hdrcheck.sh

index fbc5c62..596f973 100644
@@ -70,6 +70,7 @@ config HEADERS_CHECK
 	  relevant for userspace, say 'Y', and check the headers
 	  exported to $(INSTALL_HDR_PATH) (usually 'usr/include' in
 	  your build tree), to make sure they're suitable.
+	  Note: Enabling this check requires perl to be installed
 
 config DEBUG_KERNEL
 	bool "Kernel debugging"

deleted file mode 100755
index 3159858..0000000
@@ -1,10 +0,0 @@
-#!/bin/sh
-
-for FILE in `grep '^[ \t]*#[ \t]*include[ \t]*<' $2 | cut -f2 -d\< | cut -f1 -d\> | egrep ^linux\|^asm` ; do
-    if [ ! -r $1/$FILE ]; then
-	echo $2 requires $FILE, which does not exist in exported headers
-	exit 1
-    fi
-done
-# FIXME: List dependencies into $3
-touch $3

1.5.1.rc3.20.gaa453

$ make headers_check
$ sed -i /auxvec.h/d include/asm/Kbuild 
$ make headers_check
$ sed -i /auxvec.h/d include/asm-generic/Kbuild.asm
$ make headers_check

Why doesn't it recheck <linux/auxvec.h> and fail?

dwmw2

Because asm/auxvec.h gets pulled by asm-generic/Kbuild.asm

Sam

Not after the 4th command in the above sequence, it doesn't.

dwmw2

Missed that.
This fixes it.

Sam

index 6d74b54..7fa28b8 100644
@@ -51,7 +51,7 @@ ifeq ($(obj)$(ALTARCH),include/asm-$(ARCH)$(BIASMDIR))
 endif
 
 # file generated during checktime listing depfiles and incfiles
-depsfile  := $(INSTALL_HDR_PATH)/$(_dst)/.check.cmd
+depsfile  := $(INSTALL_HDR_PATH)/$(_dst)/.check.d
 checkfile := $(INSTALL_HDR_PATH)/$(_dst)/check
 
 

@@ -135,15 +135,16 @@ echo "\# endif" ;							\
 echo "\#endif /* $$STUBDEF */" ;					\
 ) > $@
 
-.PHONY: __headersinst __headerscheck
+PHONY += __headersinst __headerscheck
 
 ifdef HDRCHECK
 __headerscheck: $(subdir-y) $(checkfile)
 	$(Q):
 
-.PHONY: FORCE
+PHONY += FORCE
+targets += $(checkfile)
 $(checkfile): FORCE
-	$(if $?, $(call cmd,check))
+	$(call if_changed,check)
 	$(Q)touch $(checkfile)
 
 # Include autogenerated file listing depfiles and incfiles

@@ -151,6 +152,15 @@ ifneq ($(wildcard $(depsfile)),)
   include $(depsfile)
 endif
 
+targets := $(wildcard $(sort $(targets)))
+cmd_files := $(wildcard $(foreach f,$(targets),$(dir $(f)).$(notdir $(f)).cmd))
+
+ifneq ($(cmd_files),)
+  include $(cmd_files)
+endif
+
+
+
 $(checkfile) : $(depfiles)
 # dummy rule - cannot be made
 $(depfiles) : ;

@@ -195,7 +205,8 @@ altarch-dir: $(subdir-y) $(header-y) $(unifdef-y) $(objhdr-y)
 	$(Q)$(MAKE) $(hdrinst)=include/asm dst=include/asm$(BIASMDIR)
 
 # Recursion
-.PHONY: $(subdir-y)
+PHONY += $(subdir-y)
 $(subdir-y):
 	$(Q)$(MAKE) $(hdrinst)=$(obj)/$@ dst=$(_dst)/$@ rel=../$(rel)
 
+.PHONY: $(PHONY)

Actually I don't keep a tree for header stuff (except for the automatic export-to-userspace one at linux/kernel/git/dwmw2/kernel-headers.git which just tracks Linus' tree.

I figure that keeping headers in shape is a task for _everyone_, which is sort of why we now have the infrastructure for exporting them and checking on that.

Could you include it in your kbuild tree, or would you like me to (try to remember to) send it to Linus myself after 2.6.22 is out?

dwmw2

I can have it in kbuild.git - no problem.
I'm reworking the header support a bit and will post patches for review/test when ready. So we will end up with something a bit different than what I poster the other week.

Sam