Skip to contentSkip to conversation

kbuild: refactor headers_install and support compile-test of UAPI headers

PATCH In review Masahiro Yamada 4 participants 26 messages Jun 4, 2019 – Jun 22, 2019

Subsystems (11)
Networking Network drivers Documentation PowerPC +7 more subsystems
lore
Share & tools
lore

Masahiro Yamada

author
#0 lore raw

cover kbuild: refactor headers_install and support compile-test of UAPI headers

Multiple people have suggested to compile-test UAPI headers.

Currently, Kbuild provides simple sanity checks by headers_check but they are not enough to catch bugs.

The most recent patch I know is David Howells' work:
https://patchwork.kernel.org/patch/10590203/

I agree that we need better tests for UAPI headers, but I want to integrate it in a clean way.

The idea that has been in my mind is to compile each header to make sure the selfcontainedness.

Recently, Jani Nikula proposed a new syntax 'header-test-y'.
https://patchwork.kernel.org/patch/10947005/

So, I implemented UAPI compile-testing on top of that.

When adding a new feature, cleaning the code first is a good practice.

[1] Remove headers_install_all

This target installs UAPI headers of all architectures in a single tree.
It does not make sense to compile test of headers from multiple arches at the same time. Hence, removed.

[2] Split header installation into 'make headers' and 'make headers_install'

To compile-test UAPI headers, we need a work-directory somewhere to save objects and .*.cmd files.

usr/include/ will be the work-directory.

Since we cannot pollute the final destination of headers_install,

I split the header installation into two stages.

'make headers' will build up
the ready-to-install headers in usr/include,
which will be also used as a work-directory for the compile-test.

'make headers_install' will copy headers
from usr/include to $(INSTALL_HDR_PATH)/include.

[3] Support compile-test of UAPI headers

This is implemented in usr/include/Makefile

Jani Nikula (1):
kbuild: add support for ensuring headers are self-contained

Masahiro Yamada (14):
kbuild: remove headers_{install,check}_all
kbuild: remove stale dependency between Documentation/ and
headers_install
kbuild: make gdb_script depend on prepare0 instead of prepare
kbuild: fix Kconfig prompt of CONFIG_HEADERS_CHECK
kbuild: add CONFIG_HEADERS_INSTALL and loosen the dependency of
samples
kbuild: remove build_unifdef target in scripts/Makefile
kbuild: build all prerequisite of headers_install simultaneously
kbuild: add 'headers' target to build up ready-to-install uapi headers
kbuild: re-implement Makefile.headersinst without directory descending
kbuild: move hdr-inst shorthand to top Makefile
kbuild: simplify scripts/headers_install.sh
kbuild: deb-pkg: do not run headers_check
fixup: kbuild: add support for ensuring headers are self-contained
kbuild: compile test UAPI headers to ensure they are self-contained

 Documentation/kbuild/headers_install.txt |   7 --
 Documentation/kbuild/makefiles.txt       |  13 ++-
 Makefile                                 |  56 +++++-----
 arch/arc/configs/tb10x_defconfig         |   1 +
 arch/nds32/configs/defconfig             |   1 +
 arch/parisc/configs/a500_defconfig       |   1 +
 arch/parisc/configs/b180_defconfig       |   1 +
 arch/parisc/configs/c3000_defconfig      |   1 +
 arch/parisc/configs/default_defconfig    |   1 +
 arch/powerpc/configs/ppc6xx_defconfig    |   1 +
 arch/s390/configs/debug_defconfig        |   1 +
 include/uapi/{linux => }/Kbuild          |   6 +-
 init/Kconfig                             |  20 ++++
 lib/Kconfig.debug                        |  25 +++--
 samples/Kconfig                          |  14 ++-
 samples/Makefile                         |   4 +-
 scripts/Kbuild.include                   |   6 --
 scripts/Makefile                         |   5 -
 scripts/Makefile.build                   |   9 ++
 scripts/Makefile.headersinst             | 132 ++++++++++-------------
 scripts/Makefile.lib                     |   3 +
 scripts/cc-system-headers.sh             |   8 ++
 scripts/headers.sh                       |  29 -----
 scripts/headers_install.sh               |  48 ++++-----
 scripts/package/builddeb                 |   2 +-
 usr/.gitignore                           |   1 -
 usr/Makefile                             |   2 +
 usr/include/.gitignore                   |   3 +
 usr/include/Makefile                     | 132 +++++++++++++++++++++++
 29 files changed, 329 insertions(+), 204 deletions(-)
 rename include/uapi/{linux => }/Kbuild (77%)
 create mode 100755 scripts/cc-system-headers.sh
 delete mode 100755 scripts/headers.sh
 create mode 100644 usr/include/.gitignore
 create mode 100644 usr/include/Makefile
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 1/15 kbuild: remove headers_{install,check}_all

headers_install_all does not make much sense any more because different architectures export different set of uapi/linux/ headers. As you see in include/uapi/linux/Kbuild, the installation of a.out.h, kvm.h, and kvm_para.h is arch-dependent. So, headers_install_all repeats the installation/removal of them.

If somebody really thinks it is useful to do headers_install for all architectures, it would be possible by small shell-scripting, but the top Makefile do not have to provide entry targets just for that purpose.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Documentation/kbuild/headers_install.txt |  7 ------
 Documentation/kbuild/makefiles.txt       |  6 +----
 Makefile                                 | 15 ++----------
 scripts/headers.sh                       | 29 ------------------------
 4 files changed, 3 insertions(+), 54 deletions(-)
 delete mode 100755 scripts/headers.sh
Documentation/kbuild/headers_install.txt +0 -7
index f0153adb95e2..f07e34eab70b 100644
@@ -39,12 +39,5 @@ INSTALL_HDR_PATH indicates where to install the headers. It defaults to
 An 'include' directory is automatically created inside INSTALL_HDR_PATH and
 headers are installed in 'INSTALL_HDR_PATH/include'.
 
-The command "make headers_install_all" exports headers for all architectures
-simultaneously.  (This is mostly of interest to distribution maintainers,
-who create an architecture-independent tarball from the resulting include
-directory.)  You also can use HDR_ARCH_LIST to specify list of architectures.
-Remember to provide the appropriate linux/asm directory via "mv" or "ln -s"
-before building a C library with headers exported this way.
-
 The kernel header export infrastructure is maintained by David Woodhouse
 <[email protected]>.
Documentation/kbuild/makefiles.txt +1 -5
index d65ad5746f94..bac301a73a86 100644
@@ -897,11 +897,7 @@ When kbuild executes, the following steps are followed (roughly):
 --- 6.2 Add prerequisites to archheaders:
 
 	The archheaders: rule is used to generate header files that
-	may be installed into user space by "make header_install" or
-	"make headers_install_all".  In order to support
-	"make headers_install_all", this target has to be able to run
-	on an unconfigured tree, or a tree configured for another
-	architecture.
+	may be installed into user space by "make header_install".
 
 	It is run before "make archprepare" when run on the
 	architecture itself.
Makefile +2 -13
index 004d67a4405f..7745bdd84861 100644
@@ -1181,34 +1181,23 @@ headerdep:
 #Default location for installed headers
 export INSTALL_HDR_PATH = $(objtree)/usr
 
-# If we do an all arch process set dst to include/arch-$(SRCARCH)
-hdr-dst = $(if $(KBUILD_HEADERS), dst=include/arch-$(SRCARCH), dst=include)
-
 PHONY += archheaders archscripts
 
 PHONY += __headers
 __headers: $(version_h) scripts_basic uapi-asm-generic archheaders archscripts
 	$(Q)$(MAKE) $(build)=scripts build_unifdef
 
-PHONY += headers_install_all
-headers_install_all:
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/headers.sh install
-
 PHONY += headers_install
 headers_install: __headers
 	$(if $(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/Kbuild),, \
 	  $(error Headers not exportable for the $(SRCARCH) architecture))
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include
-	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi $(hdr-dst)
-
-PHONY += headers_check_all
-headers_check_all: headers_install_all
-	$(Q)$(CONFIG_SHELL) $(srctree)/scripts/headers.sh check
+	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include
 
 PHONY += headers_check
 headers_check: headers_install
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include HDRCHECK=1
-	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi $(hdr-dst) HDRCHECK=1
+	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include HDRCHECK=1
 
 ifdef CONFIG_HEADERS_CHECK
 all: headers_check
scripts/headers.sh deleted +0 -29
deleted file mode 100755
index e0f883eb39a2..000000000000
@@ -1,29 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: GPL-2.0
-# Run headers_$1 command for all suitable architectures
-
-# Stop on error
-set -e
-
-do_command()
-{
-	if [ -f ${srctree}/arch/$2/include/asm/Kbuild ]; then
-		make ARCH=$2 KBUILD_HEADERS=$1 headers_$1
-	else
-		printf "Ignoring arch: %s\n" ${arch}
-	fi
-}
-
-archs=${HDR_ARCH_LIST:-$(ls ${srctree}/arch)}
-
-for arch in ${archs}; do
-	case ${arch} in
-	um)        # no userspace export
-		;;
-	*)
-		if [ -d ${srctree}/arch/${arch} ]; then
-			do_command $1 ${arch}
-		fi
-		;;
-	esac
-done
Signature · 1 line
2.17.1
Acked-by on v1 1/15
#2 lore raw

Re: [PATCH 01/15] kbuild: remove headers_{install,check}_all

Quoted 9 lines · Masahiro Yamada

headers_install_all does not make much sense any more because different architectures export different set of uapi/linux/ headers. As you see in include/uapi/linux/Kbuild, the installation of a.out.h, kvm.h, and kvm_para.h is arch-dependent. So, headers_install_all repeats the installation/removal of them.

If somebody really thinks it is useful to do headers_install for all architectures, it would be possible by small shell-scripting, but the top Makefile do not have to provide entry targets just for that purpose.

Agreed, good to see this be dropped.

Acked-by Sam Ravnborg <[email protected]>
author
#3 lore raw

PATCH 2/15 kbuild: remove stale dependency between Documentation/ and headers_install

In reply to Masahiro Yamada

Commit 8e2faea877eb ("Make Documenation depend on headers_install") dates back to 2014, which is before Sphinx was introduced for the kernel documentation.

Since none of DOC_TARGET requires headers_install, it is strange to run it only for the single target "Documentation/".

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Makefile +1 -2
index 7745bdd84861..67b866d19269 100644
@@ -1745,8 +1745,7 @@ build-dir = $(patsubst %/,%,$(dir $(build-target)))
 PHONY += /
 /: ./
 
-# Make sure the latest headers are built for Documentation
-Documentation/ samples/: headers_install
+samples/: headers_install
 %/: prepare FORCE
 	$(Q)$(MAKE) KBUILD_MODULES=1 $(build)=$(build-dir)
Signature · 1 line
2.17.1
author
#4 lore raw

PATCH 3/15 kbuild: make gdb_script depend on prepare0 instead of prepare

In reply to Masahiro Yamada

'gdb_script' needs headers generated by ./Kbuild, which is visited by 'prepare0'. None of 'gdb_script' depends on 'prepare'.

Loosen the dependency.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
index 67b866d19269..8348939765d3 100644
@@ -1541,7 +1541,7 @@ $(DOC_TARGETS): scripts_basic FORCE
 # ---------------------------------------------------------------------------
 
 PHONY += scripts_gdb
-scripts_gdb: prepare
+scripts_gdb: prepare0
 	$(Q)$(MAKE) $(build)=scripts/gdb
 	$(Q)ln -fsn $(abspath $(srctree)/scripts/gdb/vmlinux-gdb.py)
Signature · 1 line
2.17.1
author
#5 lore raw

PATCH 4/15 kbuild: fix Kconfig prompt of CONFIG_HEADERS_CHECK

In reply to Masahiro Yamada

Prior to commit 257edce66d31 ("kbuild: exploit parallel building for CONFIG_HEADERS_CHECK"), the sanity check of exported headers was done as a side-effect of build rule of vmlinux.

That commit is good, but I missed to update the prompt of the Kconfig entry. For the sake of preciseness, lets' say "when building 'all'".

Signed-off-by Masahiro Yamada <[email protected]>

---

 lib/Kconfig.debug | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
lib/Kconfig.debug +3 -3
index cbdfae379896..7cdcb962358c 100644
@@ -306,11 +306,11 @@ config DEBUG_FS
 	  If unsure, say N.
 
 config HEADERS_CHECK
-	bool "Run 'make headers_check' when building vmlinux"
+	bool "Run sanity checks on uapi headers when building 'all'"
 	depends on !UML
 	help
-	  This option will extract the user-visible kernel headers whenever
-	  building the kernel, and will run basic sanity checks on them to
+	  This option will extract the user-visible kernel headers when
+	  building the 'all' target, and will run basic sanity checks on them to
 	  ensure that exported files do not attempt to include files which
 	  were not exported, etc.
Signature · 1 line
2.17.1
author
#6 lore raw

PATCH 5/15 kbuild: add CONFIG_HEADERS_INSTALL and loosen the dependency of samples

In reply to Masahiro Yamada

Commit 5318321d367c ("samples: disable CONFIG_SAMPLES for UML") used a big hammer to fix the build errors under the samples/ directory, while only some samples actually include uapi headers from usr/include.

Introduce CONFIG_HEADERS_INSTALL since 'depends on HEADERS_INSTALL' is clearer than 'depends on !UML'. If this option is enabled, uapi headers are installed before starting directory descending.

I added 'depends on HEADERS_INSTALL' to per-sample CONFIG options. This allows UML to compile some samples.

$ make ARCH=um allmodconfig samples/
  [ snip ]
  CC [M]  samples/configfs/configfs_sample.o
  CC [M]  samples/kfifo/bytestream-example.o
  CC [M]  samples/kfifo/dma-example.o
  CC [M]  samples/kfifo/inttype-example.o
  CC [M]  samples/kfifo/record-example.o
  CC [M]  samples/kobject/kobject-example.o
  CC [M]  samples/kobject/kset-example.o
  CC [M]  samples/trace_events/trace-events-sample.o
  CC [M]  samples/trace_printk/trace-printk.o
  AR      samples/vfio-mdev/built-in.a
  AR      samples/built-in.a
Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile                              |  8 ++++----
 arch/arc/configs/tb10x_defconfig      |  1 +
 arch/nds32/configs/defconfig          |  1 +
 arch/parisc/configs/a500_defconfig    |  1 +
 arch/parisc/configs/b180_defconfig    |  1 +
 arch/parisc/configs/c3000_defconfig   |  1 +
 arch/parisc/configs/default_defconfig |  1 +
 arch/powerpc/configs/ppc6xx_defconfig |  1 +
 arch/s390/configs/debug_defconfig     |  1 +
 lib/Kconfig.debug                     | 19 ++++++++++++++-----
 samples/Kconfig                       | 14 +++++++++++---
 samples/Makefile                      |  4 ++--
 12 files changed, 39 insertions(+), 14 deletions(-)
Makefile +4 -4
index 8348939765d3..ce5a9551860d 100644
@@ -1053,9 +1053,6 @@ vmlinux: scripts/link-vmlinux.sh autoksyms_recursive $(vmlinux-deps) FORCE
 
 targets := vmlinux
 
-# Some samples need headers_install.
-samples: headers_install
-
 # The actual objects are generated when descending,
 # make sure no implicit rule kicks in
 $(sort $(vmlinux-deps)): $(vmlinux-dirs) ;

@@ -1199,6 +1196,10 @@ headers_check: headers_install
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include HDRCHECK=1
 	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include HDRCHECK=1
 
+ifdef CONFIG_HEADERS_INSTALL
+prepare: headers_install
+endif
+
 ifdef CONFIG_HEADERS_CHECK
 all: headers_check
 endif

@@ -1745,7 +1746,6 @@ build-dir = $(patsubst %/,%,$(dir $(build-target)))
 PHONY += /
 /: ./
 
-samples/: headers_install
 %/: prepare FORCE
 	$(Q)$(MAKE) KBUILD_MODULES=1 $(build)=$(build-dir)
arch/arc/configs/tb10x_defconfig +1 -0
index 5b5119d2b5d5..dc739bd093e3 100644
@@ -94,6 +94,7 @@ CONFIG_CONFIGFS_FS=y
 CONFIG_DEBUG_INFO=y
 CONFIG_STRIP_ASM_SYMS=y
 CONFIG_DEBUG_FS=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_DEBUG_SECTION_MISMATCH=y
 CONFIG_MAGIC_SYSRQ=y
arch/nds32/configs/defconfig +1 -0
index 65ce9259081b..40313a635075 100644
@@ -92,6 +92,7 @@ CONFIG_DEBUG_INFO=y
 CONFIG_DEBUG_INFO_DWARF4=y
 CONFIG_GDB_SCRIPTS=y
 CONFIG_READABLE_ASM=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_DEBUG_SECTION_MISMATCH=y
 CONFIG_MAGIC_SYSRQ=y
arch/parisc/configs/a500_defconfig +1 -0
index 5acb93dcaabf..390c0bc09179 100644
@@ -167,6 +167,7 @@ CONFIG_NLS_ISO8859_1=m
 CONFIG_NLS_ISO8859_15=m
 CONFIG_NLS_UTF8=m
 CONFIG_DEBUG_FS=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_MAGIC_SYSRQ=y
 # CONFIG_DEBUG_BUGVERBOSE is not set
arch/parisc/configs/b180_defconfig +1 -0
index 83ffd161aec5..bdf1fe2b217f 100644
@@ -91,6 +91,7 @@ CONFIG_NLS_ASCII=m
 CONFIG_NLS_ISO8859_1=m
 CONFIG_NLS_ISO8859_15=m
 CONFIG_NLS_UTF8=m
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_KERNEL=y
arch/parisc/configs/c3000_defconfig +1 -0
index 8d41a73bd71b..ed4d49575b38 100644
@@ -140,6 +140,7 @@ CONFIG_NLS_ISO8859_1=m
 CONFIG_NLS_ISO8859_15=m
 CONFIG_NLS_UTF8=m
 CONFIG_DEBUG_FS=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_MUTEXES=y
arch/parisc/configs/default_defconfig +1 -0
index 52c9050a7c5c..fcfd9eaadf9b 100644
@@ -184,6 +184,7 @@ CONFIG_NLS_KOI8_R=m
 CONFIG_NLS_KOI8_U=m
 CONFIG_NLS_UTF8=y
 CONFIG_DEBUG_FS=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_KERNEL=y
arch/powerpc/configs/ppc6xx_defconfig +1 -0
index 7c6baf6df139..463aa3e53084 100644
@@ -1124,6 +1124,7 @@ CONFIG_NLS_KOI8_R=m
 CONFIG_NLS_KOI8_U=m
 CONFIG_DEBUG_INFO=y
 CONFIG_UNUSED_SYMBOLS=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_MAGIC_SYSRQ=y
 CONFIG_DEBUG_KERNEL=y
arch/s390/configs/debug_defconfig +1 -0
index b0920b35f87b..994e03fad424 100644
@@ -586,6 +586,7 @@ CONFIG_GDB_SCRIPTS=y
 CONFIG_FRAME_WARN=1024
 CONFIG_READABLE_ASM=y
 CONFIG_UNUSED_SYMBOLS=y
+CONFIG_HEADERS_INSTALL=y
 CONFIG_HEADERS_CHECK=y
 CONFIG_DEBUG_SECTION_MISMATCH=y
 CONFIG_MAGIC_SYSRQ=y
lib/Kconfig.debug +14 -5
index 7cdcb962358c..6a6ea4219d1e 100644
@@ -305,14 +305,23 @@ config DEBUG_FS
 
 	  If unsure, say N.
 
+config HEADERS_INSTALL
+	bool "Install uapi headers to usr/include"
+	depends on !UML
+	help
+	  This option will install uapi headers (headers exported to user-space)
+	  into the usr/include directory for use during the kernel build.
+	  This is unneeded for building the kernel itself, but needed for some
+	  user-space program samples. It is also needed by some features such
+	  as uapi header sanity checks.
+
 config HEADERS_CHECK
 	bool "Run sanity checks on uapi headers when building 'all'"
-	depends on !UML
+	depends on HEADERS_INSTALL
 	help
-	  This option will extract the user-visible kernel headers when
-	  building the 'all' target, and will run basic sanity checks on them to
-	  ensure that exported files do not attempt to include files which
-	  were not exported, etc.
+	  This option will run basic sanity checks on uapi headers when
+	  building the 'all' target, for example, ensure that they do not
+	  attempt to include files which were not exported, etc.
 
 	  If you're making modifications to header files which are
 	  relevant for userspace, say 'Y', and check the headers
samples/Kconfig +11 -3
index d63cc8a3e0df..71b5e833dd9e 100644
@@ -1,7 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 menuconfig SAMPLES
 	bool "Sample kernel code"
-	depends on !UML
 	help
 	  You can build and test sample kernel code here.
 

@@ -95,16 +94,24 @@ config SAMPLE_CONFIGFS
 
 config SAMPLE_CONNECTOR
 	tristate "Build connector sample -- loadable modules only"
-	depends on CONNECTOR && m
+	depends on CONNECTOR && HEADERS_INSTALL && m
 	help
 	  When enabled, this builds both a sample kernel module for
 	  the connector interface and a user space tool to communicate
 	  with it.
 	  See also Documentation/connector/connector.txt
 
+config SAMPLE_HIDRAW
+	bool "hidraw sample"
+	depends on HEADERS_INSTALL
+
+config SAMPLE_PIDFD
+	bool "pidfd sample"
+	depends on HEADERS_INSTALL
+
 config SAMPLE_SECCOMP
 	bool "Build seccomp sample code"
-	depends on SECCOMP_FILTER
+	depends on SECCOMP_FILTER && HEADERS_INSTALL
 	help
 	  Build samples of seccomp filters using various methods of
 	  BPF filter construction.

@@ -156,6 +163,7 @@ config SAMPLE_ANDROID_BINDERFS
 
 config SAMPLE_VFS
 	bool "Build example programs that use new VFS system calls"
+	depends on HEADERS_INSTALL
 	help
 	  Build example userspace programs that use new VFS system calls such
 	  as mount API and statx().  Note that this is restricted to the x86
samples/Makefile +2 -2
index debf8925f06f..7d6e4ca28d69 100644
@@ -4,14 +4,14 @@
 obj-$(CONFIG_SAMPLE_ANDROID_BINDERFS)	+= binderfs/
 obj-$(CONFIG_SAMPLE_CONFIGFS)		+= configfs/
 obj-$(CONFIG_SAMPLE_CONNECTOR)		+= connector/
-subdir-y				+= hidraw
+subdir-$(CONFIG_SAMPLE_HIDRAW)		+= hidraw
 obj-$(CONFIG_SAMPLE_HW_BREAKPOINT)	+= hw_breakpoint/
 obj-$(CONFIG_SAMPLE_KDB)		+= kdb/
 obj-$(CONFIG_SAMPLE_KFIFO)		+= kfifo/
 obj-$(CONFIG_SAMPLE_KOBJECT)		+= kobject/
 obj-$(CONFIG_SAMPLE_KPROBES)		+= kprobes/
 obj-$(CONFIG_SAMPLE_LIVEPATCH)		+= livepatch/
-subdir-y				+= pidfd
+subdir-$(CONFIG_SAMPLE_PIDFD)		+= pidfd
 obj-$(CONFIG_SAMPLE_QMI_CLIENT)		+= qmi/
 obj-$(CONFIG_SAMPLE_RPMSG_CLIENT)	+= rpmsg/
 subdir-$(CONFIG_SAMPLE_SECCOMP)		+= seccomp
Signature · 1 line
2.17.1
author
#7 lore raw

PATCH 6/15 kbuild: remove build_unifdef target in scripts/Makefile

In reply to Masahiro Yamada

Since commit 2aedcd098a94 ("kbuild: suppress annoying "... is up to date." message"), if_changed and friends nicely suppress "... is up to date"

We do not need per-Makefile tricks.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile         | 2 +-
 scripts/Makefile | 5 -----
 2 files changed, 1 insertion(+), 6 deletions(-)
Makefile +1 -1
index ce5a9551860d..e998c40c94b4 100644
@@ -1182,7 +1182,7 @@ PHONY += archheaders archscripts
 
 PHONY += __headers
 __headers: $(version_h) scripts_basic uapi-asm-generic archheaders archscripts
-	$(Q)$(MAKE) $(build)=scripts build_unifdef
+	$(Q)$(MAKE) $(build)=scripts scripts/unifdef
 
 PHONY += headers_install
 headers_install: __headers
scripts/Makefile +0 -5
index 9d442ee050bd..16bcb8087899 100644
@@ -31,11 +31,6 @@ always		:= $(hostprogs-y) $(hostprogs-m)
 # The following hostprogs-y programs are only build on demand
 hostprogs-y += unifdef
 
-# These targets are used internally to avoid "is up to date" messages
-PHONY += build_unifdef
-build_unifdef: $(obj)/unifdef
-	@:
-
 subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins
 subdir-$(CONFIG_MODVERSIONS) += genksyms
 subdir-$(CONFIG_SECURITY_SELINUX) += selinux
Signature · 1 line
2.17.1
author
#8 lore raw

PATCH 7/15 kbuild: build all prerequisite of headers_install simultaneously

In reply to Masahiro Yamada

Currently, scripts/unifdef is compiled after scripts_basic, uapi-asm-generic, archheaders, and archscripts.

The proper dependency is just scripts_basic. There is no problem to compile scripts/unifdef and other headers at the same time.

Split scripts_unifdef out in order to allow more parallel building.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
Makefile +5 -2
index e998c40c94b4..f9c206eb3583 100644
@@ -1181,8 +1181,7 @@ export INSTALL_HDR_PATH = $(objtree)/usr
 PHONY += archheaders archscripts
 
 PHONY += __headers
-__headers: $(version_h) scripts_basic uapi-asm-generic archheaders archscripts
-	$(Q)$(MAKE) $(build)=scripts scripts/unifdef
+__headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts
 
 PHONY += headers_install
 headers_install: __headers

@@ -1204,6 +1203,10 @@ ifdef CONFIG_HEADERS_CHECK
 all: headers_check
 endif
 
+PHONY += scripts_unifdef
+scripts_unifdef: scripts_basic
+	$(Q)$(MAKE) $(build)=scripts scripts/unifdef
+
 # ---------------------------------------------------------------------------
 # Kernel selftest
Signature · 1 line
2.17.1
author
#9 lore raw

PATCH 8/15 kbuild: add 'headers' target to build up ready-to-install uapi headers

In reply to Masahiro Yamada

In Linux build system, build targets and installation targets are separated.

Examples are:

- 'make vmlinux' -> 'make install'
- 'make modules' -> 'make modules_install'
- 'make dtbs' -> 'make dtbs_install'
- 'make vdso' -> 'make vdso_install'

The intention is to run the build targets under the normal privilege, then the installation targets under the root privilege since we need the write permission to the system directories.

We have 'make headers_install" but the corresponding 'make headers' stage does not exist. The purpose of headers_install is to provide the kernel interface to C library. So, nobody would try to install headers to /usr/include directly.

If 'sudo make INSTALL_HDR_PATH=/usr/include headers_install' were run, some build artifacts in the kernel tree would be owned by root because some of uapi headers are generated by 'uapi-asm-generic', 'archheaders' targets.

Anyway, I believe it makes sense to split the header installation into two stages.

[1] 'make headers'
Process headers in uapi directories by scripts/headers_install.sh and copy them to usr/include

[2] 'make headers_install'

    Copy '*.h' verbatim from usr/include to $(INSTALL_HDR_PATH)/include

For the backward compatibility, 'headers_install' depends on 'headers'.

Some samples expect uapi headers in usr/include. So, the 'headers' target is useful to build up them in the fixed location usr/include irrespective of INSTALL_HDR_PATH.

Another benefit is to stop polluting the final destination with the time-stamp files '.install' and '.check'. Maybe you can see them in your toolchains.

Lastly, my main motivation is to prepare for compile-testing uapi headers. To build something, we have to creating an object and .*.cmd somewhere. The usr/include/ will be the work directory for that.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile                     | 23 +++++++++++++++--------
 lib/Kconfig.debug            |  4 +---
 scripts/Makefile.headersinst |  8 ++++----
 3 files changed, 20 insertions(+), 15 deletions(-)
Makefile +15 -8
index f9c206eb3583..3c172dd516ff 100644
@@ -262,7 +262,7 @@ old_version_h := include/linux/version.h
 clean-targets := %clean mrproper cleandocs
 no-dot-config-targets := $(clean-targets) \
 			 cscope gtags TAGS tags help% %docs check% coccicheck \
-			 $(version_h) headers_% archheaders archscripts \
+			 $(version_h) headers headers_% archheaders archscripts \
 			 %asm-generic kernelversion %src-pkg
 no-sync-config-targets := $(no-dot-config-targets) install %install \
 			   kernelrelease

@@ -1178,25 +1178,32 @@ headerdep:
 #Default location for installed headers
 export INSTALL_HDR_PATH = $(objtree)/usr
 
-PHONY += archheaders archscripts
-
-PHONY += __headers
-__headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts
+quiet_cmd_headers_install = INSTALL $(INSTALL_HDR_PATH)/include
+      cmd_headers_install = \
+	mkdir -p $(INSTALL_HDR_PATH); \
+	rsync -mrl --include='*/' --include='*\.h' --exclude='*' \
+	usr/include $(INSTALL_HDR_PATH)
 
 PHONY += headers_install
-headers_install: __headers
+headers_install: headers
+	$(call cmd,headers_install)
+
+PHONY += archheaders archscripts
+
+PHONY += headers
+headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts
 	$(if $(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/Kbuild),, \
 	  $(error Headers not exportable for the $(SRCARCH) architecture))
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include
 	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include
 
 PHONY += headers_check
-headers_check: headers_install
+headers_check: headers
 	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include HDRCHECK=1
 	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include HDRCHECK=1
 
 ifdef CONFIG_HEADERS_INSTALL
-prepare: headers_install
+prepare: headers
 endif
 
 ifdef CONFIG_HEADERS_CHECK
lib/Kconfig.debug +1 -3
index 6a6ea4219d1e..0031a31d98c2 100644
@@ -324,9 +324,7 @@ config HEADERS_CHECK
 	  attempt to include files which were not exported, etc.
 
 	  If you're making modifications to header files which are
-	  relevant for userspace, say 'Y', and check the headers
-	  exported to $(INSTALL_HDR_PATH) (usually 'usr/include' in
-	  your build tree), to make sure they're suitable.
+	  relevant for userspace, say 'Y'.
 
 config OPTIMIZE_INLINING
 	bool "Allow compiler to uninline functions marked 'inline'"
scripts/Makefile.headersinst +4 -4
index 3d1ebaabd1b6..1af6d0b06585 100644
@@ -41,7 +41,7 @@ ifeq ($(skip-inst),)
 kbuild-file := $(srctree)/$(obj)/Kbuild
 -include $(kbuild-file)
 
-installdir    := $(INSTALL_HDR_PATH)/$(dst)
+installdir    := usr/$(dst)
 gendir        := $(objtree)/$(subst include/,include/generated/,$(obj))
 header-files  := $(notdir $(wildcard $(srcdir)/*.h))
 header-files  := $(filter-out $(no-export-headers), $(header-files))

@@ -60,10 +60,10 @@ output-files  := $(addprefix $(installdir)/, $(all-files))
 oldheaders    := $(patsubst $(installdir)/%,%,$(wildcard $(installdir)/*.h))
 unwanted      := $(filter-out $(all-files),$(oldheaders))
 
-# Prefix unwanted with full paths to $(INSTALL_HDR_PATH)
+# Prefix unwanted with full paths to objtree
 unwanted-file := $(addprefix $(installdir)/, $(unwanted))
 
-printdir = $(patsubst $(INSTALL_HDR_PATH)/%/,%,$(dir $@))
+printdir = $(patsubst %/,%,$(dir $@))
 
 quiet_cmd_install = INSTALL $(printdir) ($(words $(all-files))\
                             file$(if $(word 2, $(all-files)),s))

@@ -81,7 +81,7 @@ quiet_cmd_check = CHECK   $(printdir) ($(words $(all-files)) files)
       cmd_check = for f in $(all-files); do                          \
                   echo "$(installdir)/$${f}"; done                      \
                   | xargs                                            \
-                  $(PERL) $< $(INSTALL_HDR_PATH)/include $(SRCARCH); \
+                  $(PERL) $< usr/include $(SRCARCH); \
 	          touch $@
 
 ifndef HDRCHECK
Signature · 1 line
2.17.1
author
#10 lore raw

PATCH 9/15 kbuild: re-implement Makefile.headersinst without directory descending

In reply to Masahiro Yamada

Since commit fcc8487d477a ("uapi: export all headers under uapi directories"), the headers in uapi directories are all exported by default although exceptional cases are still allowed by the syntax 'no-export-headers'.

The traditional directory descending has been still kept (in a somewhat hacky way), but it is no longer needed.

Get rid of it to simplify the code.

Also, handle files one by one instead of the previous per-directory processing. This will emit much more log, but I like it.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile                        |   8 +-
 include/uapi/{linux => }/Kbuild |   6 +-
 scripts/Makefile.headersinst    | 132 +++++++++++++-------------------
 3 files changed, 62 insertions(+), 84 deletions(-)
 rename include/uapi/{linux => }/Kbuild (77%)
Makefile +4 -4
index 3c172dd516ff..608af32cc758 100644
@@ -1194,13 +1194,13 @@ PHONY += headers
 headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts
 	$(if $(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/Kbuild),, \
 	  $(error Headers not exportable for the $(SRCARCH) architecture))
-	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include
-	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include
+	$(Q)$(MAKE) $(hdr-inst)=include/uapi
+	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi
 
 PHONY += headers_check
 headers_check: headers
-	$(Q)$(MAKE) $(hdr-inst)=include/uapi dst=include HDRCHECK=1
-	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi dst=include HDRCHECK=1
+	$(Q)$(MAKE) $(hdr-inst)=include/uapi HDRCHECK=1
+	$(Q)$(MAKE) $(hdr-inst)=arch/$(SRCARCH)/include/uapi HDRCHECK=1
 
 ifdef CONFIG_HEADERS_INSTALL
 prepare: headers
include/uapi/Kbuild renamed from include/uapi/linux/Kbuild +3 -3
similarity index 77%
rename from include/uapi/linux/Kbuild
rename to include/uapi/Kbuild
index 34711c5d6968..61ee6e59c930 100644
@@ -1,14 +1,14 @@
 # SPDX-License-Identifier: GPL-2.0
 ifeq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/a.out.h),)
-no-export-headers += a.out.h
+no-export-headers += linux/a.out.h
 endif
 
 ifeq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/kvm.h),)
-no-export-headers += kvm.h
+no-export-headers += linux/kvm.h
 endif
 
 ifeq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/kvm_para.h),)
 ifeq ($(wildcard $(objtree)/arch/$(SRCARCH)/include/generated/uapi/asm/kvm_para.h),)
-no-export-headers += kvm_para.h
+no-export-headers += linux/kvm_para.h
 endif
 endif
scripts/Makefile.headersinst +55 -77

164 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#11 lore raw

PATCH 10/15 kbuild: move hdr-inst shorthand to top Makefile

In reply to Masahiro Yamada

Now that hdr-inst is used only in the top Makefile, move it there from scripts/Kbuild.include.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile               | 2 ++
 scripts/Kbuild.include | 6 ------
 2 files changed, 2 insertions(+), 6 deletions(-)
Makefile +2 -0
index 608af32cc758..48bac02fb72d 100644
@@ -1190,6 +1190,8 @@ headers_install: headers
 
 PHONY += archheaders archscripts
 
+hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj
+
 PHONY += headers
 headers: $(version_h) scripts_unifdef uapi-asm-generic archheaders archscripts
 	$(if $(wildcard $(srctree)/arch/$(SRCARCH)/include/uapi/asm/Kbuild),, \
scripts/Kbuild.include +0 -6
index f12e81eeb89b..9ae4493d5bcd 100644
@@ -180,12 +180,6 @@ dtbinst := -f $(srctree)/scripts/Makefile.dtbinst obj
 # $(Q)$(MAKE) $(clean)=dir
 clean := -f $(srctree)/scripts/Makefile.clean obj
 
-###
-# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.headersinst obj=
-# Usage:
-# $(Q)$(MAKE) $(hdr-inst)=dir
-hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj
-
 # echo command.
 # Short version is used, if $(quiet) equals `quiet_', otherwise full one.
 echo-cmd = $(if $($(quiet)cmd_$(1)),\
Signature · 1 line
2.17.1
author
#12 lore raw

PATCH 11/15 kbuild: simplify scripts/headers_install.sh

In reply to Masahiro Yamada

Now that headers_install.sh is invoked per file, remove the for-loop in the shell script.

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/Makefile.headersinst |  2 +-
 scripts/headers_install.sh   | 48 +++++++++++++++---------------------
 2 files changed, 21 insertions(+), 29 deletions(-)
scripts/Makefile.headersinst +1 -1
index c96c4c26e240..d2b572a7a628 100644
@@ -58,7 +58,7 @@ $(if $(new-dirs), $(shell mkdir -p $(new-dirs)))
 ifndef HDRCHECK
 
 quiet_cmd_install = HDRINST $@
-      cmd_install = $(CONFIG_SHELL) $(srctree)/scripts/headers_install.sh $(@D) $(<D) $(@F)
+      cmd_install = $(CONFIG_SHELL) $(srctree)/scripts/headers_install.sh $< $@
 
 $(src-headers): $(dst)/%.h: $(src)/%.h $(srctree)/scripts/headers_install.sh FORCE
 	$(call if_changed,install)
scripts/headers_install.sh +20 -28

67 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#13 lore raw

PATCH 12/15 kbuild: deb-pkg: do not run headers_check

In reply to Masahiro Yamada

It is absolutely fine to add extra sanity checks in package scripts, but it is not necessary to do so.

This is already covered by the daily compile-testing (0day bot etc.) because headers_check is run as a part of the normal build process when CONFIG_HEADERS_CHECK=y.

Replace it with the newly-added "make headers".

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/package/builddeb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/package/builddeb +1 -1
index b03dd56a4782..e8ca6dc97e96 100755
@@ -130,7 +130,7 @@ if is_enabled CONFIG_MODULES; then
 fi
 
 if [ "$ARCH" != "um" ]; then
-	$MAKE -f $srctree/Makefile headers_check
+	$MAKE -f $srctree/Makefile headers
 	$MAKE -f $srctree/Makefile headers_install INSTALL_HDR_PATH="$libc_headers_dir/usr"
 fi
Signature · 1 line
2.17.1
author
#14 lore raw

PATCH 13/15 kbuild: add support for ensuring headers are self-contained

In reply to Masahiro Yamada

From: Jani Nikula <[email protected]>

Sometimes it's useful to be able to explicitly ensure certain headers remain self-contained, i.e. that they are compilable as standalone units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add headers to be tested if CONFIG_HEADER_TEST is enabled. This will generate a dummy C file per header that gets built as part of extra-y.

Cc Chris Wilson <[email protected]>
Cc Masahiro Yamada <[email protected]>
Cc Michal Marek <[email protected]>
Signed-off-by Jani Nikula <[email protected]>
Signed-off-by Masahiro Yamada <[email protected]>

---

 Documentation/kbuild/makefiles.txt |  7 +++++++
 init/Kconfig                       |  9 +++++++++
 scripts/Makefile.build             | 10 ++++++++++
 scripts/Makefile.lib               |  3 +++
 4 files changed, 29 insertions(+)
Documentation/kbuild/makefiles.txt +7 -0
index bac301a73a86..ca4b24ec0399 100644
@@ -1018,6 +1018,13 @@ When kbuild executes, the following steps are followed (roughly):
 	In this example, extra-y is used to list object files that
 	shall be built, but shall not be linked as part of built-in.a.
 
+    header-test-y
+
+	header-test-y specifies headers (*.h) in the current directory that
+	should be compile tested to ensure they are self-contained,
+	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
+	this autogenerates dummy sources to include the headers, and builds them
+	as part of extra-y.
 
 --- 6.7 Commands useful for building a boot image
init/Kconfig +9 -0
index 36894c9fb420..02d8897b91fb 100644
@@ -96,6 +96,15 @@ config COMPILE_TEST
 	  here. If you are a user/distributor, say N here to exclude useless
 	  drivers to be distributed.
 
+config HEADER_TEST
+	bool "Compile test headers that should be standalone compilable"
+	help
+	  Compile test headers listed in header-test-y target to ensure they are
+	  self-contained, i.e. compilable as standalone units.
+
+	  If you are a developer or tester and want to ensure the requested
+	  headers are self-contained, say Y here. Otherwise, choose N.
+
 config LOCALVERSION
 	string "Local version - append to kernel release"
 	help
scripts/Makefile.build +10 -0
index ae9cf740633e..2b4d56483c2e 100644
@@ -294,6 +294,16 @@ quiet_cmd_cc_lst_c = MKLST   $@
 $(obj)/%.lst: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_lst_c)
 
+# Dummy C sources for header test (header-test-y target)
+# ---------------------------------------------------------------------------
+
+quiet_cmd_header_test = HDRTEST $@
+      cmd_header_test = echo "\#include \"$(<F)\"" > $@
+
+# FIXME: would be nice to be able to limit this implicit rule to header-test-y
+$(obj)/%.header_test.c: $(src)/%.h FORCE
+	$(call if_changed,header_test)
+
 # Compile assembler sources (.S)
 # ---------------------------------------------------------------------------
scripts/Makefile.lib +3 -0
index f1f38c8cdc74..60a739a22b9c 100644
@@ -66,6 +66,9 @@ extra-y += $(patsubst %.dtb,%.dt.yaml, $(dtb-y))
 extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
 endif
 
+# Test self-contained headers
+extra-$(CONFIG_HEADER_TEST) += $(patsubst %.h,%.header_test.o,$(header-test-y))
+
 # Add subdir path
 
 extra-y		:= $(addprefix $(obj)/,$(extra-y))
Signature · 1 line
2.17.1
#15 lore raw

PATCH v2 1/1 kbuild: add support for ensuring headers are self-contained

Sometimes it's useful to be able to explicitly ensure certain headers remain self-contained, i.e. that they are compilable as standalone units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add headers to be tested if CONFIG_HEADER_TEST is enabled. This will generate a dummy C file per header that gets built as part of extra-y.

Cc Chris Wilson <[email protected]>
Cc Masahiro Yamada <[email protected]>
Cc Michal Marek <[email protected]>
Cc Sam Ravnborg <[email protected]>
Signed-off-by Jani Nikula <[email protected]>

---

v2: changes suggested by Masahiro:

http://mid.mail-archive.com/CAK7LNAT=nB0=at0X4OnHVKB=y7WwHGm4LXkrQnCw9HpjB5LooA@mail.gmail.com

---
 .gitignore                         | 1 +
 Documentation/dontdiff             | 1 +
 Documentation/kbuild/makefiles.txt | 7 +++++++
 Makefile                           | 1 +
 init/Kconfig                       | 9 +++++++++
 scripts/Makefile.build             | 9 +++++++++
 scripts/Makefile.lib               | 3 +++
 7 files changed, 31 insertions(+)
.gitignore +1 -0
index 7587ef56b92d..4bb60f0fa23b 100644
@@ -22,6 +22,7 @@
 *.elf
 *.gcno
 *.gz
+*.hdrtest.c
 *.i
 *.ko
 *.lex.c
Documentation/dontdiff +1 -0
index 5eba889ea84d..554dfe4883d2 100644
@@ -19,6 +19,7 @@
 *.grep
 *.grp
 *.gz
+*.hdrtest.c
 *.html
 *.i
 *.jpeg
Documentation/kbuild/makefiles.txt +7 -0
index d65ad5746f94..a31e54bd9ddd 100644
@@ -1022,6 +1022,13 @@ When kbuild executes, the following steps are followed (roughly):
 	In this example, extra-y is used to list object files that
 	shall be built, but shall not be linked as part of built-in.a.
 
+    header-test-y
+
+	header-test-y specifies headers (*.h) in the current directory that
+	should be compile tested to ensure they are self-contained,
+	i.e. compilable as standalone units. If CONFIG_HEADER_TEST is enabled,
+	this autogenerates dummy sources to include the headers, and builds them
+	as part of extra-y.
 
 --- 6.7 Commands useful for building a boot image
Makefile +1 -0
index 11358153d8f2..b347be697fbb 100644
@@ -1646,6 +1646,7 @@ clean: $(clean-dirs)
 		-o -name '*.dwo' -o -name '*.lst' \
 		-o -name '*.su'  \
 		-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \
+		-o -name '*.hdrtest.c' \
 		-o -name '*.lex.c' -o -name '*.tab.[ch]' \
 		-o -name '*.asn1.[ch]' \
 		-o -name '*.symtypes' -o -name 'modules.order' \
init/Kconfig +9 -0
index 36894c9fb420..02d8897b91fb 100644
@@ -96,6 +96,15 @@ config COMPILE_TEST
 	  here. If you are a user/distributor, say N here to exclude useless
 	  drivers to be distributed.
 
+config HEADER_TEST
+	bool "Compile test headers that should be standalone compilable"
+	help
+	  Compile test headers listed in header-test-y target to ensure they are
+	  self-contained, i.e. compilable as standalone units.
+
+	  If you are a developer or tester and want to ensure the requested
+	  headers are self-contained, say Y here. Otherwise, choose N.
+
 config LOCALVERSION
 	string "Local version - append to kernel release"
 	help
scripts/Makefile.build +9 -0
index ae9cf740633e..ee0319560513 100644
@@ -294,6 +294,15 @@ quiet_cmd_cc_lst_c = MKLST   $@
 $(obj)/%.lst: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_lst_c)
 
+# Dummy C sources for header test (header-test-y target)
+# ---------------------------------------------------------------------------
+
+quiet_cmd_header_test = HDRTEST $@
+      cmd_header_test = echo "\#include \"$*.h\"" > $@
+
+$(obj)/%.hdrtest.c:
+	$(call cmd,header_test)
+
 # Compile assembler sources (.S)
 # ---------------------------------------------------------------------------
scripts/Makefile.lib +3 -0
index f1f38c8cdc74..3e630fcaffd1 100644
@@ -66,6 +66,9 @@ extra-y += $(patsubst %.dtb,%.dt.yaml, $(dtb-y))
 extra-$(CONFIG_OF_ALL_DTBS) += $(patsubst %.dtb,%.dt.yaml, $(dtb-))
 endif
 
+# Test self-contained headers
+extra-$(CONFIG_HEADER_TEST) += $(patsubst %.h,%.hdrtest.o,$(header-test-y))
+
 # Add subdir path
 
 extra-y		:= $(addprefix $(obj)/,$(extra-y))
Signature · 1 line
2.20.1
Reviewed-by on v2 1/1
#16 lore raw

Re: [PATCH v2] kbuild: add support for ensuring headers are self-contained

Quoted 8 lines · Jani Nikula

Sometimes it's useful to be able to explicitly ensure certain headers remain self-contained, i.e. that they are compilable as standalone units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add headers to be tested if CONFIG_HEADER_TEST is enabled. This will generate a dummy C file per header that gets built as part of extra-y.

Quoted 5 lines
Cc Chris Wilson <[email protected]>
Cc Masahiro Yamada <[email protected]>
Cc Michal Marek <[email protected]>
Cc Sam Ravnborg <[email protected]>
Signed-off-by Jani Nikula <[email protected]>

Looks good, thanks.

Reviewed-by Sam Ravnborg <[email protected]>
author
#17 lore raw

Re: [PATCH v2] kbuild: add support for ensuring headers are self-contained

Quoted 16 lines · Sam Ravnborg
Quoted 8 lines · Jani Nikula

Sometimes it's useful to be able to explicitly ensure certain headers remain self-contained, i.e. that they are compilable as standalone units, by including and/or forward declaring everything they depend on.

Add special target header-test-y where individual Makefiles can add headers to be tested if CONFIG_HEADER_TEST is enabled. This will generate a dummy C file per header that gets built as part of extra-y.

Quoted 5 lines
Cc Chris Wilson <[email protected]>
Cc Masahiro Yamada <[email protected]>
Cc Michal Marek <[email protected]>
Cc Sam Ravnborg <[email protected]>
Signed-off-by Jani Nikula <[email protected]>
Quoted 4 lines

Looks good, thanks.

Reviewed-by Sam Ravnborg <[email protected]>

Applied to linux-kbuild.
Thanks.

Signature · 2 lines
Best Regards
Masahiro Yamada
author
#18 lore raw

PATCH 14/15 fixup: kbuild: add support for ensuring headers are self-contained

In reply to Masahiro Yamada

This is needed to avoid regeneration of header-test C files.

This will go away when Jani send v2.

Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/Makefile.build | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
scripts/Makefile.build +2 -3
index 2b4d56483c2e..067e98628664 100644
@@ -298,10 +298,9 @@ $(obj)/%.lst: $(src)/%.c FORCE
 # ---------------------------------------------------------------------------
 
 quiet_cmd_header_test = HDRTEST $@
-      cmd_header_test = echo "\#include \"$(<F)\"" > $@
+      cmd_header_test = echo "\#include \"$*.h\"" > $@
 
-# FIXME: would be nice to be able to limit this implicit rule to header-test-y
-$(obj)/%.header_test.c: $(src)/%.h FORCE
+$(obj)/%.header_test.c:
 	$(call if_changed,header_test)
 
 # Compile assembler sources (.S)
Signature · 1 line
2.17.1
#19 lore raw

Re: [PATCH 14/15] fixup: kbuild: add support for ensuring headers are self-contained

Quoted 3 lines · Masahiro Yamada

This is needed to avoid regeneration of header-test C files.

This will go away when Jani send v2.

Done, in reply to patch 13.

Thanks,
Jani.

Quoted 24 lines
Signed-off-by Masahiro Yamada <[email protected]>

---

 scripts/Makefile.build | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
scripts/Makefile.build +2 -3
index 2b4d56483c2e..067e98628664 100644
@@ -298,10 +298,9 @@ $(obj)/%.lst: $(src)/%.c FORCE
 # ---------------------------------------------------------------------------
 
 quiet_cmd_header_test = HDRTEST $@
-      cmd_header_test = echo "\#include \"$(<F)\"" > $@
+      cmd_header_test = echo "\#include \"$*.h\"" > $@
 
-# FIXME: would be nice to be able to limit this implicit rule to header-test-y
-$(obj)/%.header_test.c: $(src)/%.h FORCE
+$(obj)/%.header_test.c:
 	$(call if_changed,header_test)
 
 # Compile assembler sources (.S)
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
author
#20 lore raw

PATCH 15/15 kbuild: compile test UAPI headers to ensure they are self-contained

In reply to Masahiro Yamada

Multiple people have suggested compile-testing UAPI headers to ensure they can be really included from user-space. "make headers_check" is obviously not enough to catch bugs, and we often leak references to kernel-space definition to user-space.

The most recent patch I know is David Howells' work:

https://patchwork.kernel.org/patch/10590203/

While I agree that we should do this, we must consider how it can be integrated cleanly. That is why it has not been supported in the mainline yet.

The idea that has been in my mind is to compile every UAPI header so that it can be included without relying on any include order.

Please note usr/include/ is built with a completely different set of compiler flags. The header search path is set to $(objtree)/usr/include since UAPI headers should not include unexported headers.

We use -std=gnu89 for the kernel space since the kernel code highly depends on GNU extensions. On the other hand, UAPI headers should be written in somewhat strict C, so they are compiled with -std=c89. This will catch C++ style comments, the keyword 'inline', etc. ('__inline__' should be used instead).

There is additional compiler requirement for building under usr/include. because many of UAPI headers include <stdlib.h>, <sys/ioctl.h>, <sys/time.h>, etc. directly or indirectly.

You can use kernel.org prebuilt toolchains for building the kernel
(https://mirrors.edge.kernel.org/pub/tools/crosstool/index.html)
but they do not provide <stdlib.h> etc.

If you want to compile test UAPI headers, you need to use full-featured compilers, which are usually provided by distributions.

I added scripts/cc-system-headers.sh to check if necessary system headers are available, which CONFIG_UAPI_HEADER_TEST depends on.

For now, a lot of headers need to be excluded because they cannot be compiled standalone, but this is a good start point.

Signed-off-by Masahiro Yamada <[email protected]>

---

 Makefile                     |   2 +-
 init/Kconfig                 |  11 +++
 scripts/cc-system-headers.sh |   8 +++
 usr/.gitignore               |   1 -
 usr/Makefile                 |   2 +
 usr/include/.gitignore       |   3 +
 usr/include/Makefile         | 132 +++++++++++++++++++++++++++++++++++
 7 files changed, 157 insertions(+), 2 deletions(-)
 create mode 100755 scripts/cc-system-headers.sh
 create mode 100644 usr/include/.gitignore
 create mode 100644 usr/include/Makefile
Makefile +1 -1
index 48bac02fb72d..0d54b073c415 100644
@@ -1363,7 +1363,7 @@ CLEAN_DIRS  += $(MODVERDIR) include/ksym
 CLEAN_FILES += modules.builtin.modinfo
 
 # Directories & files removed with 'make mrproper'
-MRPROPER_DIRS  += include/config usr/include include/generated          \
+MRPROPER_DIRS  += include/config include/generated          \
 		  arch/$(SRCARCH)/include/generated .tmp_objdiff
 MRPROPER_FILES += .config .config.old .version \
 		  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \
init/Kconfig +11 -0
index 02d8897b91fb..9a26f0e7e3fb 100644
@@ -105,6 +105,17 @@ config HEADER_TEST
 	  If you are a developer or tester and want to ensure the requested
 	  headers are self-contained, say Y here. Otherwise, choose N.
 
+config UAPI_HEADER_TEST
+	bool "Compile test UAPI headers"
+	depends on HEADER_TEST && HEADERS_INSTALL
+	depends on $(success,$(srctree)/scripts/cc-system-headers.sh $(CC))
+	help
+	  Compile test headers exported to user-space to ensure they are
+	  self-contained, i.e. compilable as standalone units.
+
+	  If you are a developer or tester and want to ensure the UAPI
+	  headers are self-contained, say Y here. Otherwise, choose N.
+
 config LOCALVERSION
 	string "Local version - append to kernel release"
 	help
scripts/cc-system-headers.sh new file +8 -0
new file mode 100755
index 000000000000..1b3db369828c
@@ -0,0 +1,8 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+
+cat << "END" | $@ -E -x c - -o /dev/null >/dev/null 2>&1
+#include <stdlib.h>
+#include <sys/ioctl.h>
+#include <sys/time.h>
+END
usr/.gitignore +0 -1
index 8e48117a3f3d..be5eae1df7eb 100644
@@ -7,4 +7,3 @@ initramfs_data.cpio.gz
 initramfs_data.cpio.bz2
 initramfs_data.cpio.lzma
 initramfs_list
-include
usr/Makefile +2 -0
index 4a70ae43c9cb..6a89eb019275 100644
@@ -56,3 +56,5 @@ $(deps_initramfs): klibcdirs
 $(obj)/$(datafile_y): $(obj)/gen_init_cpio $(deps_initramfs) klibcdirs
 	$(Q)$(initramfs) -l $(ramfs-input) > $(obj)/$(datafile_d_y)
 	$(call if_changed,initfs)
+
+subdir-$(CONFIG_UAPI_HEADER_TEST) += include
usr/include/.gitignore new file +3 -0
new file mode 100644
index 000000000000..a0991ff4402b
@@ -0,0 +1,3 @@
+*
+!.gitignore
+!Makefile
usr/include/Makefile new file +132 -0

132 lines not on this page · raw message

Signature · 1 line
2.17.1
#21 lore raw

Re: [PATCH 15/15] kbuild: compile test UAPI headers to ensure they are self-contained

Makefile CLEAN_DIRS += $(MODVERDIR) include/ksym · line 1369

Quoted 11 lines · Masahiro Yamada
Makefile +1 -1
@@ -1363,7 +1363,7 @@ CLEAN_DIRS  += $(MODVERDIR) include/ksym
 CLEAN_FILES += modules.builtin.modinfo
 
 # Directories & files removed with 'make mrproper'
-MRPROPER_DIRS  += include/config usr/include include/generated          \
+MRPROPER_DIRS  += include/config include/generated          \
                  arch/$(SRCARCH)/include/generated .tmp_objdiff
 MRPROPER_FILES += .config .config.old .version \
                  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \

This change seems to have caused a minor regression:

$ make clean ; make clean
find: ‘*’: No such file or directory

Any idea?

Arnd

author
#22 lore raw

Re: [PATCH 15/15] kbuild: compile test UAPI headers to ensure they are self-contained

Quoted 18 lines · Arnd Bergmann

Makefile CLEAN_DIRS += $(MODVERDIR) include/ksym · line 1369

Quoted 11 lines · Masahiro Yamada
Makefile +1 -1
@@ -1363,7 +1363,7 @@ CLEAN_DIRS  += $(MODVERDIR) include/ksym
 CLEAN_FILES += modules.builtin.modinfo
 
 # Directories & files removed with 'make mrproper'
-MRPROPER_DIRS  += include/config usr/include include/generated          \
+MRPROPER_DIRS  += include/config include/generated          \
                  arch/$(SRCARCH)/include/generated .tmp_objdiff
 MRPROPER_FILES += .config .config.old .version \
                  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \

This change seems to have caused a minor regression:

Quoted 2 lines
$ make clean ; make clean
find: ‘*’: No such file or directory

Hmm, I cannot reproduce this.

I checked the latest linux-next.

masahiro@grover:~/ref/linux-next$ git describe
next-20190621
masahiro@grover:~/ref/linux-next$ make clean; make clean
masahiro@grover:~/ref/linux-next$
Quoted 3 lines

Any idea?

Arnd

Signature · 2 lines
Best Regards
Masahiro Yamada
author
#23 lore raw

Re: [PATCH 15/15] kbuild: compile test UAPI headers to ensure they are self-contained

Quoted 25 lines · Masahiro Yamada
Quoted 18 lines · Arnd Bergmann

Makefile CLEAN_DIRS += $(MODVERDIR) include/ksym · line 1369

Quoted 11 lines · Masahiro Yamada
Makefile +1 -1
@@ -1363,7 +1363,7 @@ CLEAN_DIRS  += $(MODVERDIR) include/ksym
 CLEAN_FILES += modules.builtin.modinfo
 
 # Directories & files removed with 'make mrproper'
-MRPROPER_DIRS  += include/config usr/include include/generated          \
+MRPROPER_DIRS  += include/config include/generated          \
                  arch/$(SRCARCH)/include/generated .tmp_objdiff
 MRPROPER_FILES += .config .config.old .version \
                  Module.symvers tags TAGS cscope* GPATH GTAGS GRTAGS GSYMS \

This change seems to have caused a minor regression:

Quoted 2 lines
$ make clean ; make clean
find: ‘*’: No such file or directory

Hmm, I cannot reproduce this.

Quoted 9 lines

I checked the latest linux-next.

masahiro@grover:~/ref/linux-next$ git describe
next-20190621
masahiro@grover:~/ref/linux-next$ make clean; make clean
masahiro@grover:~/ref/linux-next$

Ah, now I was able to reproduce it.

Will fix it soon. Thanks.

Signature · 2 lines
Best Regards
Masahiro Yamada
author
#24 lore raw
In reply to Masahiro Yamada
Quoted 7 lines · Masahiro Yamada

Multiple people have suggested to compile-test UAPI headers.

Currently, Kbuild provides simple sanity checks by headers_check but they are not enough to catch bugs.

Quoted 8 lines

The most recent patch I know is David Howells' work:
https://patchwork.kernel.org/patch/10590203/

I agree that we need better tests for UAPI headers, but I want to integrate it in a clean way.

The idea that has been in my mind is to compile each header to make sure the selfcontainedness.

For convenience, I pushed this series at

git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
uapi-header-test-v1

(13/15 was replaced with v2)

If you want to test it quickly, please check-out it, then

  $ make -j8 allmodconfig usr/

(As I noted in the commit log, you need to use a compiler that provides <stdlib.h>, <sys/time.h>, etc.)

Signature · 2 lines
Best Regards
Masahiro Yamada
author
#25 lore raw
In reply to Masahiro Yamada
Quoted 66 lines · Masahiro Yamada

Multiple people have suggested to compile-test UAPI headers.

Currently, Kbuild provides simple sanity checks by headers_check but they are not enough to catch bugs.

The most recent patch I know is David Howells' work:
https://patchwork.kernel.org/patch/10590203/

I agree that we need better tests for UAPI headers, but I want to integrate it in a clean way.

The idea that has been in my mind is to compile each header to make sure the selfcontainedness.

Recently, Jani Nikula proposed a new syntax 'header-test-y'.
https://patchwork.kernel.org/patch/10947005/

So, I implemented UAPI compile-testing on top of that.

When adding a new feature, cleaning the code first is a good practice.

[1] Remove headers_install_all

This target installs UAPI headers of all architectures in a single tree.
It does not make sense to compile test of headers from multiple arches at the same time. Hence, removed.

[2] Split header installation into 'make headers' and 'make headers_install'

To compile-test UAPI headers, we need a work-directory somewhere to save objects and .*.cmd files.

usr/include/ will be the work-directory.

Since we cannot pollute the final destination of headers_install,

I split the header installation into two stages.

'make headers' will build up
the ready-to-install headers in usr/include,
which will be also used as a work-directory for the compile-test.

'make headers_install' will copy headers
from usr/include to $(INSTALL_HDR_PATH)/include.

[3] Support compile-test of UAPI headers

This is implemented in usr/include/Makefile

Jani Nikula (1):
kbuild: add support for ensuring headers are self-contained

Masahiro Yamada (14):
kbuild: remove headers_{install,check}_all
kbuild: remove stale dependency between Documentation/ and
headers_install
kbuild: make gdb_script depend on prepare0 instead of prepare
kbuild: fix Kconfig prompt of CONFIG_HEADERS_CHECK
kbuild: add CONFIG_HEADERS_INSTALL and loosen the dependency of
samples
kbuild: remove build_unifdef target in scripts/Makefile

Quoted 8 lines

kbuild: build all prerequisite of headers_install simultaneously
kbuild: add 'headers' target to build up ready-to-install uapi headers
kbuild: re-implement Makefile.headersinst without directory descending
kbuild: move hdr-inst shorthand to top Makefile
kbuild: simplify scripts/headers_install.sh
kbuild: deb-pkg: do not run headers_check
fixup: kbuild: add support for ensuring headers are self-contained
kbuild: compile test UAPI headers to ensure they are self-contained

Series, applied to linux-kbuild.

Quoted 44 lines
 Documentation/kbuild/headers_install.txt |   7 --
 Documentation/kbuild/makefiles.txt       |  13 ++-
 Makefile                                 |  56 +++++-----
 arch/arc/configs/tb10x_defconfig         |   1 +
 arch/nds32/configs/defconfig             |   1 +
 arch/parisc/configs/a500_defconfig       |   1 +
 arch/parisc/configs/b180_defconfig       |   1 +
 arch/parisc/configs/c3000_defconfig      |   1 +
 arch/parisc/configs/default_defconfig    |   1 +
 arch/powerpc/configs/ppc6xx_defconfig    |   1 +
 arch/s390/configs/debug_defconfig        |   1 +
 include/uapi/{linux => }/Kbuild          |   6 +-
 init/Kconfig                             |  20 ++++
 lib/Kconfig.debug                        |  25 +++--
 samples/Kconfig                          |  14 ++-
 samples/Makefile                         |   4 +-
 scripts/Kbuild.include                   |   6 --
 scripts/Makefile                         |   5 -
 scripts/Makefile.build                   |   9 ++
 scripts/Makefile.headersinst             | 132 ++++++++++-------------
 scripts/Makefile.lib                     |   3 +
 scripts/cc-system-headers.sh             |   8 ++
 scripts/headers.sh                       |  29 -----
 scripts/headers_install.sh               |  48 ++++-----
 scripts/package/builddeb                 |   2 +-
 usr/.gitignore                           |   1 -
 usr/Makefile                             |   2 +
 usr/include/.gitignore                   |   3 +
 usr/include/Makefile                     | 132 +++++++++++++++++++++++
 29 files changed, 329 insertions(+), 204 deletions(-)
 rename include/uapi/{linux => }/Kbuild (77%)
 create mode 100755 scripts/cc-system-headers.sh
 delete mode 100755 scripts/headers.sh
 create mode 100644 usr/include/.gitignore
 create mode 100644 usr/include/Makefile

--
2.17.1

_______________________________________________
linux-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-riscv

Signature · 2 lines
Best Regards
Masahiro Yamada