kbuild: building with a mostly-clean /usr/src/linux and O=
PATCH 1/1 kbuild: building with a mostly-clean /usr/src/linux and O=
When compiling a kernel with a separate output directory,
/var/tmp/kernel-obj> cp $CONFIG_FILE .config /var/tmp/kernel-obj> make -C /usr/src/linux O=pwd
the main kernel Makefile first tries to make sure that /usr/src/linux is a clean source tree by checking if /usr/src/linux contains .config or the include/asm symlink. This test is more restrictive than necessary, and can be relaxed with a few additional changes so that /usr/src/linux may contain a few additional critical files, and may still be used as the kernel compilation source.
The additional files I have in mind are include/asm, include/linux/version.h, include/linux/autoconf.h, and perhaps also include/asm/offset.h. With these files present, it's possible to use kernel headers from user-space. I know that's evil and frowned upon, but it's still done once in a while, and sometimes it's the least painful path compared to the alternatives. The kbuild changes required to allow this IMHO are harmless; please consider for inclusion.
Thanks,
Andreas.
Makefile +9 -8
@@ -465,7 +465,7 @@ ifeq ($(KBUILD_EXTMOD),) scripts: scripts_basic include/config/MARKER $(Q)$(MAKE) $(build)=$(@) -scripts_basic: include/linux/autoconf.h +scripts_basic: $(objtree)/include/linux/autoconf.h # Objects we will link into vmlinux / subdirs we need to visit init-y := init/
@@ -491,11 +491,11 @@ include .config # If .config is newer than include/linux/autoconf.h, someone tinkered # with it and forgot to run make oldconfig -include/linux/autoconf.h: .config +$(objtree)/include/linux/autoconf.h: .config $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig else # Dummy target needed, because used as prerequisite -include/linux/autoconf.h: ; +$(objtree)/include/linux/autoconf.h: ; endif # The all: target is the default when no target is given on the
@@ -757,7 +757,7 @@ $(vmlinux-dirs): prepare-all scripts prepare2: ifneq ($(KBUILD_SRC),) @echo ' Using $(srctree) as source for kernel' - $(Q)if [ -h $(srctree)/include/asm -o -f $(srctree)/.config ]; then \ + $(Q)if [ -f $(srctree)/.config ]; then \ echo " $(srctree) is not clean, please run 'make mrproper'";\ echo " in the '$(srctree)' directory.";\ /bin/false; \
@@ -769,7 +769,7 @@ endif # prepare1 creates a makefile if using a separate output directory prepare1: prepare2 outputmakefile -prepare0: prepare1 include/linux/version.h include/asm include/config/MARKER +prepare0: prepare1 $(objtree)/include/linux/version.h $(objtree)/include/asm include/config/MARKER ifneq ($(KBUILD_MODULES),) $(Q)rm -rf $(MODVERDIR) $(Q)mkdir -p $(MODVERDIR)
@@ -808,14 +808,15 @@ export CPPFLAGS_vmlinux.lds += -P -C -U$ # hard to detect, but I suppose "make mrproper" is a good idea # before switching between archs anyway. -include/asm: +include/asm: $(objtree)/include/asm +$(objtree)/include/asm: @echo ' SYMLINK $@ -> include/asm-$(ARCH)' $(Q)if [ ! -d include ]; then mkdir -p include; fi; @ln -fsn asm-$(ARCH) $@ # Split autoconf.h into include/linux/config/* -include/config/MARKER: include/linux/autoconf.h +include/config/MARKER: $(objtree)/include/linux/autoconf.h @echo ' SPLIT include/linux/autoconf.h -> include/config/*' @scripts/basic/split-include include/linux/autoconf.h include/config @touch $@
@@ -839,7 +840,7 @@ define filechk_version.h ) endef -include/linux/version.h: $(srctree)/Makefile FORCE +$(objtree)/include/linux/version.h: $(srctree)/Makefile FORCE $(call filechk,version.h) # ---------------------------------------------------------------------------
scripts/Makefile.lib +1 -1
@@ -112,7 +112,7 @@ __cpp_flags = $(_cpp_flags) else # Prefix -I with $(srctree) if it is not an absolute path -addtree = $(if $(filter-out -I/%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1))) $(1) +addtree = $(1) $(if $(filter-out -I/%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1))) # Find all -I options and call addtree flags = $(foreach o,$($(1)),$(if $(filter -I%,$(o)),$(call addtree,$(o)),$(o)))
Quoted 12 lines · Andreas Gruenbacher
When compiling a kernel with a separate output directory,
/var/tmp/kernel-obj> cp $CONFIG_FILE .config/var/tmp/kernel-obj> make -C/usr/src/linuxO=pwdthe main kernel
Makefilefirst tries to make sure that/usr/src/linuxis a clean source tree by checking if/usr/src/linuxcontains .config or theinclude/asmsymlink. This test is more restrictive than necessary, and can be relaxed with a few additional changes so that/usr/src/linuxmay contain a few additional critical files, and may still be used as the kernel compilation source.
Quoted 7 lines
The additional files I have in mind are
include/asm,include/linux/version.h,include/linux/autoconf.h, and perhaps alsoinclude/asm/offset.h. With these files present, it's possible to use kernel headers from user-space. I know that's evil and frowned upon, but it's still done once in a while, and sometimes it's the least painful path compared to the alternatives. The kbuild changes required to allow this IMHO are harmless; please consider for inclusion.
I've already included below patch from you.
It was included in -linus last night.
Do we really need more?
Sam
diff-tree d80e22460968ec7986c82fd7d207ebe3de59e03d (from c5f75eca120de6587e67a1951ce3e6912e2c6879)
Author: Sam Ravnborg <sam@mars.(none)> Date: Thu Jul 14 20:22:39 2005 +0000
kbuild: Don't fail if include/asm symlink exists
From: Andreas Gruenbacher <[email protected]>
We're having the following situation: There are user-space applications that include kernel headers directly. With a completely unconfigured /usr/src/linux tree, including most headers fails because essential files are not there:
include/asm
include/linux/autoconf.h
include/linux/version.h
So we create these files. On the other hand, we want to use /usr/src/linux as read-only source for building kernels or additional modules. Now when building a kernel with a separate output directory (O=), there is a check in the main makefile for the include/asm symlink. There is no real need for this check: if we ensure that $(objdir)/include/asm is always created as the patch does, $(srctree)/include/asm becomes irrelevant.
Makefile +4 -3
@@ -765,11 +765,11 @@ $(vmlinux-dirs): prepare-all scripts # 2) Create the include2 directory, used for the second asm symlink prepare2: ifneq ($(KBUILD_SRC),) @echo ' Using $(srctree) as source for kernel' - $(Q)if [ -h $(srctree)/include/asm -o -f $(srctree)/.config ]; then \ + $(Q)if [ -f $(srctree)/.config ]; then \ echo " $(srctree) is not clean, please run 'make mrproper'";\ echo " in the '$(srctree)' directory.";\ /bin/false; \ fi; $(Q)if [ ! -d include2 ]; then mkdir -p include2; fi;
@@ -777,11 +777,12 @@ ifneq ($(KBUILD_SRC),) endif # prepare1 creates a makefile if using a separate output directory prepare1: prepare2 outputmakefile -prepare0: prepare1 include/linux/version.h include/asm include/config/MARKER +prepare0: prepare1 include/linux/version.h $(objtree)/include/asm \ + include/config/MARKER ifneq ($(KBUILD_MODULES),) $(Q)rm -rf $(MODVERDIR) $(Q)mkdir -p $(MODVERDIR) endif
@@ -816,11 +817,11 @@ export CPPFLAGS_vmlinux.lds += -P -C -U$ # FIXME: The asm symlink changes when $(ARCH) changes. That's # hard to detect, but I suppose "make mrproper" is a good idea # before switching between archs anyway. -include/asm: +$(objtree)/include/asm: @echo ' SYMLINK $@ -> include/asm-$(ARCH)' $(Q)if [ ! -d include ]; then mkdir -p include; fi; @ln -fsn asm-$(ARCH) $@ # Split autoconf.h into include/linux/config/*
Quoted 2 lines · Sam Ravnborg
I've already included below patch from you.
It was included in -linus last night.
That was close.
Quoted 1 line
Do we really need more?
So it seems I'm afraid: With the version of this patch that just went in, Jan Beulich <[email protected]> found a bug when building vmlinux.lds on i386. He triggered it by by putting poisoned version.h and autoconf.h files in /usr/src/linux.
With the additional changes (rediff against linux-2.6.13-git6 below), things work as expected. There may be a slightly more minimalistic solution that also works --- I didn't want to risk strange failures.
(To not confuse people, let me add that this all doesn't matter if the source tree is distclean.)
Cheers,
Andreas.
Additional fixes for building with O= against a non-distclean tree
Makefile +8 -7
@@ -462,7 +462,7 @@ ifeq ($(KBUILD_EXTMOD),) scripts: scripts_basic include/config/MARKER $(Q)$(MAKE) $(build)=$(@) -scripts_basic: include/linux/autoconf.h +scripts_basic: $(objtree)/include/linux/autoconf.h # Objects we will link into vmlinux / subdirs we need to visit init-y := init/
@@ -488,11 +488,11 @@ include .config # If .config is newer than include/linux/autoconf.h, someone tinkered # with it and forgot to run make oldconfig -include/linux/autoconf.h: .config +$(objtree)/include/linux/autoconf.h: .config $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig else # Dummy target needed, because used as prerequisite -include/linux/autoconf.h: ; +$(objtree)/include/linux/autoconf.h: ; endif # The all: target is the default when no target is given on the
@@ -798,7 +798,7 @@ endif # prepare1 creates a makefile if using a separate output directory prepare1: prepare2 outputmakefile -prepare0: prepare1 include/linux/version.h include/asm \ +prepare0: prepare1 $(objtree)/include/linux/version.h $(objtree)/include/asm \ include/config/MARKER ifneq ($(KBUILD_MODULES),) $(Q)rm -rf $(MODVERDIR)
@@ -838,14 +838,15 @@ export CPPFLAGS_vmlinux.lds += -P -C -U$ # hard to detect, but I suppose "make mrproper" is a good idea # before switching between archs anyway. -include/asm: +include/asm: $(objtree)/include/asm +$(objtree)/include/asm: @echo ' SYMLINK $@ -> include/asm-$(ARCH)' $(Q)if [ ! -d include ]; then mkdir -p include; fi; @ln -fsn asm-$(ARCH) $@ # Split autoconf.h into include/linux/config/* -include/config/MARKER: include/linux/autoconf.h +include/config/MARKER: $(objtree)/include/linux/autoconf.h @echo ' SPLIT include/linux/autoconf.h -> include/config/*' @scripts/basic/split-include include/linux/autoconf.h include/config @touch $@
@@ -869,7 +870,7 @@ define filechk_version.h ) endef -include/linux/version.h: $(srctree)/Makefile FORCE +$(objtree)/include/linux/version.h: $(srctree)/Makefile FORCE $(call filechk,version.h) # ---------------------------------------------------------------------------
scripts/Makefile.lib +1 -1
@@ -98,7 +98,7 @@ __cpp_flags = $(_cpp_flags) else # Prefix -I with $(srctree) if it is not an absolute path -addtree = $(if $(filter-out -I/%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1))) $(1) +addtree = $(1) $(if $(filter-out -I/%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1))) # Find all -I options and call addtree flags = $(foreach o,$($(1)),$(if $(filter -I%,$(o)),$(call addtree,$(o)),$(o)))
Unfortunately this isn't sufficient, yet. In the architecture-specific makefiles asm-offsets.s (or however the specific architectures call this) now need their dependencies on include/linux/version.h changed (I wonder whether it wouldn't be more efficient to centralize these dependencies into an architecture-independent file, perhaps at once applying consistent naming across all architectures), and arch/um/Makefile additionally has a dangling dependency on include/linux/autoconf.h now.
greping all Make* across the tree easily shows all locations needing additional changes.
Jan
Quoted 1 line
Quoted 1 line
Quoted 1 line
Andreas Gruenbacher <[email protected]> 07.09.05 03:03:44 >>>
Quoted 2 lines · Sam Ravnborg
I've already included below patch from you.
It was included in -linus last night.
That was close.
Quoted 1 line
Do we really need more?
So it seems I'm afraid: With the version of this patch that just went in, Jan Beulich <[email protected]> found a bug when building vmlinux.lds on i386. He triggered it by by putting poisoned version.h andautoconf.h files in /usr/src/linux.
With the additional changes (rediff against linux-2.6.13-git6 below), things work as expected. There may be a slightly more minimalistic solution that also works --- I didn't want to risk strange failures.
(To not confuse people, let me add that this all doesn't matter if the source tree is distclean.)
Cheers,
Andreas.
Additional fixes for building with O= against a non-distclean tree
Makefile +4 -4
@@ -462,7 +462,7 @@ ifeq ($(KBUILD_EXTMOD),) scripts: scripts_basic include/config/MARKER $(Q)$(MAKE) $(build)=$(@) -scripts_basic: include/linux/autoconf.h +scripts_basic: $(objtree)/include/linux/autoconf.h # Objects we will link into vmlinux / subdirs we need to visit init-y := init/
@@ -488,11 +488,11 @@ include .config # If .config is newer than include/linux/autoconf.h, someone tinkered # with it and forgot to run make oldconfig -include/linux/autoconf.h: .config +$(objtree)/include/linux/autoconf.h: .config $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig else # Dummy target needed, because used as prerequisite -include/linux/autoconf.h: ; +$(objtree)/include/linux/autoconf.h: ; endif # The all: target is the default when no target is given on the
@@ -798,7 +798,7 @@ endif # prepare1 creates a makefile if using a separate output directory prepare1: prepare2 outputmakefile -prepare0: prepare1 include/linux/version.h include/asm \ +prepare0: prepare1 $(objtree)/include/linux/version.h
$(objtree)/include/asm \
include/config/MARKER
ifneq ($(KBUILD_MODULES),)
$(Q)rm -rf $(MODVERDIR)
@@ -838,14 +838,15 @@ export CPPFLAGS_vmlinux.lds += -P -C -U$
# hard to detect, but I suppose "make mrproper" is a good idea
# before switching between archs anyway.
-include/asm:
+include/asm: $(objtree)/include/asm +$(objtree)/include/asm:
@echo ' SYMLINK $@ -> include/asm-$(ARCH)' $(Q)if [ ! -d include ]; then mkdir -p include; fi; @ln -fsn asm-$(ARCH) $@
# Split autoconf.h into include/linux/config/*
-include/config/MARKER: include/linux/autoconf.h +include/config/MARKER: $(objtree)/include/linux/autoconf.h
@echo ' SPLIT include/linux/autoconf.h -> include/config/*'
@scripts/basic/split-include include/linux/autoconf.hinclude/config
@touch $@
@@ -869,7 +870,7 @@ define filechk_version.h
)
endef
-include/linux/version.h: $(srctree)/Makefile FORCE +$(objtree)/include/linux/version.h: $(srctree)/Makefile FORCE
$(call filechk,version.h)
#
---------------------------------------------------------------------------
scripts/Makefile.lib +1 -3
@@ -98,7 +98,7 @@ __cpp_flags = $(_cpp_flags) else # Prefix -I with $(srctree) if it is not an absolute path -addtree = $(if $(filter-out -I/%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1))) $(1) +addtree = $(1) $(if $(filter-out -I/%,$(1)),$(patsubst -I%,-I$(srctree)/%,$(1))) # Find all -I options and call addtree
flags = $(foreach o,$($(1)),$(if $(filter -I%,$(o)),$(call
addtree,$(o)),$(o)))
Quoted 6 lines · Jan Beulich
Unfortunately this isn't sufficient, yet. In the architecture-specific makefiles asm-offsets.s (or however the specific architectures call this) now need their dependencies on
include/linux/version.hchanged (I wonder whether it wouldn't be more efficient to centralize these dependencies into an architecture-independent file, perhaps at once applying consistent naming across all architectures)
This topic has been on my mind to implment over a long time. So I took a stamp on it and it hit -linus this night.
Dependencies on arch/$(ARCH)/kernel/asm-offsets.s is now tracked like any other dependency - so we now at last have full dependency checks for asm-offsets.h.
Take a look at the Kbuild file in the top-level directory.
With respect to the patch posted below I recall I had to zap two $(objtree) to fix alpha build.
Patch by Jan Dittmer - see 946dc121d7d1c606f6bbeb8ae778963a1e2ff59c
But that may have changed now after introducing generic asm-offsets.h support.
Sam