kbuild: Improve Kernel build with separated output

kbuild: Improve Kernel build with separated output

Patch is improved based on input from lkml. The changelog has seen most changes better describing impact of the change.

To improve support for kernel build using separate output directories the following changes are implemented:
1) A Makefile is generated in the output directory allowing one to execute make in both the source and the output directory.
2) A new symlink 'source' is introduced. The symlink is set to point to the kernel source when installing modules.

When a kernel is built with source and output files mixed this patch will not introduce any change in behaviour.

When the kernel is built with separate output directory there were previously no way to locate the output files. Two options was considered:
a) Adding a new symlink pointing to the output files "object"
b) Let the build symlink point to the output files and introduce a new symlink "source" pointing to the kernel source

Option b) were selected based on the follwoing rationale. Option b) allowed ordinary external modules that did not need direct access to the kernel source to continue to build with no modifications.

In comparsion option a) required all external modules to use O= syntax (or set KBUILD_OUTPUT) in the case where the kernel was build using separate output directories. This situation was foreseen to result in a lot of reports like:
"Module compile with vendor-a but not vendor-b - so vendor-b need to fix their tree".
This sort of reports was expected for _all_ external modules, no matter how simple they were. Selecting option b) limit this to external modules that require direct access to kernel soruce.

So to re-iterate. Before this patch external modules could be built when using O= - and it was always required when the kernel used separate output directories.
Now external modules that needs direct access to source needs to follow the source symlink to find the kernel source - when the kernel uses separate output directory.

With this change the following command works independent of the kernel being build with separate output directory, or with output and source mixed.

make -C /lib/modules/uname -r/build M=pwd

[Substituting M=... with SUBDIRS=pwdmodules give same effect].

When the kernel is built using separate output directory the above invocation of make will invoke the generated Makefile located in the output directory - that again will invoke the Makefile located in the kernel source tree root.

Patch includes contributions from:
Andreas Gruenbacher <[email protected]> and Geert Uytterhoeven <[email protected]>

diff -Nru a/Makefile b/Makefile
@@ -608,14 +608,24 @@
 # A multi level approach is used. prepare1 is updated first, then prepare0.
 # prepare-all is the collection point for the prepare targets.
 
-.PHONY: prepare-all prepare prepare0 prepare1
+.PHONY: prepare-all prepare prepare0 prepare1 prepare2
+
+# prepare 2 generate Makefile to be placed in output directory, if
+# using a seperate output directory. This allows convinient use
+# of make in output directory
+prepare2:
+	$(Q)if [ ! $(srctree) -ef $(objtree) ]; then       \
+	$(CONFIG_SHELL) $(srctree)/scripts/mkmakefile      \
+	    $(srctree) $(objtree) $(VERSION) $(PATCHLEVEL) \
+	    > $(objtree)/Makefile;                         \
+	fi 
 
 # prepare1 is used to check if we are building in a separate output directory,
 # and if so do:
 # 1) Check that make has not been executed in the kernel src $(srctree)
 # 2) Create the include2 directory, used for the second asm symlink
 
-prepare1:
+prepare1: prepare2
 ifneq ($(KBUILD_SRC),)
 	@echo '  Using $(srctree) as source for kernel'
 	$(Q)if [ -h $(srctree)/include/asm -o -f $(srctree)/.config ]; then \

@@ -725,6 +735,12 @@
 modules_prepare: prepare-all scripts
 
 # Target to install modules
+# Modules are pr. default installed in /lib/modules/$(KERNELRELEASE)/...
+# Within this directory create two symlinks:
+# build => link to the directory containing the output files of the kernel build
+# source => link to the directory containing the source for the kernel
+# source and build are equal except for the case when the kernel is build using
+# a separate output directory
 .PHONY: modules_install
 modules_install: _modinst_ _modinst_post
 

@@ -736,9 +752,13 @@
 		sleep 1; \
 	fi
 	@rm -rf $(MODLIB)/kernel
-	@rm -f $(MODLIB)/build
+	@rm -f $(MODLIB)/source
 	@mkdir -p $(MODLIB)/kernel
-	@ln -s $(TOPDIR) $(MODLIB)/build
+	@ln -s $(srctree) $(MODLIB)/source
+	@if [ ! $(objtree) -ef  $(MODLIB)/build ]; then \
+		rm -f $(MODLIB)/build ; \
+		ln -s $(objtree) $(MODLIB)/build ; \
+	fi
 	$(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst
 
 # If System.map exists, run depmod.  This deliberately does not have a

diff -Nru a/scripts/mkmakefile b/scripts/mkmakefile
@@ -0,0 +1,31 @@
+#!/bin/sh
+# Generates a small Makefile used in the root of the output
+# directory, to allow make to be started from there.
+# The Makefile also allow for more convinient build of external modules
+
+# Usage
+# $1 - Kernel src directory
+# $2 - Output directory
+# $3 - version
+# $4 - patchlevel
+
+
+cat << EOF
+# Automatically generated by $0: don't edit
+
+VERSION = $3
+PATCHLEVEL = $4
+
+KERNELSRC    := $1
+KERNELOUTPUT := $2
+
+MAKEFLAGS += --no-print-directory
+
+all:
+	\$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT)
+
+%::
+	\$(MAKE) -C \$(KERNELSRC) O=\$(KERNELOUTPUT) \$@
+
+EOF
+

I would vote against doing that. Or at the very least don't overwrite one that might already be there. I, for one, have a very specific makefile in my build (object) directories. Anyone sufficiently skilled to be building kernels outside the source tree, and/or those with the specific need to be doing so will already have makefiles and/or shell scripts to suit their needs. Making the option a user specified target ala "make makefiles" would be a better/safer choice; if the user wants or needs a makefile in their object directory, then they have a simple option to make themselves one -- no knowledge of GNU Make necessary.

And in my case(s), the source and object symlinks will be of no value. The next "bk pull" and build invalidates those links. And once the kernel(s) are installed on their respective target machines, there won't be any source or object trees.

--Ricky

You cold just rename your Makefile to makefile. Then GNU Make will select that one as first choice.

Today kbuild unconditionally overwrite the Makefile, because it depends on the following:
- Top level Makefile (VERSION)
- scripts/mkmakefile script
- Source directory
- Output directory

In total too many factors to check for so the more brutal approach used.

Do you use your specific Makefile for something others could benefit from?

Sam