PATCH 2.4.4.3: 3rdparty driver support for kbuild

The attached patch, against kernel 2.4.4-pre3, adds a feature I call "3rd-party support."

When vendors put additional kernel modules and drivers in their kernel package, typically they do so via a patch. For the case where totally new file(s) are added to a kernel, this results in patch conflicts time and again, because all these patches are updating central files:
drivers/foo/Config.in and drivers/foo/Makefile.

The attached patch allows the addition of new kernel features without patching.

A vendor can simply have their kernel package build process untar a tarball inside the 3rdparty directory, or 'cp' a .c file into the 3rdparty directory. You simply run the "mkbuild.pl" script to make the kernel build and configuration system aware of the new kernel module, and then rebuild with your normal kernel build process. Additional details are included in Documentation/3rdparty.txt, in the patch.

Jeff Garzik       | "Give a man a fish, and he eats for a day. Teach a
Building 1024     |  man to fish, and a US Navy submarine will make sure
MandrakeSoft      |  he's never hungry again." -- Chris Neufeld

Index: linux_2_4/Makefile

diff -u linux_2_4/Makefile:1.1.1.85 linux_2_4/Makefile:1.1.1.85.2.1
@@ -124,7 +124,7 @@
 		 drivers/net/net.o \
 		 drivers/media/media.o
 LIBS		=$(TOPDIR)/lib/lib.a
-SUBDIRS		=kernel drivers mm fs net ipc lib
+SUBDIRS		=kernel drivers mm fs net ipc lib 3rdparty
 
 DRIVERS-n :=
 DRIVERS-y :=

@@ -180,6 +180,7 @@
 
 DRIVERS += $(DRIVERS-y)
 
+include 3rdparty/Makefile.drivers
 
 # files removed with 'make clean'
 CLEAN_FILES = \

@@ -219,7 +220,8 @@
 	.menuconfig.log \
 	include/asm \
 	.hdepend scripts/mkdep scripts/split-include scripts/docproc \
-	$(TOPDIR)/include/linux/modversions.h
+	$(TOPDIR)/include/linux/modversions.h \
+	3rdparty/Makefile.meta
 # directories removed with 'make mrproper'
 MRPROPER_DIRS = \
 	include/config \

@@ -407,7 +409,19 @@
 	rm -rf $(CLEAN_DIRS)
 	$(MAKE) -C Documentation/DocBook clean
 
-mrproper: clean archmrproper
+3rdpartyproper:
+	@echo "#" > 3rdparty/Config.in
+	@echo "# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT." >> 3rdparty/Config.in
+	@echo "#" >> 3rdparty/Config.in
+	@echo "#" > 3rdparty/Makefile
+	@echo "# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT." >> 3rdparty/Makefile
+	@echo "#" >> 3rdparty/Makefile
+	@echo 'include $$(TOPDIR)/Rules.make' >> 3rdparty/Makefile
+	@echo "#" > 3rdparty/Makefile.drivers
+	@echo "# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT." >> 3rdparty/Makefile.drivers
+	@echo "#" >> 3rdparty/Makefile.drivers
+
+mrproper: clean archmrproper 3rdpartyproper
 	find . \( -size 0 -o -name .depend \) -type f -print | xargs rm -f
 	rm -f $(MRPROPER_FILES)
 	rm -rf $(MRPROPER_DIRS)

Index: linux_2_4/3rdparty/Makefile.drivers

diff -u /dev/null linux_2_4/3rdparty/Makefile.drivers:1.1.4.1
@@ -0,0 +1,3 @@
+#
+# THIS IS AN AUTOMATICALLY GENERATED FILE. DO NOT EDIT.
+#

Index: linux_2_4/3rdparty/mkbuild.pl

Index: linux_2_4/Documentation/3rdparty.txt

Index: linux_2_4/arch/i386/config.in

diff -u linux_2_4/arch/i386/config.in:1.1.1.48 linux_2_4/arch/i386/config.in:1.1.1.48.2.1
@@ -377,3 +377,6 @@
 #bool 'Debug kmalloc/kfree' CONFIG_DEBUG_MALLOC
 bool 'Magic SysRq key' CONFIG_MAGIC_SYSRQ
 endmenu
+
+source 3rdparty/Config.in
+

Already covered by my 2.5 makefile rewrite[1] which has explicit support for third party kernel source. Shadow trees are designed specifically to handle this problem. I don't see the point of adding a script which will only be used in 2.4, especially when the vendors would have to change their tarball format for 2.5 formats.

(3) The kernel is constructed from multiple source trees and built in a separate object tree.

As for (2) you have a separate object directory for each set of config options. But instead of manually building a single source tree from various patches, the patch sets are kept separate and kbuild 2.5 logically constructs a single kernel source tree from the various source trees. This is called a shadow tree system because the patch sets shadow the main kernel source tree.

[1] http://sourceforge.net/projects/kbuild

I don't see how multiple source trees can be merged automatically with 100% accuracy.

What happens when you build with linus-tree, xfs-tree, and reiserfs-tree, and both XFS and reiserfs touch struct file_operations, in conflicting ways? It takes human intervention to figure that stuff out and resolve such conflicts.

If you support multiple source trees -> single build tree, it sounds like are you trying to integrate cvs/diff/similar into the kernel build system, which is whacky...

Jeff

Jeff Garzik      | The difference between America and England is that
Building 1024    | the English think 100 miles is a long distance and
MandrakeSoft     | the Americans think 100 years is a long time.
                 |      (random fortune)

I agree, multiple source trees only work 100% for non-overlapping code. It does not matter how you implement separate source, the moment it overlaps you need human intervention. That is true for my makefile-2.5 as well as your Perl method.

Where "non-overlapping" needs to be construed broadly to include "not logically conflicting", and not merely as overlapping diffs.

/Jonathan Lundell.