Separate obj/src dir

Based on some initial work by Kai Germaschewski I have made a working prototype of separate obj/src tree.

Usage example:
#src located in ~/bk/linux-2.5.sepobj
mkdir ~/compile/v2.5
cd ~/compile/v2.5
sh ../../kb/v2.5/kbuild

Prints out:
SRCTREE=/home/sam/bk/linux-2.5.sepobj OBJTREE=/home/sam/compile/v2.5
And then the normal make process starts after a short period.

The kbuild shell script takes a verbatim copy of all Makefiles, all Kconfig files and all defconfigs. I did not even look into using symlinks, I was not sure how they work across NFS and the like.

The VPATH feature of make is used to let make locate the src in the directory where the kbuild script is located.

I had to do some trikery with the CFLAGS to make it work. The processing done in the flags define in Makefile.build is rather unpleasent and I have another solution in mind I will give a try soon.
I will try to include files via a directory in the root of OBJTREE and then create symlinks towards the directories present in SRCTREE - but need to play a bit more with that.

Another drawback is that when a .h file exist in the SRCTREE but not in the OBJTREE the generated dependencies will point out the .h file located in SRCTREE. This happens for generated .h files, and therefore a simple check is made in kbuild to check that the SRCTREE is cleaned/mrpropered.

kconfig did not have an option to read the Kconfig files + defconfig from somewhere else than current directory, therefore they are copied. But that should be trivial to do. Possible solutions:
1) Command line option:
-r path-to-rrot-of-tree
2) Deduct it from the argument given, but then kconfig needs to know a bit too much about the kernel src tree.
3) Utilise the environment variable $(srctree), which is anyway valid.

Comments expected...

Sam

===== Makefile 1.346 vs edited =====

@@ -136,13 +136,18 @@
 export quiet Q KBUILD_VERBOSE
 
 #	Paths to obj / src tree
-
+ifneq ($(wildcard .tmp_make_config),)
+include .tmp_make_config
+src     := $(srctree)
+obj     := .
+VPATH   := $(srctree)
+else
 src	:= .
 obj	:= .
 srctree := .
 objtree := .
-
-export srctree objtree
+endif
+export srctree objtree VPATH
 
 # 	Make variables (CC, etc...)
 

@@ -304,7 +309,7 @@
 	set -e
 	$(if $(filter .tmp_kallsyms%,$^),,
 	  echo '  Generating build number'
-	  . scripts/mkversion > .tmp_version
+	  $(CONFIG_SHELL) $(src)/scripts/mkversion > .tmp_version
 	  mv -f .tmp_version .version
 	  $(Q)$(MAKE) -f scripts/Makefile.build obj=init
 	)

@@ -406,7 +411,11 @@
 
 include/asm:
 	@echo '  Making asm->asm-$(ARCH) symlink'
+ifeq ($(srctree),$(objtree))
 	@ln -s asm-$(ARCH) $@
+else
+	@ln -s $(src)/include/asm-$(ARCH) $@
+endif
 
 # 	Split autoconf.h into include/linux/config/*
 

@@ -585,7 +594,7 @@
 	tar -cvz $(RCS_TAR_IGNORE) -f $(KERNELPATH).tar.gz $(KERNELPATH)/. ; \
 	rm $(KERNELPATH) ; \
 	cd $(TOPDIR) ; \
-	. scripts/mkversion > .version ; \
+	$(CONFIG_SHELL) $(src)/scripts/mkversion > .version ; \
 	rpm -ta $(TOPDIR)/../$(KERNELPATH).tar.gz ; \
 	rm $(TOPDIR)/../$(KERNELPATH).tar.gz
 

===== scripts/Makefile.build 1.11 vs edited =====

@@ -84,26 +84,34 @@
 $(multi-objs-y:.o=.s)   : modname = $(modname-multi)
 $(multi-objs-y:.o=.lst) : modname = $(modname-multi)
 
+# Allow gcc to locate header files in srctree, if we use separate objtree
+ifeq ($(srctree),$(objtree))
+flags = $($(1))
+else
+flags = -I$(obj) -I$(srctree)/$(src) $($(1)) $(patsubst -I%,-I$(srctree)/%,$(filter -I%,$($(1))))
+endif
+
 quiet_cmd_cc_s_c = CC $(quiet_modtag)  $@
-cmd_cc_s_c       = $(CC) $(c_flags) -S -o $@ $< 
+cmd_cc_s_c       = $(CC) $(call flags,c_flags) -S -o $@ $< 
 
 %.s: %.c FORCE
 	$(call if_changed_dep,cc_s_c)
 
 quiet_cmd_cc_i_c = CPP $(quiet_modtag) $@
-cmd_cc_i_c       = $(CPP) $(c_flags)   -o $@ $<
+cmd_cc_i_c       = $(CPP) $(call flags,c_flags) -o $@ $<
 
 %.i: %.c FORCE
 	$(call if_changed_dep,cc_i_c)
 
 quiet_cmd_cc_o_c = CC $(quiet_modtag)  $@
-cmd_cc_o_c       = $(CC) $(c_flags) -c -o $@ $<
+cmd_cc_o_c       = $(CC) $(call flags,c_flags) -c -o $@ $<
 
 %.o: %.c FORCE
 	$(call if_changed_dep,cc_o_c)
 
 quiet_cmd_cc_lst_c = MKLST   $@
-cmd_cc_lst_c       = $(CC) $(c_flags) -g -c -o $*.o $< && sh scripts/makelst $*.o System.map $(OBJDUMP) > $@
+cmd_cc_lst_c       = $(CC) $(call flags,c_flags) -g -c -o $*.o $< && \
+		     sh $(src)/scripts/makelst $*.o System.map $(OBJDUMP) > $@
 
 %.lst: %.c FORCE
 	$(call if_changed_dep,cc_lst_c)

@@ -116,17 +124,14 @@
 $(real-objs-m)      : modkern_aflags := $(AFLAGS_MODULE)
 $(real-objs-m:.o=.s): modkern_aflags := $(AFLAGS_MODULE)
 
-a_flags = -Wp,-MD,$(depfile) $(AFLAGS) $(NOSTDINC_FLAGS) \
-	  $(modkern_aflags) $(EXTRA_AFLAGS) $(AFLAGS_$(*F).o)
-
 quiet_cmd_as_s_S = CPP $(quiet_modtag) $@
-cmd_as_s_S       = $(CPP) $(a_flags)   -o $@ $< 
+cmd_as_s_S       = $(CPP) $(call flags,a_flags) -o $@ $< 
 
 %.s: %.S FORCE
 	$(call if_changed_dep,as_s_S)
 
 quiet_cmd_as_o_S = AS $(quiet_modtag)  $@
-cmd_as_o_S       = $(CC) $(a_flags) -c -o $@ $<
+cmd_as_o_S       = $(CC) $(call flags,a_flags) -c -o $@ $<
 
 %.o: %.S FORCE
 	$(call if_changed_dep,as_o_S)

@@ -217,8 +222,7 @@
 # Create executable from a single .c file
 # host-csingle -> Executable
 quiet_cmd_host-csingle 	= HOSTCC  $@
-      cmd_host-csingle	= $(HOSTCC) -Wp,-MD,$(depfile) \
-			  $(HOSTCFLAGS) $(HOST_EXTRACFLAGS) \
+      cmd_host-csingle	= $(HOSTCC) $(call flags,hostc_flags) \
 			  $(HOST_LOADLIBES) -o $@ $<
 $(host-csingle): %: %.c FORCE
 	$(call if_changed_dep,host-csingle)

@@ -235,9 +239,7 @@
 # Create .o file from a single .c file
 # host-cobjs -> .o
 quiet_cmd_host-cobjs	= HOSTCC  $@
-      cmd_host-cobjs	= $(HOSTCC) -Wp,-MD,$(depfile) \
-			  $(HOSTCFLAGS) $(HOST_EXTRACFLAGS) \
-			  $(HOSTCFLAGS_$(@F)) -c -o $@ $<
+      cmd_host-cobjs	= $(HOSTCC) $(call flags,hostc_flags) -c -o $@ $<
 $(host-cobjs): %.o: %.c FORCE
 	$(call if_changed_dep,host-cobjs)
 

@@ -253,18 +255,14 @@
 
 # Create .o file from a single .cc (C++) file
 quiet_cmd_host-cxxobjs	= HOSTCXX $@
-      cmd_host-cxxobjs	= $(HOSTCXX) -Wp,-MD,$(depfile) \
-			  $(HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
-			  $(HOSTCXXFLAGS_$(@F)) -c -o $@ $<
+      cmd_host-cxxobjs	= $(HOSTCXX) $(call flags,hostcxx_flags) -c -o $@ $<
 $(host-cxxobjs): %.o: %.cc FORCE
 	$(call if_changed_dep,host-cxxobjs)
 
 # Compile .c file, create position independent .o file
 # host-cshobjs -> .o
 quiet_cmd_host-cshobjs	= HOSTCC  -fPIC $@
-      cmd_host-cshobjs	= $(HOSTCC) -Wp,-MD,$(depfile) -fPIC\
-			  $(HOSTCFLAGS) $(HOST_EXTRACFLAGS) \
-			  $(HOSTCFLAGS_$(@F)) -c -o $@ $<
+      cmd_host-cshobjs	= $(HOSTCC) $(call flags,hostc_flags) -fPIC -c -o $@ $<
 $(host-cshobjs): %.o: %.c FORCE
 	$(call if_changed_dep,host-cshobjs)
 

===== scripts/Makefile.lib 1.3 vs edited =====

@@ -128,6 +128,14 @@
 c_flags        = -Wp,-MD,$(depfile) $(CFLAGS) $(NOSTDINC_FLAGS) \
 	         $(modkern_cflags) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o) \
 	         $(basename_flags) $(modname_flags) $(export_flags) 
+a_flags        = -Wp,-MD,$(depfile) $(AFLAGS) $(NOSTDINC_FLAGS) \
+		 $(modkern_aflags) $(EXTRA_AFLAGS) $(AFLAGS_$(*F).o)
+hostc_flags    = -Wp,-MD,$(depfile) \
+		 $(HOSTCFLAGS) $(HOST_EXTRACFLAGS) \
+		 $(HOSTCFLAGS_$(*F).o)
+hostcxx_flags  = -Wp,-MD,$(depfile) \
+	 	 $(HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
+		 $(HOSTCXXFLAGS_$(*F).o)
 
 # Finds the multi-part object the current object will be linked into
 modname-multi = $(subst $(space),_,$(strip $(foreach m,$(multi-used),\

@@ -139,7 +147,7 @@
 quiet_cmd_shipped = SHIPPED $@
 cmd_shipped = cat $< > $@
 
-%:: %_shipped
+$(obj)/%:: $(src)/%_shipped
 	$(call cmd,shipped)
 
 # Commands useful for building a boot image

@@ -0,0 +1,28 @@
+#!/bin/sh
+OBJTREE=$PWD
+cd `dirname $0`
+SRCTREE=$PWD
+cd $OBJTREE
+echo OBJTREE $OBJTREE
+echo SRCTREE $SRCTREE
+if [ "$SRCTREE" != "$OBJTREE" ]; then
+  if [ -f $SRCTREE/.config -o -d $SRCTREE/include/asm ]; then
+    echo '$SRCTREE contains generated files, please run "make mrproper" in the SRCTREE'
+  else
+    for a in `cd $SRCTREE; find -type d`; do
+      mkdir -p $a;
+    done
+    for a in `cd $SRCTREE; find -name Makefile\* -o -name Kconfig\* -o -name defconfig`; do
+      cp -f $SRCTREE/$a $a
+    done
+
+    ( echo "srctree := $SRCTREE";
+      echo "objtree := $OBJTREE";
+    ) > .tmp_make_config
+    touch Rules.make
+  make $*
+  fi
+else
+  rm -f .tmp_make_config
+  make $*
+fi

[SNIPPED...]

I have a question; "What problem is this supposed to solve?" This looks like a M$ism to me. Real source trees don't look like this. If you don't have write access to the source- code tree, you are screwed on a real project anyway. That's why we have CVS, tar and other tools to provide a local copy.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Bush : The Fourth Reich of America

Two problems (at least):

1) You want to compile your kernel based on two different configurations, but sharing the same src. No need to have a duplicate of all src.
- There are other ways to do this using symlinks

2) You have the src located on a read-only filesystem. I have been told this is the case for some SCM systems.

People has requested this feature at several occasions, and here it is based on the current build system.
It's not ready for inclusion (obviously), and you shall also see this as a way to check that this is considered usefull by someone.

Sam

Hmmm. If your source is located on a read-only file-system, you can't modify it. You are therefore doomed to use only "known working" distributions that are known to work with all possible module combinations (these don't exist). So you get to compile the kernel just as a test exercise or a gimmick like; "what did you do today?..." answer; "I compiled the kernel..." This seems to not be very practical since the purpose of compiling the kernel was to add something or fix something. Now, its just to see if it compiles.

Different configurations are handled with different ".config" files.

I think all you did was increase the compile time by writing output files to different directories than the ones currently in cache. There are a lot of negatives. It would be a shame for you to waste a great deal of time on something that would not be accepted into the distribution. Remember the earlier make modules where the new objects went into a separate directory with sym-links? That got changed. I think it got changed for good reasons.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
Bush : The Fourth Reich of America

Well, this isn't necessarily for everybody, if it's of no use for you, that's okay. Note that the patch doesn't (unless it has bugs ;) change the normal behavior at all. It really only adds a script to set up a separate objdir. If you don't want that, don't use that script ;)

--Kai

And different .config files results in different kernels. Please note that .config files are also located in OBJTREE.

Cosider something like the following:

~/bk/linux-2.5
~/compile/arm <= Used to check that the kernel compiles for ARM
~/compile/allmod <= My config with a lot of modules ~/compile/machine <= The config I use on my machine
~/compile/work <= That's my sandbox

All configurations share the same src.
During the last copule of days we have seen some header file clean-ups. It would have been nice if they tried compiling in all four configurations listed above.
But if I switch .config then it is often a recompile of everything, whereas the above setup can minimize it.
The header file cleanup is maybe not the best example because touching a few key header files requires recompilation of everything anyway.

But my point is that there is a good use of different configurations based on the same src.

Others that have asked for separate obj dir may step in here, explaining why they thougt it was good.

Sam

Why? The only person arguing against it is RBJ, and people only ever read his mails for the amusement value.

Of course it's a useful feature.

--
dwmw2

Definitely. I already use home-grown scripts to maintain multiple object-trees, with the sources symlinked to a single source tree, because I generally maintain several different platforms simultaneously.

Having one source tree makes things _much_ easier, as I don't have to keep track of changes in a bunch of separate trees (it also saves a bit of space, which is nice)!

-Miles

Suburbia: where they tear out the trees and then name streets after them.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I think that your example for testing is the most valid one.

In development, you normally have different source trees (hardlinked if you don't have a terabyte of hard drive space to spare), and use an editor that breaks hard-links (eg, emacs). You might as well build in the source directory, since you'll likely keep reworking it.

In release testing (aka release engineering, or more accurately: release fumbling-in-the-dark), you need to test a few different configurations. Sure, you could just build a set of symlink or hardlink trees, but it'd be very useful to be able to "make multiconfigs" and have a representative set of kernels built (either for later testing, or at least to ensure that the new kernel will build without modules, without networking, without IDE, and so on). Sure, it will take a while if you build everything, but that is why God^WTridge gave us ccache.

Brad

- --
http://linux.conf.au. 22-25Jan2003. Perth, Aust. I'm registered. Are you? -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE92rNGW6pHgIdAuOMRAmcDAKCnzBOh6/6+zouOlM2bi1z2mcEmSACghEnp MzXw5OsNmbZfzBMVGrFmNr0=
=7DkQ
-----END PGP SIGNATURE-----

There are plenty of other good reasons for this. Offhand:

- Speeds up grepping through the source tree for things if you don't have to look at object files. SCCS dirs still make this a pain, I may hack grep -r to optionally skip them.

- Guarantees that doing a build should not affect anything in the srcdir; this is handy when preparing release tarballs etc.

- Multiple builds from the same source (different architectures, configs, compilers, whatever) without copying the source.

Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

Or when you store multiple versions of the source tree using cp -rl, patch, and same. I do not want to accidentally touch any file in hardlinked trees, since it would screw up the file in all trees.

Currently I use separate trees for compiling, which (A) consumes more disk space, and (B) is much slower to maintain (e.g. diff knows about hard links).

If I can compile in a separate directory, I can directly use one of the hardlinked trees as the source tree.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

You modify it by other means than open()/read()/write()/close()

Regards
Henning

"In einem Abw�gungsprozess, wollen wir weiter regieren, hat sich die
SPD und die Bundesregierung und auch der Bundesfinanzminister f�rs
Weiterregieren entschieden und gegen die Ehrlichkeit" -- Oswald
Metzger, B�ndnis '90/Die Gr�nen, 12.11.2002

-- 
Henning Schmiedehausen     "Interpol und Deutsche Bank, FBI und
[email protected]            Scotland Yard, Flensburg und das BKA
[email protected]    haben unsere Daten da." -- Kraftwerk, 1981

It can be really nice to maintain a bunch of different architectures at the same time from the same tree. It also makes it really easy to "clean" a tree.

On the other hand, I do wonder whether ccache could be used to get the same effect. Sam?

---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

I have never taken the time to look into ccache, so I dunno.

Sam

ccache helps a lot if you change your .config back to one previously compiled, but it still doesn't offer you the option to keep multiple .configs around at the same time.

Also, of course it doesn't help with keeping the source tree clean of non-source files ;)

Wrt the original patch, I like it, one preliminary comment is that I think symlinks are nicer than copying. They are faster, shouldn't cause any trouble on NFS, make uses "stat" and not "lstat", so it gets the timestamps right, too. And if you edit a Makefile/Kconfig in the source tree, you rather want that to take effect immediately, I guess ;)

--Kai

Second try on the script.
Create symlinks as suggested, and optimised find a liitle.

Sam

#!/bin/sh
#
# This script is used to build a kernel from a separate directory. # The location of this script is assumed to be the root of

# the kernel tree.
# Usage:
# kernel src located in:
# ~/kernelsrc
# compile in:
# ~/compile
# cd ~/compile   <= Change to the directory where the compile shall take place
# ../kernelsrc/kbuild
#
# Arguments to kbuild is the same as used to make in the kernel build
# kbuild prints out SRCTREE and OBJTREE when started, and then makes a mirror
# of relevant files from the kernelsrc.

# files we do not care about in the kernel src
RCS_FIND_IGNORE="-name SCCS -o -name BitKeeper -o -name .svn -o -name CVS"

OBJTREE=$PWD
cd dirname $0
SRCTREE=$PWD
cd $OBJTREE
echo OBJTREE $OBJTREE
echo SRCTREE $SRCTREE
if [ "$SRCTREE" != "$OBJTREE" ]; then
if [ -f $SRCTREE/.config -o -d $SRCTREE/include/asm ]; then
echo '$SRCTREE contains generated files, please run "make mrproper" in the SRCTREE'
else
for a in `cd $SRCTREE; \
find \( $RCS_FIND_IGNORE \) -prune -o -name Makefile\* -o -name Kconfig\* -o -name defconfig; do if [ ! -d dirname $a` ]; then
mkdir -p $a
fi
ln -fs $SRCTREE/$a $a
done

( echo "srctree := $SRCTREE";
echo "objtree := $OBJTREE";
) > .tmp_make_config
touch Rules.make
make $*
fi
else
rm -f .tmp_make_config
make $*
fi

Having Trees read-only checked out? Having Trees on "pseudo filesystems" backed by a SCM?

This is the same thing we do in Java Land for ages. I personally like it buy your taste may vary.

Regards
Henning

"In einem Abw�gungsprozess, wollen wir weiter regieren, hat sich die
SPD und die Bundesregierung und auch der Bundesfinanzminister f�rs
Weiterregieren entschieden und gegen die Ehrlichkeit" -- Oswald
Metzger, B�ndnis '90/Die Gr�nen, 12.11.2002

-- 
Henning Schmiedehausen     "Interpol und Deutsche Bank, FBI und
[email protected]            Scotland Yard, Flensburg und das BKA
[email protected]    haben unsere Daten da." -- Kraftwerk, 1981

Sam Ravnborg writes:

<snip>

I wonder how hard it would be to do this for other files types. It would be sort of handy to be able to copy a single file out of the source tree into the build tree, and have the build use the copy in the build tree. Example: you want to test a one liner in drivers/scsi/sd.c, you could just copy sd.c into the build tree, and make the change and test it out. That could be a huge space savings. That would help out those of us that are stuck with tiny hard drives in our laptops :)

Excellent, I never really thought about it until I read this email, but it makes perfect sense(to me anyways).

<snip>

It actually works. But that is a side-effect and not something I had planned.
I tried your example and kbuild uses the sd.c located in OBJTREE if present. make searches lokal directory before VPATH directory. And when sd.c is deleted the commandline changes and the original file is compiled again.

But I can see several drawbacks with this, especially dealing with .h files, or .c files included by other .c files.
So this is not a trustable feature.

Sam

For that you probably want to use the hardlinked trees approach. Just do a cp -al linux-2.5 scratch; then change your file and build from the copy.

Simon

PGP public key Id 0x144A991C, or http://himi.org/stuff/himi.asc
(crappy) Homepage: http://himi.org
doe #237 (see http://www.lemuria.org/DeCSS) 
My DeCSS mirror: ftp://himi.org/pub/mirrors/css/ 

One word of caution to vi users: you WILL forget to unlink one day. So, to protect me from myself, every time I untar a Linus' tree, I immediately do:
find linux-2.5.foo -type f | xargs chmod a-w
Then, the attached script reduces the typing. Run "deal dir/file.c" before editing as if it was "bk edit dir/file.c".
Once you are done I always do:
diff -urN -X dontdiff linux-2.5.48 linux-2.5.48-sparc > x.diff
This is a blazingly fast with hardlinked tree (cp -al).

-- Pete

#!/bin/sh
# This script dealiases a hardlinked file for editing.

set -e

if [ $# != 1 ]; then
echo "Usage: deal file" >&2
exit 1
fi

f=$1

mv $f $f~
cp $f~ $f
chmod u+w $f

But Kconfigs and defconfigs belong to the sources, don't they?

Suppose you have a main source tree and multiple objdirs with a purpose to test different .configs. Now if you update the main tree (including it's Kconfigs), objdirs are broken. The copies of Kconfigs are obsoleted and maybe unrelated at all. Also makefiles.

Nevermind, found your script with symlinks.