Skip to contentSkip to conversation

External kernel modules, second try

PATCH Andreas Gruenbacher 7 participants 18 messages Mar 7, 2004 Kbuild

lore
Share & tools
lore
#0 lore raw

Hello,

here is the patch I posted previously that adds support for modversions in external kernel modules that are built outside the main kernel tree (first posting archived here: http://lwn.net/Articles/69148/). Relative to the original version, the attached version also works when compiling with O=.

The patch also adds a modules_add target that does the equivalent of modules_install for one external module. In addition, it makes clean and mrproper work for external modules. Makefiles of modules that live outside the main tree currently need to provide their own clean targets to remove the files kbuild creates; this is pretty messy.

The third change is to remove one instance of temporary file creation inside the main kernel tree while external modules are built. I think there are still other cases where temp files in the kernel tree are used. IMHO they should all go away, so that a ``make -C $KERNEL_SOURCE modules SUBDIRS=$PWD'' works against a read-only tree.

Thanks,

Signature · 2 lines
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
Makefile +35 -13
Index: linux-2.6.1/Makefile
===================================================================
@@ -289,7 +289,10 @@ export CPPFLAGS NOSTDINC_FLAGS OBJCOPYFL
 export CFLAGS CFLAGS_KERNEL CFLAGS_MODULE 
 export AFLAGS AFLAGS_KERNEL AFLAGS_MODULE
 
-export MODVERDIR := .tmp_versions
+# When compiling out-of-tree modules, put MODVERDIR in the module source
+# tree rather than in the kernel source tree. The kernel source tree might
+# even be read-only.
+export MODVERDIR := $(patsubst %,%/,$(firstword $(filter ../% /%,$(SUBDIRS)))).tmp_versions
 
 # The temporary file to save gcc -MD generated dependencies must not
 # contain a comma
@@ -749,6 +752,9 @@ endif
 _modinst_post: _modinst_
 	if [ -r System.map ]; then $(DEPMOD) -ae -F System.map $(depmod_opts) $(KERNELRELEASE); fi
 
+modules_add:
+	$(Q)$(MAKE) -rR -f $(srctree)/scripts/Makefile.modinst
+
 else # CONFIG_MODULES
 
 # Modules not configured
@@ -820,11 +826,28 @@ clean-dirs += $(addprefix _clean_,$(ALL_
 $(clean-dirs):
 	$(Q)$(MAKE) $(clean)=$(patsubst _clean_%,%,$@)
 
+.PHONY: _clean _mrproper
+ifeq ($(filter ../% /%,$(SUBDIRS)),)
+
+clean_subdirs := .
+clean: _clean archclean $(clean-dirs)
+	$(call cmd,rmclean)
+mrproper distclean: clean _mrproper archmrproper
+	$(call cmd,mrproper)
+
+else
+
+clean_subdirs := $(filter ../% /%,$(SUBDIRS))
+clean: _clean
+mrproper distclean: _mrproper
+	$(Q)rm -rf $(MODVERDIR)
+
+endif
+
 quiet_cmd_rmclean = RM  $$(CLEAN_FILES)
 cmd_rmclean	  = rm -f $(CLEAN_FILES)
-clean: archclean $(clean-dirs)
-	$(call cmd,rmclean)
-	@find . $(RCS_FIND_IGNORE) \
+_clean:
+	@find $(clean_subdirs) $(RCS_FIND_IGNORE) \
 	 	\( -name '*.[oas]' -o -name '*.ko' -o -name '.*.cmd' \
 		-o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \) \
 		-type f -print | xargs rm -f
@@ -833,15 +856,13 @@ clean: archclean $(clean-dirs)
 #
 quiet_cmd_mrproper = RM  $$(MRPROPER_DIRS) + $$(MRPROPER_FILES)
 cmd_mrproper = rm -rf $(MRPROPER_DIRS) && rm -f $(MRPROPER_FILES)
-mrproper distclean: clean archmrproper
-	@echo '  Making $@ in the srctree'
-	@find . $(RCS_FIND_IGNORE) \
+_mrproper:
+	@find $(clean_subdirs) $(RCS_FIND_IGNORE) \
 	 	\( -name '*.orig' -o -name '*.rej' -o -name '*~' \
 		-o -name '*.bak' -o -name '#*#' -o -name '.*.orig' \
 	 	-o -name '.*.rej' -o -size 0 \
 		-o -name '*%' -o -name '.*.cmd' -o -name 'core' \) \
 		-type f -print | xargs rm -f
-	$(call cmd,mrproper)
 
 # Generate tags for editors
 # ---------------------------------------------------------------------------
@@ -1045,13 +1066,14 @@ cmd = @$(if $($(quiet)cmd_$(1)),echo '  
 define filechk
 	@set -e;				\
 	echo '  CHK     $@';			\
-	mkdir -p $(dir $@);			\
-	$(filechk_$(1)) < $< > [email protected];		\
-	if [ -r $@ ] && cmp -s $@ [email protected]; then	\
-		rm -f [email protected];			\
+	tmp=$$(/bin/mktemp /tmp/kbuild.XXXXXX);	\
+	$(filechk_$(1)) < $< > $$tmp;		\
+	if [ -r $@ ] && cmp -s $@ $$tmp; then	\
+		rm -f $$tmp;			\
 	else					\
 		echo '  UPD     $@';		\
-		mv -f [email protected] $@;		\
+		mkdir -p $(dir $@);		\
+		mv -f $$tmp $@;			\
 	fi
 endef
 
scripts/modpost.c +109 -9

168 lines not on this page · raw message

scripts/modpost.h +1 -0
Index: linux-2.6.1/scripts/modpost.h
===================================================================
@@ -70,6 +70,7 @@ struct module {
 	const char *name;
 	struct symbol *unres;
 	int seen;
+	int skip;
 	struct buffer dev_table_buf;
 };
 
scripts/Makefile.modinst +6 -4
Index: linux-2.6.1/scripts/Makefile.modinst
===================================================================
@@ -29,11 +29,13 @@ quiet_cmd_modules_install = INSTALL $@
 $(filter-out ../% /%,$(modules)):
 	$(call cmd,modules_install)
 
-# Modules built outside just go into extra
+# Modules built outside go into extra by default
 
-quiet_cmd_modules_install_extra = INSTALL $(obj-m:.o=.ko)
-      cmd_modules_install_extra = mkdir -p $(MODLIB)/extra; \
-			    	  cp $@ $(MODLIB)/extra
+MOD_DIR ?= extra
+
+quiet_cmd_modules_install_extra = INSTALL $(@F)
+      cmd_modules_install_extra = mkdir -p $(MODLIB)/$(MOD_DIR); \
+			    	  cp $@ $(MODLIB)/$(MOD_DIR)
 
 $(filter     ../% /%,$(modules)):
 	$(call cmd,modules_install_extra)
scripts/Makefile.modpost +12 -2
Index: linux-2.6.1/scripts/Makefile.modpost
===================================================================
@@ -51,11 +51,21 @@ $(modules:.ko=.mod.c): __modpost ;
 
 # Extract all checksums for all exported symbols
 
-quiet_cmd_modpost = MODPOST
-      cmd_modpost = scripts/modpost $(filter-out FORCE,$^)
+# Are we building out-of-tree modules?
 
+ifeq ($(filter ../% /%,$(SUBDIRS)),)
+modver_file := modversions
+quiet_cmd_modpost = MODPOST
+      cmd_modpost = scripts/modpost $(filter-out FORCE,$^) -o $(modver_file)
 __modpost: $(wildcard vmlinux) $(modules:.ko=.o) FORCE
 	$(call if_changed,modpost)
+else
+modver_file := $(wildcard $(srctree)/modversions)
+quiet_cmd_modpost = MODPOST
+      cmd_modpost = scripts/modpost $(if $(modver_file),-i $(modver_file)) $^
+__modpost: $(modules:.ko=.o)
+	$(call cmd,modpost)
+endif
 
 targets += __modpost
 
#1 lore raw
Quoted 7 lines · Andreas Gruenbacher

Hello,

here is the patch I posted previously that adds support for modversions in external kernel modules that are built outside the main kernel tree (first posting archived here: http://lwn.net/Articles/69148/). Relative to the original version, the attached version also works when compiling with O=.

Hi Andreas.
I have started to look into this.
The changes im Makefile when you use SUBDIRS as a flag does not look pretty.

What I have in mind is something like this:
- Get rid of current use of SUBDIRS. It is no longer used in any arch Makefiles.
- Introduce a KBUILD_EXTMOD variable that is only set when building external modules
- Introduce a new method to be used when compiling external modules:
make M=dir-to-module
- Keeping the SUbDIRS notation for backward compatibility
- When using SUBDIRS or M= the 'modules' target is implicit.
- make clean and make mrproper/distclen only deletes files in the external module directory (as done in your patch)
- Error out if any updadtes are requires in the kernel tree
- Find a magic way to include a Kconfig file for the external module
- Allow kbuild Makefiles to be named Kbuild, so local stuff can be in a file named Makefile file
note: this can be achieved using makefile/Makefile today but it makes sense since there is not much 'Make' syntax left in kbuild makefiles today.

Above will not be made in one go. My next step is to make a patch for the first four steps - to see the actual impact on current makefiles.

Comments welcome!

Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes)

Quoted 2 lines

The patch also adds a modules_add target that does the equivalent of modules_install for one external module.

Looks good.

Quoted 5 lines

The third change is to remove one instance of temporary file creation inside the main kernel tree while external modules are built. I think there are still other cases where temp files in the kernel tree are used. IMHO they should all go away, so that a ``make -C $KERNEL_SOURCE modules SUBDIRS=$PWD'' works against a read-only tree.

Agree - should be easy to test using a CD.
Are there an easy way to mount just a directory structure RO?

Sam

#2 lore raw
Quoted 4 lines

Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes)

distributions don't like to install the vmlinux since it's big(ish) and means customers need to download a new vmlinux at each kernel erratum. The same information is btw also present in System.map so imo the real solution is to make modpost use System.map instead ;)

Quoted 3 lines
Quoted 1 line

modules SUBDIRS=$PWD'' works against a read-only tree.

Agree - should be easy to test using a CD.
Are there an easy way to mount just a directory structure RO?

not without making it a separate partition or loopback device.

#3 lore raw

Hello Arjan,

Quoted 8 lines · Arjan van de Ven
Quoted 3 lines

Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes)

distributions don't like to install the vmlinux since it's big(ish) and means customers need to download a new vmlinux at each kernel erratum. The same information is btw also present in System.map so imo the real solution is to make modpost use System.map instead ;)

System.map doesn't have the hashes, and it's missing the symbols from module files. External modules may require symbols that live in another module. There are also a number of other minor differences that could probably be worked around.

Now it would be possible to extract the modver symbols from the installed vmlinux and .ko files when needed, but note that we may be building modules for kernels that are not currently running, and for which those binaries are not even installed. So this sounds like a bad idea.

Cheers,

Signature · 2 lines
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
#4 lore raw
Quoted 7 lines · Andreas Gruenbacher

Hello Arjan,

Quoted 4 lines · Arjan van de Ven
Quoted 3 lines

Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes)

Quoted 6 lines
Quoted 4 lines

distributions don't like to install the vmlinux since it's big(ish) and means customers need to download a new vmlinux at each kernel erratum. The same information is btw also present in System.map so imo the real solution is to make modpost use System.map instead ;)

System.map doesn't have the hashes,

are you sure ? I could have sworn it had.

Quoted 2 lines

and it's missing the symbols from
module files.

sure but the module files are generally installed...

Quoted 5 lines

Now it would be possible to extract the modver symbols from the installed vmlinux and .ko files when needed, but note that we may be building modules for kernels that are not currently running, and for which those binaries are not even installed. So this sounds like a bad idea.

I don't personally care about those; you need SOME stuff to build against obviously, and vmlinux is well over the top I agree that. But assuming the .ko's for the modules are there...you need those to use the kernel anyway.

#5 lore raw
Quoted 5 lines · Arjan van de Ven
Quoted 2 lines · Andreas Gruenbacher

and it's missing the symbols from
module files.

sure but the module files are generally installed...

Not when building for five different configurations on one machine. (And this is not a theoretical case.)

Quoted 10 lines
Quoted 5 lines

Now it would be possible to extract the modver symbols from the installed vmlinux and .ko files when needed, but note that we may be building modules for kernels that are not currently running, and for which those binaries are not even installed. So this sounds like a bad idea.

I don't personally care about those; you need SOME stuff to build against obviously, and vmlinux is well over the top I agree that. But assuming the .ko's for the modules are there...you need those to use the kernel anyway.

The .ko's are really not there. And even if they were, I don't think we want to teach modpost to poke around in /lib/modules when it can be avoided easily.

Cheers,

Signature · 2 lines
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
#6 lore raw

GPL 3

hi,

in all the license discussions that have gone on here, i haven't seen one on the supposed gpl 3 happen in a while, so i was wondering, is this really going to happen? and does anyone know of the date of it's proposed launch? and where is a copy of this so i can look at it in the meantime?

thanks

mark.

#7 lore raw

Re: GPL 3

Quoted 6 lines · mark kandianis

hi,

in all the license discussions that have gone on here, i haven't seen one on the supposed gpl 3 happen in a while, so i was wondering, is this really going to happen? and does anyone know of the date of it's proposed launch? and where is a copy of this so i can look at it in the meantime?

This is NOT the proper place to discuss GPL3. Please let's not have another flame war over licenses here.

Signature · 2 lines
Måns Rullgård
[email protected]
#8 lore raw

Re: GPL 3

Quote from [email protected]:

Quoted 6 lines
Quoted 3 lines · mark kandianis

hi,

in all the license discussions that have gone on here, i haven't seen=

one

Quoted 8 lines
Quoted 1 line

on the supposed gpl 3 happen in a while, so i was wondering, is this =

really

Quoted 1 line

going to happen? and does anyone know of the date of it's proposed l=

aunch?

Quoted 1 line

and where is a copy of this so i can look at it in the meantime?

This is NOT the proper place to discuss GPL3. Please let's not have another flame war over licenses here.

Why not, it's the weekend, and the troll-o-meter hasn't seen much action for a while since the last flamewar about BitKeeper.

:-)

John.

#9 lore raw
In reply to Arjan van de Ven
Quoted 4 lines · Arjan van de Ven
Quoted 2 lines

and it's missing the symbols from
module files.

sure but the module files are generally installed...

I'm aiming for a situation where I can build external modules, using an almost clean kernel src tree.

Which means that my "make clean on steroids" patch maybe is a bit too effective.
The distintion point could be:

make clean leaves only enough to build external modules. Otherwise we need a third (fourth) cleaning target, which is not desireable.

Sam

#10 lore raw
Quoted 7 lines · Sam Ravnborg
Quoted 4 lines · Arjan van de Ven
Quoted 2 lines

and it's missing the symbols from
module files.

sure but the module files are generally installed...

I'm aiming for a situation where I can build external modules, using an almost clean kernel src tree.

Personally I don't see the point. I'm perfectly happy with the current situation with the exception of not using System.map and co.

From a distribution kernel pov; I already ship a subset of files for building modules against (basically include/, the KConfig and makefiles), which only not 100% works because I don't ship vmlinux.

#11 lore raw
Quoted 9 lines · Arjan van de Ven
Quoted 7 lines · Sam Ravnborg
Quoted 4 lines · Arjan van de Ven
Quoted 2 lines

and it's missing the symbols from
module files.

sure but the module files are generally installed...

I'm aiming for a situation where I can build external modules, using an almost clean kernel src tree.

Quoted 6 lines

Personally I don't see the point. I'm perfectly happy with the current situation with the exception of not using System.map and co.

From a distribution kernel pov; I already ship a subset of files for building modules against (basically include/, the KConfig and makefiles), which only not 100% works because I don't ship vmlinux.

We have tried that with our latest round of kernels (still 2.4), and the results have been mixed. You need various headers outside include/ for some obscure external modules. Amazingly there are even external modules that make use of kernel C files.

All in all, in the end I changed my mind. I now think that it's better to build modules against a clean kernel source tree that additionally has the modversions file copied in. This already works when using O=. With the SUBDIRS= approach, the kernel source tree must include a few compiled files (scripts/ stuff), and it cannot be read-only.

I'm still undecided whether it makes sense to disallow the SUBDIRS= approach completely and only allow building with O=. (Note that this doesn't change the modversion dump file argument.) When building with SUBDIRS=, you ideally want a (read-only) kernel source tree that can adapt to different configurations (e.g., by doing like this:

make -C $KERNEL_SOURCE modules SUBDIRS=$PWD FLAVOR=bigsmp

), the default being the running kernel. The RedHat kernel has had a partial solution for merging autoconf.h. I have patches that implement a more complete solution that I'd happily send them to an appropriate place for discussion, but I don't think this would make much sense in mainline. Particularly because, while O= building has a slightly higher overhead, it gets rid of those problems, anyway.

Cheers,

Signature · 2 lines
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
#12 lore raw
Quoted 6 lines · Andreas Gruenbacher
Quoted 3 lines

From a distribution kernel pov; I already ship a subset of files for building modules against (basically include/, the KConfig and makefiles), which only not 100% works because I don't ship vmlinux.

We have tried that with our latest round of kernels (still 2.4), and the results have been mixed. You need various headers outside include/ for

2.6 has the biggest offender (scsi) fixed.

Quoted 2 lines

some obscure external modules. Amazingly there are even external modules that make use of kernel C files.

I can't imagine that being the case anymore, and for sure it won't be binary only ones ;)

Quoted 2 lines

), the default being the running kernel. The Red Hat kernel has had a partial solution for merging autoconf.h.

It's a gross hack that we thankfully got rid of finally!

#13 lore raw
In reply to Andreas Gruenbacher
Quoted 6 lines · Andreas Gruenbacher

All in all, in the end I changed my mind. I now think that it's better to build modules against a clean kernel source tree that additionally has the modversions file copied in. This already works when using O=. With the SUBDIRS= approach, the kernel source tree must include a few compiled files (scripts/ stuff), and it cannot be read-only.

Quoted 7 lines

I'm still undecided whether it makes sense to disallow the SUBDIRS= approach completely and only allow building with O=. (Note that this doesn't change the modversion dump file argument.) When building with SUBDIRS=, you ideally want a (read-only) kernel source tree that can adapt to different configurations (e.g., by doing like this:

make -C $KERNEL_SOURCE modules SUBDIRS=$PWD FLAVOR=bigsmp

This is already possible.
You can do:
make -C $KERNEL_SRC SUBDIRS=$PWD O=output-dir modules

or with my proposed syntax:
make -C $KERNEL_SRC M=$PWD O=output-dir

The files relevant for the module will be located in the $PWD dir, since they use absolute paths.

Quoted 2 lines

), the default being the running kernel.

I do not want to have potentially distro specific solutions. So it depends if we can find a solution that most will agree on.

Sam

#14 lore raw
In reply to Sam Ravnborg
Quoted 8 lines · Sam Ravnborg
Quoted 7 lines · Andreas Gruenbacher

Hello,

here is the patch I posted previously that adds support for modversions in external kernel modules that are built outside the main kernel tree (first posting archived here: http://lwn.net/Articles/69148/). Relative to the original version, the attached version also works when compiling with O=.

Quoted 4 lines

Hi Andreas.
I have started to look into this.
The changes im Makefile when you use SUBDIRS as a flag does not look pretty.

Agreed. A more generic approach wouldn't hurt. You know now which itches we have with this area of kbuild; I'm convinced we will work something out.

Quoted 9 lines

What I have in mind is something like this:
- Get rid of current use of SUBDIRS. It is no longer used in any arch Makefiles.
- Introduce a KBUILD_EXTMOD variable that is only set when building external modules
- Introduce a new method to be used when compiling external modules:
make M=dir-to-module
- Keeping the SUbDIRS notation for backward compatibility
- When using SUBDIRS or M= the 'modules' target is implicit.

Why not keep the SUBDIRS notation for external modules only then? That's what was documented to work since a long time; I see no big benefit in changing it if it can be avoided.

Quoted 2 lines

- make clean and make mrproper/distclen only deletes files in the external module directory (as done in your patch)

Yes.

Quoted 1 line

- Error out if any updates are requires in the kernel tree

Yes.

Quoted 1 line

- Find a magic way to include a Kconfig file for the external module

This is where it gets pretty messy. You would also have a different configuration in the external module. I have no clear idea how that can work reasonably cleanly.

Quoted 6 lines

- Allow kbuild Makefiles to be named Kbuild, so local stuff can be in a file named Makefile file

note: this can be achieved using makefile/Makefile today but it makes sense since there is not much 'Make' syntax left in kbuild makefiles today.

The Makefile can already include both the kbuild and local stuff (same snippet I sent you in personal communication already):

------------------------- 8< ------------------------- # Standard kbuild makefile constructs go here:

   # mod-y := ...

# Set to something different to install somewhere else:
# MOD_DIR := extra

.PHONY: modules install clean modules_add

   modules modules_add clean:
           $(MAKE) -C $(KERNEL_SOURCE) $@ SUBDIRS=$(CURDIR)
   install : modules_add
   ------------------------- 8< -------------------------

OTOH, if by local stuff you mean userspace, that *really* ought to go in a different directory. Module writers for some reason don't like this position, but we are building the modules for a whole bunch of kernels (with different configurations), and userspace is only built once per architecture. Merging the kernel and user-space parts is just wrong (TM).

Quoted 2 lines

Above will not be made in one go. My next step is to make a patch for the first four steps - to see the actual impact on current makefiles.

Yes, fair enough.

Quoted 4 lines

Comments welcome!

Could you explain what is the actually gain of using the modversions file your patch creates. (modpost changes)

Now with mainline, when building external modules they will end up not having modversions. This is caused by the way .tmp_versions is handled, and is a real problem. There are two different ways how we are building external modules today:

(1) after the kernel source tree was just compiled, so the kernel source tree still contains all the object files,

(2) in a separate step, against an almost clean kernel source tree. Almost-clean means the tree contains a set of configuration files, and the modversions dump file.

The modversions dump file elegantly solves both cases.

Quoted 11 lines
Quoted 2 lines

The patch also adds a modules_add target that does the equivalent of modules_install for one external module.

Looks good.

Quoted 5 lines

The third change is to remove one instance of temporary file creation inside the main kernel tree while external modules are built. I think there are still other cases where temp files in the kernel tree are used. IMHO they should all go away, so that a ``make -C $KERNEL_SOURCE modules SUBDIRS=$PWD'' works against a read-only tree.

Agree - should be easy to test using a CD.
Are there an easy way to mount just a directory structure RO?

Not sure what you mean exactly. My main motivation is this: we have a whole bunch of external modules that we build one after the other. Those external modules are notoriously nasty: We had cases where the modules fondled in kernel headers in the original tree. Of course then the next modules would build against a broken tree. To stop and detect such madness, I want to give modules a kernel source tree to which they have no write access, by chowning to a different user. Trees on read-only media are irrelevant IMHO.

Cheers,

Signature · 2 lines
Andreas Gruenbacher <[email protected]>
SUSE Labs, SUSE LINUX AG
#15 lore raw
Quoted 5 lines · Andreas Gruenbacher

Now with mainline, when building external modules they will end up not having modversions. This is caused by the way .tmp_versions is handled, and is a real problem. There are two different ways how we are building external modules today:

Quoted 8 lines

(1) after the kernel source tree was just compiled, so the kernel source tree still contains all the object files,

(2) in a separate step, against an almost clean kernel source tree. Almost-clean means the tree contains a set of configuration files, and the modversions dump file.

The modversions dump file elegantly solves both cases.

However, one little problem still remains: What if external modules from one package need to use symbols from modules which are also external, but in a different package? We have a similar situation in 2.4.x with lirc modules, which reference symbols from bttv (and bttv is built in a separate package together with some other out-of-tree v4l modules, because they need a common tuner module).

Suggestions:

- Create a subdirectory for modversions dump files (the dump file for the kernel will go into modversions/kernel).

- Allow multiple "-i DUMPFILE" options in modpost. Wildcard handling will be in the Makefiles:

	modver_files := $(wildcard $(srctree)/modversions/*)
	modpost_input_flags := $(addprefix -i ,$(modver_files))

- Make "-o NEWDUMPFILE" work properly even after "-i DUMPFILE" (mark all symbols which were read from a dump file and do not output such symbols when writing the new dump file).

- Add a way to specify the output dump filename for modpost when building external modules.

#16 lore raw
Quoted 4 lines · Sergey Vlasov

However, one little problem still remains: What if external modules from one package need to use symbols from modules which are also external, but in a different package?

We want to concentrate on getting the simple case right, then we can add more functionality later.

Sam

#17 lore raw
In reply to Andreas Gruenbacher
Quoted 11 lines · Andreas Gruenbacher
Quoted 10 lines

What I have in mind is something like this:
- Get rid of current use of SUBDIRS. It is no longer used in any arch Makefiles.
- Introduce a KBUILD_EXTMOD variable that is only set when building external modules
- Introduce a new method to be used when compiling external modules:
make M=dir-to-module
- Keeping the SUbDIRS notation for backward compatibility
- When using SUBDIRS or M= the 'modules' target is implicit.

Quoted 3 lines

Why not keep the SUBDIRS notation for external modules only then? That's what was documented to work since a long time; I see no big benefit in changing it if it can be avoided.

Since a long time is from 2.5.5x roughly.
The SUBDIRS= notation is not intuitive, and the M= notation follow the other similar options: O=, C=.

SUBDIRS will be kept, but warned for in 2.7. Removed when 2.8 opens or similar. So I see no breakage here.

Quoted 5 lines
Quoted 1 line

- Find a magic way to include a Kconfig file for the external module

This is where it gets pretty messy. You would also have a different configuration in the external module. I have no clear idea how that can work reasonably cleanly.

If the solution becomes messy then I will just drop it. I do not see a big need here anyway.
It will also create troubles: Can we modify .config etc.

Quoted 9 lines
Quoted 6 lines

- Allow kbuild Makefiles to be named Kbuild, so local stuff can be in a file named Makefile file

note: this can be achieved using makefile/Makefile today but it makes sense since there is not much 'Make' syntax left in kbuild makefiles today.

The Makefile can already include both the kbuild and local stuff (same snippet I sent you in personal communication already):

I know - but the incentive here was to seperate stuff out that does not belong in a Kbuild makefile.
In 2.7 I may write a simple parser to create one single big Makefile, and then it will be good to have very simple Makefiles only.

Quoted 5 lines

Now with mainline, when building external modules they will end up not having modversions. This is caused by the way .tmp_versions is handled, and is a real problem. There are two different ways how we are building external modules today:

Quoted 8 lines

(1) after the kernel source tree was just compiled, so the kernel source tree still contains all the object files,

(2) in a separate step, against an almost clean kernel source tree. Almost-clean means the tree contains a set of configuration files, and the modversions dump file.

The modversions dump file elegantly solves both cases.

You already convinced me about the usefullness of the modversions file. Can you try to make a patch that _only_ incorporate the modversions functionality. This we could ask Andrew to apply when reviewed. See it as first step.

Then I will try to work out something for the functionality outlined before.

Quoted 11 lines
Quoted 2 lines

Agree - should be easy to test using a CD.
Are there an easy way to mount just a directory structure RO?

Not sure what you mean exactly. My main motivation is this: we have a whole bunch of external modules that we build one after the other. Those external modules are notoriously nasty: We had cases where the modules fondled in kernel headers in the original tree. Of course then the next modules would build against a broken tree. To stop and detect such madness, I want to give modules a kernel source tree to which they have no write access, by chowning to a different user. Trees on read-only media are irrelevant IMHO.

Actually we agree, I just did not think of chowing the files/dirs to catch the problems.

Sam