Skip to contentSkip to conversation

sparse error: unable to open 'stdarg.h'

DISCUSSION Timur Tabi 6 participants 19 messages May 17, 2005 – May 22, 2005

lore
Share & tools
lore

Timur Tabi

author
#0 lore raw

I'm trying to run sparse on my external module, but sparse complains about not being able to find stdarg.h. I know this bug was supposed to have been fixed back in January, but I'm using the latest code, so I can't explain what's wrong. I've tried this on a couple different 2.6 kernels.

Here's the output I get:

make -C /lib/modules/2.6.8-24-smp/source SUBDIRS=/root/AMSO1100/software/host/linux/sys/devccil C=1 V=2 make[2]: Entering directory `/usr/src/linux-2.6.8-24'
CHECK /root/AMSO1100/software/host/linux/sys/devccil/devnet.c
include/linux/kernel.h:10:11: error: unable to open 'stdarg.h'
make[3]: *** [/root/AMSO1100/software/host/linux/sys/devccil/devnet.o] Error 1
make[2]: *** [_module_/root/AMSO1100/software/host/linux/sys/devccil] Error 2
make[2]: Leaving directory `/usr/src/linux-2.6.8-24'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/AMSO1100/software/host/linux/sys/devccil'
make: *** [build] Error 2

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
author
#1 lore raw
Quoted 1 line · Timur Tabi

make -C /lib/modules/2.6.8-24-smp/source SUBDIRS=/root/AMSO1100/software/host/linux/sys/devccil C=1 V=2

When I replace V=2 with V=1 (don't know how that happened), I get this output:

sparse -D__i386__=1 -Wp,-MD,/root/AMSO1100/software/host/linux/sys/devccil/.devnet.o.d -nostdinc -iwithprefix include -D__KERNEL__ -Iinclude -Wall -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -fomit-frame-pointer -pipe -msoft-float -mpreferred-stack-boundary=2 -funit-at-a-time -fno-unit-at-a-time -march=i586 -mregparm=3 -mcpu=i686 -Iinclude/asm-i386/mach-generic -Iinclude/asm-i386/mach-default -DCCNOPRINTF -DX86_32 -DEXPORT_SYMTAB -Wall -I/root/AMSO1100/software/host/linux/include -I/root/AMSO1100/software/host/common/include -I/root/AMSO1100/software/common/include -I/root/AMSO1100/software/common/include/clustercore -I/root/AMSO1100/software/host/linux/common -I/root/AMSO1100/software/host/linux/sys/devccil -DREMAP_API_CHANGE -DINCLUDE_CURRENT -DNET_DEVICE_HAS_IW -DPCI_SAVE_STATE_BUFFER -DQDISC_LIST_HEAD -DPCI_DMA_CPU -DUSE_GUP -DCCIL_KDAPL -DCCTHREADSAFE -Wa,-aldh=/root/AMSO1100/software/host/linux/sys/devccil/devnet.lst -DMODULE -DKBUILD_BASENAME=devnet -DKBUILD_MODNAME=ccil /root/AMSO1100/software/host/linux/sys/devccil/devnet.c ;

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
#2 lore raw

Review

Quoted 11 lines · Timur Tabi
Quoted 2 lines · Timur Tabi

make -C /lib/modules/2.6.8-24-smp/source SUBDIRS=/root/AMSO1100/software/host/linux/sys/devccil C=1 V=2

When I replace V=2 with V=1 (don't know how that happened), I get this output:

sparse -D__i386__=1

(quoted diff) +0 -2
-Wp,-MD,/root/AMSO1100/software/host/linux/sys/devccil/.devnet.o.d 
-nostdinc -iwithprefix include -D__KERNEL__ -Iinclude  -Wall 

Newer kbuild's do not use the -nostdinc -iwithprefix include trick. Instead they use -nostdinc -isystem gcc --print-file-name=include

Wich is a more reliable way to find stdarg.h. Newer sparse understands this too.

Please post make V=1 output with a newer kernel.

Sam

author
#3 lore raw
Quoted 7 lines · Sam Ravnborg

Newer kbuild's do not use the -nostdinc -iwithprefix include trick. Instead they use -nostdinc -isystem gcc --print-file-name=include

Wich is a more reliable way to find stdarg.h. Newer sparse understands this too.

Please post make V=1 output with a newer kernel.

I downloaded and install 2.6.11.10, and everything works. Thanks!

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
#4 lore raw
In reply to Timur Tabi

It is missing the gcc default include path.

Check your pre-processor.h in sparse to see that match your gcc include path or not.

If not, remove pre-processor.h and recompile sparse should solve the problem.

Chris

Quoted 34 lines · Timur Tabi

I'm trying to run sparse on my external module, but sparse complains about not being able to find stdarg.h. I know this bug was supposed to have been fixed back in January, but I'm using the latest code, so I can't explain what's wrong. I've tried this on a couple different 2.6 kernels.

Here's the output I get:

make -C /lib/modules/2.6.8-24-smp/source SUBDIRS=/root/AMSO1100/software/host/linux/sys/devccil C=1 V=2 make[2]: Entering directory `/usr/src/linux-2.6.8-24'
CHECK /root/AMSO1100/software/host/linux/sys/devccil/devnet.c
include/linux/kernel.h:10:11: error: unable to open 'stdarg.h' make[3]: *** [/root/AMSO1100/software/host/linux/sys/devccil/devnet.o] Error 1
make[2]: *** [_module_/root/AMSO1100/software/host/linux/sys/devccil] Error 2
make[2]: Leaving directory `/usr/src/linux-2.6.8-24'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/AMSO1100/software/host/linux/sys/devccil'
make: *** [build] Error 2

Signature · 12 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
author
#5 lore raw
Quoted 7 lines · Christopher Li

It is missing the gcc default include path.

Check your pre-processor.h in sparse to see that match your gcc include path or not.

If not, remove pre-processor.h and recompile sparse should solve the problem.

Nope, that didn't fix it. I deleted pre-process.h and re-ran "make", and it created a new one:

#define GCC_INTERNAL_INCLUDE "/usr/lib/gcc-lib/i586-suse-linux/3.3.4/include"

vic1:~/sparse-bk # ll /usr/lib/gcc-lib/i586-suse-linux/3.3.4/include/stdarg.h -rw-r--r-- 1 root root 4325 Oct 1 2004 /usr/lib/gcc-lib/i586-suse-linux/3.3.4/include/stdarg.h

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
#6 lore raw
Quoted 4 lines · Timur Tabi

Nope, that didn't fix it. I deleted pre-process.h and re-ran "make", and it created a new one:

Quoted 8 lines

#define GCC_INTERNAL_INCLUDE
"/usr/lib/gcc-lib/i586-suse-linux/3.3.4/include"

vic1:~/sparse-bk # ll /usr/lib/gcc-lib/i586-suse-linux/3.3.4/include/stdarg.h -rw-r--r-- 1 root root 4325 Oct 1 2004 /usr/lib/gcc-lib/i586-suse-linux/3.3.4/include/stdarg.h

That is wired. Can you try to edit a test.c contain just one line:

#include <stdarg.h>

run sparse on that test.c and see if you get any complain or not?

If you still get complain about file not found. Can you run "strace -e trace=file ./check test.c" and show me the out put?

Chris

PS, here is what I get:

[chrisl@64m sparse-be]$ strace -e trace=file ./check test.h execve("./check", ["./check", "test.h"], [/* 26 vars */]) = 0
open("/etc/ld.so.preload", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=58983, ...}) = 0
open("/lib/tls/libc.so.6", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0755, st_size=1539996, ...}) = 0
open("test.h", O_RDONLY) = 3
open("/usr/include/stdarg.h", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/include/stdarg.h", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/gcc-lib/i386-redhat-linux/3.2.2/include/stdarg.h", O_RDONLY) = 3 [chrisl@64m sparse-be]$

author
#7 lore raw
Quoted 5 lines · Christopher Li

That is wired. Can you try to edit a test.c contain just one line:

#include <stdarg.h>

run sparse on that test.c and see if you get any complain or not?

I did "sparse test.c" and got no output whatsoever. No files were created, either.

Quoted 3 lines

If you still get complain about file not found. Can you run "strace -e trace=file ./check test.c" and show me the out put?

I get the same thing:

execve("sparse-bk/check", ["sparse-bk/check", "test.c"], [/* 57 vars */]) = 0
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=74426, ...}) = 0
open("/lib/tls/libc.so.6", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0755, st_size=1359489, ...}) = 0
open("test.c", O_RDONLY) = 3
open("/usr/include/stdarg.h", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/local/include/stdarg.h", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/gcc-lib/i586-suse-linux/3.3.4/include/stdarg.h", O_RDONLY) = 3

There must be something specific about how kbuild calls sparse.

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
#8 lore raw
Quoted 10 lines · Timur Tabi
Quoted 5 lines · Christopher Li

That is wired. Can you try to edit a test.c contain just one line:

#include <stdarg.h>

run sparse on that test.c and see if you get any complain or not?

I did "sparse test.c" and got no output whatsoever. No files were created, either.

The sparse checker does not create files. That means your sparse is fine. It should locate the stdarg.h

Quoted 1 line

There must be something specific about how kbuild calls sparse.

I think I know that it is. There is a "-nostdinc" in the sparse options, which I saw it in the other email you send out. It drop the internal include path. Gcc is does the same thing.

gcc -c -nostdinc /tmp/test.c
/tmp/test.c:1:22: no include path in which to find stdarg.h

Chris

author
#9 lore raw
Quoted 6 lines · Christopher Li

I think I know that it is. There is a "-nostdinc" in the sparse options, which I saw it in the other email you send out. It drop the internal include path. Gcc is does the same thing.

gcc -c -nostdinc /tmp/test.c
/tmp/test.c:1:22: no include path in which to find stdarg.h

That option is set in the a_flags variable. I'm looking through the kbuild files (Makefile, etc) to see why a_flags is being used to build my driver.

As far as I'm concerned, this is a bug in kbuild, and I think it only shows up with external modules.

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
#10 lore raw
Quoted 11 lines · Timur Tabi
Quoted 6 lines · Christopher Li

I think I know that it is. There is a "-nostdinc" in the sparse options, which I saw it in the other email you send out. It drop the internal include path. Gcc is does the same thing.

gcc -c -nostdinc /tmp/test.c
/tmp/test.c:1:22: no include path in which to find stdarg.h

That option is set in the a_flags variable. I'm looking through the kbuild files (Makefile, etc) to see why a_flags is being used to build my driver.

Can you post a copy of you makefile. Then I may be able to tell you why.

Sam

author
#11 lore raw
Quoted 1 line · Sam Ravnborg

Can you post a copy of you makefile. Then I may be able to tell you why.

Ok, but it's really huge and ugly:

# Set some global variables

ifndef SOFTWARE
# Specifying 'export' turns SOFTWARE into a real environment variable, # not just a makefile variable. This means that when make calls itself # recursively in Section 1, the child process will already have SOFTWARE # defined, and won't try to re-define it. In Section 2, the current # directory is ${KERNEL_SOURCE}, so calculating SOFTWARE is impossible. export SOFTWARE = ${shell cd ../../../..; /bin/pwd}
endif

# Only if the Config.mk file exists will the next line include it. # (Inside Ammasso, the needed information is derived differently.) ifneq (${wildcard ${SOFTWARE}/../Config.mk},)

    include ${SOFTWARE}/../Config.mk

endif

ifndef KERNEL_SOURCE

    ${error KERNEL_SOURCE environment variable not defined.}

endif

# O points to where a 2.6 kernel tree put its build targets. Normally, the

# targets are placed in the same directory as the source, but you can use the
# "O=" option on the 'make' command line to override that.  If the kernel is
# built with the "O=" option, then all external modules must also be built
# with that option.  However, we also use the O= path here to determine where
# to find certain files.

ifdef O

    KERNEL_BUILD=${O}
else
    KERNEL_BUILD=${KERNEL_SOURCE}

endif

# Determine the kernel version. We only really care about 2.4 vs 2.6, so this

# simple code will work with version.h files that have multiple UTS_RELEASEs.
# Fail if version.h doesn't exist.

ifeq (${wildcard ${KERNEL_BUILD}/include/linux/version.h},)

    ${error ${KERNEL_BUILD}/include/linux/version.h does not exist.}

endif

KERNEL_VERSION=${shell grep -m 1 UTS_RELEASE ${KERNEL_BUILD}/include/linux/version.h | cut -f 2 -d'"'}

ifndef KERNEL_VERSION

    ${error Kernel version not found in ${KERNEL_BUILD}/include/linux/version.h.}

endif

ifndef KERNEL_CODE

    ${error KERNEL_CODE environment variable not defined.}

endif

# This makefile is divided into three sections. # Section 1 is the first pass of a kbuild-style makefile # Section 2 is the second pass of a kbuild-style makefile # Section 3 is the tradition 2.4-compatible makefile

# The concept of a two-pass kbuild-style makefile is taken from
# "Driver porting: compiling external modules" http://lwn.net/Articles/21823/

ifneq (${shell expr ${KERNEL_VERSION} : '2.4'}, 0)
SECTION = 3
else
ifeq (${KERNELRELEASE},)
SECTION = 1
else
SECTION = 2
endif
endif

# Check which type of build we want: release, debug, or trace.

ifeq (${CCRELEASE}, 1)
# Release build
BUILD_VERSION=release
EXTRA_CFLAGS += -DCCNOPRINTF
else

    ifeq (${CCDEBUGFAST}, 1)
        # Trace build
        BUILD_VERSION=trace
        EXTRA_CFLAGS += -DCCDEBUG -DCCNOPRINTF
    else
        # Debug build
        BUILD_VERSION=debug
        EXTRA_CFLAGS += -g -DCCDEBUG
    endif

endif

# Set the target directory for output files

TARGET_DIR = obj_${KERNEL_CODE}_${BUILD_VERSION}

# -------------------------- SECTION 1 -----------------------------

# Kbuild pass #1

ifeq (${SECTION}, 1)

# If O= is specified on the make command line, then you must have write

# access to KERNEL_SOURCE, otherwise the build will fail.  So we only pass
# O= if it is specified in Config.mk.

ifdef O

    KO=O=${KERNEL_BUILD}

endif

all:
# We need to manually copy files to the object directory

# because kbuild always dumps the targets in the current directory
	@rm -rf .tmp_versions ${TARGET_DIR}
	@mkdir -p ${TARGET_DIR}
	@${MAKE} -C ${KERNEL_SOURCE} SUBDIRS=${shell pwd} ${KO}
	@mv *.o *.ko *.lst ${TARGET_DIR}
	@rm -f .*.cmd *.mod.c

clean:

	@rm -rf .tmp_versions ${TARGET_DIR}

endif # Section 1

# ---------------------------- COMMON ------------------------------

# Here we calculate variables that are common to sections 2 and 3

ifneq (${SECTION}, 1)

# Determine where include/asm points to.

TEMP_KERNEL_ARCHDIR=$(subst -, ,${shell cd ${KERNEL_BUILD}/include/asm && pwd -P}) KERNEL_ARCHDIR=${word ${words ${TEMP_KERNEL_ARCHDIR}}, ${TEMP_KERNEL_ARCHDIR}}

# Specify the gcc parameters for this hardware platform.

ifeq (${PLATFORM}, x86_64)

    EXTRA_CFLAGS += -DX86_64 -mno-red-zone -mcmodel=kernel
else
    ifeq (${PLATFORM}, x86_32)
        EXTRA_CFLAGS += -DX86_32
    else
        ${error Unsupported platform}
    endif

endif

# Set the compiler paramaters. AMSO_CFLAGS is an environment variable

EXTRA_CFLAGS += -DEXPORT_SYMTAB -Wall ${AMSO_CFLAGS}
EXTRA_CFLAGS += -I${SOFTWARE}/host/linux/include -I${SOFTWARE}/host/common/include EXTRA_CFLAGS += -I${SOFTWARE}/common/include -I${SOFTWARE}/common/include/clustercore EXTRA_CFLAGS += -I${SOFTWARE}/host/linux/common -I${SOFTWARE}/host/linux/sys/devccil

# Define DO_MUNMAP_API_CHANGE if this kernel uses the version of do_unmap()

# that has four parameters instead of just three.

ifneq (${shell grep -c -m 1 do_munmap.*acct ${KERNEL_SOURCE}/include/linux/mm.h}, 0)

    EXTRA_CFLAGS += -DDO_MUNMAP_API_CHANGE

endif

# Define REMAP_API_CHANGE if this kernel uses the version of remap_page_range()

# that has five parameters

ifneq (${shell grep -c -m 1 remap_page_range.*vm_area_struct ${KERNEL_SOURCE}/include/linux/mm.h}, 0)

    EXTRA_CFLAGS += -DREMAP_API_CHANGE

endif

# Define REMAP_PFN_RANGE if the function remap_pfn_range() exists in mm.h # This function deprecates remap_page_range(). remap_pfn_range() uses a page

# index rather than a physical address, which allows it to support >4GB of RAM
# on 32-bit systems.

ifneq (${shell grep -c -m 1 remap_pfn_range ${KERNEL_SOURCE}/include/linux/mm.h}, 0)

    EXTRA_CFLAGS += -DREMAP_PFN_RANGE

endif

# Define INCLUDE_SYSCALL if linux/syscall.h exists.

ifneq (${wildcard ${KERNEL_SOURCE}/include/linux/syscall.h},)

    EXTRA_CFLAGS += -DINCLUDE_SYSCALL

endif

# Define INCLUDE_SYSTEM if interrupt.h does not include asm/system.h # Red Hat 8.0 (and maybe others) has a bug in interrupt.h where it forgets

# to include this header file.

ifeq (${shell grep -c asm/system\.h ${KERNEL_SOURCE}/include/linux/interrupt.h}, 0)

    EXTRA_CFLAGS += -DINCLUDE_SYSTEM

endif

# Define INCLUDE_CURRENT if hw_irq.h does not include sched.h and current.h # Red Hat 7.3 (and maybe others) has a bug in hw_irq.h where it forgets

# to include these header files.

ifeq (${shell grep -c asm/current\.h ${KERNEL_SOURCE}/include/asm-${KERNEL_ARCHDIR}/hw_irq.h}, 0)

    EXTRA_CFLAGS += -DINCLUDE_CURRENT

endif

# Define NET_DEVICE_HAS_IW if the net_device structure has a field called "wireless_handlers"

ifneq (${shell grep -cw wireless_handlers ${KERNEL_SOURCE}/include/linux/netdevice.h}, 0)

    EXTRA_CFLAGS += -DNET_DEVICE_HAS_IW

endif

# Define PCI_SAVE_STATE_BUFFER if function pci_save_state() has a parameter called "buffer"

ifneq (${shell grep -c pci_save_state.*buffer ${KERNEL_SOURCE}/include/linux/pci.h}, 0)

    EXTRA_CFLAGS += -DPCI_SAVE_STATE_BUFFER

endif

# Define QDISC_LIST_HEAD if the net_device.qdisc_list structure is of type # "struct list_head" instead of "struct Qdisc *". This change was made in 2.6.8

ifneq (${shell grep -cw list_head.*qdisc_list ${KERNEL_SOURCE}/include/linux/netdevice.h}, 0)

    EXTRA_CFLAGS += -DQDISC_LIST_HEAD

endif

# Define PCI_DMA_CPU if the function pci_dma_sync_single_for_cpu() exists. # If so, then we need to call this function instead of pci_dma_sync_single().

ifneq (${wildcard ${KERNEL_SOURCE}/include/asm-generic/pci-dma-compat.h},)

    ifneq (${shell grep -c pci_dma_sync_single_for_cpu 
${KERNEL_SOURCE}/include/asm-generic/pci-dma-compat.h}, 0)
        EXTRA_CFLAGS += -DPCI_DMA_CPU
    endif

endif

# Define SIGNAL_RLIM if rlim[] is a member of signal_struct instead of

# task_struct.  Since both signal_struct and task_struct are defined in
# sched.h, we need to grep the code for an actual usage of rlim[], i.e.
# "current->signal->rlim[]" as opposed to "current->rlim[]".  It looks like
# mm.h is a good candidate for this check.
ifneq (${shell grep -c -m 1 "signal->rlim" ${KERNEL_SOURCE}/include/linux/mm.h}, 0)
    EXTRA_CFLAGS += -DSIGNAL_RLIM
endif

# Define USE_GUP if get_user_pages() truly pins down pages. Kernels prior to # 2.6.7 had a bug in get_user_pages() that would unpin a page under extreme

# memory pressure.
ifneq (${wildcard ${KERNEL_SOURCE}/mm/rmap.c},)
    ifneq (${shell grep -c -m 1 try_to_unmap_one.*vm_area_struct 
${KERNEL_SOURCE}/mm/rmap.c}, 0)
        EXTRA_CFLAGS += -DUSE_GUP
        USE_GUP=1
    endif
endif

ifndef USE_GUP
# If we can't use get_user_pages(), then we should use mlock(). Normally, # we would need a kernel that support non-root mlock(), but the driver # actually overrides any limitiations on mlock(). EXTRA_CFLAGS += -DUSE_MLOCK
USE_MLOCK=1

# Define USE_MLOCK if this kernel supports mlock for non-root processes. # If so, then we libccil needs to call mlock instead of having the driver # do it. We also skip all the code to determine the sys_mlock() calling method.
# ifneq (${shell grep -c -m 1 can_do_mlock ${KERNEL_SOURCE}/include/linux/mm.h}, 0)
# EXTRA_CFLAGS += -DUSE_MLOCK
# USE_MLOCK=1
# endif
endif

# If neither USE_MLOCK nor USE_GUP is defined, then we want the driver to

# call sys_mlock for us.
ifndef USE_MLOCK
    ifndef USE_GUP
        EXTRA_CFLAGS += -DUSE_SYSCALL
        USE_SYSCALL=1
    endif
endif

ifdef USE_MLOCK
SYSCALL_METHOD = "Using mlock() system call"
endif
ifdef USE_GUP
SYSCALL_METHOD = "Using get_user_pages()"
endif
ifdef USE_SYSCALL
# Here we determine which method we will use to call sys_mlock(). # The rule is:

# If /proc/k[all]syms exists, then scan it for a list of syscalls # If /proc/k[all]syms doesn't exists or we can't find at least 2 syscalls,
# then scan the kernel source tree for a list of syscalls
# If that doesn't work either, then look up the sys_mlock entry point in
# /boot/System.map-uname -r
# If that also doesn't work, and if we're x86-32, then use KERNEL_SYSCALLS # Otherwise, we can't call sys_mlock(). Exit with failure.

ifneq (${wildcard /proc/ksyms},)
SYMFILE = /proc/ksyms
else
ifneq (${wildcard /proc/kallsyms},)
SYMFILE = /proc/kallsyms
endif
endif

# A list of system calls we look for
SYSCALLS = open close read write lseek wait4

ifdef SYMFILE
# The kernel symbol file is readable, so let's use it
SYMLIST = ${shell x=1; for i in ${SYSCALLS}; do if [ "`grep -sh -m 1 --mmap
\"[^t] sys_$$i\(_R[_[:alnum:]]*\|\)$$\" ${SYMFILE}" ]; then echo "-DSYSCALL$$x=$$i"; x=expr $$x + 1`; fi; done}
else
# The kernel symbol file is not readable, so we need to scan the source tree
# We also need to search the architecture-specific trees
ARCH=${KERNEL_SOURCE}/arch/${KERNEL_ARCHDIR}/kernel/*
SYMLIST = ${shell x=1; for i in ${SYSCALLS}; do if [ "`grep -sh -m 1 --mmap
EXPORT_SYMBOL[_GPL]*\(sys_$$i ${KERNEL_SOURCE}/fs/* ${KERNEL_SOURCE}/kernel/* ${ARCH}" ]; then echo "-DSYSCALL$$x=$$i"; x=expr $$x + 1`; fi; done}
endif
ifeq (${shell expr echo ${SYMLIST} | wc -w \>= 2}, 1)
# There are enough syscalls, so let's use the list
EXTRA_CFLAGS += ${SYMLIST}
SYSCALL_METHOD = "Using system call table method"
else
SYMFILE = /boot/System.map-${shell uname -r}
# There aren't enough syscalls, so let's try System.map-`uname -r
ifeq (${shell grep -cw "\(sys_mlock\|sys_munlock\)" ${SYMFILE}}, 2)
# We found sys_mlock and sys_munlock, so we'll use their addresses.
# The addresses will be passed in on the insmod command-line,
# by our loader script.
EXTRA_CFLAGS += -DSYSTEM_MAP
SYSCALL_METHOD = "Using System.map method"
else
# We couldn't find sys_m[un]lock, so search for other syscalls in
# System.map. We are assuming that System.map contains entries
# only for exported functions, not every function in the kernel.
SYMLIST = ${shell x=1; for i in ${SYSCALLS}; do if [ "`grep -sh -m 1 --mmap
\"[^t] sys_$$i\(_R[_[:alnum:]]*\|\)$$\" ${SYMFILE}" ]; then echo "-DSYSCALL$$x=$$i"; x=expr $$x + 1`; fi; done}
ifeq (${shell expr echo ${SYMLIST} | wc -w \>= 2}, 1)
# There are enough syscalls, so let's use the list
EXTRA_CFLAGS += ${SYMLIST}
SYSCALL_METHOD = "Using system call table method from System.map"
else
# Nothing so far has worked, so if we're x86-32 we can use
# kernel syscalls, otherwise we give up.
ifeq (${PLATFORM}, x86_32)
EXTRA_CFLAGS += -DKERNEL_SYSCALLS
SYSCALL_METHOD = "Using kernel syscall method"
else
# None of our options are going to work, so just give up
${error Could not determine system call method}
endif
endif
endif
endif
endif

# Support for KDAPL
EXTRA_CFLAGS += -DCCIL_KDAPL -DCCTHREADSAFE

# Source files

CFILES = \
        devnet.c \
        ccilnet.c \
        devccil.c \
        devccil_adapter.c \
        devccil_rnic.c \
        devccil_mem.c \
        devccil_vq.c \
        devccil_eh.c \
        devccil_cq.c \
        devccil_mq.c \
        devccil_pd.c \
        devccil_srq.c \
        devccil_qp.c \
        devccil_mm.c \
        devccil_ep.c \
        devccil_wrappers.c \
        devccil_ae.c \
        devccil_logging.c
COMMON_CFILES   = \
		cc_cq_common.c \
                cc_mq_common.c \
                cc_qp_common.c

# -------------------------- SECTION 2 -----------------------------

# Kbuild pass #2

ifeq (${SECTION}, 2)

SOFTWARE = ${shell cd ${SUBDIRS}/../../../..; pwd}

# Generate an assembly listing for each file

EXTRA_CFLAGS += -Wa,-aldh=$*.lst

# Fix the paths for the common .c files CFILES += $(addprefix ../../common/, ${COMMON_CFILES})

# Linker parameters

obj-m := ccil.o

ccil-objs := ${CFILES:.c=.o}

syscall:

	@echo ${SYSCALL_METHOD}

endif # Section 2

# -------------------------- SECTION 3 -----------------------------

# Kernel 2.4-compatible makefile

ifeq (${SECTION}, 3)

include ${SOFTWARE}/header_gcc.mk

COMMON_OFILES:=$(patsubst %.c, ${SOFTWARE}/host/linux/sys/devccil/${TARGET_DIR}/%.o, ${COMMON_CFILES}) COMMON_CFILES:=$(addprefix ${SOFTWARE}/host/linux/common/,${COMMON_CFILES})

# Here we try to identify the Scyld kernel, because it has peculiarities. # If linux/fs.h does not have a prototype for sys_read, but asm/unistd.h does,

# and it's X86_64, then we assume that this is the Scyld kernel.
# This kernel needs to have __KERNEL_SYSCALLS__ defined in order to pick up
# the sys_xxx prototypes.  It also needs MODVERSIONS defined and modversions.h
# needs to be included in every source file.

ifeq (${PLATFORM}, x86_64)

    ifeq (${shell grep -cw sys_read ${KERNEL_SOURCE}/include/linux/fs.h}, 0)
        ifeq (${shell grep -cw -m 1 sys_read 
${KERNEL_SOURCE}/include/asm-${KERNEL_ARCHDIR}/unistd.h}, 1)
            EXTRA_CFLAGS += -D__KERNEL_SYSCALLS__ -DMODVERSIONS -include linux/modversions.h
        endif
    endif

endif

# Reset CFLAGS because we don't like the way header_gcc.mk initializes it

CFLAGS = ${EXTRA_CFLAGS} -O2 -fno-strict-aliasing -D__KERNEL__ -DMODULE -I${KERNEL_SOURCE}/include

# Generate an assembly listing for each file

COMMON_CFLAGS := ${CFLAGS}
CFLAGS += -Wa,-aldh=${TARGET_DIR}/$*.lst

# Linker parameters

LDFLAGS += -r -E -d --whole-archive

all: syscall ${TARGET_DIR} ${COMMON_OFILES} ${TARGET_DIR}/ccil.o

syscall:

	@echo ${SYSCALL_METHOD}

${COMMON_OFILES}: ${COMMON_CFILES}

	${CC} -c ${COMMON_CFLAGS} -Wa,-aldh=/$*.lst -o $@ ${SOFTWARE}/host/linux/common/$(notdir 

$*).c

include ${SOFTWARE}/footer_gcc.mk

${TARGET_DIR}/ccil.o: ${OBJECTS} ${COMMON_OFILES}

	${LD} ${LDFLAGS} -o $@ ${OBJECTS} ${COMMON_OFILES}

clean::

	@rm -fr ${TARGET_DIR}

unload:
@echo "Unloading ccil.o"
-@rmmod ccil

load: unload

	@echo "Loading ${TARGET_DIR}/ccil.o"
	@./loadccil.bash ${TARGET_DIR}/ccil.o

endif # Section 3

endif # ifneq (${SECTION}, 1)

Signature · 7 lines
Timur Tabi
Staff Software Engineer
[email protected]

One thing a Southern boy will never say is,
"I don't think duct tape will fix it."
     -- Ed Smylie, NASA engineer for Apollo 13
#12 lore raw

Kbuild trick

Hi Timur.
Some tricks/hints.

For both kernel 2.4 and 2.6 you can split up your makefile like this:
makefile <= all the external modules specific part Makefile <= the kbuild specific part

Recent 2.6 kernels looks for a fine named Kbuild before Makefile so you can use the file Kbuild for a 2.6 compatible kbuild file, and Makefile for a 2.4 compatible kbuild file.

Quoted 5 lines

# Only if the Config.mk file exists will the next line include it. # (Inside Ammasso, the needed information is derived differently.) ifneq (${wildcard ${SOFTWARE}/../Config.mk},)

    include ${SOFTWARE}/../Config.mk

endif

-include does the same

Quoted 4 lines

# Determine the kernel version. We only really care about 2.4 vs 2.6, so this

# simple code will work with version.h files that have multiple 
UTS_RELEASEs.

In the kbuild part of the file you can check for KERNELRELEASE. And a lot of what is below ought to be in the kbuild part of the file.

Quoted 3 lines

# If O= is specified on the make command line, then you must have write

# access to KERNEL_SOURCE, otherwise the build will fail.  So we only pass
# O= if it is specified in Config.mk.

O= does not require write access to KERNEL_SOURCE. Thats one one the main concpets. KERNEL_SOURCE may be owned by whoever.

Quoted 7 lines

# Define DO_MUNMAP_API_CHANGE if this kernel uses the version of do_unmap()

# that has four parameters instead of just three.

ifneq (${shell grep -c -m 1 do_munmap.*acct
${KERNEL_SOURCE}/include/linux/mm.h}, 0)

    EXTRA_CFLAGS += -DDO_MUNMAP_API_CHANGE

endif

A macro can simplify this:

feature = $(if $(shell grep -m 1 $(1) $(srctree)/include/$(2)), $(3))

Usage:
EXTRA_CFLAGS += $(call feature, do_munmap.*acct, linux/mm.h, -DDO_MUNMAP_API_CHANGE) EXTRA_CFLAGS += $(call feature, remap_page_range.*vm_area_struct, \
linux/mm.h, -DREMAP_API_CHANGE)

etc..

Quoted 8 lines

ifneq (${wildcard /proc/ksyms},)
SYMFILE = /proc/ksyms
else
ifneq (${wildcard /proc/kallsyms},)
SYMFILE = /proc/kallsyms
endif
endif

This looks wrong. Either you shall check kernel source or running kernel.
But mixing it like this is to call for troubles.

Quoted 22 lines

# Source files

CFILES = \
        devnet.c \
        ccilnet.c \
        devccil.c \
        devccil_adapter.c \
        devccil_rnic.c \
        devccil_mem.c \
        devccil_vq.c \
        devccil_eh.c \
        devccil_cq.c \
        devccil_mq.c \
        devccil_pd.c \
        devccil_srq.c \
        devccil_qp.c \
        devccil_mm.c \
        devccil_ep.c \
        devccil_wrappers.c \
        devccil_ae.c \
        devccil_logging.c
Quoted 5 lines
COMMON_CFILES   = \
		cc_cq_common.c \
                cc_mq_common.c \
                cc_qp_common.c

In kbuild files the usual pattern is to specify the .o files not the source files. So for anyone familiar with kbuild files this looks wrong. And be explicit. No reason to hide directories behind the addprefix macro below. This is not an obscufating contest.

Quoted 6 lines

# Fix the paths for the common .c files CFILES += $(addprefix ../../common/, ${COMMON_CFILES})

# Generate an assembly listing for each file

EXTRA_CFLAGS += -Wa,-aldh=$*.lst

If needed you can use: make cc_mp_common.lst to let kbuild generate them. Dunno if it works for external modules btw.

Quoted 10 lines

# Linker parameters

obj-m := ccil.o

ccil-objs := ${CFILES:.c=.o}

syscall:

	@echo ${SYSCALL_METHOD}

As you have noticed in another mail - this does not work. An untested approch would be to use:
.PHONY: syscall
$(obj)/ccil.o: syscall

Sam

#13 lore raw

Re: Kbuild trick

Quoted 3 lines · Sam Ravnborg

For both kernel 2.4 and 2.6 you can split up your makefile like this:
makefile <= all the external modules specific part Makefile <= the kbuild specific part

You could also use our fake-2.6-kbuild-for-2.4 makefile,
retrievable via:

svn cat -r 2205 http://oss.oracle.com/projects/ocfs2/src/trunk/Kbuild-24.make

Just include it and set your target to build-modules,
clean-modules, or install-modules.

Joel

Signature · 8 lines
To spot the expert, pick the one who predicts the job will take the
longest and cost the most.

Joel Becker
Senior Member of Technical Staff
Oracle
E-mail: [email protected]
Phone: (650) 506-8127
#14 lore raw

Re: Kbuild trick

Quoted 9 lines · Joel Becker
Quoted 3 lines · Sam Ravnborg

For both kernel 2.4 and 2.6 you can split up your makefile like this:
makefile <= all the external modules specific part Makefile <= the kbuild specific part

You could also use our fake-2.6-kbuild-for-2.4 makefile,
retrievable via:

svn cat -r 2205 http://oss.oracle.com/projects/ocfs2/src/trunk/Kbuild-24.make

$ which svn
which: no svn in (/bin:/usr/bin:/.....)

Care to post a copy?

Sam

#15 lore raw

Re: Kbuild trick

On Sat, 21 May 2005 07:12:17 +0200 Sam Ravnborg wrote:

| On Fri, May 20, 2005 at 04:43:53PM -0700, Joel Becker wrote:
| > On Fri, May 20, 2005 at 09:37:06PM +0200, Sam Ravnborg wrote:
| > > For both kernel 2.4 and 2.6 you can split up your makefile like this:
| > > makefile <= all the external modules specific part
| > > Makefile <= the kbuild specific part
| > 
| > 	You could also use our fake-2.6-kbuild-for-2.4 makefile,
| > retrievable via:
| > 
| > svn cat -r 2205 http://oss.oracle.com/projects/ocfs2/src/trunk/Kbuild-24.make
| 
| $ which svn
| which: no svn in (/bin:/usr/bin:/.....)
| 
| Care to post a copy?

svn == Subversion, at http://subversion.tigris.org/

oh, you mean post a copy of Kbuild-24.make; yes, that makes sense.

---
~Randy

#16 lore raw

Re: Kbuild trick

In reply to Sam Ravnborg
Quoted 3 lines · Sam Ravnborg
Quoted 1 line

svn cat -r 2205 http://oss.oracle.com/projects/ocfs2/src/trunk/Kbuild-24.make

Care to post a copy?

That would be too logical. Here you go. The first file is a
sample Makefile for both 2.6 and 2.4, stripped out of our history. Any mistakes are mine. The second is Kbuild-24.make. The files expect KERNELDIR and KERNELINC to be set, and a 2.4 build needs to set GCCINC as well. Use your favorite autoconf/anti-autoconf system.

Joel

Signature · 8 lines
"Well-timed silence hath more eloquence than speech."  
         - Martin Fraquhar Tupper

Joel Becker
Senior Member of Technical Staff
Oracle
E-mail: [email protected]
Phone: (650) 506-8127

obj-m += myfs.o
myfs-objs := inode.o super.o

EXTRA_CFLAGS += -DSOMETHING

ifeq ($(KERNELRELEASE),)
# Called as "make" part of makefile. This executes kbuild

all: build-myfs

clean: clean-myfs

install: install-myfs

ifneq ($(KERNEL_26),)
# This is a 2.6 kenrel, forward to kbuild

build-myfs:

	$(MAKE) -C $(KERNELDIR) M=`pwd` modules

clean-myfs:

	$(MAKE) -C $(KERNELDIR) M=`pwd` clean

install-myfs:

	$(MAKE) -C $(KERNELDIR) M=`pwd` modules_install

else
# This is a 2.4 kernel, use our own

include Kbuild-24.make

endif

else
# Called from 2.6 kbuild

STAMP_DIR = pwd

endif

#
# This is a Makefile for 2.4 external module builds. Red Hat # Enterprise Linux does not support the usual:
#
# make -C /lib/modules/uname -r/build SUBDIRS=pwd modules
#
# So we must hand-roll our compile lines. This is in a seperate

# file to keep the regular Makefile readable.
#
# This uses the Makebo schema.  So, the compile parts are filled in.
# All Makefiles using this code should set MODULE_CFLAGS and
# MODULE_DEFINES for any extra cflags/defines needed.
#

WARNINGS = -Wall -Wstrict-prototypes
OPTIMIZE = -O2

KERNEL_DEFINES = -D__KERNEL__ -DMODULE -DLINUX -DEXPORT_SYMTAB KERNEL_CFLAGS = -pipe -nostdinc -fno-strict-aliasing -fno-common \
-fomit-frame-pointer
KERNEL_LDADD = -nostdlib

#
# These arch-specific flags would normally be provided by the

# kbuild system.  Hope we got them right.
#

ifeq ($(PROCESSOR),ppc64)
MACH_CFLAGS += -m64 -fsigned-char -fno-builtin -msoft-float \
-mminimal-toc
MACH_LDADD += -m elf64ppc
endif
ifeq ($(PROCESSOR),x86_64)
MACH_CFLAGS += -m64 -mcmodel=kernel
endif
ifeq ($(PROCESSOR),s390x)
MACH_CFLAGS += -fno-strength-reduce -fpic MACH_LDADD += -m elf64_s390
endif

INCLUDES = -I. -I$(KERNELINC) -I$(GCCINC)
DEFINES = $(KERNEL_DEFINES)

CFLAGS = $(OPTIMIZE) $(KERNEL_CFLAGS) $(MACH_CFLAGS) $(MODVERSIONS) \
	$(WARNINGS) $(EXTRA_CFLAGS) $(CFLAGS_$(*F).o)

LDADD = $(KERNEL_LDADD) $(MACH_LDADD)

#
# Ripping off some kbuild 2.6 tricks
#

# Modules that have multiple parts.
multi-used-m := $(sort $(foreach m, $(obj-m), $(if $(strip $($(m:.o=-objs))), $(m)))) multi-objs-m := $(foreach m, $(multi-used-m), $($(m:.o=-objs)))

# Modules that don't.
single-used-m := $(sort $(filter-out $(multi-used-m), $(obj-m))) link_multi_deps = $($(@:.o=-objs))

#
# It's impossible, AFAICT, to generate the dependancies for each

# individual object:parts set.  In fact, kbuild 2.6 doesn't try either.
# They use a simpler trick...they force the rebuild like this, but
# call if_changed on each target to avoid the useless rebuilds.
# This is a 2.4 hack, so I don't care that much :-)
#
$(multi-used-m): $(multi-objs-m)
	$(CC) -Wl,-r -o $@ $(LDADD) $(link_multi_deps)

build-modules: $(single-used-m) $(multi-used-m)

# Kbuild uses INSTALL_MOD_PATH for rooted installs. Makebo uses # DESTDIR. Folks really should set INSTALL_MOD_PATH, but in case they # don't...
INSTALL_MOD_PATH ?= $(DESTDIR)
MODLIB = $(INSTALL_MOD_PATH)$(MODULEDIR)

install-modules: $(obj-m)

	$(TOPDIR)/mkinstalldirs $(MODLIB)/$(INSTALL_MOD_DIR)
	for file in $(obj-m); do \
	  $(INSTALL_DATA) $$file $(MODLIB)/$(INSTALL_MOD_DIR)/$$file; \
	done

clean-modules:

	rm -f *.o *.p *.s
#17 lore raw

Re: Kbuild trick

In reply to Sam Ravnborg

One of the question that I haven't yet managed to solve properly is how do we manage kernel modules which have its C files in multiple underlying directories.

say :
/src/Makefile
/src/main.c
/src/module1/Makefile
/src/module1/module1.c
/src/module1/module_stuff.c

And the 3 C files should be built into a single module at the top level. I tried something like this in the top level Makefile, but it does not work.

obj-m += mymodule.o
main-objs=main.o module1/

I could use,
main-objs=main.o module1/module1.o module1/module_stuff.o

But I don't think that is a good idea. I looked at Documentation/kbuild/, but no luck.

Any suggestions?

Thanks,
Kedar.

Quoted 133 lines · Sam Ravnborg

Hi Timur.
Some tricks/hints.

For both kernel 2.4 and 2.6 you can split up your makefile like this:
makefile <= all the external modules specific part Makefile <= the kbuild specific part

Recent 2.6 kernels looks for a fine named Kbuild before Makefile so you can use the file Kbuild for a 2.6 compatible kbuild file, and Makefile for a 2.4 compatible kbuild file.

Quoted 5 lines

# Only if the Config.mk file exists will the next line include it. # (Inside Ammasso, the needed information is derived differently.) ifneq (${wildcard ${SOFTWARE}/../Config.mk},)

    include ${SOFTWARE}/../Config.mk

endif

-include does the same

Quoted 4 lines

# Determine the kernel version. We only really care about 2.4 vs 2.6, so this

# simple code will work with version.h files that have multiple
UTS_RELEASEs.

In the kbuild part of the file you can check for KERNELRELEASE. And a lot of what is below ought to be in the kbuild part of the file.

Quoted 3 lines

# If O= is specified on the make command line, then you must have write

# access to KERNEL_SOURCE, otherwise the build will fail.  So we only pass
# O= if it is specified in Config.mk.

O= does not require write access to KERNEL_SOURCE. Thats one one the main concpets. KERNEL_SOURCE may be owned by whoever.

Quoted 7 lines

# Define DO_MUNMAP_API_CHANGE if this kernel uses the version of do_unmap()

# that has four parameters instead of just three.

ifneq (${shell grep -c -m 1 do_munmap.*acct
${KERNEL_SOURCE}/include/linux/mm.h}, 0)

    EXTRA_CFLAGS += -DDO_MUNMAP_API_CHANGE

endif

A macro can simplify this:

feature = $(if $(shell grep -m 1 $(1) $(srctree)/include/$(2)), $(3))

Usage:
EXTRA_CFLAGS += $(call feature, do_munmap.*acct, linux/mm.h, -DDO_MUNMAP_API_CHANGE) EXTRA_CFLAGS += $(call feature, remap_page_range.*vm_area_struct, \
linux/mm.h, -DREMAP_API_CHANGE)

etc..

Quoted 8 lines

ifneq (${wildcard /proc/ksyms},)
SYMFILE = /proc/ksyms
else
ifneq (${wildcard /proc/kallsyms},)
SYMFILE = /proc/kallsyms
endif
endif

This looks wrong. Either you shall check kernel source or running kernel.
But mixing it like this is to call for troubles.

Quoted 22 lines

# Source files

CFILES = \
        devnet.c \
        ccilnet.c \
        devccil.c \
        devccil_adapter.c \
        devccil_rnic.c \
        devccil_mem.c \
        devccil_vq.c \
        devccil_eh.c \
        devccil_cq.c \
        devccil_mq.c \
        devccil_pd.c \
        devccil_srq.c \
        devccil_qp.c \
        devccil_mm.c \
        devccil_ep.c \
        devccil_wrappers.c \
        devccil_ae.c \
        devccil_logging.c
Quoted 5 lines
COMMON_CFILES   = \
              cc_cq_common.c \
                cc_mq_common.c \
                cc_qp_common.c

In kbuild files the usual pattern is to specify the .o files not the source files. So for anyone familiar with kbuild files this looks wrong. And be explicit. No reason to hide directories behind the addprefix macro below. This is not an obscufating contest.

Quoted 6 lines

# Fix the paths for the common .c files CFILES += $(addprefix ../../common/, ${COMMON_CFILES})

# Generate an assembly listing for each file

EXTRA_CFLAGS += -Wa,-aldh=$*.lst

If needed you can use: make cc_mp_common.lst to let kbuild generate them. Dunno if it works for external modules btw.

Quoted 10 lines

# Linker parameters

obj-m := ccil.o

ccil-objs := ${CFILES:.c=.o}

syscall:

      @echo ${SYSCALL_METHOD}

As you have noticed in another mail - this does not work. An untested approch would be to use:
.PHONY: syscall
$(obj)/ccil.o: syscall

Sam
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#18 lore raw

Re: Kbuild trick

Quoted 11 lines · Kedar Sovani

One of the question that I haven't yet managed to solve properly is how do we manage kernel modules which have its C files in multiple underlying directories.

say :
/src/Makefile
/src/main.c
/src/module1/Makefile
/src/module1/module1.c
/src/module1/module_stuff.c

Quoted 5 lines

And the 3 C files should be built into a single module at the top level. I tried something like this in the top level Makefile, but it does not work.

obj-m += mymodule.o
main-objs=main.o module1/

This create a number of modules - sometimes a good way to do it. We have several examples in the kernel using this method.

Quoted 6 lines

I could use,
main-objs=main.o module1/module1.o module1/module_stuff.o

But I don't think that is a good idea. I looked at Documentation/kbuild/, but no luck.

This is one way to do it.
The other way as used by the kernel is to avoid spreading the .c files in multiple directories.

Sam