parallel make problems (-mm) -now found in 2.6.5-rc1 also
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Sam,
randy dunlap is on vacation this week,
(and i was last week ) so i am trying to get a resolution to the parallel make problem.
Tested your patch below and it didn't help us. The patch doesn't apply clean past 2.6.4-mm1.
We now see this exact failure against 2.6.5-rc1 for our 8-way machines, so we have much concern.
Is there another fix around we could try?
cliffw
OSDL
FYI - repeat of sample error:
------------
STP: found 8 processor(s)
STP: stp_robokern changing into Linux directory
STP: stp_robokern correcting possible permission issues
STP: stp_robokern running 'make mrproper'
STP: stp_robokern calling stp_sanitize-config
STP: stp_robokern 'make clean'
STP: stp_robokern 'make -s -j8'
CHK include/linux/version.h
UPD include/linux/version.h
SYMLINK include/asm -> include/asm-i386/bin/sh: line 1: scripts/fixdep: Permission denied make[1]: *** [scripts/empty.o] Error 1 make[1]: *** Waiting for unfinished jobs....
make: *** [scripts] Error 2
make: *** Waiting for unfinished jobs....
optimize && ? optimize && ? optimize && ? optimize && ?
--------------------------
Quoted 2,236 lines · On Fri, 12 Mar 2004 12:00:24 -0800 "Randy.Dunlap" <[email protected]> wrote:
Here's a fix from Sam Ravnborg. It applies to latest -mm. Can you test it, please?
~Randy
Begin forwarded message:
Date: Fri, 12 Mar 2004 21:02:05 +0100
From: Sam Ravnborg <[email protected]>
To: "Randy.Dunlap" <[email protected]>
Cc: [email protected]
Subject: Re: parallel make problems (-mm)Quoted 6 lines · Randy.Dunlap
Hi Sam,
We are seeing some possible parallel build problems. This is most frequently on an 8-proc system, using make -j8. It also occurs less often on a 4-proc system, using make -j4.
Ingo complained about the error 'text file is busy' when doing make -j10, and I could reproduce it on my UP machine.
The problem was even visible with make -j2, where I could see fixdep being build twice.
The cure was not as simple as I originally expected. I had to introduce an additional step to build the very basic files in scripts, so I moved those to a separate directoryscripts/basic/The patch is attached. Any feedback appreciated.
It's on top of latest -mmMost of the patch is due to moving the three files.
Sam
# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.1868 -> 1.1869
#scripts/docproc.c1.4 -> 1.5scripts/basic/docproc.c(moved)
#scripts/split-include.c1.3 -> 1.4scripts/basic/split-include.c(moved)# scripts/fixdep.c 1.13 -> 1.14 scripts/basic/fixdep.c (moved) # scripts/Makefile.lib 1.23 -> 1.24 # scripts/Makefile 1.36 -> 1.37 # Makefile 1.466 -> 1.467#
scripts/Makefile.build1.42 -> 1.43
# (new) -> 1.1scripts/basic/Makefile
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 04/03/11 [email protected] 1.1869
# Fix dependencies in early phases
# --------------------------------------------
#
diff -Nru a/Makefile b/MakefileMakefile +21
-16@@ -304,17 +304,10 @@ # =========================================================================== # Rules shared between *config targets and build targets -# Helpers built in scripts/ - -scripts/docproc scripts/split-include : scripts ; - -.PHONY: scripts scripts/fixdep -scripts: - $(Q)$(MAKE) $(build)=scripts - -scripts/fixdep: - $(Q)$(MAKE) $(build)=scripts $@ - +# Basic helpers built in scripts/ +.PHONY: scripts_basic +scripts_basic: + $(Q)$(MAKE) $(build)=scripts/basic # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile@@ -358,9 +351,9 @@ # *config targets only - make sure prerequisites are updated, and descend # in scripts/kconfig to make the *config target -%config: scripts/fixdep FORCE +config: scripts_basic FORCE $(Q)$(MAKE) $(build)=scripts/kconfig $@ -config : scripts/fixdep FORCE +%config: scripts_basic FORCE $(Q)$(MAKE) $(build)=scripts/kconfig $@ else@@ -368,6 +361,16 @@ # Build targets only - this includes vmlinux, arch specific targets, clean # targets and others. In general all targets except *config targets. +# Additional helpers built in scripts/ +# Carefully list dependencies so we do not try to build scripts twice +# in parrallel +.PHONY: scripts +scripts: scripts_basic include/config/MARKER + $(Q)$(MAKE) $(build)=$(@) + +scripts_basic: include/linux/autoconf.h + + # That's our default target when none is given on the command line # Note that 'modules' will be added as a prerequisite as well, # in the CONFIG_MODULES part below@@ -400,7 +403,9 @@ # with it and forgot to run make oldconfig include/linux/autoconf.h: .config $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig - +else +# Dummy target needed, because used as prerequisite +include/linux/autoconf.h: ; endif include $(srctree)/arch/$(ARCH)/Makefile@@ -634,7 +639,7 @@ # Split autoconf.h into include/linux/config/* -include/config/MARKER: scripts/split-include include/linux/autoconf.h +include/config/MARKER: include/linux/autoconf.h @echo ' SPLIT include/linux/autoconf.h -> include/config/*' @scripts/split-include include/linux/autoconf.h include/config @touch $@@@ -930,7 +935,7 @@ # Documentation targets # --------------------------------------------------------------------------- -%docs: scripts/docproc FORCE +%docs: scripts FORCE $(Q)$(MAKE) $(build)=Documentation/DocBook $@ # Scripts to check various things for consistencydiff -Nru a/scripts/Makefile b/scripts/Makefile
scripts/Makefile +2
-9@@ -2,14 +2,10 @@ # scripts contains sources for various helper programs used throughout # the kernel for the build process. # --------------------------------------------------------------------------- -# fix-dep: Used to generate dependency information during build process -# split-include: Divide all config symbols up in a number of files in -# include/config/... # docproc: Preprocess .tmpl file in order to generate .sgml docs # conmakehash: Create arrays for initializing the kernel console tables -host-progs := fixdep split-include conmakehash docproc kallsyms modpost \ - mk_elfconfig pnmtologo bin2c +host-progs := conmakehash kallsyms modpost mk_elfconfig pnmtologo bin2c always := $(host-progs) empty.o modpost-objs := modpost.o file2alias.o sumversion.o@@ -17,10 +13,7 @@ subdir-$(CONFIG_MODVERSIONS) += genksyms # Let clean descend into subdirs -subdir- += lxdialog kconfig - -# fixdep is needed to compile other host programs -$(addprefix $(obj)/,$(filter-out fixdep,$(always)) $(subdir-y)): $(obj)/fixdep +subdir- += basic lxdialog kconfig # dependencies on generated files need to be listed explicitlydiff -Nru a/scripts/Makefile.build b/scripts/Makefile.build
scripts/Makefile.build +1
-1@@ -162,7 +162,7 @@ $(if $($(quiet)cmd_cc_o_c),echo ' $($(quiet)cmd_cc_o_c)';) \ $(cmd_cc_o_c); \ $(cmd_modversions) \ - scripts/fixdep $(depfile) $@ '$(cmd_cc_o_c)' > $(@D)/.$(@F).tmp; \ + scripts/basic/fixdep $(depfile) $@ '$(cmd_cc_o_c)' > $(@D)/.$(@F).tmp; \ rm -f $(depfile); \ mv -f $(@D)/.$(@F).tmp $(@D)/.$(@F).cmd endefdiff -Nru a/scripts/Makefile.lib b/scripts/Makefile.lib
scripts/Makefile.lib +1
-1@@ -249,7 +249,7 @@ @set -e; \ $(if $($(quiet)cmd_$(1)),echo ' $(subst ','\'',$($(quiet)cmd_$(1)))';) \ $(cmd_$(1)); \ - scripts/fixdep $(depfile) $@ '$(subst $$,$$$$,$(subst ','\'',$(cmd_$(1))))' > $(@D)/.$(@F).tmp; \ + scripts/basic/fixdep $(depfile) $@ '$(subst $$,$$$$,$(subst ','\'',$(cmd_$(1))))' > $(@D)/.$(@F).tmp; \ rm -f $(depfile); \ mv -f $(@D)/.$(@F).tmp $(@D)/.$(@F).cmd)diff -Nru a/scripts/basic/Makefile b/scripts/basic/Makefile
scripts/basic/Makefile new file +18
-0@@ -0,0 +1,18 @@ +### +# Makefile.basic list the most basic programs used during the build process. +# The programs listed herein is what is needed to do the basic stuff, +# such as splitting .config and fix dependency file. +# This initial step is needed to avoid files to be recompiled +# when kernel configuration changes (which is what happens when +# .config is included by main Makefile. +# --------------------------------------------------------------------------- +# fixdep: Used to generate dependency information during build process +# split-include: Divide all config symbols up in a number of files in +# include/config/... +# docproc: Used in Documentation/docbook + +host-progs := fixdep split-include docproc +always := $(host-progs) + +# fixdep is needed to compile other host programs +$(addprefix $(obj)/,$(filter-out fixdep,$(always))): $(obj)/fixdepdiff -Nru
a/scripts/basic/docproc.cb/scripts/basic/docproc.cscripts/basic/docproc.c new file +387
-0@@ -0,0 +1,387 @@ +/* +* docproc is a simple preprocessor for the template files +* used as placeholders for the kernel internal documentation. +* docproc is used for documentation-frontend and +* dependency-generator. +* The two usages have in common that they require +* some knowledge of the .tmpl syntax, therefore they +* are kept together. +* +* documentation-frontend +* Scans the template file and call kernel-doc for +* all occurrences of ![EIF]file +* Beforehand each referenced file are scanned for +* any exported sympols "EXPORT_SYMBOL()" statements. +* This is used to create proper -function and +* -nofunction arguments in calls to kernel-doc. +* Usage: docproc doc file.tmpl +* +* dependency-generator: +* Scans the template file and list all files +* referenced in a format recognized by make. +* Usage: docproc depend file.tmpl +* Writes dependency information to stdout +* in the following format: +* file.tmpl src.c src2.c +* The filenames are obtained from the following constructs: +* !Efilename +* !Ifilename +* !Dfilename +* !Ffilename +* +*/ + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <ctype.h> +#include <unistd.h> +#include <limits.h> +#include <sys/types.h> +#include <sys/wait.h> + +/* exitstatus is used to keep track of any failing calls to kernel-doc, +* but execution continues. */ +int exitstatus =0 ; + +typedef void DFL (char *); +DFL *defaultline; + +typedef void FILEONLY (char * file); +FILEONLY *internalfunctions; +FILEONLY *externalfunctions; +FILEONLY *symbolsonly; + +typedef void FILELINE (char * file,char * line); +FILELINE * singlefunctions; +FILELINE * entity_system; + +#define MAXLINESZ 2048 +#define MAXFILES 250 +#define KERNELDOCPATH "scripts/" +#define KERNELDOC "kernel-doc" +#define DOCBOOK "-docbook" +#define FUNCTION "-function" +#define NOFUNCTION "-nofunction" + +void usage (void ) +{ +fprintf (stderr,"Usage: docproc {doc|depend} file\n" ); +fprintf (stderr,"Input is read from file.tmpl. Output is sent to stdout\n" ); +fprintf (stderr,"doc: frontend when generating kernel documentation\n" ); +fprintf (stderr,"depend: generate list of files referenced within file\n" ); +} + +/* +* Execute kernel-doc with parameters givin in svec +*/ +void exec_kernel_doc (char **svec) +{ +pid_t pid; +int ret; +/* Make sure output generated so far are flushed */ +fflush (stdout); +switch (pid=fork ()) { +case -1 : +perror ("fork" ); +exit (1 ); +case 0 : + execvp(KERNELDOCPATH KERNELDOC, svec); +perror ("exec " KERNELDOCPATH KERNELDOC ); +exit (1 ); +default : +waitpid (pid, &ret ,0 ); + } +if (WIFEXITED (ret)) + exitstatus |=WEXITSTATUS (ret); +else + exitstatus =0xff ; +} + +/* Types used to create list of all exported symbols in a number of files */ +struct symbols +{ +char *name; +}; + +struct symfile +{ +char *filename; +struct symbols *symbollist; +int symbolcnt; +}; + +struct symfile symfilelist[MAXFILES ]; +int symfilecnt =0 ; + +void add_new_symbol (struct symfile *sym,char * symname) +{ + sym->symbollist = +realloc (sym->symbollist, (sym->symbolcnt +1 ) *sizeof (char *)); + sym->symbollist[sym->symbolcnt++].name =strdup (symname); +} + +/* Add a filename to the list */ +struct symfile *add_new_file (char * filename) +{ + symfilelist[symfilecnt++].filename =strdup (filename); +return &symfilelist[symfilecnt -1 ]; +} +/* Check if file already are present in the list */ +struct symfile *filename_exist (char * filename) +{ +int i; +for (i=0 ; i < symfilecnt; i++) +if (strcmp (symfilelist[i].filename, filename) ==0 ) +return &symfilelist[i]; +return NULL ; +} + +/* +* List all files referenced within the template file. +* Files are separated by tabs. +*/ +void adddep (char * file) {printf ("\t%s" , file); } +void adddep2 (char * file,char * line) { line = line;adddep (file); } +void noaction (char * line) { line = line; } +void noaction2 (char * file,char * line) { file = file; line = line; } + +/* Echo the line without further action */ +void printline (char * line) {printf ("%s" , line); } + +/* +* Find all symbols exported with EXPORT_SYMBOL and EXPORT_SYMBOL_GPL +* in filename. +* All symbols located are stored in symfilelist. +*/ +void find_export_symbols (char * filename) +{ +FILE * fp; +struct symfile *sym; +char line[MAXLINESZ ]; +if (filename_exist (filename) ==NULL ) { + sym =add_new_file (filename); + fp =fopen (filename,"r" ); +if (fp ==NULL ) + { +fprintf (stderr,"docproc: " ); +perror (filename); + } +while (fgets (line,MAXLINESZ , fp)) { +char *p; +char *e; +if (((p =strstr (line,"EXPORT_SYMBOL_GPL" )) !=0 ) || + ((p =strstr (line,"EXPORT_SYMBOL" )) !=0 )) { +/* Skip EXPORT_SYMBOL{_GPL} */ +while (isalnum (*p) || *p =='_' ) + p++; +/* Remove paranteses and additional ws */ +while (isspace (*p)) + p++; +if (*p !='(' ) +continue ;/* Syntax error? */ +else + p++; +while (isspace (*p)) + p++; + e = p; +while (isalnum (*e) || *e =='_' ) + e++; + *e ='\0' ; +add_new_symbol (sym, p); + } + } +fclose (fp); + } +} + +/* +* Document all external or internal functions in a file. +* Call kernel-doc with following parameters: +* kernel-doc -docbook -nofunction function_name1 filename +* function names are obtained from all the the src files +* by find_export_symbols. +* intfunc uses -nofunction +* extfunc uses -function +*/ +void docfunctions (char * filename,char * type) +{ +int i,j; +int symcnt =0 ; +int idx =0 ; +char **vec; + +for (i=0 ; i <= symfilecnt; i++) + symcnt += symfilelist[i].symbolcnt; + vec =malloc ((2 +2 * symcnt +2 ) *sizeof (char *)); +if (vec ==NULL ) { +perror ("docproc: " ); +exit (1 ); + } + vec[idx++] =KERNELDOC ; + vec[idx++] =DOCBOOK ; +for (i=0 ; i < symfilecnt; i++) { +struct symfile * sym = &symfilelist[i]; +for (j=0 ; j < sym->symbolcnt; j++) { + vec[idx++] = type; + vec[idx++] = sym->symbollist[j].name; + } + } + vec[idx++] = filename; + vec[idx] =NULL ; +printf ("<!-- %s -->\n" , filename); +exec_kernel_doc (vec); +fflush (stdout); +free (vec); +} +void intfunc (char * filename) {docfunctions (filename,NOFUNCTION ); } +void extfunc (char * filename) {docfunctions (filename,FUNCTION ); } + +/* +* Document spåecific function(s) in a file. +* Call kernel-doc with the following parameters: +* kernel-doc -docbook -function function1 [-function function2] +*/ +void singfunc (char * filename,char * line) +{ +char *vec[200 ];/* Enough for specific functions */ +int i, idx =0 ; +int startofsym =1 ; + vec[idx++] =KERNELDOC ; + vec[idx++] =DOCBOOK ; + +/* Split line up in individual parameters preceeded by FUNCTION */ +for (i=0 ; line[i]; i++) { +if (isspace (line[i])) { + line[i] ='\0' ; + startofsym =1 ; +continue ; + } +if (startofsym) { + startofsym =0 ; + vec[idx++] =FUNCTION ; + vec[idx++] = &line[i]; + } + } + vec[idx++] = filename; + vec[idx] =NULL ; +exec_kernel_doc (vec); +} + +/* +* Parse file, calling action specific functions for: +* 1) Lines containing !E +* 2) Lines containing !I +* 3) Lines containing !D +* 4) Lines containing !F +* 5) Default lines - lines not matching the above +*/ +void parse_file (FILE *infile) +{ +char line[MAXLINESZ ]; +char * s; +while (fgets (line,MAXLINESZ , infile)) { +if (line[0 ] =='!' ) { + s = line +2 ; +switch (line[1 ]) { +case 'E' : +while (*s && !isspace (*s)) s++; + *s ='\0' ; +externalfunctions (line+2 ); +break ; +case 'I' : +while (*s && !isspace (*s)) s++; + *s ='\0' ; +internalfunctions (line+2 ); +break ; +case 'D' : +while (*s && !isspace (*s)) s++; + *s ='\0' ; +symbolsonly (line+2 ); +break ; +case 'F' : +/* filename */ +while (*s && !isspace (*s)) s++; + *s++ ='\0' ; +/* function names */ +while (isspace (*s)) + s++; +singlefunctions (line +2 , s); +break ; +default : +defaultline (line); + } + } +else { +defaultline (line); + } + } +fflush (stdout); +} + + +int main (int argc,char *argv[]) +{ +FILE * infile; +if (argc !=3 ) { +usage (); +exit (1 ); + } +/* Open file, exit on error */ + infile =fopen (argv[2 ],"r" ); +if (infile ==NULL ) { +fprintf (stderr,"docproc: " ); +perror (argv[2 ]); +exit (2 ); + } + +if (strcmp ("doc" , argv[1 ]) ==0 ) + { +/* Need to do this in two passes. +* First pass is used to collect all symbols exported +* in the various files. +* Second pass generate the documentation. +* This is required because function are declared +* and exported in different files :-(( +*/ +/* Collect symbols */ + defaultline = noaction; + internalfunctions = find_export_symbols; + externalfunctions = find_export_symbols; + symbolsonly = find_export_symbols; + singlefunctions = noaction2; +parse_file (infile); + +/* Rewind to start from beginning of file again */ +fseek (infile,0 ,SEEK_SET ); + defaultline = printline; + internalfunctions = intfunc; + externalfunctions = extfunc; + symbolsonly = printline; + singlefunctions = singfunc; + +parse_file (infile); + } +else if (strcmp ("depend" , argv[1 ]) ==0 ) + { +/* Create first part of dependency chain +* file.tmpl */ +printf ("%s\t" , argv[2 ]); + defaultline = noaction; + internalfunctions = adddep; + externalfunctions = adddep; + symbolsonly = adddep; + singlefunctions = adddep2; +parse_file (infile); +printf ("\n" ); + } +else + { +fprintf (stderr,"Unknown option: %s\n" , argv[1 ]); +exit (1 ); + } +fclose (infile); +fflush (stdout); +return exitstatus; +} +diff -Nru
a/scripts/basic/fixdep.cb/scripts/basic/fixdep.cscripts/basic/fixdep.c new file +381
-0@@ -0,0 +1,381 @@ +/* +* "Optimize" a list of dependencies as spit out by gcc -MD +* for the kernel build +* =========================================================================== +* +* Author Kai Germaschewski +* Copyright 2002 by Kai Germaschewski <[email protected]> +* +* This software may be used and distributed according to the terms +* of the GNU General Public License, incorporated herein by reference. +* +* +* Introduction: +* +* gcc produces a very nice and correct list of dependencies which +* tells make when to remake a file. +* +* To use this list as-is however has the drawback that virtually +* every file in the kernel includes <linux/config.h> which then again +* includes <linux/autoconf.h> +* +* If the user re-runs make *config, linux/autoconf.h will be +* regenerated. make notices that and will rebuild every file which +* includes autoconf.h, i.e. basically all files. This is extremely +* annoying if the user just changed CONFIG_HIS_DRIVER from n to m. +* +* So we play the same trick that "mkdep" played before. We replace +* the dependency on linux/autoconf.h by a dependency on every config +* option which is mentioned in any of the listed prequisites. +* +* To be exact, split-include populates a tree in include/config/, +* e.g. include/config/his/driver.h, which contains the #define/#undef +* for the CONFIG_HIS_DRIVER option. +* +* So if the user changes his CONFIG_HIS_DRIVER option, only the objects +* which depend on "include/linux/config/his/driver.h" will be rebuilt, +* so most likely only his driver ;-) +* +* The idea above dates, by the way, back to Michael E Chastain, AFAIK. +* +* So to get dependencies right, there two issues: +* o if any of the files the compiler read changed, we need to rebuild +* o if the command line given to the compile the file changed, we +* better rebuild as well. +* +* The former is handled by using the -MD output, the later by saving +* the command line used to compile the old object and comparing it +* to the one we would now use. +* +* Again, also this idea is pretty old and has been discussed on +* kbuild-devel a long time ago. I don't have a sensibly working +* internet connection right now, so I rather don't mention names +* without double checking. +* +* This code here has been based partially based on mkdep.c, which +* says the following about its history: +* +* Copyright abandoned, Michael Chastain, <mailto:[email protected]>. +* This is a C version of syncdep.pl by Werner Almesberger. +* +* +* It is invoked as +* +* fixdep <depfile> <target> <cmdline> +* +* and will read the dependency file <depfile> +* +* The transformed dependency snipped is written to stdout. +* +* It first generates a line +* +* cmd_<target> = <cmdline> +* +* and then basically copies the .<target>.d file to stdout, in the +* process filtering out the dependency on linux/autoconf.h and adding +* dependencies on include/config/my/option.h for every +* CONFIG_MY_OPTION encountered in any of the prequisites. +* +* It will also filter out all the dependencies on *.ver. We need +* to make sure that the generated version checksum are globally up +* to date before even starting the recursive build, so it's too late +* at this point anyway. +* +* The algorithm to grep for "CONFIG_..." is bit unusual, but should +* be fast ;-) We don't even try to really parse the header files, but +* merely grep, i.e. if CONFIG_FOO is mentioned in a comment, it will +* be picked up as well. It's not a problem with respect to +* correctness, since that can only give too many dependencies, thus +* we cannot miss a rebuild. Since people tend to not mention totally +* unrelated CONFIG_ options all over the place, it's not an +* efficiency problem either. +* +* (Note: it'd be easy to port over the complete mkdep state machine, +* but I don't think the added complexity is worth it) +*/ + +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/mman.h> +#include <unistd.h> +#include <fcntl.h> +#include <string.h> +#include <stdlib.h> +#include <stdio.h> +#include <limits.h> +#include <ctype.h> +#include <netinet/in.h> + +#define INT_CONF ntohl(0x434f4e46) +#define INT_ONFI ntohl(0x4f4e4649) +#define INT_NFIG ntohl(0x4e464947) +#define INT_FIG_ ntohl(0x4649475f) + +char *target; +char *depfile; +char *cmdline; + +void usage (void ) + +{ +fprintf (stderr,"Usage: fixdep <depfile> <target> <cmdline>\n" ); +exit (1 ); +} + +void print_cmdline (void ) +{ +printf ("cmd_%s := %s\n\n" , target, cmdline); +} + +char * str_config =NULL ; +int size_config =0 ; +int len_config =0 ; + +/* +* Grow the configuration string to a desired length. +* Usually the first growth is plenty. +*/ +void grow_config (int len) +{ +while (len_config + len > size_config) { +if (size_config ==0 ) + size_config =2048 ; + str_config =realloc (str_config, size_config *=2 ); +if (str_config ==NULL ) + {perror ("fixdep:malloc" );exit (1 ); } + } +} + + + +/* +* Lookup a value in the configuration string. +*/ +int is_defined_config (const char * name,int len) +{ +const char * pconfig; +const char * plast = str_config + len_config - len; +for ( pconfig = str_config +1 ; pconfig < plast; pconfig++ ) { +if (pconfig[-1 ] =='\n' + && pconfig[len] =='\n' + && !memcmp (pconfig, name, len)) +return 1 ; + } +return 0 ; +} + +/* +* Add a new value to the configuration string. +*/ +void define_config (const char * name,int len) +{ +grow_config (len +1 ); + +memcpy (str_config+len_config, name, len); + len_config += len; + str_config[len_config++] ='\n' ; +} + +/* +* Clear the set of configuration strings. +*/ +void clear_config (void ) +{ + len_config =0 ; +define_config ("" ,0 ); +} + +/* +* Record the use of a CONFIG_* word. +*/ +void use_config (char *m,int slen) +{ +char s[PATH_MAX ]; +char *p; + +if (is_defined_config (m, slen)) +return ; + +define_config (m, slen); + +memcpy (s, m, slen); s[slen] =0 ; + +for (p = s; p < s + slen; p++) { +if (*p =='_' ) + *p ='/' ; +else + *p =tolower ((unsigned char )*p); + } +printf (" $(wildcard include/config/%s.h) \\\n" , s); +} + +void parse_config_file (char *map,size_t len) +{ +int *end = (int *) (map + len); +/* start at +1, so that p can never be < map */ +int *m = (int *) map +1 ; +char *p, *q; + +for (; m < end; m++) { +if (*m ==INT_CONF ) { p = (char *) m ; goto conf; } +if (*m ==INT_ONFI ) { p = (char *) m-1 ; goto conf; } +if (*m ==INT_NFIG ) { p = (char *) m-2 ; goto conf; } +if (*m ==INT_FIG_ ) { p = (char *) m-3 ; goto conf; } +continue ; + conf: +if (p > map + len -7 ) +continue ; +if (memcmp (p,"CONFIG_" ,7 )) +continue ; +for (q = p +7 ; q < map + len; q++) { +if (!(isalnum (*q) || *q =='_' )) + goto found; + } +continue ; + + found: +use_config (p+7 , q-p-7 ); + } +} + +/* test is s ends in sub */ +int strrcmp (char *s,char *sub) +{ +int slen =strlen (s); +int sublen =strlen (sub); + +if (sublen > slen) +return 1 ; + +return memcmp (s + slen - sublen, sub, sublen); +} + +void do_config_file (char *filename) +{ +struct stat st; +int fd; +void *map; + + fd =open (filename,O_RDONLY ); +if (fd <0 ) { +fprintf (stderr,"fixdep: " ); +perror (filename); +exit (2 ); + } +fstat (fd, &st); +if (st.st_size ==0 ) { +close (fd); +return ; + } + map =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fd,0 ); +if ((long ) map ==-1 ) { +perror ("fixdep: mmap" ); +close (fd); +return ; + } + +parse_config_file (map, st.st_size); + +munmap (map, st.st_size); + +close (fd); +} + +void parse_dep_file (void *map,size_t len) +{ +char *m = map; +char *end = m + len; +char *p; +char s[PATH_MAX ]; + + p =strchr (m,':' ); +if (!p) { +fprintf (stderr,"fixdep: parse error\n" ); +exit (1 ); + } +memcpy (s, m, p-m); s[p-m] =0 ; +printf ("deps_%s := \\\n" , target); + m = p+1 ; + +clear_config (); + +while (m < end) { +while (m < end && (*m ==' ' || *m =='\\' || *m =='\n' )) + m++; + p = m; +while (p < end && *p !=' ' ) p++; +if (p == end) { +do p--;while (!isalnum (*p)); + p++; + } +memcpy (s, m, p-m); s[p-m] =0 ; +if (strrcmp (s,"include/linux/autoconf.h" ) && +strrcmp (s,".ver" )) { +printf (" %s \\\n" , s); +do_config_file (s); + } + m = p +1 ; + } +printf ("\n%s: $(deps_%s)\n\n" , target, target); +printf ("$(deps_%s):\n" , target); +} + +void print_deps (void ) +{ +struct stat st; +int fd; +void *map; + + fd =open (depfile,O_RDONLY ); +if (fd <0 ) { +fprintf (stderr,"fixdep: " ); +perror (depfile); +exit (2 ); + } +fstat (fd, &st); +if (st.st_size ==0 ) { +fprintf (stderr,"fixdep: %s is empty\n" ,depfile); +close (fd); +return ; + } + map =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fd,0 ); +if ((long ) map ==-1 ) { +perror ("fixdep: mmap" ); +close (fd); +return ; + } + +parse_dep_file (map, st.st_size); + +munmap (map, st.st_size); + +close (fd); +} + +void traps (void ) +{ + static char test[] __attribute__((aligned(sizeof(int)))) = "CONF"; + +if (*(int *)test !=INT_CONF ) { +fprintf (stderr,"fixdep: sizeof(int) != 4 or wrong endianess? %#x\n" , + *(int *)test); +exit (2 ); + } +} + +int main (int argc,char *argv[]) +{ +traps (); + +if (argc !=4 ) +usage (); + + depfile = argv[1 ]; + target = argv[2 ]; + cmdline = argv[3 ]; + +print_cmdline (); +print_deps (); + +return 0 ; +}diff -Nru
a/scripts/basic/split-include.cb/scripts/basic/split-include.cscripts/basic/split-include.c new file +226
-0@@ -0,0 +1,226 @@ +/* +* split-include.c +* +* Copyright abandoned, Michael Chastain, <mailto:[email protected]>. +* This is a C version of syncdep.pl by Werner Almesberger. +* +* This program takes autoconf.h as input and outputs a directory full +* of one-line include files, merging onto the old values. +* +* Think of the configuration options as key-value pairs. Then there +* are five cases: +* +* key old value new value action +* +* KEY-1 VALUE-1 VALUE-1 leave file alone +* KEY-2 VALUE-2A VALUE-2B write VALUE-2B into file +* KEY-3 - VALUE-3 write VALUE-3 into file +* KEY-4 VALUE-4 - write an empty file +* KEY-5 (empty) - leave old empty file alone +*/ + +#include <sys/stat.h> +#include <sys/types.h> + +#include <ctype.h> +#include <errno.h> +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> + +#define ERROR_EXIT (strExit) \ + { \ + const int errnoSave = errno; \ + fprintf(stderr, "%s: ", str_my_name); \ + errno = errnoSave; \ + perror((strExit)); \ + exit(1); \ + } + + + +int main (int argc,const char * argv []) +{ +const char * str_my_name; +const char * str_file_autoconf; +const char * str_dir_config; + +FILE * fp_config; +FILE * fp_target; +FILE * fp_find; + +int buffer_size; + +char * line; +char * old_line; +char * list_target; +char * ptarget; + +struct stat stat_buf; + +/* Check arg count. */ +if (argc !=3 ) + { +fprintf (stderr,"%s: wrong number of arguments.\n" , argv[0 ]); +exit (1 ); + } + + str_my_name = argv[0 ]; + str_file_autoconf = argv[1 ]; + str_dir_config = argv[2 ]; + +/* Find a buffer size. */ +if (stat (str_file_autoconf, &stat_buf) !=0 ) +ERROR_EXIT (str_file_autoconf); + buffer_size =2 * stat_buf.st_size +4096 ; + +/* Allocate buffers. */ +if ( (line =malloc (buffer_size)) ==NULL + || (old_line =malloc (buffer_size)) ==NULL + || (list_target =malloc (buffer_size)) ==NULL ) +ERROR_EXIT (str_file_autoconf); + +/* Open autoconfig file. */ +if ((fp_config =fopen (str_file_autoconf,"r" )) ==NULL ) +ERROR_EXIT (str_file_autoconf); + +/* Make output directory if needed. */ +if (stat (str_dir_config, &stat_buf) !=0 ) + { +if (mkdir (str_dir_config,0755 ) !=0 ) +ERROR_EXIT (str_dir_config); + } + +/* Change to output directory. */ +if (chdir (str_dir_config) !=0 ) +ERROR_EXIT (str_dir_config); + +/* Put initial separator into target list. */ + ptarget = list_target; + *ptarget++ ='\n' ; + +/* Read config lines. */ +while (fgets (line, buffer_size, fp_config)) + { +const char * str_config; +int is_same; +int itarget; + +if (line[0 ] !='#' ) +continue ; +if ((str_config =strstr (line,"CONFIG_" )) ==NULL ) +continue ; + +/* Make the output file name. */ + str_config +=sizeof ("CONFIG_" ) -1 ; +for (itarget =0 ; !isspace (str_config[itarget]); itarget++) + { +int c = (unsigned char ) str_config[itarget]; +if (isupper (c)) c =tolower (c); +if (c =='_' ) c ='/' ; + ptarget[itarget] = c; + } + ptarget[itarget++] ='.' ; + ptarget[itarget++] ='h' ; + ptarget[itarget++] ='\0' ; + +/* Check for existing file. */ + is_same =0 ; +if ((fp_target =fopen (ptarget,"r" )) !=NULL ) + { +fgets (old_line, buffer_size, fp_target); +if (fclose (fp_target) !=0 ) +ERROR_EXIT (ptarget); +if (!strcmp (line, old_line)) + is_same =1 ; + } + +if (!is_same) + { +/* Auto-create directories. */ +int islash; +for (islash =0 ; islash < itarget; islash++) + { +if (ptarget[islash] =='/' ) + { + ptarget[islash] ='\0' ; +if (stat (ptarget, &stat_buf) !=0 + &&mkdir (ptarget,0755 ) !=0 ) +ERROR_EXIT ( ptarget ); + ptarget[islash] ='/' ; + } + } + +/* Write the file. */ +if ((fp_target =fopen (ptarget,"w" )) ==NULL ) +ERROR_EXIT (ptarget); +fputs (line, fp_target); +if (ferror (fp_target) ||fclose (fp_target) !=0 ) +ERROR_EXIT (ptarget); + } + +/* Update target list */ + ptarget += itarget; + *(ptarget-1 ) ='\n' ; + } + +/* +* Close autoconfig file. +* Terminate the target list. +*/ +if (fclose (fp_config) !=0 ) +ERROR_EXIT (str_file_autoconf); + *ptarget ='\0' ; + +/* +* Fix up existing files which have no new value. +* This is Case 4 and Case 5. +* +* I re-read the tree and filter it against list_target. +* This is crude. But it avoids data copies. Also, list_target +* is compact and contiguous, so it easily fits into cache. +* +* Notice that list_target contains strings separated by \n, +* with a \n before the first string and after the last. +* fgets gives the incoming names a terminating \n. +* So by having an initial \n, strstr will find exact matches. +*/ + + fp_find =popen ("find * -type f -name \"*.h\" -print" ,"r" ); +if (fp_find ==0 ) +ERROR_EXIT ("find" ); + + line[0 ] ='\n' ; +while (fgets (line+1 , buffer_size, fp_find)) + { +if (strstr (list_target, line) ==NULL ) + { +/* +* This is an old file with no CONFIG_* flag in autoconf.h. +*/ + +/* First strip the \n. */ + line[strlen (line)-1 ] ='\0' ; + +/* Grab size. */ +if (stat (line+1 , &stat_buf) !=0 ) +ERROR_EXIT (line); + +/* If file is not empty, make it empty and give it a fresh date. */ +if (stat_buf.st_size !=0 ) + { +if ((fp_target =fopen (line+1 ,"w" )) ==NULL ) +ERROR_EXIT (line); +if (fclose (fp_target) !=0 ) +ERROR_EXIT (line); + } + } + } + +if (pclose (fp_find) !=0 ) +ERROR_EXIT ("find" ); + +return 0 ; +}diff -Nru
a/scripts/docproc.cb/scripts/docproc.cscripts/docproc.c deleted +0
-387@@ -1,387 +0,0 @@ -/* -* docproc is a simple preprocessor for the template files -* used as placeholders for the kernel internal documentation. -* docproc is used for documentation-frontend and -* dependency-generator. -* The two usages have in common that they require -* some knowledge of the .tmpl syntax, therefore they -* are kept together. -* -* documentation-frontend -* Scans the template file and call kernel-doc for -* all occurrences of ![EIF]file -* Beforehand each referenced file are scanned for -* any exported sympols "EXPORT_SYMBOL()" statements. -* This is used to create proper -function and -* -nofunction arguments in calls to kernel-doc. -* Usage: docproc doc file.tmpl -* -* dependency-generator: -* Scans the template file and list all files -* referenced in a format recognized by make. -* Usage: docproc depend file.tmpl -* Writes dependency information to stdout -* in the following format: -* file.tmpl src.c src2.c -* The filenames are obtained from the following constructs: -* !Efilename -* !Ifilename -* !Dfilename -* !Ffilename -* -*/ - -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <ctype.h> -#include <unistd.h> -#include <limits.h> -#include <sys/types.h> -#include <sys/wait.h> - -/* exitstatus is used to keep track of any failing calls to kernel-doc, -* but execution continues. */ -int exitstatus =0 ; - -typedef void DFL (char *); -DFL *defaultline; - -typedef void FILEONLY (char * file); -FILEONLY *internalfunctions; -FILEONLY *externalfunctions; -FILEONLY *symbolsonly; - -typedef void FILELINE (char * file,char * line); -FILELINE * singlefunctions; -FILELINE * entity_system; - -#define MAXLINESZ 2048 -#define MAXFILES 250 -#define KERNELDOCPATH "scripts/" -#define KERNELDOC "kernel-doc" -#define DOCBOOK "-docbook" -#define FUNCTION "-function" -#define NOFUNCTION "-nofunction" - -void usage (void ) -{ -fprintf (stderr,"Usage: docproc {doc|depend} file\n" ); -fprintf (stderr,"Input is read from file.tmpl. Output is sent to stdout\n" ); -fprintf (stderr,"doc: frontend when generating kernel documentation\n" ); -fprintf (stderr,"depend: generate list of files referenced within file\n" ); -} - -/* -* Execute kernel-doc with parameters givin in svec -*/ -void exec_kernel_doc (char **svec) -{ -pid_t pid; -int ret; -/* Make sure output generated so far are flushed */ -fflush (stdout); -switch (pid=fork ()) { -case -1 : -perror ("fork" ); -exit (1 ); -case 0 : - execvp(KERNELDOCPATH KERNELDOC, svec); -perror ("exec " KERNELDOCPATH KERNELDOC ); -exit (1 ); -default : -waitpid (pid, &ret ,0 ); - } -if (WIFEXITED (ret)) - exitstatus |=WEXITSTATUS (ret); -else - exitstatus =0xff ; -} - -/* Types used to create list of all exported symbols in a number of files */ -struct symbols -{ -char *name; -}; - -struct symfile -{ -char *filename; -struct symbols *symbollist; -int symbolcnt; -}; - -struct symfile symfilelist[MAXFILES ]; -int symfilecnt =0 ; - -void add_new_symbol (struct symfile *sym,char * symname) -{ - sym->symbollist = -realloc (sym->symbollist, (sym->symbolcnt +1 ) *sizeof (char *)); - sym->symbollist[sym->symbolcnt++].name =strdup (symname); -} - -/* Add a filename to the list */ -struct symfile *add_new_file (char * filename) -{ - symfilelist[symfilecnt++].filename =strdup (filename); -return &symfilelist[symfilecnt -1 ]; -} -/* Check if file already are present in the list */ -struct symfile *filename_exist (char * filename) -{ -int i; -for (i=0 ; i < symfilecnt; i++) -if (strcmp (symfilelist[i].filename, filename) ==0 ) -return &symfilelist[i]; -return NULL ; -} - -/* -* List all files referenced within the template file. -* Files are separated by tabs. -*/ -void adddep (char * file) {printf ("\t%s" , file); } -void adddep2 (char * file,char * line) { line = line;adddep (file); } -void noaction (char * line) { line = line; } -void noaction2 (char * file,char * line) { file = file; line = line; } - -/* Echo the line without further action */ -void printline (char * line) {printf ("%s" , line); } - -/* -* Find all symbols exported with EXPORT_SYMBOL and EXPORT_SYMBOL_GPL -* in filename. -* All symbols located are stored in symfilelist. -*/ -void find_export_symbols (char * filename) -{ -FILE * fp; -struct symfile *sym; -char line[MAXLINESZ ]; -if (filename_exist (filename) ==NULL ) { - sym =add_new_file (filename); - fp =fopen (filename,"r" ); -if (fp ==NULL ) - { -fprintf (stderr,"docproc: " ); -perror (filename); - } -while (fgets (line,MAXLINESZ , fp)) { -char *p; -char *e; -if (((p =strstr (line,"EXPORT_SYMBOL_GPL" )) !=0 ) || - ((p =strstr (line,"EXPORT_SYMBOL" )) !=0 )) { -/* Skip EXPORT_SYMBOL{_GPL} */ -while (isalnum (*p) || *p =='_' ) - p++; -/* Remove paranteses and additional ws */ -while (isspace (*p)) - p++; -if (*p !='(' ) -continue ;/* Syntax error? */ -else - p++; -while (isspace (*p)) - p++; - e = p; -while (isalnum (*e) || *e =='_' ) - e++; - *e ='\0' ; -add_new_symbol (sym, p); - } - } -fclose (fp); - } -} - -/* -* Document all external or internal functions in a file. -* Call kernel-doc with following parameters: -* kernel-doc -docbook -nofunction function_name1 filename -* function names are obtained from all the the src files -* by find_export_symbols. -* intfunc uses -nofunction -* extfunc uses -function -*/ -void docfunctions (char * filename,char * type) -{ -int i,j; -int symcnt =0 ; -int idx =0 ; -char **vec; - -for (i=0 ; i <= symfilecnt; i++) - symcnt += symfilelist[i].symbolcnt; - vec =malloc ((2 +2 * symcnt +2 ) *sizeof (char *)); -if (vec ==NULL ) { -perror ("docproc: " ); -exit (1 ); - } - vec[idx++] =KERNELDOC ; - vec[idx++] =DOCBOOK ; -for (i=0 ; i < symfilecnt; i++) { -struct symfile * sym = &symfilelist[i]; -for (j=0 ; j < sym->symbolcnt; j++) { - vec[idx++] = type; - vec[idx++] = sym->symbollist[j].name; - } - } - vec[idx++] = filename; - vec[idx] =NULL ; -printf ("<!-- %s -->\n" , filename); -exec_kernel_doc (vec); -fflush (stdout); -free (vec); -} -void intfunc (char * filename) {docfunctions (filename,NOFUNCTION ); } -void extfunc (char * filename) {docfunctions (filename,FUNCTION ); } - -/* -* Document spåecific function(s) in a file. -* Call kernel-doc with the following parameters: -* kernel-doc -docbook -function function1 [-function function2] -*/ -void singfunc (char * filename,char * line) -{ -char *vec[200 ];/* Enough for specific functions */ -int i, idx =0 ; -int startofsym =1 ; - vec[idx++] =KERNELDOC ; - vec[idx++] =DOCBOOK ; - -/* Split line up in individual parameters preceeded by FUNCTION */ -for (i=0 ; line[i]; i++) { -if (isspace (line[i])) { - line[i] ='\0' ; - startofsym =1 ; -continue ; - } -if (startofsym) { - startofsym =0 ; - vec[idx++] =FUNCTION ; - vec[idx++] = &line[i]; - } - } - vec[idx++] = filename; - vec[idx] =NULL ; -exec_kernel_doc (vec); -} - -/* -* Parse file, calling action specific functions for: -* 1) Lines containing !E -* 2) Lines containing !I -* 3) Lines containing !D -* 4) Lines containing !F -* 5) Default lines - lines not matching the above -*/ -void parse_file (FILE *infile) -{ -char line[MAXLINESZ ]; -char * s; -while (fgets (line,MAXLINESZ , infile)) { -if (line[0 ] =='!' ) { - s = line +2 ; -switch (line[1 ]) { -case 'E' : -while (*s && !isspace (*s)) s++; - *s ='\0' ; -externalfunctions (line+2 ); -break ; -case 'I' : -while (*s && !isspace (*s)) s++; - *s ='\0' ; -internalfunctions (line+2 ); -break ; -case 'D' : -while (*s && !isspace (*s)) s++; - *s ='\0' ; -symbolsonly (line+2 ); -break ; -case 'F' : -/* filename */ -while (*s && !isspace (*s)) s++; - *s++ ='\0' ; -/* function names */ -while (isspace (*s)) - s++; -singlefunctions (line +2 , s); -break ; -default : -defaultline (line); - } - } -else { -defaultline (line); - } - } -fflush (stdout); -} - - -int main (int argc,char *argv[]) -{ -FILE * infile; -if (argc !=3 ) { -usage (); -exit (1 ); - } -/* Open file, exit on error */ - infile =fopen (argv[2 ],"r" ); -if (infile ==NULL ) { -fprintf (stderr,"docproc: " ); -perror (argv[2 ]); -exit (2 ); - } - -if (strcmp ("doc" , argv[1 ]) ==0 ) - { -/* Need to do this in two passes. -* First pass is used to collect all symbols exported -* in the various files. -* Second pass generate the documentation. -* This is required because function are declared -* and exported in different files :-(( -*/ -/* Collect symbols */ - defaultline = noaction; - internalfunctions = find_export_symbols; - externalfunctions = find_export_symbols; - symbolsonly = find_export_symbols; - singlefunctions = noaction2; -parse_file (infile); - -/* Rewind to start from beginning of file again */ -fseek (infile,0 ,SEEK_SET ); - defaultline = printline; - internalfunctions = intfunc; - externalfunctions = extfunc; - symbolsonly = printline; - singlefunctions = singfunc; - -parse_file (infile); - } -else if (strcmp ("depend" , argv[1 ]) ==0 ) - { -/* Create first part of dependency chain -* file.tmpl */ -printf ("%s\t" , argv[2 ]); - defaultline = noaction; - internalfunctions = adddep; - externalfunctions = adddep; - symbolsonly = adddep; - singlefunctions = adddep2; -parse_file (infile); -printf ("\n" ); - } -else - { -fprintf (stderr,"Unknown option: %s\n" , argv[1 ]); -exit (1 ); - } -fclose (infile); -fflush (stdout); -return exitstatus; -} -diff -Nru
a/scripts/fixdep.cb/scripts/fixdep.cscripts/fixdep.c deleted +0
-381@@ -1,381 +0,0 @@ -/* -* "Optimize" a list of dependencies as spit out by gcc -MD -* for the kernel build -* =========================================================================== -* -* Author Kai Germaschewski -* Copyright 2002 by Kai Germaschewski <[email protected]> -* -* This software may be used and distributed according to the terms -* of the GNU General Public License, incorporated herein by reference. -* -* -* Introduction: -* -* gcc produces a very nice and correct list of dependencies which -* tells make when to remake a file. -* -* To use this list as-is however has the drawback that virtually -* every file in the kernel includes <linux/config.h> which then again -* includes <linux/autoconf.h> -* -* If the user re-runs make *config, linux/autoconf.h will be -* regenerated. make notices that and will rebuild every file which -* includes autoconf.h, i.e. basically all files. This is extremely -* annoying if the user just changed CONFIG_HIS_DRIVER from n to m. -* -* So we play the same trick that "mkdep" played before. We replace -* the dependency on linux/autoconf.h by a dependency on every config -* option which is mentioned in any of the listed prequisites. -* -* To be exact, split-include populates a tree in include/config/, -* e.g. include/config/his/driver.h, which contains the #define/#undef -* for the CONFIG_HIS_DRIVER option. -* -* So if the user changes his CONFIG_HIS_DRIVER option, only the objects -* which depend on "include/linux/config/his/driver.h" will be rebuilt, -* so most likely only his driver ;-) -* -* The idea above dates, by the way, back to Michael E Chastain, AFAIK. -* -* So to get dependencies right, there two issues: -* o if any of the files the compiler read changed, we need to rebuild -* o if the command line given to the compile the file changed, we -* better rebuild as well. -* -* The former is handled by using the -MD output, the later by saving -* the command line used to compile the old object and comparing it -* to the one we would now use. -* -* Again, also this idea is pretty old and has been discussed on -* kbuild-devel a long time ago. I don't have a sensibly working -* internet connection right now, so I rather don't mention names -* without double checking. -* -* This code here has been based partially based on mkdep.c, which -* says the following about its history: -* -* Copyright abandoned, Michael Chastain, <mailto:[email protected]>. -* This is a C version of syncdep.pl by Werner Almesberger. -* -* -* It is invoked as -* -* fixdep <depfile> <target> <cmdline> -* -* and will read the dependency file <depfile> -* -* The transformed dependency snipped is written to stdout. -* -* It first generates a line -* -* cmd_<target> = <cmdline> -* -* and then basically copies the .<target>.d file to stdout, in the -* process filtering out the dependency on linux/autoconf.h and adding -* dependencies on include/config/my/option.h for every -* CONFIG_MY_OPTION encountered in any of the prequisites. -* -* It will also filter out all the dependencies on *.ver. We need -* to make sure that the generated version checksum are globally up -* to date before even starting the recursive build, so it's too late -* at this point anyway. -* -* The algorithm to grep for "CONFIG_..." is bit unusual, but should -* be fast ;-) We don't even try to really parse the header files, but -* merely grep, i.e. if CONFIG_FOO is mentioned in a comment, it will -* be picked up as well. It's not a problem with respect to -* correctness, since that can only give too many dependencies, thus -* we cannot miss a rebuild. Since people tend to not mention totally -* unrelated CONFIG_ options all over the place, it's not an -* efficiency problem either. -* -* (Note: it'd be easy to port over the complete mkdep state machine, -* but I don't think the added complexity is worth it) -*/ - -#include <sys/types.h> -#include <sys/stat.h> -#include <sys/mman.h> -#include <unistd.h> -#include <fcntl.h> -#include <string.h> -#include <stdlib.h> -#include <stdio.h> -#include <limits.h> -#include <ctype.h> -#include <netinet/in.h> - -#define INT_CONF ntohl(0x434f4e46) -#define INT_ONFI ntohl(0x4f4e4649) -#define INT_NFIG ntohl(0x4e464947) -#define INT_FIG_ ntohl(0x4649475f) - -char *target; -char *depfile; -char *cmdline; - -void usage (void ) - -{ -fprintf (stderr,"Usage: fixdep <depfile> <target> <cmdline>\n" ); -exit (1 ); -} - -void print_cmdline (void ) -{ -printf ("cmd_%s := %s\n\n" , target, cmdline); -} - -char * str_config =NULL ; -int size_config =0 ; -int len_config =0 ; - -/* -* Grow the configuration string to a desired length. -* Usually the first growth is plenty. -*/ -void grow_config (int len) -{ -while (len_config + len > size_config) { -if (size_config ==0 ) - size_config =2048 ; - str_config =realloc (str_config, size_config *=2 ); -if (str_config ==NULL ) - {perror ("fixdep:malloc" );exit (1 ); } - } -} - - - -/* -* Lookup a value in the configuration string. -*/ -int is_defined_config (const char * name,int len) -{ -const char * pconfig; -const char * plast = str_config + len_config - len; -for ( pconfig = str_config +1 ; pconfig < plast; pconfig++ ) { -if (pconfig[-1 ] =='\n' - && pconfig[len] =='\n' - && !memcmp (pconfig, name, len)) -return 1 ; - } -return 0 ; -} - -/* -* Add a new value to the configuration string. -*/ -void define_config (const char * name,int len) -{ -grow_config (len +1 ); - -memcpy (str_config+len_config, name, len); - len_config += len; - str_config[len_config++] ='\n' ; -} - -/* -* Clear the set of configuration strings. -*/ -void clear_config (void ) -{ - len_config =0 ; -define_config ("" ,0 ); -} - -/* -* Record the use of a CONFIG_* word. -*/ -void use_config (char *m,int slen) -{ -char s[PATH_MAX ]; -char *p; - -if (is_defined_config (m, slen)) -return ; - -define_config (m, slen); - -memcpy (s, m, slen); s[slen] =0 ; - -for (p = s; p < s + slen; p++) { -if (*p =='_' ) - *p ='/' ; -else - *p =tolower ((unsigned char )*p); - } -printf (" $(wildcard include/config/%s.h) \\\n" , s); -} - -void parse_config_file (char *map,size_t len) -{ -int *end = (int *) (map + len); -/* start at +1, so that p can never be < map */ -int *m = (int *) map +1 ; -char *p, *q; - -for (; m < end; m++) { -if (*m ==INT_CONF ) { p = (char *) m ; goto conf; } -if (*m ==INT_ONFI ) { p = (char *) m-1 ; goto conf; } -if (*m ==INT_NFIG ) { p = (char *) m-2 ; goto conf; } -if (*m ==INT_FIG_ ) { p = (char *) m-3 ; goto conf; } -continue ; - conf: -if (p > map + len -7 ) -continue ; -if (memcmp (p,"CONFIG_" ,7 )) -continue ; -for (q = p +7 ; q < map + len; q++) { -if (!(isalnum (*q) || *q =='_' )) - goto found; - } -continue ; - - found: -use_config (p+7 , q-p-7 ); - } -} - -/* test is s ends in sub */ -int strrcmp (char *s,char *sub) -{ -int slen =strlen (s); -int sublen =strlen (sub); - -if (sublen > slen) -return 1 ; - -return memcmp (s + slen - sublen, sub, sublen); -} - -void do_config_file (char *filename) -{ -struct stat st; -int fd; -void *map; - - fd =open (filename,O_RDONLY ); -if (fd <0 ) { -fprintf (stderr,"fixdep: " ); -perror (filename); -exit (2 ); - } -fstat (fd, &st); -if (st.st_size ==0 ) { -close (fd); -return ; - } - map =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fd,0 ); -if ((long ) map ==-1 ) { -perror ("fixdep: mmap" ); -close (fd); -return ; - } - -parse_config_file (map, st.st_size); - -munmap (map, st.st_size); - -close (fd); -} - -void parse_dep_file (void *map,size_t len) -{ -char *m = map; -char *end = m + len; -char *p; -char s[PATH_MAX ]; - - p =strchr (m,':' ); -if (!p) { -fprintf (stderr,"fixdep: parse error\n" ); -exit (1 ); - } -memcpy (s, m, p-m); s[p-m] =0 ; -printf ("deps_%s := \\\n" , target); - m = p+1 ; - -clear_config (); - -while (m < end) { -while (m < end && (*m ==' ' || *m =='\\' || *m =='\n' )) - m++; - p = m; -while (p < end && *p !=' ' ) p++; -if (p == end) { -do p--;while (!isalnum (*p)); - p++; - } -memcpy (s, m, p-m); s[p-m] =0 ; -if (strrcmp (s,"include/linux/autoconf.h" ) && -strrcmp (s,".ver" )) { -printf (" %s \\\n" , s); -do_config_file (s); - } - m = p +1 ; - } -printf ("\n%s: $(deps_%s)\n\n" , target, target); -printf ("$(deps_%s):\n" , target); -} - -void print_deps (void ) -{ -struct stat st; -int fd; -void *map; - - fd =open (depfile,O_RDONLY ); -if (fd <0 ) { -fprintf (stderr,"fixdep: " ); -perror (depfile); -exit (2 ); - } -fstat (fd, &st); -if (st.st_size ==0 ) { -fprintf (stderr,"fixdep: %s is empty\n" ,depfile); -close (fd); -return ; - } - map =mmap (NULL , st.st_size,PROT_READ ,MAP_PRIVATE , fd,0 ); -if ((long ) map ==-1 ) { -perror ("fixdep: mmap" ); -close (fd); -return ; - } - -parse_dep_file (map, st.st_size); - -munmap (map, st.st_size); - -close (fd); -} - -void traps (void ) -{ - static char test[] __attribute__((aligned(sizeof(int)))) = "CONF"; - -if (*(int *)test !=INT_CONF ) { -fprintf (stderr,"fixdep: sizeof(int) != 4 or wrong endianess? %#x\n" , - *(int *)test); -exit (2 ); - } -} - -int main (int argc,char *argv[]) -{ -traps (); - -if (argc !=4 ) -usage (); - - depfile = argv[1 ]; - target = argv[2 ]; - cmdline = argv[3 ]; - -print_cmdline (); -print_deps (); - -return 0 ; -}diff -Nru
a/scripts/split-include.cb/scripts/split-include.cscripts/split-include.c deleted +0
-227@@ -1,226 +0,0 @@ -/* -* split-include.c -* -* Copyright abandoned, Michael Chastain, <mailto:[email protected]>. -* This is a C version of syncdep.pl by Werner Almesberger. -* -* This program takes autoconf.h as input and outputs a directory full -* of one-line include files, merging onto the old values. -* -* Think of the configuration options as key-value pairs. Then there -* are five cases: -* -* key old value new value action -* -* KEY-1 VALUE-1 VALUE-1 leave file alone -* KEY-2 VALUE-2A VALUE-2B write VALUE-2B into file -* KEY-3 - VALUE-3 write VALUE-3 into file -* KEY-4 VALUE-4 - write an empty file -* KEY-5 (empty) - leave old empty file alone -*/ - -#include <sys/stat.h> -#include <sys/types.h> - -#include <ctype.h> -#include <errno.h> -#include <fcntl.h> -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <unistd.h> - -#define ERROR_EXIT (strExit) \ - { \ - const int errnoSave = errno; \ - fprintf(stderr, "%s: ", str_my_name); \ - errno = errnoSave; \ - perror((strExit)); \ - exit(1); \ - } - - - -int main (int argc,const char * argv []) -{ -const char * str_my_name; -const char * str_file_autoconf; -const char * str_dir_config; - -FILE * fp_config; -FILE * fp_target; -FILE * fp_find; - -int buffer_size; - -char * line; -char * old_line; -char * list_target; -char * ptarget; - -struct stat stat_buf; - -/* Check arg count. */ -if (argc !=3 ) - { -fprintf (stderr,"%s: wrong number of arguments.\n" , argv[0 ]); -exit (1 ); - } - - str_my_name = argv[0 ]; - str_file_autoconf = argv[1 ]; - str_dir_config = argv[2 ]; - -/* Find a buffer size. */ -if (stat (str_file_autoconf, &stat_buf) !=0 ) -ERROR_EXIT (str_file_autoconf); - buffer_size =2 * stat_buf.st_size +4096 ; - -/* Allocate buffers. */ -if ( (line =malloc (buffer_size)) ==NULL - || (old_line =malloc (buffer_size)) ==NULL - || (list_target =malloc (buffer_size)) ==NULL ) -ERROR_EXIT (str_file_autoconf); - -/* Open autoconfig file. */ -if ((fp_config =fopen (str_file_autoconf,"r" )) ==NULL ) -ERROR_EXIT (str_file_autoconf); - -/* Make output directory if needed. */ -if (stat (str_dir_config, &stat_buf) !=0 ) - { -if (mkdir (str_dir_config,0755 ) !=0 ) -ERROR_EXIT (str_dir_config); - } - -/* Change to output directory. */ -if (chdir (str_dir_config) !=0 ) -ERROR_EXIT (str_dir_config); - -/* Put initial separator into target list. */ - ptarget = list_target; - *ptarget++ ='\n' ; - -/* Read config lines. */ -while (fgets (line, buffer_size, fp_config)) - { -const char * str_config; -int is_same; -int itarget; - -if (line[0 ] !='#' ) -continue ; -if ((str_config =strstr (line,"CONFIG_" )) ==NULL ) -continue ; - -/* Make the output file name. */ - str_config +=sizeof ("CONFIG_" ) -1 ; -for (itarget =0 ; !isspace (str_config[itarget]); itarget++) - { -int c = (unsigned char ) str_config[itarget]; -if (isupper (c)) c =tolower (c); -if (c =='_' ) c ='/' ; - ptarget[itarget] = c; - } - ptarget[itarget++] ='.' ; - ptarget[itarget++] ='h' ; - ptarget[itarget++] ='\0' ; - -/* Check for existing file. */ - is_same =0 ; -if ((fp_target =fopen (ptarget,"r" )) !=NULL ) - { -fgets (old_line, buffer_size, fp_target); -if (fclose (fp_target) !=0 ) -ERROR_EXIT (ptarget); -if (!strcmp (line, old_line)) - is_same =1 ; - } - -if (!is_same) - { -/* Auto-create directories. */ -int islash; -for (islash =0 ; islash < itarget; islash++) - { -if (ptarget[islash] =='/' ) - { - ptarget[islash] ='\0' ; -if (stat (ptarget, &stat_buf) !=0 - &&mkdir (ptarget,0755 ) !=0 ) -ERROR_EXIT ( ptarget ); - ptarget[islash] ='/' ; - } - } - -/* Write the file. */ -if ((fp_target =fopen (ptarget,"w" )) ==NULL ) -ERROR_EXIT (ptarget); -fputs (line, fp_target); -if (ferror (fp_target) ||fclose (fp_target) !=0 ) -ERROR_EXIT (ptarget); - } - -/* Update target list */ - ptarget += itarget; - *(ptarget-1 ) ='\n' ; - } - -/* -* Close autoconfig file. -* Terminate the target list. -*/ -if (fclose (fp_config) !=0 ) -ERROR_EXIT (str_file_autoconf); - *ptarget ='\0' ; - -/* -* Fix up existing files which have no new value. -* This is Case 4 and Case 5. -* -* I re-read the tree and filter it against list_target. -* This is crude. But it avoids data copies. Also, list_target -* is compact and contiguous, so it easily fits into cache. -* -* Notice that list_target contains strings separated by \n, -* with a \n before the first string and after the last. -* fgets gives the incoming names a terminating \n. -* So by having an initial \n, strstr will find exact matches. -*/ - - fp_find =popen ("find * -type f -name \"*.h\" -print" ,"r" ); -if (fp_find ==0 ) -ERROR_EXIT ("find" ); - - line[0 ] ='\n' ; -while (fgets (line+1 , buffer_size, fp_find)) - { -if (strstr (list_target, line) ==NULL ) - { -/* -* This is an old file with no CONFIG_* flag in autoconf.h. -*/ - -/* First strip the \n. */ - line[strlen (line)-1 ] ='\0' ; - -/* Grab size. */ -if (stat (line+1 , &stat_buf) !=0 ) -ERROR_EXIT (line); - -/* If file is not empty, make it empty and give it a fresh date. */ -if (stat_buf.st_size !=0 ) - { -if ((fp_target =fopen (line+1 ,"w" )) ==NULL ) -ERROR_EXIT (line); -if (fclose (fp_target) !=0 ) -ERROR_EXIT (line); - } - } - } - -if (pclose (fp_find) !=0 ) -ERROR_EXIT ("find" ); - -return 0 ; -} --
Signature · 2 lines
The church is near, but the road is icy. The bar is far, but i will walk carefully. - Russian proverb
Quoted 6 lines · cliff white
Hi Sam,
randy dunlap is on vacation this week,
(and i was last week ) so i am trying to get a resolution to the parallel make problem.
Quoted 7 lines
Tested your patch below and it didn't help us. The patch doesn't apply clean past 2.6.4-mm1.
We now see this exact failure against 2.6.5-rc1 for our 8-way machines, so we have much concern.
Is there another fix around we could try?
I've updated the patch since, and it is now in Linus latest - thanks to Andrew. A good sign you do not have the latest version is the fact that you still refer to fixdep in scripts/ (my bad in the frst version).
Could you either try with Linus latest, or at least take a verbatim copy of the Makefile from there.
I have reports of succes with make -j10 from others.
If it still fails please mail me the output without and with V=1, then I will dig into it.
Sam
Quoted 21 lines · On Tue, 23 Mar 2004 06:41:24 +0100 Sam Ravnborg <[email protected]> wrote:
Quoted 6 lines · cliff white
Hi Sam,
randy dunlap is on vacation this week,
(and i was last week ) so i am trying to get a resolution to the parallel make problem.Quoted 7 lines
Tested your patch below and it didn't help us. The patch doesn't apply clean past 2.6.4-mm1.
We now see this exact failure against 2.6.5-rc1 for our 8-way machines, so we have much concern.
Is there another fix around we could try?
I've updated the patch since, and it is now in Linus latest - thanks to Andrew. A good sign you do not have the latest version is the fact that you still refer to fixdep in
scripts/(my bad in the frst version).
Could you either try with Linus latest, or at least take a verbatim copy of theMakefilefrom there.
Quoted 4 lines
I have reports of succes with make -j10 from others.
If it still fails please mail me the output without and with V=1, then I will dig into it.
Everything's happy on the STP machines.
2.6.5-rc2 and 2.6.5-rc2-mm1 compiled just fine on the 8-ways. Thanks
cliffw
Quoted 8 lines
Sam
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 2 lines
The church is near, but the road is icy. The bar is far, but i will walk carefully. - Russian proverb