Safety of -j N when building kernels?

Ok, stupid question of the moment-

I always read about the kernel compilation benchmarks people run on the ultra-snazzy new machines, but do people actually run the kernels thus generated?

I have visions of a process being backgrounded to generate some files, and not completing before the one of the old files gets linked into the kernel (because not all files were listed as dependencies, for example).

So are the kernel's current Makefiles really SMP safe -- can one really run multiple jobs when building Linux kernels? Any horror stories, or am I just paranoid?

Thanks
-Eric

--------------------------------------------
 Eric H. Weigle   CCS-1, RADIANT team
 [email protected]     Los Alamos National Lab
 (505) 665-4937   http://home.lanl.gov/ehw/
--------------------------------------------

I usually do
make menuconfig
make -j3 dep
make -j3 bzImage modules
make -j3 bzlilo modules_install
where I separate the compile and install part. But, this depends on what you are compiling. I usually have problem with Compaq and Hamradio stuffs, so I have them commented out.

William Park, Open Geometry Consulting, <[email protected]>
8-CPU Cluster, Hosting, NAS, Linux, LaTeX, python, vim, mutt, tin

I think it's safe... I ran a make -j once (dual pIII 933MHz, 1Gb RAM). The load average went to 200+! In the end, the kernel was fine and booted all right... I don't do that anymore because it takes longer building with a sane number in 'make -j N'...

--------------------------------------------------
Fernando Korndorfer - [email protected]
Quatro Informatica Ltda.
Novo Hamburgo - RS/Brasil
--------------------------------------------------

The only major issue I've seen is the build may fail if you run out
of file handles, or other resources. The build will fail with an "unable to fork" error. When I was at VA Linux I often compiled kernel for use with a make -j 16, or -j 8. I seem to remember having to play with ulimit, and /proc/?/file-max to get enough file handles.

PS- You should also consider logging the output of your compile to a file. As your other jobs will continue for sometime before the make fails. Often preventing you from easily finding the actual compile error.

of course you should only really need to log the stderr or redirect stderr to somewhere you can see it.

make dep is not parallel safe. On some architectures make -jN dep will generate corrupt data, especially with module symbol versions. You might get away with it.

It is not safe on any architecture to make -jN *config dep bzImage modules in one command, you need three separate commands for make -jN *config, make -jN dep and make -jN bzImage modules. The install step must also be a separate command.

Building bzImage and modules should be parallel safe, as long as your machine can take the cpu load and has enough file descriptors. Sometimes people do not understand makefiles so the odd driver may not build correctly. aic7xxx around 2.4.19-pre5 was not parallel safe.

Of course, in kbuild 2.5, all of this is parallel safe. make -j *config install works fine.