Suggestions request for speed-up kernel compilation
Hi All,
Sorry for my bad English.
I'm a kernel newbie and I would be very happy to receive some tips about kernel compilation from experienced developers.
I have recently elaborated some trivial patches. For each one I have done a build test at minimum and I find myself spend a lot of time waiting my Core Duo to accomplish this task. Make is a very smart tool but after a git pull or a git fetch origin/git reset --hard origin it can't help as desired.
Unfortunately I don't have a clustered compilation farm under my control. :-)
Anyone can suggest me how mitigate that issue?
Tips, RTFM, links will be appreciated as long as referred to real used technique.
Thanks in advance and sorry for stupid question.
Ciao,
Alessio
Quoted 13 lines · Alessio Igor Bogani
Hi All,
Sorry for my bad English.
I'm a kernel newbie and I would be very happy to receive some tips about kernel compilation from experienced developers.
I have recently elaborated some trivial patches. For each one I have done a build test at minimum and I find myself spend a lot of time waiting my Core Duo to accomplish this task. Make is a very smart tool but after a git pull or a git fetch origin/git reset --hard origin it can't help as desired.
Quoted 3 lines
Unfortunately I don't have a clustered compilation farm under my control. :-)
Anyone can suggest me how mitigate that issue?
The kernel build system does a good job detecting what to build. So if you touches a .h file or you change a CONFIG_ symbol used by a header file you will see a lot of rebuilds - sometimes more than you expected.
Try to do:
make V=2
to see why a certain file is rebuild.
If you know you only have to build a single file you can use:
make kernel/foo.o
If you want to build a directory - including sub-directories:
make kernel/
If you want to build a specific module use:
make drivers/net/foobar.ko
make help can give you a few more hints.
And try to avoid "make clean" / "make mrproper" - as this deletes all .o files.
Sam
Quoted 9 lines · Alessio Igor Bogani
I'm a kernel newbie and I would be very happy to receive some tips about kernel compilation from experienced developers.
I have recently elaborated some trivial patches. For each one I have done a build test at minimum and I find myself spend a lot of time waiting my Core Duo to accomplish this task. Make is a very smart tool but after a git pull or a git fetch origin/git reset --hard origin it can't help as desired.
Quoted 3 lines
Unfortunately I don't have a clustered compilation farm under my control. :-)
Anyone can suggest me how mitigate that issue?
What helps me with this is using ccache (compiler cache) - it keeps the compile results inside a cache and when compiling the same file again it fetches the result from there.
on debian it is just:
apt-get install ccache
export PATH=/usr/lib/ccache:$PATH
...work on kernel like normal...
c'ya
sven
Signature · 1 line
The lights are fading out, once more...
Quoted 17 lines · Alessio Igor Bogani
Hi All,
Sorry for my bad English.
I'm a kernel newbie and I would be very happy to receive some tips about kernel compilation from experienced developers.
I have recently elaborated some trivial patches. For each one I have done a build test at minimum and I find myself spend a lot of time waiting my Core Duo to accomplish this task. Make is a very smart tool but after a git pull or a git fetch origin/git reset --hard origin it can't help as desired.
Unfortunately I don't have a clustered compilation farm under my control. :-)
Anyone can suggest me how mitigate that issue?
Quoted 7 lines
Tips, RTFM, links will be appreciated as long as referred to real used technique.
Thanks in advance and sorry for stupid question.
Ciao,
Alessio
check you're using enough make jobs for your processor & memory. On my AMDX2 I find 'make -j 8' is quickest for a full rebuild. regards
Richard
Quoted 4 lines · Alessio Igor Bogani
Hi All,
Sorry for my bad English.
Quoted 8 lines
I'm a kernel newbie and I would be very happy to receive some tips about kernel compilation from experienced developers.
I have recently elaborated some trivial patches. For each one I have done a build test at minimum and I find myself spend a lot of time waiting my Core Duo to accomplish this task. Make is a very smart tool but after a git pull or a git fetch origin/git reset --hard origin it can't help as desired.
ccache, for normal times, as suggested in other reply. But if you or mainline or something you've pulled change the headers, there's not much to be done, unfortunately...
Quoted 3 lines · Alessio Igor Bogani
I'm a kernel newbie and I would be very happy to receive some tips about kernel compilation from experienced developers.
Some tricks:
If you just want to test that the file you've been hacking on compiles you can use make dir/path/file.o
Use the minimum config file for your machines,
except for occasional fuller verification builds
Set the right -j make level. You have to experiment what is best.
Not enabling CONFIG_DEBUGINFO speeds up compilation significantly
When you use ccache make sure you have the cache directory on the same partition and enable CCACHE_HARDLINK=1. Alternatively what also works (but is a bit more expensive) is to put the ccache cache on a SSD.
What works very well is to throw some money at it -- compilation scales very well in a cluster. Get the cheapest quad core you can get as an additional machine and use a tool like icecream
(http://en.opensuse.org/Icecream) to distribute the compilation.
You need special wrappers like ftp://firstfloor.org/pub/ak/smallsrc/icecache.c to combine icecream with ccache
I also found using the "gold" linker in latest binutils speeds up builds because especially when you already have parallel compilations the linker is often a bottleneck.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.