2.5 include file shakeup.
after yesterdays cleanup removing sched.h inclusion from fs/, I looked at the dependancy graph for sched.h[1], and noticed that even with the removal of the explicit #include <linux/fs.h>, it was still being sucked in via <linux/capability.h>
Ripping this out meant breakage in various parts of the tree, who until now were relying on xxx including sched.h including fs.h these things are now including fs.h.
The next step is to split up fs.h some more, as some things are including it for trivial bits, but sucking in things like the superblock includes for every fs. I've already started this by moving ERR_PTR and friends into <linux/err.h>
Patch is at ftp://ftp.kernel.org/pub/linux/kernel/people/davej/patches/2.5/misc/include-cleanup-1.diff.gz
Not tested on anything other than x86, and I expect some breakage. In most cases, it should be missing includes of err.h, some others may fs.h (but better would be to include whatever fs.h provides) non-x86 testers/patches are more than welcomed.
Is all this worth it ? Take a look at the updated dependancy graph after the cleanups[2], and I think you'll agree things look much more sensible. We also gain a little speed increase on the compile..
make dep bzImage on a 866MHz Cyrix3 with a fast disk..
real user sys
2.5.3 12m37.110s 11m8.580s 0m47.590s
2.5.3+cleanup 12m8.053s 11m0.670s 0m47.450s
make dep on a quad ppro with a _slow_ disk.
real user sys 2.5.3 2m50.229s 1m51.370s 0m12.640s 2.5.3+cleanup 1m44.634s 1m32.580s 0m10.200s
make -j4 bzImage on the same quad
2.5.3 9m11.167s 31m8.060s 2m20.950s 2.5.3+cleanup 9m8.546s 30m33.020s 2m18.710s
Further compile time decreases should be possible by looking a little harder at various places that are including sched.h, and also when we get to the aforementioned fs.h cleanup. Currently some of the compile fixes introduced in this patch are taking the easy way out, and including fs.h, but this is after all, the first phase of this cleanup.
Comments?
[1] ftp://ftp.kernel.org/pub/linux/kernel/people/davej/misc/schedh-before.ps.gz
[2] ftp://ftp.kernel.org/pub/linux/kernel/people/davej/misc/schedh-after.ps.gz
Signature · 2 lines
Dave Jones. http://www.codemonkey.org.uk SuSE Labs.
Quoted 5 lines · Dave Jones
after yesterdays cleanup removing
sched.hinclusion fromfs/, I looked at the dependancy graph for sched.h[1], and noticed that even with the removal of the explicit #include <linux/fs.h>, it was still being sucked in via <linux/capability.h>
Quoted 8 lines
Ripping this out meant breakage in various parts of the tree, who until now were relying on xxx including
sched.hincludingfs.hthese things are now includingfs.h.The next step is to split up
fs.hsome more, as some things are including it for trivial bits, but sucking in things like the superblock includes for every fs. I've already started this by movingERR_PTRand friends into <linux/err.h>
Just checking - you realize that getting the super_block includes out of fs.h is easy, right? In fact I already did it in my Unbork fs.h (1..4) set of patches last month, at least I set a pattern using ext2 as an example, which is trivially extended for al filesystems. Now, I'm just waiting for one of two things to happen: Al to decide he's finished mucking around in there and I can submit the patch to Linus, or Al will feel threatened again and submit a similar patch to Linus. Either way we win, because the kernel gets better right? (Except that the second scenerio creates considerably more friction that necessary, as we saw last week.)
Quoted 3 lines
[...]
Is all this worth it ?
You bet it is, you are preaching to the choir.
Quoted 1 line
Take a look at the updated dependancy graph after the cleanups[2],
Oh I know all about it, because I first did a version of this for myself almost a year ago, and the complilation speedup was *remarkable*. That's not even the biggest thing, I just find it much easier to work with and feel better about it when the kernel doesn't doesn't have its thumb tied to its nose.
;-)
Signature · 1 line
Daniel
Quoted 8 lines · Daniel Phillips
Quoted 4 lines
The next step is to split up
fs.hsome more, as some things are including it for trivial bits, but sucking in things like the superblock includes for every fs. I've already started this by movingERR_PTRand friends into <linux/err.h>Just checking - you realize that getting the
super_blockincludes out offs.his easy, right? In fact I already did it in my Unborkfs.h(1..4) set of patches last month
Ah yes, I had forgotten about that thread. I'll go look through the archives later today, to make sure I don't duplicate any work already done 8-)
Quoted 5 lines
, at least I set a pattern using ext2 as an example, which
is trivially extended for al filesystems. Now, I'm just waiting for one of two things to happen: Al to decide he's finished mucking around in there and I can submit the patch to Linus, or Al will feel threatened again and submit a similar patch to Linus.
Matthew Wilcox came up with a suggestion of fs.h fathering an inode.c which may not be an entirely bad idea given how many files pull in fs.h just for the inode definition, but don't need any of the fluff that goes along with it.
The patch I posted yesterday still has 1-2 flaws (I introduced two circular dependancies in fs.h Ahem, frighteningly easy to do unless you keep generating the graphs and looking what changed) I'll fix it up, and post a corrected version tonight sometime.
Quoted 3 lines
I just find it much easier to work with and feel
better about it when the kernel doesn't doesn't have its thumb tied to its nose.
Nice choice of words. Quite.
Signature · 2 lines
| Dave Jones. http://www.codemonkey.org.uk | SuSE Labs
Quoted 13 lines · Dave Jones
sensible. We also gain a little speed increase on the compile..
make dep bzImage on a 866MHz Cyrix3 with a fast disk..
real user sys
2.5.3 12m37.110s 11m8.580s 0m47.590s
2.5.3+cleanup 12m8.053s 11m0.670s 0m47.450smake dep on a quad ppro with a _slow_ disk.
real user sys 2.5.3 2m50.229s 1m51.370s 0m12.640s 2.5.3+cleanup 1m44.634s 1m32.580s 0m10.200smake -j4 bzImage on the same quad
Quoted 8 lines
2.5.3 9m11.167s 31m8.060s 2m20.950s 2.5.3+cleanup 9m8.546s 30m33.020s 2m18.710sFurther compile time decreases should be possible by looking a little harder at various places that are including
sched.h, and also when we get to the aforementionedfs.hcleanup. Currently some of the compile fixes introduced in this patch are taking the easy way out, and includingfs.h, but this is after all, the first phase of this cleanup.
Given amount of (trivially fixable) breakage we _regularly_ see in the form of missing #includes, I want to find a way to stop it once and for all time.
What about generating one rather big include file out of .config first, and including it in each .c file? Aha, device drivers... Ok, what about this:
foo.c (core kernel source)
=====
/* All kernel infrastructure structs, funcs, global vars, macros... */ include <kernel_common.h>
...
ide_viaxxx.c (device driver)
============
/* All kernel infrastructure structs, funcs, global vars, macros... */ include <kernel_common.h>
/* Device specific bits not belonging to kernel infrastructure */ include <driver_ide.h>
...
This can slow down compilation on a box with fast disk and slow CPU, but can _speed up_ comilation if disk is slow and CPU is fast.
Why? Parsing .h file contents (declarations, almost no code) is much faster than .c, feeding unneeded declarations is cheaper that it seems to be.
Note that in future CPU/disk speed gap will only increase.
Surely I can miss a reason why this won't work at all (messed up dependencies?), feel free to enlighten me.
--
vda