Skip to contentSkip to conversation

2.5 include file shakeup.

DISCUSSION Dave Jones 3 participants 4 messages Feb 2, 2002 – Feb 4, 2002

lore
Share & tools
lore

Dave Jones

author
#0 lore raw

after yesterdays cleanup removing sched.h inclusion from fs/, I looked at the dependancy graph for sched.h[1], and noticed that even with the removal of the explicit #include <linux/fs.h>, it was still being sucked in via <linux/capability.h>

Ripping this out meant breakage in various parts of the tree, who until now were relying on xxx including sched.h including fs.h these things are now including fs.h.

The next step is to split up fs.h some more, as some things are including it for trivial bits, but sucking in things like the superblock includes for every fs. I've already started this by moving ERR_PTR and friends into <linux/err.h>

Patch is at ftp://ftp.kernel.org/pub/linux/kernel/people/davej/patches/2.5/misc/include-cleanup-1.diff.gz

Not tested on anything other than x86, and I expect some breakage. In most cases, it should be missing includes of err.h, some others may fs.h (but better would be to include whatever fs.h provides) non-x86 testers/patches are more than welcomed.

Is all this worth it ? Take a look at the updated dependancy graph after the cleanups[2], and I think you'll agree things look much more sensible. We also gain a little speed increase on the compile..

make dep bzImage on a 866MHz Cyrix3 with a fast disk..
real user sys
2.5.3 12m37.110s 11m8.580s 0m47.590s
2.5.3+cleanup 12m8.053s 11m0.670s 0m47.450s

make dep on a quad ppro with a _slow_ disk.

                real       user       sys
2.5.3           2m50.229s  1m51.370s  0m12.640s
2.5.3+cleanup   1m44.634s  1m32.580s  0m10.200s

make -j4 bzImage on the same quad

2.5.3           9m11.167s  31m8.060s  2m20.950s
2.5.3+cleanup   9m8.546s   30m33.020s 2m18.710s

Further compile time decreases should be possible by looking a little harder at various places that are including sched.h, and also when we get to the aforementioned fs.h cleanup. Currently some of the compile fixes introduced in this patch are taking the easy way out, and including fs.h, but this is after all, the first phase of this cleanup.

Comments?

[1] ftp://ftp.kernel.org/pub/linux/kernel/people/davej/misc/schedh-before.ps.gz
[2] ftp://ftp.kernel.org/pub/linux/kernel/people/davej/misc/schedh-after.ps.gz

Signature · 2 lines
Dave Jones.                    http://www.codemonkey.org.uk
SuSE Labs.
#1 lore raw
Quoted 5 lines · Dave Jones

after yesterdays cleanup removing sched.h inclusion from fs/, I looked at the dependancy graph for sched.h[1], and noticed that even with the removal of the explicit #include <linux/fs.h>, it was still being sucked in via <linux/capability.h>

Quoted 8 lines

Ripping this out meant breakage in various parts of the tree, who until now were relying on xxx including sched.h including fs.h these things are now including fs.h.

The next step is to split up fs.h some more, as some things are including it for trivial bits, but sucking in things like the superblock includes for every fs. I've already started this by moving ERR_PTR and friends into <linux/err.h>

Just checking - you realize that getting the super_block includes out of fs.h is easy, right? In fact I already did it in my Unbork fs.h (1..4) set of patches last month, at least I set a pattern using ext2 as an example, which is trivially extended for al filesystems. Now, I'm just waiting for one of two things to happen: Al to decide he's finished mucking around in there and I can submit the patch to Linus, or Al will feel threatened again and submit a similar patch to Linus. Either way we win, because the kernel gets better right? (Except that the second scenerio creates considerably more friction that necessary, as we saw last week.)

Quoted 3 lines

[...]

Is all this worth it ?

You bet it is, you are preaching to the choir.

Quoted 1 line

Take a look at the updated dependancy graph after the cleanups[2],

Oh I know all about it, because I first did a version of this for myself almost a year ago, and the complilation speedup was *remarkable*. That's not even the biggest thing, I just find it much easier to work with and feel better about it when the kernel doesn't doesn't have its thumb tied to its nose.

;-)

Signature · 1 line
Daniel
author
#2 lore raw
In reply to Dave Jones
Quoted 8 lines · Daniel Phillips
Quoted 4 lines

The next step is to split up fs.h some more, as some things are including it for trivial bits, but sucking in things like the superblock includes for every fs. I've already started this by moving ERR_PTR and friends into <linux/err.h>

Just checking - you realize that getting the super_block includes out of fs.h is easy, right? In fact I already did it in my Unbork fs.h (1..4) set of patches last month

Ah yes, I had forgotten about that thread. I'll go look through the archives later today, to make sure I don't duplicate any work already done 8-)

Quoted 5 lines

, at least I set a pattern using ext2 as an example, which
is trivially extended for al filesystems. Now, I'm just waiting for one of two things to happen: Al to decide he's finished mucking around in there and I can submit the patch to Linus, or Al will feel threatened again and submit a similar patch to Linus.

Matthew Wilcox came up with a suggestion of fs.h fathering an inode.c which may not be an entirely bad idea given how many files pull in fs.h just for the inode definition, but don't need any of the fluff that goes along with it.

The patch I posted yesterday still has 1-2 flaws (I introduced two circular dependancies in fs.h Ahem, frighteningly easy to do unless you keep generating the graphs and looking what changed) I'll fix it up, and post a corrected version tonight sometime.

Quoted 3 lines

I just find it much easier to work with and feel
better about it when the kernel doesn't doesn't have its thumb tied to its nose.

Nice choice of words. Quite.

Signature · 2 lines
| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs
#3 lore raw
In reply to Dave Jones
Quoted 13 lines · Dave Jones

sensible. We also gain a little speed increase on the compile..

make dep bzImage on a 866MHz Cyrix3 with a fast disk..
real user sys
2.5.3 12m37.110s 11m8.580s 0m47.590s
2.5.3+cleanup 12m8.053s 11m0.670s 0m47.450s

make dep on a quad ppro with a _slow_ disk.

                real       user       sys
2.5.3           2m50.229s  1m51.370s  0m12.640s
2.5.3+cleanup   1m44.634s  1m32.580s  0m10.200s

make -j4 bzImage on the same quad

Quoted 8 lines
2.5.3           9m11.167s  31m8.060s  2m20.950s
2.5.3+cleanup   9m8.546s   30m33.020s 2m18.710s

Further compile time decreases should be possible by looking a little harder at various places that are including sched.h, and also when we get to the aforementioned fs.h cleanup. Currently some of the compile fixes introduced in this patch are taking the easy way out, and including fs.h, but this is after all, the first phase of this cleanup.

Given amount of (trivially fixable) breakage we _regularly_ see in the form of missing #includes, I want to find a way to stop it once and for all time.

What about generating one rather big include file out of .config first, and including it in each .c file? Aha, device drivers... Ok, what about this:

foo.c (core kernel source)
=====
/* All kernel infrastructure structs, funcs, global vars, macros... */ include <kernel_common.h>
...

ide_viaxxx.c (device driver)
============
/* All kernel infrastructure structs, funcs, global vars, macros... */ include <kernel_common.h>
/* Device specific bits not belonging to kernel infrastructure */ include <driver_ide.h>
...

This can slow down compilation on a box with fast disk and slow CPU, but can _speed up_ comilation if disk is slow and CPU is fast.
Why? Parsing .h file contents (declarations, almost no code) is much faster than .c, feeding unneeded declarations is cheaper that it seems to be.

Note that in future CPU/disk speed gap will only increase.

Surely I can miss a reason why this won't work at all (messed up dependencies?), feel free to enlighten me.
--
vda