Hardwired drivers are going away?
I don't mind how people do it, but there are currently two significant penalties associated with modules on x86. I think other architectures have analogues.
1) The main kernel is contiguous in physical memory and is mapped with large (4 MB) pages. This reduces pressure on the TLB. Modules are loaded in vmalloc memory, which uses small pages, and therefore competes for TLB space. This is a performance penalty, especially as most current machines have undersized TLBs already. (A 64-entry TLB with 4K pages maps 256K at a time. On-chip L2 caches are this large or larger. Thus, as a crude approximation, every L2 miss also causes a TLB miss.)
Also, vmalloc space is limited, and contributes to the "why isn't the kernel seeing all of my 1 GB?" question.
2) Space for module code is allocated in page units. Thus, each module wastes an average of 2K. If I'm going to have dozens of modules loaded, small machines are going to notice.
On a related subject, I'd like a kernel image to consist of a series of concatenated "chunks". The first one is the kernel proper, and then come various modules and initrd image(s). (There may be an "EOF" chunk, or just the absence of a magic number.)
All the boot loader has to do is copy the image into contiguous memory and jump to it. The start of the image (ideally, it'd be PIC) then relocates itself to wherever it wants and starts assembling the pieces.
Whether this is done by a sort of linker before the kernel boots, or if the chunks are all ramdisk components and form a userland that the kernel boots to which assemble the rest of the kernel is not particularly important. I just want the unnecessary bits to get thrown away once we've booted. (Kernel bloat should NOT be designed in.)
That would make assembling a custom boot image simple enough for a fancy boot loader (or fancy tftp server) to do.
Quoted 5 lines
1) The main kernel is contiguous in physical memory and is mapped with large (4 MB) pages. This reduces pressure on the TLB. Modules are
2) Space for module code is allocated in page units. Thus, each module wastes an average of 2K. If I'm going to have dozens of modules loaded, small machines are going to notice.
If at boot time we keep a big chunk of ram free at the kernel end and just load modules one after each other into that space until we get into real paging that problem goes away
From: Alan Cox <[email protected]>
Date: Tue, 15 Jan 2002 12:44:38 +0000 (GMT)
If at boot time we keep a big chunk of ram free at the kernel end and just load modules one after each other into that space until we get into real paging that problem goes away
And we do have module_map/module_unmap interfaces already so it's easy to toy with this.
I've always been meaning to do a "alloc 4MB page at boot, lock into TLB, carve module pages out of that, vmalloc when that runs out" on sparc64.
Quoted 7 lines · David S. Miller
From: Alan Cox <[email protected]>
Date: Tue, 15 Jan 2002 12:44:38 +0000 (GMT)If at boot time we keep a big chunk of ram free at the kernel end and
just load modules one after each other into that space until we get into real paging that problem goes away
Quoted 6 lines
And we do have module_map/module_unmap interfaces already so it's easy to toy with this.
I've always been meaning to do a "alloc 4MB page at boot, lock into TLB, carve module pages out of that, vmalloc when that runs out" on sparc64.
How would this handle modules being unloaded? Are you going to try to pack the space, try to re-use the holes (allowing fragmentation), handle unloading the last module only (high water mark), or just ignore it and basically not allow modules to ever be unloaded?
Just curious...
Rob