Abbott and Costello meet Crunch Time -- Penultimate 2.5 merge candidate list.

This is the next to last posting of this list. (When abbott and costello meet the monster, its time is almost up.) There will be at most one more, tomorrow, and it may just be a repost of this cc'd to Linus. (Those of you waiting for the last minute, this would be it.)

Linus is definitely going to ignore the majority of this list. The best most of these patches can hope for is an explicit rejection, rather than a drop-on-the-floor that might have been because he forgot it or didn't see it. If this list generates explicit rejections, it will have done its job.

I didn't exactly sort the list, but I did move some stuff that's more likely to go in after the freeze to the end.

David Miller: I have no IPV6 tree from you.

Reiser4 people: I still haven't seen a patch for it. (You're in the list because Hans is capable of making puppy eyes through email.)

Alan: I have no 32 bit dev_t patch. And did this mean anything:
http://lwn.net/Articles/11583/

Everybody else, please proofread your entry. I'm not looking to add anything else to the list, but I am looking for last-minute objections.

And so:

================================= Intro ====================================

Linus is back from the Linux Lunacy Cruise. Photos are available here:
http://www.geekcruises.com/Linux_Lunacy_02/pages/

And a few more here:
http://www.lnxfan.org/

The following features aim to be ready for submission to Linus by Monday, October 28th (that's today, guys), to be considered for inclusion (in 2.5.45) before the feature freeze on Thursday, October 31 (halloween).

This list is just pending features trying to get in before feature freeze. It's primarily for features that need more testing, or might otherwise get forgotten in the rush. If you want to know what's already gone in, or what's being worked on for the next development cycle, or self-contained things that might be merged during the stable series, check out:
http://kernelnewbies.org/status

Thanks to Rusty Russell and Guillaume Boissiere, whose respective 2.5 merge candidate lists have been ruthlessly strip-mined in the process of assembling this. And to everybody who's emailed stuff.

And now, in no particular order:

============================ Pending features: =============================

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

1) Andrew Morton's -mm tree. (Andrew Morton, editor.)

Andrew Morton's -mm tree collates several other projects, including:

The ext2/ext3 Extended Attributes and Access Control Lists patch from Ted Tso and Andreas Gruenbacher (ext23-*.patch), Page Table Sharing from Daniel Phillips and Dave McCracken (shpte-ng.patch), a bunch of huge page upgrades from Bill Irwin (hugetlb*.patch), the orlov allocator, Ingo's generic nonlinear mappings...

Stuff. Lots of stuff.

You can get Andrew Morton's MM tree from the following URL, including a broken-out patches directory and a description file. (The latest version as of this writing is -mm5.)

http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.44

Issues: Did Ed Tomlinson's page table bug get fixed?

http://lists.insecure.org/lists/linux-kernel/2002/Oct/7147.html

----------------------------------------------------------------------------

2) Device mapper for Logical Volume Manager (LVM2) (LVM2 team) (in -ac)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536883428443&w=2

Download:
http://people.sistina.com/~thornber/patches/2.5-stable/

Home page:
http://www.sistina.com/products_lvm.htm

Note: this is in the 2.5-ac tree, available at:
http://www.kernel.org/pub/linux/kernel/people/alan/

----------------------------------------------------------------------------

3) EVMS (Enterprise Volume Management System) (IBM, Contact: Kevin Corry)

Fighting with LVM2 for a place in the tree, a bigger solution to a bigger set of problems:

Home page:
http://sourceforge.net/projects/evms

Home page:
http://evms.sourceforge.net

Download:
http://evms.sourceforge.net/patches/

Some related discussions:
http://marc.theaimsgroup.com/?t=103359686900003&r=1&w=2
http://marc.theaimsgroup.com/?t=103439913000001&r=1&w=2
http://marc.theaimsgroup.com/?w=2&r=1&s=%5Bpatch%5D+evms+core&q=t

----------------------------------------------------------------------------

4) New kernel configuration system (Roman Zippel)

Announcement:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/9043.html

Download:
http://www.xs4all.nl/~zippel/lc/

Linus has actually looked fairly favorably on this one so far:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/3250.html

And an AOL for it:

http://lists.insecure.org/lists/linux-kernel/2002/Oct/8255.html

----------------------------------------------------------------------------

5) Linux Trace Toolkit (LTT) (Karim Yaghmour)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7016.html

Download:
http://opersys.com/ftp/pub/LTT/ExtraPatches/patch-ltt-linux-2.5.44-vanilla-021026-2.2.bz2

User tools:
http://opersys.com/ftp/pub/LTT/TraceToolkit-0.9.6pre2.tgz

----------------------------------------------------------------------------

6) Kernel Probes (IBM, contact: Vamsi Krishna S)

Kprobes announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528410215211&w=2

Download Base Kprobes Patch:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528425615302&q=raw

KProbes->DProbes patches:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528454215523&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528454015520&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528485415813&q=raw

Download gzipped tarball patches from official IBM site:
http://www-124.ibm.com/linux/patches/?project_id=141

The DProbes Home Page:
http://oss.software.ibm.com/developerworks/opensource/linux/projects/dprobes

A good explanation of the difference between kprobes, dprobes, and kernel hooks:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103532874900445&w=2

And a clarification: just kprobes is being submitted for 2.5.45, (and optionally some basic dprobes support,) but not the whole of dprobes:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536827928012&w=2

----------------------------------------------------------------------------

7) Posix support and High Resolution timers (George Anzinger)

George has a patch that provides posix support, and on top of that a patch to provide high resolution timers. He talks about it here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103562196700924&w=2

The project's home page is here:
http://high-res-timers.sourceforge.net/

Downloadable patches may be found here:
http://sourceforge.net/project/showfiles.php?group_id=20460&release_id=118345

And descriptions for each patch (in the order they should be applied) are available here (although in this case the archive truncates the patches themselves, use the download link above):
http://marc.theaimsgroup.com/?l=linux-kernel&m=103553654329827&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103557676007653&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103557677207693&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103558349714128&w=2

Linus had concerns with this one a while back:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/3463.html

----------------------------------------------------------------------------

8) Posix timers alternate implementation (Jim Houston)

Jim Houston has an alternate patch to provide posix support (but not high resolution timers on top of it, yet).

http://marc.theaimsgroup.com/?l=linux-kernel&m=103549000027416&q=raw

----------------------------------------------------------------------------

9) Linux Kernel Crash Dumps (Matt Robinson, LKCD team)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103553563728914&w=2
And again:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536576625905&w=2

Download:
http://lkcd.sourceforge.net/download/latest/

----------------------------------------------------------------------------

10) Rewrite of the console layer (James Simmons)

Announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103487329526903&w=2

Home page:
http://linuxconsole.sourceforge.net/

Downloadable patch:
http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz

Bitkeeper tree:
bk://fbdev.bkbits.net/fbdev-2.5

----------------------------------------------------------------------------

11) IPv6 upgrades and crypto API (Yoshifuji Hideyaki)

The Usagi ipv6 upgrades have been available for a while, and their author would like to see them in 2.5:

README:
ftp://ftp.linux-ipv6.org/pub/usagi/patch/ipsec/README.IPSEC

Downloadable patch here:
ftp://ftp.linux-ipv6.org/pub/usagi/patch/ipsec/ipsec-2.5.43-ALL-03.patch.gz

Dave Miller is doing a major ipv6 layer rewrite, but no patch has been sent to the list yet. Ironically, James Morris has a Crypto API patch on top of Dave Miller's tree:

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103559983324080&w=2

Download:
http://samba.org/~jamesm/crypto/

----------------------------------------------------------------------------

12) MMU-less processor support (Greg Ungerer)

Most recent announcement (with links):
http://marc.theaimsgroup.com/?l=linux-kernel&m=103578189421588&w=2

A version of this is in the 2.5-ac tree. An announcement of a patch against that is here:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103578338922236&w=2

----------------------------------------------------------------------------

13) sys_epoll (I.E. /dev/poll) (Davide Libenzi)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103542994232004&w=2

Homepage:
http://www.xmailserver.org/linux-patches/nio-improve.html

Download:
http://www.xmailserver.org/linux-patches/sys_epoll-2.5.44-last.diff

Linus participated repeatedly in a thread on this one too, expressing concerns which (hopefully) have been addressed. See:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6428.html

----------------------------------------------------------------------------

14) CD Recording/sgio patches (Jens Axboe)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8060.html

Download patch:
http://www.kernel.org/pub/linux/kernel/people/axboe/patches/v2.5/2.5.44/sgio-15.bz2

This should be in Alan Cox's tree as of 2.4.44-ac4.

----------------------------------------------------------------------------

15) In-kernel module loader (Rusty Russell.)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6214.html

Download patch:
http://www.kernel.org/pub/linux/kernel/people/rusty/patches/module-x86-18-10-2002.2.5.43.diff.gz

----------------------------------------------------------------------------

16) Unlimited groups patch (Tim Hockin.)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761319825&w=2

Download patch set from:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524717119443&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761819834&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761619831&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103524761519829&q=raw

----------------------------------------------------------------------------

17) Initramfs (Al Viro)

Way back when, Al said:
http://www.cs.helsinki.fi/linux/linux-kernel/2001-30/0110.html

Download (most recent patch so far):
ftp://ftp.math.psu.edu/pub/viro/N0-initramfs-C40

And Linus recently made happy noises about the idea:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/1110.html

----------------------------------------------------------------------------

18) Kernel Hooks (IBM contact: Vamsi Krishna S.)

Website:
http://www-124.ibm.com/linux/projects/kernelhooks/

Download site:
http://www-124.ibm.com/linux/patches/?patch_id=595

Posted patch:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103364774926440&w=2

----------------------------------------------------------------------------

19) NMI request/release interface (Corey Minyard)

He says:

...

There was a lot of back and forth, resulting in the latest patch (version 8):
http://home.attbi.com/~minyard/linux-nmi-v8.diff

----------------------------------------------------------------------------

20) DriverFS Topology (Matthew Dobson)

Announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103523702710396&w=2

Patches:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540707113401&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540757613962&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540758013984&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540757513957&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103540757813966&q=raw

----------------------------------------------------------------------------

21) Advanced TCA Disk Hotswap (Steven Dake)

Announcement of most recent patch, with links:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103558466315221&w=2

Steven's comments:

----------------------------------------------------------------------------

22) Mobile IPV6 (contact: Antti Tuominen)

Antti Tuominen says:

----------------------------------------------------------------------------

23) SCSI multi-path I/O (Patrick Mansfield)

Announcement with URLs:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8736.html

) VFS Intent Lookup Patch (Peter Braam)

Download:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103552797823568&q=raw

----------------------------------------------------------------------------

24) NUMA Scheduler Upgrade

Erich Focht and Michael Hohnbaum have two different NUMA scheduler patches.

Michael has a stripped down NUMA scheduler, which he says was created because the full Node Affine NUMA Scheduler didn't look like it would be ready for 2.5. He talks about it here, with links to patches:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103548635122591&w=2

Meanwhile, Erich Focht says the full Node Affine Numa Scheduler is indeed ready for 2.5, and already in use at customer sites. He makes his case here, with links to patches, home page, LWN review, etc:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103549657202782&w=2

Here's Erich's scheduler's home page:
http://home.arcor.de/efocht/sched/

The most current version of the patches are downloadable from:

http://home.arcor.de/efocht/sched/01-numa_sched_core-2.5.44-10a.patch
http://home.arcor.de/efocht/sched/02-numa_sched_ilb-2.5.44-10.patch

Martin J. Bligh has been testing both NUMA schedulers, and wandering back and forth in his endorsement. At first he was leaning towards Erich's patch, now he seems to be leaning towards Michael's.

That thread is here:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8904.html

----------------------------------------------------------------------------

25) ptrace over fork (Daniel Jacobowitz)

At the last possible second, this was submitted:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103574480632057&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103574480232051&q=raw
http://marc.theaimsgroup.com/?l=linux-kernel&m=103574511932225&q=raw

----------------------------------------------------------------------------

26) Kexec, launch new linux kernel from Linux (Eric W. Biederman)

Announcement with links:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6584.html

And this thread is just too brazen not to include:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7952.html

Most recent status announcement:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8879.html

----------------------------------------------------------------------------

27) Nanosecond support in stat.

Discussion thread:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8983.html

Download:
ftp://ftp.firstfloor.org/pub/ak/v2.5/nsec-2.5.44-2.bz2

----------------------------------------------------------------------------

28) Digital Video Broadcasting Layer (LinuxTV team)

Home page:
http://www.linuxtv.org:81/dvb/

Download:
http://www.linuxtv.org:81/download/dvb/

This is also in Alan Cox's 2.5 tree.

----------------------------------------------------------------------------

29) Reiser4.

I don't have a patch yet, but Hans Reiser is very insistent that this will be ready by halloween. (VERY insistent.) I'll let him speak for himself:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/8793.html

And again (promises, promises):
http://lists.insecure.org/lists/linux-kernel/2002/Oct/9082.html

Still no patch at the time of this writing, though. In theory it should show up here:
http://namesys.com/download.html

In the meantime, all I can find on Reiser4 is some kind of hybrid marketing brochure/design document thing:
http://www.reiserfs.org/v4/v4.html

http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Hi Rob,

I have a couple of modifications to the kernel probes entry to clarify kprobes vs dprobes issues.

Call these KProbes->DProbes support patches

Please modify this to:

The kprobes home page:
http://www-124.ibm.com/linux/projects/kprobes

Thank you,
Vamsi.

Vamsi Krishna S.
Linux Technology Center,
IBM Software Lab, Bangalore.
Ph: +91 80 5044959
Internet: [email protected]

The kprobes homepage says:

Extensions to kprobes

kprobes has been developed from the full Dynamic Probes patch. It includes the essential mechanism to allow probes to exist in kernel space. The RPN Language Interpreter, Watchpoints and User-space probes extensions, which are part of the Dynamic Probes Package, will be available as add-on patches from the website.

At the moment, we are proposing for kprobes inclusion into the kernel which includes the following four patches:
* kprobes base:
Provides the interface described above
* debug register management:
[snip]
* kwatch points:
[snip]
* user space probes:

The first paragraph seems to say that only the base patch is being submitted and the watchpoint and user-space extensions can be retrieved from the site.

But then it goes on to say that you are proposing those for inclusion also. I'm confused and I've been using your patches. Also, that first paragraph mentions "add-on" patches while all along I thought your intention was to have enough of dprobes in the kernel so that patching wasn't necessary.

Skip

Hi Skip,

That is not the intention, I will reword that first paragraph on the website to clarify this.

Yes, our intention is to have enough of infrastructure in the kernel to do something like dprobes as an external module without patching the kernel.

dprobes provides kernel / user space probes and kernel space watchpoints along with an RPN interpreter and communication with the user space in the form a char device. Now, here is what various bits of kprobes patches do:

1. kprobes - base patch
Enables implementing a part of dprobes (kernel space probes) without further patching the kernel.

2. debug register management patch
3. kwatch points patch
Enables implementing another part of dprobes (kernel space watchpoints) without further patching the kernel.

4. user space probes patch
Enables implementing another part of dprobes (user space probes) without further patching the kernel.

Another point to note is that kprobes should be seen as an independent facility, a sort of infrastructure on top of which more comprehensive tools such as dprobes could be built. kprobes infrastructure could potentially be used in other tools such as kdb/kgdb too.

We would like all four patches to be in the kernel. However, the base patch has been on the list for a few months, has been looked at and commented upon by other kernel developers including Rusty and Linus. So, we strongly hope for its inclusion. Once kprobes is in, other patches (notably the user space probes patch) could be considered a straight forward enhancement to kprobes and may be included even after the freeze. OTOH if Linus agrees to include all four patches, that will be great :-)

Thanks for your comments, hope this clarifies the issues.

Vamsi Krishna S.
Linux Technology Center,
IBM Software Lab, Bangalore.
Ph: +91 80 5044959
Internet: [email protected]

A concise and very useful summary of upcoming/potential features, not only for you kernel hackers but also for people using linux and trying to keep half an eye on whats brewing...

I wish something like this was available all the time and not just in the run-up to a freeze

Andrew

While not 100% accurate, kernelnewbies.org has done a very good job of tracking changes and pending changes during the 2.5 development life-cycle. I'm sure with another pair of eyes, they might be able to fully cover every potential development in a kernel's lifecycle.

*SIGH* This makes me nostalgic for Jim Pick's kernelnotes.org, a shame he didn't have time to maintain it anymore.

Cheers,
Nicholas

I got bored a few days back, and hacked up a quick list of do's and don'ts for potential 2.5 testers to use. (In a cunning ploy hoping to get more people hammering on 2.5
post freeze to get bugs the exposure they deserve).

A first draft is at http://www.codemonkey.org.uk/post-halloween-2.5.txt

Dave

| Dave Jones.        http://www.codemonkey.org.uk

On the subject of testing; I use Gentoo with the 2.5.* kernels. In the light of a recent coment by Alan Cox regarding linux from scratch, is there much point in my posting bug reports?

On re-read my question smells of troll. Please treat it as the straight forward question it was intended to be. (For the record, I run RedHat servers as well, but do most of my testing with gentoo workstations)

Its always worth posting a bug report. Its rare that something appears that is unique to some setup and therefore looks suspiciously like a tool problem.

Thanks.

Ordinarily there's Guillaume's 2.5 status list
(http://kernelnewbies.org/status), and you might be able to bug Rusty into doing something like this on a more regular basis, since he seems inclined.

Rob

http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Knock off initramfs off...I'll be posting something that supercedes it (IMO) within the next 72 hours.

Aside from providing full untar suppprt, it DRAMATICALLY cleans up do_mounts.c and moves all the initrd code that was there to initrd.c. Infact I pretty much entirly rewrote initrd, so it makes sense. (purging prehistoric junk, etc.)

Dave

lol

untar - cpio is better.
initrd - 99% moved out of the kernel
do_mounts - moved out of the kernel completely initramfs - should be ready for Linus in the next day or so.

None of that junk -- and a whole lot more -- needs to be in the kernel at all.

Jeff

CPIO is commonly used and supported by NO ONE. (rpm, whoppee) Kernels even come tar'ed. KISS....

Great...you just killed the high level embedded linux market, and the ability to play boot games from GRUB. (Network, etc) Initrd is a good **OPTION* to have to fall back on...

And he's willing to completely purge initrd and do_mounts NOW???

Fire away with the 100K+ bloated POS. I'm backwards compatible, could easily add 'linked kernel image' support, and only increase the current code by 20K.

Do you have any serious sysadmin, clustering, or emebedded system IMPLEMENTATION experience?

Dave

The time is now 22:48 (Totalitarian)  -  http://www.ccops.org/

Please don't get personal, or you'll end up in peoples kill files.

ARM is basically embedded today, and I support initramfs. I don't believe your "embedded system" argument holds any water. Yes, it is a different way of doing things, but it can (and does here) support initrd images.

Russell King ([email protected])                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

It's not personal. He made a wild 'No one needs any of this[initrd]' statement. Is he coming from 'this looks like a good idea'(coder) or 'I can prove this is a good idea through experience examples.' (systems engineer)

I can prove to you (for one thing) that when the shit hits the fan (disaster recovery) initrd is a good option to have.

High level embedded systems...low level systems could care less.

The point to this is:

Linking an image into the kernel LOOKS nice, but I would sure not want to deploy it because it becomes an adminstation nightmare. Even with initrd, a single file can be a problem...this is why my patch supports extracting multiple tar.gz archives....to maintain configuration modularity.

Having an image in the kernel is not a bad thing...making it the only option is.

Dave

The time is now 22:48 (Totalitarian)  -  http://www.ccops.org/

Irrelevant to this problem. cpio unpack code is smaller, and its format allows easy and painless concatenation.

Correct -- and after the initramfs merge, initrd behavior will be completely unchanged.

Nothing is being purged. Things are being moved to userspace, making the kernel smaller and less bloated. The kernel's behavior to the end user is 100% unchanged.

initramfs decreases the kernel size by a load, thank you very much.

Further, any initrd solution is bloated -- you are using a ram disk and disk-based filesystem, the sum of which equates to ramfs -- with additional wasted memory for filesystem and ramdisk overhead.

Jeff

Both formats are simple. But cpio is simpler.

Umm. No. See initramfs below. initrds have always been an evil abomination. They do things terribly wrong and eliminating them will _save_ space in embedded linux system. When busybox is compiled without the special-case initrd workarounds, it does get a bit smaller...

Perhaps you have misunderstood. initramfs is simply an initrd infrastructure that is done right. You don't need to use klibc in your initramfs if you don't want to. Its just a piece of normal unhosed-up userspace. Populate it as you see fit.

Yes he has plenty of experience. He also has good taste. IMHO the embedded world (as well as everyone else) wants initramfs -- it is a major improvement.

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

I guess I'm part of the `embedded world,' and I don't want _either_ because they _both use RAM_!

[Well, OK, actually it'd be nice to have something like initramfs + some other sort of fetch-the-bits-directly-from-ROM FS which I could mix-n-match; anyway initramfs has got to be better than initrd...]

-Miles

Love is a snowmobile racing across the tundra.  Suddenly it flips over,
pinning you underneath.  At night the ice weasels come.  --Nietzsche

hehe :)

It should be pretty easy to populate initramfs from ROM...

Jeff

Actually what I was trying to say was that often I don't want to copy from ROM to RAM, I just want to have file reads get the bits directly from ROM (to avoid using, um, RAM).

Currently I do this by using a romfs filesystem stored in an MD device which is reading from ROM.

-Miles

[|nurgle|]  ddt- demonic? so quake will have an evil kinda setting? one that 
            will  make every christian in the world foamm at the mouth? 
[iddt]      nurg, that's the goal 

Yep, that was my assumption.

If your ROM is directly addressable, i.e. not read over a single-bit bus or anything, it should be doable. I'm not saying that initramfs will do this out of the box :) but going from initramfs to "initromfs" should not be a huge leap...

However, that said, things also depend on what proggies you are running in your initramfs. You may be running code that only runs at startup when the kernel boots, in which case the best space utilization would be to uncompress a compressed image out of ROM to RAM, use it to bootstrap, and then free [unlink] all the initramfs files that are no longer needed.

Jeff

Do you mean by putting the `internal' format that initramfs normally uses in RAM, in ROM, and skip the initial decompression step?

Or do you mean have it somehow avoid copying the data areas of the cpio stream (i.e. store pointers from the tree-in-ram to the actual data blocks in ROM).

I guess the latter sounds cleaner... it would also have the advantage that you could have a tree with the bulk of data in ROM, but which allowed new files to be written (which would be stored in RAM).

Hmmm...sounds very intersting...

-Miles

I'd rather be consing.

Now _that_ would be extremely useful!

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Well, Linux wants file data in pages, so you would need to be able to get your kernel to point to page-aligned ROM regions. Otherwise you are stuck with the same issues as fs/romfs or fs/cramfs or most other Linux filesystems -- you gotta copy the data into a RAM page before the Linux VFS will look at it[1]. For the initramfs cpio image itself, it can either piggyback inside the kernel image, or be loaded separately via bootloader, from ROM, or some other magic means. Unpacking [what is essentially] pointers into the ROM would probably involve custom userland code in an initramfs which mounts and populates a ROM-based filesystem.

So one way or another you can have all of _your own_ data in ROM, but I don't want to paint too rosy a picture -- if you want to be smarter than fs/romfs is now, it will take some additional work. And of course the ramfs-based rootfs will still be needed as the underlying writable fs.

Jeff

[1] or you can create your own file_operations and eliminate this restriction... with a bunch of ROM-specific custom code that totally bypasses the pagecache

Typical embedded initrds do fairly disgusting tricks to "work around" the limitations of themselves. These tricks are solved cleanly by initramfs, but I'll guess that the reason embedded people will complain is because it is different, and embedded people don't like to unlearn old tricks.

Here's two things that initramfs does that there is no way in hell an initrd can ever do:

- once you've finished with various stuff, you can remove it and thereby free up the space that file was occupying for use by anything without having to wait for the whole filesystem to become unused.

- there's no need to mount a ramfs filesystem, or a blockdev /dev/ram ext2fs-formatted filesystem for /tmp, /etc or /var/run (etc) since / is already a ramfs filesystem, thereby removing:
+ extra symlinks for writable files in these directories
+ extra mount points (with associated kernel structures)
+ ext2fs
+ rd

Russell King ([email protected])                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

An implementation flaw I think I pointed out to Al a year ago. You can not set a size limit on your root othewise. That's why I mount a '/dev/tmpfs/'.

My patch will cometh later today. I need some sleep right now.

Dave

The time is now 22:48 (Totalitarian)  -  http://www.ccops.org/

I imagine so. But that still leaves everything in RAM. On a system with just 1 or 2 MB of ram (I have run Linux on such things :-) there really isn't much point in trying to use any sortof ramfs. Its just not going to work. For that type of application you really want something like JFFS, JFFS2 and whatnot living in ROM/flash. But I'm sure someone could find a reason for a *ramfs to be useful, even then.... :)

-Erik

--
Erik B. Andersen http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

It depends on what you're talking about... if it's just the do_mount.c and mount-root-filesystem code, that code gets unlinked and removed from RAM completely after the kernel booting completes.

If it's other filesystem data you want hanging around after boot completes, check out my reply to Miles... it's doable.

Jeff

Erik

#1 I'll be reviewing initramfs and adding loading image from the kernel support. I don't deny it's a godo thig to have.

#2 My main bitch at jeff was he said if initramfs goes in initrd comes out. initrd shodul not come out.

My patch is the best of both because

My patch supports this (well it can be in about 5 lines of code)

Right now for backwards compatiblity I have a initrd_from_floppy option (which replaces load_ramdisk=). I was debating changing that to initrd_from= and then you can spec /dev/fd0, /dev/fd1, or in your case a memory device linux understandings.

But I mostly dimissed it because that is the job of the boot loader....but you bring up a good point where it could be used.

The time is now 22:48 (Totalitarian)  -  http://www.ccops.org/

So what are the advantages of initrd over initramfs? Is it just historical compatibility, or are is there more?

-Miles

Come now, if we were really planning to harm you, would we be waiting here, 
 beside the path, in the very darkest part of the forest?

I didn't intend to get into a crazy discussion like this now...

After I release my patch later today, and people see how it works, then we can discuss this is a meaningful manner.

Until then, I'll end further comments to this thread.

Dave

The time is now 22:48 (Totalitarian)  -  http://www.ccops.org/

What part of "kernel's behavior is 100% unchanged" did you not understand?

Stop spreading FUD.

You said this where?

You DID Say:

Excuse me if I take this to mean something different then:
"kernel's behavior is 100% unchanged"

Message-ID: <[email protected]>
Exact quote: "The kernel's behavior to the end user is 100% unchanged."

"To:" header: [email protected]

You appear to be unaware of early userspace. Moving code out of the kernel does _not_ mean eliminating that code completely. I was hoping it was obvious that it is impossible to eliminate the tasks that do_mounts.c performs -- otherwise a Linux system would never have a root filesystem mounted.

Being unaware of early userspace implies that you are not familiar with initramfs. Which implies you wish you merge your own code in place of something you do not understand.

Jeff

My appologies, I thought you meant do_mounts.c was being yanked entirly if initramfs goes in.

Actually it's very possible to eliminate it....require an initramfs image to have scripts and tools (mount and pivotroot) to do it all and mount that as the root. (An interesting but pretty bad idea IMO) This is what I thought you meant.

I'll accept your apology after you see how nicely I cleaned up do_mounts.c (It's 11K now) However I did not know the 'rootfs' was called 'early userspace'. (Or was my above assumption correct?)

Jeff, be nice. I'm trying very hard myself...and if you knew me, you'd know it's a difficult task. : >

Dave

Apology accepted -- and note my apology further down below ;-)

That's what is going to happen. The initramfs image is completely removed from RAM after all this occurs, combined with moving code out of the kernel to early userspace, is how the overall kernel size shrinks.

Looking at a kernel cleanliness perspective, mounting root, unpacking initrd images, etc. get more and more complex -- with one simply copying userspace code into the kernel to get the job done. In order to do NFS root for example, the kernel has an internal BOOTP and DHCP client, to snag an IP. This mess simply does not belong in the kernel. Never did.

The absolutely smallest possible code is - unpack a cpio archive onto a ramfs.

If you want to support initrd --in kernel-- you must add in much more code: ram disk device, and filesystem of your choice (minix/ext2/whatever). _Any_ initrd code in the kernel is quite simply too much. It doesn't matter how much you clean up the do_mounts.c code, "zero bytes" will always be smaller :)

lol

As promised above, I do apologize for jumping all over you. :) I should have much more tact and approach posts with a more educational perspective.

Further, if you are patient with _me_, I think I can explain how initramfs can (a) keep your existing initrd images working just fine or (b) help you shrink your startup images even more.

Jeff

On Wednesday 30 October 2002 3:51, Erik Andersen wrote:

Erik,

untar runs about 5K...same as 'un-cpio'. No differece there. But not from userland. Tar is used en masse, cpio isn't. It's the only reason to use tar over cpio...I feel it's a good one.

Erik

#1 I'll be reviewing initramfs and adding loading images from the kernel support. I don't deny it's a good thing to have.

#2 My main complaint is Jeff said if initramfs goes in initrd comes out. initrd should not come out. Let me clarify: the abilty of the bootloader to load images/archives for the kernel to extract should not come out.

My patch is the best of both because, it re-writes initrd properly within a sane framework. (Not to mention I scrubed the hell out of do_mounts.)

If you want to get rid of all the backwards compatible stuff (IE identifing and loading raw images to /dev/ram0, pivoting to /initrd) that's fine with me. The code is layed out now so I can litterally cut it out 10K of that junk in 30 seconds. Better yet I can ifdef it for the poor souls that still need it.

Dave

Wrong. un-cpio is obviously smaller. Just look at the generated assembly... on any platform.

IOW you'd rather bloat the kernel because tarballs are popular...

There is no need to add anything.

No need for this, initramfs means that initrd and do_mounts are moved out of the kernel.

Or better yet use initramfs, where it simply doesn't exist in the kernel image at all :)

Jeff

Red hat uses cpio all over the place. RPM is cpio based, and the drivers in a Red Hat boot disk or driver disk are in a cpio file as well (and their cdrom boot process uses the same code, so that's also cpio). This almost certainly means Mandrake uses the same stuff, and between the two of them we are over 50% of both the Linux installed base, new sales, and new installs. (Take your pick of what you want to measure.)

Yeah, cpio is a pain and change to use, but so is tar. You're just used to it. To get the behavior you want creating a tarball, your option list is probably something like "tar cvjfpC tarball.tbz dirname .". Hands up everybody who thinks cvjfpC is intuitive? Yes you could instead do "cd dirname; tar cvp * | bzip2 > tarball.tbz" if that strikes you as more newbie-friendly. (This is assuming you know that that "v" goes to stderr rather than standard out when it detects that stdout has been redirected, but then you had to know to use "." rather than "*" for obvious reasons. :)

And of course this is assuming you're using gnu tar (which still doesn't officially support bzip by the way, j support was a patch last I checked). The older versions wanted a dash before the options list. (Try the version of tar in busybox sometime...)

By that logic, people should stick with windows. :)

The objection is "people shouldn't have to learn a new tool when upgrading to a new kernel". A tool that has been around since the 1970's, I'm might add...

Rob

http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Hi!

I am asking once more for help with IRQ setup for Cardbus controller on ATI U1/Ali 1535+ chipset. This time with more exact information. This is what I get:

lspci -v -s 00:0a.0
00:0a.0 CardBus bridge: O2 Micro, Inc.: Unknown device 6972

        Subsystem: Hewlett-Packard Company: Unknown device 0024
        Flags: bus master, slow devsel, latency 32, IRQ 5
        Memory at 80000000 (32-bit, non-prefetchable) [size=4K]
        Bus: primary=00, secondary=02, subordinate=05, sec-latency=32
        Memory window 0: 81000000-81100000
        I/O window 0: 00003000-0000307f
        I/O window 1: 00000000-00000003
        16-bit legacy interface ports at 0001

Which is consistent with Windows IRQ 5

But dump_pirq gives:
Interrupt routing table found at address 0xfdf10:
Version 1.0, size 0x00d0
Interrupt router is device 00:07.0
PCI exclusive interrupt mask: 0x0000 []
Compatible router: vendor 0x10b9 device 0x1533
....(skipped)

Device 00:0a.0 (slot 0): CardBus bridge
INTA: link 0x06, irq mask 0x00a0 [5,7]

Device 00:12.0 (slot 0): Ethernet controller
INTA: link 0x02, irq mask 0x0880 [7,11]

Device 00:0c.0 (slot 0):
INTA: link 0x03, irq mask 0x0658 [3,4,6,9,10]

Interrupt router at 00:07.0: AcerLabs Aladdin M1533 PCI-to-ISA bridge
INT1 (link 1): irq 10
INT2 (link 2): irq 11
INT3 (link 3): unrouted
INT4 (link 4): unrouted
INT5 (link 5): unrouted
INT6 (link 6): irq 11
INT7 (link 7): irq 3
INT8 (link 8): irq 5
Serial IRQ: [enabled] [continuous] [frame=21] [pulse=12]

So INT6 instead of 5 gets 11 but it is not shown in lspci.

Could somebody, please, tell me what and where should be fixed (I guess in pci-irq.c) to get it correct.

BR,

Jacek

The reason for using cpio is that while the cpio command was clearly engineered to give "find" some competition the file format it uses is actually quite sane, while the tar file format is a bit crufty, and the standards compliant version of it has some irritating limitations

We only care about the file format for the purposes of an initrd

Alan

Somebody who strogly dislikes cpio could just write wrapper accepting tar-style options. Or add a --cpio option to GNU tar, that switches to using the cpio format. One could even try to auto-detect the format when reading :-)

- Werner (hates cpio, but not enough)

  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/

Well, FWIW, "pax" deprecates both cpio and tar.

http://www.opengroup.org/onlinepubs/007904975/utilities/pax.html

In theory pax turns cpio and tar into shell scripts.

Jeff

I thought shell archives went out in the 1980's for security reasons...

Rob

http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Well, pax is cute, particular the name, but it's just the internal engine, with yet another incompatible interface :-( Of course, a tar-to-cpio wrapper could use pax as an alternative to cpio ...

- Werner

  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/

I'm sorry but if you can't work cpio you probably shouldnt be hacking kernels anyway. If you can work it and have to deal with cow-orkers who can't then write them a nice drag and drop cpio builder.

They all work with initramfs

I do and I don't see the problem. The one reason you want to keep initrd around is ROM and there are better ways of doing "initromfs"