[2.6] Most likely to be merged by Halloween... THE LIST

I broke down my status list into 3 categories:
- likely to be merged before the Halloween feature freeze
- likely not to be ready by Halloween
- ongoing work

Do you think the breakdown is realistic?

-- Guillaume

PS: Remember, no more than 15-20 big features get usually
merged in 3 months :-)

-------------------------------------------
Before feature freeze:

o New VM with reverse mappings                    (Rik van Riel)
o Add Linux Security Module (LSM)                 (LSM team)
o Rewrite of the console layer                    (James Simmons)
o New MTRR (Memory Type Range Register) driver    (Patrick Mochel)
o Add support for CPU clock/voltage scaling       (Erik Mouw, Dave Jones, Russell King, Arjan van de Ven)
o Strict address space accounting                 (Alan Cox)
o USB gadget support                              (Stuart Lynne, Greg Kroah-Hartman)
o Add hardware sensors drivers                    (lm_sensors team)
o Serial driver restructure                       (Russell King)
o Add User-Mode Linux (UML)                       (Jeff Dike)
o Direct pagecache <-> BIO disk I/O               (Andrew Morton)
o More complete NetBEUI stack                     (Arnaldo Carvalho de Melo, from Procom donated code)
o Fix device naming issues                        (Patrick Mochel, Greg Kroah-Hartman)

After feature freeze:

o Improved i2o (Intelligent Input/Ouput) layer    (Alan Cox)
o Read-Copy Update Mutual Exclusion               (Dipankar Sarma, Rusty Russell, Andrea Arcangeli, LSE Team)
o New kernel build system (kbuild 2.5)            (Keith Owens)
o PCMCIA Zoom video support                       (Alan Cox)
o Add XFS (A journaling filesystem from SGI)      (XFS team)
o New IO scheduler                                (Jens Axboe)
o Per-mountpoint read-only, union-mounts, unionfs (Al Viro)
o Asynchronous IO (aio) support                   (Ben LaHaise)
o EVMS (Enterprise Volume Management System)      (EVMS team)
o LVM (Logical Volume Manager) v2.0               (LVM team)
o Dynamic Probes                                  (Suparna Bhattacharya, dprobes team)
o Page table sharing                              (Daniel Phillips)
o ext2/ext3 online resize support                 (Andreas Dilger)
o UDF Write support for CD-R/RW (packet writing)  (Jens Axboe, Peter Osterlund)
o Better event logging for enterprise systems     (Larry Kessler, evlog team)
o Full compliance with IPv6                       (Alexey Kuznetzov, Jun Murai, Yoshifuji Hideaki, USAGI team)
o UMSDOS (Unix under MS-DOS) Rewrite              (Al Viro)
o Scalable Statistics Counter                     (Ravikiran Thirumalai)
o Linux Kernel Crash Dumps                        (Matt Robinson, LKCD team)
o Add support for NFS v4                          (NFS v4 team)
o ext2/ext3 large directory support: HTree index  (Daniel Phillips, Christopher Li, Ted Ts'o)
o Zerocopy NFS                                    (Hirokazu Takahashi)
o Remove the 2TB block device limit               (Peter Chubb)
o SCTP (Stream Control Transmission Protocol)     (lksctp team)
o High resolution timers                          (George Anzinger, etc.)
o Overhaul PCMCIA support                         (David Woodhouse, David Hinds)
o Reiserfs v4                                     (Reiserfs team)
o Serial ATA support                              (Andre Hedrick)
o InfiniBand support                              (InfiniBand team)
o New lightweight library (klibc)                 (Greg Kroah-Hartman)
o Replace initrd by initramfs                     (H. Peter Anvin, Al Viro)
o Add thrashing control                           (Rik van Riel)
o Remove all hardwired drivers from kernel        (Alan Cox, etc.)
o Generic parameter/command line interface        (Keith Owens)
o New mount API                                   (Al Viro)

Ongoing work:

o Fix long-held locks for low scheduling latency  (Andrew Morton, Robert Love, etc.)
o Better support of high-end NUMA machines        (NUMA team)
o Remove use of the BKL (Big Kernel Lock)         (Alan Cox, Robert Love, Neil Brown, Dave Hansen, etc.)
o Change all drivers to new driver model          (All maintainers)

Unless I start to get some help with this, or a Zarus or some other USB gadget type device to test with, I don't think this will happen.

I do have a single volunteer, but can always use more :)

thanks,

greg k-h

Just to be a bit clearer (as it seems some people don't fully understand this item), I need people who have Linux running WITHIN a USB gadget, like a Zarus. This does not include the Palm devices, which are USB gadgets, but do not run Linux themselves.

thanks,

greg k-h

So, Greg, you mean generic USB-Device support? instead USB-Host support as is in the main tree.

There is some nice work for this in the ARM-Linux kernel specific to the Strong-ARM chip, but I was able to adapt it to another ARM chip I was working on easily. (Though not finished yet due to a bug in the USB-D controller h/w) It's got the s/w to act like a ethernet device or a pure bit pipe.

Have you looked closely at arch/arm/mach-sa1100/usb* recently? It's not far from being generalized. The Zaurus I believe already uses this work to act like an ethernet USB device, and iPaq folks on handhelds.org have been using it for quite some time.

Thanks,
Shane Nay.

(BTW- Great to hear this is on the agenda for the mainline)

Yep. I'm going to be providing Greg with access to such a system today (hopefully, since I'm getting rather a large number of requests for other things people want doing today...)

Russell King ([email protected])                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Exactly.

Yes, the arm code is nice. But the Zarus uses the code from Lineo, which I have merged into the main tree at:

	bk://linuxusb.bkbits.net/usbd-2.5

I really want to merge the arm and Lineo code together, so people don't have to maintain two different trees which work on the same devices, doing much the same thing. That's the goal at least :)

The code also needs some serious cleanups, which is why I am asking for help from people with one of these devices.

thanks,

greg k-h

Likely.

Unsure, depends on the LSM folks.

In progress of being merged from -dj -> mainline.

in -dj, being pushed next week when I get home.

A few last-minute niggles to work out, and this should go into mainline soon.

Probably a trivial merge once the more important vm changes have happened.

As per Greg's comment.

There are a few folks doing various i2c work outside the lm_sensors scope, which could find its way into the tree if its ready in time. The actual sensors stuff, unknown. There were issues with some laptops, I don't know if they got resolved ?

Hopefully ready in time.

Ready-to-merge.

Andrew ?

Depending on how busy Arnaldo gets I guess. Non-critical.

Yep. For example, how /dev/disk/ or whatever ends up working is still unclear to myself and a few other folks.

Other items you listed in the post-freeze section that really needs doing before the freeze imo:

LVM1 in 2.5 is badly borken, Sistina don't want to fix it, and whilst there is some work getting it 'mostly functional' in my tree, ongoing development of LVM1 is counterproductive when there is a superior solution in the wings. All depends on whether they can pull it off before the freeze of course.

Again, this is likely to happen sooner rather than after the freeze. This will break large amounts of kernel code, and having non-compilable kernels /after/ the freeze is undesired considering we are then in the 'get things stable' zone.

Quite a few of these you listed either should wait until 2.7 or get in before the freeze. Some of them require non-trivial changes to critical areas which we really shouldn't be poking too much in an alleged 'stabilisation series'.

Instead I would rename this section 'non critical features' or 'nice to have' features. If they don't make it in time, there's always 2.7

Dave

| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

May as well make this one more widely known:

We are going to try for this before Halloween, Internal code is being cleaned up an sanitized now, external changes should start getting the same treatment soon and get aired in public for, hopefully short, discussion.

Steve

Kai, what are your intentions w/ kbuild 2.5?

I hope I'm not speaking just for myself when I say I feel it'd be a great pity if it weren't merged.

T.

Here's a list of other things I am hoping to see merged:

shared pagetables
large page support
NUMA aware multipath IO
NUMA aware scheduler extensions
ia32 discontigmem support for NUMA machines
NUMA aware slab allocator
CONFIG_NONLINEAR (in some form, quite possibly a cut down version) shared pagetables
large page support
rcu rtcache
rcu dcache

Im not sure the complexity of shared pagetables is worth it. On ppc64 it will be a real pain to support since we rely on a 1:1 mapping between linux and ppc64 ptes.

Unless its a large gain over using large pages (I doubt that will be the case on sane chips with large TLBs) or conditional per architecture then I think we should avoid it.

I do think we should get large page support in ASAP.

Anton

Thanks Martin.

I am confident that we will have large page support in 2.5 by then ready and tested as described at OLS.

-- Hubertus Franke  ([email protected])

This is starting to get presented for merging right now (see my previous posts to lkml for an example.)

thanks,

greg k-h

A few of the features above, need this feature, so I would move it up to "before feature freeze". Well I hope it happens, that way we can actually get most of the above done :)

And there are patches available for this on Al Viro's site, but I don't know the current state of them.

thanks,

greg k-h

My patch for this, while a bit large, is simple enough it can be merged after the freeze. E.g., if rmap is merged this is just "more rmap work" and sane.

I would bump this down to "After feature freeze..." although I see no reason it cannot go in sooner (I just do not consider it a pre-freeze item in terms of its impact).

Easily 90% of this stuff should _not_ go in after the freeze. It either needs to make it in now or hold its breath until 2.7.

I would much prefer a rush of merges now and less "improper" merges after the freeze. In fact, I do not care much what we do now as long as we treat the freeze as a freeze and work solely to stabilize stuff.

Robert Love

Agreed. What we need is a 2.6 that is relatively easy to stabilise, not one that'll take until 2.6.20 to become stable.

If some of the other features turn out to be needed, they can always be backported from 2.7. In fact, backporting from 2.7 is probably _quicker_ than throwing in too many features just before the feature freeze and then try to stabilise them all at once together...

regards,

Rik

Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Before I start asking what about XXX, is there a list of what major stuff is already considered to be in? I don't see some things on your list, perhaps you regard them as done.

I really hope these do get in!

I fear Keith might go SPC if this had to wait for 2.7

I thought these were all progressing nicely

Definitely want to stabilize these

Hopefully this is close as well

could ease in 2.6.x if not?

This really shouldn't wait for 2.8!

This would help db folks now, and who knows how big a single drive will be before 2.7?

Sure would be nice if it worked on desktops as well as laptops

I sure would like to see documentation improvements on this list! For 2.6 it would be beautiful is no features went into /proc/sys unless they went into the Documentation directory as well.

bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Bit by bit, either parts of Keith's work, or orthogonal ideas are making it in. Whether the big chunks make it by halloween remains to be seen.

All are aiming for halloween, but none (afaik) have put forward anything that can be merged yet. The XFS & LVM folks are still cleaning bits and getting things presentable, whilst Ben is still at work with aio I guess (judging by his relative silence since the summit 8-)

"would be nice".

This has been around for an age, but I haven't seen anything for 2.5 yet. Then again, I dropped off the packet-writing mailing list a long time ago, so I'm not sure how up to date those folks are.

Davem's call I guess. ISTR the USAGI work was a rather large patch which if in Davem's shoes, I'd be rather dubious about taking 'all-in-one'.

Last I saw of this patch it was still against something like 2.4.1, so they have a lot of catch up to do. This fact asides, if it doesn't touch common code, there's no reason it can't go in post-feature freeze in the same way as a driver/additional fs. Depends how much it touches. That said, are there really that many NFSv4 hosts out there that make this a *must have* feature ? Are any other *nix vendors shipping NFSv4 yet?

Agreed, a pretty important feature.

"works for me". Admittedly I've not played with pcmcia much, but it seems at least if you choose the right hardware it works fine.

ObRelated: There was some shouting about a sysctlfs at some point which could clean up a lot of the crap in /proc/sys at the expense of an extra mount. Al ?

Dave.

| Dave Jones.        http://www.codemonkey.org.uk
| SuSE Labs

Hi,

Well, I still see unnecessary recompiles. There's a lot of stuff to do here, I think, which was already done in kbuild-2.5.

Seemingly. On Hawkeye, I sometimes get warnings of that kind (svc:
unknown version (4)). However, I can't tell what kind of system the clients were. I've also heard about this warning from Potsdam and Frankfurt.

I ack here, our arrays are mostly filled up with only some ten gigabytes left.

Regards, Thunder

(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

Patches for 2.5 can be found here:

        http://w1.894.telia.com/~u89404340/patches/packet/2.5/

The most recent patch is for 2.5.25. As far as I know, there are only two remaining problems with the 2.5 patch:

1. Some drives require a "synchronize cache" command to be inserted before a read command if the previous command was a write. This is implemented in the 2.4 patch in a rather ugly way, but not in 2.5.

2. The 2.5 version uses "bio" objects instead of buffer_heads, so the 2.4 functionality to fill in missing parts of packets with data from the buffer cache doesn't work. This affects performance but not correctness.

Jens thinks the cache sync commands should be handled at the level below the packet writing module, in ide-cd.c and sr.c. I agree, but haven't implemented it simply because I haven't figured out how to do it yet.

Regarding the use of cached data for partial packets, Jens suggested using the page cache instead of the buffer cache. I haven't figured out how to do this either.

Peter Osterlund - [email protected]
http://w1.894.telia.com/~u89404340

Btw, there is one more potential problem. A new block major number is allocated for the pktcdvd device. Is this still forbidden? Are there better ways to do this now?

Peter Osterlund - [email protected]
http://w1.894.telia.com/~u89404340

Why a new number? What's wrong with the official 97?

Jens Axboe

It is still using 97. I didn't know it was official until it got into the kernel tree. I think Linus once said he wouldn't accept patches adding device numbers, because he wanted people to think up something better than device numbers.

I was thinking about this old message:

	http://www.uwsg.indiana.edu/hypermail/linux/kernel/0105.1/1042.html

If that's no longer a problem, then fine.

Peter Osterlund - [email protected]
http://w1.894.telia.com/~u89404340

97 was allocated a loong time ago, it predates the above message.

Jens Axboe

Uh, I have seen no patches from David for this? Have I somehow missed this?

I would dearly love to see this functionality get added. I would also like to hear what the implementation plan is. Will the need for /etc/pcmcia go away? Will the new code do a better job of knowing what drivers to load without system configuration files? How will device removal be handled? Will we still need cardctl and cardmgr?

I just went through a process yesterday of trying to get some PCMCIA wireless PC Card configuration. It isn't at all straightforward, even with pcmcia-cs.

Miles

Before the freeze it should be fine, after all if working IDE is optional in a development kernel IPV6 is certainly not a must-have (unless it seriously break IPV4, obviously).

Not that I have used, this was more of a preemptive effort to get it in 2.6 rather than 2.8, since most vendors will be shipping in less than a year if you believe their unofficial comments. And if the benefits are there it would be a good thing to have first, perhaps.

I haven't had it work since about 2.6 on a BP6 with an adaptor, but it did work in 2.4.early. Of course it could be related to the new IDE stuff making compact flash fail, but even building a uni kernel doesn't help. I bought a cheap used laptop just to use as a card reader, but I'd rather not stay that way ;-) Guess for now that's the "right hardware."

Thanks for all the additional feedback.

bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

The list you are looking for is all the items in the merged category in the status list: http://www.kernelnewbies.org/status/

Several of the items marked merged are still being worked on (IDE, etc.), but for those, the bulk of the work should have been done already.

Cheers,

-- Guillaume

My projects:

(1) parallelizing pagefaults
(2) parallelizing page replacement
(3) allocating pte_chains from highmem
(4) deferred coalescing page allocation
(5) removing embedding waitqueue heads / hashing all waitqueues
(6) removing the global tasklist and all iterations over it

Cheers,
Bill

Some comments on the 'EVMS vs LVM2' threads:

I am only petitioning for the driver called 'device-mapper' to be included in the kernel. This is a *much* lower level volume manager than either the EVMS or LVM1 drivers. I am *not* petitioning for EVMS not to be included.

People are getting understandably confused between device-mapper and LVM2:

*) device-mapper is a driver, intended to provide an extensible (via
the definition of new targets) framework capable of support *anything* that volume management applications should want to do.

*) LVM2 is a userland application that uses the device-mapper driver to
provide a set of tools very similar to LVM1. Currently LVM2 is the only userland application that uses this driver, leading people to associate the two far too strongly.

It would be good if other volume managers embrace device-mapper allowing us to work together on the kernel side, and compete in userland. Kernel development takes *far* too much manpower for us to be duplicating work. For example I released the LVM2 vs EVMS snapshot benchmarks in the hope of encouraging EVMS to move over to device-mapper, unfortunately 2 months later a reply is posted stating that they have now developed equivalent (but broken) code :(

Sistina and IBM *are* both competing with their volume managers, but I feel that this competition should be occuring in userland - and certainly is not relevant to this list. For instance EVMS appears to do Volume + FS management whereas LVM2 does just volume management - in no way does device-mapper preclude FS management, yet that is the impression that some of the postings to the list have been giving.

- Joe

Competition has its own benefits.

Supposing both device-mapper and (the kernel part of) EVMS get into the tree, there's nothing stopping you from submitting a patch to make EVMS use device-mapper. If there's already equivalent code in EVMS, that just makes the job easier.

I'm firmly in the 'we need both' camp.

EVMS is a full-bloated^W blown enterprise solution, ready to go with every imaginable bell and whistle. Device-mapper represents the classic Linux minimalist approach. Hopefully, with the two side-by-side in the tree, both will evolve more rapidly.

Daniel

On Mon, 2002-07-22 at 16:22, Daniel Phillips wrote:developed equivalent

So we end up with twice as much code to debug and lots of incompatibilities when people want to switch around. It would be far better if the two sets of userspace code could at least agree on a common kernel interface

If there is something important in only one then that matters. If there are important features in each that are not in the other then that really proves they should merge the projects

If that were a problem, Linux would only have one filesystem.

Oh, absolutely.

I dunno about that. There's more of interest in a subsystem than just what features it has. Relying only on what I've seen in this thread, it would seem natural for EVMS to depend on device-mapper - but why is it necessary to force the issue immediately, beyond hashing out a suitable interface?

Daniel

The device mapper functionality is generic. We only have one VFS, we only need one mapping layer. What cool stuff people build on top of the mapping layer is another matter. If the mapping layer supports all of the requirements its done right, if not its borked and wants fixing first.

Based on reading this thread and some articles referenced and searched, it would appear that EVMS and LVM2 could/should use the same underpining, such as device-mapper. If you have two competing planes they should still land on the same runway. That way other software could be written which did still other things, or the same things in different ways. Having two kinds of lowest level under them seems like more code to maintain than is necessary, and may lead to conflicts if both are in use. I know the EVMS folks say that won't happen, and there's a good reason for having two groups inventing the wheel, but I'd feel better if ther was one wheel and a single API which all *VM* software could use.

I suspect that there's som NIH involved...

bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.