July 10, 2002

Check out http://www.kernelnewbies.org/status/ for the latest kernel status update.

Many link changes this week to reflect new URLs for several projects. Also removed some URLs that were not very useful or pointing to patches through the Bitkeeper Web interface that kept changing (I think the Bitmover team is working on a fix - I'll put them back when that's done).

Finally, on the cleanup side, I am planning on removing the following entries that have been submitted a while back and don't look like there are going to be accepted for inclusion anytime soon:
- Better event logging for enterprise systems
- Linux booting ELF images
- First pass at LinuxBIOS support
- Build option for Linux Trace Toolkit (LTT)
- Scalable CPU bitmasks

Also on the planned deletion list:
- Add thrashing control
- Generic parameter/command line interface

As usual, feedback welcome!
Enjoy,

-- Guillaume

----------------------------------
Linux Kernel 2.5 Status - July 10th, 2002
(Latest kernel release is 2.5.25)

Features:

Merged

o in 2.5.1+   Rewrite of the block IO (bio) layer             (Jens Axboe)
o in 2.5.2    Initial support for USB 2.0                     (David Brownell, Greg 
Kroah-Hartman, etc.)
o in 2.5.2    Per-process namespaces, late-boot cleanups      (Al Viro, Manfred 
Spraul)
o in 2.5.2+   New scheduler for improved scalability          (Ingo Molnar)
o in 2.5.2+   New kernel device structure (kdev_t)            (Linus Torvalds, etc.)
o in 2.5.3    IDE layer update                                (Andre Hedrick)
o in 2.5.3    Support reiserfs external journal               (Reiserfs team)
o in 2.5.3    Generic ACL (Access Control List) support       (Nathan Scott)
o in 2.5.3    PnP BIOS driver                                 (Alan Cox, Thomas 
Hood, Dave Jones, etc.)
o in 2.5.3+   New driver model & unified device tree          (Patrick Mochel)
o in 2.5.4    Add preempt kernel option                       (Robert Love, 
MontaVista team)
o in 2.5.4    Support for Next Generation POSIX Threading     (NGPT team)
o in 2.5.4+   Porting all input devices over to input API     (Vojtech Pavlik, 
James Simmons)
o in 2.5.5    Add ALSA (Advanced Linux Sound Architecture)    (ALSA team)
o in 2.5.5    Pagetables in highmem support                   (Ingo Molnar, Arjan 

van de Ven)
o in 2.5.5 New architecture: AMD 64-bit (x86-64) (Andi Kleen, x86-64
Linux team)
o in 2.5.5 New architecture: PowerPC 64-bit (ppc64) (Anton Blanchard,
ppc64 team)

o in 2.5.5+   IDE subsystem rewrite                           (Martin Dalecki)
o in 2.5.6    Add JFS (Journaling FileSystem from IBM)        (JFS team)
o in 2.5.6    per_cpu infrastructure                          (Rusty Russell)
o in 2.5.6    HDLC (High-level Data Link Control) update      (Krzysztof Halasa)
o in 2.5.6    smbfs Unicode and large file support            (Urban Widmark) 
o in 2.5.7    New driver API for Wireless Extensions          (Jean Tourrilhes)
o in 2.5.7    Video for Linux (V4L) redesign                  (Gerd Knorr)
o in 2.5.7    Futexes (Fast Lightweight Userspace Semaphores) (Rusty Russell, etc.)
o in 2.5.7+   NAPI network interrupt mitigation               (Jamal Hadi Salim, 
Robert Olsson, Alexey Kuznetsov)
o in 2.5.7+   ACPI (Advanced Configuration & Power Interface) (Andy Grover, ACPI 
team)
o in 2.5.8    Syscall interface for CPU task affinity         (Robert Love)
o in 2.5.8    Radix-tree pagecache                            (Momchil Velikov, 
Christoph Hellwig)
o in 2.5.8+   Delayed disk block allocation                   (Andrew Morton)
o in 2.5.9    Smarter IRQ balancing                           (Ingo Molnar)
o in 2.5.11   Replace old NTFS driver with NTFS TNG driver    (Anton Altaparmakov)
o in 2.5.11   Fast walk dcache                                (Hanna Linder)
o in 2.5.11+  Rewrite of the framebuffer layer                (James Simmons)
o in 2.5.12+  Rewrite of the buffer layer                     (Andrew Morton)
o in 2.5.14   Support for IDE TCQ (Tagged Command Queueing)   (Jens Axboe)
o in 2.5.14   Bluetooth support (no longer experimental!)     (Maxim Krasnyansky, 
Bluetooth team)
o in 2.5.17   New quota system supporting plugins             (Jan Kara)
o in 2.5.17+  Move ISDN4Linux to CAPI based interface         (Kai Germaschewski, 
ISDN4Linux team)
o in 2.5.18   Software suspend (to disk & RAM)                (Pavel Machek)
o in 2.5.23   More complete IEEE 802.2 stack                  (Arnaldo, Jay 

Schullist, from Procom donated code)
o in 2.5.23+ Hotplug CPU support (Rusty Russell)

o in -dj      Rewrite of the console layer                    (James Simmons)
o in -dj      New MTRR (Memory Type Range Register) driver    (Patrick Mochel)
o in -dj      Add support for CPU clock/voltage scaling       (Erik Mouw, Dave 
Jones, Russell King, Arjan van de Ven)
o in -ac      Strict address space accounting                 (Alan Cox)
o in -ac      PCMCIA Zoom video support                       (Alan Cox)
o in -ac      More complete NetBEUI stack                     (Arnaldo Carvalho de 

Melo, from Procom donated code)
o in -ac Improved i2o (Intelligent Input/Ouput) layer (Alan Cox)

o Ready Better event logging for enterprise systems (Larry Kessler, evlog
team)

o Ready       Linux booting ELF images                        (Eric Biederman)
o Ready       First pass at LinuxBIOS support                 (Eric Biederman)
o Ready       Build option for Linux Trace Toolkit (LTT)      (Karim Yaghmour)
o Ready       New kernel build system (kbuild 2.5)            (Keith Owens)
o Ready       Read-Copy Update Mutual Exclusion               (Dipankar Sarma, 
Rusty Russell, Andrea Arcangeli, LSE Team)
o Ready       USB gadget support                              (Stuart Lynne, Greg 
Kroah-Hartman)
o Ready       Scalable CPU bitmasks                           (Russ Weight)
o Ready       Add hardware sensors drivers                    (lm_sensors team)

o Beta        Serial driver restructure                       (Russell King)
o Beta        New IO scheduler                                (Jens Axboe)
o Beta        Add XFS (A journaling filesystem from SGI)      (XFS team)
o Beta        New VM with reverse mappings                    (Rik van Riel)
o Beta        Fix long-held locks for low scheduling latency  (Andrew Morton, 
Robert Love, etc.)
o Beta        Add Linux Security Module (LSM)                 (LSM team)
o Beta        Per-mountpoint read-only, union-mounts, unionfs (Al Viro)
o Beta        EVMS (Enterprise Volume Management System)      (EVMS team)
o Beta        LVM (Logical Volume Manager) v2.0               (LVM team)
o Beta        Dynamic Probes                                  (Suparna 
Bhattacharya, dprobes team)
o Beta        Page table sharing                              (Daniel Phillips)
o Beta        ext2/ext3 online resize support                 (Andreas Dilger)
o Beta        Add User-Mode Linux (UML)                       (Jeff Dike)
o Beta        UDF Write support for CD-R/RW (packet writing)  (Jens Axboe, Peter 
Osterlund)
o Beta        Asynchronous IO (aio) support                   (Ben LaHaise)
o Beta        Direct pagecache <-> BIO disk I/O               (Andrew Morton)

o Alpha       Better support of high-end NUMA machines        (NUMA team)
o Alpha       Full compliance with IPv6                       (Alexey Kuznetzov, 
Jun Murai, Yoshifuji Hideaki, USAGI team)
o Alpha       UMSDOS (Unix under MS-DOS) Rewrite              (Al Viro)
o Alpha       Scalable Statistics Counter                     (Ravikiran Thirumalai)
o Alpha       Linux Kernel Crash Dumps                        (Matt Robinson, LKCD 
team)
o Alpha       Add support for NFS v4                          (NFS v4 team)
o Alpha       ext2/ext3 large directory support: HTree index  (Daniel Phillips, 
Christopher Li, Ted Ts'o)
o Alpha       Remove use of the BKL (Big Kernel Lock)         (Alan Cox, Robert 
Love, Neil Brown, Dave Hansen, etc.)
o Alpha       Zerocopy NFS                                    (Hirokazu Takahashi)
o Alpha       Change all drivers to new driver model          (All maintainers)
o Alpha       Remove the 2TB block device limit               (Peter Chubb)
o Alpha       SCTP (Stream Control Transmission Protocol)     (lksctp team)

o Started Overhaul PCMCIA support (David Woodhouse,
David Hinds)

o Started     Reiserfs v4                                     (Reiserfs team)
o Started     Serial ATA support                              (Andre Hedrick)
o Started     InfiniBand support                              (InfiniBand team)
* Started     Fix device naming issues                        (Patrick Mochel, Greg 

Kroah-Hartman)

o Draft #2    New lightweight library (klibc)                 (Greg Kroah-Hartman)
o Draft #3    Replace initrd by initramfs                     (H. Peter Anvin, Al 
Viro)
o Planning    Add thrashing control                           (Rik van Riel)
o Planning    Remove all hardwired drivers from kernel        (Alan Cox, etc.)
o Planning    Generic parameter/command line interface        (Keith Owens)
o Planning    New mount API                                   (Al Viro)

Cleanups:

Merged

o in 2.5.3    Break Configure.help into multiple files        (Linus Torvalds)
o in 2.5.3    Untangle sched.h & fs.h include dependancies    (Dave Jones, Roman 

Zippel)
o in 2.5.4 Per network protocol slabcache & sock.h (Arnaldo Carvalho de
Melo)
o in 2.5.4 Per filesystem slabcache & fs.h (Daniel Phillips,
Jeff Garzik, Al Viro)

o in 2.5.6    Killing kdev_t for block devices                (Al Viro)
o in 2.5.18+  ->getattr() ->setattr() ->permission() changes  (Al Viro)
o in 2.5.21   Split up x86 setup.c into managable pieces      (Patrick Mochel)
o in 2.5.23+  Major MD tool (RAID 5) cleanup                  (Neil Brown)

o Ready Switch to ->get_super() for file_system_type (Al Viro)

o Beta        file.h and INIT_TASK                            (Benjamin LaHaise)
o Beta        Proper UFS fixes, ext2 and locking cleanups     (Al Viro)
o Beta        Lifting limitations on mount(2)                 (Al Viro)
o Beta        Remove dcache_lock                              (Maneesh Soni, IBM 

team)

o Started Reorder x86 initialization (Dave Jones, Randy
Dunlap)

Have some free time and want to help? Check out the Kernel Janitor TO DO list for a list of source code cleanups you can work on. A great place to start learning more about kernel internals!

I've had the mechanism working for well over a year now, but still don't have a proper policy for load control implemented.

This is (in limited form) ready for merging and has been stability tested by Andrew Morton. A patch should go to Linus soon...

kind regards,

Rik

Bravely reimplemented by the knights who say "NIH".

http://www.surriel.com/		http://distro.conectiva.com/

Are there any reasons why these don't make it into 2.5?

cu
Adrian

You only think this is a free country. Like the US the UK spends a lot of
time explaining its a free country because its a police state.
								Alan Cox

Hi,

Linus was scared we could break old syslog parsers.

Reason unknown. I expect some statement here.

Possibly no one had one? I have some boxes running prettily on LinuxBIOS, can't imagine why it isn't merged into 2.5. But I can't imagine what's speaking against KBuild-2.5 either. I can speak it, I love it.

Nobody seemed to be interested in this toolkit. The (s|l)trace toolkit and kdb seemed to be sufficient for the most developers. (I don't whine here either.)

Seems they got lost.

Regards, Thunder

(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

Hi,

???
Have you actually tried it? I can only encourage everyone to try it and play a bit with it. It's a great debug tool and it can't be replaced by any of the mentioned tools. It's very useful in situations where printk doesn't work anymore.

bye, Roman

Hi,

It might not have been noticed publicly, since no one showed interest in it...

Regards, Thunder

(Use http://www.ebb.org/ungeek if you can't decode)
------BEGIN GEEK CODE BLOCK------
Version: 3.12
GCS/E/G/S/AT d- s++:-- a? C++$ ULAVHI++++$ P++$ L++++(+++++)$ E W-$
N--- o?  K? w-- O- M V$ PS+ PE- Y- PGP+ t+ 5+ X+ R- !tv b++ DI? !D G
e++++ h* r--- y- 
------END GEEK CODE BLOCK------

It's somewhat unfair to compare LTT to s/ltrace or kdb because they don't serve the same purposes. The other thread on "Enhanced profiling" should have made this very clear by now.

I've spoken to many key kernel developers about this and they all saw its inclusion as being positive, but they also all said the same thing:
it's really Linus' decision.

In light of the recent discussions, it would be really nice to get a definitive statement about LTT's inclusion in 2.5.

Cheers,

Karim

===================================================
Karim Yaghmour
[email protected]
Embedded and Real-Time Linux Expert
===================================================

It has been pointed out to you at least once that it would stand a much better chance if you were to follow the kernel coding style, for one ...

And things like :

+#ifndef CONFIG_SMP /* On an SMP machine NMIs are used to implement a watchdog and will hang
+ the machine if traced. */
+ TRACE_TRAP_ENTRY(2, regs->eip);
+#endif
+

aren't very encouraging.

just my 2p
john

"I know I believe in nothing but it is my nothing"
	- Manic Street Preachers

And if you had actually cared to follow that thread with Roman Zippel to its logical conclusion, you would have noticed that patches including an update to match the coding style have been made available:
http://marc.theaimsgroup.com/?l=linux-kernel&m=102106555627527&w=2

Care to comment on why? All what the above says is that this trace point shouldn't be active on an SMP box. That's one of the very rare cases (if not the only case) where such a build-condition is added. And if this really is too much for the kernel crowd's stomach then it is easily remedied.

I would have thought you had something a little bit more substantial to stand against LTT's inclusion.

Karim

===================================================
Karim Yaghmour
[email protected]
Embedded and Real-Time Linux Expert
===================================================

This is still in discussion. The real issue is not breaking syslog parsers (in fact its a way to make sure they dont break in future) but to get

- Accurate reporting
- Error classifications
- Error manuals
- Translations
etc

done in a way that doesnt make the kernel ugly. Thats non trivial. I need to schedule a discussion with some IBM folks about part of this

Alan

Please don't presume to speak for me on that.

Daniel

This isn't dead at all. Rusty and I talked about this at the kernel summit, and there was no loud verbal dissention (that I remember).

Unfortunately, progress has been temporarily hampered by a few things, like the factd that the people that this type of stuff actually turns on are few in number, and distracted by several other projects...

-pat

Em Wed, Jul 10, 2002 at 01:11:08AM -0400, Guillaume Boissiere escreveu:

Not really, still not submitted.

- Arnaldo

As of 2.5.25, we have HZ=1000 (on x86) and a scalable user-space exported clock_t that remains at 100 HZ to keep user-space compatible. This is attributed to the Commander in Chief, Linus Torvalds.

Robert Love

But jiffies now wrap at 49.7 days, right? If so, did Tim Schmielau's jiffies wrap patches go in as well? ISTR they went in -dj.

Didn't Red Hat change HZ to 1000 (or 1024) in Limbo as well? How did they handle that?

-- v --

[email protected]

George Anzinger's 64-bit jiffies are in 2.5.

Tim's code to better utilize them is in 2.5 I _think_.

Yes, RedHat's current devel kernel is using HZ=1000. I am not sure how they handled it. What we have in 2.5 now is correct.

Robert Love

Why was the rate incremented to maintain interactive performance? Wasn't that the whole idea of the pre-empt work? Does the burden of pre-empt actually require this?

It seems that the added inefficiency of these extra interrupts is going to drag performance down.

} George Anzinger's 64-bit jiffies are in 2.5.
}
} Tim's code to better utilize them is in 2.5 I _think_.
}
} > Didn't Red Hat change HZ to 1000 (or 1024) in Limbo as well? How did they } > handle that?
}
} Yes, RedHat's current devel kernel is using HZ=1000. I am not sure how } they handled it. What we have in 2.5 now is correct.
}
} Robert Love
}
} -
} To unsubscribe from this list: send the line "unsubscribe linux-kernel" in } the body of a message to [email protected]
} More majordomo info at http://vger.kernel.org/majordomo-info.html
} Please read the FAQ at http://www.tux.org/lkml/

I did not say it was increased to improve interactivity response - and it certainly has little or nothing to do with kernel preemption being merged.

I suspect a big benefit would be poll/select performance. I think this is why RedHat increased HZ in their kernels.

You would have to ask Linus exactly what his intentions were.

Robert Love

Red Hat Limbo ChangeLog says:

"The kernel used in this release supports the following list of improvements and new features. The kernel is based on the 2.4.19- pre10-ac2 release for this beta."

"HZ=1000 on i686 and Athlon means that the system clock ticks 10 times as fast as on other x86 platforms (i386 and i586); HZ=100 has been the Linux default on x86 platforms for the entire history of the Linux kernel. This change provides better interactive response, lower latency response from some programs, and better response from the scheduler. We have adjusted the /proc filesystem to report numbers as if using the default HZ=100, but it is possible that issues could arise -- please test and report bugs, as always."

So they aim for interactive response. Otoh, I think they don't include pre-emp nor any low-lat work. I might be wrong. But the network console and crash dump functionality they include (by Ingo Molner, I reckon) seems sweet.

-- v --

[email protected]

Bizarrely in many cases it increases throughput

Sometimes - Beowulf folks already sometimes hack the clock down to 20Hz or less. This is best approached on sane hardware by extending the S/390 stuff for no regular ticks.

I can attest to this. We see the same thing with the preemptible kernel (throughput increases on certain workloads).

My guess would be the better process response applies the same to
throughput: sooner to wake up, sooner to run, sooner to be done.

Robert Love

(I missed part of this thread. I hope I correcltly deduced that you guys are talking about the improved disk troughput when increasing the HZ clock rate ... )

IMHO, This is a hint that there is something not quite right with the scheduler.

This effect has been reported here a couple of times.

If increasing the timer rate improves disk throughput that means that the disk-reading process is not scheduled immediately following the disk interrupt, but is somehow left waiting until the next timer tick....

It should be scheduled "immediately" even if there is another cpu-eating process: the scheduling heuristics should help there...

Roger.

** [email protected] ** http://www.BitWizard.nl/ ** +31-15-2137555 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* There are old pilots, and there are bold pilots. 
* There are also old, bald pilots. 

Yes, they did, but the implementation is sligtly different. I would expect it will change to Linus' implementation in the next Limbo kernel update.

Justin