delete devfs

Hm, seems kernel.org dropped my big patch, so the patch below can be found at:

	www.kernel.org/pub/linux/kernel/people/gregkh/misc/2.6/devfs-delete-2.6.8-rc2.patch

----- Forwarded message from Greg KH <[email protected]> -----

Date: Wed, 21 Jul 2004 02:49:37 -0400
From: Greg KH <[email protected]>
To: Andrew Morton <[email protected]>
Cc: [email protected], [email protected]
Subject: [PATCH] delete devfs

Ok, to test out the new development model, here's a nice patch that simply removes the devfs code. No commercial distro supports this for 2.6, and both Gentoo and Debian have full udev support for 2.6, so it is not needed there either. Combine this with the fact that Richard has sent me a number of good udev patches to fix up some "emulate devfs with udev" minor issues, I think we can successfully do this now.

Andrew, please apply this to your tree and feel free to send it to Linus when you think it should be there.

thanks,

greg k-h

<PATCH SNIPPED>

----- End forwarded message -----

Am Mittwoch, 21. Juli 2004 16:15 schrieb Greg KH:

May I point out that 2.6 is supposed to be a _stable_ series?

Regards
Oliver

Yeah, sounds like a good series to drop unsupported code from... ;-)

Sincerely,
Lars Marowsky-Brée <[email protected]>

High Availability & Clustering	    \ ever tried. ever failed. no matter.
SUSE Labs, Research and Development | try again. fail again. fail better.
SUSE LINUX AG - A Novell company    \ 	-- Samuel Beckett

Quoting GregKH: `to test out the new development model', i.e. the one established at the Kernel Summit yesterday afternoon :-)

Well, looking at the late (or early :-) hour he sent out the patch, it looks like he did need a few beers before he gained enough confidence in this `new development model' :-)

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

You didn't pay attention to the first sentance that I wrote for the patch :)

And as Lars points out, the code is unmaintained, unused, and buggy. All good reasons to rip out it out at any moment in time.

thanks,

greg k-h

Unused? Since when does every Linux user use a vendor supplied kernel? I have no use for devfs, never used it in the past, and I'm a happy udev user now, but that doesn't change the fact that there are many devfs users out there.

What does this gain us right now?

Jesse

Jesse Stockall <[email protected]>

It fixes an obviously broken chunk of code that is not maintained by _anyone_. And it will clean up all device drivers a _lot_ to have this gone, which will benifit everyone in the long run.

As for "right now"? Why not? I'm just embracing the new development model of the kernel :)

thanks,

greg k-h

Agreed, but this 'broken' chunk of code is 'working' for a lot of people (whether or not this is due to pure luck is not the point)

That's the point that Oliver and I raised, the "leave it till 2.7" (not breaking things for real world users) argument seems stronger than the "rip it now" (because it makes things cleaner, easier to code, etc) argument.

Devfs should never have made it in the kernel in the first place, but ripping devfs out in the middle of a stable series does not solve any problems, it creates them.

Is keeping devfs around for 2.6 really that much or a burden? When was the last time you saw any mails on lkml asking for devfs support?

Jesse

Jesse Stockall <[email protected]>

Pure luck.

The kernel development model (the whole stable/development tree thing) has changed based on the discussions at the kernel summit yesterday. See lwn.net for more details. That is why I sent this patch at this point in time.

thanks,

greg k-h

Got any other sources for the info that don't require a subscription? I even live in Ottawa, if you have something available :)

Jesse

Jesse Stockall <[email protected]>

Am Donnerstag, 22. Juli 2004 00:05 schrieb Greg KH:

Interesting, but we are not talking about an _internal_ API here. It's about blocking the upgrade path. System using a stable kernel will needlessly stop working after an upgrade to another stable kernel.

Regards
Oliver

There is no such block. udev has a full devfs compatibility mode, I made sure of that before every suggesting that a change like this happen.

Userspace tools need to be upgraded/added due to different kernel changes all the time. This is just another one of them.

Also, everyone please, consider these points about the current devfs code:
- it is unmaintained, and has been for years.
- it contains known bugs (race conditions), that are pretty much unsolvable with the current architecture of the code, that have been pointed out many times, for years.
- there is almost no functionality that devfs provides that is not provided with udev[1]
- no distro supports devfs
- no active, respected, kernel developer wants to see devfs remain in the kernel tree.

Yes, this has always seemed to be a hot topic. And yes, I did really push a lot of people's buttons by posting this patch[2], but I did it for two reasons:
- It was going to be my first 2.7.0 patch anyway.
- Based on the discussions at the kernel summit, the development model has changed. This patch was a test to see if anything has really changed or not :)

I'm sorry about the fact that this change in development model was not really made public before I posted it. That probably caused the most confusion in this thread, and with hindsight, I should have waited a few days for that information to have gotten out to the whole world before submitting it[3].

Again, I apologize.

thanks,

greg k-h

[1] ok, yeah, the floppy driver will not get loaded automatically if you try to open a /dev node that is not present at that time. But if you still rely on this model of behavior to properly load your modules, you somehow missed the memo that stated that the kernel has changed from automatically loading modules based on userspace needs (with kmod), to one where they are loaded based on the presence of a device. This has all been discussed in detail many times in the past on lkml. See the udev FAQ for some other details about this.

[2] It was said that I rubbed the lamp with this patch, causing Richard to suddenly appear in public at OLS within 8 hours. I personally doubt this, and publicly welcome Richard back to the kernel development community.

[3] Note to self, sending patches at 2:30am after spending the evening in deep technical discussions[4] with other kernel developers is not the best thing to do.

[4] Ok, they weren't that deep...

Am Donnerstag, 22. Juli 2004 08:49 schrieb Greg KH:

A smooth upgrade is replacing the kernel image, rerun lilo and reboot. There's very good reason to remove devfs from 2.7.0, but almost none to remove it from 2.6.X (unless Richard turns up and rewrites it from scratch) Breaking existing setup is acceptable only if you can do nothing else.

Regards
Oliver

I tend to agree with you even though I still have to read the "MOM" regarding what has been discussed yesterday.

If I'm not wrong, there is not plan to fork a 2.7 kernel anytime soon, so this kind of patch that brokes only a few users will probably be included in 2.6.*

Ciao,

paoloc.doesntexist.org

Minor nit: this is not true, maybe if you are talking only about desktop/server distros. MontaVista Linux 3.1 uses devfs by default.

However, with udev devfs compatibility it easy for that distro to migrate users IMHO. So, let's apply this patch so mvista doesn't ship another release with devfs. :)

-Matt

Hi listmembers,

I'm not a kernel developer, but recently I've been testing many development (ie. -mm and -rc) kernels and I run a network containing quite a lot of Linux boxes, so I'm involved (a little) in the kernel development or at least I'm affected by it to some extent. Anyway, I have an idea that I think you may find interesting.

1. Background

There apparently is some code in the kernel tree that is buggy and not maintained by anyone. The recent attempts to remove some parts of it (devfs, cryptoloop) have been opposed, as it turns out that they are still in use.

OTOH, because this code is present in the mainline kernel, the users of the kernel can expect that the code will be supported by kernel developers, which is not correct. Therefore the code should be removed from the kernel, so that it's not used by any new users who may expect it to be supported (there are many other reasons for removing it, but this one alone is sufficient, IMHO).

Having said that, it is not very nice to pull rugs from under people in general, so before the unmaintained code is removed from the kernel, its current users should be given some time to accommodate to the upcoming changes. Therefore the unsupported code should be made clearly distinguishable from the rest of the kernel code and documented as such, in order to indicate to the users that it may be removed at any time.

2. Proposal

I propose to introduce a new configuration option CONFIG_UNSUPPORTED, such that if it is not set, the unmaintained/unsupported code will not be compiled into the kernel. Moreover,
* IMO the option should not be set by default, which would require a user action to include the unsupported code into the kernel,
* IMO the option should be documented as to indicate that the code marked with the help of it is not supported by kernel developers and may be removed from the kernel at any time without notification.

I think that this would be fair enough wrt. users, who would be able to learn that the code is not maintained and may be removed at any time without notification, and they should not expect to get any support ftom the kernel developers wrt. this code, and it's generally not a good idea to file any bug reports regarding this code, because the bugs in it will not be fixed anyway.

OTOH, it would give the kernel developers a means to mark unsupported/unmaintained code as such in advance, without harming any users in the short run.

Yours,
rjw

Rafael J. Wysocki
[tel. (+48) 605 053 693]
----------------------------
For a successful technology, reality must take precedence over public 
relations, for nature cannot be fooled.
					-- Richard P. Feynman

Quoting 2.6 MAINTAINERS:

<-- snip -->

PCMCIA SUBSYSTEM

L:      http://lists.infradead.org/mailman/listinfo/linux-pcmcia
S:      Unmaintained

<-- snip -->

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Sure, but does it mean "It's buggy and may be killed at any time. And BTW don't compile it in if you don't know what you're doing, because it may kill your dog and noone will help you - you have been warned"?

The idea is essentially not about saying that something is unmaintained, but about marking chunks of code slated for removal in a manner that's clearly visible to users.

Yours,
rjw

Rafael J. Wysocki
[tel. (+48) 605 053 693]
----------------------------
For a successful technology, reality must take precedence over public 
relations, for nature cannot be fooled.
					-- Richard P. Feynman

Not entirely true - it is "looked after" but not specifically maintained. I'm happy to act as a patch collection service for it, and do certain development on it, but I have enough to handle without having to diagnose every guys laptop problems. 8)

I guess I should find a better way to express this in my sig as well!

Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Stuff is marked OBSOLETE in Kconfig files, and text for menu option is reflecting this.
This can well be overlooed if one does 'make oldconfig' only, but at least documented this way.

Sam

Good point.

A better name for this config option might be CONFIG_OBSOLETE, with the semantics described in R. J. Wysocki's original post.

- Steve

Personaly as many of my friends (those who use and care about kernel) we think that devfs is (was) the only resonable solution for /dev tree, and should be only one maintained. Requirement of userspace software for /dev is just a one big mistake. IMO in year time someone will have another brilliant idea, and will rip udev off. I don't think that's a good solution. IMO (and that was my humble opinion) devfs should be maintained instead of rewriting thing again, and creating problems (udev is not present unless userspace is up, etc). so for me, and many others devfs should stay as only solution :-)
I am pretty shocked that there is no expierenced developer to maintain devfs. I would be delighted to do it, but I guess it's over my schledue. I don't have time enough to maintain my KDE stuff atm, not to mention kernel bits...

(no flames please, just giving my opinion).

--
GJ

Maintaining device registration and persistence all from userspace is wonderful. Try it, you'll like it. Not to mention that for the developers udev is much more easily maintained than kernel code (devfs). Your concern about something else ripping off udev a year from now is misplaced; the beauty of free software is the best solution generally wins out. That "something else" will have to be pretty damn amazing to upstage udev.

Mike

I wouldn't disagree with this statement. I'd be very surprised if 2.7.2 would still contain devfs.

Could anyone please explain this mysterious "new development model of the kernel"?

Is this some personal fight from you against Linus or someone else you are trying to bring to linux-kernel, or WTF has happened???

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

No fighting is going on here. I know lwn.net has already reported about this, see there for details. I don't have the time to write it up right now due to being at OLS.

thanks,

greg k-h

Seems to be subscribers only...

Regards: David Weinehall

 /) David Weinehall <[email protected]> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/

Ok, is there anywhere else that isn't subscriber-only that has the scoop?

--
Brian Gerst

For those who weren't here, the basic scoop is this:

- 2.6 is doing great, despite the fact that (by Andrew's reckoning) some 10mb/month of patches are going into it.

- Linus is majorly pleased with how the partnership with Andrew has been working, and few people feel that he shouldn't be. He is a little concerned about breaking a highly effective process when 2.7 forks.

- There is not a whole lot of pressure to create a 2.7 now, but a lot of interest in getting patches into the mainstream quickly.

The end result is that there may not be a 2.7 for a while. Instead, significant patches will continue to go into 2.6, after a vetting period in -mm. Andrew stated his willingness to consider, for example, four-level page tables, MODULE_PARM removal, API changes, and more. 2.7 will only be created when it becomes clear that there are sufficient patches which are truly disruptive enough to require it. When 2.7 *is* created, it could be highly experimental, and may turn out to be a throwaway tree.

Andrew's vision, as expressed at the summit, is that the mainline kernel will be the fastest and most feature-rich kernel around, but not, necessarily, the most stable. Final stabilization is to be done by distributors (as happens now, really), but the distributors are expected to merge their patches quickly.

Anybody disagree with that summary?

jon

Jonathan Corbet
Executive editor, LWN.net
[email protected]

Thanksfor this mail, this is exactly the information that was missing.

Discussing the contents:
Changes that remove functionally like Greg's patch are hopefully still 2.7 stuff - 2.6 is a stable kernel series and smooth upgrades inside a stable kernel series are a must for many users.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

I don't necessarily agree that such changes in the userspace interface should be tied to the kernel version number, really. That's a three or four year warning period, which is unreasonably long. Six to twelve months should be long enough for udev-based replacements to stabilise and propagate out into distributions.

That being said, mid-2005 would be an appropriate time to remove devfs. If that schedule pushes things along faster than they would otherwise have progressed, well, good.

Nothing is cast in stone here btw - we're pushing the envelope, trying new things, keeping that which works well and reexamining things which perhaps don't work so well. Feel free to disagree - we're listening.

Users have had the 6-12 month warning about devfs for a while now :) And udev is currently available in the latest distro versions of:
- Red Hat
- SuSE
- Gentoo
- Debian
- Mandrake

While devfs is only supported in Gentoo at this time (and udev fills that support issue for those users.)

Ok, if people think that would really change anything, I'll wait a year. I'm patient :)

thanks,

greg k-h

No, they had a three year warning. "It'll be gone in 2.8".

Delete 100 lines per week ;)

That could be done by sending in smaller patches that remove devfs calls from drivers. If nothing in the kernel is using devfs, then there is no reason to keep it around anymore...

Just a thought.

josh

I've still some bug report of people using home-compiled devfs kernels on Debian. So people still use it. You say "devfs" is buggy, but it works on nearly all cases, so people tend not to switch.

The worse is the lack of stable name of devices, in udev too. I.e. microcode loader (Intel CPU) needs a device, which was so named (last time I controlled):

  # device name in LANANA / devices.txt
  DEVICE=/dev/cpu/microcode
  # device name in devfsd
  DEVICE2=/dev/misc/microcode
  # device name in udev
  DEVICE3=/dev/microcode

If we a coherent *default* device name scheme, the switching from a kernel utility to other would be trivial.

ciao
cate

Note: /dev/cpu/microcode was also created by devfs until recent 2.4 kernels and the whole 2.6 serie.

When was the last time you _used_ the microcode device?

Yeah, there are still a small number of drivers that are not in sysfs, so udev doesn't know about them, but right now I'm guessing we cover about 95%. I'm waiting for someone else to fix up the rest, if they really have one of those odd devices :)

thanks,

greg k-h

Hi Andrew,

my personal opinon is that this new development model isn't a good idea from the point of view of users:

There's much worth in having a very stable kernel. Many people use for different reasons self-compiled ftp.kernel.org kernels. In 2.4, it took until at about 2.4.18 or 2.4.22 [1] until it was reasonable stable. Today, most code in the 2.4 kernel has had several years of testing and it's therefore quite stable even in unusual configurations. Besides this, an upgrade like from 2.4.25 to 2.4.26 is pretty low-risk since there shouldn't be any changes that might break existing setups.

If your work together with Linus is so effective, why can't you both do all the changes in a new 2.7 tree that includes also all incompatible and potential dangerous changes as well as the removal of obsolete code like devfs or OSS. I don't see the negative effect if a 2.7 branch was created today and together with a feature freeze for 2.7 three months from now this might result in a 2.8.0 before christmas [2] that contains all the new features/removals/changes while 2.6 will evolve further into a rock-solid stable kernel.

cu
Adrian

[1] there are different opinions on the exact version number, but it was

    definitely not 2.4.10

[2] perhaps a bit optimistic, but it shouldn't be years from now

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Well. We'll see. 2.6 is becoming stabler, despite the fact that we're adding features.

I wouldn't be averse to releasing a 2.6.20.1 which is purely stability fixes against 2.6.20 if there is demand for it. Anyone who really cares about stability of kernel.org kernels won't be deploying 2.6.20 within a few weeks of its release anyway, so by the time they doodle over to kernel.org they'll find 2.6.20.2 or whatever.

4kb stacks were added after 2.6.0 and now 4KSTACKS=y results in Oops'es under some circumstances if using XFS.

2.6 currently still becomes stabler, but every new/changed feature bears the risk of breaking something.

Who will maintain the many subtrees of 2.6 this implies?

Even after 2.6.20 was already released, you might have to release a 2.6.19.5 with a few additional security fixes since according to your advice users should continue to use 2.6.19 for a few weeks which implies that someone will have to maintain at least the 2.6.19 tree for at least a few weeks after the release of 2.6.20 .

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

How is this any different then the old dev model with very short release cycles? (Other then keeping a "2." prefixed forever)

Currently, the old stable tree will be supported for several years even after the new stable tree opens (2.0 and 2.2 seem to be quite dead today and might even miss several security fixes but 2.4 is still well-maintained). If you use a 2.4 kernel today you know that you can follow 2.4 for the next year without risking big breakages.

With the new dev model, I assume noone will maintain a 2.6.8 tree for several years.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

I wont recommend this, as it screws with some (most?) things trying to detect kernel version running from uname =)

Martin Schlemmer

Andrew Morton ([email protected]) scribbled:

I have to agree with Adrian, the first thing I always do with a new distro is rip out the kernel and drop in a vanilla from kernel.org. I've been biten too many times by distro kernels. :(

imho, I feel there are two main concerns with changing the development model:

1.) Need to have readily identifiable stable versions w/o
following lkml.
2.) Understanding the changing of version numbers in light of
this change of strategy.

Ideas:

wrt (1), I think the -rc? system would be simplest. 2.6.20 is stable, 2.6.20-rc3 is not.

wrt (2), assuming the naming stays the same:

major++ = major overhaul of core system. minor++ = overhaul to drivers (or subset thereof). patch++ = testing patches survived, appear stable. extra++ = next set of testing patches applied.

Sure, this would mean version numbers start to creap up, but nothing is stopping a kernel version 2.11.x (what?! where's my 3.0.1? We were definitely supposed to have a 3.0 around here somewhere... Where's my meds? *frowns*).

tia,

Cooper.

So instead of even minor numbers indicating stability, you have pushed two levels down so that higher sub-revision (minorminorminor?) numbers indicate increased levels of stability?

Kinda makes sense.

Does that mean that 2.6.21 and 2.6.20.1 are two separate forks of 2.6.20, one for development, and the other for stability?

How is this fundamentally different from how it was done before with odd/even minor numbers?

It's like the details have been changed to give the illusion that development will go faster, when in reality, the fundamental approach hasn't really changed.

Kernel 2.4 continues to be actively supported for several years after the release of kernel 2.6 .

How long do you assume will kernel 2.6.20 be supported after the release of kernel 2.6.21?

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

I'd like to throw my opinion into the discussion at this point too, for what it's worth.

I think the idea of forking off certain releases in the 2.6.x.0 form, to only recieve bugfixes and security updates, is a very good idea. A couple of points against it were raised above, but I think if it were approached the right way, they wouldn't be issues.

There wouldn't be a huge maintenance overhead, as

-- The forks would only need to happen occasionally. When 2.6.y has advanced
past 2.6.x (y > x) sufficiently (i.e. it has significant new functionality) and it has been released for sufficient time to iron out obvious bugs, then if it comes to be considered a particularly stable release, it would be a good candidate for freezing at 2.6.y.0. I would think this sort of thing would probably only need to happen once or maybe twice per every ten or so traditional releases.

-- The only maintenance that would be needed on these frozen versions would
be a backport of any critical bugfixes / security issues, when they occur.

The table on the front page at kernel.org could be augmented with an extra row, showing the most recent frozen release from the stable tree. Users who want a recent _vanilla_ kernel that is unchanging and hopfully highly stable, could choose it.

IMO the process wouldn't mirror the old 2.x / 2.y model because it is much more fine-grained. With the old model, changes have to be backported to a kernel that is significantly older, and which potentially has seen fundamental changes in the releases between (i mean between 2.x -> 2.y). With the new model, a release 'freeze' could be made whenever deemed necessary, and since it will be a lot closer on the timeline to where the main development is happening, backporting the critical stuff should be a lot less of a headache.

The important part would be to not go overboard with release freezing, so there wouldn't be 10 frozen kernels to backport to. _That_ would be a headache.

There's my $0.02. :)

	Ben

Leave that to the vendors, they already do that.

Whats wrong with adding features which touch major parts of the code only to 2.7, and perhaps bacport them if they proof to be worth it?

Greetings
Bernd

eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/

I guess it's pretty similar in practice. I brought it up because the idea of freezing releases at 2.6.x.0 is more fine-grained, and as such it seemed to me that it would be less of a maintenance overhead.

Although labels aside, I suppose the two systems are acheiving the same thing in the end.

--
Ben

Someone has to do the actual backporting.

Krzysztof Halasa

Well, thats not a problem, since if a feature is not backported nobody is interested and it wont affect the stable stability.

Greetings
Bernd

eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/

I think so.

I assume the numbering will stay the same, i.e.
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 8
EXTRAVERSION =-rc2

will eventually become
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 8
EXTRAVERSION =

and then possibly

VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 8
EXTRAVERSION =.1 (or -pl1 etc.)

so it won't require changing scripts.

I think so. The scheme is somehow similar to -AC (Alan Cox') tree - and we all know that it (the process etc) was working very well.

Krzysztof Halasa

There certainly is. But the contribution that having a separate 2.7/2.8 kernel at the head (Linus, et. al.) end has to a user having a stable kernel is not what you think it is.

The direction presented to us now is having smaller, more continuous, steps in the head end, over having fewer larger, more disruptive steps. Do we do all the incompatible changes in a big chunk, once every couple of years, or do we do them one a week, ongoing.

Now, I repeat, this is at the head end. End users who want stability aren't feeding directly off kernel.org anyway.

The affect downstream on real users is this. If the head end operates in big chunky style, then as this big change flows downstream, it makes transitions for distros, service providers and middlemen more costly and difficult, creating expenses that must be passed on to the end user.

Yes - damming up the flow of changes creates stability. But if you're a middleman, you don't need Linus to choose where to build the dam, and when to open the flood gates. It's more efficient for you to choose for yourself when to do that damming, based on your particular resources and customer needs, rather than have to deal with hundred year floods and draughts imposed on you by Zeus.

The end user always gets their stability, if only because they won't upgrade a system that is "working".

And every change at the head end is disruptive for some poor slob. The only perfectly compatible change is no change at all. We delude ourselves if we think we can separate the "safe" fixes and additions from the "unsafe" incompatible changes. Better that we should expend some energy on smoothing out and minimizing the cost of change to those downstream from us. This needs to be done the old-fashioned way, one change at a time.

The question is whether we impose, on all those downstream from us, occasional hundred year floods, or do we provide a steady stream, and let them build their own little beaver dams, as suits their various and diverse needs and business cycles.

The great lesson of capitalism over communism is that a thousand free workers and investors, each optimizing their own little plot or portfolio, beats a single centrally imposed five year plan, even if the man pulling the levers is a genius such as Linus or Lenin (sorry, Linus, couldn't resist ...).

                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <[email protected]> 1.650.933.1373

It depend on your definition of "end users" and "stability".

You might be right for people buying for many $$$ distributions with support to run Oracle on it.

But I know many people who run ftp.kernel.org kernels on many workstations and small servers e.g. in universities.

Currently there's one transition to a new major release of the kernel every few years. You wait until a new major release of the kernel has matured, test whether both the transition and the new kernel work, and update the kernel.

An update to a new minor version of the kernel was until now relatively cheep since breakages that require further investigation were relatively seldom.

And what happens if you are a distribution, kernel 2.6.15 is the current kernel containing many important updates, but $nontrivial_feature added in 2.6.10 broke $architecture your distribution supports. This means you have to support both kernel versions with security updates creating expenses that must be passed on to the end user.

I vaguely remember such issues in the past like DECstation support was broken since 2.4.20, and some m68k subarchs that were at least until recently not working in any kernel more recent than 2.2 .

Such breakages might occur more often in the future during a stable series.

How do such end users get security updates?

But what to do if some part of the kernel (let's call it "XFS") has some problems (let's call it "Oops") with a new feature (let's call it "4kb stacks on i386") introduced in a kernel during a stable kernel series (let's call it "2.6.6") and this isn't fixed by the maintainers (let's call them "SGI") for some time (let's call it "until now")?

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

That is their choice, but there's no particular need to run a kernel.org kernel. Unless you're messing around with the kernel or have a hot requirement for some new feature, why would running a stable kernel from e.g. Debian not suffice? Debian is free and freely available, and it's not the only distribution that is that way.

You can't have it both ways. If all you care about is stability, you can run Debian stable, and be rock solid. If you want to play with the latest code, you can download a kernel.org kernel. There is no shortage of sources of kernels. It would simply mean that the kernel.org one would have slightly less of a guarantee to be "stable", although as was originally reported, 2.6 is going very well, and despite the flow of changes, there isn't a lot of terrible breakage happening.

[...]

No, you have several choices. For instance, you can continue to ship 2.6.10, or you can fix the problem. The reason Red Hat shipped a different gcc back in the 7.x days and were subjected to much flamage was because they wanted a c++ compiler that worked on architectures other than x86 (alpha), and they put in effort to obtain such. Using Red Hat as an example again, RHEL3 is based on 2.4.21. The current update is still labelled as 2.4.21. It has fixes from later kernels but it isn't based on 2.4.26. So this doesn't involve a radical change from how things are today. They backport security fixes as do pretty much all distros.

[...]

The distros apply security fixes and make updated kernels available on a regular basis.

Umm... compile the kernel with 8k stacks? Nobody is forcing you to use the new option. And as has been pointed out by Chris, you are still at risk of a stack overflow even with the 8k stacks if you take an interrupt at the wrong time and it happens to be a heavy stack user. Or you could run 2.4.26. Or...

The reasoning is that the advantages are heavily outweighing the disadvantages, and I have to agree.

Regards,

Tim

Tim Wright <[email protected]>
Splhi

One idea might be to fork off "stable" 2.6.x.y (2.6.15.1 for example) branches every now and them. Analogous to vendor kernels, but maintained by [email protected]. Compared to the 2.6.x.0 in question, these would only get security fixes and important bug fixes. The maintainer would need to pick a suitable (stable) 2.6.x.0 as basis every once in an appropriate while.

I don't know if that's feasible. Just an idea.

Anyway, as (one kind of) end user, I do welcome the new development model. I'll get the newest features in manageable manner, and if I don't fancy that I can resort to vendor (Fedora) kernels.

-- v --

[email protected]

You *do* realize that the kernel in the Fedora development tree is actually *ahead* of the released kernel.org tree, right?

The current kernel-2.6.7-1.494.src.rpm is based on 2.6.8-rc1-bk5, with a bunch of RedHat/Fedora patches on top of that.

And the 2.6.5-1.358 kernel that shipped in Fedora Core 2 is actually a 2.6.6-rc3-bk3 with patches on top of that.

I think you meant the RHEL series of kernels - current there is 2.4.21-15.0.3.EL, with backports of security/bug fixes.

You are absolutely right, Fedora was a bad example. RHEL is better.

In the past, my experience, shared by many users, I'm sure, has been that distribution kernels generally give you worse performance (IME RH) and less stability (IME Fedora).

There's an increasing amount of hardware out there in wide-spread use, which have no drivers in either kernel.org tree or distribution trees. The fragmentation between the distributions already make it impossible to get those drivers to compile on anything but the kernel.org tree, unless the author of the driver is wealthy and has the resources and floorspace to have a few different machines with different distributions installed, and the time and resources for creating workarounds and Makefile trickery for each and every one. I don't mean binary drivers here, as they are usually backed by some corporation and target the usual distributions...

Thus, we have a whole generation of users out there who grew up with the idea that the distribution kernel is just some bloated, bug-ridden and mostly incompatible monstrosity that is only barely good for bootstrapping kernel.org kernel before starting to try compile the drivers for their hardware.

Trying to change this idea is of course difficult, as everyone is afraid of change. "Will the drivers break next release?", "Will I have to stay with an old and exploitable kernel sometime in the future when the drivers no longer compile on anything but kernel.org X.Y.Z, when distro is X.Y.(Z-3)-secfix42, and kernel.org is up to X.Y.Z+5?"

It might very well be that pushing out a large portion of the dev burden to the periphery will be good in the long term for the development of the kernel, but in short-term, I only see the fragmentation problem getting worse. I hope I can be brutally proven absolutely wrong, though. :-)

I have to agree with you here. My experience is that the vendor kernels are usually build to fit a wide variety of systems and include support for a huge amount of features since there's always some of their customers that need feature X or feature Y, so they include all of them, which leads to a kernel that runs slower than it has to and has a bigger potential for problems (more features included == more stuff that can blow up in your face).

[snip]

Indeed. That's my personal attitude to vendor kernels, and I know it's shared by most of my Linux using friends.
First thing you do after getting your distribution of choice installed is to go to kernel.org, grab the latest stable kernel, build it with the features you need and then leave the vendor kernel far behind for good. Personally this is what I do for both my personal systems and my servers at work - and that's pretty common, and since the latest stable kernel.org kernel tends to actually /be/ stable that approach has worked well for years.

Also not all vendors keep up with security fixes equally well, so it's a common (at least in my experience) attitude that if you want to keep up-to-date security wise you should just keep up with the kernel.org kernels and you are resonably safe.

Also, it is usually a pretty safe bet that if you need to use third party modules, you are very good off with a kernel.org kernel as that tends to be the reference kernel that stuff gets tested against (personal experience, I have nothing concrete to back that up with).

If the stable kernel.org kernel stops being stable and reliable a lot of users will be badly disappointed and will be forced to either stay with old insecure kernels or be forced to use vendor kernels with all the bloat that implies. That would be a sad state of afairs in my oppinion.

I guess the perfect situation would be if the kernel.org kernel would be stable enough and feature rich enough that the vendors didn't /need/ to supply anything else than the stock kernel from kernel.org - how to get to that point I don't know though.

just my 0.02euro

--
Jesper Juhl <[email protected]>

--- Paul Jackson <[email protected]> a écrit : > > There's much worth in having a very stable kernel.

;-)

... and break existing compatibility, and make the system crash twice a day ...

Are you sure ? There are a number of distribution who use the
stable kernel from kernel.org and some of them are much faster
(if you remember, compiling a kernel to suit your needs
sometimes improve performance).
One size _does not_ fit all.

And with new devepment model this expenses will be passed to the

end user when the kernel will not be stable enough and will
crash. Do you you remember the 8k vs 4k stack problem for
Nvidia binary kernel module ?

So now the world is divided in gods (i.e distributions) and we, mere mortals who should pray to the gods to give us a stable
kernel ? And if we commit a sin our machine will crash ?

They will get the stability through the new discovered remote security hole because they don't upgrade their system because "the system is working" and they don't need to upgrade ?

You are wrong here. You never saw the two systems working. In comunism: One size fits all. This size is made to accomodate
80% of the population.
In capitalism: There are two sizes: one for rich and one for the poor. The one for the poor is almost bullshit but is cheap and anybody can afford it and the one for the rich is verry expensive and only few can afford it.
Now do a s/poor/kernel.org kernel/ and s/rich/distribution kernels/ and you see what your development model is.

=====
--
A mouse is a device used to point at
the xterm you want to type in.
Kim Alm on a.s.r.

Vous manquez d’espace pour stocker vos mails ? Yahoo! Mail vous offre GRATUITEMENT 100 Mo ! Créez votre Yahoo! Mail sur http://fr.benefits.yahoo.com/

Le nouveau Yahoo! Messenger est arrivé ! Découvrez toutes les nouveautés pour dialoguer instantanément avec vos amis. A télécharger gratuitement sur http://fr.messenger.yahoo.com

This seems to be where a lot of the misunderstanding is. Did anybody notice just how divergent the distributors' 2.4 (and prior) kernels were from the mainline? If you wanted a kernel with that level of features and stability, you had to get it from them - or apply hundreds of patches yourself.

One of the goals of the process now is to get those distributor patches into the mainline quickly. My expectation is that the mainline kernel will be far closer to what the distributors ship than it has been in a long time, and the mainline will be more stable for it. Just the opposite of what a lot of people are saying.

jon

The 2.4.18 kernel source in Debian stable in has at about 300kB patches applied (gzip'ed 60kB), and many of them fix security bugs that were reported since 2.4.18 was released.

OTOH, distributions might have to ship more kernel versions and provide security updates for more kernels if features in newer 2.6 kernels break architectures or subarchitectures they support.

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Well, back in v2.4 "hot-stop", most of the patches merged into distro's kernels were not "trustable" enough to be merged into v2.4 mainline, and I had no capability of reviewing them myself and make a good enough judgment of whether they should be included or not.

Another point I had against merging some of those patches was that most of them were targeted at "enterprise" users and benefit almost only them (eg finer-grained locking, etc).

To resume, I prefered to be more "conservative".

Of course, fortunately Andrew is much more capable of doing judgements on "trustability" of patches and so forth.

Obviously its a good thing to try to keep the differences between distro's kernels and mainline kernels small, and Andrew is targetting that.

You want a stable kernel, but you also want to rely on binary-only kernel modules?

The Linux kernel people cannot guarantee stability with binary-only kernel modules. And the Linux kernel people cannot solve that problem...

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

--- Geert Uytterhoeven <[email protected]> a écrit : > On

No. I wasn't clear on that one. My example was wrong. AFAIK the 8k/4k stack kernel problem were causing problems for other people too.

I underestand that.
However hpa told me that the stability of the 2.6 kernel will not suffer.

Calin

=====
--
A mouse is a device used to point at
the xterm you want to type in.
Kim Alm on a.s.r.

Vous manquez d’espace pour stocker vos mails ? Yahoo! Mail vous offre GRATUITEMENT 100 Mo ! Créez votre Yahoo! Mail sur http://fr.benefits.yahoo.com/

Le nouveau Yahoo! Messenger est arrivé ! Découvrez toutes les nouveautés pour dialoguer instantanément avec vos amis. A télécharger gratuitement sur http://fr.messenger.yahoo.com

And akpm posted that he intended to remove cryptoloop, while others are calling for the end to devfs. Not having features disappear is part of stable, I would think, not just "not oops more often."

   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me

OTOH removing things declared "obsolete" for a long time doesn't make it unstable - does it?

Krzysztof Halasa

Obsolete for a long time? This is a new feature in 2.6! It was just added and was the hook that got some people to go to 2.6 in some cases, to have some useful security in laptops. To remove it would effectively block following newer kernels when security holes are found, since dm-crypt would mean installing new software, training the support group, reformatting the disk to generate a partition to use, etc.

When major new features just added to a stable series vanish after they are adopted I most definitely do call that unstable.

And dm-crypt is vulnerable to the same types of attacks, it's just harder to use.

bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Well, actually I was thinking about removing devfs in 2005.

Krzysztof Halasa

Right. Aaaand if I remember correctly you may download the source for, say, a RHEL kernel for gratis from Redhat (or a whitebox distro) and make menuconfig it and compile it the same way you do vanilla kernels.

In fact, a single tree is far better for stable development in general. Vanilla things that break get spotted sooner, we avoid 2.x.0 the syndrome, and distro kernels in general may get more eyes from clueful people that otherwise would ignore them and leave the problems to less resourceful users.

"Dont Panic" :)

Ragnar Hojland - Project Manager
Linalco "Specialists in Linux and Free Software"
http://www.linalco.com  Tel: +34-91-4561700

Do I read that correctly? You are suggesting doing big disruptive changes every week? I know what you mean, but this is just what many people don't want, being told that they have to give up cyrptoloop or devfs to get a better scheduler.

Having been bitten by vendor kernels repeatedly, I would say that some are and some aren't. And people who want minor new features like a device driver or better scheduler don't want to rebuild a system from a base install just to get there. It's one thing to get all the new features of a 2.N to 2.{N+2} conversion, but it isn't worth a major reconfig to get there.

Do you suggest that many disruptive changes are less expensive than one every few years? There should be something between RHEL "stable for five years" and FC2 "run up2date on the hour" models. The stable series has been that in the past, and dropping any disruptive change into the stable series seems to belie the term stable.

So everyman becomes his own vendor, if you want a minor feature you have to patch it into an old kernel which still works for you? This is a major change from the way stable vs. development has ever worked... It's not the pace of little things which troubles me, it's Reiser versions, and vanishing cryptoloop and devfs, and things like that which require major changes in a system.

No matter how you spin it, you are still talking disruptive change over and over vs. an occasional major rethink.

Of course making the end users far more dependent on vendors didn't enter into this at all...

   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me

I will have to bow out of trying to further justify what was decided at the Summitt with my marginalia. My explanatory gloss are not sufficient to persuade some dissenters ... so be it.

My gut is that this is a good direction. I trust others can speak to it with more authority.

                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <[email protected]> 1.650.933.1373

I'm currently wondering whether part of our discussion might be about a non-issue:

It's true that there's not a pressing need for opening 2.7 today.

But do you assume that this will still be true one year from now?

If 2.7 will open during the next 12 months, "mid-2005" would still be after 2.7 opened, and such non-urgent cleanups could then go into 2.7 .

cu
Adrian

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Here is a portion of the story; hopefully posting a snippet will not offend John Corbet. You guys really should just subscribe to LWN.. the following is just a taste of the insight LWN has to offer:

<snip>
Linus talked about how happy just about everybody is with 2.6. It has been almost two years since the alleged 2.5 feature freeze, but there still is no great pressure to start a new development series. Linus
asks: could things just go on the way they are for a while yet, until enough pressure forms to force the 2.7 fork?

Bdale Garbee pointed out that, in the absence of a 2.7, many people will conclude that 2.6 has not yet stabilized sufficiently. There may be a need to do the fork just to convince people that 2.6 is ready. Alan Cox had a different idea: given that there is not a great deal of stuff to merge into 2.7, perhaps the developers could actually do a six-month release cycle for a change?

Andrew pointed out that, during the 2.6 process, he and Linus have been merging patches at a rate of about 10MB/month. There is, he says, no reason to believe that things will not continue that way. The traditional stabilization mechanism, where almost no patches are accepted for long periods of time, does not strike him as a good idea. Instead, Andrew would like to see a 2.6 tree which continues to change and evolve, and let the distributors do the final stabilization work. In his vision of the future, the kernel.org kernel will be the most featureful and fastest kernel out there, but it will not necessarily be the most stable.

The idea here is that restricting changes creates an incredible "patch pressure," which eventually leads to massive amounts of changes going into the kernel suddenly. At that point, things really do become unstable. It is better to keep the flow rate on patches higher; that keeps the developers happy and gets new code out to users quicker. Andrew really believes this: there are, seemingly, very few patches that he is not willing to accept into 2.6 - as long as they make sense and survive testing in -mm.

These patches include API changes, incidentally. Stable internal kernel APIs have never been guaranteed, but the developers have usually tried to not make big changes during a stable kernel series. That looks to change now. Among other things, it was said that API changes should be merged before an eventual 2.7 fork, since that would make synchronization between the two trees easier. Your editor, who really would like to see Linux Device Drivers not go obsolete before it hits the shelves, finds this idea somewhat dismaying.

What may happen is that Linus creates a 2.7 tree in the near future, but that tree will be restricted to truly experimental, destabilizing changes. This tree may have no future: if it doesn't work out, or can't be kept in sync with 2.6, it might simply be dropped. Or it could yet develop into 2.8, if that makes sense.

But lwn.net has only reported it to its subscribers. I'm not one of those and I don't indend to become one in predictable future. I'm sorry, but I don't consider lwn.net as a publicly available source of information.

In fact, _you_ decided to speak of this _in_ _public_, so please give away at least _some_ information that _you_ have or a link that is available to _the_ _public_.

Yours,
rjw

Rafael J. Wysocki
[tel. (+48) 605 053 693]
----------------------------
For a successful technology, reality must take precedence over public 
relations, for nature cannot be fooled.
					-- Richard P. Feynman

Adrian Bunk <[email protected]> :
[...]

It would not be nice with LWN.

Patch was submitted to -mm. -mm filters a lot lately.

Can't we have a peaceful week on l-k for once ? :o)

--
Ueimor

If essential information for contributing to the Linux kernel would be available only for paying costumers of LWN this was my last day on linux-kernel. [1]

Which patch are you talking about?
Greg's patch was made against 2.6.8-rc2 ...

cu
Adrian

[1] This is not meant against the LWN business model.
But in this case this is essential information that should have been posted to linux-kernel before the first patch appeared.

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Adrian Bunk <[email protected]> :
[...]

Greg asked Andrew to apply the patch to its own tree. So there is nothing terribly different from what has been seen for quite some time.

At worst (?) everybody will know what happens in a few days. I am not in a hurry.

--
Ueimor

Information on LWN is just delayed one week. So with a bit of patience...

Sam

[...]

Thats not how it appears to us non-subscribers. The embargo is 2 weeks, and then whatever was there under the $ marker apparently goes off to /dev/null, we can never see it. Or at least theres no obvious link that goes around that time period, allowing us full access to $ stuff after the 2 week embargo.

I cannot blme LWN for wanting to be self-supporting, but IMO its utility as a news source was severely reduced by those subscription only changes that appear to permenently restrict the non-subscriber in actual fact. This was not the announced intentions back then IIRC.

Cheers, Gene
There are 4 boxes to be used in defense of liberty. 
Soap, ballot, jury, and ammo.
Please use in that order, starting now.  -Ed Howdershelt, Author
Additions to this message made by Gene Heskett are Copyright 2004, 
Maurice E. Heskett, all rights reserved.

I'm pretty sure it is 1 week actually... and I think the $ sign goes off to /dev/null, but not the stuff that was under it.

I've been non-subscriber for a long period and never had trouble finding old 'subscriber-only' articles. So you are doing something wrong.

Sam

On Thursday 22 July 2004 17:06, [email protected] wrote:
[...]

Right, and I've already apologized to Jon privately. I just wasn't doing the obvious because the last time I tried it, well over a year ago, it didn't work. My mistake. Sorry for the noise

Cheers Sam, Gene
There are 4 boxes to be used in defense of liberty. 
Soap, ballot, jury, and ammo.
Please use in that order, starting now.  -Ed Howdershelt, Author
Additions to this message made by Gene Heskett are Copyright 2004, 
Maurice E. Heskett, all rights reserved.

It's utility as a news-source is such that it's well worth splashing out on the $5/month for a subscription.

regards,

Paul Jakma	[email protected]	[email protected]	Key ID: 64A2FF6A
Fortune:
You need no longer worry about the future.  This time tomorrow you'll be dead.

There anything more about this in print?

Thanks,

Martin Schlemmer

On Jul 21 2004, at 17:19, Jesse Stockall was caught saying:

If the devfs user community really wants to keep using devfs, they can volunteer to maintain it as a separate project & patch.

A single maintained solution/implementation for a given problem in the main tree. This is a good thing.

~Deepak

Deepak Saxena - dsaxena at plexity dot net - http://www.plexity.net/

"Unlike me, many of you have accepted the situation of your imprisonment and
 will die here like rotten cabbages." - Number 6

The new Debian installer requires devfs on several architectures, even for 2.6 installs. That's unlikely to get changed before release.

Matthew Garrett | [email protected]

file a bug and get the clueless to fix it

--cw

Great, if you want to rely on this, and you get around to using a kernel that doesn't have it in it, just add it to the other patches you apply to your kernel.

thanks,

greg k-h

Oh, sure, I've no objection to the damn thing dying - it was just to point out that (rightly or wrongly) there /is/ someone using it.

Matthew Garrett | [email protected]

What's keeping those architectures from switching to udev?

Debian     - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
Subversion - http://subversion.tigris.org/
WatchGuard - http://www.watchguard.com/

The Debian installer did not have a choice: until very recently udev was not quite up to the task and having all devices on a filesystem simply takes too much space. So for the next Debian release the installer will have to rely on devfs. But since it is based on 2.4 kernels anyway that isn't a problem.

For the release after sarge d-i will probably have to switch to a 2.6 kernel with hotplug, but that is a long way off at the moment.

Wichert.

Wichert Akkerman <[email protected]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

IMHO, udev was realy since beginning March already - prob earlier. Sure, some device drivers did not support it (and some still do not), but there are others ways to get past that ...

Martin Schlemmer

pretty nice - i like it