kernel changes
Hey guys,
I have been watching development of 2.4 since 2.4.2, I wonder wether there would be reversion to old process where kernel source will be solidified before starting development branch. Sysadmins where I work are uneasy to move to linux from solaris, because of erratic changes to the kernel such as replacement of the hardware monitoring code, rewriting of network card drivers - all of which broke some or other software, or simply did not work. I myself update kernel as time permits, usually every 0.0.2+ I have spent nearly two days tracing problem of my network, that ended up in source code of the driver that have been radically changed for "better". Now I don't trust 2.4 line kernel to work *at all*, so cautiously keep all old kernels in the /boot, when upgrading.
It seems I am not only one and wonder if something will be done about that. Regards,
p.
Signature · 4 lines
Research causes cancer in rats. 110461387 http://gpg.md5.ca http://perlpimp.com
Quoted 4 lines
been radically changed for "better". Now I don't trust 2.4 line kernel to work *at all*, so cautiously keep all old kernels in the /boot, when upgrading.
It seems I am not only one and wonder if something will be done about that.
You certainly aren't the only one. 2.4.10 both really alarms me and doesn't survive over night on my test box
Quoted 7 lines · Alan Cox
Quoted 4 lines
been radically changed for "better". Now I don't trust 2.4 line kernel to work *at all*, so cautiously keep all old kernels in the /boot, when upgrading.
It seems I am not only one and wonder if something will be done about that.You certainly aren't the only one. 2.4.10 both really alarms me and doesn't survive over night on my test box.
It would be better to be premature starting 2.5.x than to risk damaging the reputation of Linux by pretending an unproven kernel is stable.
Lets start 2.5.x and let Alan sort 2.4.x out.
Andrew
The answer is to treat all linus/ac/aa/... kernels as development kernels. Don't treat anything as stable until it's been through a real QA cycle. I've heard Suse, RedHat and the like don't do a bad job at this.
Quoted 4 lines
The answer is to treat all linus/ac/aa/... kernels as development kernels. Don't treat anything as stable until it's been through a real QA cycle. I've heard Suse, RedHat and the like don't do a bad job at this.
We try. If you want to QA your own kernel the cerberus test suite is publically available - and indeed the VA guys are to thank for its origins.
Alan
Quoted 9 lines · Alan Cox
Quoted 4 lines
The answer is to treat all linus/ac/aa/... kernels as development kernels. Don't treat anything as stable until it's been through a real QA cycle. I've heard Suse, RedHat and the like don't do a bad job at this.
We try. If you want to QA your own kernel the cerberus test suite is publically available - and indeed the VA guys are to thank for its origins.
Alan
For Linux QA/Testing, you may also want to check out the Linux Test Project. We just had a new release that added about 400 new testcases which is way up from what we had before. We also have descriptions for all the tests now too.
-Paul Larson
Quoted 5 lines · Mark Frazer
The answer is to treat all linus/ac/aa/... kernels as development kernels. Don't treat anything as stable until it's been through a real QA cycle. I've heard Suse, RedHat and the like don't do a bad job at this.
I agree, except that the earlier 2.4.0 kernels they are using are also buggy to one degree or another (for instance, the Tulip driver didn't start working for my cards untill about 2.4.8, and by then the VM seems to have gone berserk...) Hell, many of the official kernels won't even compile with certain options turned on, which is really frightening, and makes it hard to run regression tests...
It's almost like we need to fork off a 2.5 kernel just to let the pressure for massive change off, and let less panic'ed changes go into the 2.4 series...
The fact that there's a 20MB patch to get from Linus to AC is also disconcerting!!
Ben
Quoted 6 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 3 lines
Ben Greear <[email protected]> <[email protected]> President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
Quoted 3 lines · Pavel Zaitsev
Now I don't trust 2.4 line
kernel to work *at all*, so cautiously keep all old kernels in the /boot, when upgrading.
Well, it's a good idea to always keep a few old kernels in /boot but I certainly identify with your point. I like to run the bleeding edge kernels at home but lately I've been having doubts. I've been looking for a stable kernel since 2.4.0-test9. While 2.4.0-test9 is not exactly bug free either, any later kernel either reboots at random (all later test versions and early 2.4.x versions) or locks up hard on my SMP machine. This does not appear to have anything to do with load, indeed it often happens with the machine completely idle. It can take hours or days before this occurs but sooner or later it does. Of course, nothing ever gets written to the logs. With 2.4.8 I almost thought I had hit paydirt, but no such luck. 2.4.9 crashed right off the bat. 2.4.10 seems more unstable than most. Enough so that I finally patched in ext3 and installed journals on my file systems just to give them a semblance of stability (and to speed up the random reboots).
That said, I might just have a hardware problem. However, something in the new kernels seems to touch it off. Any ideas and tools to track this down, besides intuition and brute force, would be appreciated. [Right now I'm running with swap disabled, on somebody's suggestion. Let's see what happens.]
Regards,
Mika Liljeberg
Quoted 3 lines · you [Pavel Zaitsev]
I have been watching development of 2.4 since 2.4.2, I wonder wether there would be reversion to old process where kernel source will be solidified before starting development branch.
I think you can think of each new 2.4.x kernel as a candidate for solification. The part of the linux community a like me (and perhaps you?) that can't contribute directly to the kernel development in form of patches, can help by testing the kernels with a great variety of setups. Linus, Alan, all the other guys are working their asses off getting the code as good, clean and fast as possible. But you can't exactly expect them to verify each change with thousands of different machines (think of the variety of just PC hw, then think of all the other platforms linux runs on), like MS does. Even the linux driver developers don't have access to all the hw they support. MS doesn't release nearly as often and they have a huge QA department running the test suites for each change set. They also have a direct sopprt of hw vendors (who in most cases to the drivers for MS and use all theK interim knowledge and analyzer equipment in that.)
I've always thought when Linux or alan puts out a new kernel, it's up to the community to do the testing part.
If you need something stable, stick with 2.2 or something that has gone through a test cycle like the RH kernels.
If you have the hardware, help testing and report bugs.
That's the way I always figured Linux community works, and IMHO considering the facts, it does result in pretty impressive development pace and even stability (much thanks to the pure skill the developers have).
It does, however, seem that in 2.4 the pace has been so rapid that for each 2.4.n bug found and fixed in 2.4.(n+1), there have been a few new bugs in 2.4.(n+1). I'd expect that to get better as 2.5 forks of and the 2.4 pace slows down.
Just my two cents.
Quoted 4 lines · Dan Maas
e.g. I consider it extremely embarrassing that fundamental drivers like aic7xxx, emu10k1, tulip, etc. are breaking regularly in the mainline kernels. I haven't had any trouble with things like this in Windows for several years now...
Well. I just installed the SRP (security rollup patch MS came up with instead of releasing proper SP7) on a NT4SP6 box. The SRP erraneously detected the box was SMP and installed SMP versions for 2 out of 6 kernel files. Needless to say it didn't work and was a pain to trace and fix.
Similarly, I just updated to the newest NVidia display driver for NT4. It didn't boot. Not even to VGA mode. BSOD right in start. It took me hours and a non-NVidia display adapter to clean up the mess. And that driver had been QA'd at NVidia for months.
So shit happens on (supposedly stable version of) windows as well. And it is usually more of a pain to clean up, since MS gives you little tools to work with in case of such problem.
Of course that's not an excuse and I'd certainly like to see 2.4 to be more stable.
-- v --
Quoted 7 lines · Ville Herva
Quoted 3 lines · you [Pavel Zaitsev]
I have been watching development of 2.4 since 2.4.2, I wonder wether there would be reversion to old process where kernel source will be solidified before starting development branch.
I think you can think of each new 2.4.x kernel as a candidate for solification. The part of the linux community a like me (and perhaps you?)
One aspect that bothers me is the absence of a success criteria. The current competition for best VM is a good example. The fact is that every operating system will fail with a high enough load. The best you can hope for is a better degradation then the prior release. At the moment both 2.4.10 and 2.4.9-ac16 are better then 2.2.19. But people keep testing under higher and higher loads and (surprise) they both fail... initiating a search for better degradation logic.
Without a success criteria, this process cannot end.
Other examples are recent updates for multi-quad NUMA machines and changes to handle locking problems on 8-way machines.
john alvord
Quoted 1 line · In article <[email protected]> you wrote:
One aspect that bothers me is the absence of a success criteria.
I disagree here. Red Hat uses "must pass the cerberus test" as one of the criteria for kernels. The are other similar criteria, most are obvious (must boot :). All other distributions have similar tests and a few even use the cerberus testsuite as well.
Maybe your problem is "absence of tests before Linus releases", well even that isn't fully true as distros run these tests on -pre kernels as well (or -ac kernels, which are mostly in sync with -pre kernels)...
Quoted 3 lines
The current competition for best VM is a good example. The fact is that every operating system will fail with a high enough load. The best you can hope for is a better degradation then the prior release.
There are a few basic creteria here as well, and 2.4.10 fails on some of them so far:
1) Must not kill processes as long as there is plenty of swap or (possibly dirty) cache memory
2) Must not deadlock (as that is a code-bug)
3) Must not livelock without any progress
Note that no 2.4 kernel so far really achieves 1) in the presence of highmem; the obvious deadlocks are just pushed further by tuning.
Quoted 3 lines
At the moment both 2.4.10 and 2.4.9-ac16 are better then 2.2.19. But people keep testing under higher and higher loads and (surprise) they both fail... initiating a search for better degradation logic.
2.4.10 isn't better than 2.2.19 given the criteria above. 2.4.10aa2 might be though... and 2.4.9acX+Rik's patches are solid in testing.
Greetings,
Arjan van de Ven
Quoted 8 lines · On Sat, 29 Sep 2001 18:23:15 +0100, [email protected] wrote:
Quoted 1 line · In article <[email protected]> you wrote:
One aspect that bothers me is the absence of a success criteria.
I disagree here. Red Hat uses "must pass the cerberus test" as one of the criteria for kernels. The are other similar criteria, most are obvious (must boot :). All other distributions have similar tests and a few even use the cerberus testsuite as well.
I believe you. I was worrying about the developer transition from 2.4 to 2.5, not the excellent work that the distributors do.
Quoted 4 lines
Maybe your problem is "absence of tests before Linus releases", well even that isn't fully true as distros run these tests on -pre kernels as well (or -ac kernels, which are mostly in sync with -pre kernels)...
As mentioned, I am not worried about the distributions.
Quoted 5 lines
Quoted 4 lines
The current competition for best VM is a good example. The fact is that every operating system will fail with a high enough load. The best you can hope for is a better degradation then the prior release.
Quoted 7 lines
There are a few basic creteria here as well, and 2.4.10 fails on some of them so far:
1) Must not kill processes as long as there is plenty of swap or (possibly dirty) cache memory
2) Must not deadlock (as that is a code-bug)
3) Must not livelock without any progress
Is this criteria explicity accepted by all parties? And is it the only criteria?
Quoted 10 lines
Note that no 2.4 kernel so far really achieves 1) in the presence of highmem; the obvious deadlocks are just pushed further by tuning.
Quoted 3 lines
At the moment both 2.4.10 and 2.4.9-ac16 are better then 2.2.19. But people keep testing under higher and higher loads and (surprise) they both fail... initiating a search for better degradation logic.
2.4.10 isn't better than 2.2.19 given the criteria above. 2.4.10aa2 might be though... and 2.4.9acX+Rik's patches are solid in testing.
I read all sorts of reports, many positive and some negative.
john alvord
Quoted 4 lines · --On Saturday, September 29, 2001 09:45 AM -0700 John Alvord <[email protected]>…
...
Without a success criteria, this process cannot end.Other examples are recent updates for multi-quad NUMA machines
I'm not sure why these bother you - if you don't turn on CONFIG_MULTIQUAD, the code impact is practically non-existant.
Martin.