Skip to contentSkip to conversation

Linux stability despite unstable hardware

OT Timothy Miller 6 participants 6 messages May 21, 2004 – May 24, 2004

lore
Share & tools
lore
#0 lore raw

I have had some issues recently with memory errors when using aggressive memory timings. Although memory tests pass fine, gcc would tend to crash and would generate incorrect code when compiling other things. Gcc couldn't even build itself properly under those conditions.

The really interesting thing is that the Linux kernel was totally unaffected. Compiling the Linux kernel is often thought of as a stressful thing for a system, yet compiling a kernel with a broken gcc on a system with intermittent memory errors goes through error free, and the kernel is 100% stable when running.

But until the memory errors were fixed, things like KDE wouldn't build without gcc crashing.

So, what is it about Linux that makes it build properly with a broken GCC and run perfectly despite memory errors?

_________________________________________________________________
Watch LIVE baseball games on your computer with MLB.TV, included with MSN Premium! http://join.msn.click-url.com/go/onm00200439ave/direct/01/

#1 lore raw
Quoted 2 lines · you

So, what is it about Linux that makes it build properly with a broken GCC and run perfectly despite memory errors?

probably just the sources... i built a bunch of gentoo on a flaky machine, and ran into trouble with mozilla. it was mozilla 1.6 that demostrated the problems and prevented me from continuing to build gnome. i ended up rebuilding the whole thing on a speedy box over a weekend, for a far weaker architecture that wasn't flaky (despite 4 or 5 brutal soldering adventures on the motherboard!!!)

Signature · 3 lines
Rob Couto [[email protected]]
computer safety tip: use only a non-conducting, static-free hammer.
--
#2 lore raw
In reply to Timothy Miller
Quoted 4 lines · Timothy Miller

I have had some issues recently with memory errors when using aggressive memory timings. Although memory tests pass fine, gcc would tend to crash and would generate incorrect code when compiling other things. Gcc couldn't even build itself properly under those conditions.

Really? memtest86 <http://www,memtest86.com> did finish its run without errors? I'm sure they'd want to get their hands on the broken RAM...

Quoted 5 lines

The really interesting thing is that the Linux kernel was totally unaffected. Compiling the Linux kernel is often thought of as a stressful thing for a system, yet compiling a kernel with a broken gcc on a system with intermittent memory errors goes through error free, and the kernel is 100% stable when running.

Sure enough, if the kernel lives (mostly) in unaffected memory, and furthermore doesn't bang on it quite as hard as a compiler run does (the kernel only runs for a tiny fraction of the time).

Signature · 4 lines
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
#3 lore raw
Quoted 8 lines · Horst von Brand
Quoted 1 line · Timothy Miller

I have had some issues recently with memory errors when using aggressive memory timings. Although memory tests pass fine, gcc would tend to crash and would generate incorrect code when compiling other things. Gcc couldn't even build itself properly under those conditions.

Really? memtest86 <http://www,memtest86.com> did finish its run without errors? I'm sure they'd want to get their hands on the broken RAM...

Or these days <http:/www.memtest.org/>

#4 lore raw
In reply to Timothy Miller
Quoted 7 lines · Timothy Miller

I have had some issues recently with memory errors when using aggressive memory timings. Although memory tests pass fine, gcc would tend to crash and would generate incorrect code when compiling other things. Gcc couldn't even build itself properly under those conditions.

The really interesting thing is that the Linux kernel was totally unaffected. Compiling the Linux kernel is often thought of as a stressful thing for a system, yet compiling a kernel with a broken gcc on a system with intermittent memory errors goes through error free, and the kernel is 100% stable when running.

But until the memory errors were fixed, things like KDE wouldn't build without gcc crashing.

So, what is it about Linux that makes it build properly with a broken GCC and run perfectly despite memory errors?

It could just be heat buildup in an critical area when under sustained heavy load. It may take a while for enough heat to build up to cause problems. I just recently found one of these. It would take 4-6 hours of heavy intensive processing before an error would happen. I placed a fan pointing at the motherboard chipset and memory to keep them cooler and the problem seams to have gone away.

For testing I wrote a script that kept compiling the kernel again and agian in a while(true) loop. Effectively a repeat until crash loop. For each compile it saved the stdout/stderr output and diffed it against the first run. Any differnces were flagged for checking latter.

- Bryan

#5 lore raw
In reply to Timothy Miller
Quoted 11 lines · Timothy Miller

I have had some issues recently with memory errors when using aggressive memory timings. Although memory tests pass fine, gcc would tend to crash and would generate incorrect code when compiling other things. Gcc couldn't even build itself properly under those conditions.

The really interesting thing is that the Linux kernel was totally unaffected. Compiling the Linux kernel is often thought of as a stressful thing for a system, yet compiling a kernel with a broken gcc on a system with intermittent memory errors goes through error free, and the kernel is 100% stable when running.

Quoted 5 lines

But until the memory errors were fixed, things like KDE wouldn't build without gcc crashing.

So, what is it about Linux that makes it build properly with a broken GCC and run perfectly despite memory errors?

Been there, seen that too.

I think it has to do with smaller files. This reduces the memory pressure and the power requirements of the overall system. Once the compiler gets loaded in memory, it stayes there, small files (without too many includes) keep the buffer requirements down, and reuses memory that may be better.

I fixed mine by just treating everything as the next lower grade memory (switched the BIOS from 60ns to 70ns). I didn't even see a significant performance reduction either.