File Corruption in Kernel 2.4.18

A large directory tree (70652 files, 7.6G) is copied recursively to an
empty destination directory using the following commands :

mkdir aminet1/
cp -a aminet aminet1/

The source and destination directories are then compared using
the following commands:

diff -r aminet aminet1/aminet > difflist

A few of the files at the copy destination, typically three or four, will
usually be corrupt while the source files will be correct. Occasionally the copy will be done without any corrupt files at the destination. The mem=nopentium option appears to have no effect on this. An overnight test using the memtest86 utility shows no memory errors. The corruption in each file occurs in precise 4096 byte blocks. An overnight test using the memtest86 utility shows no memory errors. The corruption in each file occurs in precise 4096 byte blocks. System logs show no evidence of any trouble, and no kernel panics, warning messages or crashes are observed. If there is any other user activity while the copy is running, the system will frequently lock up requiring a hard reset and reboot. This forces a file system check due to the lack of a clean unmount. System logs also show no evidence of any trouble after the lockup, and no kernel panics or other messages have been observed.

If a tar file is made of the source directory and then extracted, and the
resultant extracted directory compared with the original, similar effects are observed.

Are there any kernel boot or build parameters which could be used
to give additional diagnostics ?

motherboard : ASYS-A7V
Linux version : Slackware 8
Kernel : 2.4.18
hard disk : ATA100 IBM-DTLA-307045 45gb
hd controller : Promise Technology, Inc. 20265
cpu : 900mhz AMD Athlon

This could possibly be a problem with your hard drive. Judging from the model number, you have a 45GB IBM DeskStar 75GXP, one of the first IBM drives to earn the nickname "DeathStar" for its high failure rate. What does IBM's Drive Fitness Test tell you?

I'll see about performing your test tonight; I've got a hefty little DivX directory I can throw around as I wait for j2sdk-1.4.0 to finish compiling. Such a test should be sufficient...

This could also be a recurrence of ye olde VIA686B PCI+IDE issue. IIRC, some VIA686B motherboards that had that flaw were effectively unfixable, simply because certain motherboard manufacturers spotted the problem before everyone else (even VIA?) and tried their own partial kludge fixes for it. Gotta love VIA.

Kelledin
"If a server crashes in a server farm and no one pings it, does 
it still cost four figures to fix?"

Asus A7V is Via KT133 based, right? It has additional Promise ide controller?

Stock 2.4.18, no patches? Which filesystem are you using? Ext2, ext3, other?

So the harddisk is connected to Promise, not Via? You have no other harddisks?

I had enormous trouble with a KT133(A or not) based mobo (Abit-KT7(A)-RAID in past - it would just corrupt data when transferring big files from the additional ide controller (HPT370 in this case). The Via ide controller didn't show this behaviour.

- This happened on 2.2.20, 2.4.15, 2.4.18preX + ide-patch.
- Memtest86 showed nothing
- Network activity seemed to have to do with it
- Changing the NIC to another PCI slot and tweaking bios params seemed to help, but eventually it happened again
- I eventually concluded that KT133 corrupts PCI transfers under load, which was found out by others in 'net as well.
- Tried bios updates and contacting Via, Highpoint, Abit. Highpoint and Abit never cared to answer. Neither did Via until I spotted an Via employee on viahardware.com forum. She said they'd investigate the issue, never heard of her since.
- Ditched the mobo fo good, bought an Abit ST6R, and never had a problem since. You may be lucky just switching the drive to Via ide.

First reports on the corruption:

http://marc.theaimsgroup.com/?l=linux-kernel&m=100651892331843&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=100669782329815&w=2
http://groups.google.com/groups?q=We+first+reported+disk+corruption+with+a+VIA+KT133A+based+board&hl=en&lr=&ie=UTF-8&oe=utf-8&selm=linux.kernel.00c201c1a033%241cf46700%24b71c64c2%40viasys.com&rnum=3

There was a long thread on forums.viahardware.com as well
(http://forums.viaarena.com/messageview.cfm?catid=6&threadid=7171&start=21),
but it seems they have sensored it away for good.

I've also received reports of similar experiences from a number of people since I wrote to linux-kernel about this.

I repoduced the problem with wrchk utility I wrote
(http://iki.fi/v/tmp/wrchk.c) but it seems you can do it with you directory tree copying.

-- v --

[email protected]

I got a Abit-KT7-RAID with a AMD Thunderbird 800 and also have seen lots of trouble. Finally I have figured out that reducing the memory bus clock to 100MHz (instead of 133MHz) make my system pretty stable (My memory modules can take 133MHz! I checked the specs.). Maybe that chipset memory tweaks that the linux kernel does are not enough to fix all memory problems...

[snip]

Well, after messing around with the mobo for almost 2 years, it finally seems to be stable. But I wish I could have done useful stuff with my computer during that time.

[snip]

I got to check this out!

bye,
Wilfried

I must apologize for the delay in replying to the many helpful
responses I received on this problem, and I'd like to say "thank you very much" (Domo Arigato Gozaimasu) to the many who looked into this on my behalf....:-)

Here's the status so far:

I ran the e2fcsk utility which did not detect any problems on the
hard drive. I had not known about the IBM DFT utility until it was suggested by one of the responses, so I picked that up and tried it. I ran the quick test, which immediately indicated two corrupt sectors. I ran the Corrupt Sector Repair Utility (which does not seem to be documented in the manual that comes with DFT) after backing up, and repeated the tests a couple of times. There were no more complaints from DFT. I will be reloading my test directory, which I had to dump before the backup, and I will repeat the copy test on Monday after that to see if the file corruption still occurs.

I do not know what caused the corrupted sectors, but I am giving
serious thought to a new mother board (to get rid of the chipset and Promise controller), and perhaps a replacement for the "IBM DeathStar". I'll let you all know the outcome of the tests on Monday. I am still curious about the precise 4k damaged blocks.

I did a simultaneous:
wrck /dev/hd[fh] 0 64 2
The two disks were connected to the HPT-370 controller and both were configured as slave (the masters are configured into an ataraid-0 and contain my root partition). The test disk were IBM DLTA 307030 (30GB) with updated firmware. These disks are locked down to ata-44 by the kernel and I only got a maximum I/O speed of 21.7 MB/s. During the read phase one of the disks always slowed down, while the other disk proceeded at normal speed. In the first run I got 7.2 MB/s and at the second run the other disk slowed down to crawling 5.3 MB/s, but the test was completed without any errors. *joy* However I am not that the test did stress the chipset enough to trigger the error because of the throughput is so low. My mainboard is a abit kt7-raid (VIA KT133 chipset), BIOS version 3R. Memory bus was reduced to 100 MHz (SDR). Linux kernel 2.4.18 tainted by NVidia(TM). ;)
DivX 5.0 seems to be a good stability test for VIA chipset based motherboards. It finds errors that not even memtest could detect.

greetings,
Wilfried

PS: I will do another run on my raid-0 root partition. The 2 disks that are part of the raid run at ata-100 (Maxtor 40GB).