ext2fs corruption again
Hello.
For about 3 weeks I sent a report that I've got very strange kernel error messages.
I changed my harddrive to IBM 75 GB because someone said that IBM's 40 GB harddisks are not very stable.
Today I've got these from the kernel (with the new hd):
Sep 15 10:01:58 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4215 Sep 15 10:01:58 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4217 Sep 15 10:01:59 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4234 Sep 15 10:01:59 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4236 Sep 15 10:01:59 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4239 Sep 15 10:02:03 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4847 Sep 15 10:02:03 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4848 Sep 15 10:02:03 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4852 Sep 15 10:02:03 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4855 Sep 15 10:02:06 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_new_block: Allocating block in system zone - block = 174 Sep 15 10:02:06 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: Freeing blocks in system zones - Block = 179, count = 3 Sep 15 10:02:09 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4839
Then I did an e2fsck on that device (hda5) and the errors occured after the check (and a complete reboot) again:
Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4163 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4166 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4131 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4132 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4155 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4156 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4157 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4161 Sep 15 10:10:38 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 4162 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 716 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 717 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 720 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 723 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 724 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 725 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 726 Sep 15 10:10:43 adlib kernel: EXT2-fs error (device ide0(3,5)): ext2_free_blocks: bit already cleared for block 727
I've written down all e2fsck messages by hand. ;-) And I compared them.
The following messages from e2fsck are always the same even on the old and on the new hd. Here they are:
Duplicate/bad bock(s) in inode: 97: 643 Duplicate/bad bock(s) in inode: 100: 649 Duplicate/bad bock(s) in inode: 101: 650 651 Duplicate/bad bock(s) in inode: 102: 652 Duplicate/bad bock(s) in inode: 103: 653 656 Duplicate/bad bock(s) in inode: 104: 659 660 Duplicate/bad bock(s) in inode: 105: 661 662 663 664 665 666 Duplicate/bad bock(s) in inode: 106: 667 668 Duplicate/bad bock(s) in inode: 107: 669 671 Duplicate/bad bock(s) in inode: 108: 672 673 674 Duplicate/bad bock(s) in inode: 110: 678
Inodes 643-678 are always connected to faults.
The following files are always in connection with these errors:/var/log/wtmp/var/log/messages
The old hd was hda: IBM-DTLA-305040, ATA DISK drive. The new is: hda: IBM-DTLA-307075, ATA DISK drive.
hdparm says:
Model=IBM-DTLA-307075, FwRev=TXAOA50C, SerialNo=YSDYSFN9998 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=40 BuffType=DualPortCache, BuffSize=1916kB, MaxMultSect=16, MultSect=8 CurCHS=17475/15/63, CurSects=-78446341, LBA=yes, LBAsects=150136560 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 *udma2
AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-5 T13 1321D revision 1 : ATA-2 ATA-3 ATA-4 ATA-5
I currently use linux 2.4.9 and e2fsprogs 1.23 and fileutils-4.1 and a modified RedHat 6.2. These errors only occured with linux>=2.4.5-ac11.
I might say this is definitely an error with ext2 !
Kristian
·· · · reach me :: · ·· ·· · · ·· · ·· · ··· · ·
:: http://www.korseby.net :: http://www.tomlab.de
[email protected] ....::
Quoted 7 lines · Kristian
Hello.
For about 3 weeks I sent a report that I've got very strange kernel error messages.
I changed my harddrive to IBM 75 GB because someone said that IBM's 40 GB harddisks are not very stable.
Just to get it out of the way, can you open your computer and check what country the disk is manufactured in? There has been some complaints on this list about IBM-disks fabricated in Hungary.
/David Weinehall
_ _
// David Weinehall <[email protected]> /> Northern lights wander \\
// Project MCA Linux hacker // Dance across the winter sky // \> http://www.acc.umu.se/~tao/ </ Full colour fire </
Quoted 6 lines · David Weinehall
Quoted 5 lines · Kristian
Hello.
For about 3 weeks I sent a report that I've got very strange kernel error messages.
Quoted 6 lines
Quoted 2 lines
I changed my harddrive to IBM 75 GB because someone said that IBM's 40 GB harddisks are not very stable.
Just to get it out of the way, can you open your computer and check what country the disk is manufactured in? There has been some complaints on this list about IBM-disks fabricated in Hungary.
It's not just the disks made in Hungary, I've had 3 IBM drives go bad on me in the last week after 3-4 months of operation 2 15GB 75GXPs made in Thailand (bad sectors), 1 40GB 40GV also made in Thailand (started making bad scratching noise, BIOS wouldn't detect it after that). Still have a number of the 75GXPs in service, but I'm keeping my eye on them.
Kristian's problem looks like it could be hardware problems of some sort leading to corruption.
-Dave
Quoted 8 lines · David Rees
It's not just the disks made in Hungary, I've had 3 IBM drives go bad on me in the last week after 3-4 months of operation 2 15GB 75GXPs made in Thailand (bad sectors), 1 40GB 40GV also made in Thailand (started making bad scratching noise, BIOS wouldn't detect it after that). Still have a number of the 75GXPs in service, but I'm keeping my eye on them.
Kristian's problem looks like it could be hardware problems of some sort leading to corruption.
I think so. Someone send me a link where there was described that especially that 75 GB drives are causing such severe corruption.
But that drives seem to have these errors from the beginning. I just putted off the packaging yesterday of that 75 GB drive. Mostly that errors occured when the disk was totally off for a moment and only on my root-partition.
Is it possible to detect which file currently own a specific inode ?
Thanks anyway.
Kristian
·· · · reach me :: · ·· ·· · · ·· · ·· · ··· · ·
:: http://www.korseby.net :: http://www.tomlab.de
[email protected] ....::
Signature · 4 lines
·· · · reach me :: · ·· ·· · · ·· · ·· · ··· · ·
:: http://www.korseby.net
:: http://www.tomlab.de
[email protected] ....::Hi!
Quoted 11 lines
Quoted 7 lines
Hello.
For about 3 weeks I sent a report that I've got very strange kernel error messages.
I changed my harddrive to IBM 75 GB because someone said that IBM's 40 GB harddisks are not very stable.
Just to get it out of the way, can you open your computer and check what country the disk is manufactured in? There has been some complaints on this list about IBM-disks fabricated in Hungary.
Install crc loop device, and if disk does silent errors, you'll know.
Pavel
drivers/block/loop.c +108 -3
@@ -69,6 +69,7 @@#include <linux/slab.h> #include <asm/uaccess.h> +#include <asm/checksum.h> #include <linux/loop.h>
@@ -107,7 +108,6 @@ in = loop_buf; out = raw_buf; } - key = lo->lo_encrypt_key; keysize = lo->lo_encrypt_key_size;for (i =0 ; i < size; i++)
@@ -115,11 +115,106 @@return 0 ; } -static int none_status (struct loop_device *lo,struct loop_info *info) +#define ID printk(KERN_ERR "crc: info about (%s, %d, %d) ", kdevname(lo->lo_device), real_block, blksize); + + +static int transfer_crc (struct loop_device *lo,int cmd,char *raw_buf, +char *loop_buf,int size,int real_block) { +struct buffer_head *bh; +int blksize =1024 , nsect;/* Size of block on auxilary media */ +int cksum; +u32 *data; + nsect = blksize /4 ; + +if (!lo->second_device) { +ID ;printk ("reading from not-yet-setup crc device can result in armagedon. Dont try again.\n" ); +return -1 ; + } + bh =getblk (lo->second_device,1 +real_block/nsect, blksize); +if (!bh) { +ID ;printk ("getblk returned NULL.\n" ); +return -1 ; + } +if (!buffer_uptodate (bh)) { +ll_rw_block (READ ,1 , &bh); +wait_on_buffer (bh); +if (!buffer_uptodate (bh)) { +ID ;printk ("could not read block with CRC\n" ); + goto error; + } + } + + data = (u32 *) bh->b_data; +if (cmd ==READ ) + cksum =csum_partial_copy_nocheck (raw_buf, loop_buf, size,0 ); +else + cksum =csum_partial_copy_nocheck (loop_buf, raw_buf, size,0 ); + +if (cmd ==READ ) { +if (le32_to_cpu (data[real_block%nsect]) != cksum) { +if (lo->lo_encrypt_key_size ==0 ) {/* Normal mode */ +ID ;printk ("wrong checksum reading, is %x, should be %x\n" , cksum,0x1234 ); + goto error; + }else { +ID ;printk ("wrong checksum repairing, setting to %x\n" , cksum ); + goto repair; + } + } + }else { + repair: + data[real_block%nsect] =cpu_to_le32 (cksum); +mark_buffer_uptodate (bh,1 ); +mark_buffer_dirty (bh); + } + +brelse (bh);return 0 ; +error: +brelse (bh); +return -1 ; + } +static int ioctl_crc (struct loop_device *lo,int cmd,unsigned long arg) +{ +struct file *file; +struct inode *inode; +int error; + +printk ("Entering ioctl_crc\n" ); +if (cmd !=LOOP_CRC_SET_FD ) +return -EINVAL ; + + error = -EBADF ; + file =fget (arg); +if (!file) +return -EINVAL ; + + error = -EINVAL ; + inode = file->f_dentry->d_inode; +if (!inode) { +printk (KERN_ERR "ioctl_crc: NULL inode?!?\n" ); + goto out; + } + +if (S_ISBLK (inode->i_mode)) { + error =blkdev_open (inode, file); + lo->second_device = inode->i_rdev; +printk ("loop_crc: Registered device %x\n" , lo->second_device ); +return error; + }else { + out: +fput (file); +return -EINVAL ; + } +} + +static int none_status (struct loop_device *lo,struct loop_info *info) +{ +return 0 ; +} +static int xor_status (struct loop_device *lo,struct loop_info *info) {if (info->lo_encrypt_key_size <=0 )
@@ -139,10 +234,19 @@ init: xor_status }; +struct loop_func_table crc_funcs = { + number:LO_CRYPT_CRC , + transfer: transfer_crc, + init: none_status, + ioctl: ioctl_crc +}; +/* xfer_funcs[0] is special - its release function is never called */ struct loop_func_table *xfer_funcs[MAX_LO_CRYPT ] = { &none_funcs, - &xor_funcs + &xor_funcs, +NULL ,NULL ,NULL ,NULL ,NULL , + &crc_funcs, };#define MAX_DISK_SIZE 1024*1024*1024
@@ -728,6 +832,7 @@ lo->transfer =NULL ; lo->ioctl =NULL ; lo->lo_device =0 ; + lo->second_device =0 ; lo->lo_encrypt_type =0 ; lo->lo_offset =0 ; lo->lo_encrypt_key_size =0 ;
include/linux/loop.h +3 -0
@@ -28,6 +28,7 @@int lo_number;int lo_refcnt;kdev_t lo_device; +kdev_t second_device;int lo_offset;int lo_encrypt_type;int lo_encrypt_key_size;
@@ -119,6 +120,7 @@#define LO_CRYPT_BLOW 4#define LO_CRYPT_CAST128 5#define LO_CRYPT_IDEA 6 +#define LO_CRYPT_CRC 7#define LO_CRYPT_DUMMY 9#define LO_CRYPT_SKIPJACK 10#define MAX_LO_CRYPT 20
@@ -150,5 +152,6 @@#define LOOP_CLR_FD 0x4C01#define LOOP_SET_STATUS 0x4C02#define LOOP_GET_STATUS 0x4C03 +#define LOOP_CRC_SET_FD 0x4C04#endif
Signature · 2 lines
I'm [email protected]. "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [email protected]
Quoted 1 line · Pavel Machek
Install crc loop device, and if disk does silent errors, you'll know.
Where do you store the CRCs? It appears that they are written to another block device. Also, how do you initialize the CRC table for an existing filesystem?
What would make this considerably more useful is to be able to write the CRCs into a regular file, as it would be a bit of a pain to have a partition for each CRC loop device to store the CRCs in.
Otherwise, it looks very useful, and could be handy in tracking down reports like this where it is unclear where the data corruption is.
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- DogbertHi!
Quoted 5 lines
Quoted 1 line
Install crc loop device, and if disk does silent errors, you'll know.
Where do you store the CRCs? It appears that they are written to another block device. Also, how do you initialize the CRC table for an existing filesystem?
You just read the device in special mode. That forces it to recompute crc-s.
Quoted 3 lines
What would make this considerably more useful is to be able to write the CRCs into a regular file, as it would be a bit of a pain to have a partition for each CRC loop device to store the CRCs in.
By -o loop, you can turn regular file into blockdevice ;-).
Quoted 2 lines
Otherwise, it looks very useful, and could be handy in tracking down reports like this where it is unclear where the data corruption is.
Pavel
Signature · 2 lines
I'm [email protected]. "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [email protected]