raid5 disaster
Hi,
I had a working raid5 setup with 5 SATA disks, 3 attached to a Promise TX4 and 2 more attached to the mainboard controller.
It has been working flawlessly for a long time, but I had to add a sat card to the machine so I also upgraded to 2.6.16.16
I don't know if there was some problem with that kernel, but CPU usage was almost 100% when writing to the raid, so I decided to go back to me old and trusty 2.6.15.1, but when shutting down the system it wouldn't finish so I had to power it down.
On the next reboot I saw this:
md: Autodetecting RAID arrays. md: invalid superblock checksum on sdc1 md: sdc1 has invalid sb, not importing! md: invalid superblock checksum on sde1 md: sde1 has invalid sb, not importing! md: autorun ... md: considering sdd1 ... md: adding sdd1 ... md: adding sdb1 ... md: adding sda1 ... md: created md0 md: bind<sda1> md: bind<sdb1> md: bind<sdd1> md: running: <sdd1><sdb1><sda1> raid5: device sdd1 operational as raid disk 1 raid5: device sdb1 operational as raid disk 0 raid5: device sda1 operational as raid disk 4 raid5: not enough operational devices for md0 (2/5 failed)
RAID5 conf printout:
--- rd:5 wd:3 fd:2
disk 0, o:1, dev:sdb1
disk 1, o:1, dev:sdd1
disk 4, o:1, dev:sda1
raid5: failed to run raid set md0
So sdc1 and sde1 have an invalid superblock (I assume this was because there was some I/O activity when I switched it down).
Now, as you suppose, I'd like to access my data.
This is what I get from the faulty (and one of the working disks) with the 'examine' parameter:
/dev/sda1:
Magic : a92b4efc
Version : 00.90.03
UUID : 8e47d871:51e2f219:52b05fbf:44206fa0 Creation Time : Sat Jan 21 00:20:33 2006
Raid Level : raid5
Raid Devices : 5
Total Devices : 5
Preferred Minor : 0
Update Time : Tue May 23 19:06:48 2006
State : clean
Active Devices : 5
Working Devices : 5
Failed Devices : 0
Spare Devices : 0
Checksum : ba51512f - correct
Events : 0.3551144
Layout : left-symmetric
Chunk Size : 128K
Number Major Minor RaidDevice State
this 4 8 1 4 active sync /dev/sda1
0 0 8 17 0 active sync /dev/sdb1 1 1 8 49 1 active sync /dev/sdd1 2 2 8 65 2 active sync /dev/sde1 3 3 8 33 3 active sync /dev/sdc1 4 4 8 1 4 active sync /dev/sda1
/dev/sdc1:
Magic : a92b4efc
Version : 00.90.03
UUID : 8e47d871:00000000:00000000:260f0100 Creation Time : Sat Jan 21 00:20:33 2006
Raid Level : raid5
Raid Devices : 16777216
Total Devices : 0
Preferred Minor : 5058
Update Time : Fri Dec 13 20:45:52 1901
State : active
Active Devices : -2147483648
Working Devices : -2147483648
Failed Devices : -2147483648
Spare Devices : -2147483648
Checksum : 80000000 - expected 2255ae19
Events : -2147483648.-2147483648Floating point exception
/dev/sde1:
Magic : a92b4efc
Version : 00.90.03
UUID : 8e47d871:00000000:00000000:260f0100 Creation Time : Sat Jan 21 00:20:33 2006
Raid Level : raid5
Raid Devices : 16777216
Total Devices : 0
Preferred Minor : 5058
Update Time : Fri Dec 13 20:45:52 1901
State : active
Active Devices : -2147483648
Working Devices : -2147483648
Failed Devices : -2147483648
Spare Devices : -2147483648
Checksum : 80000000 - expected 2255ae37
Events : -2147483648.-2147483648FS on the raid is XFS.
I've been crawling through the list and noticed I could create the array again and data would still be there and I should be able to mount the fs. Am I correct?
Is this the only solution?
I've assembled this command for mdadm:
mdadm -C -l5 -n5
-c=128 /dev/md0 /dev/sdb1 /dev/sdd1 /dev/sde1 /dev/sdc1 /dev/sda1
I took the devices order from the mdadm output on a working device. Is this the way it's supposed to be the command assembled?
Is there anything alse I should consider or any other valid solution to gain access to my data?
Thanks,
Signature · 1 line
Bruno Seoane
Quoted 8 lines · Bruno Seoane
mdadm -C -l5 -n5
-c=128/dev/md0/dev/sdb1/dev/sdd1/dev/sde1/dev/sdc1/dev/sda1I took the devices order from the mdadm output on a working device. Is this the way it's supposed to be the command assembled?
Is there anything alse I should consider or any other valid solution to gain access to my data?
If you create the array, it will immediately start resyncing unless you list one of the devices in your command line as "missing". Just pick one (ideally one of the ones that isn't getting picked up anyway) and put 'missing' in its place.
Using "missing" is the only way to have it be read-only in the data regions. That'll let you make a mistake and still be able to recover data after you find the right command line.
-Mike