Recovery raid5 after sata cable failure.
Hi all,
I have raid5 array working without problem for some months. A SATA cable
failed and de raid5 works fine keeping the superblock persistent, but now, I can't get the old device inserted into the array.
This is the array just now:
root@Torero-2:/mnt/raid5 # mdadm --detail /dev/md0 /dev/md0:
Version : 00.90.01
Creation Time : Tue May 31 19:37:37 2005
Raid Level : raid5
Array Size : 1367507456 (1304.16 GiB 1400.33 GB)
Device Size : 195358208 (186.31 GiB 200.05 GB) Raid Devices : 8
Total Devices : 7
Preferred Minor : 0
Persistence : Superblock is persistent
Update Time : Sun Jul 17 13:06:17 2005
State : clean, degraded
Active Devices : 7
Working Devices : 7
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
UUID : c4ed8e45:2a036953:92bff479:7cf5bac9
Events : 0.162797
Number Major Minor RaidDevice State
0 8 1 0 active sync /dev/sda1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 3 active sync /dev/sdd1
4 8 65 4 active sync /dev/sde1
5 8 81 5 active sync /dev/sdf1
6 8 97 6 active sync /dev/sdg1
7 0 0 - removed
And I try to re-add the old disk in this way:
root@Torero-2:/mnt/raid5 # mdadm /dev/md0 -a /dev/sdh1
mdadm: Cannot open /dev/sdh1: Device or resource busy
What is wrong? What I am doing bad? Sdh1 is absolutely unused, so I
don't understand the error "resource busy"
Thanks,
Paco Zafra.
Quoted 5 lines · On Sunday July 17, [email protected] wrote:
Hi all,
I have raid5 array working without problem for some months. A SATA cable
failed and de raid5 works fine keeping the superblock persistent, but now, I can't get the old device inserted into the array.
...
Quoted 8 lines
And I try to re-add the old disk in this way:
root@Torero-2:/mnt/raid5 # mdadm
/dev/md0-a/dev/sdh1
mdadm: Cannot open/dev/sdh1: Device or resource busyWhat is wrong? What I am doing bad? Sdh1 is absolutely unused, so I
don't understand the error "resource busy"
Well, it definitely is busy...
Maybe it is still part of md0, but marked as 'faulty'.
If so (cat /proc/mdstat would tell you) you need to remove it first.
mdadm /dev/md0 -r /dev/sdh1
mdadm /dev/md0 -a /dev/sdh1
NeilBrown
I already tried that:
root@Torero-2:~ # cat /proc/mdstat
Personalities : [linear] [raid5]
md0 : active raid5 sdg1[6] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
1367507456 blocks level 5, 512k chunk, algorithm 2 [8/7] [UUUUUUU_]
unused devices: <none>
root@Torero-2:~ # mdadm /dev/md0 -r /dev/sdh1
mdadm: hot remove failed for /dev/sdh1: No such device or address root@Torero-2:~ # mdadm /dev/md0 -a /dev/sdh1
mdadm: Cannot open /dev/sdh1: Device or resource busy root@Torero-2:~ #
With no luck :(
Paco.
Quoted 35 lines
-----Mensaje original-----
De: Neil Brown [mailto:[email protected]] Enviado el: lunes, 18 de julio de 2005 0:11
Para: Francisco Zafra
CC: [email protected]
Asunto: Re: Recovery raid5 after sata cable failure.Quoted 3 lines · On Sunday July 17, [email protected] wrote:
Hi all,
I have raid5 array working without problem for some
months. A SATA
Quoted 2 lines
cable failed and de raid5 works fine keeping the superblock persistent, but now, I can't get the old device inserted
into the array.
...Quoted 7 lines
And I try to re-add the old disk in this way:
root@Torero-2:/mnt/raid5 # mdadm
/dev/md0-a/dev/sdh1
mdadm: Cannot open/dev/sdh1: Device or resource busyWhat is wrong? What I am doing bad? Sdh1 is absolutely
unused, so I
Quoted 1 line
don't understand the error "resource busy"
Well, it definitely is busy...
Maybe it is still part of md0, but marked as 'faulty'.
If so (cat/proc/mdstatwould tell you) you need to remove it first.
mdadm/dev/md0-r/dev/sdh1
mdadm/dev/md0-a/dev/sdh1NeilBrown
Quoted 5 lines · On Monday July 18, [email protected] wrote:
I already tried that:
root@Torero-2:~ # cat
/proc/mdstat
Personalities : [linear] [raid5]
md0 : active raid5 sdg1[6] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
Quoted 7 lines
1367507456 blocks level 5, 512k chunk, algorithm 2 [8/7] [UUUUUUU_]unused devices: <none>
root@Torero-2:~ # mdadm/dev/md0-r/dev/sdh1
mdadm: hot remove failed for/dev/sdh1: No such device or address root@Torero-2:~ # mdadm/dev/md0-a/dev/sdh1
mdadm: Cannot open/dev/sdh1: Device or resource busy
Uhm, you might have a buggy version of mdadm. If you have 1.10.0, get an upgrade.
Otherwise either sdh1 or sdh must be:
open by some process with O_EXCL
open via a /dev/raw/* device
part of an md device (which it obviously isn't)
part of a dm device
mounted as a filesystem
an external-journal device for a jfs or ext3 or xfs filesystem in use as a swap device
open for writing under a security level of 1 (whatever that means..) an mtd device that is open
(those are all the places that I can find that take an exclusive lock
on a block device).
NeilBrown
Hi Neil,
Since some hours I am trying to solved it with the last version:
root@Torero-2:~ # mdadm --version
mdadm - v2.0-devel-2 - DEVELOPMENT VERSION NOT FOR REGULAR USE - 7 July 2005
With the same results :(
I really don't think it is locked I dd it in act of desperation and I have no problems:
root@Torero-2:~ # dd if=/dev/zero of=/dev/sdh bs=1k count=1000 1000+0 records in
1000+0 records out
1024000 bytes transferred in 0.417862 seconds (2450570 bytes/sec)
No locked or anything... I have really get out of ideas with this...
Thanks for all your help.
Paco.
Quoted 43 lines
-----Mensaje original-----
De: Neil Brown [mailto:[email protected]] Enviado el: lunes, 18 de julio de 2005 11:41
Para: Francisco Zafra
CC: [email protected]
Asunto: RE: Recovery raid5 after sata cable failure.Quoted 5 lines · On Monday July 18, [email protected] wrote:
I already tried that:
root@Torero-2:~ # cat
/proc/mdstat
Personalities : [linear] [raid5]
md0 : active raid5 sdg1[6] sdf1[5] sde1[4] sdd1[3] sdc1[2]sdb1[1] sda1[0]
Quoted 8 lines
1367507456 blocks level 5, 512k chunk, algorithm 2 [8/7][
UUUUUUU_]unused devices: <none>
root@Torero-2:~ # mdadm/dev/md0-r/dev/sdh1
mdadm: hot remove failed for/dev/sdh1: No such device or address root@Torero-2:~ # mdadm/dev/md0-a/dev/sdh1
mdadm: Cannot open/dev/sdh1: Device or resource busyUhm, you might have a buggy version of mdadm. If you have 1.10.0, get an upgrade.
Otherwise either sdh1 or sdh must be:
open by some process withO_EXCL
open via a/dev/raw/*device
part of an md device (which it obviously isn't)
part of a dm device
mounted as a filesystem
an external-journal device for a jfs or ext3 or xfs filesystem in use as a swap device
open for writing under a security level of 1 (whatever that means..) an mtd device that is open(those are all the places that I can find that take an exclusive lock on a block device).
NeilBrown
Quoted 4 lines · Francisco Zafra
Hi Neil, Since some hours I am trying to solved it with the last version:
root@Torero-2:~ # mdadm --version
mdadm - v2.0-devel-2 - DEVELOPMENT VERSION NOT FOR REGULAR USE - 7 July 2005
Quoted 9 lines
With the same results :(
I really don't think it is locked I dd it in act of desperation and I have no problems:
root@Torero-2:~ # dd if=/dev/zero of=/dev/sdh bs=1k count=1000 1000+0 records in
1000+0 records out
1024000 bytes transferred in 0.417862 seconds (2450570 bytes/sec)
Asking a silly question perhaps..
fuser /dev/sdh
Regards,
Brad
Signature · 3 lines
"Human beings, who are almost unique in having the ability to learn from the experience of others, are also remarkable for their apparent disinclination to do so." -- Douglas Adams
safe to test SATA array by pulling cables?
In reply to Francisco ZafraI've read conflicting views on whether it's safe to pull either or both the SATA data cable or power cable from a disk in an array (when they are NOT in a hotswap cage) to test whether things works as expected during a real life disk failure.
Is there a consensus on this? Theoretically either should be handled by the controller (a 3ware 9500s) since this is a real life possibility, but will either event cause damage to the disk? I'd suspect that removal of the data cable would simulate a disk loss with less physical trauma to the disk in question.
Signature · 3 lines
Cheers, Harry Harry J Mangalam - 949 856 2847 (vox; email for fax) - [email protected] <<plain text preferred>>