Corrupted ext4 filesystem after mdadm manipulation error

Hi,

For the third time, I had to change a failed drive from my home linux RAID5 box. Previous one went right and this time, I don't know what I did wrong, but I broke my RAID5. Well, at least, he didn't want to start. /dev/sdb was the failed drive /dev/sdc and /dev/sdd are OK.

I tried to reassemble the RAID with this command after I replace sdb and create a new partition :

~# mdadm -Cv /dev/md0 --assume-clean --level=5 --raid-devices=3 /dev/sdc1 /dev/sdd1 /dev/sdb1
-> '-C' was not a good idea here

Well, I guess I did an another mistake here, I should have done this instead :
~# mdadm -Av /dev/md0 --assume-clean --level=5 --raid-devices=3 /dev/sdc1 /dev/sdd1 missing

Maybe this wipe out my data...
Let's go futher, then, pvdisplay, pvscan, vgdisplay returns empty information

Google helped me, and I did this :
~# dd if=/dev/md0 bs=512 count=255 skip=1 of=/tmp/md0.txt

	[..]
	physical_volumes {
		pv0 {
			id = "5DZit9-6o5V-a1vu-1D1q-fnc0-syEj-kVwAnW"
			device = "/dev/md0"
			status = ["ALLOCATABLE"]
			flags = []
			dev_size = 7814047360
			pe_start = 384
			pe_count = 953863
		}
	}
	logical_volumes {

		lvdata {
			id = "JiwAjc-qkvI-58Ru-RO8n-r63Z-ll3E-SJazO7"
			status = ["READ", "WRITE", "VISIBLE"]
			flags = []
			segment_count = 1
	[..]

Since I saw lvm information, I guess I haven't lost all information yet...

I tried an unhoped command :
~# pvcreate --uuid "5DZit9-6o5V-a1vu-1D1q-fnc0-syEj-kVwAnW"
--restorefile /etc/lvm/archive/lvm-raid_00302.vg /dev/md0 Then,

~# vgcfgrestore lvm-raid

~# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
lvdata lvm-raid -wi-a- 450,00g /dev/md0(148480)
lvmp lvm-raid -wi-a- 80,00g /dev/md0(263680)
Then :
~# lvchange -ay /dev/lvm-raid/lv*

I was quite happy until now.
Problem appears now when I try to mount those 2 LV (lvdata & lvmp) as ext4 partition :
~# mount /home/foo/RAID_mp/

~# mount | grep -i mp
/dev/mapper/lvm--raid-lvmp on /home/foo/RAID_mp type ext4 (rw)

~# df -h /home/foo/RAID_mp
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/lvm--raid-lvmp 79G 61G 19G 77% /home/foo/RAID_mp

Here is the big problem
~# ls -la /home/foo/RAID_mp
total 0

I did a LVM R/W snapshot on the /dev/mapper/lvm--raid-lvmp LV, I fsck it. I recover 50% of the files only, all located in lost-+found/ directory with names heading with #xxxxx.

I would like to know if there is a last chance to recover my data ?

Thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

NEVER USE "CREATE" ON FILESYSTEMS OR RAID ARRAYS UNLESS YOU KNOW WHAT YOU ARE DOING!
CREATE destroys things in the creation process, especially with the --force option.

The create argument is only done to create a new array, it will start with two drives as 'good' drives and the last will likely be the degraded drive, so it will start resyncing and blowing away data on the last drive. If you used the --assume clean argument, and it DID NOT resync the drives, you might be able to recreate the array with the two good disks, provided you know the original order.

If you used the --create option, and didn't have your disks in the same order they were originally in, you probably lost your data.

Since you replaced a disk, with no data (or worse, with bad data), you should have assembled the array, in degraded mode WITHOUT the --assume-clean argument.

If C & D contain your data, and B used to..
mdadm --assemble /dev/md0 missing /dev/sdc1 /dev/sdd1 You might have to --force the assembly. If it works, and it runs in degraded mode, mount your filesystem and take a backup.

Next, then add your replacement drive back in:
mdadm --add /dev/md0 /dev/sdb1
(Note, if sdb1 has some superblock data, you might have to --zero-superblock first)

Good luck.

Quoted 106 lines · L.M.J

Up please :-(

Quoted 3 lines · L.M.J

Hi,

For the third time, I had to change a failed drive from my home linux

RAID5 box. Previous one went right and

Quoted 1 line

this time, I don't know what I did wrong, but I broke my RAID5. Well, at

least, he didn't want to

Quoted 3 lines

start. /dev/sdb was the failed drive /dev/sdc and /dev/sdd are OK.

I tried to reassemble the RAID with this command after I replace sdb and

create a new partition :

Quoted 2 lines

~# mdadm -Cv /dev/md0 --assume-clean --level=5 --raid-devices=3

/dev/sdc1 /dev/sdd1 /dev/sdb1

Quoted 3 lines

-> '-C' was not a good idea here

Well, I guess I did an another mistake here, I should have done this

instead :

Quoted 1 line

~# mdadm -Av /dev/md0 --assume-clean --level=5 --raid-devices=3

/dev/sdc1 /dev/sdd1 missing

Quoted 3 lines

Maybe this wipe out my data...
Let's go futher, then, pvdisplay, pvscan, vgdisplay returns empty

information

Quoted 25 lines

Google helped me, and I did this :
~# dd if=/dev/md0 bs=512 count=255 skip=1 of=/tmp/md0.txt

      [..]
      physical_volumes {
              pv0 {
                      id = "5DZit9-6o5V-a1vu-1D1q-fnc0-syEj-kVwAnW"
                      device = "/dev/md0"
                      status = ["ALLOCATABLE"]
                      flags = []
                      dev_size = 7814047360
                      pe_start = 384
                      pe_count = 953863
              }
      }
      logical_volumes {

              lvdata {
                      id = "JiwAjc-qkvI-58Ru-RO8n-r63Z-ll3E-SJazO7"
                      status = ["READ", "WRITE", "VISIBLE"]
                      flags = []
                      segment_count = 1
      [..]

Quoted 1 line

Since I saw lvm information, I guess I haven't lost all information

yet...

Quoted 9 lines

I tried an unhoped command :
~# pvcreate --uuid "5DZit9-6o5V-a1vu-1D1q-fnc0-syEj-kVwAnW"
--restorefile /etc/lvm/archive/lvm-raid_00302.vg /dev/md0 Then,

~# vgcfgrestore lvm-raid

~# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert

Devices

Quoted 1 line

   lvdata lvm-raid -wi-a- 450,00g

              /dev/md0(148480)

Quoted 1 line

   lvmp   lvm-raid -wi-a-  80,00g

              /dev/md0(263680)

Quoted 5 lines

Then :
~# lvchange -ay /dev/lvm-raid/lv*

I was quite happy until now.
Problem appears now when I try to mount those 2 LV (lvdata & lvmp) as

ext4 partition :

Quoted 7 lines

~# mount /home/foo/RAID_mp/

~# mount | grep -i mp
/dev/mapper/lvm--raid-lvmp on /home/foo/RAID_mp type ext4 (rw)

~# df -h /home/foo/RAID_mp
Filesystem Size Used Avail Use%

Mounted on

Quoted 1 line

      /dev/mapper/lvm--raid-lvmp   79G   61G   19G  77%

/home/foo/RAID_mp

Quoted 6 lines

Here is the big problem
~# ls -la /home/foo/RAID_mp
total 0

I did a LVM R/W snapshot on the /dev/mapper/lvm--raid-lvmp LV, I fsck

it. I recover 50% of the files only,

Quoted 9 lines

all located in lost-+found/ directory with names heading with #xxxxx.

I would like to know if there is a last chance to recover my data ?

Thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Your data is split 3 ways.. 50% on one disk, 50% on another disk, and one disk worth of parity.

Now, it's not that simple, because the data is not continuous.. It is written across the three drives in chinks, with the parity alternating between the three drives.

If you were able to recover 50%, it probably means one disk contains valid data.

Were you able to recover anything larger than your chunk size? Are larger files (Mp3s and or movies) actually playable? Likely not.

You might get lucky trying to assemble the array in degraded mode with the 2 good disks, as long as the array didn't resync your new disk + good disk to the other good disk...

If added properly, it would have resynced the two good disks with the blank disk. Try doing a 'hd /dev/sdb1' to see if there is data on the new disk

Quoted 157 lines

Hello Scott,

Do you think I've lost my data 100% for sure ? fsck recovered 50% of the
files, don't you thing there is
still something to save ?

Thanks

Quoted 3 lines · Scott D'Vileskis

NEVER USE "CREATE" ON FILESYSTEMS OR RAID ARRAYS UNLESS YOU KNOW WHAT YOU ARE DOING!
CREATE destroys things in the creation process, especially with the

--force

Quoted 3 lines

option.

The create argument is only done to create a new array, it will start

with

Quoted 1 line

two drives as 'good' drives and the last will likely be the degraded

drive,

Quoted 1 line

so it will start resyncing and blowing away data on the last drive. If

you

Quoted 22 lines

used the --assume clean argument, and it DID NOT resync the drives, you might be able to recreate the array with the two good disks, provided you know the original order.

If you used the --create option, and didn't have your disks in the same order they were originally in, you probably lost your data.

Since you replaced a disk, with no data (or worse, with bad data), you should have assembled the array, in degraded mode WITHOUT the --assume-clean argument.

If C & D contain your data, and B used to..
mdadm --assemble /dev/md0 missing /dev/sdc1 /dev/sdd1 You might have to --force the assembly. If it works, and it runs in degraded mode, mount your filesystem and take a backup.

Next, then add your replacement drive back in:
mdadm --add /dev/md0 /dev/sdb1
(Note, if sdb1 has some superblock data, you might have to --zero-superblock first)

Quoted 15 lines

Good luck.

Quoted 10 lines · L.M.J

Up please :-(

Quoted 3 lines · L.M.J

Hi,

For the third time, I had to change a failed drive from my home linux

RAID5 box. Previous one went right and

Quoted 1 line

this time, I don't know what I did wrong, but I broke my RAID5.

Well, at

Quoted 4 lines

Quoted 4 lines

least, he didn't want to

Quoted 3 lines

start. /dev/sdb was the failed drive /dev/sdc and /dev/sdd are OK.

I tried to reassemble the RAID with this command after I replace sdb

and

Quoted 42 lines

Quoted 42 lines

create a new partition :

Quoted 2 lines

~# mdadm -Cv /dev/md0 --assume-clean --level=5 --raid-devices=3

/dev/sdc1 /dev/sdd1 /dev/sdb1

Quoted 3 lines

-> '-C' was not a good idea here

Well, I guess I did an another mistake here, I should have done this

instead :

Quoted 1 line

~# mdadm -Av /dev/md0 --assume-clean --level=5 --raid-devices=3

/dev/sdc1 /dev/sdd1 missing

Quoted 3 lines

Maybe this wipe out my data...
Let's go futher, then, pvdisplay, pvscan, vgdisplay returns empty

information

Quoted 25 lines

Google helped me, and I did this :
~# dd if=/dev/md0 bs=512 count=255 skip=1 of=/tmp/md0.txt

      [..]
      physical_volumes {
              pv0 {
                      id = "5DZit9-6o5V-a1vu-1D1q-fnc0-syEj-kVwAnW"
                      device = "/dev/md0"
                      status = ["ALLOCATABLE"]
                      flags = []
                      dev_size = 7814047360
                      pe_start = 384
                      pe_count = 953863
              }
      }
      logical_volumes {

              lvdata {
                      id = "JiwAjc-qkvI-58Ru-RO8n-r63Z-ll3E-SJazO7"
                      status = ["READ", "WRITE", "VISIBLE"]
                      flags = []
                      segment_count = 1
      [..]

Quoted 1 line

Since I saw lvm information, I guess I haven't lost all information

yet...

Quoted 1 line

Quoted 8 lines

Quoted 8 lines

Quoted 8 lines

I tried an unhoped command :
~# pvcreate --uuid "5DZit9-6o5V-a1vu-1D1q-fnc0-syEj-kVwAnW"
--restorefile /etc/lvm/archive/lvm-raid_00302.vg /dev/md0 Then,

~# vgcfgrestore lvm-raid

~# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy%

Convert

Quoted 27 lines

Quoted 27 lines

Devices

Quoted 1 line

   lvdata lvm-raid -wi-a- 450,00g

              /dev/md0(148480)

Quoted 1 line

   lvmp   lvm-raid -wi-a-  80,00g

              /dev/md0(263680)

Quoted 5 lines

Then :
~# lvchange -ay /dev/lvm-raid/lv*

I was quite happy until now.
Problem appears now when I try to mount those 2 LV (lvdata & lvmp) as

ext4 partition :

Quoted 7 lines

~# mount /home/foo/RAID_mp/

~# mount | grep -i mp
/dev/mapper/lvm--raid-lvmp on /home/foo/RAID_mp type ext4 (rw)

~# df -h /home/foo/RAID_mp
Filesystem Size Used Avail Use%

Mounted on

Quoted 1 line

      /dev/mapper/lvm--raid-lvmp   79G   61G   19G  77%

/home/foo/RAID_mp

Quoted 6 lines

Here is the big problem
~# ls -la /home/foo/RAID_mp
total 0

I did a LVM R/W snapshot on the /dev/mapper/lvm--raid-lvmp LV, I fsck

Quoted 8 lines

Quoted 8 lines

it. I recover 50% of the files only,

Quoted 7 lines

all located in lost-+found/ directory with names heading with #xxxxx.

I would like to know if there is a last chance to recover my data ?

Thanks
--
To unsubscribe from this list: send the line "unsubscribe

linux-raid" in

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 7 lines

Quoted 2 lines

Quoted 2 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-raid"

in

Quoted 3 lines

Quoted 3 lines

the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Note, if you dare to --create the array again with your two previous disks, you'll want to create the array with the 'missing' disk in the right place.

mdadm --create /dev/md0 missing /dev/sdc1 /dev/sdd1 (Assuming your original array was sdb, sdc, sdd)

You'll probably need a force and maybe a start-degraded.

Then, I would try the recovery on the resulting drive.

~# mdadm -Cv /dev/md0 --assume-clean --level=5 --raid-devices=3 /dev/sdc1 /dev/sdd1 /dev/sdb1

Why did my block counts change? The disk partitions weren't touched or changed at any point. Shouldn't I have gotten the same size?

The created device isn't work. There is suppose to be luks encrypted volume there but luksOpen reports there is no luks header. (and there use to be). Would the odd change in size indicate total corruption?

Quoted 2 lines · Jeff Wiegley

Why did my block counts change? The disk partitions weren't touched or changed at any point. Shouldn't I have gotten the same size?

Defaults in mdadm has changed over time, so data offsets might be different. In order to get the exact same data offset you need to use the same mdadm version as was originally used, or at least know the values it used and use mdadm 3.3 that allows you to specify these data at creation time,

Quoted 3 lines

The created device isn't work. There is suppose to be luks encrypted volume there but luksOpen reports there is no luks header. (and there use to be). Would the odd change in size indicate total corruption?

No, the change in size indicates that data offsets are not the same so your beginning of volume is now in the wrong place.

If it's mdadm then could I just grab old copies of mdadm sources and compile them one version after the other and try each one?

Quoted 2 lines · Jeff Wiegley

If it's mdadm then could I just grab old copies of mdadm sources and compile them one version after the other and try each one?

As far as I know, it's mdadm version specific. If you look in the archives I'm sure you'll be able to find the old offsets and you can use the latest mdadm with those offsets and hopefully things will work.

Any thoughts why the recreated array would satisfy and pass cryptsetup's sanity checks but the resulting decrypted data is not recognizable as xfs?

Quoted 2 lines · Jeff Wiegley

If it's mdadm then could I just grab old copies of mdadm sources and compile them one version after the other and try each one?

As far as I know, it's mdadm version specific. If you look in the archives I'm sure you'll be able to find the old offsets and you can use the latest mdadm with those offsets and hopefully things will work.

I have been replying directly to you, not to the mailing list, since your case seems to be a case of user-screwed-up-his-own-data, and not a problem with mdadm/linux raid, nor a problem that will necessarily help someone else (since it is not likely someone will create a mess in exactly the same manner you have) .

To summarize:
1) You lost a disk. Even down a disk, you should have been able to run/start the array (in degraded mode) with only 2 disks, mounted the filesystem, etc.

2) You then should have simply partitioned and then --add 'ed the new disk. mdadm would have written a superblock to the new disk, and resynced the
data

I assume your original disks were in the order sdb, sdc, sdd.

Unfortunately, you might have clobbered your drives by recreating the array. You certainly clobbered your superblocks and changed the order when you did this:

Quoted 1 line

~# mdadm -Cv /dev/md0 --assume-clean --level=5 --raid-devices=3 /dev/sdc1

/dev/sdd1 /dev/sdb1

You changed the order, but because of the assume-clean, it shouldn't have started a resync of the data. Your file system probably had a fit though.

Hindsight is 20/20, a mistake was made, it happens to all of us at some point or another, (I've lost arrays and filesystems with careless use of 'dd' once upon a time, once I was giving a raid demo to a friend with loop devices, mistyped something, and blew something away)

IMPORTANT: At any point did your drives do a resync?

Assuming no, and assuming you haven't done any other writing to your disks(besides rewriting the superblocks), you can probably correct the order of your drives by reissuing the --create command with the two original drives, in the proper order, and the missing drive as the placeholder. (This will rewrite the superblocks again, but hopefully in the right order)
mdadm -Cv /dev/md0 --level=5 --raid-devices=3 missing /dev/sdc1 /dev/sdd1

If you can start that array (it will be degraded with only 2/3 drives) you should be able to mount and recover your data. You may need to run a full fsck again since your last fsck probably made a mess.

Assuming you can mount and copy your data, you can then --add your 'new' drive to the array with the --add argument. (Note, you'll have to clear it's superblock or mdadm will object)

Well, what should I do now ? mkfs everything and restart from scratch ?

Quoted 1 line · L.M.J

Well, what should I do now ? mkfs everything and restart from scratch

?

Most likely this is your only option. First you overwrote the superblocks
so the drives came in the wrong order (most likely) and then you ran a read/write fsck on it.

The thing to do is to make sure md is read only and use fsck in readonly-mode as a diagnostics tool to see if everything is right. If you
get it wrong and do fsck in read/write mode it's going to change things

destructively.

As a last ditch effort, try the --create again but with the two potentially good disks in the right order:

mdadm --create /dev/md0 --level=5 --raid-devices=3 missing /dev/sdc1 /dev/sdd1

Drive B has bogus data on it., since it was resync'd with C & D in the wrong order. Fortunately, your --add should have only have changed B, not C & D.

As a last ditch effort, try the --create again but with the two potentially good disks in the right order:

mdadm --create /dev/md0 --level=5 --raid-devices=3 missing /dev/sdc1 /dev/sdd1

Note: The following is where I have reproduced your problem with loop devices

#Create 3 200MB files

root@Breadman:/home/scott# mkdir raidtesting
root@Breadman:/home/scott# cd raidtesting/
root@Breadman:/home/scott/raidtesting# fallocate -l200000000 sdb
root@Breadman:/home/scott/raidtesting# fallocate -l200000000 sdc
root@Breadman:/home/scott/raidtesting# fallocate -l200000000 sdd
root@Breadman:/home/scott/raidtesting# losetup /dev/loop2 sdb
root@Breadman:/home/scott/raidtesting# losetup /dev/loop3 sdc
root@Breadman:/home/scott/raidtesting# losetup /dev/loop4 sdd
root@Breadman:/home/scott/raidtesting# mdadm --create /dev/md0 -n3 -l5
/dev/loop2 /dev/loop3 /dev/loop4
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

root@Breadman:/home/scott/raidtesting# cat /proc/mdstat
md0 : active raid5 loop4[3] loop3[1] loop2[0]
       388096 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

root@Breadman:/home/scott/raidtesting# mkfs.reiserfs /dev/md0
mkfs.reiserfs 3.6.21 (2009 www.namesys.com)
<SNIP>
ReiserFS is successfully created on /dev/md0.
root@Breadman:/home/scott/raidtesting# mkdir temp
root@Breadman:/home/scott/raidtesting# mount /dev/md0 temp/

#Then I copied a file to it:

root@Breadman:/home/scott/raidtesting# md5sum temp/systemrescuecd-x86-0.4.3.iso
b88ce25b156619a9a344889bc92b1833  temp/systemrescuecd-x86-0.4.3.iso

#And failed a disk

root@Breadman:/home/scott/raidtesting# umount temp/
root@Breadman:/home/scott/raidtesting# mdadm --fail /dev/md0 /dev/loop2
mdadm: set /dev/loop2 faulty in /dev/md0
root@Breadman:/home/scott/raidtesting# cat /proc/mdstat
md0 : active raid5 loop4[3] loop3[1] loop2[0](F)
       388096 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]

#Stopped array, removed disk, replaced disk by creating a new file

root@Breadman:/home/scott/raidtesting# mdadm --stop /dev/md0
mdadm: stopped /dev/md0
root@Breadman:/home/scott/raidtesting# losetup -d /dev/loop2
root@Breadman:/home/scott/raidtesting# rm sdb
root@Breadman:/home/scott/raidtesting# fallocate -l200000000 sdb-new
root@Breadman:/home/scott/raidtesting# losetup /dev/loop2 sdb-new

#WRONG: Create array in wrong order

root@Breadman:/home/scott/raidtesting# mdadm --create /dev/md0
--assume-clean -l5 -n3 /dev/loop3 /dev/loop4 /dev/loop2
mdadm: /dev/loop3 appears to be part of a raid array:
        level=raid5 devices=3 ctime=Fri Apr 25 09:10:31 2014
mdadm: /dev/loop4 appears to be part of a raid array:
        level=raid5 devices=3 ctime=Fri Apr 25 09:10:31 2014
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
root@Breadman:/home/scott/raidtesting# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0]
[raid1] [raid10]
md0 : active raid5 loop2[2] loop4[1] loop3[0]
       388096 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]

root@Breadman:/home/scott/raidtesting# mount /dev/md0 temp/
mount: you must specify the filesystem type

#Nope, doesn't mount, filesystem clobbered, or not?

root@Breadman:/home/scott/raidtesting# mdadm --stop /dev/md0
mdadm: stopped /dev/md0

#Recreate the array, with missing disk in the right place

root@Breadman:/home/scott/raidtesting# mdadm --create /dev/md0 -l5 -n3
missing /dev/loop3 /dev/loop4
mdadm: /dev/loop3 appears to be part of a raid array:
        level=raid5 devices=3 ctime=Fri Apr 25 09:17:38 2014
mdadm: /dev/loop4 appears to be part of a raid array:
        level=raid5 devices=3 ctime=Fri Apr 25 09:17:38 2014
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
root@Breadman:/home/scott/raidtesting# mount /dev/md0 temp/
root@Breadman:/home/scott/raidtesting# ls temp/
systemrescuecd-x86-0.4.3.iso
root@Breadman:/home/scott/raidtesting# md5sum temp/systemrescuecd-x86-0.4.3.iso
b88ce25b156619a9a344889bc92b1833  temp/systemrescuecd-x86-0.4.3.iso

#Notice we are in degraded mode

root@Breadman:/home/scott/raidtesting# cat /proc/mdstat
md0 : active raid5 loop4[2] loop3[1]
       388096 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]

#Add our replacement disk:

root@Breadman:/home/scott/raidtesting# mdadm --add /dev/md0 /dev/loop2
mdadm: added /dev/loop2

root@Breadman:/home/scott/raidtesting# cat /proc/mdstat
md0 : active raid5 loop2[3] loop4[2] loop3[1]
       388096 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [_UU]
       [============>........]  recovery = 62.1% (121316/194048)
finish=0.0min speed=12132K/sec

#After a while (short while with 200MB loop devices):

root@Breadman:/home/scott/raidtesting# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0]
[raid1] [raid10]
md0 : active raid5 loop2[3] loop4[2] loop3[1]
       388096 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU]
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html