Recovery speed at 1MB/s/device, unable to change

I created a raid array:
---

# mdadm --create /dev/md_d0 -ap --level=5 --raid-devices=2 \
/dev/sda1 missing
---

Then partitioned it:
---
# LANGUAGE=C fdisk -l /dev/md_d0

Disk /dev/md_d0: 250.0 GB, 250056605696 bytes
2 heads, 4 sectors/track, 61048976 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

      Device Boot      Start         End      Blocks   Id  System
/dev/md_d0p1               1    61048976   244195902   83  Linux

---

Then made a ext3 filesystem in /dev/md_d0p1 with mke2fs.

Then I added another device:
---

# mdadm --manage /dev/md_d0 --add /dev/sdb1
---

Currently status is:
---

# cat /proc/mdstat
Personalities : [raid0] [raid5] [raid4]
md_d0 : active raid5 sdb1[2] sda1[0]
      244195904 blocks level 5, 64k chunk, algorithm 2 [2/1] [U_]
      [>....................]  recovery =  1.0% (2588800/244195904)
finish=2100.4min speed=1916K/sec

unused devices: <none>
---

The speed is only 2000K/sec, even after I set:
---

# cat /proc/sys/dev/raid/speed_limit_min
10000
# cat /proc/sys/dev/raid/speed_limit_max
400000
---

The system is about 90% idle, so there should be more bandwidth.

---

# cat /proc/version
Linux version 2.6.17-rc2-git4 ([email protected]) (gcc version 4.0.1
(4.0.1-5mdk for Mandriva Linux release 2006.0)) #1 Sun Apr 23 00:56:30
EEST 2006
---

Any tips?

Anssi Hannula

Hmm... I don't know if this is related, but something seems to be really wrong.

I run the following set of commands:

# dd if=/dev/zero of=ohase count=5K
# dd if=/dev/zero of=ohase2 count=5K
# losetup /dev/loop0 ohase
# losetup /dev/loop1 ohase2
# mdadm --create /dev/md_d1 -ap --level=5 --raid-devices=2 /dev/loop0
missing
# fdisk /dev/md_d1
(created one partition)
# mkfs.ext3 /dev/md_d1p1
# mount /dev/md_d1p1 /mnt/test
# echo jopajoo > /mnt/iso/test
# mdadm --manage /dev/md_d1 --add /dev/loop1
# mdadm --grow /dev/md_d1 --raid-devices=3 --backup-file backupfile
mdadm: Need to backup 128K of critical section..
mdadm: /dev/md_d1: Cannot get array details from sysfs

Strace shows that it's trying to access
"/sys/block/md_d4/md/component_size".

Why is this?

Anssi Hannula

Because I didn't test my code properly :-(

Following patch should fix it.

Re: the recovery speed problem: I would try dding from one drive to the other and see what sort of speed you get then:
dd if=/dev/sda of=/dev/sdb bs=1024k count=1000
or something like that.

Thanks,
NeilBrown

### Diffstat output

 ./sysfs.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff ./sysfs.c~current~ ./sysfs.c
@@ -206,7 +206,7 @@ unsigned long long get_component_size(in
 			minor(stb.st_rdev));
 	else
 		sprintf(fname, "/sys/block/md_d%d/md/component_size",
-			minor(stb.st_rdev)/16);
+			minor(stb.st_rdev)/64);
 	fd = open(fname, O_RDONLY);
 	if (fd < 0)
 		return 0;

(sorry, prev post missed the list)

Okay, will test.

# export LANGUAGE=C; export LANG=C; export LC_ALL=C; date; dd
if=/dev/sda skip=20000 of=/dev/sdb bs=1024k count=1000; date
Mon Apr 24 01:14:48 EEST 2006
1000+0 records in
1000+0 records out
Mon Apr 24 01:15:42 EEST 2006

So the speed is about 20MB/s.

Anssi Hannula

(resend, prev post missed the list)

It appears you missed another occurrence (patch attached).

However, it got stuck after
mdadm: Need to backup 128K of critical section..

I did ctrl-c, and according to /proc/mdstat it was successfully reshaped, nor can I restart the reshape.

But "umount /mnt/test" (/mnt/test is where /dev/md_d1p1 is mounted) blocks, and this time it becomes unkillable.

/proc/mdstat reports:
md_d1 : active raid5 loop1[1] loop0[0]

      4992 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

Anssi Hannula

@@ -69,7 +69,7 @@
 			sprintf(sra->name, "md%d", minor(stb.st_rdev));
 		else
 			sprintf(sra->name, "md_d%d",
-				minor(stb.st_rdev)/16);
+				minor(stb.st_rdev)/64);
 	} else {
 		if (devnum >= 0)
 			sprintf(sra->name, "md%d", devnum);

[...strange problems...]

The rest of my problems, including the slow speed of reconstruction were solved by reboot and I'm unable to reproduce them.

Neil, thank you very much for the speedy replies :)

Anssi Hannula