RAID 1 partition with hot spare shows [UUU] ?

Hi,

Sorry to trouble people, and I am sure you have better things to do, but I can't find an answer to the following and don't know where else to go.

I have been struggling with this problem for weeks and having read all I can I still don't know the answer.

I have a server running a version of CentOS 5.x Yes, mdadm is old at
2.6.9 but it isn't possible to update it currently.

A year or so ago I clean installed with a software RAID 1 array using /dev/sda & /dev/sdb and two partitions, md1 & md2 configure automatically on install.

I restored data to the RAID and then added manually added a third drive /dev/sdc as a spare.

All appeared hunky dory, but whilst trying to figure a slightly different problem on a different machine, I went back to the first one to check how it was configured. Although I am sure all looked normal when I had last looked, this time is looked a bit strange.

Unfortunately I don't have an exact copy of things before I started messing about but it looked something like this :

cat /proc/mdstat revealed :

Personalities : [raid1]
md1 : active raid1 sdc1[2] sdb1[1] sda1[0]

      104320 blocks [3/3] [UUU]

md2 : active raid1 sdc2(S) sdb2[1] sda2[0]

      244091520 blocks [2/2] [UU]

unused devices: <none>

I don't understand how md1 shows [UUU] ??

On my other machine which has a similar configuration it shows the following which I expect :

Personalities : [raid1]
md1 : active raid1 sda1[2](S) hdc1[1] hda1[0]

      104320 blocks [2/2] [UU]

md2 : active raid1 sda2[2](S) hdc2[1] hda2[0]

      312464128 blocks [2/2] [UU]

unused devices: <none>

I thought I could fail and remove the drive, dd/fdisk/reformat, sfdisk and then try to re add it back to the array effectively as a new drive. No joy.

If I just fail and remove it md1 shows as [UU_]

I have tried checking mdadm.conf which has the following :

DEVICE partitions
ARRAY /dev/md1 level=raid1 num-devices=2
uuid=8833ba3d:ca592541:20c7be04:42cbbdf1 spares=1
ARRAY /dev/md2 level=raid1 num-devices=2
uuid=43a5b70d:9733da5c:7dd8d970:1e476a26 spares=1

Somewhere along the line the RAID is remembering the earlier configuration but having changed stuff left right and Cambridge, I can't seem to get it to forget.

I have tried different variations of mdadm.conf, and tried to rebuild initrd but that didn't fix it and I am clean out of ideas where to go next. mdadm.conf seems to be ignored.

Undoubtedly it will take some clever tweaking and I'm scared witless at trashing the array as I am in a different country from the hardware and would struggle to get back to fix it !

Any advice on how to put it back to RAID 1 with a 'hot' spare would be appreciated.

B. Rgds
John

You have a RAID1 with 3 devices. What it difficult to comprehend about that. Each block is written to all three devices.

You don't want that? Change it.

   mdadm --grow /dev/md1 --raid-disks=4

now it has 4 devices - though one will be missing.

   mdadm --grow /dev/md1 --raid-disks=2

now it has 2 devices. Actually that won't work until you mark one of the devices as failed, so

   mdadm /dev/md1 --fail /dev/sdc1
   mdadm --grow /dev/md1 --raid-disks=2

But maybe you really wanted a 3-disk RAID1 before - it is a configuration that certainly has a place.

Next you'll be telling me that a RAID5 cannot be made with just 2 devices! :-)

NeilBrown

Can't you make it with just one device, if you make a two-device raid5 with one drive missing?

mvh.,

David

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Probably a misunderstanding of the terminology - in Hicksville where I live mirrored usually just means a pair ;-)

I didn't realise that a RAID 1 could have more than two devices. Most documentation around always illustrates a RAID 1 with a pair of drives and hence the confusion.

Logical if you know it (RAID 1) can have more devices.

Illogical naming if what you actually want to do is shrink the array............ like many things in life, once you know, you know.

And there lies the answer to my problem. For that I can't thank you enough !

No I didn't, but I now understand.

Me ? I know jack ...... though I know one more thing than I did yesterday, and that is good enough for me !

Many thanks for the assistance.

B. Rgds
John
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJPjn5zAAoJEPGW9QpGvPXME0sH/0p9dpJkZvWDDxP2sMjjDlE5 0xO8v3BY8frdHT803EYaQUkxOStgtkgMd8+MaZIK3M5Z8DM2BpbDuGyz39FvZ7Ai r+TWygB2lESxgnFI1PbSeZq+qZholIpS+TD8cJ25gDImXjD9mtkN8sx/cMmcVP7k CHzPY5cPOGCHNPC1V1BOYAU82+fjGZP8asQJT/TjUTQ/DlWuGVkDSENYGkWNY84T a7MuF40StY6/RG1X3ETiTlk/iq2+1j7V6I+tQlTIqWyREb9T82O2YJ2HzTeN5uN7 YXc/asVRjELmIQHode/Xq9WXkGZpQskdHqLAPyP1lKc1aNyKARsyaGHKvYZj/94= =a+Ob
-----END PGP SIGNATURE-----

A lot of hardware raid systems (and software systems, such as Window's "dynamic disks") are limited to 2 devices in a raid1 set. The world of Linux md raid is very different - here flexibility is the key. Amongst other things, you can have raid1 sets with any number of devices - you've seen it with 3 devices, but you can also make them with just one device. And you can also use all sorts of different disks in the set, not just identical ones.

You might well ask what use a 1-disk raid1 "mirror" is. The answer is not much use on its own - but very useful in that you can add and remove disks whenever you want. Suppose, for example, you have a single disk in the machine, and you want to exchange it for a new, bigger disk. If the disk is a 1-disk raid1 mirror, you can do it quickly and safely without going off-line (except if you need to power-off to do the actual disk swapping). Attach your new, bigger disk. Add it to the raid1 array. Wait for the sync to complete (using the disk normally all this time). Fail and remove the original disk. Your new disk is now a 1-disk "mirror" with the same data. Grow the raid to fill the new disk, then grow your filesystems (or physical volumes for LVM) to use the new space.

There are lots of cool things you can do when you are not limited to hardware raid mindsets!