Best way to image devices in an LVM environment

Hello,

I have a server I'm currently finalising the setup of, which uses two 80G SATA drives in a RAID-1 (md) with LVM on top. When I've got it how I want it I need to replace the 80G drives with a much larger pair.

Now, I've kept notes about my setup of course so I can just put the new disks in and do it all over again, but that strikes me as a big waste of time.

I also of course have space elsewhere to take an image of the md devices with dd and I could copy them back over the top once md is set up again on the new disks. The majority of space though is in my single LVM volume group and only 18.5G of this 72.5G is used, so taking an 80G image seems to also be rather a waste of time.

Here's the setup:

/dev/md0 (sda1, sdb1) is swap
/dev/md1 (sda2, sdb2) is /boot
/dev/md2 (sda3, sdb3) is /
/dev/md4 (sda5, sdb5) is an LVM PV and VG 'mainvg'

Everything else is LVs within mainvg and all filesystems are ext3.

So how would you accomplish the task of replacing sda and sdb with larger disks in the least amount of effort? Downtime is not a problem, taking backups of the data is not a problem, booting off a live cd is not a problem; the goal is to find the least labour-intensive way to do this.

The best I can think of booting off a live cd, taking a dd of each md device to another server, installing the new drives, setting up the md devices again and then overwriting them with the images. Will I run into problems with their UUIDs being overwritten?

Cheers,
Andy

As I understand it -- and as I've proved before in this list, my understanding is fundamentally flawed :-) -- there is a way to do this.

Basically you have to force one disk to fail using the raid administration software. You then remove the disk, replace it with the bigger disk, and rebuild it from the first one. The partition sizes may need adjusting. Then you force the other disk to fail, and replace it with the larger one, and then rebuild it from the other large disk.

Hopefully someone can flesh this out for me, or you may now have a starting point for some googling.

Jim

Well I would then be left with two large disks where sd[ab]5 (and the sd[ab]4 primary partition that sd[ab]5 is the only logical partition within) does not extend all the way to the end of the disk. I wasn't aware that you could resize RAID-1 md arrays though, I thought it was restricted to RAID-5. Is this not the case?

Cheers,
Andy

To add another "I'm not an expert" opinion:
Supposing you can get all 4 disks in the server at the same time, you could add your new disk's large partition to your existing logical volume, then do a pvmove. The advantage over the previous suggestion is that you don't need to resize your RAID device when you're done (although you'd still want to resize your filesystem to take advantage of all the new space).

Although now that I look at the instructions for doing this
(http://tldp.org/HOWTO/LVM-HOWTO/removeadisk.html; section 13.5.1), it seems that is a dangerous operation. If it worked though, it would probably be the least labor intensive.

Matt

I can't; it's a rackmount server with only two disk bays and two SATA ports on the motherboard. I could have it open with another 2-port SATA controller on a PCI card, but this seems to be going a bit far.

Since I only want to give the extra space to the single volume group then it seems I should be able to follow this method:

http://www.debian-administration.org/articles/424

(basically involves doing what Jim said: failing out one disk, replacing it with a bigger one, failing the other, replacing it, then expanding partitions and filesystems.)

Seems that "mdadm --grow" works even with a RAID-1 which I wasn't aware of.

The only problem then remaining would be how would I grow the single pv/vg out to the full extent of the new larger block device it would be on. There doesn't seem to be a pvresize in Debain Sarge's lvm2 package..

LVM version: 2.01.04 (2005-02-09)
Library version: 1.01.00-ioctl (2005-01-17)
Driver version: 4.5.0

I see from:

http://www.robot101.net/2005/05/20/resizing-lvm2-physical-volumes/

that lvm2 has had pvresize since 2.02.00 so if I upgrade userland tools (my kernel is 2.6.16.19) then would I just need to do a pvresize after expanding the block device that my single pv/vg is on?

Cheers,
Andy

Would it not be easier to just replace with a larger disk - mirror to the same size, and have another partition afterwards.

You end up with 2pvs (both mirrored, I would expect), rather than the "nicer" answer of a single pv, but as long as the whole thing is then controlled through LVM is there really any loss as a result?

Graham

Hmm yes as you say I think I could just make another partition in the empty space, RAID-1 that, put a PV on the new MD device and then vgextend onto it I suppose. It just seems a litlle messy though to needlessly have the extra partitions and if I can fix it before the machine is put in production then I feel like I probably should.

Cheers,
Andy